脚本如何实现文件内容抽取功能

本文目录导读：

1. Python实现（最灵活）
2. Shell脚本（Linux/Unix环境）
3. PowerShell（Windows环境）
4. 高级功能实现
5. 实用工具函数
6. 使用建议

Python实现（最灵活）

基础文本抽取

import re
import os
def extract_content(file_path, pattern):
    """从文件中抽取符合正则表达式的内容"""
    results = []
    try:
        with open(file_path, 'r', encoding='utf-8') as f:
            content = f.read()
            matches = re.findall(pattern, content)
            results.extend(matches)
    except Exception as e:
        print(f"读取文件错误: {e}")
    return results
# 示例：抽取所有邮箱地址
emails = extract_content('data.txt', r'[\w\.-]+@[\w\.-]+\.\w+')
print(emails)

结构化文件抽取

import json
import csv
import xml.etree.ElementTree as ET
def extract_json_fields(json_file, fields):
    """从JSON文件抽取指定字段"""
    with open(json_file, 'r', encoding='utf-8') as f:
        data = json.load(f)
    extracted = []
    if isinstance(data, list):
        for item in data:
            extracted.append({field: item.get(field) for field in fields})
    return extracted
def extract_csv_columns(csv_file, columns, delimiter=','):
    """从CSV文件抽取指定列"""
    with open(csv_file, 'r', encoding='utf-8') as f:
        reader = csv.DictReader(f, delimiter=delimiter)
        return [{col: row[col] for col in columns if col in row} 
                for row in reader]
# 示例使用
json_data = extract_json_fields('data.json', ['name', 'email', 'age'])
csv_data = extract_csv_columns('data.csv', ['name', 'phone'])

Shell脚本（Linux/Unix环境）

#!/bin/bash
# 抽取文件中的特定模式
extract_pattern() {
    local file=$1
    local pattern=$2
    grep -oP "$pattern" "$file"
}
# 抽取CSV文件的指定列
extract_csv_column() {
    local file=$1
    local column=$2
    local delimiter=${3:-,}
    cut -d"$delimiter" -f"$column" "$file"
}
# 抽取文件的关键字行
extract_keyword_lines() {
    local file=$1
    local keyword=$2
    grep -n "$keyword" "$file"
}
# 使用示例
# extract_pattern "log.txt" "error.*"
# extract_csv_column "data.csv" 1,3,5

PowerShell（Windows环境）

# 抽取文本文件中的匹配行
function Extract-Content {
    param(
        [string]$FilePath,
        [string]$Pattern,
        [string]$OutputFile
    )
    $matches = Select-String -Path $FilePath -Pattern $Pattern
    if ($OutputFile) {
        $matches | Select-Object -ExpandProperty Line | Out-File -FilePath $OutputFile
    }
    return $matches
}
# 抽取CSV文件列
function Extract-CsvColumns {
    param(
        [string]$CsvFile,
        [string[]]$Columns
    )
    Import-Csv $CsvFile | Select-Object $Columns
}
# 示例
# Extract-Content -FilePath "log.txt" -Pattern "ERROR|WARNING"
# Extract-CsvColumns -CsvFile "data.csv" -Columns "Name", "Email"

高级功能实现

批量文件处理

import glob
import os
def batch_extract(directory, file_pattern, extraction_func):
    """批量处理目录中的文件"""
    results = {}
    for file_path in glob.glob(os.path.join(directory, file_pattern)):
        filename = os.path.basename(file_path)
        results[filename] = extraction_func(file_path)
    return results
# 示例：处理所有.log文件
def extract_errors(file_path):
    errors = []
    with open(file_path, 'r') as f:
        for line in f:
            if 'ERROR' in line:
                errors.append(line.strip())
    return errors
all_errors = batch_extract('./logs', '*.log', extract_errors)

按条件抽取

class ConditionExtractor:
    def __init__(self, conditions):
        """
        conditions: [(column_index, operator, value), ...]
        """
        self.conditions = conditions
    def extract_from_csv(self, csv_file, delimiter=','):
        result = []
        with open(csv_file, 'r', encoding='utf-8') as f:
            reader = csv.reader(f)
            headers = next(reader)
            for row in reader:
                if self._check_conditions(row):
                    result.append(row)
        return result
    def _check_conditions(self, row):
        for col_idx, operator, value in self.conditions:
            cell = row[col_idx]
            if operator == '>':
                if not (float(cell) > float(value)):
                    return False
            elif operator == 'contains':
                if value not in cell:
                    return False
        return True

实用工具函数

import re
from collections import Counter
def extract_html_tags(html_file, tag):
    """抽取HTML文件中的特定标签内容"""
    pattern = f'<{tag}[^>]*>(.*?)</{tag}>'
    with open(html_file, 'r', encoding='utf-8') as f:
        return re.findall(pattern, f.read(), re.DOTALL)
def extract_unique_values(file_path, separator=None):
    """抽取文件中唯一的值"""
    with open(file_path, 'r') as f:
        if separator:
            values = f.read().split(separator)
        else:
            values = [line.strip() for line in f]
    return list(set(filter(None, values)))
def extract_word_frequency(file_path):
    """抽取文件中的单词频率"""
    with open(file_path, 'r', encoding='utf-8') as f:
        words = re.findall(r'\b\w+\b', f.read().lower())
    return Counter(words).most_common(10)

使用建议

1. 简单文本匹配：使用Shell或PowerShell
2. 复杂数据处理：使用Python
3. 大文件处理：使用流式处理避免内存溢出
4. 正则表达式模式：根据具体需求定制pattern
5. 错误处理：添加适当的异常处理
6. 编码问题：注意文件编码格式

这些脚本可以根据具体需求进行定制和组合使用。