本文目录导读：
我来介绍几种常用的脚本来转换文本编码为UTF-8（最通用的编码格式）：
使用 iconv (Linux/macOS)
# 基本用法
iconv -f 原编码 -t UTF-8 输入文件 > 输出文件
# 示例：将GBK编码转为UTF-8
iconv -f GBK -t UTF-8 source.txt > target.txt
# 批量转换当前目录下所有.txt文件
for file in *.txt; do
iconv -f GBK -t UTF-8 "$file" > "utf8_$file"
done
使用 Python 脚本
#!/usr/bin/env python3
import os
import sys
def convert_to_utf8(file_path, source_encoding='auto'):
try:
# 读取文件
with open(file_path, 'rb') as f:
content = f.read()
# 自动检测编码
if source_encoding == 'auto':
import chardet
detected = chardet.detect(content)
source_encoding = detected['encoding']
# 解码并重新编码为UTF-8
text = content.decode(source_encoding)
with open(file_path, 'w', encoding='utf-8') as f:
f.write(text)
print(f"已转换: {file_path} ({source_encoding} -> UTF-8)")
except Exception as e:
print(f"转换失败 {file_path}: {e}")
# 使用示例
if __name__ == "__main__":
convert_to_utf8("example.txt", "gbk") # 指定编码
# 或
convert_to_utf8("example.txt") # 自动检测
使用 PowerShell (Windows)
# 基本命令
Get-Content -Path "原文件.txt" -Encoding 原编码 | Out-File -FilePath "新文件.txt" -Encoding utf8
# 示例：ANSI转UTF-8
Get-Content -Path "source.txt" -Encoding Default | Out-File -FilePath "target.txt" -Encoding utf8
# 批量处理
Get-ChildItem -Path "C:\path\to\files" -Filter "*.txt" | ForEach-Object {
$content = Get-Content -Path $_.FullName -Encoding Default
$newPath = Join-Path $_.Directory "utf8_$($_.Name)"
$content | Out-File -FilePath $newPath -Encoding utf8
}
使用 Node.js 脚本
const fs = require('fs');
const iconv = require('iconv-lite');
// 安装依赖：npm install iconv-lite
function convertToUtf8(inputFile, outputFile, sourceEncoding) {
// 读取原始文件
const buffer = fs.readFileSync(inputFile);
// 转换为UTF-8
const text = iconv.decode(buffer, sourceEncoding);
fs.writeFileSync(outputFile, text, 'utf-8');
console.log(`已转换: ${inputFile} -> ${outputFile}`);
}
// 使用
convertToUtf8('source.txt', 'target.txt', 'gbk');
批量转换脚本（综合版本）
#!/usr/bin/env python3
"""
文本编码批量转换工具
支持递归处理目录
"""
import os
import sys
import chardet
def detect_encoding(file_path):
"""检测文件编码"""
with open(file_path, 'rb') as f:
raw_data = f.read(10000) # 只读取前10000字节
result = chardet.detect(raw_data)
return result['encoding']
def convert_directory_to_utf8(directory_path, extensions=None):
"""批量转换目录下的所有文件"""
if extensions is None:
extensions = ['.txt', '.csv', '.html', '.xml', '.json', '.log']
for root, dirs, files in os.walk(directory_path):
for file in files:
if any(file.endswith(ext) for ext in extensions):
file_path = os.path.join(root, file)
try:
# 检测编码
encoding = detect_encoding(file_path)
# 读取并转换
with open(file_path, 'r', encoding=encoding) as f:
content = f.read()
# 写回UTF-8
with open(file_path, 'w', encoding='utf-8') as f:
f.write(content)
print(f"✓ {file_path} ({encoding} -> UTF-8)")
except Exception as e:
print(f"✗ {file_path}: {e}")
# 使用
if __name__ == "__main__":
if len(sys.argv) > 1:
target_dir = sys.argv[1]
else:
target_dir = "." # 当前目录
convert_directory_to_utf8(target_dir)
使用建议
- 先备份：转换前备份原始文件
- 确定原编码：使用
file命令或chardet检测原始编码 - 常见编码：
- Windows: GBK/GB2312, ANSI
- Linux: UTF-8
- Mac: UTF-8, 中文(Unicode)
选择哪个脚本取决于你的操作系统和需求：
- Linux/Mac: iconv 最简单
- Windows: PowerShell 自带
- 需要跨平台: Python 脚本最灵活
