脚本如何实现文件内容拆分功能

本文目录导读：

1. Shell脚本 (Linux/Unix)
2. Python脚本
3. PowerShell脚本 (Windows)
4. Node.js脚本
5. 常用命令行工具
6. 选择建议

Shell脚本 (Linux/Unix)

按行数拆分

#!/bin/bash
# split_by_lines.sh
# 用法: ./split_by_lines.sh input.txt 100
INPUT_FILE=$1
LINES_PER_FILE=$2
if [ -z "$INPUT_FILE" ] || [ -z "$LINES_PER_FILE" ]; then
    echo "用法: $0 <文件名> <每个文件行数>"
    exit 1
fi
# 使用split命令按行拆分
split -l $LINES_PER_FILE $INPUT_FILE "split_"

按文件大小拆分

#!/bin/bash
# 按文件大小拆分，例如每个文件10M
split -b 10M large_file.txt "part_"

Python脚本

按行数拆分

#!/usr/bin/env python3
# split_file.py
import os
import sys
def split_file_by_lines(input_file, lines_per_file=1000):
    """按行数拆分文件"""
    file_counter = 1
    line_counter = 0
    output_file = None
    try:
        with open(input_file, 'r', encoding='utf-8') as f:
            for line in f:
                # 每lines_per_file行创建新文件
                if line_counter % lines_per_file == 0:
                    if output_file:
                        output_file.close()
                    filename = f"split_{file_counter}.txt"
                    output_file = open(filename, 'w', encoding='utf-8')
                    file_counter += 1
                output_file.write(line)
                line_counter += 1
            if output_file:
                output_file.close()
    except Exception as e:
        print(f"错误: {e}")
        sys.exit(1)
if __name__ == "__main__":
    if len(sys.argv) != 3:
        print("用法: python3 split_file.py <输入文件> <每个文件行数>")
        sys.exit(1)
    input_file = sys.argv[1]
    lines_per_file = int(sys.argv[2])
    split_file_by_lines(input_file, lines_per_file)

按关键字拆分

#!/usr/bin/env python3
# split_by_keyword.py
def split_by_keyword(input_file, keyword):
    """按关键字拆分文件，关键字作为分割点"""
    file_counter = 1
    output_file = open(f"part_{file_counter}.txt", 'w', encoding='utf-8')
    with open(input_file, 'r', encoding='utf-8') as f:
        for line in f:
            if keyword in line:
                output_file.close()
                file_counter += 1
                output_file = open(f"part_{file_counter}.txt", 'w', encoding='utf-8')
            output_file.write(line)
    output_file.close()
# 使用示例
split_by_keyword("data.txt", "---分割线---")

PowerShell脚本 (Windows)

按行数拆分

# split_file.ps1
$inputFile = "large.txt"
$linesPerFile = 500
$lineCount = 0
$fileCounter = 1
$outputFile = "part_$fileCounter.txt"
Get-Content $inputFile | ForEach-Object {
    $lineCount++
    if ($lineCount -gt $linesPerFile) {
        $fileCounter++
        $outputFile = "part_$fileCounter.txt"
        $lineCount = 1
    }
    $_ | Out-File -FilePath $outputFile -Append
}

分割

# 按特定标记分割
$inputFile = "data.txt"
$splitMarker = "CHAPTER"
$content = Get-Content $inputFile -Raw
$chapters = $content -split $splitMarker
for ($i=1; $i -lt $chapters.Length; $i++) {
    $chapters[$i] | Out-File "chapter_$i.txt"
}

Node.js脚本

// split_file.js
const fs = require('fs');
const readline = require('readline');
const inputFile = process.argv[2];
const linesPerFile = parseInt(process.argv[3]) || 1000;
let fileIndex = 1;
let lineCount = 0;
let writeStream = fs.createWriteStream(`part_${fileIndex}.txt`);
const rl = readline.createInterface({
    input: fs.createReadStream(inputFile),
    crlfDelay: Infinity
});
rl.on('line', (line) => {
    if (lineCount >= linesPerFile) {
        writeStream.end();
        fileIndex++;
        writeStream = fs.createWriteStream(`part_${fileIndex}.txt`);
        lineCount = 0;
    }
    writeStream.write(line + '\n');
    lineCount++;
});
rl.on('close', () => {
    writeStream.end();
    console.log('文件拆分完成');
});

常用命令行工具

# 按大小拆分（Linux/Mac）
split -b 1M input.txt output_prefix
# 按行数拆分
split -l 1000 input.txt output_prefix
# 加数字后缀
split -d -l 10 input.txt part_
# 按空格或分隔符拆分
csplit input.txt '/PATTERN/' '{*}'

选择建议

• Linux用户: 首选split命令，简单高效
• Windows用户: 使用PowerShell脚本
• 跨平台需求: Python或Node.js脚本
• 复杂分割规则: Python更适合
• 处理大文件: Node.js或Python流式处理更高效

选择哪种方法取决于你的具体需求、操作系统和使用场景。