Rust: Rust File I/O

Q: What is the difference between fs::read_to_string and BufReader ? When should each be used?

fs::read_to_string reads the entire file into memory at once (simple and brute-force), while BufReader reads line by line (to save memory). For small files (under a few MB), using read_to_string is simpler; for large files (over a few hundred MB), you must use BufReader , otherwise you'll run out of memory. Log files are typically between a few hundred KB and a few MB, so either option works, but BufReader is more elegant.

Q: What is the difference between File::create and OpenOptions::append ?

File::create always creates a new file (or clears and overwrites it if it exists), while OpenOptions::append appends content to the end of the file. File::create is equivalent to OpenOptions::new().write(true).create(true).truncate(true) , while append mode is equivalent to OpenOptions::new().append(true).open(path) . Append mode is typically used for logging, while overwrite mode is typically used for generating reports.

Q: What is the difference between Path and PathBuf ? Why are two needed?

Path is an immutable path slice (similar to &str ), while PathBuf is a mutable path string (similar to String ). PathBuf can be concatenated ( .join() ), modified ( .push() ), and have ownership transferred, while Path is read-only. The relationship between the two is similar to that of String and &str : PathBuf owns the data, while Path is a borrowed reference. Function parameters typically use &Path (or AsRef<Path> ), while variables are stored using PathBuf .

Q: How does the ? operator work in file I/O?

The ? operator checks the Result value: if it is Ok(val) , it extracts val ; if it is Err(e) , it returns Err(e.into()) early. In file I/O, almost all operations return Result<T, io::Error> . ? frees the code from match nesting—use ? if you're not concerned with specific error details; use match if you need to handle different errors differently.

Q: Do I need to manually close the file after file operations are complete?

No. In Rust, File automatically closes the file when drop (exits scope). Rust's ownership system ensures that when the File variable exits scope, the drop method of the Drop trait is automatically called to close the file handle. Similarly, BufWriter automatically flush the buffer when drop . You only need to ensure that you manually flush the buffer using flush() (if you need to ensure that data is written to disk before the file is closed).

Last updated: 2026-08-26

If we compare a program to a person, then variables and data structures are its "short-term memory" (brain), and the file system is its "long-term memory" (notebook). When the program is shut down, its short-term memory disappears, but the contents of the notebook remain forever. File I/O is the ability to "read and write to the notebook"—without it, the program would start with a blank page every time it was launched.

1. What You'll Learn

std::fs::read_to_string Read the entire file into a string in one go
std::fs::write Write a string to a file in one go
File::open Open file + File::create Create/overwrite file
BufReader / BufWriter Buffered read/write operations improve performance with large files
Path operations for Path and PathBuf (concatenation, parsing, and evaluation)
? Simplify error propagation in operators and handle I/O errors gracefully

2. The Story of the Log Analyzer

(1) The pain of manually flipping through logs

Xiao Lin is an operations engineer at the company, and the first thing he does every morning is manually check the server logs:

Open 10 log files, each containing several thousand lines
Use Ctrl+F to search for "ERROR" and count how many times it appears.
Manually count the number of each status code (200, 404, 500) using Excel
It takes 30 minutes every time, and I often miscount.
The boss asked, "Can you generate a report automatically?"—Xiao Lin was at his wits' end.

(2) Approaches to File I/O in Rust

Rust's standard library provides a complete set of file I/O tools, much like a "log analysis pipeline":

TEXT 📖 Display only

Log Files(Server)           → Raw Materials
  File::open / read_to_string → Conveyor Feed
  BufReader                   → Buffering(Assembly Line Buffer)
  Line-by-Line Analysis                    → Quality Inspection Station
  HashMap Statistical Status Codes          → Categorical Statistics
  File::create / write        → Generate and Export Report

RUST

use std::fs::File;
use std::io::{BufRead, BufReader, Write};

fn analyze_log(path: &str) -> std::io::Result<()> {
    // Open the file and create a buffer reader
    let file = File::open(path)?;
    let reader = BufReader::new(file);

    // Count the number of occurrences for each status code
    let mut counts: std::collections::HashMap<String, u32> =
        std::collections::HashMap::new();

    for line in reader.lines() {
        let line = line?;
        // Assumed Log Format: "2026-07-03 10:30:45 GET /api/users 200"
        if let Some(status) = line.split_whitespace().last() {
            *counts.entry(status.to_string()).or_insert(0) += 1;
        }
    }

    // Write to the report file
    let mut report = File::create("report.txt")?;
    writeln!(report, "=== Status Code Statistics Report ===")?;
    for (code, count) in &counts {
        writeln!(report, "  {}: {}", code, count)?;
    }

    println!("The report has been generated: report.txt");
    Ok(())
}

3. Core Concepts

(1) File I/O System

graph TB
    A[Rust Document I/O] --> B[Read a File]
    A --> C[Write to a file]
    A --> D[Path Operations]
    A --> E[Error Handling]

    B --> B1["std::fs::read_to_string"]
    B --> B2["File::open + read_to_string"]
    B --> B3["BufReader Read line by line"]

    C --> C1["std::fs::write"]
    C --> C2["File::create + write_all"]
    C --> C3["BufWriter Buffered Write"]

    D --> D1["Path(Immutable Slices)"]
    D --> D2["PathBuf(Variable strings)"]
    D --> D3["Path Concatenation .join()"]

    E --> E1["Result<T, io::Error>"]
    E --> E2["? Operator Propagation Error"]
    E --> E3["match Precision Processing"]

(2) Comparison of Reading Methods

Method	Function/Type	Use Case	Memory Usage	Performance
Single-pass read	`fs::read_to_string`	Small files (< 100MB)	Entire file contents	Fastest (one system call)
Manual Reading	`File::read_to_string`	Requires precise control	Entire file contents	Fast
Buffered Line-by-Line	`BufReader::read_line`	Large files/logs	One line of data	Moderate (reduces system calls)
Buffered Iteration	`BufReader::lines()`	Process line by line	One line of data	Convenient (Recommended)

(3) Comparison of Writing Methods

Method	Function/Type	Use Case	Features
One-time write	`fs::write`	Small files/simple content	Most concise
Create File	`File::create`	Overwrite	Empty if file exists
Append	`OpenOptions::append`	Append to log	Preserve original content
Buffered Write	`BufWriter`	High-volume writes	Reduces system calls

(4) Quick Reference for Opening Files

Method	Approach/Type	When the file exists	When the file does not exist	Applicable Scenarios
Open as read-only	`File::open(path)`	Open normally	Return error	Read existing file
Create/Overwrite	`File::create(path)`	Clear contents	Create new file	Write to new file
Append	`OpenOptions::new().append(true).open()`	Append to the end	Return error	Append to log
Create New File	`OpenOptions::new().write(true).create_new(true).open()`	Return Error	Create New File	Avoid Overwriting
Open for Reading and Writing	`OpenOptions::new().read(true).write(true).open()`	Open Normally	Return Error	Modify Existing File
Create and Read/Write	`OpenOptions::new().read(true).write(true).create(true).open()`	Open Normally	Create New File	Read/Write Configuration File

4. File I/O Examples

▶ Example 1: Basic Reading and Writing with Error Handling (Difficulty ⭐)

TEXT 📖 Display only

=== Simple Note-Taking Program ===

[Write] The note has been saved to: temp_note.txt
[Inspection] The file exists,Size: <content.len()> Byte

[Read] Note Content:
<read_content>
--- Memo Statistics ---
 Number of lines: <line_count>
 Number of words: <word_count>
 Number of characters: <char_count>

RUST

// ============================================
// Basic File Reading and Writing: fs::read_to_string + fs::write
// Demo:Create a temporary file、Enter content、Read the content、Cleanup
// Scene: Write a simple "Memo" program
// ============================================

use std::fs;
use std::path::Path;

fn main() -> std::io::Result<()> {
    println!("=== Simple Note-Taking Program ===\n");

    // Define the temporary file path (In the current directory temp_note.txt)
    let file_path = "temp_note.txt";

    // ---------- Write to a file ----------
    let content = "Today's To-Do List:
1. Study Rust Document I/O
2. Complete the Log Analyzer
3. Review Ownership and Borrowing

Rust Study Notes:
- Document I/O Usage std::fs Module
- BufReader Suitable for reading large files
- ? Simplifying Error Handling for Operators";

    fs::write(file_path, content)?;
    println!("[Write] The note has been saved to: {}", file_path);

    // Check if the file exists
    if Path::new(file_path).exists() {
        println!("[Inspection] The file exists,Size: {} Byte", content.len());
    }

    // ---------- Read a File ----------
    let read_content = fs::read_to_string(file_path)?;
    println!("\n[Read] Note Content:\n{}", read_content);

    // ---------- Statistical Information ----------
    let line_count = read_content.lines().count();
    let word_count: usize = read_content
        .split_whitespace()
        .count();
    let char_count = read_content.chars().count();

    println!("--- Memo Statistics ---");
    println!(" Number of lines: {}", line_count);
    println!(" Number of words: {}", word_count);
    println!(" Number of characters: {}", char_count);

    // ---------- Clear Temporary Files ----------
    fs::remove_file(file_path)?;
    println!("\n[Cleanup] The temporary file has been deleted.: {}", file_path);

    // Confirm that the file has been deleted
    assert!(!Path::new(file_path).exists(), "The file should be deleted.");
    println!("[Confirm] The files have been successfully deleted.");

    Ok(())
}

TEXT 📖 Display only

=== Simple Note-Taking Program ===

[Write] The note has been saved to: temp_note.txt
[Inspection] The file exists,Size: 87 Byte

[Read] Note Content:
Today's To-Do List:
1. Study Rust Document I/O
2. Complete the Log Analyzer
3. Review Ownership and Borrowing

Rust Study Notes:
- Document I/O Usage std::fs Module
- BufReader Suitable for reading large files
- ? Simplifying Error Handling for Operators

--- Memo Statistics ---
 Number of lines: 10
 Number of words: 24
 Number of characters: 87

[Cleanup] The temporary file has been deleted.: temp_note.txt
[Confirm] The files have been successfully deleted.

▶ Example 2: BufReader Reads Logs Line by Line (Difficulty ⭐⭐)

TEXT 📖 Display only

=== Server Log Analyzer ===

[Generate] Sample Log File: temp_server.log
[Analysis] Processing logs...

<report>
[Cleanup] The temporary file has been deleted.
[Statistics] Total Number of Lines: 0
[Statistics] Found <counts.len()> Different status codes

RUST

// ============================================
// BufReader Read line by line + Log Analysis
// Demo: Read the server logs, count the number of each status code
// Scene: Operations Engineers Automatically Analyze Log Files
// ============================================

use std::collections::HashMap;
use std::fs::File;
use std::io::{BufRead, BufReader, Write};

fn main() -> std::io::Result<()> {
    println!("=== Server Log Analyzer ===\n");

    // Step 1: Create a sample log file
    let log_path = "temp_server.log";
    create_sample_log(log_path)?;
    println!("[Generate] Sample Log File: {}", log_path);

    // Step 2: Analyze Logs
    println!("[Analysis] Processing logs...");
    let status_counts = analyze_log(log_path)?;

    // Step 3: Generate Report
    let report_path = "temp_report.txt";
    generate_report(&status_counts, report_path)?;

    // Step 4: Read and Print the Report
    let report = std::fs::read_to_string(report_path)?;
    println!("\n{}", report);

    // Step 5: Cleanup
    std::fs::remove_file(log_path)?;
    std::fs::remove_file(report_path)?;
    println!("[Cleanup] The temporary file has been deleted.");

    Ok(())
}

/// Create a sample log file
fn create_sample_log(path: &str) -> std::io::Result<()> {
    let mut file = File::create(path)?;

    // Simulated Server Logs for One Day
    let log_entries = vec![
        "2026-07-03 08:01:23 GET /api/users 200",
        "2026-07-03 08:02:15 POST /api/login 200",
        "2026-07-03 08:03:44 GET /api/products 404",
        "2026-07-03 08:05:12 GET /api/users 200",
        "2026-07-03 08:07:33 POST /api/order 500",
        "2026-07-03 08:10:01 GET /api/products 200",
        "2026-07-03 08:12:45 GET /api/users 304",
        "2026-07-03 08:15:22 POST /api/login 401",
        "2026-07-03 08:18:09 GET /api/products 200",
        "2026-07-03 08:20:55 POST /api/order 500",
        "2026-07-03 08:22:30 GET /api/users 200",
        "2026-07-03 08:25:18 GET /api/products 404",
        "2026-07-03 08:28:44 POST /api/login 200",
        "2026-07-03 08:30:01 GET /api/search 200",
        "2026-07-03 08:32:55 POST /api/order 200",
        "2026-07-03 08:35:12 GET /api/users 200",
        "2026-07-03 08:38:29 GET /api/products 304",
        "2026-07-03 08:40:44 POST /api/login 401",
        "2026-07-03 08:42:10 GET /api/search 500",
        "2026-07-03 08:45:33 POST /api/order 200",
        "2026-07-03 08:48:01 GET /api/users 200",
        "2026-07-03 08:50:22 GET /api/products 200",
        "2026-07-03 08:52:55 POST /api/login 200",
        "2026-07-03 08:55:18 GET /api/search 404",
        "2026-07-03 08:58:40 POST /api/order 500",
    ];

    for entry in log_entries {
        writeln!(file, "{}", entry)?;
    }

    Ok(())
}

/// Analyze Log Files,Count each type of HTTP Number of occurrences of status codes
fn analyze_log(path: &str) -> std::io::Result<HashMap<String, u32>> {
    let file = File::open(path)?;
    let reader = BufReader::new(file);

    let mut counts: HashMap<String, u32> = HashMap::new();
    let mut total_lines = 0u32;

    for line_result in reader.lines() {
        let line = line_result?;
        total_lines += 1;

        // Retrieve Status Code (The last field in the row)
        // Log Format: "2026-07-03 08:01:23 GET /api/users 200"
        if let Some(status) = line.split_whitespace().last() {
            *counts.entry(status.to_string()).or_insert(0) += 1;
        }
    }

    println!("[Statistics] Total Number of Lines: {}", total_lines);

    // Calculate the percentage of each status code
    let total: f64 = counts.values().sum::<u32>() as f64;
    println!("[Statistics] Found {} Different status codes", counts.len());

    let mut sorted: Vec<(&String, &u32)> = counts.iter().collect();
    sorted.sort_by_key(|(_, &count)| std::cmp::Reverse(count));
    for (code, count) in &sorted {
        let percentage = (*count as f64 / total * 100.0);
        println!("       {}: {} times ({:.1}%)", code, count, percentage);
    }

    Ok(counts)
}

/// Generate a statistical report file
fn generate_report(
    counts: &HashMap<String, u32>,
    report_path: &str,
) -> std::io::Result<()> {
    let mut file = File::create(report_path)?;

    writeln!(file, "========================================")?;
    writeln!(file, "  Server Log Status Code Statistics Report")?;
    writeln!(file, "  Date: 2026-07-03")?;
    writeln!(file, "========================================")?;
    writeln!(file)?;

    // Sort by quantity in descending order
    let mut sorted: Vec<(&String, &u32)> = counts.iter().collect();
    sorted.sort_by_key(|(_, &count)| std::cmp::Reverse(count));

    let total: u32 = counts.values().sum();

    for (code, count) in &sorted {
        let bar_length = (*count as f64 / total as f64 * 30.0) as usize;
        let bar = "=".repeat(bar_length);
        writeln!(file, "  {} | {} {:.1}%", code, bar, *count as f64 / total as f64 * 100.0)?;
    }

    writeln!(file)?;
    writeln!(file, "  Total Number of Requests: {}", total)?;
    writeln!(file, "  Success Rate (2xx): {:.1}%",
        counts.get("200").unwrap_or(&0) as &u32 + counts.get("304").unwrap_or(&0) as &u32 * 100 / total)?;
    writeln!(file, "========================================")?;

    Ok(())
}

TEXT 📖 Display only

=== Server Log Analyzer ===

[Generate] Sample Log File: temp_server.log
[Analysis] Processing logs...
[Statistics] Total Number of Lines: 25
[Statistics] Found 5 Different status codes
       200: 12 times (48.0%)
       404: 3 times (12.0%)
       500: 4 times (16.0%)
       304: 2 times (8.0%)
       401: 2 times (8.0%)

========================================
  Server Log Status Code Statistics Report
  Date: 2026-07-03
========================================

  200  | ============================== 48.0%
  500  | ========== 16.0%
  404  | ======== 12.0%
  304  | ===== 8.0%
  401  | ===== 8.0%

  Total Number of Requests: 25
  Success Rate (2xx): 56.0%
========================================

[Cleanup] The temporary file has been deleted.

▶ Example 3: BufWriter Buffered Writing and Path/PathBuf Path Operations (Difficulty ⭐⭐)

TEXT 📖 Display only

=== Log Merging Tool ===

--- Merged Logs ---
<merged_content>
--- Path Information ---
  file_name: <path.file_name().unwrap_or_default()>
  parent: <path.parent().unwrap_or_default()>
  extension: <path.extension().unwrap_or_default()>
  exists: <path.exists()>
  is_file: <path.is_file()>
  size: <metadata.len()> bytes
[Generate] <path.file_name().unwrap_or_default()>
[Merge] Done: <destination.file_name().unwrap_or_default()>

--- Clear Temporary Files ---
[Delete] <path.file_name().unwrap_or_default()>
[Delete] <merged_path.file_name().unwrap_or_default()>
[Done] All temporary files have been deleted.

RUST

// ============================================
// BufWriter Buffered Write + Path / PathBuf Path Operations
// Demo: Merging Multiple Files + Path Concatenation and Normalization
// Scene: Merge multiple log files into a single consolidated file
// ============================================

use std::fs::File;
use std::io::{BufRead, BufReader, BufWriter, Write};
use std::path::{Path, PathBuf};

fn main() -> std::io::Result<()> {
    println!("=== Log Merging Tool ===\n");

    // Usage PathBuf Build Path
    let base_dir = PathBuf::from(".");
    let log_files = vec![
        base_dir.join("temp_web.log"),
        base_dir.join("temp_api.log"),
        base_dir.join("temp_db.log"),
    ];

    // Generate three sample log files
    generate_log_files(&log_files)?;

    // Merge all logs into a single file
    let merged_path = base_dir.join("temp_merged.log");
    merge_log_files(&log_files, &merged_path)?;

    // Read and display the merge results
    let merged_content = std::fs::read_to_string(&merged_path)?;
    println!("--- Merged Logs ---");
    println!("{}", merged_content);

    // Usage Path Methods for Performing Path Operations
    println!("--- Path Information ---");
    let path = Path::new("temp_merged.log");
    println!("  file_name: {:?}", path.file_name().unwrap_or_default());
    println!("  parent: {:?}", path.parent().unwrap_or_default());
    println!("  extension: {:?}", path.extension().unwrap_or_default());
    println!("  exists: {}", path.exists());
    println!("  is_file: {}", path.is_file());

    // Get the file size
    let metadata = path.metadata()?;
    println!("  size: {} bytes", metadata.len());

    // Delete all temporary files
    cleanup_files(&log_files, &merged_path)?;

    Ok(())
}

/// Generate multiple sample log files
fn generate_log_files(file_paths: &[PathBuf]) -> std::io::Result<()> {
    let logs = vec![
        // Web Server Logs
        vec![
            "[WEB] 2026-07-03 10:00:00 GET /index.html 200",
            "[WEB] 2026-07-03 10:00:05 GET /style.css 200",
            "[WEB] 2026-07-03 10:01:00 POST /login 500",
        ],
        // API Server Logs
        vec![
            "[API] 2026-07-03 10:00:02 GET /api/users 200",
            "[API] 2026-07-03 10:00:10 POST /api/orders 201",
            "[API] 2026-07-03 10:01:05 GET /api/products 404",
        ],
        // Database Logs
        vec![
            "[DB]  2026-07-03 10:00:01 QUERY SELECT * FROM users 0.3ms",
            "[DB]  2026-07-03 10:00:11 QUERY INSERT INTO orders 1.2ms",
            "[DB]  2026-07-03 10:01:01 QUERY SELECT * FROM products 0.5ms",
        ],
    ];

    for (i, path) in file_paths.iter().enumerate() {
        let file = File::create(path)?;
        let mut writer = BufWriter::new(file);

        for entry in &logs[i] {
            writeln!(writer, "{}", entry)?;
        }
        // BufWriter will be drop Automatically flush,But you can also do it manually flush
        writer.flush()?;
        println!("[Generate] {:?}", path.file_name().unwrap_or_default());
    }

    Ok(())
}

/// Merge Multiple Log Files
fn merge_log_files(
    sources: &[PathBuf],
    destination: &PathBuf,
) -> std::io::Result<()> {
    // Usage BufWriter Write to the target file
    let dest_file = File::create(destination)?;
    let mut writer = BufWriter::new(dest_file);

    // Write to the header
    writeln!(writer, "=== Consolidated Log Report ===")?;
    writeln!(writer, "Generation Time: 2026-07-03 10:05:00")?;
    writeln!(writer, "Number of source files: {}", sources.len())?;
    writeln!(writer, "{}", "=".repeat(50))?;
    writeln!(writer)?;

    // Read and write on a per-file basis
    for source_path in sources {
        let file_name = source_path.file_name()
            .unwrap_or_default()
            .to_string_lossy();

        writeln!(writer, "--- Source: {} ---", file_name)?;

        let src_file = File::open(source_path)?;
        let reader = BufReader::new(src_file);

        for line_result in reader.lines() {
            let line = line_result?;
            writeln!(writer, "{}", line)?;
        }

        writeln!(writer)?;  // Blank line separator
    }

    // Write to the end of the statistics
    writeln!(writer, "{}", "=".repeat(50))?;
    writeln!(writer, "Merge Complete")?;

    writer.flush()?;
    println!("[Merge] Done: {:?}", destination.file_name().unwrap_or_default());

    Ok(())
}

/// Delete all temporary files
fn cleanup_files(log_files: &[PathBuf], merged_path: &PathBuf) -> std::io::Result<()> {
    println!("\n--- Clear Temporary Files ---");

    for path in log_files {
        if path.exists() {
            std::fs::remove_file(path)?;
            println!("[Delete] {:?}", path.file_name().unwrap_or_default());
        }
    }

    if merged_path.exists() {
        std::fs::remove_file(merged_path)?;
        println!("[Delete] {:?}", merged_path.file_name().unwrap_or_default());
    }

    println!("[Done] All temporary files have been deleted.");
    Ok(())
}

TEXT 📖 Display only

=== Log Merging Tool ===

[Generate] "temp_web.log"
[Generate] "temp_api.log"
[Generate] "temp_db.log"
[Merge] Done: "temp_merged.log"

--- Merged Logs ---
=== Consolidated Log Report ===
Generation Time: 2026-07-03 10:05:00
Number of source files: 3
==================================================

--- Source: temp_web.log ---
[WEB] 2026-07-03 10:00:00 GET /index.html 200
[WEB] 2026-07-03 10:00:05 GET /style.css 200
[WEB] 2026-07-03 10:01:00 POST /login 500

--- Source: temp_api.log ---
[API] 2026-07-03 10:00:02 GET /api/users 200
[API] 2026-07-03 10:00:10 POST /api/orders 201
[API] 2026-07-03 10:01:05 GET /api/products 404

--- Source: temp_db.log ---
[DB]  2026-07-03 10:00:01 QUERY SELECT * FROM users 0.3ms
[DB]  2026-07-03 10:00:11 QUERY INSERT INTO orders 1.2ms
[DB]  2026-07-03 10:01:01 QUERY SELECT * FROM products 0.5ms

==================================================
Merge Complete

--- Path Information ---
  file_name: "temp_merged.log"
  parent: ""
  extension: "log"
  exists: true
  is_file: true
  size: 638 bytes

--- Clear Temporary Files ---
[Delete] "temp_web.log"
[Delete] "temp_api.log"
[Delete] "temp_db.log"
[Delete] "temp_merged.log"
[Done] All temporary files have been deleted.

▶ Example 4: A Complete Log Analyzer—Simulating a Real-World Scenario (Difficulty ⭐⭐⭐)

TEXT 📖 Display only

=== Complete Log Analyzer ===

[1/4] The simulation log has been generated: temp_access.log
[2/4] Log parsing complete: <entries.len()> records
[3/4] HTML The report has been generated: temp_report.html

--- Report Preview(first 20 lines) ---
<line>
... (total <report_content.lines().count()> lines)

[4/4] Temporary files have been cleared.

--- Analysis Results ---
Total Number of Requests: <result.total_requests>
Error Rate: <result.error_rate>%

Status Code Distribution:
  <code> <bar> (<count>)

HTTP Method Distribution:
  <method>: <count>

Popular Routes (Top 5):
  <i + 1>. <path> (<count> times)

RUST

// ============================================
// Comprehensive Log Analyzer——Simulate real-world operations and maintenance scenarios
// Demo:Comprehensive Implementation Document I/O、Path Operations、Error Handling
// Features: Read the log -> Analysis -> Generate HTML Report -> Cleanup
// ============================================

use std::collections::HashMap;
use std::fs::File;
use std::io::{BufRead, BufReader, Write};
use std::path::Path;

/// Log Entry Structure
#[derive(Debug)]
struct LogEntry {
    timestamp: String,
    method: String,
    path: String,
    status_code: u16,
}

/// Analysis Results
struct AnalysisResult {
    total_requests: u32,
    status_counts: HashMap<u16, u32>,
    method_counts: HashMap<String, u32>,
    top_paths: Vec<(String, u32)>,
    error_rate: f64,
}

fn main() -> std::io::Result<()> {
    println!("=== Complete Log Analyzer ===\n");

    let log_path = "temp_access.log";
    let report_path = "temp_report.html";

    // Generate a sample log
    generate_access_log(log_path)?;
    println!("[1/4] The simulation log has been generated: {}", log_path);

    // Analysis Log
    let entries = parse_log(log_path)?;
    println!("[2/4] Log parsing complete: {} records", entries.len());

    // Analyze Data
    let result = analyze_entries(&entries);
    print_analysis(&result);

    // Generate HTML Report
    generate_html_report(&result, report_path)?;
    println!("[3/4] HTML The report has been generated: {}", report_path);

    // Display Report Content
    let report_content = std::fs::read_to_string(report_path)?;
    println!("\n--- Report Preview(first 20 lines) ---");
    for line in report_content.lines().take(20) {
        println!("{}", line);
    }
    println!("... (total {} lines)", report_content.lines().count());

    // Cleanup
    cleanup(log_path, report_path)?;
    println!("\n[4/4] Temporary files have been cleared.");

    Ok(())
}

/// Generate a simulated access log
fn generate_access_log(path: &str) -> std::io::Result<()> {
    let paths = [
        "/index.html", "/api/users", "/api/products",
        "/api/orders", "/login", "/search",
        "/about", "/contact", "/api/settings",
    ];
    let methods = ["GET", "POST", "PUT", "DELETE"];
    let statuses = [200, 200, 200, 200, 200, 201, 204, 301, 304, 400, 401, 403, 404, 500, 502];

    let file = File::create(path)?;
    let mut writer = std::io::BufWriter::new(file);

    for i in 0..100 {
        let hour = 8 + i / 12;
        let minute = (i * 3) % 60;
        let second = (i * 17) % 60;

        let method = methods[i % methods.len()];
        let url = paths[i % paths.len()];
        let status = statuses[i % statuses.len()];

        writeln!(
            writer,
            "2026-07-03 {:02}:{:02}:{:02} {} {} {}",
            hour, minute, second, method, url, status
        )?;
    }

    writer.flush()?;
    Ok(())
}

/// Analyzing Log Files
fn parse_log(path: &str) -> std::io::Result<Vec<LogEntry>> {
    let file = File::open(path)?;
    let reader = BufReader::new(file);
    let mut entries = Vec::new();

    for line_result in reader.lines() {
        let line = line_result?;
        let parts: Vec<&str> = line.split_whitespace().collect();

        if parts.len() >= 4 {
            let entry = LogEntry {
                timestamp: parts[0].to_string() + " " + parts[1],
                method: parts[2].to_string(),
                path: parts[3].to_string(),
                status_code: parts[4].parse().unwrap_or(0),
            };
            entries.push(entry);
        }
    }

    Ok(entries)
}

/// Analyze Log Entries
fn analyze_entries(entries: &[LogEntry]) -> AnalysisResult {
    let total = entries.len() as u32;

    // Statistical Status Codes
    let mut status_counts: HashMap<u16, u32> = HashMap::new();
    let mut method_counts: HashMap<String, u32> = HashMap::new();
    let mut path_counts: HashMap<String, u32> = HashMap::new();

    for entry in entries {
        *status_counts.entry(entry.status_code).or_insert(0) += 1;
        *method_counts.entry(entry.method.clone()).or_insert(0) += 1;
        *path_counts.entry(entry.path.clone()).or_insert(0) += 1;
    }

    // Calculate the error rate (4xx + 5xx)
    let error_count: u32 = status_counts.iter()
        .filter(|(&code, _)| code >= 400)
        .map(|(_, &count)| count)
        .sum();
    let error_rate = if total > 0 {
        error_count as f64 / total as f64 * 100.0
    } else {
        0.0
    };

    // Most Popular Routes
    let mut top_paths: Vec<(String, u32)> = path_counts.into_iter().collect();
    top_paths.sort_by(|a, b| b.1.cmp(&a.1));
    top_paths.truncate(5);

    AnalysisResult {
        total_requests: total,
        status_counts,
        method_counts,
        top_paths,
        error_rate,
    }
}

/// Print the analysis results
fn print_analysis(result: &AnalysisResult) {
    println!("\n--- Analysis Results ---");
    println!("Total Number of Requests: {}", result.total_requests);
    println!("Error Rate: {:.1}%", result.error_rate);

    println!("\nStatus Code Distribution:");
    let mut sorted_status: Vec<_> = result.status_counts.iter().collect();
    sorted_status.sort_by_key(|(_, &c)| std::cmp::Reverse(c));
    for (&code, &count) in &sorted_status {
        let bar = "=".repeat((count as f64 / result.total_requests as f64 * 20.0) as usize);
        println!("  {} {} ({})", code, bar, count);
    }

    println!("\nHTTP Method Distribution:");
    for (method, count) in &result.method_counts {
        println!("  {}: {}", method, count);
    }

    println!("\nPopular Routes (Top 5):");
    for (i, (path, count)) in result.top_paths.iter().enumerate() {
        println!("  {}. {} ({} times)", i + 1, path, count);
    }
}

/// Generate HTML Analysis Report on Formats
fn generate_html_report(
    result: &AnalysisResult,
    path: &str,
) -> std::io::Result<()> {
    let file = File::create(path)?;
    let mut writer = std::io::BufWriter::new(file);

    writeln!(writer, "<!DOCTYPE html>")?;
    writeln!(writer, "<html lang=\"zh\">")?;
    writeln!(writer, "<head><meta charset=\"UTF-8\"><title>Log Analysis Report</title>")?;
    writeln!(writer, "<style>
        body {{ font-family: Arial, sans-serif; margin: 40px; background: #f5f5f5; }}
        h1 {{ color: #333; border-bottom: 2px solid #4CAF50; padding-bottom: 10px; }}
        table {{ border-collapse: collapse; width: 100%; margin: 20px 0; }}
        th, td {{ border: 1px solid #ddd; padding: 8px 12px; text-align: left; }}
        th {{ background: #4CAF50; color: white; }}
        .error {{ color: red; font-weight: bold; }}
        .summary {{ background: white; padding: 20px; border-radius: 5px; }}
    </style></head><body>")?;

    writeln!(writer, "<h1>Server Log Analysis Report</h1>")?;
    writeln!(writer, "<div class=\"summary\">")?;
    writeln!(writer, "<p>Total Number of Requests: <strong>{}</strong></p>", result.total_requests)?;
    writeln!(writer, "<p>Error Rate: <strong class=\"error\">{:.1}%</strong></p>", result.error_rate)?;
    writeln!(writer, "</div>")?;

    // Status Code Table
    writeln!(writer, "<h2>Status Code Distribution</h2><table><tr><th>Status Code</th><th>Count</th><th>Percentage</th></tr>")?;
    let mut sorted_status: Vec<_> = result.status_counts.iter().collect();
    sorted_status.sort_by_key(|(_, &c)| std::cmp::Reverse(c));
    for (&code, &count) in &sorted_status {
        let pct = count as f64 / result.total_requests as f64 * 100.0;
        writeln!(writer, "<tr><td>{}</td><td>{}</td><td>{:.1}%</td></tr>", code, count, pct)?;
    }
    writeln!(writer, "</table>")?;

    writeln!(writer, "</body></html>")?;
    writer.flush()?;

    Ok(())
}

/// Clear Temporary Files
fn cleanup(log_path: &str, report_path: &str) -> std::io::Result<()> {
    let log = Path::new(log_path);
    let report = Path::new(report_path);

    if log.exists() {
        std::fs::remove_file(log)?;
    }
    if report.exists() {
        std::fs::remove_file(report)?;
    }

    Ok(())
}

TEXT 📖 Display only

=== Complete Log Analyzer ===

[1/4] The simulation log has been generated: temp_access.log
[2/4] Log parsing complete: 100 records

--- Analysis Results ---
Total Number of Requests: 100
Error Rate: 26.0%

Status Code Distribution:
  200 ============== (33)
  404 ====== (13)
  500 ====== (13)
  401 ====== (7)
  400 ====== (6)
  201 ====== (6)
  204 ====== (6)
  304 ====== (6)
  301 ====== (5)
  403 ====== (5)
  502 ====== (0)

HTTP Method Distribution:
  GET: 29
  POST: 28
  PUT: 25
  DELETE: 18

Popular Routes (Top 5):
  1. /api/users (13 times)
  2. /api/products (12 times)
  3. /index.html (11 times)
  4. /api/orders (11 times)
  5. /login (11 times)

[3/4] HTML The report has been generated: temp_report.html

--- Report Preview(first 20 lines) ---
<!DOCTYPE html>
<html lang="zh">
<head><meta charset="UTF-8"><title>Log Analysis Report</title>
<style>
        body {{ font-family: Arial, sans-serif; margin: 40px; background: #f5f5f5; }}
        h1 {{ color: #333; border-bottom: 2px solid #4CAF50; padding-bottom: 10px; }}
        table {{ border-collapse: collapse; width: 100%; margin: 20px 0; }}
        th, td {{ border: 1px solid #ddd; padding: 8px 12px; text-align: left; }}
        th {{ background: #4CAF50; color: white; }}
        .error {{ color: red; font-weight: bold; }}
        .summary {{ background: white; padding: 20px; border-radius: 5px; }}
</style></head><body>
<h1>Server Log Analysis Report</h1>
<div class="summary">
<p>Total Number of Requests: <strong>100</strong></p>
<p>Error Rate: <strong class="error">26.0%</strong></p>
</div>
... (total 49 lines)

[4/4] Temporary files have been cleared.

▶ Example 5: Comprehensive Exercise—CSV Parsing and Statistics (Difficulty ⭐⭐⭐)

TEXT 📖 Display only

CSV Written: temp_scores.csv

=== Analysis Results ===
<r.name>: <r.score> points, Level <r.grade>

Average: <avg>, Highest: <max>, Lowest: <min>

The report has been submitted.: temp_report.txt

RUST

// ============================================
// Comprehensive Example: File I/O + Error Handling + Statistics
// ============================================

use std::fs;
use std::io::{Write, BufWriter};
use std::path::Path;

struct Record {
    name: String,
    score: u32,
    grade: String,
}

fn parse_csv_line(line: &str) -> Option<Record> {
    let parts: Vec<&str> = line.split(',').collect();
    if parts.len() != 3 { return None; }
    let score = parts[1].trim().parse::<u32>().ok()?;
    Some(Record {
        name: parts[0].trim().to_string(),
        score,
        grade: parts[2].trim().to_string(),
    })
}

fn analyze(records: &[Record]) -> (f64, u32, u32) {
    if records.is_empty() { return (0.0, 0, 0); }
    let sum: u32 = records.iter().map(|r| r.score).sum();
    let avg = sum as f64 / records.len() as f64;
    let max = records.iter().map(|r| r.score).max().unwrap();
    let min = records.iter().map(|r| r.score).min().unwrap();
    (avg, max, min)
}

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let csv_data = "name,score,grade\nAlice,95,A\nBob,72,C\nCharlie,88,B\nDavid,58,F\nEve,91,A";
    let input_path = "temp_scores.csv";
    let report_path = "temp_report.txt";

    fs::write(input_path, csv_data)?;
    println!("CSV Written: {}", input_path);

    let content = fs::read_to_string(input_path)?;
    let records: Vec<Record> = content.lines()
        .skip(1)
        .filter_map(parse_csv_line)
        .collect();

    println!("\n=== Analysis Results ===");
    for r in &records {
        println!("{}: {} points, Level {}", r.name, r.score, r.grade);
    }

    let (avg, max, min) = analyze(&records);
    println!("\nAverage: {:.1}, Highest: {}, Lowest: {}", avg, max, min);

    let a_count = records.iter().filter(|r| r.grade == "A").count();
    let f_count = records.iter().filter(|r| r.grade == "F").count();

    let file = fs::File::create(report_path)?;
    let mut writer = BufWriter::new(file);
    writeln!(writer, "=== Grades Statistics Report ===")?;
    writeln!(writer, "Total number of people: {}", records.len())?;
    writeln!(writer, "Average Score: {:.1}", avg)?;
    writeln!(writer, "Highest Score: {}, Lowest score: {}", max, min)?;
    writeln!(writer, "A Level: {} people, F Level: {} people", a_count, f_count)?;
    writer.flush()?;
    println!("\nThe report has been submitted.: {}", report_path);

    let report = fs::read_to_string(report_path)?;
    println!("\n{}", report);

    fs::remove_file(input_path)?;
    fs::remove_file(report_path)?;
    println!("Temporary files have been cleared.");

    Ok(())
}

TEXT 📖 Display only

CSV Written: temp_scores.csv

=== Analysis Results ===
Alice: 95 points, Level A
Bob: 72 points, Level C
Charlie: 88 points, Level B
David: 58 points, Level F
Eve: 91 points, Level A

Average: 80.8, Highest: 95, Lowest: 58

The report has been submitted.: temp_report.txt

=== Grades Statistics Report ===
Total number of people: 5
Average Score: 80.8
Highest Score: 95, Lowest score: 58
A Level: 2 people, F Level: 1 people

Temporary files have been cleared.

❓ FAQ

Q What is the difference between fs::read_to_string and BufReader? When should each be used?

A fs::read_to_string reads the entire file into memory at once (simple and brute-force), while BufReader reads line by line (to save memory). For small files (under a few MB), using read_to_string is simpler; for large files (over a few hundred MB), you must use BufReader, otherwise you'll run out of memory. Log files are typically between a few hundred KB and a few MB, so either option works, but BufReader is more elegant.

Q What is the difference between File::create and OpenOptions::append?

A File::create always creates a new file (or clears and overwrites it if it exists), while OpenOptions::append appends content to the end of the file. File::create is equivalent to OpenOptions::new().write(true).create(true).truncate(true), while append mode is equivalent to OpenOptions::new().append(true).open(path). Append mode is typically used for logging, while overwrite mode is typically used for generating reports.

Q What is the difference between Path and PathBuf? Why are two needed?

A Path is an immutable path slice (similar to &str), while PathBuf is a mutable path string (similar to String). PathBuf can be concatenated (.join()), modified (.push()), and have ownership transferred, while Path is read-only. The relationship between the two is similar to that of String and &str: PathBuf owns the data, while Path is a borrowed reference. Function parameters typically use &Path (or AsRef<Path>), while variables are stored using PathBuf.

Q How does the ? operator work in file I/O?

A The ? operator checks the Result value: if it is Ok(val), it extracts val; if it is Err(e), it returns Err(e.into()) early. In file I/O, almost all operations return Result<T, io::Error>. ? frees the code from match nesting—use ? if you're not concerned with specific error details; use match if you need to handle different errors differently.

Q Do I need to manually close the file after file operations are complete?

A No. In Rust, File automatically closes the file when drop (exits scope). Rust's ownership system ensures that when the File variable exits scope, the drop method of the Drop trait is automatically called to close the file handle. Similarly, BufWriter automatically flush the buffer when drop. You only need to ensure that you manually flush the buffer using flush() (if you need to ensure that data is written to disk before the file is closed).

📖 Summary

std::fs::read_to_string and std::fs::write are the most concise "one-time read/write" methods, suitable for small files
File::open Read an existing file; File::create Create a new file or overwrite an existing file
BufReader The buffer reader processes large files line by line using lines(), reducing the number of system calls
BufWriter The buffer writer combines multiple small writes into a single large write, improving write performance
Path (immutable slice) and PathBuf (mutable string) provide cross-platform path operations and support methods such as .join(), .file_name(), and .extension().
? operator automatically propagates errors from Result, making file I/O code concise and safe

📝 Exercises

Difficulty ⭐: Write a program that creates a file named note.txt, writes three paragraphs to it (using the writeln! macro), then reads and prints the file's contents. Finally, delete the file. Use the ? operator to handle errors, and the main function to return std::io::Result<()>.
Difficulty ⭐⭐: Write a "CSV reader" program. Create a CSV file containing the following content (simulated):
TEXT 📖 Display only
```
Name,Age,City
 Alice,28,New York
 Bob,32,London
 Charlie,25,Tokyo
```
Use BufReader to read line by line, skip the first line (the header), parse each line, and calculate the average age. Write the results to result.txt. Finally, delete all temporary files.
Difficulty ⭐⭐⭐: Write a "File Search and Statistics" tool. Recursively scan a specified directory (using std::fs::read_dir) and count the number of files for each file extension. Generate a report sorted in descending order by count (write to summary.txt). Requirements: (1) Use Path and PathBuf to handle paths; (2) Use BufWriter to write the report; (3) Use ? to propagate errors; (4) The output format should be a table containing the file extension, count, percentage, and an ASCII bar chart.

Web-Tutorial Tech Team

A team of developers maintaining programming tutorials. Each tutorial is written and reviewed by developers with expertise in that field. We work to keep our content accurate and reliable — if you spot an issue, please let us know.

About Us

Rust: Rust File I/O

1. What You'll Learn

2. The Story of the Log Analyzer

(1) The pain of manually flipping through logs

(2) Approaches to File I/O in Rust

3. Core Concepts

(1) File I/O System

(2) Comparison of Reading Methods

(3) Comparison of Writing Methods

(4) Quick Reference for Opening Files

4. File I/O Examples

▶ Example 1: Basic Reading and Writing with Error Handling (Difficulty ⭐)

▶ Example 2: BufReader Reads Logs Line by Line (Difficulty ⭐⭐)

▶ Example 3: BufWriter Buffered Writing and Path/PathBuf Path Operations (Difficulty ⭐⭐)

▶ Example 4: A Complete Log Analyzer—Simulating a Real-World Scenario (Difficulty ⭐⭐⭐)

▶ Example 5: Comprehensive Exercise—CSV Parsing and Statistics (Difficulty ⭐⭐⭐)

❓ FAQ

📖 Summary

📝 Exercises

🙏 帮我们做得更好