Rust: Rust Iterators: Lazy Data Processing Pipelines
Last updated: 2026-08-26
An iterator is Rust’s “lazy data processing pipeline”—it doesn’t compute results immediately, but instead produces elements one by one, allowing you to process data sequences declaratively through chained calls.
An iterator is like a factory assembly line: data enters at one end, goes through a series of processes (filtering, transformation, extraction, aggregation), and is finally produced as a finished product at the other end. Each process does only one thing, but when combined, they can accomplish complex processing tasks.
1. The Story of an Assembly-Line Factory
(1) Pain: Processing data with loops is tedious and time-consuming
Xiao Ming is the assembly line supervisor at the Rust factory. He needs to process a batch of parts data:
- Sort out all qualified products (even-numbered items)
- Mark each part twice
- Take only the first 5
- Statistical Totals
He wrote it using a traditional for loop:
fn main() {
let parts = vec![1, 2, 3, 4, 5, 6, 7, 8, 9, 10];
let mut result = vec![];
let mut count = 0;
for &part in &parts {
if part % 2 == 0 { // Steps1: Filter
let doubled = part * 2; // Steps2: Convert
result.push(doubled);
count += 1;
if count == 5 { // Steps3: Excerpt
break;
}
}
}
let sum: i32 = result.iter().sum();
println!("Result: {:?}, Sum: {}", result, sum);
}
Although the code runs, the logic is scattered throughout the program. If the requirement changes to "skip the first two again" or "take an even number of items again," the entire loop would have to be rewritten.
(2) The Iterator Pipeline Approach
Rewrite the logic above using chained iterator calls:
fn main() {
let parts = vec![1, 2, 3, 4, 5, 6, 7, 8, 9, 10];
let sum: i32 = parts.iter()
.filter(|&&n| n % 2 == 0) // Steps1: Filter for Even Numbers
.map(|&n| n * 2) // Steps2: multiply by2
.take(5) // Steps3: Take the first 5
.sum(); // Steps4: Sum
println!("Sum: {}", sum);
}
The code has shifted from "how" to "what." Each line represents an independent process that can be inserted, deleted, or reordered at any time—just like adjusting workstations on an assembly line.
2. Conceptual Diagrams
The following Mermaid diagram illustrates the complete lazy evaluation process of the Iterator trait method chain from iter() to collect():
graph LR
A["Raw Data<br/>1..=20"] --> B["iter()<br/>Create an iterator"]
B --> C["filter(|n| n%2==0)<br/>Filter for Even Numbers"]
C --> D["map(|n| n*3)<br/>Each element ×3"]
D --> E["skip(2)<br/>Skip the first 2"]
E --> F["take(5)<br/>Take the first 5"]
F --> G["collect()<br/>Consumer: Triggered Evaluation"]
H["Lazy Evaluation: Build the pipeline only<br/>Do not calculate immediately"] -.-> C
H -.-> D
H -.-> E
H -.-> F
I["Consumer-Driven Execution<br/>Produce the final result"] -.-> G
G --> J["Results<br/>[18, 24, 30, 36, 42]"]
3. What You'll Learn
- The Iterator trait and the
nextmethod: The core contract of an iterator; understanding hownext()produces elements one by one - Iterator adapters:
map/filter/take/skip/chainand other conversion methods - Consumer:
collect/sum/count/fold, etc.—methods that drive the iterator to execute - Lazy evaluation: The adapter is not executed immediately; it waits until the consumer is called.
- Custom Iterators: Implement
Iterator traitfor your own types - Practical Guide to Chain Calls: Combining adapters and consumers to complete data processing tasks
4. Core Concepts
graph TB
A[Iterator Iterator] --> B[Iterator Adapters<br>Adapter]
A --> C[Consumer Consumer]
B --> D["map(|x| x+1) Convert"]
B --> E["filter(|x| x>0) Filter"]
B --> F["take(n) Take the first n"]
B --> G["skip(n) Skip n"]
B --> H["chain(other) Concat"]
B --> I["enumerate() Add an index"]
C --> J["collect() Collected into a set"]
C --> K["sum() Sum"]
C --> L["count() Count"]
C --> M["fold(init, fn) Collapse"]
C --> N["for x in iter Loop"]
B -.->|"❌ Inertia<br>If it isn't called, it won't run."| C
C --> O["Triggered Evaluation"]
(1) Iterator Adapters vs. Consumers
| Feature | Iterator Adapter | Consumer |
|---|---|---|
| Function | Convert an iterator (from one iterator to another) | Drive the iterator and produce the final result |
| Return Value | New Type for Iterator |
Specific Values (e.g., Vec<T>, i32, usize) |
| Lazy | Lazy (not executed immediately) | Greedy (executed immediately) |
| Typical Method | map, filter, take, skip, chain |
collect, sum, count, fold, for_each |
| Chain Position | Intermediate Step | Final Step |
| Example | `.map( | x |
(2) Quick Reference for Common Iterator Adapters
| Adapter | Function | Example | Result |
|---|---|---|---|
map(f) |
Apply the f transformation to each element | [1,2,3].iter().map(|x| x*2) |
2, 4, 6 |
filter(p) |
Keep elements that satisfy condition p | [1..=5].filter(|x| x%2==0) |
2, 4 |
take(n) |
Take only the first n elements | [1..].take(3) |
1, 2, 3 |
skip(n) |
Skip the first n elements | [1..=5].skip(2) |
3, 4, 5 |
chain(it) |
Append another iterator | [1,2].iter().chain([3,4].iter()) |
1, 2, 3, 4 |
enumerate() |
Assign an index to each element (i, val) |
['a','b'].iter().enumerate() |
(0,'a'), (1,'b') |
zip(it) |
Pair the two iterators one by one | [1,2].iter().zip(['a','b'].iter()) |
(1,'a'), (2,'b') |
rev() |
Reverse Iterator | [1..=3].rev() |
3, 2, 1 |
(3) Quick Reference for Common Consumer Methods
| Consumer | Effect | Example | Return Type | Is Greedy |
|---|---|---|---|---|
collect() |
Collect as a set | iter.collect::<Vec<_>>() |
B: FromIterator |
Yes |
sum() |
Sum | iter.sum::<i32>() |
S: Sum |
Yes |
count() |
count | iter.count() |
usize |
Yes |
fold(init, f) |
Cumulative calculation | iter.fold(0, |acc, x| acc + x) |
Initial value type | Yes |
reduce(f) |
Collapse without initial value | iter.reduce(|a, b| a + b) |
Option<Item> |
Yes |
for_each(f) |
Execute one by one (no return value) | iter.for_each(|x| println!(x)) |
() |
Yes |
any(p) |
Does a match exist? | iter.any(|x| x > 0) |
bool |
Yes |
all(p) |
Are all conditions met? | iter.all(|x| x > 0) |
bool |
Yes |
find(p) |
Find the first one that meets the criteria | iter.find(|x| *x > 3) |
Option<Item> |
Yes |
max() / min() |
Max/Min | iter.max() |
Option<Item> |
Yes |
5. Examples
▶ Example 1: The Iterator trait and the next() method—Understanding the Essence of Iterators (Difficulty ⭐)
Output:
<iter.next()>
<iter.next()>
<iter.next()>
<iter.next()>
<iter.next()>
<iter.next()>
<iter.next()>
for loop #0: <val>
// ============================================
// Manual Invocation next() Understanding How Iterators Work
// ============================================
fn main() {
let numbers = vec![10, 20, 30, 40, 50];
// iter() Returns an iterator,Does not consume vectors
let mut iter = numbers.iter();
// next() Every time it returns Option<&T>
// Some(&value) Indicates that there is another element
// None Indicates the end of the iteration
println!("{:?}", iter.next()); // Some(10)
println!("{:?}", iter.next()); // Some(20)
println!("{:?}", iter.next()); // Some(30)
println!("{:?}", iter.next()); // Some(40)
println!("{:?}", iter.next()); // Some(50)
println!("{:?}", iter.next()); // None
println!("{:?}", iter.next()); // None (A subsequent call still returns None)
// for A loop is next() syntactic sugar
let mut count = 0;
let iter2 = numbers.iter();
for val in iter2 {
println!("for loop #{}: {}", count, val);
count += 1;
}
}
Output:
Some(10)
Some(20)
Some(30)
Some(40)
Some(50)
None
None
for loop #0: 10
for loop #1: 20
for loop #2: 30
for loop #3: 40
for loop #4: 50
The core contract of an iterator is the
next()method: each call returnsSome(element), and when exhausted, it returnsNone. Theforloop is syntactic sugar that repeatedly callsnext()until it encountersNone. Understanding this means you understand the fundamentals of all iterators.
▶ Example 2: Chaining Iterator Adapters—map / filter / take / skip (Difficulty: ⭐⭐)
Output:
Pipeline result: <result>
Chained: <combined>
Indexed: <indexed>
// ============================================
// Combine Multiple Adapters to Build a Data Processing Pipeline
// ============================================
fn main() {
// Raw Data: 1 to 20
let data = 1..=20;
// Pipeline: Filter for Even Numbers → multiply by 3 → Skip the first 2 → Take the first 5
let result: Vec<i32> = data
.filter(|&n| n % 2 == 0) // [2,4,6,8,10,12,14,16,18,20]
.map(|n| n * 3) // [6,12,18,24,30,36,42,48,54,60]
.skip(2) // [18,24,30,36,42]
.take(5) // [18,24,30,36,42]
.collect(); // Triggered Evaluation,Collected Vec
println!("Pipeline result: {:?}", result);
// Another pipeline: use chain to Concat two slices
let first = vec!["A", "B", "C"];
let second = vec!["X", "Y", "Z"];
let combined: Vec<&str> = first.iter()
.chain(second.iter())
.copied()
.collect();
println!("Chained: {:?}", combined);
// Use enumerate to index elements
let fruits = vec!["apple", "banana", "cherry"];
let indexed: Vec<(usize, &str)> = fruits.iter()
.enumerate()
.map(|(i, &name)| (i + 1, name))
.collect();
println!("Indexed: {:?}", indexed);
}
Output:
Pipeline result: [18, 24, 30, 36, 42]
Chained: ["A", "B", "C", "X", "Y", "Z"]
Indexed: [(1, "apple"), (2, "banana"), (3, "cherry")]
An adapter chain is like the arrangement of workstations on an assembly line: each adapter does only one thing, and data flows sequentially through each workstation.
collect()is the demand signal at the end of the pipeline—without it, the workers in the pipeline won’t start working (lazy evaluation).
▶ Example 3: Custom Iterators—Implementing the Iterator trait for Your Type (Difficulty: ⭐⭐)
Output:
Fibonacci up to 50:
fib(<i>) = <n>
Even Fibonacci numbers up to 100:
<even_fibs>
// ============================================
// Custom Fibonacci Iterator
// Implementation Iterator trait Make any type iterable
// ============================================
// Fibonacci Sequence Generator
struct Fibonacci {
current: u64,
next: u64,
max: u64,
}
impl Fibonacci {
fn new(max: u64) -> Self {
Fibonacci {
current: 0,
next: 1,
max,
}
}
}
// Implementation Iterator trait It is the core of custom iterators.
impl Iterator for Fibonacci {
// Item The type of the elements returned by the iterator
type Item = u64;
// next() Back Option<Self::Item>
// Some(value) Indicates that there is another element
// None Indicates the end of the iteration
fn next(&mut self) -> Option<Self::Item> {
if self.current > self.max {
return None;
}
let result = self.current;
// Update to the next Fibonacci number
let new_next = self.current + self.next;
self.current = self.next;
self.next = new_next;
Some(result)
}
}
fn main() {
println!("Fibonacci up to 50:");
let fib = Fibonacci::new(50);
// Fibonacci It can now be used in for In a loop
for (i, n) in fib.enumerate() {
println!(" fib({}) = {}", i, n);
}
// It can also be used in conjunction with an adapter chain
println!("\nEven Fibonacci numbers up to 100:");
let even_fibs: Vec<u64> = Fibonacci::new(100)
.filter(|&n| n % 2 == 0)
.collect();
println!("{:?}", even_fibs);
}
Output:
Fibonacci up to 50:
fib(0) = 0
fib(1) = 1
fib(2) = 1
fib(3) = 2
fib(4) = 3
fib(5) = 5
fib(6) = 8
fib(7) = 13
fib(8) = 21
fib(9) = 34
Even Fibonacci numbers up to 100:
[0, 2, 8, 34]
To implement
Iterator trait, you only need to do one thing: definetype Item(element type) andfn next()(production rule). Once implemented, your type automatically gains all adapter methods (map,filter,take, etc.)—this is the embodiment of the “duck typing” concept in Rust: if you implementnext(), it can be used just like an iterator.
▶ Example 4: Consumer in Action—fold / sum / count / collect (Difficulty ⭐⭐⭐)
Output:
Sum: <total>
Count: <cnt>
10! = <factorial>
Sum (via fold): <sum_via_fold>
Count (via fold): <count_via_fold>
Doubled: <doubled>
Even set: <even_set>
Sum of first 5 even squares: <complex_result>
// ============================================
// Consumer:Drive the iterator to execute and produce the final result
// ============================================
fn main() {
let numbers = vec![1, 2, 3, 4, 5, 6, 7, 8, 9, 10];
// sum() — Sum
let total: i32 = numbers.iter().sum();
println!("Sum: {}", total);
// count() — Count
let cnt = numbers.iter().count();
println!("Count: {}", cnt);
// fold() — General Folding Operation (initial value + accumulator closure)
// Here, we calculate 10!
let factorial: u64 = (1..=10u64).fold(1, |acc, x| acc * x);
println!("10! = {}", factorial);
// fold() — Manual Implementation of sum and count
let sum_via_fold: i32 = numbers.iter().fold(0, |acc, &x| acc + x);
let count_via_fold: usize = numbers.iter().fold(0, |acc, _| acc + 1);
println!("Sum (via fold): {}", sum_via_fold);
println!("Count (via fold): {}", count_via_fold);
// collect() — Collected various types of collections
let doubled: Vec<i32> = numbers.iter().map(|&x| x * 2).collect();
println!("Doubled: {:?}", doubled);
let even_set: std::collections::HashSet<i32> = numbers.iter()
.filter(|&&x| x % 2 == 0)
.copied()
.collect();
println!("Even set: {:?}", even_set);
// General: find the sum of the first 5 even squares
let complex_result: i32 = (1..=100)
.filter(|&n| n % 2 == 0)
.map(|n| n * n)
.take(5)
.fold(0, |acc, n| acc + n);
println!("Sum of first 5 even squares: {}", complex_result);
}
Output:
Sum: 55
Count: 10
10! = 3628800
Sum (via fold): 55
Count (via fold): 10
Doubled: [2, 4, 6, 8, 10, 12, 14, 16, 18, 20]
Even set: {2, 4, 6, 8, 10}
Sum of first 5 even squares: 220
Consumers are the key components at the end of the pipeline:
sum(),count(), andfold()directly compute numerical results;collect()collects data into a set.fold()is the most general-purpose consumer—sum()andcount()are essentially specialized forms offold(). A pipeline must have a consumer to actually execute; otherwise, it’s all just talk.
▶ Example 5: Comprehensive Exercise—Implementing a Custom Iterator to Create a FizzBuzz Generator (Difficulty ⭐⭐⭐)
Output:
=== FizzBuzz (1-20) ===
<item>
1-100 Fizz occurrences: <fizz_count>
=== Fibonacci first 15 terms ===
<val>
Fibonacci < 1M sum of even numbers: <fib_sum>
// ============================================
// Comprehensive Example:Custom Iterators + Adapter Chain
// ============================================
struct FizzBuzz {
current: u32,
limit: u32,
}
impl FizzBuzz {
fn new(limit: u32) -> Self {
FizzBuzz { current: 0, limit }
}
}
impl Iterator for FizzBuzz {
type Item = String;
fn next(&mut self) -> Option<String> {
self.current += 1;
if self.current > self.limit {
return None;
}
let n = self.current;
let result = match (n % 3, n % 5) {
(0, 0) => "FizzBuzz".to_string(),
(0, _) => "Fizz".to_string(),
(_, 0) => "Buzz".to_string(),
_ => n.to_string(),
};
Some(result)
}
}
struct Fibonacci {
curr: u64,
next: u64,
}
impl Iterator for Fibonacci {
type Item = u64;
fn next(&mut self) -> Option<u64> {
let result = self.curr;
self.curr = self.next;
self.next = result + self.next;
Some(result)
}
}
fn main() {
println!("=== FizzBuzz (1-20) ===");
for item in FizzBuzz::new(20) {
print!("{} ", item);
}
println!();
let fizz_count = FizzBuzz::new(100)
.filter(|s| s.starts_with("Fizz"))
.count();
println!("1-100 Fizz occurrences: {}", fizz_count);
println!("\n=== Fibonacci first 15 terms ===");
let fib = Fibonacci { curr: 0, next: 1 };
for val in fib.take(15) {
print!("{} ", val);
}
println!();
let fib_sum: u64 = Fibonacci { curr: 1, next: 1 }
.take_while(|&x| x < 1_000_000)
.filter(|&x| x % 2 == 0)
.sum();
println!("Fibonacci < 1M sum of even numbers: {}", fib_sum);
}
Output:
=== FizzBuzz (1-20) ===
1 2 Fizz 4 Buzz Fizz 7 8 Fizz Buzz 11 Fizz 13 14 FizzBuzz 16 17 Fizz 19 Buzz
1-100 Fizz occurrences: 27
=== Fibonacci first 15 terms ===
0 1 1 2 3 5 8 13 21 34 55 89 144 233 377
Fibonacci < 1M sum of even numbers: 1089154
To create a custom iterator, simply implement the
next()method of theIteratortrait.FizzBuzzGenerators can be chained usingfilter/count;Fibonacciinfinite iterators can limit their output usingtake/take_while, and then aggregate the results usingfilter/sum.
❓ FAQ
iter(), into_iter(), and iter_mut()?iter() returns &T (an immutable reference) without transferring ownership; into_iter() returns T (ownership transfer), which consumes the original collection; iter_mut() returns &mut T (mutable reference), which allows elements to be modified. The for loop uses into_iter() by default.collect(), sum(), etc.). This is a core feature of Rust’s iterator design: zero-overhead abstraction that computes only when it’s actually needed.collect() How do I know what type to collect?.collect::<Vec<i32>>() or by declaring the variable type let v: Vec<i32> = iter.collect();. The compiler determines how to collect based on the implementation of the target type FromIterator.type Item in a custom iterator?Item is an associated type that specifies the type of Some returned by next(). For example, type Item = u64 in Iterator for Fibonacci indicates that next() returns Option<u64> each time. Associated types allow you to specify the type of elements produced by an iterator without requiring additional generic parameters.fold() and reduce()?fold() requires an initial value, while reduce() uses the first element as the initial value. fold(0, \|acc, x\| acc + x) starts counting from 0; reduce(\|acc, x\| acc + x) starts counting from the first element. fold() always returns the initial value type you specify, while reduce() returns Option<Self::Item> (or None if the iterator is empty).📖 Summary
- The Iterator trait is the foundation of Rust's iterator system; you only need to implement the
next()method, which returnsOption<Self::Item>. - Iterator adapters (
map,filter,take,skip,chain) are lazy—they only record operations and do not perform computations. - Consumers (
collect,sum,count,fold) are the endpoints of the pipeline—calling them triggers the actual computation - Chainable calls transform data-processing code from "how to do it" (imperative) to "what you want" (declarative), making it clearer and more composable
- Custom iterators only need to implement
Iterator trait; once implemented, they automatically gain access to all adapter methods. - Lazy evaluation is a core principle of iterator design—zero-overhead abstraction that computes values only when needed, avoiding unnecessary intermediate allocations.
📝 Exercises
-
Difficulty ⭐: Rewrite the following code using iterators. Extract all odd numbers from
[1, 2, 3, 4, 5, 6, 7, 8], multiply them by 10, store them in a Vec, and print them.RUST// Replace this loop with a chain of iterators let numbers = vec![1, 2, 3, 4, 5, 6, 7, 8]; let mut result = vec![]; for &n in &numbers { if n % 2 == 1 { result.push(n * 10); } } println!("{:?}", result); -
Difficulty ⭐⭐: Implement the
Iteratortrait forstruct StepRange { start: i32, end: i32, step: i32 }so that it can be used likefor n in StepRange::new(0, 10, 2), outputting 0, 2, 4, 6, 8, 10. Then usemapto square each value, and usecollectto collect them into a Vec. -
Difficulty ⭐⭐⭐: Write a function
fn word_count(text: &str) -> std::collections::HashMap<String, usize>that uses iterator methods to count the number of times each word appears in a piece of text. Requirements: Usesplit_whitespace()to split the text,mapto convert to lowercase, andfoldto construct a HashMap. Hint:HashMap’sentry()API, combined withor_insert(), makes counting easy.