Parseq

I’ve published my own parallel sequential iterator library for Rust, Parseq. Parseq provides an extension trait adding a map_parallel method to the standard iterator trait. It’s a drop-in-replacement for the standard map method.

use std::time::Duration;
use parseq::ParallelIterator;

let mut iter = [3,2,1]
  .into_iter()
  .map_parallel(|i| {
    // Insert heavy computation here ...
    std::thread::sleep(Duration::from_millis(100*i));
    2*i
  });

assert_eq!(iter.next(), Some(6));
assert_eq!(iter.next(), Some(4));
assert_eq!(iter.next(), Some(2));
assert_eq!(iter.next(), None);

See the repository for a real world example, and docs.io for the documentation.

Parseq is of course not the first crate providing a parallel map function for iterators. But I think it comes with a unique set of features:

• Parseq utilizes a configurable number of worker threads
• Parseq preserves the order of the original iterator
• Parseq is lazy in the sense that it doesn’t consume from the original iterator before next is called for the first time
• Parseq doesn’t fuse the original iterator
• Parseq uses constant space: linear in the number of threads and the size of the buffer, not in the length of the possibly infinite original iterator
• Parseq propagates panics from the given closure

In contrast, Rayon, the de-facto standard solution for Rust, doesn’t preserve the order of the original iterator. Pariter, a good sequential alternative addressing this exact issue, doesn’t support infinite iterators; though it provides more features than Parseq.

Therefore, I invite you to give Parseq a try, and send me some feedback.