Executors

A library with high-performace task executors for Rust.

Usage

Add this to your Cargo.toml:

[dependencies]
executors = "0.9"

You can use, for example, the crossbeam_workstealing_pool to schedule a number n_jobs over a number n_workers threads, and collect the results via an mpsc::channel.

use executors::*;
use executors::crossbeam_workstealing_pool;
use std::sync::mpsc::channel;

let n_workers = 4;
let n_jobs = 8;
let pool = crossbeam_workstealing_pool::small_pool(n_workers);

let (tx, rx) = channel();
for _ in 0..n_jobs {
    let tx = tx.clone();
    pool.execute(move|| {
        tx.send(1).expect("channel will be there waiting for the pool");
    });
}

assert_eq!(rx.iter().take(n_jobs).fold(0, |a, b| a + b), 8);

Rust Version

Requires at least Rust 1.26, due to crossbeam-channel requirements.

Deciding on an Implementation

To select an Executor implementation, it is best to test the exact requirements on the target hardware. The crate executor-performance provides a performance testing suite for the provided implementations. To use it clone this repository, run cargo build --release, and then check target/release/executor-performance --help to see the available options. There is also a small script to test thread-scaling with some reasonable default options.

In general, crossbeam_workstealing_pool works best for workloads where the tasks on the worker threads spawn more and more tasks. If all tasks a spawned from a single thread that isn't part of the threadpool, then crossbeam_channel_pool tends to perform best.

If you don't know what hardware your code is going to run on, use the crossbeam_workstealing_pool. It tends to perform best on all the hardware I have tested (which is pretty much Intel processors like i7 and Xeon).

If you absolutely need low response time to bursty workloads, you can compile the crate with the ws-no-park feature, which prevents the workers in the crossbeam_workstealing_pool from parking their threads, when all the task-queues are temporarily empty. This will, of course, not play well with other tasks running on the same system, although the threads are still yielded to the OS scheduler in between queue checks. See latency results below to get an idea of the performance impact of this feature.

Core Affinity

You can enable support for pinning pool threads to particular CPU cores via the "thread-pinning" feature. It will then pin by default as many threads as there are core ids. If you are asking for more threads than that, the rest will be created unpinned. You can also assign only a subset of your core ids to a thread pool by using the with_affinity(...) instead of the new(...) function.

Some Numbers

The following are some example results from my desktop machine (Intel i7-4770 @ 3.40Ghz Quad-Core with HT (8 logical cores) with 16GB of RAM). Note that they are all from a single run and thus not particularly scientific and subject to whatever else was going on on my system during the run.

Implementation abbreviations:

• TP: threadpool_executor docs
• CBCP: crossbeam_channel_pool docs
• CBWP: crossbeam_workstealing_pool docs

Throughput

These experiments measure the throughput of a fully loaded executor implementation.

Low Amplification

Testing command (where $NUM_THREADS corresponds to #Threads in the table below):

target/release/executor-performance -t $NUM_THREADS -p 2 -m 10000000 -a 1 throughput --pre 10000 --post 10000

This corresponds to a IO-handling-server-style workload, where the vast majority of tasks are coming in via the external queue, and only very few are spawned from within the executor's threads.

(Units are in mio tasks/s)

High Amplification

Testing command (where $NUM_THREADS corresponds to #Threads in the table below):

target/release/executor-performance -t $NUM_THREADS -p 2 -m 1000 -a 50000 throughput --pre 10000 --post 10000

This corresponds to a message-passing-style (or fork-join) workload, where the vast majority of tasks are spawned from within the executor's threads and only relatively few come in via the external queue.

(Units are in mio tasks/s)

Latency

These experiments measure the start-to-finish time (latency) of running a bursty workload, while sleeping in between runs. This is what typically happens in a network server, that isn't fully loaded, but waiting for a message and then running a series of tasks based on the message, before responding and waiting for the next.

These experiments report averages collected over multiple measurements with RSE<10%.

Small Bursts / Low Amplification

Testing command:

target/release/executor-performance -t 6 -p 1 -m 100 -a 1 latency -s 100

This corresponds to a very small internal workload in response to every external message. Sleep time between bursts is 100ms.

Medium Bursts / High Amplification

Testing command:

target/release/executor-performance -t 6 -p 1 -m 100 -a 100 latency -s 100

This corresponds to a significant internal workload in response to every external message. Sleep time between bursts is 100ms.

Large Bursts / Very High Amplification

Testing command:

target/release/executor-performance -t 6 -p 1 -m 100 -a 10000 latency -s 100

This corresponds to a very large internal workload in response to every external message. Sleep time between bursts is 100ms.

License

Licensed under the terms of the MIT license.

See LICENSE for details.