MadSim

Magical Deterministic Simulator for distributed systems.

Deterministic Simulation Testing

MadSim is a Rust async runtime similar to tokio, but with a key feature called deterministic simulation.

The main idea is borrowed from FoundationDB and sled simulation guide. Your code should be able to deterministically executed on top of a simulator. The simulator will amplify randomness, create chaos and inject failures into your system. A lot of hidden bugs may be revealed, which you can then deterministically reproduce them until they are fixed. If your system can survive such chaos, you will have more confidence in deploying your system in the real world.

However, implementing deterministic simulation is difficult. All I/O-related interfaces must be mocked during the simulation, and all uncertainties should be eliminated. This project is created to make that easy.

A part of the implementation of this crate is inspired by tokio-rs/simulation.

Usage

Add the following lines to your Cargo.toml:

[dependencies]
madsim = "0.2"

If your project depends on tokio or tonic, replace all the following entries:

[dependencies]
tokio = { version = "0.2", package = "madsim-tokio" }
tonic = { version = "0.2", package = "madsim-tonic" }

[dev-dependencies]
tonic-build = { version = "0.2", package = "madsim-tonic-build" }

If your dependency graph includes the following crates, replace them by our patched version:

[patch.crates-io]
quanta = { git = "https://github.com/madsys-dev/quanta.git", rev = "a819877" }
getrandom = { git = "https://github.com/madsys-dev/getrandom.git", rev = "cc95ee3" }

When built normally, these crates are identical to the original ones.

To run your code on the simulator, enable the config madsim:

RUSTFLAGS="--cfg madsim" cargo test

Now you have gotten rid of tokio/tonic and you are in the simulation world!

We provide a set of APIs to control the simulator. You can use them to kill a process, disconnect the network, inject failures, etc. Check out the documentation and search for the sim feature to learn more usages.

Ensure determinism

Developers should eliminate any randomness in the application code. That's not easy.

Here are some tips to avoid randomness:

• Use futures::select_biased instead of futures::select macro.

To make sure your code is deterministic, run your test with the following environment variable:

MADSIM_TEST_CHECK_DETERMINISM=1

Your test will be run at least twice with the same seed. If any non-determinism detected, it will panic as soon as possible.

Related Projects

• MadRaft: The labs of Raft consensus algorithm derived from MIT 6.824 and PingCAP Talent Plan.

License

Apache License 2.0