tarantool-test

tarantool-test is a CLI test framework, written for tarantool-based Rust applications.

Powered by tarantool-runner, it eliminates huge boilerplate associated with functional tests.

You have a tarantool-oriented Rust application, and you have tests that depend on tarantool symbols and you find yourself writing your own test suite? Then you've found needed utility. All you need is to write business-logic of your test-suite, bind it through tarantool-test macro and use supplied CLI-application.

Quick glance at CLI usage:

# use default entrypoint:
tarantool-test -p ./target/debug/mypackage.so

# use custom entrypoint(for example, for secondary test suite):
tarantool-test -p ./target/debug/mypackage.so -e custom_entrypoint

If you want a quick steps to setup tests at your own crate, read installation and tutorial sections.

For more detailed explanation, see details and features sections.

Installation

tarantool-test consists of two parts, merged into one crate:

1. library, which contains utilities you must use in order to prepare your own crate for tests;
2. binary utility which you install and run just like cargo test

You can install binary utility from crates.io:

cargo install --features="bin" tarantool-test

Take a note that we activated feature bin in order to install the binary. Do not activate this feature when you use tarantool-test as a library.

You can also install binary utility directly from the repository:

git clone https://git.picodata.io/picodata/picodata/tarantool-test
cd tarantool-test
cargo install --features="bin" --path tarantool-test

In order to add library part to your project, you simply use tarantool-test as a dependency: cargo add tarantool-test

Tutorial

You can see an example of usage in Makefile, target run-example executes test suite against example crate, which can be found in example subdir.

Now you want to use tarantool-test with your own crate. Following steps will use default test suite:

1. Add tarantool-test to your Cargo.toml as an ordinary dependency: cargo add tarantool-test;
2. Ensure your package could be used by the tarantool-test. Include those lines in the Cargo.toml:

[lib]
crate-type = ["lib", "cdylib"]

3. Mark desired tests with #[tarantool_test::test] attribute macro. This would allow tarantool-test to collect and run them;
4. Bind test suite you want to use by using bind_test_suite macro - add this line to lib.rs: tarantool_test::bind_test_suite!()
5. Build your crate: cargo build -p example. Check ./target/debug directory to find out the actual path of shared object you've built. It would be named something like libexample.so for Linux or libexample.dylib for Macos.
6. Launch your tests by running: tarantool-test -p ./debug/tarantool/libexample.so. Tests would be executed, and you are done! No need to cleanup anything.

If you got confused by some of the points above, check example crate or read more details below.

Features

Multiple test suites

You are not limited to the single or default test suite - you can use as many custom suites as you want, or even bind same test suite to the many entrypoints. The important thing is that tarantool-test binary can only run single entrypoint - so if you would like to run many entrypoints, you'll have to use tarantool-test multiple times, at least for now.

In order to introduce new test suite, you implement TestSuite trait for some type. Then you decide which entrypoint you'd like to bind it to, for example new_test_entrypoint, and bind test suite to it. Then your setup would look like this:

struct MyTestSuite;

impl tarantool_test::TestSuite for MyTestSuite {
    fn before_all() {}
    fn after_all() {}
    fn before_each() {}
    fn after_each() {}
}

bind_test_suite!(new_test_entrypoint, MyTestSuite);

Then you run your setup by specifying concrete entrypoint you've used: tarantool-test -p ./debug/target/myproject.so -e new_test_entrypoint.

You can also omit entrypoint name for your custom test suite while binding, then it would be bind to the default entrypoint, which is named "default_test_entrypoint": bind_test_suite!(MyTestSuite)

You can omit test suite type reference, then the default test suite would be bind to the specified entrypoint: bind_test_suite!(new_test_entrypoint).

Take a note, that default test suite is implemented for (), so the line above would have the same effect as this one: bind_test_suite!(new_test_entrypoint, ()).

NOTE: You can't use the same entrypoint twice. For example, this setup fails:

// DO NOT COMPILE!
bind_test_suite!(first_test_entrypoint);
bind_test_suite!(first_test_entrypoint);

..But you surely can reuse test suites many times! For example, this setup works:

bind_test_suite!(first_test_entrypoint, MyTestSuite);
bind_test_suite!(second_test_entrypoint, MyTestSuite);

Override tarantool-runner args

If you want, you can override some of the arguments that are passed to tarantool-runner. Just supply them after --.

For example, you've written your own tarantool initialization script and want tarantool-test to use it. You can forward it as an argument to tarantool-runner like this: tarantool-test -p my_package.so -e my_entrypoint -- -i my_init_script.lua

However, there are limitations: you can't override -p and -e arguments and override input(which goes after --). So these usages are NOT allowed:

1. tarantool-test -p my_package.so -e my_entrypoint -- -e entrypoint
2. tarantool-test -p my_package.so -e my_entrypoint -- -p package
3. tarantool-test -p my_package.so -e my_entrypoint -- -- "my input"

Details

Problem and solution

Problem: tarantool-module, which is actively used to develop tarantool applications, depends on tarantool symbols as it is, essentially, a wrapper over tarantool API. Tarantool symbols are supplied only if an application is loaded via tarantool runtime. Therefore, in order to run tests that depend on tarantool-module, we have to develop some tarantool runtime and run test suite there. As the number of projects grows, such boilerplate setups migrate from one project to another, keeping bugs and undocumented behavior.

Solution: tarantool-test introduces CLI application that runs test suite in the tarantool environment. The only boilerplate you need to write is a few lines of code to setup your test suite(or use the default one if you ain't interested in before_all or any other callbacks).

Implementation

tarantool-test is backed by tarantool-runner. It executes test setup in the provided by runner tarantool environment, so anything related to tarantool-runner is applicable here. NOTE: you don't need to install tarantool-runner yourself - it is builtin.

Currently, tarantool-test highly depends on the tarantool-module. Namely, tarantool_test::test macro is just a reexport of tarantool::test, which might change in the future. Despite the fact that marking tests with tarantool::test would also work, it is discouraged, as tarantool_test::test would someday be separated to include special features.

What tarantool-test does: whenever you use bind_tarantool_test! macro, it generates tarantool procedure with needed entrypoint name. Newly created entrypoint deserializes input and launches tests with specified test suite.

Binary utility simply wraps tarantool-runner, introducing cleaner interface.

Contributing

If you have question, problem or suggestion, feel free to reach me directly at f.telnov@picodata.io or at telegram.

You can also join Picodata chat at telegram and tag me with your question here as well. We'll be glad to see you there!