tiny-integration-tester

This Rust program is used to perform simple integration tests on programs through a RUST dyn lib or a JSON configuration file.

Usage: tiny-integration-tester [options]

This program has been tested on Linux. Although it compiles for Windows, features such as timeout and the ability to display outputs are not supported

USING RUST DYN LIB :

When you use the tiny_integration_tester command on a crate, whether it's an executable or a library, a integration_tester_{package_name} folder is generated and two basic tests, cargo test and cargo build, are run.

The contents of the new lib.rs file should look like this :

//! This crate contains integration tests for {package_name}.

mod boilerplate_structure;
mod testable;

pub use boilerplate_structure::{Attribute, IntTest};
pub use testable::Testable;

use std::ffi::OsStr;

/// Test for `cargo build` command.
#[derive(Debug)]
pub struct CargoBuilder;

// YOU CAN IMPLEMENT THE `Testable` TRAIT TO CREATE A NEW TEST ...
#[cfg_attr(rustfmt, rustfmt_skip)]
impl Testable for CargoBuilder {
    fn command(&self) -> &OsStr { "cargo".as_ref() }
    fn name(&self) -> &str { "Build program" }
    fn args(&self) -> Vec<&OsStr> { vec!["build".as_ref()] }
    fn terminate(&self) -> bool { true }
    fn show_output(&self) -> Option<bool> { Some(true) }
    fn output(&mut self, status: i32, _out: &[u8], _err: &[u8]) -> bool { status == 0 }
}

/// This main function will be exported.
#[no_mangle]
#[allow(non_snake_case)]
pub fn TESTS(_args: &[&str]) -> Result<Vec<Box<dyn Testable>>, Box<dyn std::error::Error>> {
    // ... OR YOU CAN USE THE `IntTest` STRUCTURE, WHICH IS A VERY HANDY BOILERPLATE.
    let cargo_test = Box::new(
        IntTest::builder("cargo")
            .args(["test"])
            .output(|status: i32, _out: &[u8], _err: &[u8]| status == 0)
            .set_attribute(Attribute {
                name: "Basic cargo test".to_string(),
                terminate: true,
                show_output: Some(true),
                ..Default::default()
            }),
    );

    Ok(vec![Box::new(CargoBuilder {}), cargo_test])
}

Each of the tests must implement the Testable trait and they must be returned as a Box<dyn Testable> through a collection of type Vec<Box<dyn Testable>>.

In order to achieve this, you have the choice between creating a new structure and implementing the Testable trait for it, or you can make use of the boilerplate structure IntTest for which the Testable trait implementations are already defined in the boilerplate_structure.rs file.

You also have the possibility to pass arguments from the tiny_integration_tester command line with the -a argument. You will find the list of these arguments as a parameter of the main test function in the form &[&str]. You can of course use getopt on it.

The best thing to do is to generate the documentation for the crate with cargo doc and look at all the existing implementations, which already do just about everything!

Do not change either the name or the prototype of the function : pub fn TESTS(_args: &[&str]) -> Result<Vec<Box<dyn Testable>>, Box<dyn std::error::Error>>

However, there is one last restriction, it is imperative to compile the dynamic library with the same version of rustc that was used to compile tiny_integration_tester, use the -c option if necessary. I hope to find a satisfactory solution in the future.

FOR JSON USE :

After creating a new project with the command cargo new hello, create a file named test.json like this :

[
    {
        "command" : "./target/debug/hello",
        "stdout" : "Hello, world!\n"
    },
    {
        "command" : "./target/debug/hello",
        "stdout" : "Hello, dummy!\n"
    }
]

test.json is the default json filename.

Then simply run cargo build then tiny-integration-tester -j (with -j argument) :

Executing command: "./target/debug/hello" with args: []
OK!
Executing command: "./target/debug/hello" with args: []
FAIL!
result: 1 / 2

You can set these values into the json test file :

Here, for instance, we run the command cargo build --release, setting arguments to - "args" : ["build", "--release"] ,waiting for the command to succeed - "status" : "0x0". There is no time limit - "timeout" : 0, and the output will be displayed on the screen - "show_output" : true. Finally, if an error occurs, we stop the tests - "terminate" : true :

[
    {
        "command" : "cargo",
        "args" : ["build", "--release"],
        "status" : "0x0",
        "timeout" : 0,
        "terminate" : true,
        "show_output" : true
    }
]

Same example as below :

[
    {
        "command" : "echo",
        "args" : ["hello", "world"],
        "status" : "0x0",
        "show_output" : true
    },
    {
        "command" : "cargo",
        "args" : ["test", "--release"],
        "status" : "0x0",
        "timeout" : 0,
        "terminate" : true,
        "show_output" : true
    }
]

Also, we check if stdout or stderr match a given pattern :

[
    {
        "command" : "./target/release/fibonacci",
        "args" : ["3"],
        "stdout" : "fibo(3) = 2\n",
        "timeout" : 1000
    },
    {
        "command" : "./target/release/fibonacci",
        "args" : ["1", ""],
        "status" : "0xff00",
        "stderr" : "Usage: target/debug/fibonacci POSITIF_NUMBER\n"
    },
    {
        "command" : "./target/release/fibonacci",
        "args" : ["7"],
        "stdout" : "fibo(7) = 13\n"
    },
    {
        "timeout" : 500,
        "command" : "./target/release/fibonacci",
        "args" : ["15"],
        "status" : "0x0",
        "stdout" : "fibo(15) = 610\n"
    },
    {
        "command" : "./target/release/fibonacci",
        "args" : ["164neuf"],
        "status" : "0xff00",
        "stderr" : "Bad number format\n"
    },
    {
        "command" : "./target/release/fibonacci",
        "status" : "0xff00",
        "stderr" : "Bad number format\n"
    }
]

We can set test to false so that it does not count as a test :

[
    {
        "command" : "rustc",
        "args" : ["readline-tester.rs"],
        "test" : false
    },
    {
        "command" : "rustc",
        "args" : ["env-tester.rs"],
        "test" : false
    }
]

We can also record data for stdin or manage environment variables with env_clear and envs :

[
    {
        "command" : "./readline-tester",
        "status" : "0x0",
        "stdin" : "carrots are cooked\nbananas\n",
        "stdout" : "1 carrots are cooked\n2 bananas\n"
    },
    {
        "command" : "./env-tester",
        "stdout" : "1 = foo\n2 = bar\n",
        "env_clear" : true,
        "envs" : [["1", "foo"], ["2", "bar"], ["CLUTTER_IM_MODULE", "dummy"]],
        "show_output" : true
    }
]

Be careful with trailing commas at the end in a JSON file.