Rust Test Harness

A modern, feature-rich testing framework for Rust with real Docker container management, hooks, and parallel execution.

Requirements

• Docker: This framework requires Docker to be installed and running on your system
• Rust: Rust 1.70+ with Cargo

Features

• Rust-Native Testing: Works exactly like Rust's built-in #[test] attribute
• Docker Integration: Run tests in isolated containers
• Container Hooks: NEW! before_each/after_each for per-test container lifecycle
• Test Hooks: before_all, before_each, after_each, after_all
• Parallel Execution: Run tests concurrently with configurable concurrency
• Tag-based Filtering: Organize and filter tests by tags
• Test Timeouts: Set maximum execution time for tests
• HTML Reports: Generate beautiful, interactive test reports with automatic target folder organization

Quick Start

Basic Usage (Rust-Style)

Your test harness works exactly like Rust's built-in testing framework! You can use it in mod tests blocks and outside of main() functions. The framework is fully functional:

use rust_test_harness::test_case;

pub struct Calculator {
    value: i32,
}

impl Calculator {
    pub fn new() -> Self { Self { value: 0 } }
    pub fn add(&mut self, x: i32) { self.value += x; }
    pub fn get_value(&self) -> i32 { self.value }
}

#[cfg(test)]
mod tests {
    use super::*;
    
    // Test using test_case! macro - works exactly like #[test]
    test_case!(test_calculator_new, |_ctx| {
        let calc = Calculator::new();
        assert_eq!(calc.get_value(), 0);
        Ok(())
    });
    
    // Standard Rust test that also works
    #[test]
    fn test_subtraction() {
        let mut calc = Calculator::new();
        calc.add(10);
        // Your test logic here
        assert_eq!(calc.get_value(), 10);
    }
}

Running Tests

# Run all tests (discovered by cargo test)
cargo test

# Run specific test
cargo test test_calculator_new

# Run tests with filter
cargo test calculator

# Run specific example
cargo test --example rust_style_tests

# Run specific test in an example
cargo test --example rust_style_tests test_calculator_new

Advanced Features

Docker Integration

Docker integration is available through the container hooks system (see Container Integration section for details).

Test Hooks

Important Note: Hooks are built into the framework and work automatically. You don't need to manually call them.

use rust_test_harness::test_case;

#[cfg(test)]
mod tests {
    use super::*;
    
    // The framework automatically handles hooks for:
    // - before_all: Global setup before all tests
    // - before_each: Setup before each test (including container startup)
    // - after_each: Cleanup after each test (including container shutdown)
    // - after_all: Global cleanup after all tests
    
    test_case!(my_test, |_ctx| {
        // Your test logic here
        // Hooks run automatically
        Ok(())
    });
}

What Hooks Can Do:

• ✅ Setup/teardown test databases
• ✅ Container lifecycle management (NEW!)
• ✅ Initialize test data
• ✅ Clean up test files
• ✅ Manage test configuration
• ✅ Handle test environment setup

Container Management with Hooks:

use rust_test_harness::{before_each, after_each, ContainerConfig};

let container = ContainerConfig::new("postgres:13")
    .port(5432, 5432)
    .env("POSTGRES_PASSWORD", "test");

before_each(move |ctx| {
    let container_id = container.start()?;
    ctx.set_data("db_container_id", container_id);
    Ok(())
});

after_each(move |ctx| {
    let container_id = ctx.get_data::<String>("db_container_id").unwrap();
    container.stop(&container_id)?;
    Ok(())
});

What Hooks Are NOT For:

• ❌ Complex resource orchestration (use specialized macros)
• ❌ Cross-test data sharing (use before_all/after_all instead)

Tag-based Filtering

use rust_test_harness::test_with_tags;

test_with_tags("slow_integration_test", vec!["slow", "integration"], |_ctx| {
    // This test has tags: "slow" and "integration"
    std::thread::sleep(Duration::from_secs(1));
    Ok(())
});

// Skip slow tests: TEST_SKIP_TAGS=slow cargo test

HTML Reports

Generate beautiful, interactive HTML reports for your test results. All HTML reports are automatically stored in the target/test-reports/ directory for clean project organization and easy CI/CD integration:

use rust_test_harness::run_tests_with_config;

let config = TestConfig {
    html_report: Some("test-results.html".to_string()),
    ..Default::default()
};

let result = run_tests_with_config(config);
// Generates target/test-reports/test-results.html with comprehensive test results

HTML Report Features:

• 📊 Visual Summary: Pass/fail/skip statistics with color-coded cards
• 📋 Detailed Results: Individual test results with status and error details
• 🎨 Modern Design: Responsive, mobile-friendly interface with gradients
• 🔍 Test Information: Tags, timeouts, Docker configuration, and error messages
• ⏱️ Execution Time: Total test execution duration
• 📱 Responsive Layout: Works on desktop, tablet, and mobile devices
• 🔽 Expandable Details: Click any test to view detailed metadata and configuration
• 🔍 Search Functionality: Search tests by name, status, or tags in real-time
• ⌨️ Keyboard Shortcuts: Ctrl+F (search), Ctrl+A (expand all), Ctrl+Z (collapse all)
• 🚨 Auto-Expand Failed: Failed tests automatically expand for better visibility

Environment Variable:

export TEST_HTML_REPORT=test-results.html
cargo test
# Report will be saved to: target/test-reports/test-results.html

Path Resolution:

• Relative paths (e.g., "report.html") → Automatically stored in target/test-reports/report.html
• Absolute paths (e.g., "/tmp/report.html") → Stored at the exact specified location
• Target directory: Uses CARGO_TARGET_DIR environment variable or defaults to "target"

Target Folder Benefits:

• 🗂️ Clean Project Structure: No HTML files cluttering your project root
• 🔄 CI/CD Friendly: Easy to exclude from version control and clean up
• 📁 Organized Output: All test reports in one dedicated location
• 🚀 Rust Conventions: Follows standard Rust project structure
• 🧹 Easy Cleanup: Simple to remove with cargo clean

Interactive Features

The HTML reports include several interactive features to enhance your testing experience:

🔽 Expandable Test Details

• Click on any test header to expand/collapse detailed information
• View test metadata including tags, timeouts, and Docker configuration
• Error details are automatically displayed for failed tests

🔍 Real-Time Search

• Search box filters tests by name, status, or tags
• Results update instantly as you type
• Case-insensitive search for better usability

⌨️ Keyboard Shortcuts

• Ctrl+F (or Cmd+F on Mac): Focus search box
• Ctrl+A (or Cmd+A on Mac): Expand all test details
• Ctrl+Z (or Cmd+Z on Mac): Collapse all test details

🚨 Smart Defaults

• Failed tests automatically expand for immediate visibility
• Hover effects provide visual feedback
• Responsive design adapts to all screen sizes

Framework vs. Standard Testing

Feature	Rust Standard	Rust Test Harness
Basic Tests	#[test]	test_case!() or #[test]
Test Discovery	✅	✅
Parallel Execution	✅	✅
Test Hooks	❌	✅ (Automatic)
Docker Integration	❌	✅
Tag-based Filtering	❌	✅
Test Timeouts	❌	✅
HTML Reports	❌	✅ (with target folder organization)

Note: The framework is fully functional and works correctly in all test environments. HTML reports are automatically stored in the organized target/test-reports/ directory for clean project structure.

Migration from Standard Tests

Converting from standard Rust tests is simple:

Before (Standard Rust):

#[test]
fn test_my_function() {
    assert_eq!(my_function(2, 2), 4);
}

After (Rust Test Harness):

test_case!(test_my_function, |_ctx| {
    assert_eq!(my_function(2, 2), 4);
    Ok(())
});

Or keep using #[test] unchanged:

#[cfg(test)]
mod tests {
    use super::*;
    
    #[test]
    fn test_my_function() {
        // Your existing test code works unchanged
        assert_eq!(my_function(2, 2), 4);
    }
}

ContainerConfig

The ContainerConfig struct provides a clean, builder-pattern approach to configuring containers for testing:

use rust_test_harness::ContainerConfig;
use std::time::Duration;

let container = ContainerConfig::new("redis:alpine")
    .port(6379, 6379)                    // Host port -> Container port
    .env("REDIS_PASSWORD", "secret")     // Environment variables
    .name("test_redis")                  // Container name
    .ready_timeout(Duration::from_secs(30)); // Wait for container readiness

Available Methods:

• .port(host_port, container_port) - Map host ports to container ports
• .auto_port(container_port) - Automatically assign available host port for container port
• .env(key, value) - Set environment variables
• .name(name) - Set container name
• .ready_timeout(duration) - Set readiness timeout
• .no_auto_cleanup() - Disable automatic cleanup (containers persist after tests)

Container Lifecycle Methods:

• .start() - Start container and return ContainerInfo
• .stop(container_id) - Stop container by ID

Automatic Cleanup: By default, all containers are automatically stopped and removed when tests complete. This ensures a clean environment for each test run.

Port Configuration Options:

1. Auto-Port Assignment (Recommended): The framework automatically finds available ports on your system, eliminating port conflicts:

   let container = ContainerConfig::new("nginx:alpine")
       .auto_port(80)        // Automatically assign available host port for container port 80
       .auto_port(443);      // Automatically assign available host port for container port 443
   
   let container_info = container.start()?;
   println!("Container running on: {}", container_info.ports_summary());
   println!("Web accessible at: {}", container_info.primary_url().unwrap());
   

2. User-Specified Ports: You can also specify exact port mappings when you need specific ports:

   let container = ContainerConfig::new("postgres:13-alpine")
       .port(5432, 5432)     // Map host port 5432 to container port 5432
       .port(8080, 80);      // Map host port 8080 to container port 80
   
   let container_info = container.start()?;
   // PostgreSQL will be accessible on localhost:5432
   // HTTP service will be accessible on localhost:8080
   

3. Mixed Configuration: Combine both approaches for maximum flexibility:

   let container = ContainerConfig::new("httpd:alpine")
       .port(8080, 80)       // Fixed mapping for main service
       .auto_port(443)       // Auto-assign for HTTPS
       .auto_port(9090);     // Auto-assign for metrics
   
   let container_info = container.start()?;
   println!("HTTP: localhost:8080");
   if let Some(https_port) = container_info.host_port_for(443) {
       println!("HTTPS: localhost:{}", https_port);
   }
   

ContainerInfo Object: When you start a container, you get a ContainerInfo object with:

• Port Information: Easy access to host ports and URLs
• Container Details: ID, image, name, and status
• Convenience Methods: Get URLs, port mappings, and summaries

let container_info = container.start()?;

// Get the host port for a specific container port
if let Some(host_port) = container_info.host_port_for(27017) {
    println!("MongoDB accessible at localhost:{}", host_port);
}

// Get all URLs
for url in &container_info.urls {
    println!("Service available at: {}", url);
}

// Get port summary
println!("Ports: {}", container_info.ports_summary());

🌐 Accessing Container Ports and URLs

The framework provides comprehensive access to container port information, making it easy to connect to your services:

Getting Host Ports

let container = ContainerConfig::new("postgres:13-alpine")
    .auto_port(5432)    // Auto-assign available host port
    .env("POSTGRES_PASSWORD", "testpass");

let container_info = container.start()?;

// Get the actual host port that was assigned
if let Some(host_port) = container_info.host_port_for(5432) {
    println!("PostgreSQL running on: localhost:{}", host_port);
    
    // Use in connection strings
    let conn_str = format!("postgresql://user:pass@localhost:{}/db", host_port);
}

Getting Service URLs

let web_container = ContainerConfig::new("nginx:alpine")
    .auto_port(80)
    .auto_port(443);

let web_info = web_container.start()?;

// Get ready-to-use URLs
if let Some(http_url) = web_info.url_for_port(80) {
    println!("HTTP service: {}", http_url);  // http://localhost:52341
    
    // Use directly in HTTP clients
    let response = reqwest::get(&http_url).await?;
}

// Get primary URL (first port)
if let Some(primary) = web_info.primary_url() {
    println!("Main service: {}", primary);
}

Real-World Usage Patterns

// Pattern 1: Database Testing
let db_info = postgres_container.start()?;
if let Some(port) = db_info.host_port_for(5432) {
    std::env::set_var("DATABASE_URL", format!("postgresql://localhost:{}/test", port));
    // Now your application can connect using the environment variable
}

// Pattern 2: API Testing
let api_info = api_container.start()?;
if let Some(api_url) = api_info.primary_url() {
    let client = reqwest::Client::new();
    let response = client.get(&format!("{}/health", api_url)).send().await?;
    assert_eq!(response.status(), 200);
}

// Pattern 3: Multiple Services
let web_info = web_container.start()?;
let db_info = db_container.start()?;

println!("Services started:");
println!("  Web: {}", web_info.ports_summary());
println!("  DB:  {}", db_info.ports_summary());

// All ports are different - no conflicts!

ContainerInfo Methods Reference

Method	Purpose	Example
host_port_for(container_port)	Get host port for specific container port	container_info.host_port_for(80)
url_for_port(container_port)	Get URL for specific container port	container_info.url_for_port(80)
primary_url()	Get URL for first port	container_info.primary_url()
ports_summary()	Human-readable port mappings	container_info.ports_summary()
port_mappings	All (host_port, container_port) pairs	container_info.port_mappings
urls	All service URLs	container_info.urls

Example with PostgreSQL:

let postgres = ContainerConfig::new("postgres:13")
    .port(5432, 5432)
    .env("POSTGRES_DB", "testdb")
    .env("POSTGRES_USER", "testuser")
    .env("POSTGRES_PASSWORD", "testpass")
    .name("test_postgres")
    .ready_timeout(Duration::from_secs(30));

// Use in hooks
before_each(move |ctx| {
    let container_id = postgres.start()?;
    ctx.set_data("postgres_id", container_id);
    Ok(())
});

Examples

Check out the examples/ directory for comprehensive examples:

Core Examples

• minimal_rust_style.rs - Minimal example showing Rust-style testing
• rust_style_tests.rs - Comprehensive Rust-style testing patterns
• basic_usage.rs - Basic framework usage with test_case!

Real-World Examples

• advanced_features.rs - Advanced features and patterns
• real_world_calculator.rs - Real-world application testing
• mongodb_integration.rs - NEW! Database testing with container hooks (recommended approach)
• auto_port_demo.rs - NEW! Auto-port functionality and container management demo
• container_port_access.rs - NEW! Detailed guide on accessing container ports and URLs
• cargo_test_integration.rs - Integration with cargo test

Container Management Examples

• mongodb_integration.rs - Container Hooks Pattern: Uses before_each/after_each for per-test container lifecycle

Container Hooks Pattern The container hooks approach provides excellent control and isolation:

• Each test gets a fresh container
• Automatic cleanup prevents resource leaks
• Cleaner separation of concerns
• More flexible configuration options

Running Examples

# Run a specific example
cargo run --example rust_style_tests

# Test a specific example
cargo test --example rust_style_tests

# Run specific test in an example
cargo test --example rust_style_tests test_calculator_new

# HTML reports are automatically generated in target/test-reports/
# when using examples with HTML reporting enabled

Port Access Examples:

# See auto-port functionality in action
cargo run --example auto_port_demo

# Learn how to access container ports and URLs
cargo run --example container_port_access

# Container port and URL access
cargo run --example container_port_access

# User-specified port mappings
cargo run --example user_specified_ports

Container Management Patterns

Container Lifecycle with Hooks (NEW!)

Use before_each and after_each for per-test container management:

use rust_test_harness::{test, before_each, after_each, ContainerConfig};

// Define container configuration
let mongo_container = ContainerConfig::new("mongo:5.0")
    .port(27017, 27017)
    .env("MONGO_INITDB_ROOT_USERNAME", "admin")
    .env("MONGO_INITDB_ROOT_PASSWORD", "password")
    .name("test_mongodb")
    .ready_timeout(Duration::from_secs(30));

// before_each starts a fresh container for each test
before_each(move |ctx| {
    let container_id = mongo_container.start()?;
    ctx.set_data("mongo_container_id", container_id);
    Ok(())
});

// after_each cleans up the container after each test
let mongo_container_clone = mongo_container.clone();
after_each(move |ctx| {
    let container_id = ctx.get_data::<String>("mongo_container_id").unwrap();
    mongo_container_clone.stop(&container_id)?;
    Ok(())
});

// Tests use the container
test("test_mongodb_operations", |ctx| {
    let container_id = ctx.get_data::<String>("mongo_container_id").unwrap();
    // Your test logic here
    Ok(())
});

Benefits of Container Hooks:

• ✅ Fresh Containers: Each test gets a completely isolated container
• ✅ Automatic Cleanup: No risk of resource leaks
• ✅ Easy Configuration: Builder pattern for container setup
• ✅ Clean Separation: Test logic separate from container management
• ✅ Real-world Ready: Can easily be extended to use actual Docker API

For Docker Containers

Use Container Hooks (Recommended approach): Refer to the Container Integration section above for the full example using before_each/after_each hooks.

Benefits of Container Hooks:

• Fine-grained control: Full control over container lifecycle
• Better isolation: Fresh container per test
• Automatic cleanup: Prevents resource leaks
• Flexible configuration: Custom setup/teardown logic

For Other Resources

Hooks work great for:

• Database connections
• File system operations
• Configuration management
• Test data setup/cleanup

Configuration

Set environment variables to configure test execution:

# Filter tests by name
export TEST_FILTER=integration

# Skip tests with specific tags
export TEST_SKIP_TAGS=slow,flaky

# Set maximum concurrency
export TEST_MAX_CONCURRENCY=8

# Generate HTML report
export TEST_HTML_REPORT=test-results.html

# Skip hooks (for debugging)
export TEST_SKIP_HOOKS=true

IDE Support

RustRover and Similar IDEs

test_case! macros:

• ❌ No play buttons (macros aren't expanded during compilation)
• ✅ Tests are discoverable by cargo test
• ✅ Tests run correctly

Standard #[test] functions:

• ✅ Play buttons work normally
• ✅ Full IDE integration

Recommended Approach:

#[cfg(test)]
mod tests {
    use super::*;
    
    // Use test_case! for framework features
    test_case!(test_with_framework, |_ctx| {
        // Framework-specific logic
        Ok(())
    });
    
    // Use #[test] for IDE play button support
    #[test]
    fn test_standard_rust() {
        // Standard Rust testing
        assert_eq!(2 + 2, 4);
    }
}

Framework Reliability: The framework is fully functional. All features work correctly in test environments, and HTML reports are automatically organized in the target/test-reports/ directory for clean project structure.

Why This Approach?

1. Familiar: Works exactly like Rust's built-in testing

2. Discoverable: Tests are found by cargo test automatically

3. Compatible: Existing Rust tests work unchanged

4. Enhanced: Adds powerful features without breaking existing patterns

5. Flexible: Use framework features when needed, standard patterns when not

6. Automatic: Hooks work automatically without manual setup

7. Container-Ready: NEW! Built-in container lifecycle management with hooks

8. Isolated: Each test gets fresh containers for true isolation

9. Clean: Builder pattern for container configuration

10. Docker Integration Ready: Can easily integrate with real Docker APIs

11. Organized: HTML reports automatically stored in target/test-reports/ for clean project structure

12. Reliable: Fully functional framework with comprehensive test coverage

Contributing

Contributions are welcome! This framework aims to enhance Rust's testing capabilities while maintaining full compatibility with existing testing patterns.