lmrc-pipeline

Pipeline orchestration library for LMRC Stack projects with reusable build, test, and deployment steps.

Overview

lmrc-pipeline provides a programmatic way to define and execute CI/CD pipelines for Rust projects. It offers:

• Reusable Steps: Pre-built steps for common tasks (build, test, lint, deploy)
• Composable API: Build complex pipelines from simple building blocks
• Progress Tracking: Beautiful terminal output with step status and timing
• Error Handling: Comprehensive error types with context
• Flexibility: Easy to add custom steps for project-specific needs

Installation

Add to your Cargo.toml:

[dependencies]
lmrc-pipeline = "0.1.0"
lmrc-config-validator = "0.1.0"  # For ProjectConfig

Quick Start

use lmrc_pipeline::{Pipeline, PipelineContext};
use lmrc_pipeline::steps::{BuildStep, TestStep, ClippyStep};
use lmrc_config_validator::ProjectConfig;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Load project configuration
    let config = ProjectConfig::from_file("lmrc.toml")?;

    // Create pipeline context
    let context = PipelineContext::new(config);

    // Build and run pipeline
    let report = Pipeline::new(context)
        .add_step(BuildStep::new().release())
        .add_step(TestStep::new())
        .add_step(ClippyStep::new())
        .run()
        .await?;

    println!("Pipeline completed in {:?}", report.total_duration);
    Ok(())
}

Available Steps

Cargo Steps

BuildStep

Build your Rust workspace:

use lmrc_pipeline::steps::BuildStep;

// Debug build
let step = BuildStep::new();

// Release build
let step = BuildStep::new().release();

// Specific packages
let step = BuildStep::new()
    .packages(vec!["my-app".to_string(), "my-lib".to_string()]);

TestStep

Run tests:

use lmrc_pipeline::steps::TestStep;

// All tests
let step = TestStep::new();

// Specific packages
let step = TestStep::new()
    .packages(vec!["my-app".to_string()]);

// Specific test
let step = TestStep::new()
    .test_name("integration_test".to_string());

FormatCheckStep / FormatStep

Check or apply code formatting:

use lmrc_pipeline::steps::{FormatCheckStep, FormatStep};

// Check formatting (CI)
let step = FormatCheckStep::new();

// Apply formatting (development)
let step = FormatStep::new();

ClippyStep

Run clippy linter:

use lmrc_pipeline::steps::ClippyStep;

// Deny warnings (default)
let step = ClippyStep::new();

// Allow warnings
let step = ClippyStep::new().deny_warnings(false);

// Specific packages
let step = ClippyStep::new()
    .packages(vec!["my-app".to_string()]);

Infrastructure Steps (Placeholder)

The following steps are defined but not yet fully implemented. They serve as placeholders for future functionality:

• DockerBuildStep: Build Docker images
• ProvisionStep: Provision infrastructure (Hetzner Cloud)
• SetupK8sStep: Setup Kubernetes/K3s cluster
• SetupDatabaseStep: Setup PostgreSQL database
• SetupDnsStep: Configure DNS records (Cloudflare)
• DeployStep: Deploy applications to Kubernetes

Creating Custom Steps

Implement the PipelineStep trait:

use lmrc_pipeline::{PipelineStep, PipelineContext, StepOutput, Result};
use async_trait::async_trait;
use std::time::Instant;

pub struct CustomStep {
    name: String,
}

impl CustomStep {
    pub fn new(name: String) -> Self {
        Self { name }
    }
}

#[async_trait]
impl PipelineStep for CustomStep {
    fn name(&self) -> &str {
        "custom-step"
    }

    fn description(&self) -> &str {
        "My custom pipeline step"
    }

    async fn execute(&self, ctx: &mut PipelineContext) -> Result<StepOutput> {
        let start = Instant::now();

        // Your custom logic here
        println!("Running custom step: {}", self.name);

        Ok(StepOutput::success_with_message(
            self.name(),
            start.elapsed(),
            format!("Custom step '{}' completed", self.name),
        ))
    }

    fn should_skip(&self, ctx: &PipelineContext) -> bool {
        // Optional: add skip logic
        false
    }
}

Pipeline Context

The PipelineContext provides shared state between steps:

use lmrc_pipeline::PipelineContext;
use lmrc_config_validator::ProjectConfig;
use std::path::PathBuf;

let config = ProjectConfig::from_file("lmrc.toml")?;

let context = PipelineContext::new(config)
    .with_working_dir(PathBuf::from("/path/to/project"))
    .with_dry_run(true);  // Preview without executing

// Steps can access and modify shared state
context.set_state("key".to_string(), "value".to_string());
let value = context.get_state("key");

Example: Complete CI Pipeline

use lmrc_pipeline::{Pipeline, PipelineContext};
use lmrc_pipeline::steps::*;
use lmrc_config_validator::ProjectConfig;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let config = ProjectConfig::from_file("lmrc.toml")?;
    let context = PipelineContext::new(config);

    Pipeline::new(context)
        // Check stage
        .add_step(FormatCheckStep::new())
        .add_step(ClippyStep::new())

        // Test stage
        .add_step(TestStep::new())

        // Build stage
        .add_step(BuildStep::new().release())

        // Deploy stage (when implemented)
        // .add_step(DockerBuildStep::new())
        // .add_step(DeployStep::new())

        .run()
        .await?;

    Ok(())
}

CLI Integration

The library is designed to be used in generated pipeline binaries:

use clap::{Parser, Subcommand};
use lmrc_pipeline::{Pipeline, PipelineContext};
use lmrc_pipeline::steps::*;
use lmrc_config_validator::ProjectConfig;

#[derive(Parser)]
struct Cli {
    #[command(subcommand)]
    command: Commands,
}

#[derive(Subcommand)]
enum Commands {
    Build { #[arg(long)] release: bool },
    Test,
    Check,
    Full,
}

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    let cli = Cli::parse();
    let config = ProjectConfig::from_file("lmrc.toml")?;

    match cli.command {
        Commands::Build { release } => {
            let mut step = BuildStep::new();
            if release {
                step = step.release();
            }
            Pipeline::new(PipelineContext::new(config))
                .add_step(step)
                .run()
                .await?;
        }
        Commands::Test => {
            Pipeline::new(PipelineContext::new(config))
                .add_step(TestStep::new())
                .run()
                .await?;
        }
        Commands::Check => {
            Pipeline::new(PipelineContext::new(config))
                .add_step(FormatCheckStep::new())
                .add_step(ClippyStep::new())
                .run()
                .await?;
        }
        Commands::Full => {
            Pipeline::new(PipelineContext::new(config))
                .add_step(FormatCheckStep::new())
                .add_step(ClippyStep::new())
                .add_step(TestStep::new())
                .add_step(BuildStep::new().release())
                .run()
                .await?;
        }
    }

    Ok(())
}

Error Handling

The library uses a comprehensive error type:

use lmrc_pipeline::PipelineError;

match pipeline.run().await {
    Ok(report) => {
        println!("Success! Took {:?}", report.total_duration);
    }
    Err(PipelineError::StepFailed { step, source }) => {
        eprintln!("Step '{}' failed: {}", step, source);
    }
    Err(PipelineError::CommandFailed(msg)) => {
        eprintln!("Command failed: {}", msg);
    }
    Err(e) => {
        eprintln!("Pipeline error: {}", e);
    }
}

Output Example

When you run a pipeline, you'll see beautiful terminal output:

════════════════════════════════════════════════════════════════════════════════
  Pipeline Execution - 4 steps
════════════════════════════════════════════════════════════════════════════════

  ▶ 1/4 Running cargo-fmt-check
    Check code formatting
  ✓ 1/4 DONE cargo-fmt-check (0.43s)
    Code is properly formatted

  ▶ 2/4 Running cargo-clippy
    Run clippy linter
  ✓ 2/4 DONE cargo-clippy (2.15s)
    No clippy issues found

  ▶ 3/4 Running cargo-test
    Run workspace tests
  ✓ 3/4 DONE cargo-test (5.82s)
    All tests passed

  ▶ 4/4 Running cargo-build
    Build workspace in release mode
  ✓ 4/4 DONE cargo-build (18.34s)
    Built successfully in release mode

════════════════════════════════════════════════════════════════════════════════
  ✓ Pipeline completed successfully (26.74s)
════════════════════════════════════════════════════════════════════════════════

Development Status

✅ Implemented

• Core pipeline orchestration
• Cargo build/test/format/clippy steps
• Error handling and reporting
• Progress tracking and terminal output
• Context sharing between steps

🚧 Planned

• Complete Docker image building
• Infrastructure provisioning integration
• Kubernetes deployment integration
• Database setup integration
• DNS configuration integration
• Parallel step execution
• Step caching and incremental builds
• Pipeline state persistence
• Rollback on failure

Contributing

Contributions are welcome! Areas that need work:

1. Infrastructure Steps: Complete the placeholder implementations
2. Testing: Add comprehensive unit and integration tests
3. Documentation: Improve examples and API docs
4. Performance: Optimize step execution and add caching
5. Features: Add parallel execution, retries, timeouts

License

Dual licensed under MIT OR Apache-2.0 (user's choice).

Author

Lemarc lemarc.dev@gmail.com

Related Projects

• lmrc-cli: CLI tool that uses this library
• lmrc-config-validator: Configuration validation
• lmrc-hetzner: Hetzner Cloud client
• lmrc-kubernetes: Kubernetes management
• lmrc-docker: Docker operations

Links

• Repository
• Issues
• Documentation