OpenCrates

An intelligent Rust crate generator and registry management system powered by AI

Key Features

• AI-Powered Code Generation: Advanced LLM integration for intelligent crate creation
• Multi-Provider Support: OpenAI, Codex, and extensible provider architecture
• Enterprise-Grade: Production-ready with comprehensive monitoring and security
• Template System: Sophisticated templating with reasoning patterns
• Real-time Analytics: Built-in metrics, health checks, and observability
• Docker Ready: Complete containerization with multi-stage builds

FastAPI Server & Endpoints

• RESTful API: Complete FastAPI-style server with automatic OpenAPI documentation
• WebSocket Support: Real-time communication for streaming operations
• Authentication: JWT-based auth with role-based access control
• Rate Limiting: Configurable rate limiting and request throttling
• CORS Support: Cross-origin resource sharing for web applications
• Health Checks: Comprehensive health monitoring endpoints

Enterprise Architecture

• Microservices Ready: Modular design for distributed deployments
• Database Integration: SQLite, PostgreSQL, and Redis support
• Caching Layer: Multi-tier caching with Redis backend
• Message Queues: Async processing with queue management
• Monitoring Stack: Prometheus metrics and structured logging
• Security: Input validation, sanitization, and security headers

Advanced Project Analysis

• Dependency Analysis: Smart dependency resolution and optimization
• Code Quality Metrics: Automated code quality assessment
• Performance Profiling: Built-in benchmarking and performance analysis
• Security Scanning: Vulnerability detection and security auditing

Development Tools

• CLI Interface: Rich command-line tools for developers
• IDE Integration: Language server protocol support
• Testing Framework: Comprehensive testing utilities
• Documentation Generation: Automated docs with examples

Quick Start

Installation

# Install from crates.io
cargo install opencrates

# Or add as dependency
cargo add opencrates

Docker Setup

# Pull and run
docker pull opencrates/opencrates:latest
docker run -p 8080:8080 opencrates/opencrates

# Or build locally
docker build -t opencrates .
docker-compose up

Basic Usage

use opencrates::{OpenCrates, providers::OpenAIProvider};
use anyhow::Result;

#[tokio::main]
async fn main() -> Result<()> {
    // Initialize OpenCrates
    let opencrates = OpenCrates::new().await?;
    
    // Generate a new crate
    opencrates.generate_crate(
        "my-awesome-crate",
        "A fantastic Rust library",
        vec!["async".to_string(), "serde".to_string()],
        "./output".into()
    ).await?;
    
    // Analyze existing project
    let analysis = opencrates.analyze_crate("./my-project").await?;
    println!("Project analysis: {:#?}", analysis);
    
    Ok(())
}

CLI Usage

# Generate a new crate
opencrates generate --name "my-crate" --description "A sample crate" --output ./output

# Analyze existing crate
opencrates analyze ./existing-project

# Start the server
opencrates server --host 0.0.0.0 --port 8080

# Health check
opencrates health

# Interactive mode
opencrates interactive

Configuration

OpenCrates supports comprehensive configuration through multiple sources:

Environment Variables

export OPENCRATES_ENV=production
export OPENAI_API_KEY=your_api_key_here
export OPENCRATES_DATABASE_URL=sqlite:./opencrates.db
export OPENCRATES_REDIS_URL=redis://localhost:6379
export OPENCRATES_LOG_LEVEL=info

Configuration File (opencrates.toml)

[server]
host = "127.0.0.1"
port = 8080
workers = 4

[database]
url = "sqlite:./opencrates.db"
max_connections = 10

[ai]
provider = "openai"
model = "gpt-4"
temperature = 0.7
max_tokens = 4000

[cache]
backend = "redis"
ttl = 3600
max_size = 1000

[monitoring]
enabled = true
health_check_interval = 30
enable_profiling = false

[security]
enable_auth = true
enable_rate_limiting = true
jwt_secret = "your-secret-key"

[features]
enable_web_ui = true
enable_api = true
enable_cli = true

Runtime Configuration

use opencrates::utils::config::OpenCratesConfig;

let config = OpenCratesConfig {
    server: ServerConfig {
        host: "0.0.0.0".to_string(),
        port: 3000,
        workers: 8,
        ..Default::default()
    },
    ai: AiConfig {
        provider: "openai".to_string(),
        model: "gpt-4-turbo".to_string(),
        temperature: 0.8,
        ..Default::default()
    },
    ..Default::default()
};

let opencrates = OpenCrates::new_with_config(config).await?;

Development & Testing

Prerequisites

# Install Rust (latest stable)
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

# Install required tools
cargo install cargo-audit cargo-tarpaulin

# For full development
docker-compose up -d redis postgres

Building from Source

# Clone the repository
git clone https://github.com/your-username/opencrates.git
cd opencrates

# Build in development mode
cargo build

# Build optimized release
cargo build --release

# Run with all features
cargo run --all-features -- --help

Running Tests

# Run all tests
cargo test --all-features

# Run specific test suites
cargo test --test basic_functionality
cargo test --test integration_tests
cargo test --test codex_integration

# Run with coverage
cargo tarpaulin --all-features --out html

# Benchmark tests
cargo bench

# Security audit
cargo audit

Development Scripts

# Comprehensive build and test
./build_and_test.sh

# Quick test run
./test_all.sh

# Format and lint
cargo fmt
cargo clippy --all-features -- -D warnings

# Generate documentation
cargo doc --no-deps --all-features --open

Architecture

Core Components

OpenCrates/
├── Core Engine           # Central orchestration
│   ├── Template Manager  # Code generation templates
│   ├── Provider Registry # AI provider management
│   └── Configuration     # System configuration
├── Providers/            # AI service integrations
│   ├── OpenAI           # GPT-4, GPT-3.5-turbo
│   ├── Codex            # GitHub Copilot integration
│   └── Custom          # Extensible provider system
├── Server/              # FastAPI-style HTTP server
│   ├── REST API         # RESTful endpoints
│   ├── WebSocket        # Real-time communication
│   └── GraphQL          # Advanced querying
├── CLI/                 # Command-line interface
│   ├── Interactive      # REPL-style interaction
│   ├── Batch           # Script automation
│   └── TUI             # Terminal user interface
├── Storage/            # Data persistence
│   ├── SQLite          # Embedded database
│   ├── PostgreSQL      # Production database
│   └── Redis           # Caching and sessions
└── Monitoring/         # Observability stack
    ├── Metrics         # Prometheus integration
    ├── Logging         # Structured logging
    └── Tracing         # Distributed tracing

Request Flow

[Client Request] 
    ↓
[Rate Limiting & Auth]
    ↓
[Request Validation]
    ↓
[Provider Selection]
    ↓
[AI Processing]
    ↓
[Template Application]
    ↓
[Code Generation]
    ↓
[Quality Validation]
    ↓
[Response Formation]
    ↓
[Client Response]

Data Models

// Core entities
pub struct CrateSpec {
    pub name: String,
    pub description: String,
    pub version: String,
    pub dependencies: HashMap<String, String>,
    pub features: Vec<String>,
    pub metadata: CrateMetadata,
}

pub struct GenerationRequest {
    pub prompt: String,
    pub context: String,
    pub provider: String,
    pub model: String,
    pub parameters: GenerationParameters,
}

pub struct AnalysisResult {
    pub metrics: CodeMetrics,
    pub dependencies: DependencyGraph,
    pub suggestions: Vec<Improvement>,
    pub security: SecurityReport,
}

Monitoring & Observability

Metrics Collection

# Prometheus metrics endpoint
curl http://localhost:8080/metrics

# Health check endpoint
curl http://localhost:8080/health

# System status
curl http://localhost:8080/status

Monitoring Stack

• Prometheus: Metrics collection and alerting
• Grafana: Dashboards and visualization
• Jaeger: Distributed tracing
• ELK Stack: Log aggregation and analysis

Key Metrics

• Request latency and throughput
• AI provider response times
• Code generation success rates
• Resource utilization
• Error rates and types
• Cache hit ratios

Logging

use tracing::{info, warn, error};

// Structured logging with context
info!(
    user_id = %user.id,
    crate_name = %spec.name,
    generation_time_ms = generation_time.as_millis(),
    "Crate generation completed successfully"
);

Security

Authentication & Authorization

// JWT-based authentication
#[derive(Serialize, Deserialize)]
struct Claims {
    sub: String,
    role: UserRole,
    exp: usize,
}

// Role-based access control
#[derive(Serialize, Deserialize)]
enum UserRole {
    Admin,
    Developer,
    ReadOnly,
}

Security Features

• Input Validation: Comprehensive input sanitization
• Rate Limiting: Configurable request throttling
• API Key Management: Secure API key handling
• Audit Logging: Complete audit trail
• Dependency Scanning: Automated vulnerability detection
• Secure Defaults: Security-first configuration

Best Practices

1. Environment Variables: Store secrets in environment variables
2. HTTPS Only: Force HTTPS in production
3. Regular Updates: Keep dependencies updated
4. Monitoring: Monitor for suspicious activity
5. Backup: Regular database backups
6. Access Control: Principle of least privilege

API Reference

Core Endpoints

# Crate generation
POST /api/v1/generate
Content-Type: application/json

{
  "name": "my-crate",
  "description": "Description here",
  "features": ["async", "serde"],
  "template": "library"
}

# Project analysis
POST /api/v1/analyze
Content-Type: multipart/form-data

# Health check
GET /health

# Metrics
GET /metrics

# OpenAPI specification
GET /docs

WebSocket Endpoints

// Real-time generation updates
const ws = new WebSocket('ws://localhost:8080/ws/generate');

ws.onmessage = (event) => {
    const update = JSON.parse(event.data);
    console.log('Generation progress:', update.progress);
};

Contributing

We welcome contributions! Please see our Contributing Guide for details.

Development Setup

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests
5. Ensure all tests pass
6. Submit a pull request

Code Style

• Follow Rust standard formatting (cargo fmt)
• Pass all lints (cargo clippy)
• Add documentation for public APIs
• Include tests for new functionality

License

Licensed under either of:

• Apache License, Version 2.0 (LICENSE-APACHE)
• MIT License (LICENSE-MIT)

at your option.

Changelog

Version 3.0.0 (Released!)

Major Release - Complete Rewrite

• New Architecture: Completely redesigned modular architecture
• AI Integration: Advanced LLM provider system with OpenAI and Codex
• FastAPI Server: Production-ready HTTP server with OpenAPI docs
• Enterprise Features: Monitoring, security, and scalability improvements
• Docker Support: Complete containerization with multi-stage builds
• Database Integration: SQLite, PostgreSQL, and Redis support
• Comprehensive Testing: 67+ tests covering all functionality
• Documentation: Complete API documentation and examples

Version 2.0.0

• Template system improvements
• Basic AI integration
• CLI enhancements

Version 1.0.0

• Initial release
• Basic crate generation
• Simple template system

OpenCrates - Intelligent Rust Development, Powered by AI