OxiFY - LLM Workflow Orchestration Platform

OxiFY is a graph-based LLM workflow orchestration platform built in Rust, designed to compose complex AI applications using directed acyclic graphs (DAGs). This meta-crate provides unified access to all OxiFY components.

Features

• Graph-Based Workflows: Define LLM applications as visual DAGs
• Type-Safe Execution: Compile-time guarantees for workflow structure
• Multi-Provider Support: OpenAI, Anthropic, local models, and more
• Vector Database Integration: Qdrant, in-memory vector search for RAG workflows
• Vision/OCR Processing: Multi-provider OCR with Tesseract, Surya, PaddleOCR
• MCP Support: Native support for Model Context Protocol
• REST API: Full-featured API for workflow management
• Pure Rust: No C/Fortran dependencies (COOLJAPAN Policy)

Quick Start

Add OxiFY to your Cargo.toml:

[dependencies]
oxify = "0.1"
tokio = { version = "1", features = ["full"] }

Using the Prelude

The prelude provides convenient access to commonly used types:

use oxify::prelude::*;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Build a workflow
    let workflow = WorkflowBuilder::new("simple-chat")
        .description("A simple chat workflow")
        .build()?;

    // Create an LLM node configuration
    let llm_config = LlmConfig {
        provider: "openai".to_string(),
        model: "gpt-4".to_string(),
        system_prompt: Some("You are a helpful assistant.".to_string()),
        prompt_template: "{{user_input}}".to_string(),
        temperature: Some(0.7),
        max_tokens: Some(1000),
        extra_params: serde_json::Value::Null,
    };

    Ok(())
}

Direct Module Access

You can also access individual modules directly:

use oxify::model::{Workflow, Node, NodeKind, Edge};
use oxify::engine::{Engine, ExecutionConfig};
use oxify::vector::{HnswIndex, HnswConfig, DistanceMetric};
use oxify::connect_llm::{LlmRequest, LlmResponse};

Module Overview

This meta-crate re-exports all OxiFY library crates:

Architecture

+-------------------------------------------------------------+
|                      OxiFY Platform                          |
+-------------------------------------------------------------+
|  API Layer (oxify-server)                                   |
|    +-> Authentication (oxify-authn)                         |
|    +-> Authorization (oxify-authz)                          |
|    +-> Middleware (CORS, logging, compression)              |
+-------------------------------------------------------------+
|  Workflow Engine (oxify-engine)                             |
|    +-> DAG Execution                                        |
|    +-> Node Processors (LLM, Vision, Retriever, Code)       |
|    +-> Plugin System                                        |
+-------------------------------------------------------------+
|  Connector Layer                                            |
|    +-> LLM Clients (oxify-connect-llm)                      |
|    +-> Vision/OCR (oxify-connect-vision)                    |
|    +-> Vector DB (oxify-connect-vector)                     |
|    +-> Vector Search (oxify-vector)                         |
+-------------------------------------------------------------+

Examples

Creating a Workflow

use oxify::model::{Workflow, Node, NodeKind, Edge, LlmConfig};

fn create_chat_workflow() -> Workflow {
    let mut workflow = Workflow::new("chat-bot".to_string());

    // Create nodes
    let start = Node::new("Start".to_string(), NodeKind::Start);
    let llm = Node::new("LLM".to_string(), NodeKind::Llm(LlmConfig {
        provider: "openai".to_string(),
        model: "gpt-4".to_string(),
        system_prompt: Some("You are helpful.".to_string()),
        prompt_template: "{{input}}".to_string(),
        temperature: Some(0.7),
        max_tokens: Some(1000),
        extra_params: serde_json::Value::Null,
    }));
    let end = Node::new("End".to_string(), NodeKind::End);

    let start_id = start.id;
    let llm_id = llm.id;
    let end_id = end.id;

    workflow.add_node(start);
    workflow.add_node(llm);
    workflow.add_node(end);

    workflow.add_edge(Edge::new(start_id, llm_id));
    workflow.add_edge(Edge::new(llm_id, end_id));

    workflow.validate().expect("Invalid workflow");
    workflow
}

Vector Search with HNSW

use oxify::vector::{HnswIndex, HnswConfig, DistanceMetric, SearchResult};

fn vector_search_example() {
    // Create HNSW index
    let config = HnswConfig {
        m: 16,
        ef_construction: 200,
        ef_search: 50,
        distance_metric: DistanceMetric::Cosine,
        ..Default::default()
    };

    let mut index = HnswIndex::new(384, config);

    // Add vectors
    let vectors = vec![
        vec![0.1, 0.2, 0.3], // ... 384 dimensions
        vec![0.4, 0.5, 0.6],
    ];

    for (i, vec) in vectors.iter().enumerate() {
        index.insert(i as u64, vec.clone());
    }

    // Search
    let query = vec![0.15, 0.25, 0.35]; // ... 384 dimensions
    let results = index.search(&query, 10);
}

Using LLM Providers

use oxify::connect_llm::{LlmRequest, LlmResponse, OpenAIProvider, LlmProvider};

async fn llm_example() -> Result<(), Box<dyn std::error::Error>> {
    let provider = OpenAIProvider::new("your-api-key".to_string());

    let request = LlmRequest {
        model: "gpt-4".to_string(),
        messages: vec![
            // ... messages
        ],
        temperature: Some(0.7),
        max_tokens: Some(1000),
        ..Default::default()
    };

    let response = provider.complete(&request).await?;
    println!("Response: {}", response.content);

    Ok(())
}

Node Types

OxiFY supports 16+ workflow node types:

Supported Providers

LLM Providers

• OpenAI (GPT-3.5, GPT-4, GPT-4-Turbo)
• Anthropic (Claude 3 Opus, Sonnet, Haiku)
• Ollama (Local models: Llama, Mistral, etc.)
• Gemini, Mistral, Cohere, Bedrock

Vector Databases

• Qdrant (cloud and self-hosted)
• In-memory HNSW index

Embedding Providers

• OpenAI (text-embedding-ada-002, text-embedding-3-small/large)
• Ollama (local embeddings)

Vision/OCR Providers

• Tesseract OCR
• Surya
• PaddleOCR

Performance

• LLM Response Caching: 1-hour TTL for cost savings
• Execution Plan Caching: 100-entry LRU cache
• Rate Limiting: Configurable (default 500 req/min)
• Parallel Execution: Level-based DAG parallelism
• SIMD Acceleration: Vector operations use SIMD when available
• Retry Logic: Exponential backoff with configurable limits

Individual Crates

If you only need specific functionality, you can depend on individual crates:

# Just the workflow model
oxify-model = "0.1"

# Just vector search
oxify-vector = "0.1"

# Just LLM connections
oxify-connect-llm = "0.1"

Binary Applications

The following binary applications are available separately:

• oxify-api: REST API server
• oxify-cli: Command-line workflow runner
• oxify-ui: Web-based workflow editor (coming soon)

Development Status

Version 0.1.0 - Production-ready!

• Core Infrastructure: Complete
• LLM Workflow Engine: Complete
• API Layer: Complete
• CLI Tool: Complete
• Web UI: In Progress

Related Projects

OxiFY is part of the COOLJAPAN ecosystem:

• SciRS2 - Scientific computing in Pure Rust
• NumRS2 - Numerical computing library
• ToRSh - PyTorch-like tensor library
• OxiRS - Semantic web platform

License

Apache-2.0 - See LICENSE file for details.

Author

COOLJAPAN OU (Team Kitasan)