Litho (deepwiki-rs)

English | 中文

💪🏻 High-performance AI-driven intelligent documentation generation engine (DeepWiki-like) built with Rust

📚 Automatically generates professional C4-style project architecture documentation for any software project

[!WARNING] ⚠️ As we plan to ship a torrent of features in the following months, future updates will contain breaking changes. With the architecture evolving, we'll annotate changes and highlight migration paths as we encounter them.

🚀 Help me develop this software better by sponsoring on GitHub

👋 What's Litho

Litho (also known as deepwiki-rs) is a high-performance AI documentation generation engine built with Rust, capable of automatically creating comprehensive technical documentation for software projects. Through advanced AI analysis capabilities, Litho intelligently explores project structures, identifies core modules, analyzes dependency relationships, and generates professional architecture documentation.

Litho employs a multi-engine workflow based on static code analysis and AI Multi-Agents, combining static code analysis with large language model (LLM) intelligence. The system uses specialized AI agents to conduct deep research on system context, architecture patterns, business workflows, and core module insights, ultimately generating comprehensive documentation including overview, architecture, and detailed technical topic documentation.

❤️ Like Litho? Star it 🌟 or Sponsor Me! ❤️

🌠 Features & Capabilities

• AI-Driven Multi-Stage Workflow: Implements a four-stage pipeline (Preprocessing → Research → Compose → Output) with specialized AI agents for comprehensive project analysis.
• Project Knowledge & Technical Documentation Generation: Generates professional C4 architecture-style documentation including project overview, technical architecture, and detailed Deep Dive intelligent topic documentation.
• Multi-Agent Research System: Deploys specialized AI agents for system context analysis, architecture pattern recognition, business workflow reconstruction, and core module insights.
• Comprehensive Language Support: Built-in processors for 10+ programming languages including Rust, Python, JavaScript/TypeScript, Java, Kotlin, React, Vue, Svelte, and more.
• Intelligent Code Analysis: Combines static code analysis with LLM-driven understanding to extract project structure, dependencies, and architectural patterns.
• ReAct Pattern Integration: Utilizes ReAct (Reasoning + Acting) pattern for systematic codebase exploration and analysis.
• Caching & Performance Optimization: Includes intelligent caching system and parallel processing capabilities for efficient large-scale project analysis.
• Extensible Architecture: Modular design with trait-based language processors and agent system supporting easy extension for new languages and analysis capabilities.
• Professional Documentation Output: Generates markdown and HTML documentation with proper structure, diagrams, and detailed module analysis.

🧠 How it works

Litho's documentation generation process is divided into four distinct phases: preprocessing, research, composition, and output. Each phase is executed by specialized agents, forming a pipeline-style workflow.

Four-Stage Processing Pipeline

1. Preprocessing Stage: Extracts project structure, analyzes code files with language-specific processors, and identifies core modules and dependencies.
2. Research Stage: Deploys multiple specialized AI agents to conduct deep analysis:
   • System Context Researcher: Analyzes overall system context and external integrations
   • Architecture Researcher: Identifies architectural patterns and design principles
   • Workflow Researcher: Reconstructs business processes and data flows
   • Domain Module Detector: Discovers and analyzes domain-specific modules
3. Compose Stage: Intelligent document editors synthesize research findings into professional documentation:
   • Overview Editor: Generates project overview and core functionality documentation
   • Architecture Editor: Creates comprehensive architecture documentation
   • Key Modules Insight Editor: Produces detailed core modules documentation
4. Output Stage: Formats and outputs the final documentation in the requested format.

Preprocessing Stage

This stage is responsible for extracting basic metadata from the project to provide input for subsequent AI analysis.

flowchart TD
A[Preprocessing Agent] --> B[Structure Extractor]
A --> C[Original Document Extractor]
A --> D[Code Analysis Agent]
A --> E[Relationship Analysis Agent]
B --> F[Project Structure]
C --> G[Original Document Materials]
D --> H[Core Code Insights]
E --> I[Code Dependencies]
F --> J[Store to Memory]
G --> J
H --> J
I --> J

Research Stage

This stage conducts multi-level deep research on the project through multiple agents running in parallel.

flowchart TD
A[Research Orchestrator] --> B[SystemContext Researcher]
A --> C[Domain Module Detector]
A --> D[Architecture Researcher]
A --> E[Workflow Researcher]
A --> F[Key Module Insights]
B --> G[System Context Report]
C --> H[Domain Module Report]
D --> I[Architecture Analysis Report]
E --> J[Workflow Analysis Report]
F --> K[Module Deep Insights]
G --> Memory
H --> Memory
I --> Memory
J --> Memory
K --> Memory

Composition and Output Stage

The final stage integrates research results into final documentation and persists it to disk.

flowchart TD
A[Document Composer] --> B[Overview Editor]
A --> C[Architecture Editor]
A --> D[Module Insight Editor]
B --> E[Overview Document]
C --> F[Architecture Document]
D --> G[Module Documents]
E --> H[Document Tree]
F --> H
G --> H
H --> I[Disk Outlet]
I --> J[Output Directory]

🏗️ Architecture Overview

Litho features a sophisticated modular architecture designed for high performance, extensibility, and intelligent analysis. The system implements a multi-stage workflow with specialized AI agents and comprehensive caching mechanisms.

flowchart TB
    subgraph "Core Engine"
        Workflow[Workflow Controller]
        Context[Generator Context]
        Memory[Shared Memory]
    end

    subgraph "Preprocessing Stage"
        StructureExtractor[Structure Extractor]
        LanguageProcessors[Language Processors]
        CodeAnalyzer[Code Analyzer]
    end

    subgraph "Research Stage"
        ResearchOrchestrator[Research Orchestrator]
        SystemContextAgent[System Context Agent]
        ArchitectureAgent[Architecture Agent]
        WorkflowAgent[Workflow Agent]
        DomainModuleAgent[Domain Module Agent]
    end

    subgraph "Compose Stage"
        OverviewEditor[Overview Editor]
        ArchitectureEditor[Architecture Editor]
        CoreModulesEditor[Core Modules Editor]
    end

    subgraph "Output Stage"
        SummaryGenerator[Summary Generator]
        DiskOutlet[Disk Outlet]
    end

    subgraph "Infrastructure"
        LLMClient[LLM Client]
        CacheManager[Cache Manager]
        FileTools[File Tools]
    end

    Workflow --> Context
    Context --> Memory

    Workflow --> StructureExtractor
    StructureExtractor --> LanguageProcessors
    LanguageProcessors --> CodeAnalyzer

    Workflow --> ResearchOrchestrator
    ResearchOrchestrator --> SystemContextAgent
    ResearchOrchestrator --> ArchitectureAgent
    ResearchOrchestrator --> WorkflowAgent
    ResearchOrchestrator --> DomainModuleAgent

    Workflow --> OverviewEditor
    Workflow --> ArchitectureEditor
    Workflow --> CoreModulesEditor

    Workflow --> SummaryGenerator
    SummaryGenerator --> DiskOutlet

    Context --> LLMClient
    Context --> CacheManager
    LLMClient --> FileTools

Core Modules

• Generator Workflow: Orchestrates the entire four-stage documentation generation pipeline
• Language Processor System: Supports 10+ programming languages with dedicated processors for Rust, Python, JavaScript/TypeScript, Java, Kotlin, React, Vue, Svelte, and more
• Multi-Agent Research System: Specialized AI agents for deep project analysis using ReAct pattern
• Intelligent Document Composers: AI-powered editors that synthesize research findings into professional technical documentation
• ReAct Executor LLM Client: Advanced LLM integration supporting reasoning and action loops with tool usage
• High-Speed Cache & Context Memory System: Performance optimization with intelligent caching and memory management
• File System Tools: Comprehensive file exploration and reading capabilities for project analysis

Core Process

Litho's execution process follows a clear pipeline pattern: Preprocessing → Deep Research → Document Composition → Output Storage. The entire process is driven by the launch function and injects all dependencies through GeneratorContext.

sequenceDiagram
participant Main as main.rs
participant Workflow as workflow.rs
participant Context as GeneratorContext
participant Preprocess as PreProcessAgent
participant Research as ResearchOrchestrator
participant Doc as DocumentationOrchestrator
participant Outlet as DiskOutlet
Main->>Workflow : launch(config)
Workflow->>Context : Create context (LLM, Cache, Memory)
Workflow->>Preprocess : execute(context)
Preprocess->>Context : Store project structure and metadata
Context-->>Workflow : Preprocessing complete
Workflow->>Research : execute_research_pipeline(context)
Research->>Research : Execute multiple research agents in parallel
loop Each Research Agent
Research->>StepForwardAgent : execute(context)
StepForwardAgent->>Context : Validate data sources
StepForwardAgent->>AgentExecutor : Call prompt or extract
AgentExecutor->>LLMClient : Initiate LLM request
LLMClient->>CacheManager : Check cache
alt Cache hit
CacheManager-->>LLMClient : Return cached result
else Cache miss
LLMClient->>LLM : Call LLM API
LLM-->>LLMClient : Return raw response
LLMClient->>CacheManager : Store result to cache
end
LLMClient-->>AgentExecutor : Return processed result
AgentExecutor-->>StepForwardAgent : Return result
StepForwardAgent->>Context : Store result to Memory
end
Research-->>Workflow : Research complete
Workflow->>Doc : execute(context, doc_tree)
Doc->>Doc : Call multiple composition agents to generate docs
Doc-->>Workflow : Documentation generation complete
Workflow->>Outlet : save(context)
Outlet-->>Workflow : Storage complete
Workflow-->>Main : Process finished

🖥 Getting Started

Prerequisites

• Rust (version 2024 or later)
• Cargo
• Internet connection (for LLM integration)

Installation

Option 1: Install from crates.io (Recommended)

The easiest way to install Litho is directly from crates.io:

cargo install deepwiki-rs

After installation, you can use deepwiki-rs command directly:

deepwiki-rs --help

Option 2: Build from Source

1. Clone the repository.

   git clone https://github.com/sopaco/deepwiki-rs.git
   

2. Step into the cloned project folder.

   cd deepwiki-rs
   

3. Build the project.

   cargo build --release
   

4. The compiled binary will be available in the target/release directory.

🚀 Usage

Litho provides a command-line interface for generating architecture documentation. Here are some basic usage examples:

Basic Command

# Generate project knowledge & technical documentation for current directory
litho

# Analyze specific project directory
litho -p /path/to/your/project

# Specify output directory
litho -o ./architecture-docs

# Specify project name
litho --name "My Project"

Documentation Generation

Litho generates professional project knowledge & technical documentation with the following structure:

• 1. Project Overview: Project overview, core functionality, and technology stack analysis
• 2. Architecture Overview: Comprehensive architecture design, core processes, and module breakdown
• 3. Workflow Overview: Comprehensive architecture design, core processes, and module breakdown
• 4. Deep Dive/: Detailed technical topic implementation documentation for each identified core modules topic
• __Litho_Summary_Brief__.md: Generation process summary brief
• __Litho_Summary_Detail__.md: Generation process execution details

# Generate complete project knowledge & technical documentation
litho -p ./my-project

# Generate with custom project name
litho --name "My Awesome Project" -o ./my-docs

Advanced Options

• Enable verbose logging:

  litho -v
  

• Configure LLM settings:

  litho --llm-api-base-url <api-url> --llm-api-key <api-key> --model-efficient <model-name>
  

• Skip specific stages:

  litho --skip-preprocessing  # Skip preprocessing stage
  litho --skip-research      # Skip research stage
  litho --skip-documentation # Skip documentation stage
  

• Advanced configuration:

  litho --config <config-file> --max-tokens 4000 --temperature 0.7 --max-parallels 4
  

For more detailed usage information, run:

litho --help

📁 Output Structure

Litho generates comprehensive project knowledge & technical documentation structure:

project-docs/
├── 1. Project Overview      # Project overview, core functionality, technology stack
├── 2. Architecture Overview # Overall architecture, core processes, module breakdown
├── 3. Workflow Overview     # Overall architecture, core processes, module breakdown
├── 4. Deep Dive/            # Detailed technical topic implementation documentation
│   ├── Topic1.md
│   ├── Topic2.md
│   └── ...

🤝 Contribute

Help improve Litho by reporting bugs or submitting feature requests through GitHub Issues.

Ways to Contribute

• Language Processor Development: Add support for additional programming languages by implementing the LanguageProcessor trait
• AI Agent Enhancement: Improve existing research agents or develop new specialized analysis agents
• Documentation Format Extension: Add support for new output formats beyond Markdown and HTML
• Performance Optimization: Enhance caching strategies and parallel processing capabilities
• Tool Integration: Extend the ReAct pattern with additional tools for code analysis
• Architecture Pattern Recognition: Improve the detection of architectural patterns and design principles
• International Support: Contribute to multi-language documentation generation capabilities

⚛️ Developed with

• rust - Systems programming language for performance and safety
• rig-core - Rust library for building LLM-powered applications
• tokio - Asynchronous runtime for Rust
• clap - Command line argument parser
• serde - Serialization framework
• anyhow - Error handling
• thiserror - Derive macros for error handling
• walkdir - Directory traversal
• regex - Regular expressions
• markdown - Markdown processing

🪪 License

MIT. A copy of the license is provided in the LICENSE file.

👨 About Me

🚀 Help me develop this software better by sponsoring on GitHub

An experienced internet veteran, having navigated through the waves of PC internet, mobile internet, and AI applications. Starting from an individual mobile application developer to a professional in the corporate world, I possess rich experience in product design and research and development. Currently, I am employed at Kuaishou, focusing on the R&D of universal front-end systems and AI exploration.

WeChat: dokhell

Email: dokhell@hotmail.com