Toast - LLM CLI & API Server

An unofficial CLI client and API server for Claude and DeepSeek that provides both a command-line interface and OpenAI-compatible API endpoints.

Overview

Toast is a versatile tool that provides multiple interfaces for working with LLMs:

1. A CLI client for direct interaction with Claude or DeepSeek (default)
2. An API server that provides OpenAI-compatible endpoints
3. Support for multiple LLM providers: Claude (Anthropic) and DeepSeek
4. Built-in coding agent capabilities with file operations and command execution

Features

• Interactive CLI interface for both Claude and DeepSeek
• OpenAI-compatible API server
• Support for multiple models (Opus, Sonnet, Haiku variants)
• Built-in coding agent with tool use (exec commands, file reading)
• Conversation state management
• Model mapping (gpt-4/gpt-4.1 → Claude models)
• CORS support for browser clients
• SHA3 proof-of-work solver for DeepSeek's anti-bot protection

Installation

# Clone the repository
git clone https://github.com/yourusername/toast.git
cd toast

# Build the project
cargo build --release

Usage

CLI Mode (Default)

By default, Toast runs in CLI mode with Claude. You can switch between providers:

# Use Claude (default)
./target/release/toast

# Use DeepSeek
./target/release/toast --deepseek

CLI options:

# Use Claude Opus model (claude-opus-4-20250514)
./target/release/toast --opus

# Use Claude Haiku model (claude-3-5-haiku-20241022)
./target/release/toast --haiku

# Use DeepSeek with R1 reasoning model (default for DeepSeek)
./target/release/toast --deepseek

# Use DeepSeek lite model
./target/release/toast --deepseek --haiku

CLI Commands

Toast includes a built-in coding agent that can execute commands and read files. These features work with both Claude and DeepSeek:

• Enter your message and press Enter to send to the LLM
• Use /exit or exit or x to quit
• The LLM can use # exec <command> to execute shell commands
• The LLM can use # read_file <filename> to read files
• Supports multiline commands using heredocs
• Automatic command execution based on LLM responses

API Server Mode

Run Toast as an OpenAI-compatible API server:

# Start the API server
./target/release/toast --serve

# Custom port
./target/release/toast --serve --port 8080

# Custom model
./target/release/toast --serve --model claude-sonnet-4-20250514-claude-ai

# Custom host address
./target/release/toast --serve --host 127.0.0.1

API Endpoint

The server exposes the following OpenAI-compatible endpoints:

• POST /v1/chat/completions - Chat completions API
• POST /chat/completions - Alternative endpoint
• POST / - Root endpoint (same as chat completions)
• GET /health - Health check endpoint

API Reference

Chat Completions

POST /v1/chat/completions

Request body:

{
  "model": "gpt-4.1",
  "messages": [
    {
      "role": "developer",
      "content": "You are a helpful assistant."
    },
    {
      "role": "user",
      "content": "Hello!"
    }
  ]
}

Response:

{
  "id": "f52a7702-6588-4b68-a057-bc2630d223b8",
  "object": "chat.completion",
  "created": 1683989000,
  "model": "claude-opus-4-20250514",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Hello! I'm Claude, a helpful AI assistant. How can I assist you today?"
      }
    }
  ]
}

The API supports various content formats:

• Simple string content
• Array of strings
• Structured content objects
• Content blocks with type fields

Model Mapping

The server maps OpenAI model names to Claude models:

• gpt-3.5-turbo → Claude Haiku (claude-3-5-haiku-20241022)
• gpt-4, gpt-4-turbo → Claude Sonnet (claude-sonnet-4-20250514-claude-ai)
• gpt-4o, gpt-4.1 → Claude Opus (claude-opus-4-20250514)

You can also specify a Claude model directly in the model field.

Authentication

The API server currently doesn't require authentication tokens from clients. However, you need to configure credentials for the underlying LLM providers (see Configuration section).

Configuration

Toast requires authentication credentials for each LLM provider stored in the config directory:

Claude Configuration

• macOS: ~/Library/Application Support/toast/
• Linux: ~/.config/toast/
• Windows: C:\Users\{USERNAME}\AppData\Roaming\toast\

Required files:

• cookie: Contains your Claude session cookie
• org_id: Your Claude organization ID (automatically extracted from cookie if not present)

DeepSeek Configuration

• macOS: ~/Library/Application Support/toast/deepseek/
• Linux: ~/.config/toast/deepseek/
• Windows: C:\Users\{USERNAME}\AppData\Roaming\toast\deepseek\

Required files:

• auth_token: Your DeepSeek authentication token (from Authorization header)
• cookies.json: DeepSeek cookies including Cloudflare verification

To get your DeepSeek auth token:

1. Go to chat.deepseek.com in your browser
2. Open Developer Tools (F12)
3. Go to Network tab
4. Look for Authorization header in any request
5. Save the token part (without 'Bearer ') to the auth_token file

Technical Details

• Built with Rust using Tokio for async runtime
• Uses Axum for the HTTP server
• Implements OpenAI's chat completion API specification
• Supports streaming responses in CLI mode
• Includes SHA3 proof-of-work solver for DeepSeek's anti-bot protection
• WebAssembly support for optimized cryptographic operations
• Conversation history tracking with context-based matching

Development

Building from source

cargo build

Running tests

cargo test

Dependencies

Key dependencies include:

• rquest: HTTP client with browser emulation
• axum: Web framework for the API server
• tokio: Async runtime
• clap: Command-line argument parsing
• wasmtime: WebAssembly runtime for DeepSeek POW
• sha3: Cryptographic hashing for DeepSeek verification
• dashmap: Concurrent hashmap for conversation tracking

License

MIT