| Crates.io | nntp-proxy |
| lib.rs | nntp-proxy |
| version | 0.3.0 |
| created_at | 2025-09-05 15:45:33.496295+00 |
| updated_at | 2025-11-21 22:23:47.53351+00 |
| description | High-performance NNTP proxy server with connection pooling and authentication |
| homepage | https://github.com/mjc/nntp-proxy |
| repository | https://github.com/mjc/nntp-proxy |
| max_upload_size | |
| id | 1825728 |
| size | 1,516,942 |
A high-performance NNTP proxy server written in Rust, with intelligent hybrid routing, round-robin load balancing, and TLS support.
This NNTP proxy offers three operating modes:
--routing-mode stateful) - Full NNTP proxy with complete command support--routing-mode per-command) - Pure stateless routing for maximum efficiency✅ Hybrid mode (default) - Best for:
✅ Stateful mode - Good for:
✅ Per-command routing mode - Good for:
❌ Not suitable for:
When running in per-command routing mode (--per-command-routing or -r), the proxy rejects stateful commands to maintain consistent routing:
Rejected commands (require group context):
GROUP, NEXT, LAST, LISTGROUPARTICLE 123, HEAD 123, BODY 123, STAT 123XOVER, OVER, XHDR, HDRAlways supported:
ARTICLE <message-id@example.com>LIST, HELP, DATE, CAPABILITIESPOST (if backend supports)AUTHINFO USER/PASS (handled by proxy)Rationale: Commands requiring group context (current article number, group selection) cannot work reliably when each command routes to a different backend. Use stateful mode or hybrid mode if you need these features.
Hybrid mode automatically handles the per-command routing limitations:
In stateful mode (--routing-mode stateful):
# Clone the repository
git clone https://github.com/mjc/nntp-proxy.git
cd nntp-proxy
# Build release version
cargo build --release
# Binary will be in target/release/nntp-proxy
# Enter development environment
nix develop
# Or use direnv
direnv allow
# Build and run
cargo build
cargo run
./target/release/nntp-proxy --port 8119 --config config.toml
telnet localhost 8119
The proxy includes Docker support with environment variable configuration for easy deployment.
# Build the image
docker build -t nntp-proxy .
# Run with environment variables (no config file needed!)
docker run -d \
--name nntp-proxy \
-p 8119:8119 \
-e NNTP_SERVER_0_HOST=news.example.com \
-e NNTP_SERVER_0_PORT=119 \
-e NNTP_SERVER_0_NAME="My News Server" \
-e NNTP_PROXY_ROUTING_MODE=hybrid \
nntp-proxy
The repository includes a docker-compose.yml with examples:
# Edit docker-compose.yml to set your backend servers
# Then start the proxy:
docker-compose up -d
# View logs
docker-compose logs -f
# Stop the proxy
docker-compose down
Proxy Configuration:
NNTP_PROXY_PORT - Port to listen on (default: 8119)NNTP_PROXY_ROUTING_MODE - Routing mode: standard, per-command, or hybrid (default: hybrid)NNTP_PROXY_THREADS - Number of worker threads (default: number of CPUs)NNTP_PROXY_CONFIG - Path to config file (default: /etc/nntp-proxy/config.toml)RUST_LOG - Log level: trace, debug, info, warn, or error (default: info)Backend Server Configuration:
Configure servers using indexed environment variables:
# Server 0 (required - at least one server)
NNTP_SERVER_0_HOST=news.example.com
NNTP_SERVER_0_PORT=119
NNTP_SERVER_0_NAME="Primary Server"
NNTP_SERVER_0_USERNAME=user # Optional
NNTP_SERVER_0_PASSWORD=pass # Optional
NNTP_SERVER_0_MAX_CONNECTIONS=10 # Optional
# Server 1 (optional - for load balancing)
NNTP_SERVER_1_HOST=news2.example.com
NNTP_SERVER_1_PORT=119
NNTP_SERVER_1_NAME="Secondary Server"
# Server 2, 3, 4... (add as many as needed)
Configuration Priority:
NNTP_SERVER_* env vars if presentNNTP_SERVER_* environment variables are set → use env vars onlyNote: Command-line arguments (like --port) always take precedence over both config file and environment variables.
This allows you to:
# docker-compose.yml
version: '3.8'
services:
nntp-proxy:
image: nntp-proxy
ports:
- "8119:8119"
environment:
NNTP_PROXY_ROUTING_MODE: hybrid
RUST_LOG: info
# ⚠️ SECURITY: Never hardcode credentials in compose files. Use environment variable substitution.
# Copy .env.example to .env and fill in your credentials, then reference them here.
# Three backends for round-robin load balancing
NNTP_SERVER_0_HOST: news1.example.com
NNTP_SERVER_0_PORT: 119
NNTP_SERVER_0_NAME: "Server 1"
NNTP_SERVER_0_USERNAME: ${BACKEND_USER_0}
NNTP_SERVER_0_PASSWORD: ${BACKEND_PASS_0}
NNTP_SERVER_1_HOST: news2.example.com
NNTP_SERVER_1_PORT: 119
NNTP_SERVER_1_NAME: "Server 2"
NNTP_SERVER_1_USERNAME: ${BACKEND_USER_1}
NNTP_SERVER_1_PASSWORD: ${BACKEND_PASS_1}
NNTP_SERVER_2_HOST: news3.example.com
NNTP_SERVER_2_PORT: 119
NNTP_SERVER_2_NAME: "Server 3"
NNTP_SERVER_2_USERNAME: ${BACKEND_USER_2}
NNTP_SERVER_2_PASSWORD: ${BACKEND_PASS_2}
restart: unless-stopped
The proxy uses a TOML configuration file. Create config.toml:
# Backend servers (at least one required)
[[servers]]
host = "news.example.com"
port = 119
name = "Primary News Server"
username = "your_username" # Optional
password = "your_password" # Optional
max_connections = 20 # Optional, default: 10
[[servers]]
host = "news2.example.com"
port = 119
name = "Secondary News Server"
max_connections = 10
# Health check configuration (optional)
[health_check]
interval = 30 # Seconds between checks (default: 30)
timeout = 5 # Timeout per check (default: 5)
unhealthy_threshold = 3 # Failures before marking unhealthy (default: 3)
| Field | Type | Required | Default | Description |
|---|---|---|---|---|
host |
string | Yes | - | Backend server hostname or IP |
port |
integer | Yes | - | Backend server port |
name |
string | Yes | - | Friendly name for logging |
username |
string | No | - | Authentication username |
password |
string | No | - | Authentication password |
max_connections |
integer | No | 10 | Max concurrent connections to this backend |
use_tls |
boolean | No | false | Enable TLS/SSL encryption |
tls_verify_cert |
boolean | No | true | Verify server certificates (uses system CA store) |
tls_cert_path |
string | No | - | Path to additional CA certificate (PEM format) |
connection_keepalive |
integer | No | - | Send DATE command every N seconds on idle connections (omit to disable) |
The proxy supports TLS/SSL encrypted connections to backend servers using rustls - a modern, memory-safe TLS implementation written in pure Rust.
For servers with valid SSL certificates from recognized CAs:
[[servers]]
host = "secure.newsserver.com"
port = 563 # Standard NNTPS port
name = "Secure News Server"
use_tls = true
tls_verify_cert = true # Uses system certificate store (default)
max_connections = 20
That's it! No additional certificate configuration needed. The proxy will:
For servers using certificates from a private CA:
[[servers]]
host = "internal.newsserver.local"
port = 563
name = "Internal News Server"
use_tls = true
tls_verify_cert = true
tls_cert_path = "/etc/nntp-proxy/internal-ca.pem" # PEM format
max_connections = 10
Note: The custom certificate is added to the system certificates, not replacing them.
| Operating System | Certificate Store |
|---|---|
| Linux (Debian/Ubuntu) | /etc/ssl/certs/ca-certificates.crt |
| Linux (RHEL/CentOS) | /etc/pki/tls/certs/ca-bundle.crt |
| macOS | Security.framework (Keychain) |
| Windows | SChannel (Windows Certificate Store) |
| Port | Protocol | Description |
|---|---|---|
| 119 | NNTP | Unencrypted, standard NNTP |
| 563 | NNTPS | NNTP over TLS/SSL (encrypted) |
| 8119 | Custom | Common alternative port |
✅ Always verify certificates in production (tls_verify_cert = true)
✅ Keep system certificates updated via OS package manager
✅ Use TLS 1.3 when possible (automatically negotiated by rustls)
✅ Use standard NNTPS port 563 for encrypted connections
✅ Monitor TLS handshake failures in logs
⚠️ Never set tls_verify_cert = false in production - this disables all certificate verification and is extremely insecure!
Backend servers can be configured entirely via environment variables, useful for Docker/container deployments. If any NNTP_SERVER_N_HOST variable is found, environment variables take precedence over the config file.
Per-server variables (N = 0, 1, 2, ...):
| Variable | Required | Default | Description |
|---|---|---|---|
NNTP_SERVER_N_HOST |
Yes | - | Backend hostname/IP (presence triggers env mode) |
NNTP_SERVER_N_PORT |
No | 119 | Backend port |
NNTP_SERVER_N_NAME |
No | "Server N" | Friendly name for logging |
NNTP_SERVER_N_USERNAME |
No | - | Backend authentication username |
NNTP_SERVER_N_PASSWORD |
No | - | Backend authentication password |
NNTP_SERVER_N_MAX_CONNECTIONS |
No | 10 | Max concurrent connections |
Example Docker deployment:
docker run -e NNTP_SERVER_0_HOST=news.example.com \
-e NNTP_SERVER_0_PORT=119 \
-e NNTP_SERVER_0_NAME="Primary" \
-e NNTP_SERVER_0_USERNAME=user \
-e NNTP_SERVER_0_PASSWORD=pass \
-e NNTP_SERVER_1_HOST=news2.example.com \
-e NNTP_SERVER_1_PORT=119 \
-e NNTP_PROXY_PORT=8119 \
nntp-proxy
| Field | Type | Default | Description |
|---|---|---|---|
interval |
integer | 30 | Seconds between health checks |
timeout |
integer | 5 | Health check timeout in seconds |
unhealthy_threshold |
integer | 3 | Consecutive failures before marking unhealthy |
The proxy handles authentication transparently:
Backend authentication (when credentials are configured)
username and password in server configurationClient authentication handling
AUTHINFO USER/PASS commands are intercepted by the proxynntp-proxy [OPTIONS]
| Option | Short | Environment Variable | Description | Default |
|---|---|---|---|---|
--port <PORT> |
-p |
NNTP_PROXY_PORT |
Listen port | 8119 |
--routing-mode <MODE> |
-r |
NNTP_PROXY_ROUTING_MODE |
Routing mode: hybrid, standard, per-command | hybrid |
--config <FILE> |
-c |
NNTP_PROXY_CONFIG |
Config file path | config.toml |
--threads <NUM> |
-t |
NNTP_PROXY_THREADS |
Tokio worker threads | CPU cores |
--help |
-h |
- | Show help | - |
--version |
-V |
- | Show version | - |
Note: Environment variables take precedence over default values but are overridden by command-line arguments.
# Hybrid mode with defaults (recommended)
nntp-proxy
# Custom port and config (still uses hybrid mode)
nntp-proxy --port 8120 --config production.toml
# Stateful mode (full stateful behavior)
nntp-proxy --routing-mode stateful
# Per-command routing mode (pure stateless)
nntp-proxy --routing-mode per-command
# Short form for routing modes
nntp-proxy -r stateful
nntp-proxy -r per-command
# Single-threaded for debugging
nntp-proxy --threads 1
# Production setup
nntp-proxy --port 119 --config /etc/nntp-proxy/config.toml
# Using environment variables for configuration
NNTP_PROXY_PORT=8119 \
NNTP_PROXY_THREADS=4 \
NNTP_SERVER_0_HOST=news.example.com \
NNTP_SERVER_0_PORT=119 \
NNTP_SERVER_0_NAME="Primary" \
nntp-proxy
# Docker deployment with environment variables
docker run -d \
-e NNTP_PROXY_PORT=119 \
-e NNTP_SERVER_0_HOST=news.provider.com \
-e NNTP_SERVER_0_USERNAME=myuser \
-e NNTP_SERVER_0_PASSWORD=mypass \
-e NNTP_SERVER_1_HOST=news2.provider.com \
-p 119:119 \
nntp-proxy
--routing-mode hybrid--routing-mode stateful--routing-mode per-commandThe codebase is organized into focused modules with clear responsibilities:
| Module | Purpose |
|---|---|
auth/ |
Client and backend authentication (RFC 4643 AUTHINFO) |
cache/ |
Article caching with TTL-based expiration (cache proxy binary) |
command/ |
NNTP command parsing and classification |
config/ |
Configuration loading and validation (TOML + environment variables) |
constants/ |
Buffer sizes, timeouts, and performance tuning constants |
health/ |
Backend health monitoring with DATE command probes |
network/ |
Socket optimization for high-throughput transfers |
pool/ |
Connection and buffer pooling with deadpool |
protocol/ |
RFC 3977 protocol parsing, response categorization, message-ID handling |
router/ |
Backend selection with lock-free round-robin and health awareness |
session/ |
Client session lifecycle and command/response streaming |
stream/ |
Connection abstraction supporting TCP and TLS |
tls/ |
TLS configuration and handshake management using rustls |
types/ |
Core type definitions (ClientId, BackendId) |
The protocol module centralizes all NNTP protocol knowledge:
commands.rs: Command construction helpers (QUIT, DATE, AUTHINFO, ARTICLE, etc.)responses.rs: Response constants and builders (AUTH_REQUIRED, BACKEND_UNAVAILABLE, etc.)response.rs: Response parsing with ResponseCode enum for type-safe categorization
Client Connection
↓
Send Greeting (200 NNTP Proxy Ready)
↓
Read Command
↓
Parse Command (is_stateful check)
↓
┌─ Stateless Command ─────┐ ┌─ Stateful Command ────┐
│ Route to Backend │ │ Switch to Stateful │
│ (per-command routing) │ │ Reserve Backend │
│ Execute & Stream │ │ Bidirectional Forward │
│ Return to Pool │ │ (until disconnect) │
└─────────────────────────┘ └────────────────────────┘
↓ ↓
Return to Command Reading Connection Cleanup
Client Connection
↓
Select Backend (round-robin, health-aware)
↓
Get Pooled Connection (pre-authenticated)
↓
Bidirectional Data Forwarding
↓
Connection Cleanup & Return to Pool
Client Connection
↓
Send Greeting (200 NNTP Proxy Ready)
↓
Read Command
↓
Parse Command (protocol/command.rs)
↓
Route to Healthy Backend (round-robin)
↓
Get Pooled Connection
↓
Execute Command (waits for complete response)
↓
Stream Response to Client
↓
Return Connection to Pool
↓
Repeat (serial command processing)
The proxy implements several performance optimizations:
| Optimization | Impact | Description |
|---|---|---|
| ResponseCode enum | Eliminates redundant parsing | Parse response once, reuse for multiline detection and success checks |
| Lock-free routing | ~10-15% CPU reduction | Atomic operations for backend selection instead of RwLock |
| Pre-authenticated pools | Eliminates auth overhead | Connections authenticate once during pool initialization |
| Buffer pooling | ~200+ allocs/sec saved | Reuse pre-allocated buffers in hot paths |
| Optimized I/O | Fewer syscalls | 256KB buffers for article transfers, TCP socket tuning |
| TLS 1.3 with 0-RTT | Faster reconnections | Session resumption and early data support in rustls |
| Direct byte parsing | Avoids allocations | Message-ID extraction and protocol parsing work on byte slices |
The proxy adheres to NNTP standards:
\r\n.\r\n)To generate a performance flamegraph for analysis:
# Install cargo-flamegraph (if using Nix, it's already available)
cargo install flamegraph
# Run with flamegraph profiling (per-command routing mode)
cargo flamegraph --bin nntp-proxy -- --config config.toml -r --threads 1
# Open flamegraph.svg in a browser to analyze CPU hotspots
cargo build
./target/debug/nntp-proxy
cargo build --release
./target/release/nntp-proxy
# Build optimized binary
cargo build --release
# Copy binary to deployment location
sudo cp target/release/nntp-proxy /usr/local/bin/
# Create config directory
sudo mkdir -p /etc/nntp-proxy
# Copy config
sudo cp config.toml /etc/nntp-proxy/
# Run as service (example systemd unit included)
sudo systemctl start nntp-proxy
For maximum portability, build a fully static binary:
# Install musl target
rustup target add x86_64-unknown-linux-musl
# Build static binary
cargo build --release --target x86_64-unknown-linux-musl
# Result is a static binary with no dependencies
./target/x86_64-unknown-linux-musl/release/nntp-proxy
# All tests
cargo test
# Unit tests only
cargo test --lib
# Integration tests only
cargo test --test integration_tests
# With output
cargo test -- --nocapture
# Quiet mode
cargo test --quiet
The codebase includes:
Test with telnet or netcat:
# Connect to proxy
telnet localhost 8119
# Should see greeting like:
# 200 news.example.com ready
# Try commands:
HELP
LIST ACTIVE
ARTICLE <message-id@example.com>
QUIT
For performance testing, create custom scripts that:
| Crate | Purpose |
|---|---|
tokio |
Async runtime and networking |
rustls |
Modern, memory-safe TLS implementation |
tokio-rustls |
Tokio integration for rustls |
webpki-roots |
Mozilla's CA certificate bundle |
rustls-native-certs |
System certificate store integration |
tracing / tracing-subscriber |
Structured logging framework |
anyhow |
Ergonomic error handling |
clap |
Command-line argument parsing with derive macros |
serde / toml |
Configuration parsing and serialization |
deadpool |
Generic connection pooling |
moka |
High-performance cache (for cache proxy) |
memchr |
Fast byte searching (message-ID extraction) |
tempfile - Temporary files for config testingtests/test_helpers.rs"Connection refused" when starting
lsof -i :8119--port 8120"Backend authentication failed"
"Command not supported" errors
High CPU usage
-r or --per-command-routing--threads 1Backends marked unhealthy
Control log verbosity with RUST_LOG:
# Info level (default)
RUST_LOG=info nntp-proxy
# Debug level
RUST_LOG=debug nntp-proxy
# Specific module
RUST_LOG=nntp_proxy::router=debug nntp-proxy
# Multiple modules
RUST_LOG=nntp_proxy::router=debug,nntp_proxy::health=debug nntp-proxy
Contributions welcome! Please:
./scripts/install-git-hooks.shcargo test - Run all testscargo clippy --all-targets --all-features - Run lintercargo fmt - Format codeAfter cloning the repository, install git hooks to automatically run code quality checks:
./scripts/install-git-hooks.sh
The pre-commit hook will automatically run:
cargo fmt --check - Verify code formattingcargo clippy --all-targets --all-features - Check for lint warningsTo bypass the hook temporarily (not recommended): git commit --no-verify
MIT License - see LICENSE file for details.
Built with Rust and the excellent Tokio async ecosystem.