IvoryValley

Status: Under Heavy Development - This project is in early development. APIs and features may change without notice.

A transparent deduplication proxy for Mastodon and the Fediverse.

The Problem

Following users across multiple Fediverse accounts often results in seeing the same posts repeatedly due to boosts/reposts. Your timeline can show the same content 10+ times.

The Solution

IvoryValley sits between your Mastodon client and the upstream server, filtering out duplicate posts before they reach you. It tracks seen post URIs and removes duplicates from timeline responses.

┌─────────────┐     ┌─────────────────┐     ┌──────────────┐
│   Client    │────▶│   IvoryValley   │────▶│   Mastodon   │
│  (Tusky,    │◀────│                 │◀────│   Instance   │
│   etc.)     │     │  - Filter dupes │     │              │
└─────────────┘     │  - Store URIs   │     └──────────────┘
                    │  - Pass auth    │
                    └─────────────────┘

Features

• Transparent proxying - Works with any Mastodon-compatible client
• Deduplication - Filters duplicate posts and boosts from timelines
• WebSocket streaming - Real-time filtering for streaming connections
• OAuth passthrough - No credential handling, tokens are forwarded directly
• SQLite storage - Lightweight local database for tracking seen URIs
• Configurable - CLI args, environment variables, or config file
• Health endpoint - /health endpoint for load balancers and Kubernetes probes

Installation

From Docker (Recommended)

Pull and run the latest image:

docker pull ghcr.io/jensens/ivoryvalley:latest
docker run -d \
  --name ivoryvalley \
  -p 8080:8080 \
  -v ivoryvalley-data:/data \
  -e IV_UPSTREAM_URL=https://mastodon.social \
  ghcr.io/jensens/ivoryvalley:latest

Or use docker-compose (see docker-compose.yml):

# Edit docker-compose.yml with your upstream URL
docker compose up -d

From crates.io

cargo install ivoryvalley

From GitHub Releases

Download the appropriate binary for your platform from the Releases page.

From Source

git clone https://github.com/jensens/ivoryvalley.git
cd ivoryvalley
cargo build --release

The binary will be at target/release/ivoryvalley.

Quick Start

1. Start IvoryValley

# Using Docker (recommended)
docker run -d -p 8080:8080 \
  -v ivoryvalley-data:/data \
  -e IV_UPSTREAM_URL=https://mastodon.social \
  ghcr.io/jensens/ivoryvalley:latest

# Or using the binary directly
ivoryvalley --upstream-url https://mastodon.social

2. Configure Your Client

Point your Mastodon client to your IvoryValley URL instead of your instance URL. Most clients require HTTPS - see Client Setup for detailed instructions and Local HTTPS Setup for development options.

3. Log In Normally

Use your regular credentials. IvoryValley passes authentication through to your instance transparently.

Usage

# Basic usage
ivoryvalley --upstream-url https://mastodon.social

# With custom host and port
ivoryvalley --upstream-url https://mastodon.social --host 127.0.0.1 --port 3000

# With environment variables
IV_UPSTREAM_URL=https://mastodon.social ivoryvalley

# With a config file
ivoryvalley --config /path/to/ivoryvalley.toml

Configuration

IvoryValley supports configuration via:

1. CLI arguments (highest priority)
2. Environment variables (prefixed with IV_)
3. Config file (config.toml, config.yaml, ivoryvalley.toml, or ivoryvalley.yaml)
4. Default values

Configuration Options

Option	Env Variable	Default	Description
--upstream-url	IV_UPSTREAM_URL	https://mastodon.social	Upstream Mastodon instance URL
--host	IV_HOST	0.0.0.0	Address to bind to
-p, --port	IV_PORT	8080	Port to listen on
--database-path	IV_DATABASE_PATH	ivoryvalley.db	SQLite database path
--max-body-size	IV_MAX_BODY_SIZE	52428800 (50MB)	Maximum request body size in bytes
--connect-timeout-secs	IV_CONNECT_TIMEOUT_SECS	10	HTTP connection timeout in seconds
--request-timeout-secs	IV_REQUEST_TIMEOUT_SECS	30	HTTP request timeout in seconds
--record-traffic-path	IV_RECORD_TRAFFIC_PATH	-	Path to record traffic (JSONL format)
--cleanup-interval-secs	IV_CLEANUP_INTERVAL_SECS	3600 (1 hour)	Interval between cleanup runs in seconds
--cleanup-max-age-secs	IV_CLEANUP_MAX_AGE_SECS	604800 (7 days)	Maximum age of stored URIs in seconds
-c, --config	IV_CONFIG	-	Path to configuration file

Config File Example

Create an ivoryvalley.toml file:

upstream_url = "https://mastodon.social"
host = "127.0.0.1"
port = 8080
database_path = "/var/lib/ivoryvalley/seen.db"

Or ivoryvalley.yaml:

upstream_url: "https://mastodon.social"
host: "127.0.0.1"
port: 8080
database_path: "/var/lib/ivoryvalley/seen.db"

Health Check Endpoint

The proxy exposes a /health endpoint for monitoring and orchestration systems.

Basic Health Check

curl http://localhost:8080/health

Response:

{"status": "healthy", "version": "x.y.z"}

Deep Health Check

Use the ?deep=true query parameter to include database connectivity verification:

curl http://localhost:8080/health?deep=true

Response:

{"status": "healthy", "version": "x.y.z", "checks": {"database": "ok"}}

This endpoint:

• Returns HTTP 200 when the service is healthy
• Does not require authentication
• Suitable for load balancer health checks and Kubernetes liveness/readiness probes

How It Works

1. Client sends request to IvoryValley
2. IvoryValley forwards request to upstream Mastodon instance
3. For timeline endpoints, IvoryValley:
   • Extracts post URIs (globally unique across federation)
   • Checks each URI against the seen-URI database
   • Filters out duplicates
   • Records new URIs as seen
4. Returns filtered response to client

Client Setup

IvoryValley works with any Mastodon-compatible client. You simply change the server URL from your instance (e.g., https://mastodon.social) to IvoryValley (e.g., http://localhost:8080).

Important Notes

• HTTP vs HTTPS: Most Mastodon clients (both desktop and mobile) require HTTPS for OAuth authentication. See Local HTTPS Setup for development options.
• Authentication: Your login credentials and OAuth tokens are passed through to the upstream server. IvoryValley does not store or access them.
• Same Instance: Make sure to configure IvoryValley with the same upstream URL as your Mastodon account.

Desktop Clients

Note: Despite being desktop applications, many Mastodon clients require HTTPS even for localhost connections due to OAuth security requirements and embedded browser security policies. See Local HTTPS Setup for workarounds.

Tuba (Linux GTK)

1. Install: flatpak install flathub dev.geopjr.Tuba
2. Open Tuba and click "Add Account"
3. Enter your IvoryValley HTTPS URL (e.g., https://localhost:8443 or your mkcert domain)
4. Log in with your normal Mastodon credentials

Tokodon (Linux KDE)

1. Install: apt install tokodon or via your package manager
2. Open Tokodon and add a new account
3. Enter your IvoryValley HTTPS URL as the server
4. Complete the OAuth flow normally

Whalebird (Cross-platform)

1. Download from Whalebird releases
2. Add new account and enter your IvoryValley HTTPS URL as the instance
3. Log in with your credentials

Mobile Clients

Mobile clients require HTTPS. You need to deploy IvoryValley behind a reverse proxy with a valid SSL certificate.

Recommended: Deploy with a Reverse Proxy

Use Caddy, nginx, or Traefik with a valid SSL certificate. This enables full functionality including WebSocket streaming.

Example with Caddy:

ivoryvalley.example.com {
    reverse_proxy localhost:8080
}

Tusky (Android)

1. Deploy IvoryValley with a valid HTTPS certificate
2. Open Tusky and tap "Log in"
3. Enter your IvoryValley URL as the instance
4. Complete the OAuth login

Ivory / Ice Cubes / Mona (iOS)

1. Deploy IvoryValley with a valid HTTPS certificate (iOS requires valid certificates)
2. Enter your IvoryValley URL as the instance
3. Log in normally

Web Clients

You can also access Mastodon's web interface through IvoryValley:

# Open in browser
xdg-open http://localhost:8080

Note that the web interface works best with HTTPS for all features.

Local HTTPS Setup

For local development and testing with desktop/mobile clients, you'll need HTTPS with trusted certificates. Here are recommended approaches:

Option 1: mkcert (Recommended for Development)

mkcert creates locally-trusted development certificates.

# Install mkcert
# macOS
brew install mkcert

# Linux (check your package manager, or use the binary release)
# See https://github.com/FiloSottile/mkcert#installation

# Create and install the local CA
mkcert -install

# Generate certificates for localhost
mkcert localhost 127.0.0.1 ::1

# This creates localhost+2.pem and localhost+2-key.pem

Then use a reverse proxy like Caddy to serve IvoryValley over HTTPS:

# Caddyfile
localhost:8443 {
    tls localhost+2.pem localhost+2-key.pem
    reverse_proxy localhost:8080
}

Option 2: Caddy with Automatic HTTPS

Caddy can automatically provision certificates. For local development with a custom domain:

1. Add an entry to /etc/hosts: 127.0.0.1 ivoryvalley.local
2. Use Caddy with mkcert certificates:

# Caddyfile
ivoryvalley.local {
    tls internal
    reverse_proxy localhost:8080
}

Option 3: Deploy with a Real Domain

For the most seamless experience, deploy IvoryValley on a server with a real domain name. Caddy will automatically obtain Let's Encrypt certificates:

# Caddyfile
ivoryvalley.example.com {
    reverse_proxy localhost:8080
}

Troubleshooting

Connection Issues

"Connection refused" error

• Ensure IvoryValley is running: check with curl http://localhost:8080/api/v1/instance
• Verify the port isn't blocked by a firewall
• Check if another service is using port 8080

"Timeout" errors

• Check if the upstream server is reachable: curl https://mastodon.social/api/v1/instance
• Increase timeout settings if on a slow connection:

  ivoryvalley --upstream-url https://mastodon.social \
    --connect-timeout-secs 30 \
    --request-timeout-secs 120
  

Authentication Issues

OAuth login fails or redirects to wrong URL

• Make sure your client is configured to use the IvoryValley URL (not the upstream instance)
• For HTTPS, ensure the certificate is accepted/trusted

"Unauthorized" errors after login

• Your token may have expired; try logging in again
• Make sure the upstream URL exactly matches your Mastodon instance

Deduplication Issues

Duplicates still appear

• IvoryValley only filters posts it has seen before; the first occurrence always appears
• Check if the database file exists and is writable
• Duplicates in notifications are expected (only timeline endpoints are filtered)

Want to reset seen posts?

Delete the database file to start fresh:

# Find the database location
ls -la ivoryvalley.db

# Stop IvoryValley, delete database, restart
rm ivoryvalley.db

Docker Issues

Container won't start

• Check logs: docker logs ivoryvalley
• Ensure the IV_UPSTREAM_URL environment variable is set

Database not persisting

• Make sure you're using a volume: -v ivoryvalley-data:/data
• Check volume permissions: the container runs as user 1000

Logging and Debugging

Enable debug logging to see detailed information:

# With the binary
RUST_LOG=ivoryvalley=debug ivoryvalley --upstream-url https://mastodon.social

# With Docker
docker run -e RUST_LOG=ivoryvalley=debug \
  -e IV_UPSTREAM_URL=https://mastodon.social \
  ghcr.io/jensens/ivoryvalley:latest

Log levels: error, warn, info, debug, trace

Recording Traffic for Debugging

If you need to debug API issues, you can record all traffic:

ivoryvalley --upstream-url https://mastodon.social \
  --record-traffic-path /tmp/traffic.jsonl

This creates a JSONL file with all request/response pairs.

Development

This project uses just as a command runner. Install with cargo install just.

# List all available commands
just

# Run the proxy in development mode
just dev

# Run all tests
just test

# Run all quality checks before committing
just check

Testing with Clients

See Client Setup for detailed client configuration.

For quick HTTPS testing with the web interface only:

just dev-https

Note: The development HTTPS proxy does not support WebSocket connections, so native clients (desktop/mobile apps) won't have working streaming. For full client testing, deploy with a proper reverse proxy like Caddy.

Manual Commands

# Run with logging
RUST_LOG=ivoryvalley=debug cargo run -- --upstream-url https://mastodon.social

# Check code quality
cargo clippy --all-features -- -D warnings

# Format code
cargo fmt

Contributing

See CONTRIBUTING.md for guidelines.

License

MIT