Halldyll Memory Model

Multi-user, multi-model distributed memory system for AI agents. Designed for cloud deployment with PostgreSQL + Redis backend.

Overview

A sophisticated memory management system enabling AI agents (like Halldyll) to:

• Store and retrieve conversations, facts, images, and transcriptions
• Support multiple users with complete data isolation and RBAC
• Manage multiple AI models with model-specific memory scoping
• Search semantically using vector embeddings with pgvector
• Cache intelligently with Redis for performance
• Scale horizontally in Kubernetes/RunPods environments

Architecture

┌─────────────────────────────────────────────────────┐
│              MemoryEngine (Orchestrator)            │
├─────────────────────────────────────────────────────┤
│                                                      │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────┐  │
│  │  User Mgmt   │  │ Embedding    │  │ Context  │  │
│  │ (RBAC)       │  │ Generation   │  │ Builder  │  │
│  └──────────────┘  └──────────────┘  └──────────┘  │
│                                                      │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────┐  │
│  │  Processor   │  │  Searcher    │  │ Models   │  │
│  │ (Ingestion)  │  │ (Retrieval)  │  │ Registry │  │
│  └──────────────┘  └──────────────┘  └──────────┘  │
│                                                      │
│  ┌─────────────────────────────────────────────┐   │
│  │     StoragePool (Connection Management)     │   │
│  │  ┌──────────────────┐  ┌──────────────────┐ │   │
│  │  │  PostgreSQL      │  │  Redis           │ │   │
│  │  │  (Data Layer)    │  │  (Cache Layer)   │ │   │
│  │  └──────────────────┘  └──────────────────┘ │   │
│  └─────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────┘

Features

Multi-User Support

• User account management with roles (Admin, User, Service, Viewer)
• Role-based access control (RBAC)
• Complete data isolation between users
• Audit logging for compliance

Memory Types

• Conversations: Full chat history with messages
• Facts: Learned information with confidence scoring
• Images: Generated image metadata and parameters
• Transcriptions: Audio transcription with language detection
• Summaries: Session and context summaries

Vector Search

• pgvector integration for semantic search
• Embedding generation with ONNX Runtime
• Configurable similarity thresholds
• Full-text search fallback

Distributed Architecture

• PostgreSQL for persistent storage
• Redis for caching and session management
• Connection pooling for efficiency
• Health checks and automatic failover

Cloud-Ready

• Docker Compose for local development
• Kubernetes-ready configuration
• RunPods compatible
• Horizontally scalable

Installation

Prerequisites

• Rust 1.70+
• Docker & Docker Compose (for local development)
• PostgreSQL 14+ with pgvector extension
• Redis 6+

Quick Start

1. Clone the repository

git clone https://github.com/Mr-soloDev/Halldyll-Memory-Model
cd Halldyll-Memory-Model

2. Set up environment

cp .env.example .env
docker-compose up -d

3. Run migrations

sqlx migrate run --database-url postgresql://halldyll:halldyll_dev_password@localhost/halldyll_memory

4. Add to your Cargo.toml

[dependencies]
halldyll_memory_model = "0.2"

# Or from Git:
halldyll_memory_model = { git = "https://github.com/Mr-soloDev/Halldyll-Memory-Model" }

Usage

use halldyll_memory_model::{Config, MemoryEngine};
use halldyll_memory_model::models::{Conversation, Message, Role};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Create and initialize engine
    let config = Config::default();
    let engine = MemoryEngine::new(config).await?;
    engine.initialize().await?;

    // Create a conversation
    let mut conversation = Conversation::new("My First Conversation");
    conversation.add_message(Message::new(Role::User, "Hello!"));
    conversation.add_message(Message::new(Role::Assistant, "Hi there!"));

    // Store memory
    let user_id = "user-123";
    engine.store_memory(user_id, Memory::Conversation(conversation)).await?;

    // Search memories
    let results = engine.search(user_id, "hello", 10).await?;
    println!("Found {} memories", results.len());

    Ok(())
}

Configuration

Environment Variables

# PostgreSQL
DATABASE_URL=postgresql://user:password@host:5432/db

# Redis
REDIS_URL=redis://host:6379

# Embeddings
EMBEDDING_MODEL_PATH=./models/all-MiniLM-L6-v2.onnx
EMBEDDING_DIMENSION=384

# Search
SEARCH_TOP_K=10
SIMILARITY_THRESHOLD=0.7

# Cache
CACHE_SIZE=1000

# TTL
EPHEMERAL_FACT_TTL_DAYS=30
MAX_CONTEXT_TOKENS=4000

# Database
DB_POOL_SIZE=5

# Logging
RUST_LOG=info,halldyll_memory_model=debug

Module Structure

src/
├── core/              # Fundamental types and errors
│   ├── config.rs      # Configuration
│   ├── error.rs       # Error types
│   └── id.rs          # ID generation
├── models/            # Data models
│   ├── conversation.rs
│   ├── fact.rs
│   ├── image.rs
│   ├── memory.rs
│   ├── transcription.rs
│   └── user_model.rs
├── user/              # User management
│   ├── manager.rs
│   └── permissions.rs
├── storage/           # Persistence layer
│   ├── pool.rs        # Connection pooling
│   ├── postgres.rs    # PostgreSQL operations
│   └── redis.rs       # Redis caching
├── embedding/         # Vector embeddings
├── ingest/            # Data ingestion
├── retrieval/         # Memory search
├── context/           # Prompt context building
├── engine/            # Main orchestrator
└── utils/             # Utility functions

Development

Run Tests

# Unit tests
cargo test --lib

# Integration tests (requires database)
cargo test --test '*' -- --test-threads=1

# All tests with logging
RUST_LOG=debug cargo test -- --nocapture

Build

# Debug
cargo build

# Release (optimized)
cargo build --release

Code Quality

# Format
cargo fmt

# Lint
cargo clippy -- -D warnings

# Check
cargo check

# Full validation
cargo check && cargo fmt && cargo clippy && cargo test

Generate Documentation

cargo doc --no-deps --open

Deployment

Docker

docker build -t halldyll-memory:latest .
docker run -e DATABASE_URL=postgresql://... \
           -e REDIS_URL=redis://... \
           halldyll-memory:latest

Kubernetes

apiVersion: apps/v1
kind: Deployment
metadata:
  name: halldyll-memory
spec:
  replicas: 2
  selector:
    matchLabels:
      app: halldyll-memory
  template:
    metadata:
      labels:
        app: halldyll-memory
    spec:
      containers:
      - name: memory
        image: halldyll-memory:latest
        env:
        - name: DATABASE_URL
          valueFrom:
            secretKeyRef:
              name: db-credentials
              key: url
        - name: REDIS_URL
          valueFrom:
            secretKeyRef:
              name: redis-credentials
              key: url

RunPods

Compatible with RunPods serverless and pod deployments. Ensure PostgreSQL and Redis are accessible.

API Overview

Memory Storage

pub async fn store_memory(
    &self,
    user_id: &str,
    memory: Memory,
) -> MemoryResult<()>

Search

pub async fn search(
    &self,
    user_id: &str,
    query: &str,
    limit: usize,
) -> MemoryResult<Vec<Memory>>

User Management

pub async fn create_user(
    &self,
    username: String,
    email: String,
    role: UserRole,
) -> MemoryResult<UserModel>

Performance

• Vector search: < 100ms for top-10 on 1M vectors (with proper indexing)
• Fact storage: Non-blocking, fully asynchronous
• Embedding caching: Configurable LRU cache
• Database: Connection pooling with configurable pool size

Security

• User Isolation: Complete data separation per user
• RBAC: Role-based access control with permission checking
• Audit Logging: All operations logged for compliance
• No SQL Injection: Uses parameterized queries (sqlx)
• Validation: Input validation for all user data

Testing Models

Supported model specialties:

• Qwen2.5-14B-Instruct: LLM for text generation
• Qwen2.5-VL-7B-Instruct: Vision-language for image understanding
• RealVisXL_V5.0: Ultra-realistic image generation
• ControlNet/InstantID: Image generation control and identity preservation
• InsightFace: Facial recognition

Roadmap

• GraphQL API layer
• WebSocket support for real-time memory updates
• Advanced fact extraction with LLM
• Multi-language embedding support
• Hierarchical memory compression
• Import/Export functionality
• Web dashboard for memory visualization
• Model-specific memory scoping
• Distributed tracing with OTEL

Contributing

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Commit changes (git commit -m 'Add amazing feature')
4. Push to branch (git push origin feature/amazing-feature)
5. Open a Pull Request

License

MIT License - see LICENSE file for details

Author

Roy Géryan - Halldyll Creator

Support

For issues, questions, or suggestions:

• GitHub Issues: https://github.com/Mr-soloDev/Halldyll-Memory-Model/issues
• Discussions: https://github.com/Mr-soloDev/Halldyll-Memory-Model/discussions
• Email: geryan.roy@icloud.com

Built with ❤️ for Halldyll and the AI community