pcapsql-core

Engine-agnostic PCAP protocol parsing library.

This crate provides the core parsing functionality for pcapsql, without any SQL engine dependencies. It can be used standalone for protocol analysis or as the foundation for SQL integrations (DataFusion, DuckDB).

Features

• Protocol Parsing: 17+ built-in protocol parsers
• PCAP Reading: PCAP and PCAPNG formats with gzip/zstd compression
• Memory-Mapped I/O: Efficient reading of large capture files
• Parse Caching: LRU cache to avoid redundant parsing during JOINs
• TCP Stream Reassembly: Connection tracking and application-layer parsing
• TLS Decryption: Support for TLS 1.2/1.3 with SSLKEYLOGFILE

Quick Start

use pcapsql_core::prelude::*;
use pcapsql_core::io::FilePacketSource;

// Create a protocol registry with all built-in parsers
let registry = default_registry();

// Open a PCAP file
let source = FilePacketSource::open("capture.pcap").unwrap();
let mut reader = source.reader(None).unwrap();

// Read and parse packets
reader.process_packets(1000, |packet| {
    let results = pcapsql_core::parse_packet(
        &registry,
        packet.link_type as u16,
        &packet.data,
    );

    for (protocol_name, result) in results {
        println!("{}: {} fields", protocol_name, result.fields.len());
    }
    Ok(())
}).unwrap();

Supported Protocols

Crate Features

Architecture

pcapsql-core
├── schema/     - FieldDescriptor, DataKind (engine-agnostic types)
├── protocol/   - Protocol trait, parsers, FieldValue
├── io/         - PacketSource, PacketReader, mmap support
├── pcap/       - PCAP/PCAPNG reading, compression
├── cache/      - LRU parse cache
├── stream/     - TCP reassembly, HTTP/TLS stream parsing
├── tls/        - TLS key derivation and decryption
└── format/     - Address formatting utilities

License

MIT