incpa

incpa is an incremental parser composition crate.

Incremental parsers process a chunk of input, then either produce an error, a parsed output, or an updated parser state ready for future input. This primitive, codified by ParserState::feed, allows the same parser definition to support parsing streaming input from async or sync sources, as well as other "incremental" use cases such as interactive REPL loop parsing.

The term "parser composition" emphasizes how sophisticated parsers can be defined by composing simpler parsers.

Related Crates

The incpa project functionality is separated into multiple distinct crates:

• The incpa-byte crate provides byte-wise input functionality, such as UTF8 decoding (so that any string parser can read byte-oriented sources).
• Support for async input streams is provided in the downstream incpa-tokio crate.

Example

use incpa::BaseParserError;
use incpa::primitive::remaining;
use incpa::Parser;

fn main() -> Result<(), BaseParserError> {
    let parser = define_my_parser();
    let output = parser.parse_all("Hello World!")?;
    assert_eq!(output, ("Hello", " World!".to_string()));
    Ok(())
}

fn define_my_parser() -> impl Parser<str, Output=(&'static str, String), Error=BaseParserError> {
    "Hello".then(remaining())
}

Trade-offs

There is a fundamental trade-off between streaming parsers, like incpa-based parsers, versus "zero-copy" parsers which parse values which refer back to the original input buffer.

Zero-copy parsers reduce the memory footprint and amount of copying at the cost of requiring all input to be held in memory, whereas streaming parsers can parse very large inputs at the cost of internally copying input where necessary.

Related Projects

This crate is inspired by chumsky which is an excellent and mature parser composition crate. Another inspiration is parsec in haskell-land.

Status

This crate is in the version 0.0.x phase of early proof of concept with unstable APIs.

Roadmap

Changelog