parlex

Core support library for lexers and parsers generated by parlex-gen.

Overview

Parlex is a suite of Rust-based tools for the generation of efficient lexical analyzers and parsers. The project comprises two complementary crates: parlex, the core support library, and parlex-gen, which provides the ALEX lexer generator and the ASLR parser generator. Together, these components form a cohesive and extensible framework for language parsing, analysis, and compiler front-end development.

The system is inspired by the classic lex (flex) and yacc (bison) utilities written for C, but provides a Rust-based implementation that is more composable and improves upon ambiguity resolution. Unlike lex and yacc, which interleave user-defined code with automatically generated code, Parlex maintains a strict separation between specification and implementation: grammar rules and lexer definitions are explicitly named, and user code refers to them symbolically.

The ALEX lexer generator offers expressive power comparable to that of lex or flex. It employs Rust’s standard regular expression libraries to construct deterministic finite automata (DFAs) that efficiently recognize lexical patterns at runtime. ALEX supports multiple lexical states, enabling precise and context-sensitive tokenization.

The ASLR parser generator implements the SLR(1) parsing algorithm, which is somewhat less general than the LALR(1) approach used by yacc and bison. However, ASLR introduces an important improvement: it supports dynamic runtime resolution of shift/reduce ambiguities, providing greater flexibility in languages such as Prolog, where operator definitions may be introduced or redefined dynamically.

The parlex crate serves as the core runtime and support library for the generated lexers and parsers. It defines the traits, data structures, and runtime abstractions that underpin the generated code, ensuring consistent behavior and interoperability across user-defined grammars. All lexers and parsers produced by the parlex-gen tools depend on this crate, and users extend its interfaces to build custom language processors based on their generated components.

This crate is typically used in conjunction with code generated by parlex-gen. See the parlex-gen documentation for information on generating lexers and parsers.

License

Copyright (c) 2005–2025 IKH Software, Inc.

Released under the terms of the GNU Lesser General Public License, version 3.0 or (at your option) any later version (LGPL-3.0-or-later).

See Also

• parlex-gen - Code generation tools ALEX and ASLR
• arena-terms-parser — real-world example using ALEX and ASLR