Crates.io | logos-codegen |
lib.rs | logos-codegen |
version | 0.14.0 |
source | src |
created_at | 2023-04-10 10:30:39.552861 |
updated_at | 2024-02-07 10:43:14.920704 |
description | Create ridiculously fast Lexers |
homepage | https://logos.maciej.codes/ |
repository | https://github.com/maciejhirsz/logos |
max_upload_size | |
id | 834995 |
size | 166,192 |
Create ridiculously fast Lexers.
Logos has two goals:
To achieve those, Logos:
use logos::Logos;
#[derive(Logos, Debug, PartialEq)]
#[logos(skip r"[ \t\n\f]+")] // Ignore this regex pattern between tokens
enum Token {
// Tokens can be literal strings, of any length.
#[token("fast")]
Fast,
#[token(".")]
Period,
// Or regular expressions.
#[regex("[a-zA-Z]+")]
Text,
}
fn main() {
let mut lex = Token::lexer("Create ridiculously fast Lexers.");
assert_eq!(lex.next(), Some(Ok(Token::Text)));
assert_eq!(lex.span(), 0..6);
assert_eq!(lex.slice(), "Create");
assert_eq!(lex.next(), Some(Ok(Token::Text)));
assert_eq!(lex.span(), 7..19);
assert_eq!(lex.slice(), "ridiculously");
assert_eq!(lex.next(), Some(Ok(Token::Fast)));
assert_eq!(lex.span(), 20..24);
assert_eq!(lex.slice(), "fast");
assert_eq!(lex.next(), Some(Ok(Token::Text)));
assert_eq!(lex.slice(), "Lexers");
assert_eq!(lex.span(), 25..31);
assert_eq!(lex.next(), Some(Ok(Token::Period)));
assert_eq!(lex.span(), 31..32);
assert_eq!(lex.slice(), ".");
assert_eq!(lex.next(), None);
}
For more examples and documentation, please refer to the Logos handbook or the crate documentation.
Ridiculously fast!
test identifiers ... bench: 647 ns/iter (+/- 27) = 1204 MB/s
test keywords_operators_and_punctators ... bench: 2,054 ns/iter (+/- 78) = 1037 MB/s
test strings ... bench: 553 ns/iter (+/- 34) = 1575 MB/s
Logos is very much a labor of love. If you find it useful, consider getting me some coffee. ☕
If you'd like to contribute to Logos, then consider reading the Contributing guide.
This code is distributed under the terms of both the MIT license and the Apache License (Version 2.0), choose whatever works for you.
See LICENSE-APACHE and LICENSE-MIT for details.