logos-derive2

Crates.iologos-derive2
lib.rslogos-derive2
version0.14.0
sourcesrc
created_at2024-02-06 15:44:43.312231
updated_at2024-02-07 09:38:45.19547
descriptionCreate ridiculously fast Lexers
homepagehttps://logos.maciej.codes/
repositoryhttps://github.com/maciejhirsz/logos
max_upload_size
id1129049
size17,766
Jérome Eertmans (jeertmans)

documentation

README

Logos logo

Warning: as of v0.14, Logos is released under logos2, see #365. However, the library name is still logos, so you should only change your dependencies in Cargo.toml to use the latest versions.

Logos

Test Crates.io version shield Docs Crates.io license shield

Create ridiculously fast Lexers.

Logos has two goals:

  • To make it easy to create a Lexer, so you can focus on more complex problems.
  • To make the generated Lexer faster than anything you'd write by hand.

To achieve those, Logos:

Example

 use logos::Logos;

 #[derive(Logos, Debug, PartialEq)]
 #[logos(skip r"[ \t\n\f]+")] // Ignore this regex pattern between tokens
 enum Token {
     // Tokens can be literal strings, of any length.
     #[token("fast")]
     Fast,

     #[token(".")]
     Period,

     // Or regular expressions.
     #[regex("[a-zA-Z]+")]
     Text,
 }

 fn main() {
     let mut lex = Token::lexer("Create ridiculously fast Lexers.");

     assert_eq!(lex.next(), Some(Ok(Token::Text)));
     assert_eq!(lex.span(), 0..6);
     assert_eq!(lex.slice(), "Create");

     assert_eq!(lex.next(), Some(Ok(Token::Text)));
     assert_eq!(lex.span(), 7..19);
     assert_eq!(lex.slice(), "ridiculously");

     assert_eq!(lex.next(), Some(Ok(Token::Fast)));
     assert_eq!(lex.span(), 20..24);
     assert_eq!(lex.slice(), "fast");

     assert_eq!(lex.next(), Some(Ok(Token::Text)));
     assert_eq!(lex.slice(), "Lexers");
     assert_eq!(lex.span(), 25..31);

     assert_eq!(lex.next(), Some(Ok(Token::Period)));
     assert_eq!(lex.span(), 31..32);
     assert_eq!(lex.slice(), ".");

     assert_eq!(lex.next(), None);
 }

For more examples and documentation, please refer to the Logos handbook or the crate documentation.

How fast?

Ridiculously fast!

test identifiers                       ... bench:         647 ns/iter (+/- 27) = 1204 MB/s
test keywords_operators_and_punctators ... bench:       2,054 ns/iter (+/- 78) = 1037 MB/s
test strings                           ... bench:         553 ns/iter (+/- 34) = 1575 MB/s

Acknowledgements

Thank you

Logos is very much a labor of love. If you find it useful, consider getting me some coffee. ☕

If you'd like to contribute to Logos, then consider reading the Contributing guide.

License

This code is distributed under the terms of both the MIT license and the Apache License (Version 2.0), choose whatever works for you.

See LICENSE-APACHE and LICENSE-MIT for details.

Commit count: 194

cargo fmt