Crates.io | plrs |
lib.rs | plrs |
version | 0.1.3 |
source | src |
created_at | 2021-09-10 22:00:45.04518 |
updated_at | 2021-09-10 22:00:45.04518 |
description | The multi-tool of lexical analysis and tokenization. |
homepage | |
repository | https://github.com/JakeRoggenbuck/plrs |
max_upload_size | |
id | 449494 |
size | 13,390 |
The multi-tool of lexical analysis and tokenization.
pip install plrs
maturin build
EOF_TOKEN
Tokens
Settings
Token
- part
- token
- set_part
- set_token
- __str__
- __repr__
Lexer
- new
- char_forward
- skip_over_char_set
- next
is_char_symbol
is_char_operator
is_char_whitespace
is_char_numeric
is_single_quote
is_double_quote
ends_token
is_part_numeric
tokenize