quicktok
| Crates.io | quicktok |
| lib.rs | quicktok |
| version | 0.1.1 |
| created_at | 2024-02-26 16:21:14.11093+00 |
| updated_at | 2024-02-26 16:21:14.11093+00 |
| description | Minimal, fast, multi-threaded implementation of the Byte Pair Encoding (BPE) for LLM tokenization |
| homepage | |
| repository | |
| max_upload_size | |
| id | 1153928 |
| size | 16,648 |
Krzysztof Woś (krzysztofwos)