quicktok
| Crates.io | quicktok |
| lib.rs | quicktok |
| version | 0.2.0 |
| created_at | 2024-02-26 16:21:14.11093+00 |
| updated_at | 2026-01-06 03:18:00.542885+00 |
| description | Minimal, fast, multi-threaded implementation of the Byte Pair Encoding (BPE) for LLM tokenization |
| homepage | |
| repository | |
| max_upload_size | |
| id | 1153928 |
| size | 27,882 |
Krzysztof Woś (krzysztofwos)