rten-generate
| Crates.io | rten-generate |
| lib.rs | rten-generate |
| version | 0.24.0 |
| created_at | 2024-07-05 21:09:36.400636+00 |
| updated_at | 2025-12-23 15:00:31.823081+00 |
| description | Utilities to simplify running auto-regressive models with RTen |
| homepage | https://github.com/robertknight/rten |
| repository | https://github.com/robertknight/rten |
| max_upload_size | |
| id | 1293359 |
| size | 117,989 |
Robert Knight (robertknight)
documentation
README
rten-generate is a layer on top of RTen which handles the generation loop for auto-regressive transformer models (aka. "transformer decoders" or "generative AI"). This includes managing the KV cache, sampling and post-processing logits etc.