rusty-ggml
| Crates.io | rusty-ggml |
| lib.rs | rusty-ggml |
| version | 0.0.8 |
| source | src |
| created_at | 2023-04-29 13:50:12.860946 |
| updated_at | 2023-09-02 19:28:08.210021 |
| description | Idiomatic bindings for the GGML library (pre-alpha) |
| homepage | |
| repository | https://github.com/KerfuffleV2/rusty-ggml |
| max_upload_size | |
| id | 852082 |
| size | 98,594 |
Kerfuffle (KerfuffleV2)
documentation
README
rusty-ggml
GGML bindings that aim to be idiomatic Rust rather than directly corresponding to the C/C++ interface.
GG-what?
See:
- https://github.com/ggerganov/ggml/
- https://github.com/KerfuffleV2/ggml-sys-bleedingedge ( https://crates.io/crates/ggml-sys-bleedingedge )
WIP
Not suitable for general use. Consider this to be pre-alpha code.
v0.0.8 Warning: Keeping this in sync with recent GGML changes has lagged. It compiles and seems to work but there might be weird stuff I haven't caught.
Note: There are special considerations when using GPU features like cublas, hipblas. See the ggml-sys-bleedingedge repo or crate documentation for more information
Example usage: https://github.com/KerfuffleV2/smolrsrwkv/blob/189915ec68b28d057b440f75803d3d056e150733/smolrwkv/src/ggml/graph.rs
Related
For your token sampling needs see https://github.com/KerfuffleV2/llm-samplers ( https://crates.io/crates/llm-samplers )