# wasi-nn bindings for Rust, with LLM extensions

This package is inspired by the [wasi-nn] package, but with additional support for the LLM.

---

This package contains high-level Rust bindings for [wasi-nn] system calls. It is similar in purpose
to the [WASI bindings] but this package provides optional access to a system's machine learning
functionality from WebAssembly.

[wasi-nn]: https://github.com/WebAssembly/wasi-nn
[WASI bindings]: https://github.com/bytecodealliance/wasi

> __NOTE__: These bindings are experimental (use at your own risk) and subject to upstream changes
> in the wasi-nn specification.


### Use

1. Depend on this crate in your `Cargo.toml`:
    ```toml
    [dependencies]
    wasmedge-wasi-nn = "0.7.1"
    ```

2. Use the wasi-nn APIs in your application, for example:
    ```rust
    use wasi_nn;
    let graph = GraphBuilder::new(GraphEncoding::TensorflowLite, ExecutionTarget::CPU)
        .build_from_files([model_path])?;
    let mut ctx = graph.init_execution_context()?;
    ctx.set_input(0, TensorType::F32, &input_dims, &input_buffer)?;
    ctx.compute()?;
    let output_num_bytes = ctx.get_output(0, &mut output_buffer)?;
    ```

3. Compile the application to WebAssembly:
    ```shell script
    cargo build --target=wasm32-wasi
    ```

4. Run the generated WebAssembly in a runtime supporting [wasi-nn], e.g., [Wasmtime], [WasmEdge].

[Wasmtime]: https://wasmtime.dev
[WasmEdge]: https://wasmedge.org/


### Build

To build this crate from source, use: `cargo build --target wasm32-wasi`.


### Generation

This crate contains code ([`src/generated.rs`](src/generated.rs)) generated by
[`witx-bindgen`](https://github.com/bytecodealliance/wasi/tree/main/crates/witx-bindgen). To
regenerate this code, run the following script:

```console
$ scripts/regenerate-bindings-from-witx.sh
```


### Examples

The [examples] demonstrate how to use wasi-nn from a Rust program.

[examples]: examples


### License

This project is licensed under the Apache 2.0 license. See [LICENSE] for more details.

[LICENSE]: ../LICENSE


### Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in
this project by you, as defined in the Apache-2.0 license, shall be licensed as above, without any
additional terms or conditions.