wasi-nn bindings for Rust, with LLM extensions

This package is inspired by the wasi-nn package, but with additional support for the LLM.

This package contains high-level Rust bindings for wasi-nn system calls. It is similar in purpose to the WASI bindings but this package provides optional access to a system's machine learning functionality from WebAssembly.

NOTE: These bindings are experimental (use at your own risk) and subject to upstream changes in the wasi-nn specification.

Use

1. Depend on this crate in your Cargo.toml:

   [dependencies]
   wasmedge-wasi-nn = "0.7.1"
   

2. Use the wasi-nn APIs in your application, for example:

   use wasi_nn;
   let graph = GraphBuilder::new(GraphEncoding::TensorflowLite, ExecutionTarget::CPU)
       .build_from_files([model_path])?;
   let mut ctx = graph.init_execution_context()?;
   ctx.set_input(0, TensorType::F32, &input_dims, &input_buffer)?;
   ctx.compute()?;
   let output_num_bytes = ctx.get_output(0, &mut output_buffer)?;
   

3. Compile the application to WebAssembly:

   cargo build --target=wasm32-wasi
   

4. Run the generated WebAssembly in a runtime supporting wasi-nn, e.g., Wasmtime, WasmEdge.

Build

To build this crate from source, use: cargo build --target wasm32-wasi.

Generation

This crate contains code (src/generated.rs) generated by witx-bindgen. To regenerate this code, run the following script:

$ scripts/regenerate-bindings-from-witx.sh

Examples

The examples demonstrate how to use wasi-nn from a Rust program.

License

This project is licensed under the Apache 2.0 license. See LICENSE for more details.

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in this project by you, as defined in the Apache-2.0 license, shall be licensed as above, without any additional terms or conditions.