usls

usls

usls is a cross-platform Rust library powered by ONNX Runtime for efficient inference of SOTA vision and multi-modal models(typically under 1B parameters).

📚 Documentation

• API Documentation

• Examples

🚀 Quick Start

# CPU
cargo run -r --example yolo -- --task detect --ver 8 --scale n --dtype fp16  # q8, q4, q4f16

# NVIDIA CUDA
cargo run -r -F cuda --example yolo -- --device cuda:0   # YOLOv8-n detect by default

# NVIDIA TensorRT
cargo run -r -F tensorrt --example yolo -- --device tensorrt:0

# Apple Silicon CoreML
cargo run -r -F coreml --example yolo -- --device coreml

# Intel OpenVINO
cargo run -r -F openvino -F ort-load-dynamic --example yolo -- --device openvino:CPU

# And other EPs...

⚙️ Installation

Add the following to your Cargo.toml:

[dependencies]
# Recommended: Use GitHub version
usls = { git = "https://github.com/jamjamjon/usls", features = [ "cuda" ] }

# Alternative: Use crates.io version
usls = "latest-version"

⚡ Supported Models

📦 Cargo Features

• ort-download-binaries (default): Automatically downloads prebuilt ONNXRuntime binaries for supported platforms

• ort-load-dynamic: Dynamic linking to ONNXRuntime libraries (Guide)

• video: Enable video stream reading and writing (via video-rs and minifb)

• cuda: NVIDIA CUDA GPU acceleration support

• tensorrt: NVIDIA TensorRT optimization for inference acceleration

• coreml: Apple CoreML acceleration for macOS/iOS devices

• openvino: Intel OpenVINO toolkit for CPU/GPU/VPU acceleration

• onednn: Intel oneDNN (formerly MKL-DNN) for CPU optimization

• directml: Microsoft DirectML for Windows GPU acceleration

• xnnpack: Google XNNPACK for mobile and edge device optimization

• rocm: AMD ROCm platform for GPU acceleration

• cann: Huawei CANN (Compute Architecture for Neural Networks) support

• rknpu: Rockchip NPU acceleration

• acl: Arm Compute Library for Arm processors

• nnapi: Android Neural Networks API support

• armnn: Arm NN inference engine

• tvm: Apache TVM tensor compiler stack

• qnn: Qualcomm Neural Network SDK

• migraphx: AMD MIGraphX for GPU acceleration

• vitis: Xilinx Vitis AI for FPGA acceleration

• azure: Azure Machine Learning integration

❓ FAQ

See issues or open a new discussion.

🤝 Contributing

Contributions are welcome! If you have suggestions, bug reports, or want to add new features or models, feel free to open an issue or submit a pull request.

📜 License

This project is licensed under LICENSE.