🦅 Aquila

Your personal flying courier

A modular asset server with support for OAuth/CDNs/presigned URLs and multiple file backends, meant for serving game assets but could probably be used for other things too.

[!CAUTION] This package is in early development!

What is this for?

During game development a way to serve assets remotely is often desired. This can be either to fetch at build-time in a build environment or to serve them to your users at runtime, leading to complex setups involving git LFS or Perforce and build servers or worse - manual swapping of files.

This crate aims at simplifying this process by providing a simple server, a client and a cli that can be used to serve versioned assets. At the moment, it supports:

• Serve assets to your game clients (through presigned URLs or a CDN if you want to)
• Publish assets and manifests to a server
• Streaming uploads for large files
• Minting (read-only public) tokens
• Authenticate users (custom or OAuth, see aquila_auth_mock and aquila_auth_github)

Security Notice

This crate is in early development and should not be used in production yet. You are responsible for making sure your assets are safe and secure. If you ship public read-only tokens to users, make sure you are aware of what that entails, e.g., how to invalidate and ship new ones in the case of abuse.

[!IMPORTANT] Make sure you vet any auth providers and OAuth applications and its permissions that you intend to use thoroughly before using them in production.

Ecosystem

The workspace is composed of modular crates, allowing you to pick and choose the components you need.

Core & Integration

Storage Backends

Authentication

Feature Flags

Examples

Simple server

cargo run --example simple_server --features "server fs mock_auth"

Simple client

Simple client (will publish v1.0 manifest and test.png)

cargo run --examples simple_client --features "client"

Bevy

Bevy example (uses latest manifest and test.png)

cargo run --example bevy

Custom Server

[dependencies]
aquila = { version = "0.6", features = ["server", "fs", "mock_auth"] }

use aquila::prelude::*;

#[tokio::main]
async fn main() {
    let storage = FileSystemStorage::new("./assets");
    let auth = AllowAllAuth;

    // Build
    let app = AquilaServer::default().build(storage, auth);

    // Serve
    let listener = tokio::net::TcpListener::bind("0.0.0.0:3000").await.unwrap();
    axum::serve(listener, app).await.unwrap();
}

The rest of the examples use the CLI

[!TIP] While not required, it's recommended to install the CLI to make usage easier.

Install cli

crates.io:

cargo install aquila_cli

From source:

cargo install --path crates/aquila_cli

AWS S3

You need to set the AWS_REGION, AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY env vars and/or use the AWS cli (aws configure).

Set the bucket name

set S3_BUCKET=...

Run the server

cargo run --example s3_server --features "server s3 mock_auth"

Publish v1.0 manifest and test.png

aquila publish ./assets --version "v1.0"

Streaming

aquila publish ./assets --version "v1.0" --stream

Updating/Publishing old manifests

aquila publish ./assets --version "v0.1" --no-latest

Short args are supported, see --help or -h

aquila publish ./assets -v "v1.0" -s -n

Bevy example (uses latest manifest and test.png)

cargo run --example bevy

GitHub auth and JWT Minting (for read-only tokens)

Generate & set JWT secret:

You can use the CLI to generate a secret or provide your own:

aquila generate-secret
set AQUILA_SECRET=...

Create a GitHub OAuth app

The routes are configurable, you're going to have to make sure the callback route matches (in this case /auth/callback).

Set the client id and secret env vars

SET GITHUB_CLIENT_ID=...
SET GITHUB_CLIENT_SECRET=...

Run the server

cargo run --example github_auth_server --features "server fs github_auth"

You should now be able to log in using a second terminal:

aquila login

Now set the token that you get after you've been redirected back to the callback route:

set AQUILA_TOKEN=...

You should have full access now! To mint a read-only public token:

aquila mint-token  "release-build-v1.0"

To publish all assets and a manifest:

aquila publish ./assets --version "v1.0"

Bevy example (uses latest manifest and test.png)

cargo run --example bevy

CLI commands

single file test

aquila upload ./assets/test.png

publish manifest and assets

aquila publish ./assets --version v1.0`

[!TIP] Upload/publish commands support streaming with --stream.

Bevy

As shown above in the other examples, after publishing a manifest version, you can use the assets in bevy:

cargo run --example bevy

Server curl tests

test manually

upload

curl -X POST --data-binary @./assets/test.png http://localhost:3000/assets

fetch

curl http://localhost:3000/assets/{hash} --output test_down.png

Notes

Using generics to be able to use native async traits and avoiding dyn + async_trait or Box etc. I'd be willing to revisit this though if there's a better alternative.

TODO

• add some tests
• docker images, nix flakes (a simple server example should be enough)
• meta file support and other bevy asset reader functionality (folders)
• multiple scopes, not just read/write/admin
• I experimented with a VCSProvider trait to verify the version of the manifest against the VCS, but decided against it for now, but it definitely could be useful.

License

Dual-licensed:

• MIT License (LICENSE-MIT or http://opensource.org/licenses/MIT)
• Apache License, Version 2.0 (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)

License: MIT OR Apache-2.0