cargo-green

Crates.iocargo-green
lib.rscargo-green
version0.22.0
created_at2024-05-11 10:27:16.385342+00
updated_at2025-11-14 20:22:18.963547+00
descriptionCargo plugin and $RUSTC_WRAPPER to sandbox & cache cargo builds and execute jobs remotely
homepage
repositoryhttps://github.com/fenollp/supergreen
max_upload_size
id1236783
size412,278
Pierre Fenoll (fenollp)

documentation

README

supergreen ~ Faster Rust builds!

cargo-green: a cached & remote-ready Rust projects builder.

cargo-green is

  • a cargo plugin that sets a $RUSTC_WRAPPER then calls cargo.

  • a RUSTC_WRAPPER that builds Dockerfiles

  • by forwarding rustc calls to BuildKit builders

A rusty crab character named Ferris, featuring a unique hairstyle resembling 'Ruby Road', a vibrant and textured hairdo often seen in flamboyant red

Installation

cargo install cargo-green
cargo install --locked --force --git https://github.com/fenollp/supergreen.git cargo-green

# Make sure $CARGO_HOME/bin is in your $PATH
which cargo-green

When building locally, ensure either a docker or podman client is installed.

Minimum requirements:

  • Ubuntu 22.04
  • buildkit 0.12.0
  • github.com/docker/buildx v0.11.2
  • rust 1.73

Usage

# Usage:
  cargo green supergreen env [ENV ...]               Show used values
  cargo green supergreen doc [ENV ...]               Documentation of said values
  cargo green fetch                                  Pulls images
  cargo green supergreen push                        Push cache image (all tags)
  cargo green supergreen builder [ recreate | rm ]   Manage local/remote builder(s)
  cargo green supergreen -h | --help
  cargo green supergreen -V | --version
  cargo green ...any cargo subcommand...

# Try:
  cargo clean # Start from a clean slate
  cargo green build
  cargo supergreen env CARGOGREEN_BASE_IMAGE 2>/dev/null
  cargo supergreen help

# Suggestion:
  alias cargo='cargo green'
  # Now try, within your project:
  cargo fetch
  cargo test

Remote execution

Say you have a bigger machine in your ~/.ssh/config called extra-oomph:

DOCKER_HOST=ssh://extra-oomph cargo green test

This will compile the tests on the remote machine and run then locally. TODO: also run tests remotely, like cross does: https://github.com/cross-rs/cross/blob/49cd054de9b832dfc11a4895c72b0aef533b5c6a/README.md#supported-targets

For more knobs to tune, see also:

Caching

Share your build cache with your team and CI, never feel cold starts!

[package.metadata.green]
cache-images = [ "docker-image://my.org/team/my-project", "docker-image://ghcr.io/me/fork" ]
cache-from-images = [ "docker-image://some.org/global/cache" ]

Configuration

Tune the behavior of cargo-green either through environment variables or via the package's green metadata section.

[package]
name = "my-crate"
# ...

[package.metadata.green]
registry-mirrors = [ "mirror.gcr.io", "public.ecr.aws/docker" ] # Default values

Environment variables that are prefixed with $CARGOGREEN_ override TOML settings.

export CARGOGREEN_REGISTRY_MIRRORS=mirror.gcr.io
cargo green build

$CARGOGREEN_LOG_PATH

Path to a text file to write logs.

Use by setting this environment variable (no Cargo.toml setting):

export CARGOGREEN_LOG_PATH="/tmp/my-logs.txt"
# This needs to be set: nothing is logged by default
export CARGOGREEN_LOG="info"

$CARGOGREEN_LOG

Filter logs. Equivalent to $RUST_LOG (and doesn't conflict with cargo's).

By default, writes logs under /tmp.

See https://docs.rs/env_logger/#enabling-logging

Use by setting this environment variable (no Cargo.toml setting):

export CARGOGREEN_LOG="trace,cargo_green::build=info"

$CARGOGREEN_LOG_STYLE

Style logs. Equivalent to $RUST_LOG_STYLE.

See https://docs.rs/env_logger/#disabling-colors

Use by setting this environment variable (no Cargo.toml setting):

export CARGOGREEN_LOG_STYLE="never"

$CARGOGREEN_RUNNER

Pick which executor to use: "docker" (default), "podman" or "none".

The runner gets forwarded these environment variables:

  • $BUILDKIT_COLORS
  • $BUILDKIT_HOST
  • $BUILDKIT_PROGRESS
  • $BUILDKIT_TTY_LOG_LINES
  • $BUILDX_BAKE_GIT_AUTH_HEADER
  • $BUILDX_BAKE_GIT_AUTH_TOKEN
  • $BUILDX_BAKE_GIT_SSH
  • $BUILDX_BUILDER
  • $DOCKER_BUILDKIT
  • $BUILDX_CONFIG
  • $BUILDX_CPU_PROFILE
  • $BUILDX_EXPERIMENTAL
  • $BUILDX_GIT_CHECK_DIRTY
  • $BUILDX_GIT_INFO
  • $BUILDX_GIT_LABELS
  • $BUILDX_MEM_PROFILE
  • $BUILDX_METADATA_PROVENANCE
  • $BUILDX_METADATA_WARNINGS
  • $BUILDX_NO_DEFAULT_ATTESTATIONS
  • $BUILDX_NO_DEFAULT_LOAD
  • $DOCKER_API_VERSION
  • $DOCKER_CERT_PATH
  • $DOCKER_CONFIG
  • $DOCKER_CONTENT_TRUST
  • $DOCKER_CONTENT_TRUST_SERVER
  • $DOCKER_CONTEXT
  • $DOCKER_DEFAULT_PLATFORM
  • $DOCKER_HIDE_LEGACY_COMMANDS
  • $DOCKER_HOST
  • $DOCKER_TLS
  • $DOCKER_TLS_VERIFY
  • $EXPERIMENTAL_BUILDKIT_SOURCE_POLICY
  • $HTTP_PROXY
  • $HTTPS_PROXY
  • $NO_PROXY

Use by setting this environment variable (no Cargo.toml setting):

export CARGOGREEN_RUNNER="docker"

$BUILDX_BUILDER

Sets which BuildKit builder to use, through $BUILDX_BUILDER.

See https://docs.docker.com/build/building/variables/#buildx_builder

  • Unset: creates & handles a builder named "supergreen". Upgrades it if too old, while trying to keep old cached data
  • Set to "": skips using a builder
  • Set to "supergreen": uses existing and just warns if too old
  • Set: use that as builder, no questions asked

See also

$CARGOGREEN_BUILDER_IMAGE

Sets which BuildKit builder version to use.

See https://docs.docker.com/build/builders/

Use by setting this environment variable (no Cargo.toml setting):

export CARGOGREEN_BUILDER_IMAGE="docker-image://docker.io/moby/buildkit:latest"

$CARGOGREEN_SYNTAX_IMAGE

Sets which BuildKit frontend syntax to use.

See https://docs.docker.com/build/buildkit/frontend/#stable-channel

Use by setting this environment variable (no Cargo.toml setting):

export CARGOGREEN_SYNTAX_IMAGE="docker-image://docker.io/docker/dockerfile:1"

$CARGOGREEN_REGISTRY_MIRRORS

Registry mirrors for Docker Hub (docker.io). Defaults to GCP & AWS mirrors.

See https://docs.docker.com/build/buildkit/configure/#registry-mirror

Namely hosts with maybe a port and a path:

  • dockerhub.timeweb.cloud
  • dockerhub1.beget.com
  • localhost:5000
  • mirror.gcr.io
  • public.ecr.aws/docker
registry-mirrors = [ "mirror.gcr.io", "public.ecr.aws/docker" ]

This environment variable takes precedence over any Cargo.toml settings:

# Note: values here are comma-separated.
export CARGOGREEN_REGISTRY_MIRRORS="mirror.gcr.io,public.ecr.aws/docker"

$CARGOGREEN_CACHE_IMAGES

Both read and write cached data to and from image registries

Exactly a combination of [Green::cache_from_images] and [Green::cache_to_images].

See

cache-images = [ "docker-image://my.org/team/my-project", "docker-image://some.org/global/cache" ]

This environment variable takes precedence over any Cargo.toml settings:

# Note: values here are comma-separated.
export CARGOGREEN_CACHE_IMAGES="docker-image://my.org/team/my-project,docker-image://some.org/global/cache"

$CARGOGREEN_CACHE_FROM_IMAGES

Read cached data from image registries

See also [Green::cache_images] and [Green::cache_to_images].

cache-from-images = [ "docker-image://my.org/team/my-project-in-ci", "docker-image://some.org/global/cache" ]

This environment variable takes precedence over any Cargo.toml settings:

# Note: values here are comma-separated.
export CARGOGREEN_CACHE_FROM_IMAGES="docker-image://my.org/team/my-project-in-ci,docker-image://some.org/global/cache"

$CARGOGREEN_CACHE_TO_IMAGES

Write cached data to image registries

Note that errors caused by failed cache exports are not ignored.

See also [Green::cache_images] and [Green::cache_from_images].

cache-to-images = [ "docker-image://my.org/team/my-fork" ]

This environment variable takes precedence over any Cargo.toml settings:

# Note: values here are comma-separated.
export CARGOGREEN_CACHE_TO_IMAGES="docker-image://my.org/team/my-fork"

$CARGOGREEN_FINAL_PATH

Write final containerfile to given path.

Helps e.g. create a containerfile of e.g. a binary to use for best caching of dependencies.

Use by setting this environment variable (no Cargo.toml setting):

export CARGOGREEN_FINAL_PATH="$PWD/my-bin@1.0.0.Dockerfile"

$CARGOGREEN_BASE_IMAGE

Sets the base Rust image, as an image URL (or any build context, actually).

If needing additional envs to be passed to rustc or build script, set them in the base image.

This can be done in that same config file with base-image-inline.

See also:

  • also-run
  • base-image-inline
  • additional-build-arguments

For remote builds: make sure this is accessible non-locally.

base-image = "docker-image://docker.io/library/rust:1-slim"

The value must start with docker-image:// and image must be available on the $DOCKER_HOST, eg:

export CARGOGREEN_BASE_IMAGE=docker-image://rustc_with_libs
DOCKER_HOST=ssh://my-remote-builder docker buildx build -t rustc_with_libs - <<EOF
FROM docker.io/library/rust:1.69.0-slim-bookworm@sha256:8bdd28ef184d85c9b4932586af6280732780e806e5f452065420f2e783323ca3
RUN set -eux && apt update && apt install -y libpq-dev libssl3
ENV KEY=value
EOF

This environment variable takes precedence over any Cargo.toml settings:

export CARGOGREEN_BASE_IMAGE="docker-image://docker.io/library/rust:1-slim"

$CARGOGREEN_SET_ENVS

Pass environment variables through to build runner.

May be useful if a build script exported some vars that a package then reads. See also:

  • packages

See about $GIT_AUTH_TOKEN: https://docs.docker.com/build/building/secrets/#git-authentication-for-remote-contexts

NOTE: this doesn't (yet) accumulate dependencies' set-envs values! Meaning only the top-level crate's setting is used, for all crates/dependencies.

set-envs = [ "GIT_AUTH_TOKEN", "TYPENUM_BUILD_CONSTS", "TYPENUM_BUILD_OP" ]

This environment variable takes precedence over any Cargo.toml settings:

# Note: values here are comma-separated.
export CARGOGREEN_SET_ENVS="GIT_AUTH_TOKEN,TYPENUM_BUILD_CONSTS,TYPENUM_BUILD_OP"

$CARGOGREEN_BASE_IMAGE_INLINE

Sets the base Rust image for root package and all dependencies, unless themselves being configured differently.

See also:

  • with-network
  • additional-build-arguments

In order to avoid unexpected changes, you may want to pin the image using an immutable digest.

Note that carefully crafting crossplatform stages can be non-trivial.

base-image-inline = """
FROM --platform=$BUILDPLATFORM rust:1 AS rust-base
RUN --mount=from=some-context,dst=/tmp/some-context cp -r /tmp/some-context ./
RUN --mount=type=secret,id=aws
"""

# This must also be set so digest gets pinned automatically.
base-image = "docker-image://rust:1"

This environment variable takes precedence over any Cargo.toml settings:

IFS='' read -r -d '' CARGOGREEN_BASE_IMAGE_INLINE <<"EOF"
FROM rust:1 AS rust-base
RUN --mount=from=some-context,dst=/tmp/some-context cp -r /tmp/some-context ./
RUN --mount=type=secret,id=aws
EOF
echo "$CARGOGREEN_BASE_IMAGE_INLINE" # (with quotes to preserve newlines)
export CARGOGREEN_BASE_IMAGE_INLINE
export CARGOGREEN_BASE_IMAGE=docker-image://rust:1

$CARGOGREEN_WITH_NETWORK

Controls runner's --network none (default) | default | host setting.

Set this to "default" if e.g. your base-image-inline calls curl or wget or installs some packages.

Set to none when in $CARGO_NET_OFFLINE mode. See

This environment variable takes precedence over any Cargo.toml settings:

export CARGOGREEN_WITH_NETWORK="none"

$CARGOGREEN_ADD_APT

Adds OS packages to the base image with apt install.

See also:

  • add.apk
  • add.apt-get
  • base-image
[package.metadata.green]
add.apt = [ "libpq-dev", "pkg-config" ]

This environment variable takes precedence over any Cargo.toml settings:

# Note: values here are comma-separated.
export CARGOGREEN_ADD_APT="libpq-dev,pkg-config"

# Inspect the resulting base image with:
cargo green supergreen env CARGOGREEN_BASE_IMAGE_INLINE

$CARGOGREEN_ADD_APT_GET

Adds OS packages to the base image with apt-get install.

See also:

  • add.apk
  • add.apt
  • base-image
add.apt-get = [ "libpq-dev", "pkg-config" ]

This environment variable takes precedence over any Cargo.toml settings:

# Note: values here are comma-separated.
export CARGOGREEN_ADD_APT_GET="libpq-dev,pkg-config"

# Inspect the resulting base image with:
cargo green supergreen env CARGOGREEN_BASE_IMAGE_INLINE

$CARGOGREEN_ADD_APK

Adds OS packages to the base image with apk add.

See also:

  • add.apt
  • add.apt-get
  • base-image
add.apk = [ "libpq-dev", "pkgconf" ]

This environment variable takes precedence over any Cargo.toml settings:

# Note: values here are comma-separated.
export CARGOGREEN_ADD_APK="libpq-dev,pkg-conf"

# Inspect the resulting base image with:
cargo green supergreen env CARGOGREEN_BASE_IMAGE_INLINE

$CARGOGREEN_EXPERIMENT

A comma-separated list of names of features to activate.

A name that does not match exactly is an error.

Use by setting this environment variable (no Cargo.toml setting):

export CARGOGREEN_EXPERIMENT="finalpathnonprimary,repro"

Alternatives

In no particular order:

Origins

Hacking

./hack/cli.sh ...

# Usage:           $0                              #=> generate CI
#
# Usage:           $0 ( <name@version> | <name> )  #=> cargo install name@version
# Usage:           $0   ok                         #=> cargo install all working bins
#
# Usage:           $0 ( build | test )             #=> cargo build ./cargo-green
#
# Usage:    jobs=1 $0 ..                           #=> cargo --jobs=$jobs
# Usage: offline=1 $0 ..                           #=> cargo --frozen (defaults to just: --locked)
# Usage:    rmrf=1 $0 ..                           #=> rm -rf $CARGO_TARGET_DIR/*; cargo ...
# Usage:   reset=1 $0 ..                           #=> docker buildx rm $BUILDX_BUILDER; cargo ...
# Usage:   clean=1 $0 ..                           #=> Both reset=1 + rmrf=1
# Usage:   final=1 $0 ..                           #=> Generate final Containerfile
#
# Usage:    DOCKER_HOST=.. $0 ..                   #=> Overrides machine
# Usage: BUILDX_BUILDER=.. $0 ..                   #=> Overrides builder (set to "empty" to set BUILDX_BUILDER='')

./hack/recipes.sh

Syncs ./recipes/*.Dockerfile files.

./hack/caching.sh

Verifies properties about caching crates & granularity.

docker buildx prune --all --force

./hack/hit.sh

Estimate of amount of crates reused through compilation of ./recipes/ --> ~5%! Expecting more with larger/more representative corpus + smart locking of transitive deps.

recipes/buildxargs@1.4.0.Dockerfile
8< 8< 8<
3: dep-l-utf8parse-0.2.1-522ff71b25340e24
5: dep-l-bitflags-1.3.2-70ce9f1f2fa253bc
5: dep-l-strsim-0.10.0-fd42a4ea370e31ec
5: dep-l-unicode-ident-1.0.12-4c1dc76c11b3deb8
6: dep-l-cfg-if-1.0.0-da34da6838abd7f1

Total recipes: 15
Total stages: 1065
Stages in common: 58
5.44%

When git-bisecting

all:
	cargo +nightly fmt --all
	./hack/clis.sh | tee .github/workflows/clis.yml
	./hack/self.sh | tee .github/workflows/self.yml
	CARGO_TARGET_DIR=$${CARGO_TARGET_DIR:-target/clippy} cargo clippy --locked --frozen --offline --all-targets --all-features -- --no-deps -W clippy::cast_lossless -W clippy::redundant_closure_for_method_calls -W clippy::str_to_string -W clippy::unnecessary_wraps
	RUST_MIN_STACK=8000000 cargo nextest run --all-targets --all-features --locked --frozen --offline --no-fail-fast
	git --no-pager diff --exit-code

Goals

  • seamlessly build on another machine (with more cores, more cache)
    • support remote builds by setting env DOCKER_HOST= with e.g. ssh://me@beaffy-machine.internal.net
      • Build cache is saved remotely, artifacts are saved locally
      • Tests building happens on remote machine, execution happens on local machine
  • seamlessly integrate with normal cargo usage
    • only pull sources from local filesystem
    • produce the same intermediary artefacts as local cargo does
    • fallback to normal, local rustc anytime
      • switching from this wrapper back to local rustc does necessitate a fresh build
  • wrap rustc calls in buildkit-like calls (docker, podman)
    • docker
    • podman TODO: test in CI
    • deps compatibility
      • handle Rust-only deps
      • handle all the other deps (expand this list) (use crater)
        • C deps
        • ...
    • runner compatibility
      • set .dockerignores (to be authoritative on srcs)
    • trace these outputs (STDOUT/STDERR) for debugging
  • available as a rustc wrapper through $RUSTC_WRAPPER
  • available as a cargo subcommand
    • configuration profiles (user, team, per-workspace, per-crate, CI, ...)
    • seamlessly use current/local rustc version
      • support overriding rustc base image
    • seamlessly use current/local tools (mold, ...)
      • config expressions on top of base image config?
      • just suggest an inline Dockerfile stage?
    • support CRUD-ish operations on local/remotes cache
    • [SEC] support building a crate without it having network access
  • integrate with shipping OCI images
  • share cache with the World cf. user-wide-cache
    • never rebuild a dep (for a given version of rustc, ...)
      • ensure finest cache granularity (crate-level)
      • free users from cache key built from Cargo.lock (changes on every release cut!)
    • share cache with other projects on local machine
      • fix WORKDIRs + rewrite paths with remap-path-prefix
    • share cache with CI and team
      • share cache with CI (at least for a single user)
    • [SEC] ensure private deps don't leak through/to cache
    • CLI gives the Dockerfile that cargo install's any crate
  • suggest a global cache -faciliting configuration profile
  • integrate with cross

    • build for a non-local target

    • run/test for a non-local target (with cross's same caveats ie. QEMU)

Upstream issues & patches

En vrac

Commit count: 284

cargo fmt