nixpkgs-fmt - Nix code formatter for nixpkgs

STATUS: beta

This project's goal is to format the nix code in nixpkgs to increase the consistency of the code found there. Ideally automatically with pre-commit hooks and later ofborg enforcing the format.

Demo

You can try nixpkgs-fmt in your browser. The page also provides a way for you to submit code samples if you find the output not satisfying: https://nix-community.github.io/nixpkgs-fmt/

Design decisions

You might ask yourself; why do we need yet another nix code formatter?

The main goal of nixpkgs-fmt is to provide some overall consistency in the nix code submitted to nixpkgs, our main package repository.

At this point it's important to understand that there are multiple possible outputs for a code formatter. Those outputs will depend on multiple conflicting desires and depending on how much weight is being put on each requirement the output will change.

For nixpkgs-fmt we have a few of these:

1. Minimize merge conflicts. nixpkgs is seen in a lot of pull-requests and we want to avoid them getting unnecessarily stale.
2. Only expand, don't collapse. It's up to the developer to choose if an element should be on a single line or multiple lines.
3. Respect the developer's expressivity. Empty lines can be useful as a way to separate blocks of code.
4. Only change the indent of one (+/-) per line. Not sure why but it seems like a good thing.

Corollary rules:

• because of (1). The format is quite close to what exists in nixpkgs already.
• because of (1). Don't align values vertically, a single line change can introduce a very big diff.
• because of (1). Avoid too many rules. More rules means more formatting changes that create merge conflicts.
• because of (2). Don't enforce line lengths. Line length limits also create complicated heuristics.

At the time where we started this project none of the other formatters were weighted that way.

To implement this, we needed a whitespace and comment-preserving parser which rnix provides to us. Then create an engine that follows the AST and patches the tree with rewrite rules. The nice thing about this design is that it also works on incomplete or broken nix code. We are able to format up to the part that is missing/broken, which makes it great for potential editor integration.

Most of the other formatters out there take a pretty-printing approach where the AST is parsed, and then a pretty-printer inspects and formats the AST back to code without taking spaces and newlines into account. The advantage is that it's initially easier to implement. The output is very strict and the same AST will always give the same output. One disadvantage is that the pretty-printer needs to handle all the possible combination of Nix code to make them look good.

With nixpkgs-fmt the output will depend on how the code was formatted initially. The developer still has some input on how they want to format their code. If there is no rule for a complicated case, the code will be left alone. For nixpkgs this approach will be preferable since it minimizes the diff.

Well done for reading all of this, I hope this clarifies a bit why nixpkgs-fmt exists and what role it can play.

Usage

$ nixpkgs-fmt --help 2>&1 || true

nixpkgs-fmt 0.9.0
Format Nix code

USAGE:
    nixpkgs-fmt [FLAGS] [FILE]...

FLAGS:
        --check            Only test if the formatter would produce differences
        --explain          Show which rules are violated
    -h, --help             Prints help information
        --output-format    Output syntax tree in JSON format
        --parse            Show syntax tree instead of reformatting
    -V, --version          Prints version information

ARGS:
    <FILE>...    File to reformat in place. If no file is passed, read from stdin.

Tree traversal

When nixpkgs-fmt is given a folder as a file argument, it will traverse that using the same ignore crate as ripgrep, using 8 parallel threads.

By default it will automatically ignore files reading .ignore, .gitignore, and .git/info/exclude files in that order. If additional files need to be ignored, it is also possible to add --exclude <glob> to the call.

Installation

nixpkgs-fmt is available in nixpkgs master. nix-env -i nixpkgs-fmt.

It's also possible to install it directly from this repository:

nix-env -f https://github.com/nix-community/nixpkgs-fmt/archive/master.tar.gz -i

pre-commit hook

This project can also be installed as a pre-commit hook.

Add to your project's .pre-commit-config.yaml:

-   repo: https://github.com/nix-community/nixpkgs-fmt
    rev: master
    hooks:
    -   id: nixpkgs-fmt

Make sure to have rust available in your environment.

Then run pre-commit install-hooks

Development

Install Rust and Cargo or run nix-shell to load the project dependencies.

Install pre-commit and run pre-commit install to setup the git hooks on the repository. This will allow to keep the code nicely formatted over time.

Then use cargo run to build and run the software.

Running Fuzzer

$ cargo install cargo-fuzz
$ mkdir -p ./fuzz/corpus/fmt
$ cp test_data/**.nix ./fuzz/corpus/fmt
$ rustup run nightly -- cargo fuzz run fmt

or with nix:

$ nix-shell --run "cargo fuzz run fmt"

• fmt is the name of the target in ./fuzz/Cargo.toml

Fuzzer will run indefinitely or until it finds a crash. The crashing input is written to fuzz/artifacts directory. Commit this crash- file, and it will be automatically tested by a unit-test.

Documentation

• HOWTO write new rules
• HOWTO WASM
• How we do releases

Related projects

Feel free to submit your project!

Using nixpkgs-fmt

• Emacs integration, including minor mode for format-on-save

• rnix-lsp - A Lambda Server for Nix

Formatters

• canonix - Nix formatter prototype written in Haskell using the tree-sitter-nix grammar.
• format-nix - A nix formatter using tree-sitter-nix.
• nix-format - Emacs-based Nix formatter.
• nix-lsp - Nix language server using rnix.
• nixfmt - A nix formatter written in Haskell.

Linters

• nix-linter

Parsers

• hnix - Haskell implementation of Nix including a parser. The parser is not comment-preserving.
• rnix - Rust Nix parser based on rowan
• tree-sitter-nix - Tree Sitter is a forgiving parser used by Atom for on-the-fly syntax highlighting and others. This is a implementation for Nix.

Discussions

• nixpkgs style guide
• On Nix expression formatting
• [Job] Implement a nix-fmt formatter

Sponsors

This work has been sponsored by NumTide.