derive_fuzztest
==========

_This is not an officially supported Google product._

[![Crates.io](https://img.shields.io/crates/v/derive_fuzztest)](https://crates.io/crates/derive_fuzztest) [![Docs](https://docs.rs/derive_fuzztest/badge.svg)](https://docs.rs/derive_fuzztest)

## Description

Proc macros that generates both a fuzz target for use with `cargo fuzz`, and a property test
(via `quickcheck` or `proptest`) for use with `cargo test`.

The reason for having both is that property testing allows for quick iteration to make sure the
test works, and can be checked in presubmit CI, while fuzzing can test the input space more
exhaustively and run continuously.

## Example

```rust
#![cfg_attr(fuzzing, no_main)]

#[derive_fuzztest::fuzztest]
fn transitive_ord(a: u32, b: u32, c: u32) {
    if a >= b && b >= c {
        assert!(a >= c);
    }
    if a <= b && b <= c {
        assert!(a <= c);
    }
}

#[test]
fn additional_test_here() {
    /* ... */
}
```

# Result reporting

Test functions report test failures by panicking, similar to how you would write a regular
`#[test]`. If the annotated test function completes without panicking, the test is considered to
have passed.

In some cases, you may want to discard some inputs, treating them neither as passes nor failures,
just continue onto testing another generated input. This can be done by returning a
[`TestResult`](https://docs.rs/derive_fuzztest/latest/derive_fuzztest/enum.TestResult.html) from the
annotated function. Property testing frameworks will try to generate more test cases to replace the
discarded one, up to a certain limit. For fuzzing, test results that are discarded will not be added
to the corpus.

```rust
use derive_fuzztest::TestResult;

#[derive_fuzztest::fuzztest]
fn increment(a: u8) -> TestResult {
    if a < u8::MAX {
        assert_eq!(a + 1 - 1, a);
        TestResult::Passed
    } else {
        TestResult::Discard
    }
}
 ```

## Usage

Run the generated property tests
```sh
cargo test
```

Run continuous fuzzing
```sh
cargo install cargo-fuzz
cargo +nightly fuzz run <binary name>
```

## Crate structure

If you use `#[fuzz]` or `#[fuzztest]`, the fuzz target imposes the following requirements:

* The target must be in a separate `[[bin]]` target that only contains a single fuzz target.
* The crate containing the bin target has `[package.metadata] cargo-fuzz = true`
* The bin target is annotated with `#![cfg_attr(fuzzing, no_main)]`

The recommended structure for your crate `foo` is to put your tests under `foo/fuzz/src/bin`:

```text
foo
├── fuzz
│   ├── src
│   │   └── bin
│   │       └── fuzz_target_1.rs
│   └── Cargo.toml
├── src
│   └── [project source]
└── Cargo.toml
```

This is different from the default structure generated by `cargo fuzz init` or `cargo fuzz add`
so that we can take advantage of [target
auto-discovery](https://doc.rust-lang.org/cargo/reference/cargo-targets.html#target-auto-discovery).
If you prefer, the default structure generated by `cargo fuzz` can also work, but make sure you
remove `test = false` from the generated target in `Cargo.toml`.

You will also need to declare a dependency on the `libfuzzer-sys` crate, but only if fuzzing is
requested:

```toml
[target.'cfg(fuzzing)'.dependencies]
libfuzzer-sys = "*"
```

(The reason for this conditional dependency is that `libfuzzer-sys` injects a main function to
the resulting binary, and there will be linking failures if we link that in without defining a
corresponding `fuzz_target`.)

## Features

* `quickcheck` (default) — Enable generation of
  [`quickcheck`](https://docs.rs/quickcheck/latest/quickcheck/) property tests.
* `proptest` — Enable generation of [`proptest`](https://docs.rs/proptest/latest/proptest/)
  property tests.

## Relevant reading
* [Announcing Better Support for Fuzzing with Structured Inputs in
  Rust](https://fitzgeraldnick.com/2020/01/16/better-support-for-fuzzing-structured-inputs-in-rust.html#how-is-all-this-different-from-quickcheck-and-proptest)
* [Bridging Fuzzing and Property
  Testing](https://blog.yoshuawuyts.com/bridging-fuzzing-and-property-testing/)