# Notes

Right now the benches are super basic, but at least we can get a sense of relative performance between the various ECSs covered. And it's useful to see how flecs versions progress.

## Bench history on M1 Max (Dec 2022)

Bevy 0.9.1:

- simple_insert/bevy_single - 18.307 ms
- simple_insert/bevy_batched - 3.1692 ms
- simple_iter/bevy - 120.06 µs

Hecs 0.9.0

- simple_insert/hecs_single - 10.207 ms
- simple_insert/hecs_batched - 2.9633 ms
- simple_iter/hecs - 91.739 µs

Flax 0.3.2

- simple_insert/flax_single - 51.808 ms
- simple_insert/flax_batched - 1.5515 ms
- simple_iter/flax - 237.27 µs

flecs 3.0.1

- simple_insert/flecs - 69.735 ms
- simple_iter/flecs_each - 722.49 µs 723.19 µs
- simple_iter/flecs_iter - 35.810 µs

## Bench history on M1 Max (Apr 2023)

Bevy 0.10.3:

- simple_insert/bevy_single - 14.718 ms
- simple_insert/bevy_batched - 2.9090 ms
- simple_iter/bevy - 122.76 µs

Hecs 0.10.1

- simple_insert/hecs_single - 10.123 ms
- simple_insert/hecs_batched - 2.8848 ms
- simple_iter/hecs - 91.187 µs

Flax 0.4.0

- simple_insert/flax_single - 52.398 ms
- simple_insert/flax_batched - 1.5866 ms
- simple_iter/flax - 97.714 µs large gain! on par w Hecs now

flecs 3.0.1 (unchanged)

- simple_insert/flecs - 69.735 ms
- simple_iter/flecs_each - 722.49 µs
- simple_iter/flecs_iter - 35.810 µs

## Flecs 3.1.0 Release (Apr 2023)

Other 3 similar to above

flecs 3.1.0

- simple_insert/flecs - 70.977 ms
- simple_iter/flecs_each - 724.53
- simple_iter/flecs_iter - 35.739 µs

## Flecs 3.1.2 Release (Apr 2023)

Other 3 similar to above
Similar results, slightly worse

flecs 3.1.2

- simple_insert/flecs - 75.482 ms
- simple_iter/flecs_each - 741.34 µs
- simple_iter/flecs_iter - 36.590 µs

## Flecs 3.1.3 Release (Apr 2023)

Pretty much the same results
Should add better benches once 3.2 upgrade is completed.

## Flecs 3.1.4 Release (Apr 2023)

flecs 3.1.2

- simple_insert/flecs - 59.603 ms  20% savings!
- simple_iter/flecs_each - 732.62 µs
- simple_iter/flecs_iter - 35.545 µs

## Flecs 3.1.5 Release (Apr 2023)

Pretty much the same results as for 3.1.4

## Flecs 3.2.0 Release (Apr 2023)

Pretty much the same results as for 3.1.4

## Flecs 3.2.1 Release (Apr 2023)

Pretty much the same results as for 3.1.4

## June 17, 2023

CC Optimizations thanks to TransmogrificationDevice [gH]

By simply setting NDEBUG = None we get huge gain of 60 - 70%!

Benchmarking simple_insert/flecs: Collecting 100 samples in estimated 7.1483 s (300 iterati
simple_insert/flecs     time:   [23.438 ms 23.523 ms 23.615 ms]
                        change: [-60.236% -60.047% -59.852%] (p = 0.00 < 0.05)
                        Performance has improved.

Benchmarking simple_iter/flecs_each: Collecting 100 samples in estimated 5.6289 s (25k iter
simple_iter/flecs_each  time:   [221.89 µs 222.18 µs 222.54 µs]
                        change: [-69.932% -69.849% -69.764%] (p = 0.00 < 0.05)
                        Performance has improved.

- simple_insert/flecs - 23.523 ms		60% savings!
- simple_iter/flecs_each - 222.18 µs	70% savings!
- simple_iter/flecs_iter - 34.970 µs

After lto = "fat"

Benchmarking simple_insert/flecs: Collecting 100 samples in estimated 6.9128 s (300 iterati
simple_insert/flecs     time:   [22.986 ms 23.063 ms 23.149 ms]
                        change: [-2.1922% -1.7339% -1.2837%] (p = 0.00 < 0.05)
                        Performance has improved.

Benchmarking simple_iter/flecs_each: Collecting 100 samples in estimated 5.0352 s (30k iter
simple_iter/flecs_each  time:   [166.63 µs 167.13 µs 167.77 µs]
                        change: [-24.713% -24.435% -24.117%] (p = 0.00 < 0.05)
                        Performance has improved.

Benchmarking simple_iter/flecs_iter: Collecting 100 samples in estimated 5.6149 s (35k iter
simple_iter/flecs_iter  time:   [156.15 µs 157.05 µs 157.89 µs]
                        change: [+347.81% +349.92% +351.82%] (p = 0.00 < 0.05)
                        Performance has regressed.