Crates.io | std-dev |
lib.rs | std-dev |
version | 0.1.0 |
source | src |
created_at | 2024-02-25 15:30:38.967899 |
updated_at | 2024-02-25 15:30:38.967899 |
description | Your Swiss Army knife for swiftly processing any amount of data. Implemented for industrial and educational purposes alike. |
homepage | https://github.com/Icelk/std-dev |
repository | https://github.com/Icelk/std-dev |
max_upload_size | |
id | 1152525 |
size | 309,523 |
Your Swiss Army knife for swiftly processing any amount of data. Implemented for industrial and educational purposes alike.
This codebase is well-documented and comments, in an effort to expose the wonderful algorithms of data analysis to the masses.
We're ever expanding, but for now the following are implemented.
This application supports using it both as a library (with optional cargo features), an interactive CLI program, and through piping data to it, through standard input.
It accepts any comma/space separated values. Scientific notation is supported. This is minimalistic by design, as other programs may be used to produce/modify the data before it's processed by us.
Using the subcommand completion
, std-dev automatically generates shell completions for your shell and tries to put them in the appropriate location.
When using Bash or Zsh, you should run std-dev as root, as we need root privileges to write to their completion directories.
Alternatively, use the --print
option to yourself write the completion file.
When using this as a library, I recommend disabling all features (except base
) (std-dev = { version = "0.1", default-features = false, features = ["base"] }
)
and enabling those you need.
bin
(default, binary feature): This enables the binary to compile.prettier
(default, binary feature): Makes the binary output prettier. Includes colours and prompts for interactive use.completion
(default, binary feature): Enable the ability to generate shell completions.regression
(default, library and binary feature): Enables all regression estimators. This requires nalgebra
, which provides linear algebra.ols
(default, library feature): Enables the use of OLS, which is the "default" estimator. This also enables polynomial Theil-Sen for degrees > 2 & polynomial regression in best_fit
functions.arbitrary-precision
(default, library feature): Uses arbitrary precision algebra for >10 degree polynomial regression.percentile-rand
(default, base, library feature): Enables the recommended pivot_fn
for percentile-related functions.simplify-fraction
(default, base, library feature): Fractions are simplified. Relaxes the requirements for fraction input and implements Eq & Ord for fractions.generic-impls
(default, base, library feature): Makes mean
, standard_deviation
, and percentile resolving generic over numbers. This enables you to use numerical types from other libraries without hassle.Documentation of the main branch can be found at doc.icelk.dev.
To document with information on which cargo features enables the code,
set the environment variable RUSTDOCFLAGS
to --cfg docsrs
(e.g. in Fish set -x RUSTDOCFLAGS "--cfg docsrs"
)
and then run cargo +nightly doc
.
This library aims to be as fast as possible while maintaining easily readable code.
As all algorithms are executed in linear time now, this is not as useful, but nevertheless an interesting feature. If you already have clustered data, this feature is great.
When using the clusters feature (turning your list into a ClusterList
),
calculations are done per unique value.
Say you have a dataset of infant height, in centimeters.
That's probably only going to be some 40 different values, but potentially millions of entries.
Using clusters, all that data is only processed as O(40)
, not O(millions)
. (I know that notation isn't right, but you get my point).
Creating this cluster involves adding all the values to a map. This takes O(n)
time, but is very slow compared to all other algorithms.
After creation, most operations in this library are executed in O(m)
time, where m is the count of unique values.