# Alumina An experimental deep learning library written in pure rust. Breakage expected on each release in the short term. See mnist.rs in examples or [Rusty_SR](https://github.com/millardjn/rusty_sr) for usage samples. ## Contributions Issues are a great place for discussion, problems, requests, and coordinating future work. Blatantly incorrect documentation contributions are encouraged as a way to guide efforts on docs, just submit a PR and fill a doc comment with anything from your best guess to passive aggressive nursery rhymes. ## Documentation Patchy until the library settles down, particularly until the graph abstraction is finalised and the switch to ndarray is completed. ## Progress - [x] Computation hypergraph - [x] Dense Connection and Bias operations - [x] Loss functions - [x] Mean Squared Error - [x] Categorical Cross Entropy - [x] SoftMax Cross Entropy - [ ] Binary Cross Entropy - [x] Activations - [x] Tanh - [x] Logistic - [x] Identity - [x] ReLU - [x] LeakyReLU - [x] ELU - [x] SoftMax - [x] SRGB Curves - [x] BeLU - [ ] SoftExp - [ ] SoftPlus - [x] Spatial operations - [x] Shape constraint propagation - [x] N-dimensional Convolution - [x] Arbitrary padding - [ ] Strides - [x] N-dimensional AvgPooling - [x] N-dimensional spaxel shuffling for "Sub-pixel Convolution" - [ ] N-dimensional Linear-Interpolation (backprop not finished) - [x] Global Pooling - [x] Broadcasting - [x] Data Loading - [x] Mnist - [ ] Cifar - [x] Image Folders - [x] Imagenet (ILSVRC) - [x] SGD - [ ] RMSProp - [ ] ADAM - [x] CAIN - [x] Adaptive BatchSize - [x] Adaptive Learning Rate - [ ] Adaptive Momentum - [x] Basic numerical tests - [ ] Limit Optimiser evaluation batch size to stay within memory limits - [ ] Selectively disable calculation of forward values, node derivatives and parameter derivatives - [ ] Builder patterns for operation contruction - [ ] Split Graph struct into mutable GraphBuilder and immutable Sub-Graphs - [ ] Replace 'accidentally quadratic' graph algorithms - [ ] Replace up-front allocation with Sub-Graph optimised allocation/deallocation patterns based on liveness analysis of nodes - [ ] Overhaul data ingestion, particularly buffering input processing/reads. - [ ] Move to bluss' ndarray where possible (long overdue) - [ ] Improve naming inter/intra-library consistancy - [ ] Complete Documentation - [ ] Reduce ability to express illegal states in API - [ ] Move from panics to error-chain - [ ] Guard unsafe code rigourously - [ ] Comprehensive tests - [ ] Arrayfire as an option for sgemm on APUs ### Distant - [ ] RNNs - [ ] Efficient probablistic structures (e.g. generative RNNs) - [ ] Graph optimisation passes and inplace operations - [ ] Support for both dynamic and static graphs ## License MIT