Crates.io | opensensor |
lib.rs | opensensor |
version | 0.4.0 |
source | src |
created_at | 2023-03-01 03:20:31.642014 |
updated_at | 2023-03-01 03:20:31.642014 |
description | A crate for streaming and archiving measurements from sensors |
homepage | |
repository | https://github.com/opensensordotdev/opensensor-rs |
max_upload_size | |
id | 797696 |
size | 283,248 |
Generics for implementing new sensors that interact with the OpenSensor Infrastructure as Code.
docker-compose
is now deprecated and the compose functionality is integrated into the docker compose
command. To install alongside an existing docker installation, run sudo apt-get install docker-compose-plugin
. ref.For Debian-based Linux distros, you can install opensensor-rs
's dependencies (except Docker, that require special repository configuration documented above) with the following command:
apt-get install clang build-essential lld clang zstd libzstd-dev make cmake pkg-config libssl-dev
opensensor-rs
is tested on Ubuntu 22.04 LTS, but welcomes pull requests to fix Windows or MacOS issues.
git clone https://github.com/opensensordotdev/opensensor-rs.git
opensensor-rs
won't build!./bootstrap_cluster.sh
: Start the testing Redpanda, MinIO, and monitoring stack.cargo test
: Verify all cargo tests passMeasurement
: The physical data structure representing the discrete unit of data produced by the sensor. Each Sensor
can produce multiple kinds of Measurements
.Transducer
: The interface with sensor hardware, producing a stream of Measurements
.Sensor
: The abstraction between multiple Transducers
that produce the same kind of Measurements
, grouping common validation logic together. Consumes from the mpsc channel that the Transducer
sends Measurements
on, validates them, and produces to Redpanda.Measurements
are notionally serialized using Google flatbuffers. For a sample Measurement
, Transducer
, and Sensor
implementation, see the sensor-simple
crate in the opensensor repository.
In order to make implementing new sensors as straightforward as possible, opensensor-rs
seeks to provide automatic archiving of Measurement
implementers to Parquet through Rust's Arrow bindings. Experiments for archiving arbitrary Rust structs to arrow and then parquet are documented in the archiver
directory and in arrow.rs
. Ideally, this functionality would be derivable or implementable through traits to allow arbitrary measurements to be serialized to/from parquet.
Filed this issue on arrow2, but even though their resulting PR fixed the Rust code (the tests in test_arrow.rs
now pass), the resulting parquet for any nested arrays or structs still can't be deserialized in pyarrow.
The test cases in test_arrow.rs
are based on the following examples: