narrow-derive

Crates.ionarrow-derive
lib.rsnarrow-derive
version
sourcesrc
created_at2021-07-29 22:37:29.48779
updated_at2024-12-02 15:23:14.68873
descriptionAn implementation of Apache Arrow
homepage
repositoryhttps://github.com/mbrobbel/narrow
max_upload_size
id429032
Cargo.toml error:TOML parse error at line 19, column 1 | 19 | autolib = false | ^^^^^^^ unknown field `autolib`, expected one of `name`, `version`, `edition`, `authors`, `description`, `readme`, `license`, `repository`, `homepage`, `documentation`, `build`, `resolver`, `links`, `default-run`, `default_dash_run`, `rust-version`, `rust_dash_version`, `rust_version`, `license-file`, `license_dash_file`, `license_file`, `licenseFile`, `license_capital_file`, `forced-target`, `forced_dash_target`, `autobins`, `autotests`, `autoexamples`, `autobenches`, `publish`, `metadata`, `keywords`, `categories`, `exclude`, `include`
size0
Matthijs Brobbel (mbrobbel)

documentation

http://docs.rs/narrow/

README

Narrow logo

crates.io docs.rs

An experimental (work-in-progress) statically typed implementation of Apache Arrow.

This crate provides methods to automatically generate types to support reading and writing instances of abstract data types in Arrow's in-memory data structures.

Why

  • The arrow crate provides APIs that make sense when the array types are only known at run-time. Many of its APIs require the use of trait objects and downcasting. However, for applications where types are known at compile-time, these APIs are not ergonomic.
  • Builders for nested array types are complex and error-prone.

There are other crates that aim to prevent users from having to maintain array builder code by providing derive macros. These builders typically produce type-erased arrays, whereas this crate only provides fully statically typed arrays.

Goals and non-goals

Goals

  • Provide production ready, fully statically typed, safe and efficient Arrow array implementations
  • Enable everyone using Rust to easily benefit from the Arrow ecosystem
  • Provide zero-copy interop with the arrow crate
  • Support custom buffer implementations e.g. to support accelerators
  • Explore expressing Arrow concepts using the Rust type system, and mapping Rust concepts to Arrow

Non-goals

  • Support arbitrary array types at runtime (the arrow crate supports this use case)
  • Provide compute kernels
  • Replace other Arrow implementations

Example

use narrow::{
    array::{StructArray, UnionArray},
    ArrayType, Length,
};

#[derive(ArrayType, Default, Clone, Debug, PartialEq, Eq)]
struct Foo {
    a: bool,
    b: u32,
    c: Option<String>,
}

#[derive(ArrayType, Default, Clone, Debug, PartialEq, Eq)]
struct Bar(Vec<u8>);

#[derive(ArrayType, Clone, Debug, PartialEq, Eq)]
enum FooBar {
    Foo(Foo),
    Bar(Bar),
    None,
}

let foos = vec![
    Foo {
        a: false,
        b: 0,
        c: None,
    },
    Foo {
        a: true,
        b: 42,
        c: Some("hello world".to_owned()),
    },
];
let struct_array = foos.clone().into_iter().collect::<StructArray<Foo>>();
assert_eq!(struct_array.len(), 2);
assert_eq!(struct_array.into_iter().collect::<Vec<_>>(), foos);

let foo_bars = vec![
    FooBar::Foo(Foo {
        a: true,
        b: 42,
        c: Some("hello world".to_owned()),
    }),
    FooBar::Bar(Bar(vec![1, 2, 3, 4])),
    FooBar::None,
    FooBar::None,
];
let union_array = foo_bars
    .clone()
    .into_iter()
    .collect::<UnionArray<FooBar, 3>>();
assert_eq!(union_array.len(), 4);
assert_eq!(union_array.into_iter().collect::<Vec<_>>(), foo_bars);

Features

The crate supports the following optional features:

  • derive: adds [ArrayType] derive support.
  • arrow-rs: adds array conversion methods for arrow.
  • uuid: adds ArrayType support for uuid::Uuid.

Docs

Minimum supported Rust version

The minimum supported Rust version for this crate is Rust 1.79.0.

License

Licensed under either of Apache License, Version 2.0 or MIT license at your option.

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

Commit count: 490

cargo fmt