Crates.io | json2arrow |
lib.rs | json2arrow |
version | 0.19.0 |
source | src |
created_at | 2021-02-28 09:10:01.808727 |
updated_at | 2024-09-18 02:08:52.969487 |
description | Convert JSON files to Arrow |
homepage | https://github.com/domoritz/arrow-tools/tree/main/crates/json2arrow |
repository | https://github.com/domoritz/arrow-tools |
max_upload_size | |
id | 361724 |
size | 40,669 |
Convert JSON files to Apache Arrow. This package is part of Arrow CLI tools.
You can get the latest releases from https://github.com/domoritz/arrow-tools/releases.
brew install domoritz/homebrew-tap/json2arrow
cargo install json2arrow
To avoid re-compilation and speed up installation, you can install this tool with cargo binstall
:
cargo binstall json2arrow
Usage: json2arrow [OPTIONS] <JSON> [ARROW]
Arguments:
<JSON> Input JSON file, stdin if not present
[ARROW] Output file, stdout if not present
Options:
-s, --schema-file <SCHEMA_FILE>
File with Arrow schema in JSON format
-m, --max-read-records <MAX_READ_RECORDS>
The number of records to infer the schema from. All rows if not present. Setting max-read-records to zero will stop schema inference and all columns will be string typed
-p, --print-schema
Print the schema to stderr
-n, --dry
Only print the schema
-h, --help
Print help
-V, --version
Print version
The --schema-file option uses the same file format as --dry and --print-schema.
For usage examples, see the csv2parquet
examples which shares a similar interface.
Since we use the Arrow JSON loader, we are limited to what it supports. Right now, it supports JSON line-delimited files.
{ "a": 42, "b": true }
{ "a": 12, "b": false }
{ "a": 7, "b": true }