parquet2lance

Crates.ioparquet2lance
lib.rsparquet2lance
version0.5.0
created_at2023-03-18 17:36:28.240232+00
updated_at2025-01-24 07:48:55.3234+00
descriptionConvert parquet files to lance
homepage
repositoryhttps://github.com/haoxins/parquet2lance
max_upload_size
id813875
size151,751
Xin Hao (haoxins)

documentation

README

  • Convert parquet files to lance

  • Working with GCS

    • Using gcloud auth application-default login to generate Application Default Credentials (ADC)
parquet2lance \
  -i gs://cloud-samples-data/bigquery/us-states/ \
  -o test.lance -O

parquet2lance \
  -i gs://cloud-samples-data/bigquery/us-states/us-states.parquet \
  -o test.lance -O
  • From GCS to GCS
parquet2lance \
  -i gs://cloud-samples-data/bigquery/us-states/ \
  -o gs://your-bucket-name/bigquery/us-states/

parquet2lance \
  -i gs://cloud-samples-data/bigquery/us-states/us-states.parquet \
  -o gs://your-bucket-name/bigquery/us-states/us-states.lance
  • Working with local FS
gsutil cp gs://cloud-samples-data/bigquery/us-states/us-states.parquet test.parquet

parquet2lance -i test.parquet -o test.lance -O
Commit count: 64

cargo fmt