parquet2lance

Crates.ioparquet2lance
lib.rsparquet2lance
version0.4.4
sourcesrc
created_at2023-03-18 17:36:28.240232
updated_at2024-07-30 09:51:22.066969
descriptionConvert parquet files to lance
homepage
repositoryhttps://github.com/haoxins/parquet2lance
max_upload_size
id813875
size143,015
Xin Hao (haoxins)

documentation

README

  • Convert parquet files to lance

  • Working with GCS

    • Using gcloud auth application-default login to generate Application Default Credentials (ADC)
parquet2lance \
  -i gs://cloud-samples-data/bigquery/us-states/ \
  -o test.lance -O

parquet2lance \
  -i gs://cloud-samples-data/bigquery/us-states/us-states.parquet \
  -o test.lance -O
  • From GCS to GCS
parquet2lance \
  -i gs://cloud-samples-data/bigquery/us-states/ \
  -o gs://your-bucket-name/bigquery/us-states/

parquet2lance \
  -i gs://cloud-samples-data/bigquery/us-states/us-states.parquet \
  -o gs://your-bucket-name/bigquery/us-states/us-states.lance
  • Working with local FS
gsutil cp gs://cloud-samples-data/bigquery/us-states/us-states.parquet test.parquet

parquet2lance -i test.parquet -o test.lance -O
Commit count: 62

cargo fmt