parquet_to_excel

A tool to convert parquet file to an/a excel/csv file in rust with constant memory, both a single parquet file and a folder of parquet files are supported. You can also use python or rust to call it. The python package name is parquet_to_excel too. you can install it by pip install parquet_to_excel. If you could not install this package correctly, you can try to install rust and maturin (pip install maturin) first. Then you can try again.

Functions

1. parquet_file_to_csv: convert a single parquet file to a csv file
2. parquet_files_to_csv: convert a folder of parquet files to a csv file
3. parquet_file_to_xlsx: convert a single parquet file to an excel file
4. parquet_files_to_xlsx: convert a folder of parquet files to an excel file

Rust Excamples

1. parquet to csv

use std::collections::HashMap;
use parquet_to_excel::csv::{file_to_csv, folder_to_csv};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let mut headerlabels = HashMap::new();
    headerlabels.insert("gsmc".to_string(), "公司名称".to_string());
    headerlabels.insert("col2".to_string(), "Column 2".to_string());

    let select_colmuns = HashSet::from(["sheet".into(), "yjkm".into(), "yjkmsm".into()]);

    //  parquet file to csv
    let source = r"D:\Projects\RustTool\data\.duck\yo_dxzh\source=合并报表公司主体及内部客商编码（管理责任人：刘露）.xlsx\data.parquet";
    let writer = r"data\test.csv";
    file_to_csv(source, writer, &headerlabels, &select_colmuns)?;

    //  parquet folder to csv
    let source = r"D:\Projects\RustTool\data\.duck\yo_dxzh";
    let writer = r"data\test1.csv";
    folder_to_csv(source, writer, &headerlabels, &select_colmuns)?;
    
    Ok(())
}

2. parquet to xlxs

use std::collections::HashMap;
use parquet_to_excel::xlsx::{file_to_xlsx, folder_to_xlsx};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let mut headerlabels = HashMap::new();
    headerlabels.insert("gsmc".to_string(), "公司名称".to_string());
    headerlabels.insert("col2".to_string(), "Column 2".to_string());

    let select_colmuns = HashSet::from(["sheet".into(), "yjkm".into(), "yjkmsm".into()]);

    //  parquet file to xlsx
    let source = r"D:\Projects\RustTool\data\.duck\yo_dxzh\source=xxx.xlsx\data.parquet";
    let writer = r"data\test.xlsx";
    file_to_xlsx(source, writer, Some("data".into()), None, &headerlabels, &select_colmuns)?;

    //  parquet folder to csv
    let source = r"D:\Projects\RustTool\data\.duck\yo_dxzh";
    let writer = r"data\test1.xlsx";
    folder_to_xlsx(source, writer, None, Some("gsmc".into()),&headerlabels, &select_colmuns)?;
    
    Ok(())
}

Python Example

1. parquet to xlsx

from parquet_to_excel import parquet_file_to_xlsx, parquet_files_to_xlsx

# the last three arguments are optional
parquet_file_to_xlsx(r"data\result\qid=160\a.parquet", r"out1.xlsx", "data", "", {"ddbm": "地点编码"})
parquet_files_to_xlsx(r"data\result\qid=160", r"out2.xlsx", "", "scfs", {"ddbm": "地点编码"})