madato

madato is a library and command line tool for working tabular data, and Markdown

• madato
  • Usage
    • CLI
    • Rust
    • Python
  • Details
    • Example CLI usage
  • Internals
  • Tips
  • Python
  • More Commandline
    • Sheet List
    • YAML to Markdown
    • Excel/ODS to YAML
  • Features
  • Future Goals
    • Known Issues
  • License
    • Contribution

1. madato (library) - this library, which reads YAML, CSV, JSON, XLSX/ODS and writes Markdown
2. madato (cli) - providing a helpful command line tool of the above
3. The full library is available as a python module, or a rust library.

The tools is primarly centered around getting tabular data (spreadsheets, CSVs) into Markdown.

Usage

CLI

Download from https://github.com/inosion/madato/releases

Rust

The library, if you need spreadsheet support, then add the spreadsheets feature.

madato = { version = "0", features = ["spreadsheets"] }

Python

pip install madato

Details

When generating the output:

• Filter the Rows using basic Regex over Key/Value pairs
• Limit the columns to named headings
• Re-order the columns, or repeat them using the same column feature
• Only generate a table for a named "sheet" (applicable for the XLS/ODS formats)

Madato is:

• Command Line Tool (Windows, Mac, Linux) - good for CI/CD preprocessing
• Rust Library - Good for integration into Rust Markdown tooling
• Node JS WASM API - To be used later for Atom and VSCode Extensions

Madato expects that every column has a heading row. That is, the first row are headings/column names. If a cell in that first row is blank, it will create NULL0..NULLn entries as required.

Example CLI usage

• Extract the 3rd Sheet sheet from an MS Excel Document

08:39 $ madato table --type xlsx test/sample_multi_sheet.xlsx --sheetname "3rd Sheet"
|col1|col2| col3 |col4 |                         col5                          |NULL5|
|----|----|------|-----|-------------------------------------------------------|-----|
| 1  |that| are  |wider|  value ‘aaa’ is in the next cell, but has no heading  | aaa |
|than|the |header| row |       (open the spreadsheet to see what I mean)       |     |

• Extract and reorder just 3 Columns

08:42 $ madato table --type xlsx test/sample_multi_sheet.xlsx --sheetname "3rd Sheet" -c col2 -c col3 -c NULL5
|col2| col3 |NULL5|
|----|------|-----|
|that| are  | aaa |
|the |header|     |

• Pull from the second_sheet sheet
• Only extract Heading 4 column
• Use a Filter, where Heading 4 values must only have a letter or number.

08:48 $ madato table --type xlsx test/sample_multi_sheet.xlsx --sheetname second_sheet -c "Heading 4" -f 'Heading 4=[a-zA-Z0-9]'
|        Heading 4         |
|--------------------------|
|         << empty         |
|*Some Bolding in Markdown*|
|   `escaped value` foo    |
|           0.22           |
|         #DIV/0!          |
|  “This cell has quotes”  |
|       😕 ← Emoticon       |

• Filtering on a Column, ensuring that a "+" is there in Trend Column

09:00 $ madato table --type xlsx test/sample_multi_sheet.xlsx --sheetname Sheet1 -c Rank -c Language -c Trend -f "Trend=\+"
|                         Rank                         |  Language  |Trend |
|------------------------------------------------------|------------|------|
|                          1                           |   Python   |+5.5 %|
|                          3                           | Javascript |+0.2 %|
|                          7                           |     R      |+0.0 %|
|                          12                          | TypeScript |+0.3 %|
|                          16                          |   Kotlin   |+0.5 %|
|                          17                          |     Go     |+0.3 %|
|                          20                          |    Rust    |+0.0 %|

Internals

madato uses:

• calamine for reading XLS and ODS sheets
• wasm bindings to created JS API versions of the Rust API
• regex for filtering, and serde for serialisation.
• PyO3 and Maturin for Python Support

Tips

• I have found that copying the "table" I want from a website: HTML, to a spreadsheet, then through madato gives an excellent Markdown table of the original.

Python

pip install madato

# py
from IPython.display import display, Markdown
import madato
display(Markdown(madato.spreadsheet_to_md("../test/Financial Sample.xlsx")
print(madato.spreadsheet_to_md(str(my_sample_spreadsheet)))

• For more examples see pysource/tests

More Commandline

Sheet List

You can list the "sheets" of an XLS*, ODS file with

$ madato sheetlist test/sample_multi_sheet.xlsx 
Sheet1
second_sheet
3rd Sheet

YAML to Markdown

Madato reads a "YAML" file, in the same way it can a Spreadsheet. This is useful for "keeping" tabular data in your source repository, and perhaps not the XLS.

madato table -t yaml test/www-sample/test.yml

|col3| col4  |  data1  |       data2        |
|----|-------|---------|--------------------|
|100 |gar gar|somevalue|someother value here|
|190x|       |  that   |        nice        |
|100 | ta da |  this   |someother value here|

Please see the test/www-sample/test.yml file for the expected layout of this file

Excel/ODS to YAML

Changing the output from default "Markdown (MD)" to "YAML", you get a Markdown file of the Spreadsheet.

madato table -t xlsx test/sample_multi_sheet.xslx.xlsx -s Sheet1 -o yaml
---
- Rank: "1"
  Change: ""
  Language: Python
  Share: "23.59 %"
  Trend: "+5.5 %"
- Rank: "2"
  Change: ""
  Language: Java
  Share: "22.4 %"
  Trend: "-0.5 %"
- Rank: "3"
  Change: ""
  Language: Javascript
  Share: "8.49 %"
...

If you omit the sheet name, it will dump all sheets into an order map of array of maps.

Features

• [x] Reads a formatted YAML string and renders a Markdown Table
• [x] Can take an optional list of column headings, and only display those from the table (filtering out other columns present)
• [X] Native Binary Command Line (windows, linux, osx)
• [X] Read an XLSX file and produce a Markdown Table
• [X] Read an ODS file and produce a Markdown Table
• [X] Read a CSV
• [X] Published as a Python Module
• [ ] TSV, PSV (etc) file and produce a Markdown Table
• [ ] Support Nested Structures in the YAML input
• [ ] Read a Markdown File, and select the "table" and turn it back into YAML

Future Goals

• Finish the testing and publishing of the JS WASM Bindings. (PS - it works.. (see : test/www-sample and the Makefile )
• Embed the "importing" of YAML, CSV and XLS* files into the mume Markdown Preview Enhanced Plugin. https://shd101wyy.github.io/markdown-preview-enhanced/ So we can have Awesome Markdown Documents.
• Provide a PreRenderer for `rust-lang-nursery/mdBook to "import" MD tables from files.

Known Issues

• A Spreadsheet Cell with a Date will come out as the "magic" Excel date number :-( - https://github.com/tafia/calamine/issues/116

License

Serde is licensed under either of

• Apache License, Version 2.0, (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
• MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)

at your option.

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in Serde by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.