polite

The core rusqlite × Polars bridge. polite makes it easy to move data between SQLite databases and Polars DataFrames.

Features

• Open SQLite databases (file-based only).
• Execute arbitrary SQL statements.
• Bulk-load query results into Polars DataFrames (to_dataframe) via ConnectorX.
• Write Polars DataFrames into SQLite tables (from_dataframe).

Dependencies

• Built against Polars 0.49.1 with a lightly patched fork of the latest release of ConnectorX (0.4.4, configured for only the SQLite source and Arrow destination).
  • Pins chrono =0.4.39 due to an upstream Arrow/Polars issue (this will be removed once the conflict is resolved there).

Limitations (MVP)

• Supported SQLite column types: INTEGER → Polars Int64 REAL → Polars Float64 TEXT → Polars String
• Other SQLite types are stored as String.
• Output uses Polars’ standard debug DataFrame format.
• No advanced type inference or schema evolution yet.

⚠️ Notes on SQLite backends

• polite uses ConnectorX for bulk reads into Polars.
• File-backed databases (.sqlite, .db) are required.
• In-memory databases (:memory:) are not supported — use a tempfile if you don’t want persistence.

Core functions

💡 All of these functions are also available via use polite::prelude::*;

The two basic functions provided by the library are:

• to_dataframe(db_path, sql) – run a query and return a DataFrame.
• from_dataframe(&conn, table, &df) – write a DataFrame into a table. Takes an open rusqlite::Connection, the table name to write to, and your DataFrame.

polite also provides a couple of convenience wrappers (with simplified string errors and without connection handling):

• save_dataframe(db_path, table, &df)
  Opens a connection and writes the DataFrame in one step.
  Creates and closes its own connection; use this for one-off saves.

• load_dataframe(db_path, sql)
  Wraps to_dataframe but adds context to errors (e.g. "Failed to load DataFrame from demo.db: no such table: users").
  This makes it clearer where the failure came from, especially if you’re working with multiple databases.

Why use these helpers?

These helpers don’t add new capabilities beyond the core API, but they provide more ergonomic errors.

The raw API (to_dataframe, from_dataframe) exposes detailed error variants (Query, Arrow, Polars, rusqlite, etc.), which is useful if you want to distinguish exactly what failed.

The convenience wrappers (load_dataframe, save_dataframe) normalize those into a single error variant per operation:

1. load_dataframe always yields PoliteError::Load
2. save_dataframe always yields PoliteError::Save

• ✅ You don’t have to juggle Query, Arrow, ArrowToPolars variants of PoliteError, rusqlite::Error etc.
• ✅ They’re the "safe default" for people who just want “load/save a DataFrame” and don’t care which stage failed.
• ✅ Advanced users can drop down to to_dataframe / from_dataframe for finer control and granular error inspection.

In practice, wrappers are the recommended default for most use cases. Drop down to the raw API when you want maximum control.

🎤 Demo time

use polite::prelude::*;
use polars::prelude::*;

fn main() -> anyhow::Result<(), String> {
    // Open (or create) a SQLite database
    let db_path = "polite.db";
    let conn = connect_sqlite(Some(db_path))?;

    execute_query(&conn, "CREATE TABLE friends_made (id INTEGER, name TEXT)")?;

    let nobody = load_dataframe(db_path, "SELECT * FROM friends_made")?;
    println!("🤓 I am making friends in SQLite! I don't have any there yet...\n{nobody:?}");

    // Create a table to keep your friends' names in
    execute_query(&conn, "INSERT INTO friends_made VALUES (1, 'Alice')")?;
    execute_query(&conn, "INSERT INTO friends_made VALUES (2, 'Bob')")?;
    execute_query(&conn, "INSERT INTO friends_made VALUES (3, 'Charlie')")?;

    // Query your friends back into a Polars DataFrame
    let dbf = to_dataframe(db_path, "SELECT * FROM friends_made")?;
    println!("🪄 I have lovingly restored my friends into a Polars DataFrame:\n{dbf:?}");

    // Add some more friends directly from a Polars DataFrame
    let polars_friends = df! {
        "id" => [4_i64, 5], // careful with dtypes: Polars will use i32 by default here!
        "name" => ["Dora", "Eve"],
    }?;
    
    from_dataframe(&conn, "cool_friends", &polars_friends)?;

    println!("🆒 My friends from Polars are now my friends in SQLite:\n{polars_friends:?}");

    // Combine both tables into one DataFrame.
    // ⚠️ If the `cool_friends.id` column was created as `Int32` in Polars,
    // SQLite may widen or nullify values when UNIONing with `friends_made.id`
    // (which is `INTEGER` = i64). Use `_i64` suffix in Polars literals to match.
    let all_friends = load_dataframe(
        db_path,
        "SELECT * FROM friends_made UNION ALL SELECT * FROM cool_friends ORDER BY id",
    )?;
    println!("🎉 All my friends are politely gathered in a DataFrame:\n{all_friends:?}");

    Ok(())
}

🤓 I am making friends in SQLite! I don't have any there yet...
shape: (0, 2)
┌─────┬──────┐
│ id  ┆ name │
│ --- ┆ ---  │
│ i64 ┆ str  │
╞═════╪══════╡
└─────┴──────┘
🪄 I have lovingly restored my friends into a Polars DataFrame:
shape: (3, 2)
┌─────┬─────────┐
│ id  ┆ name    │
│ --- ┆ ---     │
│ i64 ┆ str     │
╞═════╪═════════╡
│ 1   ┆ Alice   │
│ 2   ┆ Bob     │
│ 3   ┆ Charlie │
└─────┴─────────┘
🆒 My friends from Polars are now my friends in SQLite:
shape: (2, 2)
┌─────┬──────┐
│ id  ┆ name │
│ --- ┆ ---  │
│ i64 ┆ str  │
╞═════╪══════╡
│ 4   ┆ Dora │
│ 5   ┆ Eve  │
└─────┴──────┘
🎉 All my friends are politely gathered in a DataFrame:
shape: (5, 2)
┌─────┬─────────┐
│ id  ┆ name    │
│ --- ┆ ---     │
│ i64 ┆ str     │
╞═════╪═════════╡
│ 1   ┆ Alice   │
│ 2   ┆ Bob     │
│ 3   ┆ Charlie │
│ 4   ┆ Dora    │
│ 5   ┆ Eve     │
└─────┴─────────┘

Type system

Note that the type system used by rusqlite via ConnectorX is as shown here

Integration

• Use this library in Rust projects that need to bridge SQLite and Polars.
• For a quick playground, see the CLI.

Documentation

• Crate docs: docs.rs/polite
• Workspace guide: DEVELOPMENT.md

License

Licensed under the MIT License. See LICENSE for details.