modql

modql is a set of types and utilities designed to structurally express model query filters (e.g., $eq: .. $startsWith: .., $containsIn: [..]) and list options (e.g., offset, limit, order_bys). These can be easily represented in JSON.

In essence, it offers a MongoDB-like filter syntax that is storage-agnostic, has built-in support for sea-query, and can be expressed either in JSON or Rust types.

IMPORTANT: v0.4.0 is RELEASED: This release includes significant refactoring (retaining the same functionality but with cleaner naming and decoupled from sea-query) to allow derive(Fields) to provide field_names and field_refs without requiring the with-sea-query feature.

For more information, see CHANGELOG.md and MIGRATION-v03x-v04x.md.

Thanks

• Chetan Bhasin for PR #8 Use sea-query 0.32

• Andrii Yermakov for PR #2 Fix an errata in null operator for numbers

• Marc Cámara for PR #9 Add Column Casting Support to Filter Nodes

• Marc Cámara for adding the Case Insensitive support and the with-ilike support for Postgresql. PR #3

• Andrii Yermakov for fixing a "null operator for numbers" PR #2

Quick Overview

/// This is the model entity, annotated with Fields.
#[derive(Debug, Clone, modql::field::Fields, FromRow, Serialize)]
pub struct Task {
    pub id: i64,
    pub project_id: i64,

    pub title: String,
    pub done: bool,
}

/// This is a Filter, with the modql::filter::OpVals... properties
#[derive(modql::filter::FilterNodes, Deserialize, Default, Debug)]
pub struct TaskFilter {
    project_id: Option<OpValsInt64>,
    title: Option<OpValsString>,
    done: Option<OpValsBool>,
}

// -- Parsing JSON representation to TaskFilter
// This condition requires all of these rules to match (AND).
let list_filter: TaskFilter = serde_json::from_value(json! ({
    "project_id": 123,
    "title": {"$startsWith": "Hello", "$contains": "World"} ,
}))?;

// -- modql ListOptions
let list_options: modql::filter::ListOptions =
    serde_json::from_value(json! ({
        "offset": 0,
        "limit": 2,
        "order_bys": "!title" // ! for descending
    }))?;

// -- Building a sea-query select query with those condition
// Convert the TaskFilter into sea-query condition
let cond: sea_query::Condition = filter.try_into()?;
let mut query = sea_query::Query::select();

// Select only the columns corresponding to the task type.
// This is determined by the modql::field::Fields annotation.
query.from(task_table).columns(Task::sea_column_refs());

// Add the condition from the filter
query.cond_where(cond);

// Apply the list options
list_options.apply_to_sea_query(&mut query);

// and execute query
let (sql, values) = query.build_sqlx(PostgresQueryBuilder);
let entities = sqlx::query_as_with::<_, E, _>(&sql, values)
    .fetch_all(db)
    .await?;

This crate is instrumental for JSON-RPC or other types of model APIs (e.g., the joql pattern).

IMPORTANT v0.3.x represents the new version of modql, featuring the with-sea-query feature set. It is utilized in the rust10x web-app production code blueprint Episode 02. This version is somewhat incompatible with v0.2.x, mainly due to module reorganization. If you are using the rust10x/awesomeapp desktop app, please stick with v0.2.x for the time being. I plan to upgrade the codebase to v0.3.x soon.

changelog

OpVal[Type] Conditional Operators

OpVal[Type] is a filter unit that allows the expression of an operator on a given value for a specified type.

The corresponding OpVals[Type], with an "s", is typically used in filter properties, as it permits multiple operators for the same field.

The basic JSON representation of an OpVal[Type] follows the {field_name: {$operator1: value1, $operator2: value2}} format. For example:

{
    "title": {"$startsWith": "Hello", "$contains": "World"}
}

This expresses the conditions that both "startsWith" and "contains" must be met.

The following tables show the list of possible operators for each type.

OpValString Operators

OpValInt32, OpValInt64, OpValFloat64 Operators

OpValBool Operators

More Info

• modql::filter - Delivers a declarative structure that can be deserialized from JSON.
• modql::field - Provides a method get field information on a struct. The with-sea-query feature add sea-query compatible data structure from standard structs and derive.

#[derive(modql::field::Fields) provide the following

• Task::field_names() returns the property names of the struct. It can be overridden with the #[field(name="another_name")] property attribute.
• Task::field_refs() returns FieldRef { name: &'static str, rel: Option<&'static str>} for the properties. rel acts like the table name. It can be set as #[modql(rel="some_table_name")] at the struct level, or #[field(rel="special_rel_name")] at the field level.

When compiled with the with-sea-query feature, these additional functions are available on the struct:

• Task::sea_column_refs() -> Vec<ColumnRef>: Constructs sea-query select queries (with rel as the table, and name as the column name).
• Task::sea_idens() -> Vec<DynIden>: Constructs sea-query select queries, suited for simpler cases. (similar to ::field_names() but returns the sea-query DynIden).
• task.all_sea_fields().for_sea_insert() -> (Vec<DynIden>, Vec<SimpleExpr>): Used for sea-query inserts.
• task.all_sea_fields().for_sea_update() -> impl Iterator<Item = (DynIden, SimpleExpr)>: Used for sea-query updates.

Additionally, it offers:

• task.not_none_fields(): Operates similarly to the above, but only for fields where their Option is not None.

Rust types

On the Rust side, this can be expressed like this:

pub type Result<T> = core::result::Result<T, Error>;
pub type Error = Box<dyn std::error::Error>; // For early dev.
use modql::filter::{FilterGroups, FilterNode, OpValtring};

fn main() -> Result<()> {
    let filter_nodes: Vec<FilterNode> = vec![
        (
            "title",
            OpValtring::ContainsAny(vec!["Hello".to_string(), "welcome".to_string()]),
        )
            .into(),
        ("done", true).into(),
    ];
    let filter_groups: FilterGroups = filter_nodes.into();

    println!("filter_groups:\n{filter_groups:#?}");

    Ok(())
}

A Model or Store layer can take the filter_groups and serialize them into their DSL (e.g., SQL for databases).

The Filter structure is as follows:

• FilterGroups is the top level and consists of multiple FilterGroup elements. FilterGroup elements are intended to be executed with an OR operation between them.
• Each FilterGroup contains a vector of FilterNode elements, which are intended to be executed with an AND operation.
• FilterNode contains a rel (not used yet), name which represents the property name from where the value originates, and a Vec<OpVal>, representing the Operator Value.
• OpVal is an enum for type-specific OpVal[Type] entities, such as OpValString that holds the specific operation for that type along with the associated pattern value.

GitHub Repo