kronicler

Automatic performance capture and analysis for production applications in Python using a custom columnar database written in Rust.

[!IMPORTANT] Kronicler is still early in development! Currently you can install and try out the logging. Analysis features are coming soon.

Benefits of using Kronicler

• Automatic performance capturing
• Lightweight and concurrent*
• One Python dependency
• Works out-of-the-box without configuration

* concurrency is in development but not fully implemented as of version 0.1.0. Track concurrency in issue #41.

Why use Kronicler?

If you want to monitor the performance of a production application, kronicler offers efficient and lightweight logging with a single library. Kronicler lets you view runtime statistics for functions like mean and median as well as statistics for different percentiles.

A use-case for these statistics is to find functions that occasionally operate much slower than they do on average. By looking at the "slowest" speed, the standard error, and the mean, you can find functions that occasionally run much slower than expected. Sometimes it's hard to find and replicate these issues in a test environment, so keeping logs in your production application can improve your ability to find these issues.

Install (Python)

Install with Pip for Python

pip install kronicler

Usage (Python)

Kronicler provides a Python decorator called capture that will calculate the time it takes to run the function.

import kronicler

@kronicler.capture()
def my_function():
	pass

Architecture

Simplified version of the package and database architecture. The data is passed from the Python decorator called capture to the database's queue. It then consumes that queue to insert each field into its respective column. The column uses the bufferpool to operate on pages.

This does not include details on:

• How the bufferpool manages pages.
• How pages operate.
• capture, index, row, or saving and loading with metadata.

The Database

The columnar database is somewhat inspired by my previous database called RedoxQL. A lot of the structure and architecture is different as well as how data is stored.

The Bufferpool

The bufferpool is based on my bufferpool project. I had to modify it to work with the rest of this database.

Future Languages

Install and Usage for Rust is coming soon...

I plan to implement the Rust version as an attribute to be used like the following:

#[capture]
fn foo() {
	todo!()
}

Examples

Using Kronicler with FastAPI

With just two lines of code, you can add Kronicler to your FastAPI server.

from fastapi import FastAPI
import uvicorn
import kronicler

app = FastAPI()


# You need to wrap helper functions
@kronicler.capture
def foo():
    return {"Hello": "World"}

# You cannot wrap routes right now
@app.get("/")
def read_root():
    return foo()

if __name__ == "__main__":
    uvicorn.run(app, host="0.0.0.0", port=8000)

Code from tests/fastapi-test/main.py.

Using Kronicler's database directly

If you're interested in using Kronicler's database directly to add custom logging functions (or just to use a columnar database), the library is published to crates.io.

Install with Cargo for Rust

cargo install kronicler

Add as a dependency in your Cargo.toml.

[dependencies]
kronicler = "0.1.0"

Performance

Kronicler is designed to be as lightweight as possible. By adding logs to a queue concurrently*, Kronicler doesn't affect performance by much [citation needed].

For accessing logs and running calculations, Kronicler uses a columnar database design to optimize file operations when looking at lots of data from only a few columns typical of analytics tasks.

* concurrency is in development but not fully implemented as of version 0.1.0. Track concurrency in issue #41.

Analysis Web Dashboard

The Analysis Web Dashboard is still under construction. This feature will let you remotely view the logs collected from Kronicler.

Mock-up

The Web Dashboard may look something like this. It will show important information about all of the functions at the top. It will also include a graph of the functions performance over time.

This mock-up was created with Claude and the future dashboard may vary substantially.

Analytics CLI

Install the Analytics CLI

cargo install kronicler

You can view all of your data by running kr in the directory of your data:

kr --fetch all

kr --fetch <index>

You should see the data collected:

In the future, there will be many options for sorting, filtering, and viewing specific statistics.

Logging

By adding the capture decorator to your code (as seen below), Kronicler will automatically test the runtime of your function when it gets called. The results of the test get added to the database. This data can later be viewed in the Analysis Web Dashboard or the Analytics CLI.

import kronicler

@kronicler.capture()
def my_function():
	pass

Development

Building the Python package

Build the package

maturin build

Install the package

pip install --force-reinstall target/wheels/kronicler-*

Testing

You can run the testing suite with the following command:

cargo test

The tests should all succeed