deepwell

DEEPWELL

DEEPWELL is an internal backend system to provide core wiki operations for Wikijump. This is intended as an internal API consumed by the web server as part of its logical tasks, and as such it is the job of the web server to verify that the calling user has the permissions to perform the given task. DEEPWELL will assume all requests are trusted and perform them.

The lint #![forbid(unsafe_code)] is set, and therefore this crate has only safe code.

Available under the terms of the GNU Affero General Public License. See LICENSE.md.

Development

If you have sea-orm-cli installed, and have a local instance of Wikijump running, you can use the following script to autogenerate SeaORM model files:

$ scripts/generate-models.sh

Structure

The primary organization of the crate is as follows:

• api/ — Web server definition, such as its routes and related structures.
  • Exposes the internal API for use by Framerail.
• endpoints/ — Implementations for individual endpoints described above.
• services/ — "Services", or logical encapsulations of different concepts or operations.
  • For instance, the ParentService allows retrieving and storing data related to parent-child page relationships. You can think of it as "wrapping" the page_parent table.
  • Similarly, the PageService encapsulates the page table, but also wraps all the other operations contained with the logical concept of the "page", such as creating new revisions using the RevisionService as part of the "edit" method.
  • Structure definitions, organized per-service, are found here. For instance, a structured used as the body of a request is likely to be defined here as well. Some structures particular to routes are defined inline.
• locales/ — Provides localization methods, interpreting the Fluent translation files.
• models/ — Primarily auto-generated by Sea-ORM, these files allow for interfacing with the database.
• utils/ — Backend-wide utilities and other tools, divided into sections.
• web/ — For some enums, structures, and other concepts which are used throughout, usually related to interfacing with the web API in some way.
• Other modules are utilities, and are generally understandable by looking at their contents.

The routes are defined in api/, with their implementations in methods/, and the structures they rely on in services/<name>/structs.rs. The services invoked to encapsulate logical operations are at services/<name>.rs or services/<name>/service.rs.

Compilation

This executable targets the latest stable Rust. At time of writing, that is 1.77.0.

At present the crate has one feature:

• notify — Enables a feature to track the locale directory and configuration file, reloading the server if they are modified. This should be used in local builds only, not in production.

$ cargo build [--features ...] [--release]

Execution

$ cargo build --features <deploy|local> -- [-q] [-p <port>] <config-file>

There are a number of arguments beyond the ones shown. Run with --help for all relevant information.

This runs a local instance of DEEPWELL with the given configuration file. When debugging (i.e. on --features local only), you can also pass in -w or --watch-files to have the process to restart automatically when the configuration file or any localization files change.

This does not seem to work with Docker, so you should instead manually stop the api container and run it locally with the flag. That will properly watch changes and restart itself.

Testing

Tests have not yet been implemented, but when they are, run:

$ cargo test

Add -- --nocapture to the end if you want to see test output.

Linting

$ cargo fmt     # Ensure code is formatted
$ cargo clippy  # Check code for lints

Database

There are two important directories related to the management of the database (which DEEPWELL can be said to "own"). They are both fairly self-explanatory:

• migrations/ — A series of SQL files to set up the schema for a new database.
• seeder/ — Data to initially seed a fresh instance with.
• This is not part of the migrations system, instead using DEEPWELL services and methods to ensure that all invariants are properly set, rather than them needing to be manually provided as raw rows in migration files.
• This also makes modifying the initial state of an instance much easier, since editing the default start page only requires editing a regular text file.

Whether migrations and the seeder run on startup are controlled via configuration. This means they can be set by either:

• The RUN_MIGRATIONS environment variable, or the --run-migrations command-line flag.
• The RUN_SEEDER environment variable, or the --run-seeder command-line flag.

Both of these are enabled by default for local installations.

Basic database setup (creating the wikijump database and user) is done when building the container. See /install/files/postgres/init/.

Adding new migrations

Database migrations are applied using sqlx migrations, which uses the simple structure that can be seen in the migrations/ directory. New migrations can be added using the sqlx command-line utility:

$ sqlx migrate add [name_of_migration]

Migration names should be all-lowercase, snake_case, and limited to 1-3 words.

Note that, until Wikijump has a production deployment, migrations are being condensed to simplify the database system. Incremental migrations are necessary to avoid wrecking a production environment, but until one exists it does not make sense to go through the extra work of splitting all changes.

You can see sqlx migrate --help or sqlx-cli for more information.