duat-core

The core of Duat, this crate is meant to be used only for the creation of plugins for Duat.

The capabilities of duat-core are largely the same as the those of Duat, however, the main difference is the multi Ui APIs of this crate. In it, the public functions and types are defined in terms of U: Ui, which means that they can work on various different interfaces:

Quick Start

This crate is composed of a few main modules, which will be used in order to extend Duat:

• ui: Has everything to do with the interface of Duat, that includes things like:

  • Widgets: As the name implies, this is the trait for objects that will show up on the screen. The most noteworthy Widget is File, which displays the contents of a file.
  • WidgetCfgs: These are Widget builders. They are used in the setup function of Duat’s config, through the WidgetCreated and WindowCreated hooks.
  • Ui and Areas: These are used if you want to create your own interface for Duat. Very much a work in progress, so I wouldn’t recommend trying that yet.

• text: Defines the struct used to show characters on screen

  • Text: Is everything that Duat shows on screen (except Ui specific decorations). This includes a UTF-8 string and tags to modify it.
  • Tags: This is how Duat determines how Text will be displayed on screen. There are tags for styling, text alignment, spacing and all sorts of other things.
  • txt!: This macro, with syntax reminiscent of format! from Rust’s std, can be used to create Text through the text::Builder struct.

• mode: Defines how Duat will take input in order to control Widgets, includes things like:

  • Modes: have the function send_key, which takes a key and the current widget as input, and decides what to do with them. Only one Mode is active at any given time.
  • map and alias: These functions provide vim-style remapping on a given Mode, also letting you switch modes on key sequences.
  • set, set_default, reset: These functions are used in order to switch Mode on demand. Do note that the switching is done asynchronously.

• hook: Provides utilities for hooking functions in Duat

  • Hookable: An event that you want to provide hooks for, in order to trigger functions whenever it takes place.
  • add, add_grouped, remove: These functions let you add or remove functions from Hookable events. Their arguments are statically determine by said Hookables.

• cmd: Creation of commands in Duat, which can be called at runtime by the user.

  • add!: This macro lets you create a command, with one or more callers, and any number of Parameters
  • Parameter: A command argument parsed from a string. There are a bunch of predefined Parameters, and things like Vec<P> where P: Parameter, can also be as Parameters, if you want multiple of the same kind.
  • call, queue, call_notify, queue_and, etc: functions to call or queue commands, which one should be used depends on the context of the function calling them.

• form: How to stylize Text

  • Form: Has many options on what Text should look like, are the same as those found on unix terminals.
  • set, set_weak: These functions let you set forms with a name. They can be set to a Form or reference another name of a form.
  • ColorSchemes: These are general purpose Form setters, with a name that can be called from the colorscheme command

These are the elements available to you if you want to extend Duat. Additionally, there are some other things that have been left out, but they are available in the prelude, so you can just import it:

// Usually at the top of the crate, below `//!` comments:
use duat_core::prelude::*;

How to extend Duat

Duat is extended primarily through the use of Plugins from external crates, these will be plugged in the main config through the plug! macro, and are modified in place through the builder pattern.

For this demonstration, I will create a Plugin that keeps track of the word count in a File, without reparsing it every time said File changes.

Creating a Plugin

First of all, assuming that you have succeeded in following the installation instructions of duat, you should create a crate with cargo init:

cargo init --lib duat-word-count
cd duat-word-count

Wihin that crate, you’re should add the duat-core dependency:

cargo add duat-core

Or, if you’re using git dependencies:

cargo add duat-core --git https://github.com/AhoyISki/duat

Finally, you can remove everything in duat-word-count/src/lib.rs and start writing your plugin.

// In duat-word-count/src/lib.rs
use duat_core::prelude::*;

/// A [`Plugin`] to count the number of words in [`File`]s
pub struct WordCount;

impl<U: Ui> Plugin<U> for WordCount {
    fn plug(self) {
        todo!();
    }
}

In the example, WordCount is a plugin that can be included in Duat’s config crate. It will give the user the ability to get how many words are in a File, without having to reparse the whole buffer every time, given that it could be a very large file. In order to configure the Plugin, you should make use of the builder pattern, returning the Plugin on every modification.

use duat_core::prelude::*;

/// A [`Plugin`] to count the number of words in [`File`]s
pub struct WordCount(bool);

impl WordCount {
    /// Returns a new instance of the [`WordCount`] plugin
    pub fn new() -> Self {
        WordCount(false)
    }

    /// Count everything that isn't whitespace as a word character
    pub fn not_whitespace(self) -> Self {
        WordCount(true)
    }
}

impl<U: Ui> Plugin<U> for WordCount {
    fn plug(self) {
        todo!();
    }
}

Now, there is an option to exclude only whitespace, not just including regular alphanumeric characters. This would count, for example “x(x^3 + 3)” as 3 words, rather than 4.

Next, I need to add something to keep track of the number of words in a File. For Files specifically, there is a built-in way to keep track of changes through the Parser trait:

use duat_core::prelude::*;

/// A [`Parser`] to keep track of words in a [`File`]
struct WordCounter {
    words: usize,
    regex: &'static str,
}

impl<U: Ui> Parser<U> for WordCounter {
    fn parse(&mut self, pa: &mut Pass, snap: FileSnapshot, ranges: Option<&mut Ranges>) {
        todo!();
    }
}

Whenever changes take place in a File, those changes will be reported in a Moment, which is essentially just a list of Changes that took place. This Moment, in a FileSnapshot, will be sent to the Parser::parse function, in which you are supposed to change the internal state of the Parser to accomodate the Changes.

The FileSnapshot gives you a “snapshot” of what the File looked like after said Moment took place. It includes the Moment in question, the Bytes of the File’s Text, and the PrintCfg at that moment in time.

First, I’m going to write a function that figures out how many words were added or removed by a Change:

use duat_core::{prelude::*, text::Change};

fn word_diff(regex: &str, bytes: &Bytes, change: Change<&str>) -> i32 {
    let [start, _] = bytes.points_of_line(change.start().line());
    let [_, end] = bytes.points_of_line(change.added_end().line());

    // Recreate the line as it was before the change
    // behind_change is just the part of the line before the point
    // where a change starts.
    // ahead_of_change is the part of the line after the end of
    // the Change
    let mut behind_change = bytes.strs(start..change.start()).unwrap().to_string();
    let ahead_of_change = bytes.strs(change.added_end()..end).unwrap();
    // change.taken_str() is the &str that was taken by the Change
    behind_change.push_str(change.taken_str());
    // By adding these three together, I now have:
    // {behind_change}{change.taken_str()}{ahead_of_change}
    // Which is what the line looked like before the Change happened
    behind_change.extend(ahead_of_change);

    // Here, I'm just counting the number of occurances of the
    // regex in the line before and after the change.
    let words_before = behind_change.search_fwd(regex, ..).unwrap().count();
    let words_after = bytes.search_fwd(regex, start..end).unwrap().count();

    words_after as i32 - words_before as i32
}

In this method, I am calculating the difference between the number of words in the line before and after the Change took place. Here Bytes::points_of_line returns the Points where a line starts and ends. I know there are better ways to do this by comparing the text that was taken to what was added, with the context of the lines of the change, but this is just a demonstration, and the more efficient method is left as an exercise to the viewer 😉.

Now, just call this on parse:

use duat_core::{prelude::*, text::Change};

/// A [`Parser`] to keep track of words in a [`File`]
struct WordCounter {
    words: usize,
    regex: &'static str,
}

impl<U: Ui> Parser<U> for WordCounter {
    fn parse(&mut self, pa: &mut Pass, snap: FileSnapshot, _: Option<&mut Ranges>) {
        // Rust iterators are magic 🪄
        let diff: i32 = snap
            .moment
            .changes()
            .map(|change| word_diff(self.regex, &snap.bytes, change))
            .sum();

        self.words = (self.words as i32 + diff) as usize;
    }
}

And that’s it for the Parser implementation! Now, how do we add it to a File?

In order to add this Parser to a File, we’re going to need a ParserCfg, which is used for configuring Parsers before they are added:

use duat_core::prelude::*;

struct WordCounterCfg(bool);

impl<U: Ui> ParserCfg<U> for WordCounterCfg {
    type Parser = WordCounter;

    fn init(self, file: &File<U>) -> Result<ParserBox<U>, Text> {
        let regex = if self.0 { r"\S+" } else { r"\w+" };
        let words = file.bytes().search_fwd(regex, ..).unwrap().count();

        let word_counter = WordCounter { words, regex };
        Ok(ParserBox::new(file, word_counter))
    }
}

In this function, I am returning the WordCounter, with a precalculated number of words (since I have to calculate this value at some point), based on the current state of the File.

The ParserBox return value is a wrapper for “constructing the Parser”. To create a ParserBox, there are two functions: new and new_remote. The first one is essentially just a wrapper around the Parser. The second one takes a closure that will build the Parser in a second thread, this can be useful if you want to create your Parser remotely.

One thing to note is that the Parser and ParserCfg can be the same struct, it all depends on your constraints. For most Parser implementations, that may not be the case, but for this one, instead of storing a bool in WordCounterCfg, I could’ve just stored the regex directly, like this:

use duat_core::prelude::*;

impl WordCounter {
    /// Returns a new instance of [`WordCounter`]
    pub fn new() -> Self {
        WordCounter { words: 0, regex: r"\w+" }
    }
}

impl<U: Ui> ParserCfg<U> for WordCounter {
    type Parser = Self;

    fn init(self, file: &File<U>) -> Result<ParserBox<U>, Text> {
        let words = file.bytes().search_fwd(self.regex, ..).unwrap().count();

        Ok(ParserBox::new(file, Self { words, ..self }))
    }
}

But the former is done for the purpose of demonstration, since (I don’t think) this will be the case for most Parsers.

Now, to wrap this all up, the plugin needs to add this Parser to every opened File. We do this through the use of a hook:

use duat_core::prelude::*;

/// A [`Plugin`] to count the number of words in [`File`]s
pub struct WordCount(bool);

impl WordCount {
    /// Returns a new instance of the [`WordCount`] plugin
    pub fn new() -> Self {
        WordCount(false)
    }

    /// Count everything that isn't whitespace as a word character
    pub fn not_whitespace(self) -> Self {
        WordCount(true)
    }
}

impl<U: Ui> Plugin<U> for WordCount {
    fn plug(self) {
        let not_whitespace = self.0;

        hook::add::<File<U>, U>(move |pa, (mut cfg, builder)| {
            cfg.with_parser(WordCounterCfg(not_whitespace))
        });
    }
}

Now, whenever a File is opened, this Parser will be added to it. This is just one out of many types of hook that Duat provides by default. In Duat, you can even create your own, and choose when to trigger them.

However, while we have added the Parser, how is the user supposed to access this value? Well, one convenient way to do this is through a simple function:

use duat_core::prelude::*;

/// The number of words in a [`File`]
pub fn file_words<U: Ui>(file: &File<U>) -> usize {
    file.read_parser(|word_counter: &WordCounter| word_counter.words)
        .unwrap_or(0)
}

Now, we have a finished plugin:

use duat_core::{prelude::*, text::Change};

/// A [`Plugin`] to count the number of words in [`File`]s
pub struct WordCount(bool);

impl WordCount {
    /// Returns a new instance of [`WordCount`]
    pub fn new() -> Self {
        WordCount(false)
    }

    /// Count everything that isn't whitespace as a word character
    pub fn not_whitespace(self) -> Self {
        WordCount(true)
    }
}

impl<U: Ui> Plugin<U> for WordCount {
    fn plug(self) {
        let not_whitespace = self.0;

        hook::add::<File<U>, U>(move |_, (mut cfg, _)| {
            cfg.with_parser(WordCounterCfg(not_whitespace))
        });
    }
}

/// The number of words in a [`File`]
pub fn file_words<U: Ui>(file: &File<U>) -> usize {
    file.read_parser(|word_counter: &WordCounter| word_counter.words)
        .unwrap_or(0)
}

/// A [`Parser`] to keep track of words in a [`File`]
struct WordCounter {
    words: usize,
    regex: &'static str,
}

impl<U: Ui> Parser<U> for WordCounter {
    fn parse(&mut self, pa: &mut Pass, snap: FileSnapshot, _: Option<&mut Ranges>) {
        let diff: i32 = snap
            .moment
            .changes()
            .map(|change| word_diff(self.regex, &snap.bytes, change))
            .sum();

        self.words = (self.words as i32 + diff) as usize;
    }
}

struct WordCounterCfg(bool);

impl<U: Ui> ParserCfg<U> for WordCounterCfg {
    type Parser = WordCounter;

    fn init(self, file: &File<U>) -> Result<ParserBox<U>, Text> {
        let regex = if self.0 { r"\S+" } else { r"\w+" };

        let words = file.bytes().search_fwd(regex, ..).unwrap().count();

        Ok(ParserBox::new(file, WordCounter { words, regex }))
    }
}

fn word_diff(regex: &str, bytes: &Bytes, change: Change<&str>) -> i32 {
    let [start, _] = bytes.points_of_line(change.start().line());
    let [_, end] = bytes.points_of_line(change.added_end().line());

    // Recreate the line as it was before the change
    let mut line_before = bytes.strs(start..change.start()).unwrap().to_string();
    line_before.push_str(change.taken_str());
    line_before.extend(bytes.strs(change.added_end()..end).unwrap());

    let words_before = line_before.search_fwd(regex, ..).unwrap().count();
    let words_after = bytes.search_fwd(regex, start..end).unwrap().count();

    words_after as i32 - words_before as i32
}

Once you’re done modifying your plugin, you should be ready to publish it to crates.io. This is the common registry for packages (crates in Rust), and is also where Duat will pull plugins from. Before publishing, try to follow these guidelines in order to improve the usability of the plugin. Now, you should be able to just do this in the duat-word-count directory:

cargo publish

Ok, it’s published, but how does one use it?

Using plugins

Assuming that you’ve already installed duat, you should have a config crate in ~/.config/duat (or $XDG_CONFIG_HOME/duat), in it, you can call the following command:

cargo add duat-word-count@"*" --rename word-count

Then, in src/lib.rs, you can add the following:

setup_duat!(setup);
use duat::prelude::*;
use word_count::*;

fn setup() {
    plug!(WordCount::new().not_whitespace());

    hook::add::<StatusLine<Ui>>(|pa, (sl, _)| {
        sl.replace(status!(
            "{file_txt} has [wc]{file_words}[] words{Spacer}{mode_txt} {sels_txt} {main_txt}"
        ))
    });
}

Now, the default StatusLine should have word count added in, alongside the other usual things in there. It’s been added in the {file_words} part of the string, which just interpolated that function, imported by use word_count::*;, into the status line.

There are many other things that plugins can do, like create custom Widgets, Modes that can change how Duat behaves, customized commands and hooks, and many such things

Plugin examples

duat-sneak

duat-sneak, inspired by vim-sneak, lets you traverse the screen by searching through character sequences.