Crates.io | allmytoes |
lib.rs | allmytoes |
version | 0.4.0 |
source | src |
created_at | 2022-07-20 20:05:54.323945 |
updated_at | 2024-05-05 16:05:28.526771 |
description | provides thumbnails according to the freedesktop.org specification |
homepage | https://gitlab.com/allmytoes/allmytoes |
repository | https://gitlab.com/allmytoes/allmytoes |
max_upload_size | |
id | 629147 |
size | 1,076,307 |
Provides thumbnails by using the freedesktop-specified thumbnail data-base (aka XDG standard).
Thumbnails are shared with other programs via a common cache.
The (expensive) creation of thumbnails is only done if a thumbnails has not been created yet or if the original file has changed.
AllMyToes can create thumbnails for many image formats. (See “Decoding” support of the image crate.)
Additionally, AllMyToes can create thumbnails for other kind of files using other programs to create the base image. (See section Provider.)
AllMyToes is both, a non-interactive program for the shell and scripts, and a Rust library. It's abbreviated “AMT” (or “amt”) in code or structured data.
[[TOC]]
Call allmytoes
with a file as parameter. The program will print the path to a thumbnail to stdout
.
> allmytoes some_image.jpg
/home/me/.cache/thumbnails/large/b7931d1d6e0439c1a6e2e6b02c5b21a6.png
If the thumbnail already exists, AllMyToes will just return the path to the existing thumbnail file. If not, AllMyToes will create the image’s thumbnail first.
The latest freedesktop.org specification (0.9.0) defines four different thumbnail sizes: normal, large, x-large, and xx-large with maximum edge lengths of 128 px, 256 px, 512 px, and 1024 px respectively.
By default, AllMyToes returns the path to the large (256 px) thumbnail.
The size to be returned can be chosen with the -s
(--size
) option by giving a value out
of {n
, l
, x
, xx
}.
For example, to get a xx-large thumbnail, the call would look like this:
> allmytoes -sxx some_image.jpg
/home/me/.cache/thumbnails/xx-large/ad0779df58de36f038bdc4040a322bfe.png
If the input image is smaller than the requested thumbnail size, the thumbnail will have the same dimensions as the input image. A thumbnail will never be bigger than the original image.
If a thumb for the requested size does not already exist, AllMyToes will check if the input image is smaller than the requested thumb size. If so, AllMyToes will determine the smallest thumb size that still covers the input image dimensions (is bigger or equal than the input image). If that “feasible” thumb size is different from the requested thumb size, AllMyToes will start the job all over with that smaller, “feasible” thumb size. AllMyToes does that to avoid unnecessary, duplicated thumbnails for small images.
That implies that for small input images, the returned thumbnail path might not correspond to the requested thumb size, because creating and keeping thumbnails for the (bigger) requested size would be a waste of computation time and disk space.
If – for whatever reason – one must get a result path to the thumbnail of the requested size,
one can use the -F
(--force-size
) switch.
However, one should do that only if there is a really good reason.
Calling AllMyToes with the --extensive
(or -e
) option makes it print out
more information than just the path to the thumb. The provided information is
It does not matter if the thumb did already exist or if it has just been created by AllMyToes.
All data is printed as a list of key-value pairs, each tuple in one line.
The key is separated from the value by “:
”. Key-value pairs which show meta
data from the thumbnail file are preceded with a colon and are printed in
alphabetical order.
Example:
$ allmytoes --extensive some_image.jpg
Thumb path: /home/dude/.cache/thumbnails/large/c000276b8378a33e16e3fac005eebc02.png
Thumb hash: c000276b8378a33e16e3fac005eebc02
:Software: allmytoes
:Software::Version: 0.1.0
:Thumb::Image::Dimensions: 4608x3456
:Thumb::Image::Height: 3456
:Thumb::Image::Reorientation: Rotated 180 degrees, not flipped
:Thumb::Image::Width: 4608
:Thumb::MTime: 1693737198
:Thumb::Mimetype: image/jpeg
:Thumb::Size: 7777807
:Thumb::URI: file:///path/to/some_image.jpg
What meta-data is available in the thumb of course depends on the software which
created the thumb. So, the lines starting with a colon (:
) may vary.
By default, AllMyToes will print warnings and errors to stderr
.
A different log-level can be set via the environment variable RUST_LOG
.
For example, do a
export RUST_LOG=trace
to get all log entries, down to the most detailed “trace”-level.
AllMyToes can create thumbnails for non-image files if a “provider” is defined. A provider is a shell-command or a script that provides an image for a certain set of MIME types.
AllMyToes has inbuilt support for some file formats:
Format | Dependencies |
---|---|
(evince-thumbnailer or magick ) and (exiftool or pdfinfo or gs ) |
|
Postscript | (evince-thumbnailer or magick ) and (exiftool or gs ) |
SVG | inkscape or magick |
various video formats | ffmpeg (and optionally ffprobe & magick & bc for more complex thumbs) and (ffprobe or (mediainfo and bc ) or (exiftool and bc )) |
The exact MIME-types and the provider definitions can be found in /conf/provider.yaml
.
Users can use their own provider configuration in a YAML file to add support for more MIME types or to implement more fancy thumbnails.
To do so, copy /conf/provider.yaml
to ~/.config/allmytoes/provider.yaml
(or $XDG_CONFIG_HOME/allmytoes/provider.yaml
if $XDG_CONFIG_HOME
is defined)
and adapt it.
You can also use the -p
(--provider-config-file
) option to explicitly choose another provider configuration.
❗ Be aware that the provider-feature is pretty new and the format of the provider configuration will likely change here and there before it stabilizes.
The provider configuration YAML file has a list of dictionaries on the top level. Each list entry defines a provider for one or more MIME types. For each entry, there are two mandatory and two optional keys.
Key | Mandatory | Short description |
---|---|---|
mimes |
Yes | List of mime types, handled by the provider definition |
commands |
Yes | List of commands providing the thumbnail, processed top down until one suceeds |
meta |
No | Dictionary mapping thumb-meta-keys to a list of commands to provide them |
revision |
No | integer to track the version of the provider definition |
mimes
The mime type list defines for which mime types the provider is executed.
MIME types are specifies as <type>/<subtype>
, like video/x-matroska
.
If AllMyToes is called for a file for which no provider can be found,
the MIME type is printed in an error message.
MIME types for which a provider is found are printed in a debug message and also
annotated as meta-data in the thumbnail itself.
(Which you can see if you use the --extensive
(-e
) option.)
commmands
Each provider-entry must have a list of commands. Commands are shell commands (or scripts) that take the input file and return an image back to AllMyToes. AllMyToes takes that image and creates the XDG-conforming thumbnail from it. Commands provide that image by copying it to a defined location.
When one of the MIME types from the mimes
list matches the input file,
AllMyToes executes the first command from the commands
list.
If the command succeeds (exit code 0
), AllMyToes stops and uses the thumbnail which
has been provided by that command.
If the command does not succeed (exit code ≠ 0
), AllMyToes tries the next command, and so forth.
This allows to have commands which use different programs to provide the thumb.
If a user does not have the programs used in the first command, another option to calculate the thumb
may work.
If a command returns 0
but still does not provide a valid thumb,
AllMyToes returns with an error.
A command provides the thumb by copying it to a certain path in a temporary directory, which is deleted by AllMyToes afterwards.
Commands need certain information to do their job, for example the input file and the path to copy the thumbnail to. Therefore, commands can use certain variables.
Variable | Content |
---|---|
%(file)s |
The full path to the input file |
%(size)s |
The size in pixels that the longer edge should ideally have. |
%(outfile)s |
The path to the temporary file where the thumbnail shall be copied to |
%(tmpdir)s |
The temporary directory that contains %(outfile)s , and that can be used other temporary files that might be needed during the thumb-creation |
If the created image (at %(outfile)s
) is bigger than %(size)s
, AllMyToes will scale
the image down accordingly.
AllMyToes also does other necessary conversions like turning the image into a 8-bit per channel RGBA PNG
image and add all necessary meta chunks.
So, the commands can provide their thumbnails in any supported image format.
AllMyToes will take care to turn the image into a freedesktop.org-conform thumbnail afterwards.
meta
freedesktop.org-thumbnails have “meta data chunks”, meta-data attributes in key-value pairs.
(See explanation of meta-data chunks.)
Providers can also provide such meta-data entries.
Some are even recommended by the standard
(Thumb::Document::Pages
for paper-oriented documents and Thumb::Movie::Length
for the length of video files in seconds).
The meta
key in the provider specification contains a dictionary,
mapping meta-data keys to lists of commands.
For each key, the list of commands are executed like for the thumb image top down
until one of them exits successfully with 0
.
The value for the meta-chunk has to be returned via stdout
.
With the current implementation, the thumb-generation fails if none of the commands for a meta-chunk is successful. The commands for a meta-chunk can use the same variables as the commands for the image creation.
The specification of meta-chunks is optional.
revision
The revision key is intended as a “version” for the provider. It's optional and as of now, it does not have any effect other than being added as a meta-chunk to the thumbnail.
The idea is that this revision number can be used in the future to re-create thumbs if a provider is available with a higher revision.
One should consider a few things for both, commands for the images in the commands
key,
and for commands under a meta
key.
First, commands can be multi-line shell scripts, simply by using multi-line strings in the YAML file. This example for an SVG-provider shows how multi-line commands may look like:
- mimes:
- image/svg+xml
commands:
- |
which inkscape || exit 1
w=$(inkscape -W "%(file)s" | sed 's/\..*//')
h=$(inkscape -H "%(file)s" | sed 's/\..*//')
test $w -gt $h && size_arg="-w %(size)s" || size_arg="-h %(size)s"
inkscape --export-area-page --export-type=png $size_arg -o "%(tmpdir)s/outfile.png" "%(file)s" && \
mv "%(tmpdir)s/outfile.png" "%(outfile)s"
- |
convert -background none -resize %(size)sx%(size)s "%(file)s" "%(tmpdir)s/outfile.png" && \
mv "%(tmpdir)s/outfile.png" "%(outfile)s"
Secondly, commands should be POSIX shell compatible, not - for example - depend on BASH. This assures that the provider commands will work on most machines and accounts.
Third, commands should fail with a return code > 0
if some environment
preconditions are not fulfilled.
Keep in mind that a pipe of commands exits with the return code of the last command,
no matter if an earlier command did fail.
So, it's a good idea to chain programs with &&
or to extend program calls with || exit 1
to make a provider command fail properly.
The availability of required programs that are used in pipes can for example be tested
with which <program> || exit 1
.
Also remember to quote the input file (like "%(file)s"
) as the path may contain spaces.
The %(outfile)s
(and the %(tmpdir)s
) should not contain any spaces,
but it doesn't hurt to also quote them.
Let's look at an example and specify a provider that provides thumbnails for PDF and postscript files. Both can be turned into a thumbnail image with the same command, which is why it's possible to handle both in one provider.
Be aware that the real default configuration for AllMyToes uses a different provider definition for these formats to implement more specific meta-chunk commands. This definition serves only as an example.
PDF files have a MIME type application/pdf
, postscript files have application/postscript
.
- mimes: [application/pdf, application/postscript]
commands:
- evince-thumbnailer -s %(size)s "%(file)s" %(outfile)s
- convert -thumbnail %(size)sx%(size)s -background white %(file)s[0] PNG:%(outfile)s
meta:
'Thumb::Document::Pages':
- |
which exiftool || exit 1
exiftool "%(file)s" | awk '/^Page Count/ {print $4}' | sed 's/[^0-9]*//g'
- |
which gs || exit 1
gs -o /dev/null -sDEVICE=bbox "%(file)s" 2>&1 | grep HiResBoundingBox | wc -l
revision: 1
First, the MIME types for which this provider shall be used are defined.
The commands
section first tries to create the thumbnail (in %(outfile)s
)
with evince-thumbnailer
.
If evince-thumbnailer
exists, the task of providing the image is done and
the second command is not executed.
If evince-thumbnailer
is not available, the first command fails and AllMyToes
will execute the second command which uses convert
from ImageMagick.
If also convert
is not available, the thumb creation has failed and AllMyToes returns
with an error.
The evince-thumbnailer
is tried first because it's faster and we prefer that one.
If one of the two commands was successful, AllMyToes evaluates the meta
section.
In this example, we have only one meta-chunk with the meta-key Thumb::Document::Pages
.
This meta-chunk has also two commands and again, AllMyToes will first try the first and
execute the second one only if the first one fails.
The first command depends on exiftool
, the second depends on Ghostscript (gs
).
Because both of them are used in a pipe, we first check if they exist explicitly
and exit with 1
otherwise.
The example shows how a multi-line script can be used as a command.
Both of the meta-data commands return the value for the Thumb::Document::Pages
meta data chunk
as output on stdout
.
Check the default provider configuration for more examples.
My personal motivation was the use of thumbnails for image previews in terminal-based file managers like joshuto and ranger. But AllMyToes can be useful in any situation where one wants to show a size-limited image in scripted environments, maybe as icons in desktop notifications, other kind of image-overlays in terminal-based applications, or desktop widgets; wherever loading full sized-images would consume unnecessary time and CPU power.
Download the latest release as binary from the release page.
Make the downloaded file executable (chmod +x allmytoes
)
and place it in a directory which is part of your PATH
(for example mv allmytoes ~/.local/bin
).
cargo
cargo install allmytoes
cargo
cargo install --git https://gitlab.com/allmytoes/allmytoes.git
git clone https://gitlab.com/allmytoes/allmytoes.git
cd allmytoes
cargo build --release
The built binary will be at target/release/allmytoes
.
$XDG_CACHE_HOME
or ~/.cache
).
In case the cache directory does not exist, AllMyToes will terminate with an error.
(Might be changed later.)Thumbnails - all in PNG format - have certain meta-data entries, each of them a PNG “tEXt” chunk with a key and a value. The freedesktop.org thumbnail standard describes some mandatory and some optional keys. (See the “Thumbnail Creation section” in the standard.) AllMyToes implements all of them. The following table lists all meta-data entries created by AllMyToes.
Key | XDG standard | Description |
---|---|---|
Thumb::URI |
mandatory | The URI of the thumb's source file. |
Thumb::MTime |
mandatory | The mtime of the thumb's source file. (Functionally required.) |
Software |
optional | The name of the software that created the thumb. Also described by the PNG standard. This is always set to allmytoes . |
Software::Version |
- | The version of AllMyToes that created the thumb. |
AMT::ProviderRevision |
- | For thumbs from providers (non image sources), the revision of the provider as given in the config. |
Thumb::Size |
optional | The size of the thumb's source file in bytes. |
Thumb::Mimetype |
optional | The mimetype of the thumb's source file. |
Thumb::Image::Width |
optional | The width of the source image in pixel. † |
Thumb::Image::Height |
optional | The height of the source image in pixel. † |
Thumb::Image::Dimensions |
- | A string <width>x<height> , describing the dimensions of the thumb's source image. † |
Thumb::Image::Reorientation |
- | A textual description of the rotation and flipping done my AllMyToes on creation of the thumb based on the EXIF data of the thumb's source image. Only if the source of a thumb is an image format with EXIF data that contains an Orientation flag. |
Thumb::Document::Pages |
optional | The number of pages (only for PDF and postscript files). |
Thumb::Movie::Length |
optional | The length of a video in seconds (only for video files). |
† Only for image input files
Major milestones planned:
See also my development board.
Few things are tested by standard Rust unit test.
Those tests can be run with cargo test
.
The majority of AllMyToes’ functionality is tested with end-to-end tests by a BDD test framework,
Python Behave.
Those tests use real images on the file system and an allmytoes
debug build, started as a sub-process.
The BDD tests live in the test
sub-directory.
The test-definitions can be found as feature
-files in test/features
.
Have a Python >= 3.10 environment with the required dependencies.
E.g., use a virtualenv and then pip install -r test/requirements.txt
.
Have a debug build of AllMyToes. target/debug/allmytoes
will be the binary under test.
Then, to run the tests, cd
into the test
directory and run behave
.
evaluation of XDG_CACHE_HOME
MIME type probing for wrong and missing file extensions (if this fails on CI, check if the MIME-db is installed on the used docker img)
--force-creation
Return error when input file is not readable, no matter if valid thumb exists.
Updating existing but outdated thumbs
Error on non-existing, non-readable, not-decodable input-file
Deny to process files which are neither a regular nor a symlink to a regular file
All tEXt meta-data (mtime and URI are tested implicitly, of course)
#61
Tumbler is a much more mature and feature-rich tool for the same purpose, but needs to run as a service and uses DBUS for communication.
I did not use Tumbler as I wanted to have something more simple for my scripting.
AllMyToes can be used as a Rust library to obtain a thumbnail for a given image. You can find the crate on crates.io.
AllMyToes has a very small interface.
There's one struct for the configuration (AMT
) that also provides the one function (get
) to get a thumb.
Then, there's one enumeration to specify the thumbnail size (ThumbSize
),
one struct for the result (Thumb
),
and one enumeration for the possible errors (ToeErrorType
).
use std::path::Path;
use allmytoes::{AMTConfiguration, AMT, ThumbSize};
fn main() {
// The configuration for allmytoes
// Usually, the defaults are fine.
let configuration = AMTConfiguration::default();
// An instance of allmytoes that can be used to provide thumbnails
let amt = AMT::new(&configuration);
// The file for which we want a thumbnail as a `std::path::Path`.
let input_file = Path::new("/tmp/image.jpg");
// The size of the thumbnail we want
let thumb_size = ThumbSize::Large;
// Get a thumbnail
match amt.get(
input_file,
thumb_size,
) {
Ok(thumb) => {
println!("The thumb is here: {}", thumb.path)
}
Err(error) => println!(
"Error '{:?}' occurred when trying to provide the thumb. ({})",
error,
error.msg(),
),
}
}
AllMyToes is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
AllMyToes is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with AllMyToes. If not, see https://www.gnu.org/licenses/.