mkey_exporter

Export counts and sizes of Memcached keys matching regular expressions as Prometheus metrics.

For example, with the Memcached keys:

thing-1:user-1:something
thing-1:user-1:something-different
thing-1:user-2:something
thing-1:user-3:something
thing-2:user-1:something
thing-2:user-1:something-else

And the YAML configuration:

name: demo
rules:
- pattern: '^(\w+):'
  label_name: 'thing'
  label_value: '$1'
- pattern: '^\w+:([\w\-]+):'
  label_name: 'user'
  label_value: '$1'

The following Prometheus metrics would be exported:

mkey_memcached_counts{user="user-1",thing="thing-1"} 2
mkey_memcached_counts{user="user-2",thing="thing-1"} 1
mkey_memcached_counts{user="user-3",thing="thing-1"} 1
mkey_memcached_counts{user="user-1",thing="thing-2"} 2

mkey_memcached_sizes{user="user-1",thing="thing-1"} 242
mkey_memcached_sizes{user="user-2",thing="thing-1"} 56
mkey_memcached_sizes{user="user-3",thing="thing-1"} 23
mkey_memcached_sizes{user="user-1",thing="thing-2"} 127

Using these metrics, you can determine what your Memcached cluster is caching based on rules that measure what's meaningful for your application.

Features

• Export counts and sizes of cache entries in your Memcached cluster.
• Extract Prometheus labels from keys based on powerful regular expressions.
• Easy to understand YAML configuration format.
• TLS Memcached connection support.

Install

There are multiple ways to install mkey_exporter listed below.

Binaries

Binaries are published for GNU/Linux (x86_64), Windows (x86_64), and MacOS (x86_64 and aarch64) for each release.

Docker

Docker images for GNU/Linux (x86_64, arm64) are published for each release.

Cargo

mkey_exporter along with its dependencies can be downloaded and built from source using the Rust cargo tool. Note that this requires you have a Rust toolchain installed.

To install:

cargo install mkey_exporter

To install as a completely static binary (Linux only):

cargo install --target x86_64-unknown-linux-musl mkey_exporter 

To uninstall:

cargo uninstall mkey_exporter

Source

mkey_exporter along with its dependencies can be built from the latest sources on Github using the Rust cargo tool. Note that this requires you have Git and a Rust toolchain installed.

Get the sources:

git clone https://github.com/56quarters/mkey_exporter.git && cd mkey_exporter

Install from local sources:

cargo install --path .

Install a completely static binary from local sources (Linux only):

cargo install --path . --target x86_64-unknown-linux-musl

To uninstall:

cargo uninstall mkey_exporter

Usage

Examples of running and configuring mkey_exporter are given below.

Running

mkey_exporter has a single required argument: the path to a YAML configuration file. Examples of invoking mkey_exporter with a variety of options are listed below. In each of these examples, config.yaml is a valid YAML configuration file. Example configuration files are listed below in the Config section or there is an example in the repository.

You can find information about all supported options by running mkey_exporter --help.

Connecting to a local server

mkey_exporter config.yaml

Connecting to a remote server

mkey_exporter --host cache01.example.com config.yaml

Connecting to a remote server over TLS

mkey_exporter --host cache01.example.com:11211 --tls-enabled --tls-ca memcached-ca-cert.pem --tls-cert memcached-client-cert.pem --tls-key memcached-client-key.pem config.yaml 

Enabling debug logging and a quicker refresh interval

mkey_exporter --log-level debug --refresh-secs 30 config.yaml

Config

mkey_exporter buckets Memcached keys using Prometheus labels based on rules that you define. The rules parse portions of the Memcached key and turn them into Prometheus label names and values. Some example configurations and the resulting prometheus metrics that would be generated are given below.

Each rule in an mkey_exporter configuration file parses a value for a particular Prometheus label from a Memcached key. Rules are evaluated in order. The first rule that sets a value for a particular label name "wins", no other rules that set that label name will be evaluated for a Memcached key.

The configuration format is defined as:

name: example                  # Name of this configuration, used for diagnostics.
rules:                         # Array of rules to apply, in order, for each Memcached key.
- pattern: '^(\w+):'           # Regular expression to apply to the Memcached key.
  label_name: 'store'          # Name of the label to emit, this may NOT contain regular expression captures.
  label_value: '$1'            # Value of the label to emit, this MAY contain regular expression captures.
- pattern: '^\w+:([\w\-]+):'   # You may include as many rules as you want, they will be evaluated in order
  label_name: 'user'           # for every Memcached key.
  label_value: '$1'

Examples

In the following examples, only the mkey_memcached_counts metric is shown for brevity.

This configuration sets two labels for metrics based on the Memcached keys.

Keys:

user-profile:user-1:backup
user-profile:user-1:latest
user-profile:user-2:latest
user-profile:user-3:latest
user-cart:user-1:latest

Rules:

name: example
rules:
- pattern: '^(\w+):'
  label_name: 'store'
  label_value: '$1'
- pattern: '^\w+:([\w\-]+):'
  label_name: 'user'
  label_value: '$1'

Metrics:

mkey_memcached_counts{user="user-1",store="user-profile"} 2
mkey_memcached_counts{user="user-2",store="user-profile"} 1
mkey_memcached_counts{user="user-3",store="user-profile"} 1
mkey_memcached_counts{user="user-1",store="user-cart"} 1

This configuration sets two labels for metrics based on the Memcached keys using multiple rules to set values for the "store" label.

Keys:

up:user-1:backup
up:user-1:latest
up:user-2:latest
up:user-3:latest
uc:user-1:latest
uu:user-1:latest

Rules:

name: example
rules:
- pattern: '^up:'
  label_name: 'store'
  label_value: 'user-profile'
- pattern: '^uc:'
  label_name: 'store'
  label_value: 'user-cart'
- pattern: '^\w+:'
  label_name: 'store'
  label_value: 'unknown'
- pattern: '^\w+:([\w\-]+):'
  label_name: 'user'
  label_value: '$1'

Metrics:

mkey_memcached_counts{user="user-1",store="user-profile"} 2
mkey_memcached_counts{user="user-2",store="user-profile"} 1
mkey_memcached_counts{user="user-3",store="user-profile"} 1
mkey_memcached_counts{user="user-1",store="user-cart"} 1
mkey_memcached_counts{user="user-1",store="unknown"} 1

Limitations

Every evaluation loop, mkey_exporter gets a complete list of keys from the Memcached server. This means the time taken for each update will increase based on the number of keys in the server. I've tested up to 3.5 million keys running on a server local to the mkey_exporter process with decent results (approximately a 5-second update time).

License

mkey_exporter is available under the terms of the GPL, version 3.

Contribution

Any contribution intentionally submitted for inclusion in the work by you shall be licensed as above, without any additional terms or conditions.