emmett

Crates.ioemmett
lib.rsemmett
version0.1.3-alpha
sourcesrc
created_at2019-06-25 23:49:37.242253
updated_at2019-06-27 01:01:44.103754
descriptiona simple etl tool
homepagehttps://gitlab.com/andrewbanchich/emmett
repositoryhttps://github.com/andrewbanchich/emmett
max_upload_size
id143612
size142,512
Andrew Banchich (andrewbanchich)

documentation

README

API Docs

Crates.io Gitlab pipeline status (branch) Crates.io

About

emmett is a unified logging layer with the goal of serving as a drop-in Logstash replacement as well as providing added / improved features.

emmett is written in Rust and uses Tokio, which means:

  • Blazing fast ⚡⚡⚡
  • Very small resource requirements (currently needs less than 3 MB RAM for simple pipelines)
  • No garbage collector == better and more consistent performance
  • Completely asynchronous - plugins operate independently of one another
  • Multithreaded and work-stealing

Usage

emmett currently uses TOML for configuration, but will be able to parse Logstash configuration files as well as other formats like JSON in the future.

The Logstash configuration file parser is currently on hold until the plugins themselves become more stable. Merge requests are always welcome though!

# This is an emmett config file.

[[inputs]]

[inputs.http_poller]
request_timeout = 60
schedule = { cron = "* * * * * UTC" }
codec = "json"
metadata_target = "http_poller_metadata"
truststore = "/path/to/downloaded_truststore.jks"
truststore_password = "mypassword"

[[inputs.http_poller.urls]]
test1 = "https://jsonplaceholder.typicode.com/posts/1"

[[filters]]

[filters.mutate]
replace = { "id" = "yo dawg" }
copy = { "title" = "titleCopy" }
strip = ["body"]
split = { "body" = "\n", "titleCopy" = " repellat " }
capitalize = ["titleCopy"]
join = { "body" = " ... "}

[filters.json]
source = "jsonString"
target = "jsonString"
    
[filters.date]
match = [ "timestamp" , "dd/MMM/yyyy:HH:mm:ss Z" ]	

[[outputs]]

[outputs.elasticsearch]
hosts = ["localhost:9200"]
Commit count: 0

cargo fmt