AWS Lambda Log Proxy

Filter or transform logs from AWS Lambda functions before they are sent to CloudWatch Logs.

[!NOTE] This is a library for developers. If you are looking for a binary executable, see AWS Lambda Log Filter.

[!CAUTION] Possible data loss if you write tons of logs and return immediately. See possible data loss below.

Usage

Installation

Add the following to the dependencies in your Cargo.toml:

aws-lambda-log-proxy = "0.2"

or run:

cargo add aws-lambda-log-proxy

Examples

A real world case: AWS Lambda Log Filter.

Documentation

FAQ

Why We Need This?

We need a solution to realize the following features without modifying the existing code of AWS Lambda functions:

• Reduce the volume of logs to lower the costs.
• Wrap existing logs in JSON with customizable level and field name, so we can use the built-in Lambda Log-Level Filtering to filter them.
• But keep the EMF logs untouched so we can still retrieve the metrics.

How It Works?

We use AWS Lambda Runtime Proxy to spawn the handler process as a child process of the proxy (and optionally disable the _LAMBDA_TELEMETRY_LOG_FD environment variable) to intercept the logs, then suppress the invocation/next request to process logs as much as possible.

Why Not Just Grep

Actually you could just create a shell scripts with the content exec "$@" 2>&1 | grep --line-buffered xxx and use the lambda runtime wrapper script to realize a simple filter. However we need this lib to realize the following features:

• Override the _LAMBDA_TELEMETRY_LOG_FD environment variable to ensure the logs are printed to the stdout/stderr instead of the telemetry log file.
• Suppress the invocation/next request to process logs as much as possible.

Possible Data Loss

Though we tried our best to suppress the invocation/next request to process logs as much as possible, if you write tons of logs (more than thousands of lines) and return immediately, there might be some logs not processed.

As a best practice, it is your responsibility to do a thorough benchmark test against your own use case to ensure the logs are processed as expected.

CHANGELOG