Skip to content

Provide alternate strategies for exporting metrics with AWS Lambda #4157

@herin049

Description

@herin049

What problem do you want to solve?

The current AWS Lambda instrumentation library flushes all metrics after each invocation, which results in metric data-points being generated after each invocation, regardless of the configured export interval. This issue becomes increasingly problematic for Lambda functions with short execution durations (e.g. ~100ms) since even datapoints with the same attributes, will not have the ability to be aggregated if they span more than a single invocation. In addition to being operationally inefficient, because almost all observability vendors charge based on the number of datapoints ingested, this pattern results in dramatic increases in costs.

Describe the solution you'd like

There are several changes which can be employed to help reduce the number of metric datapoints being generated. The two initial changes are as follows:

  1. Provide a configuration option/environment variable to disable force flushing metrics after each invocation.
  2. Provide a configuration option/environment variable to register a signal handler for the SIGTERM shutdown event to flush all metrics before the Lambda instance is cleaned up. (see https://aws.amazon.com/blogs/compute/python-3-12-runtime-now-available-in-aws-lambda/)
  3. Either: add an enhancement to the existing PeriodicExportingMetricReader to add the ability to suspend/resume metric collection to ensure that metric collection stops before the Lambda environment is potentially frozen OR create a new SuspendableExportingMetricReader which has the same functionality.

Describe alternatives you've considered

Metric aggregation in the collector, however there seems to be limited support for this pattern.

Additional Context

No response

Would you like to implement a fix?

Yes

Tip

React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions