Generate RED metrics from spans
Generates RED metrics from spans with the spanmetrics connector: explicit latency buckets, HTTP dimensions, exemplars, and a 10k cardinality cap, after normalizing span names so route explosions don't inflate the metric space.
Turns spans into RED metrics with the spanmetrics connector: explicit histogram buckets from 2ms to 15s, http.request.method and http.response.status_code dimensions, exemplars for jumping from a latency spike straight to a trace, and an aggregation cardinality cap of 10,000 series as a blast-radius limit. resource_metrics_key_attributes pins series identity to service name, namespace, and SDK language so pod churn doesn't fragment the metrics.
Span names are normalized to semantic conventions 1.37.0 with set_semconv_span_name before metrics are generated, because high-cardinality span names are the classic way spanmetrics blows up. Traces continue to your trace backend over OTLP; the generated metrics go to your OTLP/HTTP endpoint via OTLP_GATEWAY_ENDPOINT.
Use this flow
Open the config in Telflo and it becomes a working pipeline on the canvas: adapt what's specific to you, test it against recorded traffic, and push it to your fleet over OpAMP. Free account, no card.
Components
What's in it, and why
Receives spans from SDKs and upstream collectors over gRPC and HTTP.
Runs first so backpressure reaches the receiver before the connector's in-memory aggregation can push the collector out of memory.
Normalizes span names to semantic conventions 1.37.0 before metrics are generated, since high-cardinality span names are what makes spanmetrics explode.
Batches on a 10 second timeout, on both the trace path and the generated metric path.
Computes RED metrics from spans with explicit 2ms to 15s buckets, HTTP method and status dimensions, exemplars, and a 10,000 series cardinality cap.
Ships the original traces on to the trace backend over OTLP gRPC (Tempo in the template).
Ships the generated RED metrics to the OTLP/HTTP endpoint named by OTLP_GATEWAY_ENDPOINT.
Notes
Gotchas
- 1
The cardinality cap of 10,000 series is a blast-radius limit, not a target. Once it is hit, new series are dropped rather than the collector growing without bound.
- 2
resource_metrics_key_attributespins series identity to service name, namespace, and SDK language, so pod churn does not fragment the metrics into a new series per pod. - 3
Exemplars are enabled here, which links a latency bucket to a real trace id. That only works if the trace it points at survives whatever sampling runs downstream.
- 4
The trace exporter points at
tempo.example.internal, a placeholder that has to be changed before this runs anywhere real. - 5
Metrics are generated from whatever reaches this collector. If sampling happens upstream, the RED metrics describe the sample, not production.
More flows
Related flows
Replace vendor agents on your VM fleet
Scrapes CPU, memory, load, disk, filesystem, and network metrics plus system and application logs on every VM.
Redact PII before telemetry leaves the network
Masks emails, card numbers, SSNs, bearer tokens, AWS keys, and IPs in log bodies and span attributes at a central gateway, before telemetry leaves your network.
Survive backend outages with a persistent queue
Replaces the default in-memory exporter queue with a disk-backed queue that survives collector restarts, sized in items instead of opaque requests.
Test it before your fleet runs it
Free account, no card. Open this flow in the editor, adapt it, and see what it does to real data before anything ships.