Metrics

Metrics are LeanSignal’s first-class signal, and the one every ready-made demand is built from. This page follows a metric through the whole pipeline: how it gets into the agent, how the agent stores and indexes it, how demand decides what reaches central storage, and how you explore it in the app.

The same demand-driven model applies to logs and traces — metrics is simply where it is easiest to see.

The lifecycle at a glance

   your workloads                 the agent gateway                    central
  ─────────────────      ─────────────────────────────────      ──────────────────
   OTLP / Prometheus  →   local VictoriaMetrics (all, 1 day)  →   demanded subset
   scrape / remote-write   metric index reported to the app        (dataplane VM,
                           demand filter → forward only what         30 days)
                           a dashboard / alert / rule demands

Everything is collected at full fidelity at the edge. Only the demanded subset is ever forwarded to central storage — so central cost tracks the metrics you actually use, not everything your systems emit. See Demand-driven observability for the why.

How metrics are collected

The agent gateway is an OpenTelemetry Collector distribution, so it ingests metrics through standard receivers:

Path	Endpoint on the gateway	Use it for
OTLP	4317 (gRPC) / 4318 (HTTP)	Apps with an OpenTelemetry SDK, or any OpenTelemetry Collector pipeline forwarding to the gateway
Prometheus	scrape / remote-write, per the gateway config	Existing Prometheus exporters and /metrics endpoints

From applications — point your OpenTelemetry SDK or auto-instrumentation at the gateway’s OTLP endpoint. With zero-code instrumentation this is often a single environment variable:

export OTEL_EXPORTER_OTLP_ENDPOINT="http://<gateway-host>:4317"
export OTEL_SERVICE_NAME="checkout"

From hosts and infrastructure — the ready-made demands in the integrations catalog pair a dashboard with the receiver configuration for a source (host metrics, PostgreSQL, MySQL, VictoriaMetrics, and more). Import one and the gateway starts collecting that source.

Set a meaningful service.name and an instance label

Metrics are labelled by their OpenTelemetry resource attributes. service.name and a per-instance label (host, pod, replica) are what let you tell one source from another and filter dashboards to a single instance — set them on every workload. See multi-instance dashboards for the pattern.

How metrics are processed and stored at the edge

Inside the gateway, every metric flows through an */all pipeline before any filtering:

1. Local store. Every sample is written to a co-located, single-node VictoriaMetrics (127.0.0.1:8428) with a fixed 1-day retention. This is the full-fidelity edge buffer — everything the gateway sees, whether or not anything demands it.
2. Live index. The gateway maintains an index of every metric name and timeseries it has observed and reports it up to the app over its outbound gRPC control stream. That index is what powers the metric finder in edit mode — you can discover a metric before you collect it centrally.

Because the store and index live at the edge, discovery costs you nothing centrally: you can inspect any metric the agent sees, decide it is worth keeping, and only then create demand for it.

How metrics are filtered — the demand set

An */all pipeline fans into an */filtered pipeline. The filter is the demand set: the exact list of timeseries the app has computed from what consumes metrics. Only timeseries on that list are remote-written to the central dataplane VictoriaMetrics (through vmauth); everything else stays at the edge and ages out after a day.

Three things create metric demand:

• Dashboards. Every panel query implies the timeseries needed to render it. Saving a dashboard demands them. See Build dashboards.
• Alerts. Every alert rule’s PromQL implies the timeseries needed to evaluate it — alerting on a not-yet-collected metric turns its collection on. See Alerting.
• Ingestion rules. A metric selector you add directly on the Metrics page’s Ingestion rules tab, attached to a demand — the escape hatch for forwarding a metric without a dashboard or alert.

The app recomputes the demand set whenever any of these change and pushes it to every connected agent in near real time; the agent starts (or stops) forwarding within seconds. Central storage keeps demanded timeseries for 30 days — the same retention as logs and traces. Retiring the last dashboard, alert, or rule that demands a series retires its central cost with it.

How metrics show up in the app

The Metrics page in the LeanSignal app is where you query metrics directly. It has two tabs and a source switch:

• Explore tab — a metric finder plus a PromQL query editor and graph.
• Ingestion rules tab — create and manage the metric selectors this signal forwards on demand (the custom rules described above).

The source switch in the top right selects where the query runs:

Source	What it reads	What’s there
Stored	Central dataplane VictoriaMetrics	Only the demanded subset, kept 30 days — the same data dashboards and alerts read
Available	The agent’s edge store, live over the agent’s gRPC control channel	Everything collected, for the last ~1 day

Available is the superpower of demand-driven observability: you can graph metrics nobody is paying to store centrally. It requires the agent to be Connected — the query is tunnelled down the agent’s own control stream, so the edge store is never exposed to the network. If no agent is connected, the Available source returns an error.

The finder plus Available answer “what could I collect?”; Stored answers “what am I collecting?”. When a metric earns a permanent place, put it on a dashboard or an alert, or add an ingestion rule — and that usage becomes demand.

Exploring never creates demand

Queries on the Metrics page — against either source — never change what is stored centrally. Demand comes only from saved dashboards, active alerts, and enabled ingestion rules. That makes the page the safe place to iterate on a query before committing to its cost.

Verify

1. Point an instrumented app or a host collector at the gateway’s OTLP endpoint.
2. In the app, open Metrics, switch the source to Available, and find a metric your source emits — this reads the edge store live, so it works before any demand exists.
3. Graph it, then put it on a dashboard panel (or add an Ingestion rule). Switch the source back to Stored: new samples appear from the moment the demand was saved. Nothing is backfilled.

Next steps

• Logs and Traces — the same lifecycle for the other two signals.
• Build dashboards — a saved panel is what turns a metric into demand.
• Work with demands — the unit that ties dashboards, alerts, and rules to cost.
• Agent configuration — gateway ports, endpoints, and retention.