Skip to content

Demand-driven observability

Most observability tooling is supply-driven: it ingests whatever telemetry your systems can produce, stores all of it centrally, and leaves you to figure out — and pay for — what was actually worth keeping. The result is a loop every platform team knows: store everything → bill shock → reactive pruning → repeat. Cost grows with data volume, not with the value it delivers.

LeanSignal is demand-driven: it collects only the telemetry tied to a declared need, so observability cost stays locked to value — by design, not by cleanup. Nothing is shipped to long-term storage unless something actually needs it, and new cost only ever comes from new need.

A demand is a declared need for specific telemetry: metric timeseries, log streams, and trace services. Demands can be created explicitly, but most are derived automatically from the things that consume telemetry:

  • Dashboards — every panel query implies the timeseries required to render it, and panels that query logs or traces imply their log streams and trace services the same way.
  • Alerts — every rule implies the timeseries required to evaluate it.

LeanSignal resolves these into a concrete demand set: the exact list of timeseries, log streams, and trace services that must be forwarded to satisfy current demand. You can also import ready-made demands — curated dashboards and alerts for common technologies — from the integrations catalog.

The demand set is the contract between LeanSignal and your agents:

  1. You add or change a dashboard, alert, or demand.
  2. LeanSignal recomputes the demand set.
  3. The new demand set is pushed to every connected agent in near real time.
  4. Each agent forwards the telemetry in the set to your central dataplane — and nothing else.

Because the filter is applied at the source, central ingestion and storage costs track only the telemetry that something is actually using. Every stored signal is tied to the need it serves — that is the cost–value link: you can always see which spend buys which value, and retiring a dashboard or alert retires its cost with it.

Filtering at the edge raises an obvious question: how do you build a dashboard for a metric you are not collecting centrally? LeanSignal answers it with a short full-fidelity buffer at the edge:

  • Every agent stores everything it sees at the edge — metrics in a co-located VictoriaMetrics for one day, logs in a co-located Loki and traces in a co-located Tempo for roughly the last hour — and reports the full index of available timeseries to the app.
  • While you edit a dashboard, preview queries are answered from that local store — so you explore real data, not guesses, before anything is demanded.
  • The moment you save, the demand set updates and the agents start forwarding the newly demanded timeseries to the dataplane.

You keep the discoverability of collect-everything systems without paying collect-everything storage bills — declaring demand never means whitelisting blindly. See Architecture for how the pieces fit together, including the pain-cycle vs value-cycle picture.

How this differs from sampling and pruning

Section titled “How this differs from sampling and pruning”

Plenty of tools reduce observability cost. Almost all of them work downstream: sample a fraction of the stream, shape or aggregate it in a pipeline, prune high-cardinality offenders after the bill, or cap usage at the vendor. Those are real tools with real savings — and they share a structural limit: they decide how much data to keep, not which data matters. The cost stays anchored to volume, and the risk of dropping the one series you needed stays with you.

Demand-driven collection moves the decision to the front of the pipeline and changes what it is anchored to. The filter at the edge is configured by your declared demand — the dashboards and alerts you actually run — so:

  • Nothing is a guess. The kept subset is derived from real consumers, not from a sampling rate or a hand-maintained allowlist.
  • Noise never becomes a bill. Undemanded telemetry is dropped before central storage, not discounted after it.
  • Coverage is recoverable. The edge buffer holds full fidelity, so the moment you need a new metric, demanding it starts collection — no redeploy, no instrumentation change.

And because the interfaces are open standards — OpenTelemetry in, PromQL out — adopting the model doesn’t trade cost control for lock-in.

Supply-drivenDemand-driven (LeanSignal)
Collect and store everything, prune laterStore centrally only what is demanded
Cost scales with what your systems emitCost scales with what you actually use
Surprise bills, reactive cleanupCost tracks value; new cost only from new need
  • Architecture — the pain cycle vs the value cycle, and the components that implement it.
  • Integrations — ready-made demands you can import in one step.
  • Getting started — from zero to your first demand.
Was this page helpful?