Logs

LeanSignal applies the same demand-driven model to logs that it applies to metrics. This page follows a log record through the whole pipeline: how it gets into the agent, how the agent buffers it, how demand decides which streams reach central storage, and how you search them in the app.

The lifecycle at a glance

   your workloads              the agent gateway                     central
  ─────────────────    ────────────────────────────────      ──────────────────
   OTLP / Loki push  →  local Loki (all streams, ~1 hour)  →   demanded streams
   filelog / journald   log-stream index available to the       (tenant Loki,
                        app; demand filter forwards only          30 days)
                        streams a dashboard / rule demands

Everything is collected at full fidelity at the edge. Only the demanded log streams are ever forwarded to central storage — so central cost tracks the logs you actually query, not everything your systems write. See Demand-driven observability for the why.

How logs are collected

The agent gateway accepts logs two ways — both land in the same edge store:

Path	Endpoint on the gateway	Use it for
OTLP	4317 (gRPC) / 4318 (HTTP)	Apps instrumented with an OpenTelemetry SDK, or any OpenTelemetry Collector pipeline
Loki push API	3500 (HTTP) / 3600 (gRPC)	Existing Promtail / Grafana Alloy-style shippers — repoint them, no re-instrumentation

From applications — the OTLP endpoint your app already uses for metrics carries logs too. With OpenTelemetry zero-code instrumentation this is often a single environment variable:

export OTEL_EXPORTER_OTLP_ENDPOINT="http://<gateway-host>:4317"
export OTEL_LOGS_EXPORTER="otlp"    # Java and .NET agents export logs by default

From files, journald, or Kubernetes — run a collector close to the source (an OpenTelemetry Collector with the filelog / journald receivers, or the shipper you already operate) and point its output at the gateway: OTLP to 4317/4318, or the Loki push endpoint on 3500. The gateway itself is an OpenTelemetry Collector distribution, so if it can reach the files directly you can add those receivers straight to its own config instead.

Give every source a service.name

Log streams are keyed by their OpenTelemetry resource attributes, promoted to stream labels — service_name, service_namespace, k8s_namespace_name, k8s_pod_name, and friends. Those labels are what stream selectors match, so a meaningful service.name on every source is what makes logs findable.

How logs are buffered at the edge

Inside the gateway, every log record flows through a logs/all pipeline before any filtering:

• Local store. Every record is written to a co-located, single-node Loki — a short ~1-hour full-fidelity window (max_query_lookback: 1h). This is the edge buffer: every stream the gateway sees, whether or not anything demands it.
• Queryable live. That buffer is searchable from the Logs page in the app (source: Available) over the agent’s gRPC control channel — so you can read logs nobody is storing centrally yet.

Kubernetes

The Helm chart bundles the local log store: a co-located Loki runs as its own pod by default. To use a Loki you operate yourself, set localLoki.deploy=false and point localLoki.writeEndpoint and localLoki.queryEndpoint at it. The Linux and macOS installers set the local Loki up for you.

How logs are filtered — the demand set

The logs/all pipeline fans into a logs/filtered pipeline. The filter is the set of demanded log-stream selectors. Only records whose stream matches a demanded selector are forwarded to the per-tenant Loki; everything else stays at the edge and ages out within the hour. The filter is fail-closed: an empty demand set forwards nothing.

Two things create log demand:

• Dashboards. Saving a dashboard whose panels query logs demands their stream selectors. LeanSignal extracts the LogQL selectors from the Loki panels server-side and folds them into the demand set — for example {service_name="checkout"}. See Build dashboards.
• Ingestion rules. A LogQL stream selector you add directly on the Logs page’s Ingestion rules tab, attached to a demand — forward a stream without building a dashboard for it.

A selector demands the whole stream. Filter stages after the selector (|= "error", | json) refine what you see on a query, not what is shipped. The demanded subset is recomputed and pushed to every connected agent in near real time, and central storage keeps demanded streams for 30 days — the same retention as metrics and traces.

Ready-made demands are metrics-only — with one exception

The demands in the integrations catalog are metrics-only, except the LeanSignal Agent integration: its Standard and Extended variants also demand the agent stack’s own log streams. Your log demand comes from the dashboards and ingestion rules you create — there is nothing to import to “enable” logs. Collect them, then demand what you query.

How logs show up in the app

The Logs page in the LeanSignal app is where you search log streams. It has two tabs and a source switch:

• Explore tab — a LogQL query editor, a log-volume histogram over the results, and the matching log lines below.
• Ingestion rules tab — create and manage the LogQL stream selectors this signal forwards on demand.

Search with LogQL — a label selector plus optional filter stages:

{service_name="checkout"} |= "error"

The source switch selects where the query runs:

Source	What it reads	What’s there
Stored	The per-tenant Loki	Only demanded streams, kept 30 days, from the moment each demand was saved
Available	The agent’s edge Loki, live over the agent’s gRPC control channel	Every stream collected, for the last ~1 hour

Available requires the agent to be Connected — the query is tunnelled down the agent’s own control stream, so the edge store is never exposed to the network. If no agent is connected, Available returns an error.

Exploring never creates demand

Searching on the Logs page — against either source — never changes what is stored centrally. Demand comes only from saved dashboards and enabled ingestion rules. Iterate on a query freely before committing to its cost.

Verify

1. Send some traffic through an instrumented app (or a shipper).
2. In the app, open Logs, switch the source to Available, and query {service_name="<your service>"}. This reads the edge store live, so it works before any demand exists.
3. Save a dashboard panel with that query (or add an Ingestion rule), then switch back to Stored: new data appears from the moment the demand was saved. Nothing is backfilled.

Next steps

• Metrics and Traces — the same lifecycle for the other two signals.
• Build dashboards — a saved log panel is what turns a stream into demand.
• Work with demands — the unit that ties dashboards and rules to cost.
• Agent configuration — gateway ports, endpoints, and retention.