LeanSignal Agent

Watch the health of the LeanSignal agents themselves — control-stream connectivity, pipeline throughput, export failures, queue depth, and memory — with curated dashboards and alerts you import in one step.

There is nothing to enable. The LeanSignal agent monitors itself out of the box: the underlying OpenTelemetry Collector emits its own internal telemetry, and the LeanSignal edge controller adds a handful of platform-specific metrics (metric-index sizes, control-stream connectivity). Every shipped config — the Linux/systemd install, the Docker image, and the Helm chart — already collects it. Install the agent and these metrics are already flowing.

Self-telemetry rides the same demand-driven pipeline as your application metrics: it’s stored in full in the agent’s local VictoriaMetrics, indexed, and demandable. Importing one of the demands below is exactly what forwards it to your central dataplane, so you get fleet-wide agent dashboards without collecting anything you don’t ask for.

Metric names are the ones stored in VictoriaMetrics — the collector’s built-in otelcol_* metrics plus LeanSignal’s own leansignal_edgecontroller_* metrics. See the agent’s own-telemetry reference for the full list.

Prerequisites

• A LeanSignal agent installed and connected — that is the only requirement. Self-telemetry is collected automatically; there is nothing to enable. If you haven’t deployed the agent yet, follow Install the agent (source: LeanSignal/leansignal-agent). It should show as Connected under Agents in the LeanSignal app.
• Editor or admin role in the LeanSignal app (importing a demand creates dashboards and alert rules).

Setup

Nothing to configure. Install the agent (see Install the agent) and its self-telemetry is collected automatically. Every shipped config already does three things for you:

1. Exports the collector’s own metrics (level detailed), logs, and traces via service.telemetry over OTLP to the agent’s own loopback OTLP receiver (127.0.0.1:4317).
2. Routes them through the same pipelines as your application telemetry, where the resource processor stamps the leansignal_agent_name, host_name, and os_type identity labels onto every series.
3. Stores them in the co-located edge stores — metrics land in the local VictoriaMetrics, indexed and demandable like any application metric.

The only step is importing a demand below — that is what tells the agent to forward its own metrics to your central dataplane, exactly like an application metric.

To confirm it locally on an agent host, query its VictoriaMetrics directly:

# Is the control stream up right now?
curl -s 'http://127.0.0.1:8428/api/v1/query?query=leansignal_edgecontroller_connection_up' | jq .

# List every self-telemetry series currently stored
curl -s http://127.0.0.1:8428/api/v1/label/__name__/values \
  | jq -r '.data[]' | grep -E '^(leansignal_|otelcol_|http_client_)'

Counters are born on first increment

A counter that has never fired has no series yet — for example leansignal_edgecontroller_connection_established_total on an agent that has never reached the backend. Use the always-present leansignal_edgecontroller_connection_up gauge to observe current state. The http_client_* latency histograms require the shipped detailed telemetry level (the default); dropping to normal removes them.

Where this runs

This telemetry originates inside the agent itself, so there is nothing to place or route — every agent you run, central or edge, self-reports. Use the Instance filter to tell them apart.

Multiple instances

Every self-telemetry series automatically carries leansignal_agent_name, host_name, and os_type identity labels (the agent promotes the leansignal.agent.name, host.name, and os.type resource attributes with resource_to_telemetry_conversion), so multiple agents stay distinct with no configuration. The imported dashboard has an Instance filter — view all instances at once or focus on one.

Logs and traces

The agent’s own logs are part of its self-telemetry: service.telemetry pushes them through the agent’s own logs pipeline, where they land in the co-located Loki as the leansignal-agent stream — and since agent 0.8.0 the co-located stores’ logs ride along too, as leansignal-victoria-metrics, leansignal-loki, and leansignal-tempo (collected by default on Linux and macOS installs). The Standard and Extended demands on this page include those streams, so importing one also forwards the agent stack’s logs to your tenant — on Windows, where the service captures no log file, that is the only way to read agent logs at all. The agent emits very few spans of its own; the traces it carries are your workloads’, which reach it over OTLP like every other signal.

Demand-driven, like metrics

Logs and traces land in the gateway’s edge stores (a roughly one-hour full-fidelity window) and are queryable immediately in the LeanSignal app under Logs and Traces with the source set to Available. Nothing is stored centrally until a demand covers it — and this page is the one exception to the catalog’s metrics-only rule: the Standard and Extended variants demand the agent stack’s own log streams, so importing one forwards them. Everything else stays local until a dashboard asks for it. See Logs and Traces.

Demands

LeanSignal Agent ships as a ready-made demand in three variants. You don’t copy any JSON — in the LeanSignal app go to Demands, press the ˅ arrow on the Add Demand button (the arrow opens the import menu), choose Import from catalog…, and pick the variant. Each variant below lists the slug that identifies its published bundle. See Integrations for the full import flow.

The variants are nested — Standard is a superset of Essential, Extended a superset of Standard. Pick whichever is closest to what you need and treat it as a starting point: after import everything is a normal, editable copy, so retune thresholds, add or drop panels, and adjust queries for your environment — your edits reshape the demand automatically. Start small and re-import a larger variant later if you outgrow it.

Essential

The golden signals. 1 dashboard · 4 panels · 3 alerts — stream connectivity, throughput, failures, and memory at a glance.

Import this variant from the catalog. Its demand slug:

Demand slug

leansignal-agent-otel-demand-essential

Dashboard — Control stream connected, Accepted metric points, Failed metric points, and Agent memory (RSS).

Alerts (3)

Alert	Severity	Fires when
Agent control stream down	critical	connection_up = 0 for 5m
Agent export failures	critical	failing points for 10m
Agent export queue near capacity	warning	queue > 80% full for 5m

Review the bundle JSON →

Standard

Well-rounded coverage. 1 dashboard · 13 panels · 6 alerts — adds demand-set size, sent/refused rates, queue pressure, and the agent stack’s own logs. The recommended default.

Import this variant from the catalog. Its demand slug:

Demand slug

leansignal-agent-otel-demand-standard

Dashboard — everything in Essential, plus Demanded timeseries, Sent metric points, Sent log records, Sent spans, Local metrics store disk, Local store free disk, Refused metric points, Export queue utilization, and Agent & store logs.

Alerts (6)

Alert	Severity	Fires when
Agent control stream down	critical	connection_up = 0 for 5m
Agent export failures	critical	failing points for 10m
Agent export queue near capacity	warning	queue > 80% full for 5m
Agent refusing incoming data	warning	refusing points for 10m
Agent demand set is empty	warning	demanded series = 0 for 15m
Agent restarted	warning	uptime < 5m

Review the bundle JSON →

Extended

Deep agent visibility. 1 dashboard · 22 panels · 10 alerts — index internals, reconnects, write latency, CPU, and heap for fleets you need to diagnose, not just watch.

Import this variant from the catalog. Its demand slug:

Demand slug

leansignal-agent-otel-demand-extended

Dashboard — everything in Standard, plus Known timeseries (local index), Pending index updates, Control-stream reconnects, Local log store ingest, Local trace store ingest, Delivery latency p95 — per destination, Delivery errors — per destination, Agent CPU usage (cores), and Go heap allocated.

Alerts (10)

Alert	Severity	Fires when
Agent control stream down	critical	connection_up = 0 for 5m
Agent export failures	critical	failing points for 10m
Agent export queue near capacity	warning	queue > 80% full for 5m
Agent refusing incoming data	warning	refusing points for 10m
Agent demand set is empty	warning	demanded series = 0 for 15m
Agent restarted	warning	uptime < 5m
Agent control stream flapping	warning	> 0.1 reconnects/s for 10m
Agent metric index not draining	warning	> 1000 pending updates for 15m
Agent memory high	warning	RSS > 1 GiB for 15m
Agent remote-write latency high	warning	p95 write latency > 5s for 10m

Review the bundle JSON →

Dashboards

Each variant imports one LeanSignal Agent dashboard, its panels grouped into collapsible sections. Every dashboard carries an Instance filter (on the leansignal_agent_name label), so one import covers the whole fleet — view every instance at once, or focus on one. Legends are sortable tables showing last/mean/max per series, so the outlier instance stands out. Dashboards are demand-driven: importing one tells your agents to forward exactly the timeseries its panels query, and nothing else. Panels are ordinary Perses panels — edit queries, add panels, or retune them after import, and your changes reshape the demand automatically.

Alerts

Thresholds are conservative starting points — tune them to your workload.

Caution

Imported alert rules arrive without a notification channel and start out paused — with nowhere to deliver, they do not evaluate. After importing, create or pick a channel under Alerts → Manage Channels, attach it to each rule, and resume. Rules without a channel are marked with a warning icon in the alerts list.

Next steps

• Demand-driven observability — why importing a demand also shapes what your agents collect.
• Agent configuration — the full agent config surface.