Host Metrics

Collect system-level metrics — CPU, memory, disk, filesystem, network, paging, and processes — from every host in your fleet, and get curated dashboards and alerts for them with a single import.

Host metrics are produced by the OpenTelemetry hostmetrics receiver, which scrapes the machine it runs on. In a LeanSignal deployment the topology has two tiers:

• The LeanSignal agent — a gateway, one per cluster or environment. It holds the control stream to LeanSignal, keeps full-fidelity data in its co-located VictoriaMetrics, and forwards only the demanded subset to your central dataplane.
• A per-node collector on each host you want to observe. Each host runs a lightweight OpenTelemetry Collector with the hostmetrics receiver and forwards its metrics over OTLP to the gateway — never to a remote or third-party endpoint. This scales to hundreds of nodes behind a single gateway.

host A ─ otel collector (hostmetrics) ─┐
host B ─ otel collector (hostmetrics) ─┼─ OTLP ─▶ LeanSignal agent (gateway) ─▶ dataplane
host C ─ otel collector (hostmetrics) ─┘                                          (demanded subset)

Need hardware-level metrics?

The hostmetrics receiver reads no hardware sensors — no temperatures, fan speeds, power draw, or ECC memory errors. For those (or if your fleet already runs node_exporter), use Host Metrics (node_exporter) instead — the same golden signals sourced from node_exporter, plus a Hardware section in its Extended variant. Import one flavor per fleet, not both.

Metric names follow the standard OTLP → Prometheus normalization (dots become underscores, unit and counter suffixes are added), so system.disk.io becomes system_disk_io_bytes_total. The demands below are built only on default-enabled hostmetrics metrics — utilization is derived from the standard usage counters and gauges, so a stock receiver config needs no per-metric overrides. The one exception is the Extended variant’s process-count panel, which needs the opt-in processes scraper.

Prerequisites

• A LeanSignal agent gateway deployed and connected — one per cluster or environment. If you haven’t deployed it yet, follow Install the agent; the source and release bundles live in the public LeanSignal/leansignal-agent repository. It should show as Connected under Agents in the LeanSignal app.
• Network reachability from each host to the gateway’s OTLP port — 4317 (gRPC) or 4318 (HTTP). See Agent configuration.
• An OpenTelemetry Collector on each host you want to observe (configured in Setup below).
• Editor or admin role in the LeanSignal app (importing a demand creates dashboards and alert rules).

Setup

On each host you want to observe, run an OpenTelemetry Collector with the hostmetrics receiver and point its OTLP exporter at your LeanSignal agent gateway. The gateway takes it from there — you don’t ship host metrics to a remote endpoint, only to the gateway you already run.

Any OpenTelemetry Collector build that includes the hostmetrics receiver works (for example the upstream otel/opentelemetry-collector-contrib image). A minimal per-node config:

receivers:
  hostmetrics:
    collection_interval: 15s
    # The default metric set of each scraper is all the dashboards below need —
    # no per-metric overrides.
    scrapers:
      cpu: {}
      load: {}
      memory: {}
      disk: {}
      filesystem: {}
      network: {}
      paging: {}

processors:
  # Tag every series with the host it came from so hundreds of nodes stay
  # distinguishable. Required — the queries below group by host_name.
  resourcedetection:
    detectors: [system]
    system:
      hostname_sources: [os]
  batch: {}

exporters:
  # Forward to the LeanSignal agent gateway's OTLP endpoint — not a remote
  # LeanSignal address. Replace with your gateway host/service.
  otlp:
    endpoint: leansignal-agent:4317
    tls:
      insecure: true   # in-cluster / trusted network; use TLS across untrusted links

service:
  pipelines:
    metrics:
      receivers: [hostmetrics]
      processors: [resourcedetection, batch]
      exporters: [otlp]

Multiple hosts

resourcedetection is what populates the host.name resource attribute (exported as the host_name label). Without it, every host’s metrics collapse into a single series and the dashboards and alerts below can’t tell nodes apart. The imported dashboards carry an Instance filter on host_name — view all hosts at once or focus on one.

Where to run this

The LeanSignal agent gateway is an OpenTelemetry Collector distribution, and the native installs (the Linux/systemd installer bundles) already run the hostmetrics receiver for the gateway’s own host, with the default metric set these dashboards are built on — that host shows up with no configuration at all. The Docker deployment is the exception: its shipped config does not run hostmetrics (the container does not see the host), so treat a Docker gateway’s host like any other node here. Host metrics are read from the local machine, so each additional host you want to observe runs its own OpenTelemetry Collector in agent mode (the config above), forwarding OTLP to the gateway.

For the Extended variant, also enable the processes scraper — it feeds the process-count panel and is off by default:

hostmetrics:
  scrapers:
    # …existing scrapers…
    processes: {}   # Linux/Windows only — omit on macOS

The swap panel and alert stay empty on a host with no swap configured — that is expected, not a misconfiguration.

To roll this out across a fleet, the agent installer packages a per-node edge mode: running install.sh --central-url <gateway-host>:4317 on a node installs a lightweight collector — hostmetrics included, no local stores — that forwards everything to the gateway. See Edge mode in the install guide. A DaemonSet or sidecar running the config above works too. Then confirm the metrics are arriving at the gateway by querying its local VictoriaMetrics on the gateway host:

curl -s http://127.0.0.1:8428/api/v1/label/__name__/values | grep system_

You should see system_cpu_*, system_memory_*, system_network_*, and the other system_* names, and host_name should appear as a label on them. Once a Host Metrics demand is active, the gateway forwards the demanded subset to your dataplane automatically.

Logs and traces

Round out host observability with host logs: add the filelog/journald receivers to the collector already running on the host (the edge agent, or the gateway itself when it runs there) and route them to a logs pipeline — or repoint an existing Promtail/Alloy shipper at the gateway’s Loki push endpoint (:3500). Traces are an application concern, not a host one — instrument the workloads running on the host via the runtime integrations.

Demand-driven, like metrics

Logs and traces land in the gateway’s edge stores (a roughly one-hour full-fidelity window) and are queryable immediately in the LeanSignal app under Logs and Traces with the source set to Available. Nothing is stored centrally until a dashboard demands it — the ready-made demands on this page stay metrics-only. See Logs and Traces.

Demands

Host Metrics ships as a ready-made demand in three variants. You don’t copy any JSON — in the LeanSignal app go to Demands, press the ˅ arrow on the Add Demand button (the arrow opens the import menu), choose Import from catalog…, and pick the variant. Each variant below lists the slug that identifies its published bundle. See Integrations for the full import flow.

The variants are nested — Standard is a superset of Essential, Extended a superset of Standard. Pick whichever is closest to what you need and treat it as a starting point: after import everything is a normal, editable copy, so retune thresholds, add or drop panels, and adjust queries for your environment — your edits reshape the demand automatically. Start small and re-import a larger variant later if you outgrow it.

Essential

The golden signals. 1 dashboard · 4 panels · 3 critical alerts — the smallest footprint that still tells you a host is in trouble, built entirely on default-enabled metrics.

Import this variant from the catalog. Its demand slug:

Demand slug

host-metrics-otel-demand-essential

Dashboard — Load average (with a cores reference line), Memory utilization, Filesystem utilization, and Network I/O.

Alerts (3)

Saturation is alerted on load average per core rather than the volatile CPU-utilization view — load is already time-smoothed, and normalizing by core count makes one threshold fit every host size. (CPU utilization itself lives in the Extended variant as a diagnosis panel.)

Alert	Severity	Fires when
Host load high	critical	5m load > 3× cores for 10m
Host memory usage high	critical	memory used > 90% for 5m
Host filesystem usage high	critical	any mount > 90% for 10m

Review the bundle JSON →

Standard

Well-rounded overview. 1 dashboard · 7 panels · 5 alerts, with paired warning/critical thresholds. Still default-enabled metrics only. The recommended default for most fleets.

Import this variant from the catalog. Its demand slug:

Demand slug

host-metrics-otel-demand-standard

Dashboard — everything in Essential, plus Memory usage by state, Disk I/O, and Network errors.

Alerts (5)

Alert	Severity	Fires when
Host load high	critical	5m load > 3× cores for 10m
Host load high	warning	5m load > 1.5× cores for 15m
Host memory usage high	critical	memory used > 90% for 5m
Host filesystem usage high	critical	any mount > 90% for 10m
Host filesystem usage high	warning	any mount > 80% for 15m

Review the bundle JSON →

Extended

Deep host visibility. 1 dashboard · 15 panels · 9 alerts — every hostmetrics scraper and the full alert set. For hosts you need to diagnose, not just watch.

Import this variant from the catalog. Its demand slug:

Demand slug

host-metrics-otel-demand-extended

Dashboard — everything in Standard, plus CPU utilization, CPU by state, Swap usage, Inode utilization, Disk operations, Disk busy time, Network dropped packets, and Processes by status. (The process panel needs the processes scraper — see Setup.)

Alerts (9)

Alert	Severity	Fires when
Host load high	critical	5m load > 3× cores for 10m
Host load high	warning	5m load > 1.5× cores for 15m
Host memory usage high	critical	memory used > 90% for 5m
Host memory usage high	warning	memory used > 80% for 15m
Host filesystem usage high	critical	any mount > 90% for 10m
Host filesystem usage high	warning	any mount > 80% for 15m
Host filesystem inodes high	warning	inodes > 85% for 15m
Host swap usage high	warning	swap used > 50% for 15m
Host network errors high	warning	error rate > 10/s for 5m

Review the bundle JSON →

Dashboards

Panels are grouped into collapsible sections (CPU, Memory, Filesystem, Disk, Network, Processes), and every query is grouped by host_name, so one import covers the whole fleet — view every host at once, or focus on one. Legends are sortable tables showing last/mean/max per series, so the outlier host stands out. Dashboards are demand-driven: importing one tells your agents to forward exactly the timeseries its panels query, and nothing else. Panels are ordinary Perses panels — edit queries, add panels, or retune them after import, and your changes reshape the demand automatically.

Alerts

Thresholds are conservative starting points — tune them to your fleet.

Caution

Imported alert rules arrive without a notification channel and start out paused — with nowhere to deliver, they do not evaluate. After importing, create or pick a channel under Alerts → Manage Channels, attach it to each rule, and resume. Rules without a channel are marked with a warning icon in the alerts list.

Next steps

• Demand-driven observability — why importing a demand also shapes what your agents collect.
• Agent configuration — the full agent config surface.