Go

Collect Go runtime and HTTP metrics from your services and get curated dashboards and alerts for them with a single import.

Go apps instrumented with the OpenTelemetry SDK export their metrics over OTLP to your LeanSignal agent gateway (one per cluster or environment) — never to a remote or third-party endpoint. The gateway keeps full fidelity locally and forwards only the demanded subset to your central dataplane. This integration covers the Go runtime (memory, goroutines, GC goal, scheduler latency, GOMAXPROCS/GOGC) plus HTTP server request metrics from otelhttp.

Metric names follow the standard OTLP → Prometheus normalization (dots become underscores, unit/_total suffixes are added), so go.goroutine.count becomes go_goroutine_count and http.server.request.duration becomes http_server_request_duration_seconds. The demands below are built against those names.

Prerequisites

• A LeanSignal agent gateway deployed and connected — one per cluster or environment. If you haven’t deployed it yet, follow Install the agent (source: LeanSignal/leansignal-agent). It should show as Connected under Agents in the LeanSignal app.
• Network reachability to the gateway’s OTLP port — 4317 (gRPC) or 4318 (HTTP). See Agent configuration.
• Your Go app instrumented with the OpenTelemetry SDK, exporting metrics over OTLP to the gateway — configured in Setup.
• Editor or admin role in the LeanSignal app (importing a demand creates dashboards and alert rules).

Setup

Add the runtime, OTLP-metric, and HTTP instrumentation packages, then point the SDK at your LeanSignal agent gateway (OTLP gRPC :4317 / HTTP :4318) — not a SaaS endpoint.

go get go.opentelemetry.io/contrib/instrumentation/runtime
go get go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc
go get go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp

export OTEL_EXPORTER_OTLP_ENDPOINT="http://<gateway-host>:4317"
export OTEL_EXPORTER_OTLP_PROTOCOL="grpc"    # or http/protobuf -> :4318
export OTEL_SERVICE_NAME="my-go-service"
export OTEL_SEMCONV_STABILITY_OPT_IN="http"  # emit stable http.server.request.duration

Wire up the meter provider and start the runtime producer:

exp, _ := otlpmetricgrpc.New(context.Background()) // reads OTEL_EXPORTER_OTLP_ENDPOINT
mp := sdkmetric.NewMeterProvider(
    sdkmetric.WithReader(sdkmetric.NewPeriodicReader(exp)),
    // The second reader enables go.schedule.duration:
    sdkmetric.WithReader(sdkmetric.NewPeriodicReader(exp, sdkmetric.WithProducer(runtime.NewProducer()))),
)
otel.SetMeterProvider(mp)
_ = runtime.Start(runtime.WithMinimumReadMemStatsInterval(time.Second))

// Instrument your HTTP server:
handler := otelhttp.NewHandler(mux, "server")

Metric names and opt-ins

Current runtime metrics use the go.* prefix (e.g. go_memory_used_bytes, go_goroutine_count); legacy runtime.go.* names only re-emit under OTEL_GO_X_DEPRECATED_RUNTIME_METRICS=true. The scheduler-latency panel/alert needs runtime.NewProducer() registered on a reader (as above), otherwise go_schedule_duration_seconds_* is absent. The GOMEMLIMIT-based memory alert is only meaningful when GOMEMLIMIT is set. http.server.request.duration (seconds) requires OTEL_SEMCONV_STABILITY_OPT_IN=http.

Where to send this

You have two options:

• Straight to the LeanSignal agent gateway. Point the app’s OTEL_EXPORTER_OTLP_ENDPOINT at it — nothing else in between.
• Via an OpenTelemetry Collector in agent mode. Point the app at a local collector and give that collector an otlp exporter to the gateway (leansignal-agent:4317).

Both are supported — pick whichever suits your topology. A local collector tends to fit when the app has no route to the gateway (a different network or cluster), or when you want local buffering, batching, or extra processing; exporting directly means one less process to operate.

Multiple instances

Give each running instance a stable identity so replicas stay distinct: set service.instance.id, exported as the instance label — for example OTEL_RESOURCE_ATTRIBUTES=service.instance.id=$(hostname), or your orchestrator’s pod name. The imported dashboard has an Instance filter — view all instances at once or focus on one.

Logs and traces

Go has no zero-code agent, so each signal is wired in code against the same gateway endpoint:

• Traces — set up a TracerProvider with an OTLP exporter and wrap handlers/clients with otelhttp (or your framework’s middleware); the net/http instrumentation that feeds the metrics above emits spans from the same wrappers.
• Logs — bridge slog with otelslog (or otelzap for zap) over an OTLP LoggerProvider; or keep logging to stdout and ship files with a collector or shipper instead.

See the Go OpenTelemetry docs.

Demand-driven, like metrics

Logs and traces land in the gateway’s edge stores (a roughly one-hour full-fidelity window) and are queryable immediately in the LeanSignal app under Logs and Traces with the source set to Available. Nothing is stored centrally until a dashboard demands it — the ready-made demands on this page stay metrics-only. See Logs and Traces.

Demands

Go ships as a ready-made demand in three variants. You don’t copy any JSON — in the LeanSignal app go to Demands, press the ˅ arrow on the Add Demand button (the arrow opens the import menu), choose Import from catalog…, and pick the variant. Each variant below lists the slug that identifies its published bundle. See Integrations for the full import flow.

The variants are nested — Standard is a superset of Essential, Extended a superset of Standard. Pick whichever is closest to what you need and treat it as a starting point: after import everything is a normal, editable copy, so retune thresholds, add or drop panels, and adjust queries for your environment — your edits reshape the demand automatically. Start small and re-import a larger variant later if you outgrow it.

Essential

The golden signals. 1 dashboard · 4 panels · 3 alerts — traffic, errors, latency, and goroutines at a glance.

Import this variant from the catalog. Its demand slug:

Demand slug

golang-otel-demand-essential

Dashboard — HTTP request rate, HTTP 5xx error rate, HTTP request duration p95, and Live goroutines.

Alerts (3)

Alert	Severity	Fires when
High HTTP 5xx error rate	critical	5xx ratio > 5% for 5m
High HTTP p95 latency	warning	p95 latency > 1s for 5m
Goroutine count spike	warning	> 10,000 goroutines for 10m

Review the bundle JSON →

Standard

Well-rounded runtime coverage. 1 dashboard · 8 panels · 6 alerts — adds scheduler latency, memory, and GC pacing. The recommended default.

Import this variant from the catalog. Its demand slug:

Demand slug

golang-otel-demand-standard

Dashboard — everything in Essential, plus Goroutine scheduling latency p95, Runtime memory used, Heap allocation rate, and GC heap goal.

Alerts (6)

Alert	Severity	Fires when
High HTTP 5xx error rate	critical	5xx ratio > 5% for 5m
High HTTP p95 latency	warning	p95 latency > 1s for 5m
Goroutine count spike	warning	> 10,000 goroutines for 10m
Goroutine scheduling latency high	warning	p99 schedule latency > 100ms for 10m
Runtime memory near GOMEMLIMIT	critical	post-GC memory floor > 90% of GOMEMLIMIT for 5m
High runtime memory usage	warning	runtime memory > 1 GiB for 10m

Review the bundle JSON →

Extended

Deep runtime visibility. 1 dashboard · 14 panels · 10 alerts — per-route traffic, allocation churn, and the GOMEMLIMIT/GOMAXPROCS/GOGC picture for services you need to diagnose, not just watch.

Import this variant from the catalog. Its demand slug:

Demand slug

golang-otel-demand-extended

Dashboard — everything in Standard, plus HTTP request duration p99, HTTP request rate by route, Runtime memory limit (GOMEMLIMIT), Allocation operations, GOMAXPROCS (processor limit), and GOGC target.

Alerts (10)

Alert	Severity	Fires when
High HTTP 5xx error rate	critical	5xx ratio > 5% for 5m
High HTTP p95 latency	warning	p95 latency > 1s for 5m
Goroutine count spike	warning	> 10,000 goroutines for 10m
Goroutine scheduling latency high	warning	p99 schedule latency > 100ms for 10m
Runtime memory near GOMEMLIMIT	critical	post-GC memory floor > 90% of GOMEMLIMIT for 5m
High runtime memory usage	warning	runtime memory > 1 GiB for 10m
High heap allocation rate	warning	allocating > 512 MiB/s for 10m
Sustained goroutine growth	warning	goroutines +1,000 over 30m for 15m
Elevated HTTP 4xx rate	warning	4xx ratio > 30% for 10m
Critical HTTP p99 latency	critical	p99 latency > 3s for 5m

Review the bundle JSON →

Dashboards

Each variant imports one Go dashboard, its panels grouped into collapsible sections. Every dashboard carries an Instance filter (on the instance label), so one import covers the whole fleet — view every instance at once, or focus on one. Legends are sortable tables showing last/mean/max per series, so the outlier instance stands out. Dashboards are demand-driven: importing one tells your agents to forward exactly the timeseries its panels query, and nothing else. Panels are ordinary Perses panels — edit queries, add panels, or retune them after import, and your changes reshape the demand automatically.

Alerts

Thresholds are conservative starting points — tune them to your workload.

Caution

Imported alert rules arrive without a notification channel and start out paused — with nowhere to deliver, they do not evaluate. After importing, create or pick a channel under Alerts → Manage Channels, attach it to each rule, and resume. Rules without a channel are marked with a warning icon in the alerts list.

Next steps

• Demand-driven observability — why importing a demand also shapes what your agents collect.
• Agent configuration — the full agent config surface.