Prometheus MCP Server

• mcp-servers/prometheus/internal/prom/client.go
• mcp-servers/prometheus/internal/prom/format.go
• mcp-servers/prometheus/internal/tools/tools.go
• QueryInput
• QueryRangeInput
• SeriesInput
• SearchMetricsInput
• MetricMatch
• MetricMeta
• Series

The Prometheus MCP server is Korvus's window onto metrics. It is a standalone binary, mcp-prometheus, that speaks MCP over streamable HTTP and exposes a small set of read-only PromQL tools against a Prometheus-compatible HTTP API v1 endpoint. Korvus never spawns it — the container runtime does, and Korvus reaches it over the URL configured under mcp.servers (Bundled MCP Servers Overview).

The server exists because metrics are the cheapest evidence an incident investigation can start from. Before an agent reads a log line or a trace, it usually needs to know what the service was doing: error rate, saturation, whether an alert's expression is still firing. Prometheus already stores that history, and its query API is the interface to it. Wrapping it in MCP lets the agent ask the same questions it would ask a human, using the tool surface described in MCP Client, Registry and Server Connection.

The module is deliberately narrow. mcp-servers/prometheus/internal/prom is a hand-written client for four read endpoints rather than the upstream Prometheus Go SDK — that SDK pulls in the whole Prometheus common/model tree for a handful of GETs, and this module's dependency budget is one SDK (client.go). The companion Loki MCP Server and Tempo MCP Server follow the same shape, one per signal type, and the three are deployed side by side in the example stack (Deployment Stack and Example Configuration).

How it works

The process starts in mcp-prometheus/cmd/mcp-prometheus/main.go. Two flags (or environment variables) decide where it listens and which Prometheus it talks to: -addr / MCP_PROMETHEUS_ADDR, defaulting to :9100, and -prometheus-url / PROMETHEUS_URL, defaulting to http://localhost:9090 (main.go). The timeout constants live right above run: a per-request queryTimeout, plus separate read-header, write and idle timeouts for the HTTP server (main.go).

run builds a prom.Client for the configured base URL, constructs a single MCP server named mcp-prometheus, and hands both to tools.Register (main.go). One server instance serves every session: the tools hold no per-session state and the upstream client is safe for concurrent use (main.go). The streamable HTTP handler is mounted at /mcp and a liveness probe at /healthz on the same port (main.go) — which is why the example configuration points Korvus at http://localhost:9100/mcp and not at a bare host and port (Configuration Loading and Validation).

The tool surface itself lives in mcp-servers/prometheus/internal/tools/tools.go. Register adds five tools to the server, all carrying a read-only annotation: prometheus_query, prometheus_query_range, prometheus_series, prometheus_search_metrics and prometheus_label_values (tools.go). Each tool's description and JSON-schema tags are the contract the model programs against, and they are written for a model with no prior knowledge of the deployment: every argument states its exact format, and every description says what the tool is for rather than what it wraps (tools.go).

Input arguments are typed structs. QueryInput carries a PromQL expression and an optional evaluation instant (tools.go); QueryRangeInput adds an inclusive start, end and step (tools.go); SeriesInput carries one or more series selectors and an optional window (tools.go); SearchMetricsInput carries a name pattern and a limit (tools.go).

Timestamps are parsed strictly as RFC 3339 by parseTime, and steps by parseStep, which accepts Go duration syntax but deliberately rejects a bare number so there is exactly one way to write a step (tools.go). The accepted timestamp form is documented once as TimeFormatHelp and repeated verbatim in every time argument, with a test asserting the repetitions still agree, because a schema that accepts unix seconds in one field and RFC 3339 in another invites a model to guess (tools.go).

A range query is not sent at the step the model asked for. prom.AdjustStep widens the step until the window fits within MaxPointsPerSeries, returning the step actually usable; widening rather than erroring keeps the model moving, since a coarser answer over the whole window beats a rejection it has to guess its way out of (format.go). The rendered result then tells the model that its requested step was widened, because a model comparing two range queries has no other way to notice its resolution changed (format.go).

The prom package itself is a thin HTTP client. NewClient trims the base URL and installs an http.Client whose timeout bounds each request end to end (client.go). Query omits the time parameter entirely when the instant is zero, so "now" is evaluated server-side and a clock skew between Korvus and Prometheus cannot silently shift the evaluation (client.go). QueryRange sends query, start, end and step to /api/v1/query_range (client.go). Series posts match[] selectors to /api/v1/series and returns raw label sets (client.go). LabelValues builds /api/v1/label/<name>/values with the label name path-escaped, because an unescaped name would let a caller reach a different endpoint entirely (client.go).

Query results decode into Result, which carries Prometheus's resultType — vector, matrix, scalar or string — and either a list of Series or a single scalar sample (client.go). A Series holds its label set plus either one Value (an instant query) or all Values (a range query) (client.go). Sample values stay strings exactly as Prometheus sent them: parsing to float64 and formatting again loses precision and turns NaN into something the model has to decode (client.go).

Search is the one tool that needs two endpoints. prometheus_search_metrics lists metric names via LabelValues on __name__ and then looks up metadata separately, because names come from the TSDB while metadata comes from the scrape targets and the two disagree in both directions. __name__ is the authority on what can actually be queried; a metadata failure therefore degrades the result with a warning rather than failing the search (tools.go). Client.Metadata returns per-name MetricMeta — type, HELP text and unit — keeping the first entry when targets declare the same metric differently, since a disagreement between targets is not something the model can act on (client.go).

Result rendering and the byte budget

Every tool result is rendered text, and the rendering lives in mcp-servers/prometheus/internal/prom/format.go. The whole file is organised around one constraint: Korvus's agent loop truncates any tool result above 32 KiB head-and-tail and replaces the middle with a generic notice. That notice is a worse outcome than trimming here, because it can cut a series' label set in half and cannot tell the model which series it lost, so the formatters render under the rail and let the model always see whole series plus an accurate account of what was dropped (format.go).

FormatInstant renders one line per series: the label set, then the value, then its timestamp (format.go). The label set itself is rendered deterministically by formatLabels, which puts __name__ first as the metric name and the remaining labels in sorted order (format.go).

FormatRange renders series whole or not at all — half a series reads as a series that stopped reporting, which is a false RCA signal (format.go). When not even one series fits, the formatter downsamples the first series rather than returning only a trailer, because dropping the tail would silently answer a narrower window than the one asked about. The downsampled trailer states the effective resolution and the remedy: re-run with a shorter time range, since a coarser step will not help (format.go).

The stride used for that thinning is derived from the samples' real byte cost rather than a constant, since a fixed estimate is wrong in the direction that matters — an underestimate overflows the budget it exists to respect. pointCost measures the exact line width, including the rendered timestamp rather than an assumed one, and strideToFit starts from the average cost, verifies the estimate, and widens the stride until the selection really fits (format.go).

Search hits are rendered by FormatMetricSearch, which pairs each name with its declared type and HELP text. The type and HELP are the point: a bare list of names makes the model guess which of several similar metrics answers its question, and a guess costs a whole tool call to discover it was wrong (format.go). Each hit is a MetricMatch — a name plus the MetricMeta its targets declared (format.go).

Error handling

Failures reach the model as sentences it can act on. The client classifies every failure into a QueryError whose Kind is Prometheus's own errorType (bad_data, timeout, execution, canceled) or one of two synthetic kinds for failures Prometheus never got to classify — unreachable and bad_response (client.go). A context deadline is reported as a timeout rather than as an unreachable server (client.go).

On the tool side, explain turns those kinds into remediation: a bad_data failure says the expression is invalid and points at the discovery tools, a timeout says to narrow the range, add matchers or aggregate, an execution failure says to reduce the expression's scope, and an unreachable server is explicitly flagged as an infrastructure fault the model should not retry repeatedly (tools.go).

Two bounds sit below the model. Response bodies are read through a limit reader, because a wide query_range over months legitimately produces hundreds of megabytes and the result is truncated to a few KiB for the model regardless, so reading it all would only OOM the server (client.go). And the metric-name search is capped by a default and a maximum limit, so a pattern like . cannot pull every name in the TSDB into the context window (tools.go).

Relationships

graph TD
    main["mcp-servers/prometheus/cmd/mcp-prometheus/main.go"]
    tools["mcp-servers/prometheus/internal/tools/tools.go"]
    client["mcp-servers/prometheus/internal/prom/client.go"]
    format["mcp-servers/prometheus/internal/prom/format.go"]

    main --> tools
    main --> client
    tools --> client
    tools --> format
    tools --> QueryInput
    tools --> QueryRangeInput
    tools --> SeriesInput
    tools --> SearchMetricsInput
    client --> Series
    client --> MetricMeta
    format --> MetricMatch

The Prometheus server is one of the MCP servers in the bundled set; see Bundled MCP Servers Overview for how the three signal servers are deployed together, and Korvus Overview for where the agent's tool calls enter this picture.

Reference

Generated by astrolabe from commit 0c2dca73 on branch main.