Model Providers, Embeddings and Error Kinds

• internal/model/provider.go
• internal/model/anthropic.go
• internal/model/openai.go
• internal/model/openai_embed.go
• internal/model/embedder.go
• internal/model/errors.go
• Provider
• Embedder
• Message
• ToolDef
• Usage
• CompletionRequest
• CompletionResponse
• AnthropicProvider
• OpenAIProvider
• OpenAIEmbedder
• ProviderError

internal/model is the seam between Korvus and whatever model API actually answers the questions. The package exists so the agent loop in internal/agent never learns a wire format: it builds a CompletionRequest, hands it to a Provider, and gets back a CompletionResponse plus a classified error. Every quirk — Anthropic's content blocks, OpenAI's tool-call encodings, cache breakpoints, the five different ways an upstream can say "your context is too long" — stays inside the adapters provider.go.

Two narrow interfaces sit at that seam. Provider is one request/response completion call Provider; Embedder turns text into vectors for document retrieval Embedder. They are deliberately separate: Anthropic has no embeddings endpoint, so folding Embed into Provider would force the Anthropic adapter to answer "unsupported" for something its own interface promised embedder.go.

The package's other job is error classification. Every failure that comes back from a provider is wrapped in a *ProviderError carrying an ErrorKind, and that kind is what the retry and failover logic in the agent run loop switches on ProviderError.

How it works

The neutral contract

The request type is deliberately small. CompletionRequest carries a system prompt, a slice of Message, the tools on offer, a token cap and a temperature CompletionRequest. Message is the canonical transcript entry: a Role (user, assistant, tool), text content, optional tool calls, and — for tool results — the call ID being answered, the tool name, and an error flag Message. Tool-call arguments stay raw json.RawMessage; the agent loop parses and validates them, never the adapter ToolCall. ToolDef carries a name, a description and a JSON Schema that came straight from the MCP registry ToolDef.

The response normalizes in the other direction. CompletionResponse gives back concatenated text, the requested tool calls, a normalized StopReason (end, tool_use, length, filtered) and token accounting in Usage CompletionResponse StopReason Usage.

The Anthropic adapter

AnthropicProvider wraps the official SDK client and a model ID AnthropicProvider. NewAnthropicProvider disables SDK-internal retries with option.WithMaxRetries(0), because retry policy belongs to the caller — the agent loop, which owns backoff and the one-way provider fallback NewAnthropicProvider. The same rule holds for the OpenAI adapter NewOpenAIProvider.

Complete does three interesting things. First, it attaches an ephemeral cache breakpoint to the system prompt, because the request prefix is byte-identical across a session's calls and caching it cuts input-token cost several-fold anthropic.go. Second, it decomposes each tool's JSON Schema into properties and required, since the Anthropic tool parameter wants them split rather than as one blob anthropic.go. Third, on the way back it folds text blocks into a single string and turns tool_use blocks into ToolCalls anthropic.go.

The hard part is message folding. The Messages API requires roles to alternate strictly, but RoleUser and RoleTool both render as user, so the ordinary agent tail (assistant → tool → tool → user) would arrive as three consecutive user messages and be rejected anthropicMessages. The fold merges same-role blocks into the last already-emitted message rather than into a pending buffer, which makes a zero-block message truly invisible and keeps alternation true by construction anthropic.go. A message that renders to no blocks is dropped entirely, because an empty text block and an empty content array are both rejected by the API anthropicBlocks.

Two smaller invariants live here. A tool result with an empty body is replaced by a placeholder string, since content: [{"type":"text","text":""}] is a 400 and dropping the block would leave an unanswered tool_use, which is its own 400 emptyToolResultText. And tool-call arguments are normalized to a JSON object by toolInput, because an empty json.RawMessage makes the SDK encoder fail before the HTTP call — which would otherwise be misread as a retryable timeout and burn every attempt plus the fallback provider toolInput.

The OpenAI adapter

OpenAIProvider targets any OpenAI-compatible chat-completions endpoint — OpenAI, vLLM, Mistral, OpenRouter OpenAIProvider. Its Complete translates the neutral transcript into chat messages, with the system prompt promoted to its own leading message, and maps the response's finish_reason onto StopReason openaiMessages openaiStopReason. The chat-completions tool message has no error flag, so an errored tool result is prefixed with ERROR: to keep it visible to the model openai.go.

Embeddings

OpenAIEmbedder talks to any OpenAI-compatible /v1/embeddings endpoint — OpenAI, vLLM, TEI, Ollama OpenAIEmbedder. The constructor takes the vector width and sends it as the request's dimensions parameter, so the model must accept it; one that ignores it is caught by the response width check, one that rejects it answers 400 NewOpenAIEmbedder.

Embed is defensive about ordering. The API promises no response ordering, only an index field per embedding, so results are placed by index rather than appended — appending would attach a chunk's text to another chunk's vector, a corruption no error surfaces and that only degraded retrieval would ever reveal Embed. It also rejects out-of-range indices, repeated indices, a width mismatch, and any slot left nil openai_embed.go. The float64-to-float32 narrowing happens in float32Vector, because pgvector is a float32 type and doing it here spares the store layer a conversion path it would only ever narrow float32Vector.

Error kinds

ErrorKind is a small taxonomy that exists to answer one question: is this worth retrying? ErrorKind

classifyStatus maps an HTTP status plus body text onto that taxonomy and is shared by all three adapters classifyStatus. Two cases need body sniffing: 5xx bodies that contain "overloaded", and 4xx bodies that describe a context overflow — OpenAI-compatible servers such as vLLM report overflow as a plain 400, so the message text is searched for phrasings like "context length", "context_length_exceeded" or "prompt is too long" looksLikeOverflow looksLikeOverloaded.

Each adapter has its own thin classifier that extracts the SDK's typed error, pulls the response headers, and builds a ProviderError with the kind, status code and retry hint classifyOpenAIError classifyAnthropicError classifyOpenAIEmbedError. Anything that is not a typed API error is treated as a transport failure and reported as ErrKindTimeout, which is retryable openai.go. The retry hint comes from retryAfterFromHeader, which prefers the retry-after-ms extension header and falls back to Retry-After in seconds; HTTP-date values are ignored because a wrong parse is worse than backoff retryAfterFromHeader.

KindOf is the function callers use: it unwraps a *ProviderError if there is one, and maps context deadline or cancellation onto ErrKindTimeout, defaulting everything else to timeout as well KindOf.

Relationships

graph LR
  agent[internal/agent] --> Provider
  Provider --> AnthropicProvider
  Provider --> OpenAIProvider
  AnthropicProvider --> Message
  AnthropicProvider --> ToolDef
  AnthropicProvider --> CompletionResponse
  OpenAIProvider --> CompletionRequest
  OpenAIProvider --> CompletionResponse
  CompletionRequest --> Message
  CompletionRequest --> ToolDef
  CompletionResponse --> Usage
  Embedder --> OpenAIEmbedder
  OpenAIEmbedder --> ProviderError
  AnthropicProvider --> ProviderError
  OpenAIProvider --> ProviderError
  ProviderError --> Usage

The agent loop consumes the package through exactly two entry points: it builds a CompletionRequest per call and hands it to a Provider, and it classifies whatever comes back with KindOf Agent::callModel. The CLI wiring turns configured provider definitions into AnthropicProvider or OpenAIProvider instances newProvider, and builds the Embedder separately, validating the configured vector width against the live docs_chunks.embedding column before the process starts buildEmbedder. The embedder's output feeds the documentation indexing and search path, and provider identity reaches the OpenTelemetry GenAI attributes from the same span the agent records on. See also Configuration Loading and Validation for the embedding block itself and Storage Domain Model and Store Interfaces for the column the width check compares against.

Error handling

The contract is that adapters never retry and always classify. Provider implementations must not retry internally — the agent loop owns retry policy — and must wrap every failure in a *ProviderError so the loop can classify it Provider. ProviderError implements Error and Unwrap, so errors.Is/errors.As reach the underlying cause ProviderError::Error ProviderError::Unwrap.

The consumer side is where the taxonomy pays off. The agent loop treats overflow and bad-request errors as deterministic and gives up immediately, fails over on auth failures without retrying, and treats rate limit, overloaded, server and timeout as transient enough for jittered exponential backoff, honouring RetryAfter when the provider supplies one caller.go.

Reference

Generated by astrolabe from commit 0c2dca73 on branch main.