Zoltai fake LLM provider

Zoltai is a fake LLM provider for tests and offline development. It implements the same llm.ProviderPort and llm.EmbeddingProviderPort as the real providers, so it drops into any call site with no code changes. Completions return a fortune selected from a pool. Embeddings return deterministic vectors derived from the input text. No network, no API key, no per-token cost.

Overview

Zoltai takes its name from the fortune-telling arcade machine. Ask it a question and it returns a fortune from its pool, wrapped by default in the Zoltai oracle preamble and postamble. It serves as a drop-in replacement for Anthropic, OpenAI, or any other LLM provider when running a real model is wrong. Common cases include unit tests that should not hit the network and CI pipelines without provider credentials. Demos that need to work offline fit the same shape, as does early local development against the LLM service before you choose a real provider.

Because Zoltai uses the same public ports as the real providers, swapping it in is a one-line bootstrap change. One provider instance covers the whole fake LLM stack. piko auto-detects its embedding support and auto-wires embeddings, so you write no glue. Zoltai also exercises the full LLM surface in tests, not plain completions alone. It honours tool calls, structured-output responses (JSON object and JSON schema), streaming, model listing, and usage token accounting. Your code paths run the same way they run against a real provider.

Reach for Zoltai when you need the LLM service to exist but do not care what it says. Do not reach for it when the test cares about the actual response content. Zoltai ignores the prompt entirely and returns a fortune from its pool. For deterministic same-prompt-same-output behaviour, set Seed to a fixed value so fortune selection is reproducible.

Embeddings are deterministic. The same input text always produces the same vector. They are not semantically meaningful, and two unrelated inputs may end up close together in the vector space, so do not use Zoltai embeddings to validate retrieval quality.

Configuration

import (
    "piko.sh/piko/wdk/llm/llm_provider_zoltai"
)

provider, err := llm_provider_zoltai.NewZoltaiProvider(llm_provider_zoltai.Config{
    DefaultModel:          "zoltai-1",       // model name reported in responses
    DefaultEmbeddingModel: "zoltai-embed-1", // model name reported in embedding responses
    EmbeddingDimensions:   384,              // vector size for fake embeddings
    Fortunes:              nil,              // nil = use built-in fortune pool
    FormatFortune:         nil,              // nil = wrap fortunes in the Zoltai preamble/postamble
    Seed:                  0,                // 0 = random seed at construction
})
if err != nil {
    return err
}

Every field has a default, so NewZoltaiProvider(Config{}) is a valid call. Override Fortunes to inject test-specific responses, override FormatFortune to control the wrapping text, and set Seed to a fixed value for reproducible test runs.

The provider is a separate Go module, piko.sh/piko/wdk/llm/llm_provider_zoltai, with its own go.mod. It is pure Go and needs no build tag or CGO, so it adds nothing to the build and behaves the same in compiled mode and in interpreted dev mode (dev-i). Apart from the piko framework module itself, its only direct dependencies are the test and telemetry libraries.

Close runs a bounded drain and stays idempotent. It cancels any in-flight streaming goroutines and waits up to 30 seconds for them to finish. Test teardown that exercises streaming is safe to call Close once or more than once.

Bootstrap

Register Zoltai with WithLLMProvider and WithDefaultLLMProvider. That single pair wires both completions and embeddings. piko detects that the provider also implements EmbeddingProviderPort, auto-registers it as an embedding provider under the same name, and auto-selects it as the default embedding provider:

ssr := piko.New(
    piko.WithLLMProvider("zoltai", provider),
    piko.WithDefaultLLMProvider("zoltai"),
)

WithEmbeddingProvider and WithDefaultEmbeddingProvider are not needed here. Use WithEmbeddingProvider only when you want to register the embedding side under a separate name, for example to point the default embedding provider at a different instance.

For mixed setups (real provider in production, Zoltai in tests), gate the registration on environment:

provider := selectProviderByEnv() // returns Anthropic in prod, Zoltai in tests
ssr := piko.New(
    piko.WithLLMProvider("default", provider),
    piko.WithDefaultLLMProvider("default"),
)

See also

Other LLM providers (the real ones Zoltai stands in for):

  • Anthropic Claude, long-context, strong tool use.
  • OpenAI, widest model selection, native structured-output API.
  • Gemini, cheapest competent option, multimodal-native.
  • Mistral, open-weight European provider.
  • Grok, xAI provider.
  • Ollama, local inference for offline workloads (a real alternative to Zoltai when you need real outputs offline).
  • Voyage, embeddings-only specialist.

Framework docs: