LLM API

Piko's LLM service abstracts over Anthropic, OpenAI, Gemini, Grok, Mistral, Ollama, Voyage, and Zolt.ai. It supports plain completions, streaming, tool calling, structured JSON, embeddings, retrieval-augmented generation, cost tracking, budgets, rate limits, and response caching. For task recipes see how to LLMs and embeddings. Source of truth: wdk/llm/facade.go.

Service

Function	Returns
`llm.NewService(defaultProviderName string, opts ...ServiceOption) Service`	Constructs a new service.
`llm.GetDefaultService() (Service, error)`	Returns the bootstrap-configured service.

Builders

func NewCompletionBuilder(service Service) *CompletionBuilder
func NewCompletionBuilderFromDefault() (*CompletionBuilder, error)
func NewEmbeddingBuilder(service Service) *EmbeddingBuilder
func NewEmbeddingBuilderFromDefault() (*EmbeddingBuilder, error)

Common fluent methods on CompletionBuilder include .Model(...), .Messages(...), .Tools(...), .ResponseFormat(...), .Stream(...), .Temperature(...), .MaxTokens(...), .Do(ctx). The full surface also covers RAG (WithRAGQuery, WithRAGMinScore, WithRAGFilter, WithRAGEmbeddingProvider, WithRAGEmbeddingModel, WithRAGQueryRewriter, WithRAGHybridSearch), vector context (WithVectorContext), tool choice, top-p, stop sequences, seeds, and per-call cost or rate-limit overrides. See wdk/llm/facade.go for the authoritative list.

Message helpers

NewSystemMessage(content string) Message
NewUserMessage(content string) Message
NewAssistantMessage(content string) Message
NewToolResultMessage(toolCallID, content string) Message
NewUserMessageWithImages(content string, images ...ContentPart) Message
NewUserMessageWithImageURL(content string, url, detail string) Message
NewUserMessageWithImageData(content string, data []byte, mimeType, detail string) Message

Tools and structured output

NewFunctionTool(name, description string, params JSONSchema) ToolDefinition
NewStrictFunctionTool(name, description string, params JSONSchema) ToolDefinition
ToolChoiceAuto() ToolChoice
ToolChoiceNone() ToolChoice
ToolChoiceRequired() ToolChoice
ToolChoiceSpecific(name string) ToolChoice

JSONSchemaDefinition and JSONSchema describe response formats. Set ResponseFormat.Type = ResponseFormatJSONSchema with ResponseFormat.JSONSchema to force a structured output.

Cost and budget

Function	Purpose
`NewCostCalculator()`	Cost calculator with the default pricing table.
`NewCostCalculatorWithPricing(table *PricingTable)`	Custom pricing.
`NewBudgetManager(store, calculator)`	Enforces spending limits.
`DefaultPricingTable`	The shipped pricing table.

BudgetStorePort persists spending counters. Provide a cache.Cache or custom backing store.

Rate limiting

Function	Purpose
`NewRateLimiter(store, opts...)`	Per-model, per-user, or global rate limit.
`RateLimiterStorePort`	Driven port for counter state.
`WithRateLimiterClock(clk)`	Injects a test clock.

Response cache

NewCacheManager(store, defaultTTL) returns a manager that short-circuits completions whose inputs match a previously cached request.

Memory

Function	Purpose
`NewBufferMemory(store, opts...)`	Fixed-size conversation buffer.
`NewWindowMemory(store, opts...)`	Token-window buffer.
`NewSummaryMemory(store, service, config)`	LLM-summarised long history.

Retrieval-augmented generation

Function	Purpose
`NewVectorStore(factory)`	Cache-backed vector store.
`Service.NewIngest(namespace) *IngestBuilder`	Method on `Service` (not a top-level function). Returns a fluent builder that loads, splits, transforms, and vectorises documents into `namespace`.
`NewRecursiveFSLoader(fsys, patterns...)`	Filesystem document loader.
`NewRecursiveCharacterSplitter(chunkSize, overlap) (SplitterPort, error)`	Token-agnostic chunker. Returns an error when `chunkSize` or `overlap` are invalid.
`NewMarkdownSplitter(chunkSize, overlap, opts...) (SplitterPort, error)`	Markdown-aware chunker. Returns an error when configuration is invalid.
`StripFrontmatter()`, `ExtractFrontmatter(opts...)`	Frontmatter transforms.
`PrependChunkContext()`	Attaches chunk position metadata.
`LLMQueryRewriter(opts...)`	Multi-query expansion.

Attach RAG to a completion with CompletionBuilder.WithRAG(...) options: WithRAGQuery, WithRAGMinScore, WithRAGFilter, WithRAGEmbeddingProvider, WithRAGEmbeddingModel, WithRAGQueryRewriter, WithRAGHybridSearch.

Errors

ErrProviderNotFound, ErrNoDefaultProvider, ErrProviderAlreadyExists, ErrStreamingNotSupported, ErrToolsNotSupported, ErrStructuredOutputNotSupported, ErrEmptyMessages, ErrEmptyModel, ErrInvalidTemperature, ErrInvalidTopP, ErrInvalidMaxTokens, ErrBudgetExceeded, ErrRateLimited, ErrMaxCostExceeded, ErrUnknownModelPrice, ErrProviderOverloaded, ErrProviderTimeout, ErrConversationNotFound.

Providers

Sub-package	Backend
`llm_provider_anthropic`	Claude models.
`llm_provider_openai`	OpenAI GPT and o-series.
`llm_provider_gemini`	Google Gemini.
`llm_provider_grok`	xAI Grok.
`llm_provider_mistral`	Mistral.
`llm_provider_ollama`	Local Ollama.
`llm_provider_voyage`	Voyage AI embeddings.
`llm_provider_zoltai`	Zolt.ai.

Ollama

llm_provider_ollama.NewOllamaProvider(config) performs no network requests or process startup. The first request connects to the server, starting ollama serve if AutoStart permits it. Start(ctx) and EnsureModels(ctx) allow explicit initialisation before requests arrive.

Config contains the following lifecycle limits.

Field	Default	Purpose
`StartupTimeout`	30 seconds	Wait for a spawned server to answer health checks.
`ProbeTimeout`	2 seconds	Each health or reachability request.
`StopGracePeriod`	5 seconds	Time allowed for a managed server to exit after an interrupt, before forced termination.

Image fetching

A non-nil Config.ImageFetch enables fetching image URLs from messages. Destinations must have publicly routable addresses. The provider rejects loopback, private, link-local, and cloud metadata destinations. Inline image data does not require URL fetching.

`ImageFetchConfig` field	Default	Purpose
`MaxImages`	8	Maximum image URLs per request. The provider rejects excess URLs before downloading any image.
`MaxBytes`	20 MiB	Maximum bytes per image.
`Timeout`	30 seconds	Time limit for each image fetch.
`MaxRedirects`	5	Maximum redirects per fetch. A negative value rejects all redirects.

Bootstrap options

Option	Purpose
`piko.WithLLMProvider(name, provider)`	Registers a completion provider.
`piko.WithDefaultLLMProvider(name)`	Marks the default completion provider.
`piko.WithEmbeddingProvider(name, provider)`	Registers an embedding provider.
`piko.WithDefaultEmbeddingProvider(name)`	Marks the default embedding provider.
`piko.WithLLMService(service)`	Registers a fully configured service.