Otter cache provider

In-process cache provider implementing cache.Provider with adaptive W-TinyLFU eviction, generic typed namespaces, and no external services.

Overview

Otter is the default Piko cache. It is a hashmap-backed, in-process cache that wraps maypok86/otter v2 and uses its adaptive W-TinyLFU eviction policy. Reads and writes stay inside the process, with no network round trip and no serialisation while the value type stays in the Go heap. It is the lowest-latency cache Piko ships for single-instance, in-process access.

Otter is the zero-config default. When you register no cache provider, the container registers Otter and sets it as the default, so most applications write no cache bootstrap glue at all. The provider is pure Go. It needs no CGO, no build tags, and no system libraries, so it behaves the same in interpreted dev mode (dev-i) and in compiled builds.

The character is fast and local. Cache state lives in the same Go heap as the application, so it does not survive a restart and does not synchronise across replicas. Two pods running Otter caches see two independent caches. A write on one is invisible to the other. Reach for Otter when scale is small (single-instance dev, small services, per-pod hot-data tiers) or when you wrap it as the L1 of a multilevel setup. Reach for Redis or Valkey when you need a shared cache across replicas.

The provider takes no configuration of its own. NewOtterProvider() is parameterless. Per-namespace tuning (maximum size, TTL, weight-based eviction, encoders, compression, encryption) happens through cache.NewCacheBuilder when you create the namespace. The same builder, namespaces, and transformers apply to every cache provider, so swapping Otter for Redis later is a one-line provider change.

Configuration

import (
    "piko.sh/piko/wdk/cache/cache_provider_otter"
)

provider := cache_provider_otter.NewOtterProvider()

The constructor returns a cache.Provider directly. There is no Config struct and no error to handle. Tune each namespace by building cache instances against the cache service:

builder, err := cache.NewCacheBuilder[string, User](service)
if err != nil {
    return err
}

userCache, err := builder.
    Provider("otter").
    Namespace("users").
    MaximumSize(10000).
    Build(ctx)

NewCacheBuilder returns the builder and an error, so capture both before chaining. MaximumWeight and Weigher enable weight-based eviction, and Compression and Encryption add per-namespace transformers.

Bootstrap

Otter needs no bootstrap glue. When you configure no cache provider, the container registers Otter and sets it as the default automatically:

ssr := piko.New()

ssr := piko.New(
    piko.WithCacheProvider("otter", provider),
    piko.WithDefaultCacheProvider("otter"),
)

Persistence

Otter can write a write-ahead log alongside the in-memory map, so the provider reloads cache state on restart. Enable it per namespace through PersistenceConfig, which the builder passes to the provider via Options:

userCache, err := builder.
    Provider("otter").
    Namespace("users").
    Options(cache_provider_otter.PersistenceConfig[string, User]{
        Enabled:    true,
        KeyCodec:   keyCodec,
        ValueCodec: valueCodec,
        WALConfig:  cache_provider_otter.DefaultPersistConfigNamed("users"),
    }).
    Build(ctx)

Persistence needs both KeyCodec and ValueCodec. The build fails at startup if either is missing.

Tradeoffs

Otter is in-process, so cache state does not survive restarts and does not synchronise across replicas. Scaling out across more pods lowers the per-replica hit rate unless routing keeps each request on the same pod, because each pod warms its own cache. Reach for Redis, Valkey, or a multilevel Otter plus Redis combination when horizontal scale matters.