Multilevel cache provider

Two-tier cache adapter that composes a fast L1 (local/in-process) provider with a slower L2 (remote/distributed) provider. It falls through on misses, back-populates L1 from L2 hits, and protects callers from a failing L2 with a circuit breaker.

Overview

Unlike the other cache providers, you compose multilevel per cache instance instead of registering it as a top-level cache.Provider. The adapter wraps two existing cache.ProviderPort instances into a single ProviderPort, so callers reach it through the normal cache interface. Wire it with the MultiLevel(l1, l2) builder shorthand, or call NewMultiLevelAdapter directly when you need full control.

The adapter implements the whole cache.ProviderPort surface by delegating: Get, BulkGet, Compute, Search, Query, tag invalidation, TTL writes, and the key, value, and entry iterators. Reads and computes go to L1, writes go through to both tiers, so you swap multilevel in without changing call sites.

Each lookup checks L1 first. A hit skips the L2 round trip entirely. On an L1 miss, the lookup falls through to L2. On an L2 hit, the adapter back-populates the value into L1. A single-key read back-populates L1 before it returns, so the next read is fast. A bulk read back-populates L1 on a bounded background worker, so the next read of a bulk-loaded key can still miss L1 until that worker runs. Combined, this gives you the latency of Otter for the hot working set with the cross-replica visibility of Redis or Valkey underneath.

Reach for multilevel when one tier alone is not enough. The per-process Otter cache misses cross-replica updates, and the network round trip to Redis is too slow for a high-frequency hot path. Skip it if your hit rate is already good on one tier. The wrapper adds a circuit breaker, an L2 fallback path, and an L1 back-population step that are not free.

An embedded circuit breaker handles L2 failures. After MaxConsecutiveFailures consecutive failures, the breaker opens and the adapter skips L2 for OpenStateTimeout. A Redis outage then degrades the cache to L1-only instead of blocking every request on a timeout. Writes follow the same path. The adapter writes L2 first through the breaker, then writes L1. When the breaker is open the adapter skips the L2 write and the write lands in L1 only.

Background back-population and async write-back run on goroutines bounded to runtime.NumCPU() workers. The caller acquires a worker slot before launching, so a slow L2 cannot spawn unbounded goroutines. The adapter emits OpenTelemetry counters for L1 hits, L2 hits, total misses, L2 errors, and back-populations under piko.cache.provider.multilevel.*, so you can watch the hit-rate split between tiers.

Configuration

import (
    "context"
    "time"

    "piko.sh/piko/wdk/cache"
    "piko.sh/piko/wdk/cache/cache_provider_multilevel"
)

// l1 and l2 are cache.ProviderPort[K, V] instances built from
// concrete providers (for example Otter for L1, Redis for L2).
multilevel := cache_provider_multilevel.NewMultiLevelAdapter[string, []byte](
    ctx,
    "user-cache", // metric label for this multilevel instance
    l1,           // L1 (local/fast) provider port
    l2,           // L2 (remote/distributed) provider port
    cache_provider_multilevel.Config{
        MaxConsecutiveFailures: 5,                // open L2 breaker after N failures
        OpenStateTimeout:       30 * time.Second, // hold L2 breaker open this long
        // L1ProviderName and L2ProviderName are populated by the cache builder
        // when you use the MultiLevel(...) shorthand.
    },
)

In most applications you do not call NewMultiLevelAdapter directly. You build a multilevel cache through the cache builder, which writes the glue for you. MultiLevel(l1, l2) builds both tiers from registered provider names, applies your size, expiry, and transformer settings to L1, and wires the circuit breaker. The breaker defaults to opening after 5 failures and holding open for 30 seconds.

builder, err := cache.NewCacheBuilder[string, []byte](service)
if err != nil {
    return err
}

userCache, err := builder.
    MultiLevel("otter", "redis").                  // L1 and L2 provider names
    Namespace("users").
    L2CircuitBreaker(5, 30*time.Second).           // breaker tuning
    Build(ctx)

NewCacheBuilder returns (*Builder[K, V], error), so capture the builder first and chain methods off it. Do not chain directly off the constructor result.

The builder routes multilevel construction through a reflection factory that handles a fixed set of key and value type combinations: string/any, string/string, string/[]byte, and int/any. For any other combination, including a struct value type, call NewMultiLevelAdapter directly with the concrete types.

Bootstrap

The multilevel adapter is not a top-level cache.Provider. Register your L1 and L2 providers separately, then compose them at the cache-builder level.

ssr := piko.New(
    piko.WithCacheProvider("otter", otterProvider),
    piko.WithCacheProvider("redis", redisProvider),
    piko.WithDefaultCacheProvider("otter"), // pick whichever default makes sense
)

The builder looks up the multilevel constructor in a registry populated by the adapter package's init. The cache_provider_multilevel facade imports that package, so referencing NewMultiLevelAdapter registers the constructor for you. If you wire multilevel only through the builder and never touch the facade, add a blank import so the constructor is present. Without it, Build returns an error that names the missing import.

import _ "piko.sh/piko/internal/cache/cache_adapters/provider_multilevel"

See also

Other cache providers:

  • Otter, typical L1 (in-process, fast).
  • Redis, typical L2 (distributed, shared).
  • Valkey, Redis-compatible L2 alternative.
  • Redis Cluster, sharded L2 for large working sets.
  • Valkey Cluster, sharded permissively licensed L2.

Framework docs:

External: