Module llm_cache 

Deterministic content-hash-keyed cache for local LLM inference (issue #528).

Local-model inference (Ollama, llama.cpp) is CPU/GPU intensive and non-deterministic (sampling makes outputs vary). For a 10K-page docs site that means hours of CI per build and bit-different artifacts across runs. This module fixes both: invocations are keyed on a SHA-256 of (endpoint, model, prompt, timeout); a hit returns the cached body and never crosses the wire, so builds are both fast and reproducible.

On-disk layout

Entries live under $XDG_CACHE_HOME/ssg/llm/ (Linux), ~/Library/Caches/ssg/llm/ (macOS), or %LOCALAPPDATA%\ssg\llm\ (Windows). Each entry is git-sharded:

<cache_dir>/<aa>/<bbbbbbbbbb...>.json

where aa is the first two hex chars of the key and bbbbb... is the remaining 62 chars. This keeps any single shard directory well under the FAT32/exFAT 65 K-entry ceiling even on a million- page site.

File format

Each entry is a small JSON document:

{
  "version": 1,
  "key_hex": "<full 64-char hex of key>",
  "payload_len": <usize>,
  "payload": "<the cached LLM response>"
}

key_hex and payload_len provide a cheap end-to-end integrity check — a torn mid-write (or a flipped bit on disk) yields a parse error or a length mismatch, and the entry is evicted and re-computed without surfacing a hard error to the caller.

Concurrency

Writes go to <final>.tmp.<pid>.<nanos> then rename into place. rename is atomic on every supported filesystem, so concurrent writers for distinct keys never trample one another, and concurrent writers for the same key produce a last-writer-wins outcome where every reader still sees a consistent entry.

TTL

LlmCache::get honours a configurable TTL: entries older than ttl (compared against the file’s mtime) are evicted and reported as a miss. Default is 90 days.

Structs

CacheStats

Counters returned by LlmCache::stats for the ssg cache --stats CLI subcommand. Counts are session-local — the cache file itself stores none of this.

LlmCache

Content-hash-keyed file cache for LLM inference.

Constants

DEFAULT_TTL

Default TTL for cache entries: 90 days. Matches the AC4 default from issue #528 (“cache.llm.ttl_days default 90”).