Hermes Agent prompt caching decides how much of each request your automations pay to process again. If you build pipelines on Hermes (crons, webhooks, subagent fan-outs, kanban boards), the cache rules shape both latency and token spend, and v0.23.3 rewired them around a single lifecycle controller. Below: the mechanism, the config, the failure modes and a debugging routine.
⚙️ Building automations on Hermes? Inside the AI Profit Boardroom you get the Agent OS zip included, our pipeline templates, prompt libraries and live build sessions every week. → Get access
📺 Watch: Hermes Agent OS Can Automate EVERYTHING?! 🤯
How Hermes Agent Prompt Caching Works Under the Hood
The logic lives in agent/prompt_caching.py. It keeps the prefix of a conversation in the provider's cache so repeat turns skip re-processing it, using Anthropic's cache_control breakpoints. By the docs' own estimate, multi-turn chats see input token cost fall by roughly three quarters.
Anthropic caps a single request at four breakpoints. Hermes allocates them with a strategy named system_and_3:
- Breakpoint 1: the system prompt, the most stable block in the request.
- Breakpoints 2 to 4: a rolling window over the last three non-system messages.
Each marker is either {"type":"ephemeral"} or {"type":"ephemeral","ttl":"1h"}. The function apply_anthropic_cache_control() deep-copies the message list before injecting them, so your stored history is never mutated. Plain string content becomes a text block that carries the marker, while list content gets the marker on its final element. Tool messages only receive markers on the native Anthropic API.
Which Routes Get Markers
Caching activates automatically once both conditions hold: the model belongs to Anthropic's Claude family, and the provider honours cache_control. The docs name two: Anthropic's native API and OpenRouter. On Nous Portal, prompt_cache_markers_from_catalog reads the model catalog to choose which models are marked.
Routes outside that set rely on whatever caching the provider does on its side, and Hermes tracks those with its per-session clock rather than explicit markers. If cost per run matters in a pipeline, send the long-lived agents through OpenRouter or the Anthropic API with a Claude model.
Choosing cache_ttl for Automations
# config.yaml
prompt_caching:
cache_ttl: auto # auto | 5m | 1h
With auto, Hermes looks at where a turn came from. Sessions driven by a human get the 1-hour tier. Sources the docs class as machine-paced get the 5-minute tier: subagent, cron, oneshot, webhook, kanban, api, api_server, tool, batch and curator. Side agents launched via /background, /btw and /refine also run on 5 minutes.
According to the canary notes, auto keeps the tier of the source and clamps it on every send. Pinning 5m or 1h overrides that choice for every source on the install at once, which is why most builders are better off leaving auto in place and shaping the workload instead. That applies whether the work comes from scheduled cron runs, subagent fan-outs or a kanban board.
📺 Watch: Hermes Agent Just Got 10X Faster!
Shipping client automations on Hermes? The AI Profit Boardroom shares the Agent OS, ready-made workflow templates and recorded build walkthroughs you can copy.
Cache Invalidation: The Failure Modes
Hits depend on prefix matching. Anything that changes the bytes near the front of the request throws away every cached block after it. Watch for these in your automations:
- Model identity changes. Model identity is baked into the cache key. A
/modelswitch, a primary-model fallback, or credential-pool rotation onto another account each sends the next request in with zero hits, so the entire conversation is billed undiscounted. - Mid-history edits. Inserting or deleting messages in the middle invalidates every cached block after the edit. Pipelines that rewrite earlier turns pay for it on the next call.
- System prompt mutation. Treat the system prompt as frozen for the life of a session. Hermes itself only appends a note to it on the first compaction.
- Idle expiry. When the TTL lapses with no traffic, the provider evicts the entry.
- Compression. Compacting a region drops that region's cache. The system prompt entry survives, and the three-message window re-caches within one or two turns.
If you wire in a fallback model for reliability, budget for every failover being a cold read of the full context.
Warming and the v0.23.3 Lifecycle Controller
Release v0.23.3, published 20 November 2026, merged cache handling into one prompt-cache lifecycle controller. Its notes list warming that survives long tool runs, idle-chat compaction ahead of the 1-hour expiry, Nous Portal moving onto the whole 1-hour lifecycle, and prompt_caching.cache_ttl defaulting to auto. The release before it, v0.23.2, had already put each session on one prompt-cache clock.
The warming rules builders should know:
- The warm-or-skip decision fires at 5/6 of the TTL.
- Idle keep-alives are capped at three per user message.
- Two genuine misses disable warming for the session, and
/contextreports it aswarming_off. /newclears that state.
The 19 November canary shows the internals. prompt_cache_lifecycle became a package, cache-expiry compaction now summarises from the warm prefix, and that summary writes no fresh cache entry and runs only when it comes out cheaper. A malformed prompt_caching block also can no longer break startup, which helps if you template config files across many machines.
Designing Pipelines That Stay Cached
Most cache problems in automations are design problems, not config problems. A few patterns follow straight from the prefix-matching rule:
- One model per stage. If a step needs a different model, give it its own session instead of switching mid-run, so the main session keeps its warm prefix.
- Static system prompt, dynamic tail. Put changing data such as dates, ticket IDs or customer names in the newest message, never in the system prompt.
- Append, don't rewrite. Add new tool results at the end of the history rather than editing earlier turns.
- Fresh session per unrelated job. When a run has no use for earlier context, start clean with
/newrather than dragging a cold history along. - Tight call spacing on 5-minute sources. Cron, webhook and API runs sit on the short tier, so long waits between their model calls let the entry lapse.
A Debugging Routine for Cache Misses
- Run
/contextin the affected session and look forwarming_off. - Check for a model change, failover or credential rotation just before the first miss.
- Compare the request prefix across two turns. If the system prompt or an early message changed, you have your cause.
- Compare the gap between model calls with the TTL tier the source runs on. A job on the 5-minute tier that idles longer between calls will expire.
- Confirm the route is a Claude model on Anthropic or OpenRouter, or a catalog-marked Nous Portal model.
- Reset with
/newand confirm the install is on v0.23.3 or newer.
For keeping context lean across long runs, read our Hermes memory management guide next.
Want to watch pipelines like these get built? Get inside the AI Profit Boardroom for the Agent OS, Hermes build tutorials and weekly coaching calls.
FAQ
Why are my Hermes cron runs getting cache misses?
Under auto, cron sits on the 5-minute tier. If a run pauses longer than that between model calls, or a failover swaps the model, the next call starts cold.
How should I tune cache_ttl?
Start on auto, which matches the tier to the source. Override it with 5m or 1h only once you know the pacing of every source on that install, since the pinned value covers all of them.
How does context compression interact with prompt caching?
Compression invalidates the cache for the compacted span but leaves the system prompt entry alone. The rolling window recovers within one or two turns, and v0.23.3 can compact idle chats on the warm prefix before the 1-hour entry expires.
Does prompt caching work on every model?
No. Explicit markers go to Claude models on cache_control-capable providers, plus the models Nous Portal's catalog marks. Other routes depend on the provider's own caching.
What does warming_off mean in /context?
It means two keep-alives genuinely missed, so Hermes stopped warming that session. Running /new starts a session with warming back on.
Build around those rules (one model per session, a frozen system prompt, append-only history) and Hermes Agent prompt caching carries most of the cost work for your automations.











