Hermes Agent Prompt Caching: system_and_3, cache_ttl and Cache Misses

Julian Goldie — founder, AI Profit Boardroom
By Julian Goldie · 6 min read
Get The AI Profit Stack Join AIPB →
🎯 1,000+ done-for-you AI agent workflows 📅 4 live coaching calls every week 🛡️ 7-day refund + 30-day ROI guarantee 👥 3,400+ AI operators inside

Hermes Agent prompt caching decides how much of each request your automations pay to process again. If you build pipelines on Hermes (crons, webhooks, subagent fan-outs, kanban boards), the cache rules shape both latency and token spend, and v0.23.3 rewired them around a single lifecycle controller. Below: the mechanism, the config, the failure modes and a debugging routine.

⚙️ Building automations on Hermes? Inside the AI Profit Boardroom you get the Agent OS zip included, our pipeline templates, prompt libraries and live build sessions every week. → Get access

📺 Watch: Hermes Agent OS Can Automate EVERYTHING?! 🤯

How Hermes Agent Prompt Caching Works Under the Hood

The logic lives in agent/prompt_caching.py. It keeps the prefix of a conversation in the provider's cache so repeat turns skip re-processing it, using Anthropic's cache_control breakpoints. By the docs' own estimate, multi-turn chats see input token cost fall by roughly three quarters.

Anthropic caps a single request at four breakpoints. Hermes allocates them with a strategy named system_and_3:

Each marker is either {"type":"ephemeral"} or {"type":"ephemeral","ttl":"1h"}. The function apply_anthropic_cache_control() deep-copies the message list before injecting them, so your stored history is never mutated. Plain string content becomes a text block that carries the marker, while list content gets the marker on its final element. Tool messages only receive markers on the native Anthropic API.

Which Routes Get Markers

Caching activates automatically once both conditions hold: the model belongs to Anthropic's Claude family, and the provider honours cache_control. The docs name two: Anthropic's native API and OpenRouter. On Nous Portal, prompt_cache_markers_from_catalog reads the model catalog to choose which models are marked.

Routes outside that set rely on whatever caching the provider does on its side, and Hermes tracks those with its per-session clock rather than explicit markers. If cost per run matters in a pipeline, send the long-lived agents through OpenRouter or the Anthropic API with a Claude model.

Choosing cache_ttl for Automations

# config.yaml
prompt_caching:
  cache_ttl: auto    # auto | 5m | 1h

With auto, Hermes looks at where a turn came from. Sessions driven by a human get the 1-hour tier. Sources the docs class as machine-paced get the 5-minute tier: subagent, cron, oneshot, webhook, kanban, api, api_server, tool, batch and curator. Side agents launched via /background, /btw and /refine also run on 5 minutes.

According to the canary notes, auto keeps the tier of the source and clamps it on every send. Pinning 5m or 1h overrides that choice for every source on the install at once, which is why most builders are better off leaving auto in place and shaping the workload instead. That applies whether the work comes from scheduled cron runs, subagent fan-outs or a kanban board.

📺 Watch: Hermes Agent Just Got 10X Faster!

Shipping client automations on Hermes? The AI Profit Boardroom shares the Agent OS, ready-made workflow templates and recorded build walkthroughs you can copy.

Cache Invalidation: The Failure Modes

Hits depend on prefix matching. Anything that changes the bytes near the front of the request throws away every cached block after it. Watch for these in your automations:

  1. Model identity changes. Model identity is baked into the cache key. A /model switch, a primary-model fallback, or credential-pool rotation onto another account each sends the next request in with zero hits, so the entire conversation is billed undiscounted.
  2. Mid-history edits. Inserting or deleting messages in the middle invalidates every cached block after the edit. Pipelines that rewrite earlier turns pay for it on the next call.
  3. System prompt mutation. Treat the system prompt as frozen for the life of a session. Hermes itself only appends a note to it on the first compaction.
  4. Idle expiry. When the TTL lapses with no traffic, the provider evicts the entry.
  5. Compression. Compacting a region drops that region's cache. The system prompt entry survives, and the three-message window re-caches within one or two turns.

If you wire in a fallback model for reliability, budget for every failover being a cold read of the full context.

Warming and the v0.23.3 Lifecycle Controller

Release v0.23.3, published 20 November 2026, merged cache handling into one prompt-cache lifecycle controller. Its notes list warming that survives long tool runs, idle-chat compaction ahead of the 1-hour expiry, Nous Portal moving onto the whole 1-hour lifecycle, and prompt_caching.cache_ttl defaulting to auto. The release before it, v0.23.2, had already put each session on one prompt-cache clock.

The warming rules builders should know:

The 19 November canary shows the internals. prompt_cache_lifecycle became a package, cache-expiry compaction now summarises from the warm prefix, and that summary writes no fresh cache entry and runs only when it comes out cheaper. A malformed prompt_caching block also can no longer break startup, which helps if you template config files across many machines.

Designing Pipelines That Stay Cached

Most cache problems in automations are design problems, not config problems. A few patterns follow straight from the prefix-matching rule:

A Debugging Routine for Cache Misses

  1. Run /context in the affected session and look for warming_off.
  2. Check for a model change, failover or credential rotation just before the first miss.
  3. Compare the request prefix across two turns. If the system prompt or an early message changed, you have your cause.
  4. Compare the gap between model calls with the TTL tier the source runs on. A job on the 5-minute tier that idles longer between calls will expire.
  5. Confirm the route is a Claude model on Anthropic or OpenRouter, or a catalog-marked Nous Portal model.
  6. Reset with /new and confirm the install is on v0.23.3 or newer.

For keeping context lean across long runs, read our Hermes memory management guide next.

Want to watch pipelines like these get built? Get inside the AI Profit Boardroom for the Agent OS, Hermes build tutorials and weekly coaching calls.

FAQ

Why are my Hermes cron runs getting cache misses?

Under auto, cron sits on the 5-minute tier. If a run pauses longer than that between model calls, or a failover swaps the model, the next call starts cold.

How should I tune cache_ttl?

Start on auto, which matches the tier to the source. Override it with 5m or 1h only once you know the pacing of every source on that install, since the pinned value covers all of them.

How does context compression interact with prompt caching?

Compression invalidates the cache for the compacted span but leaves the system prompt entry alone. The rolling window recovers within one or two turns, and v0.23.3 can compact idle chats on the warm prefix before the 1-hour entry expires.

Does prompt caching work on every model?

No. Explicit markers go to Claude models on cache_control-capable providers, plus the models Nous Portal's catalog marks. Other routes depend on the provider's own caching.

What does warming_off mean in /context?

It means two keep-alives genuinely missed, so Hermes stopped warming that session. Running /new starts a session with warming back on.

Build around those rules (one model per session, a frozen system prompt, append-only history) and Hermes Agent prompt caching carries most of the cost work for your automations.

What members say about the AI Profit Boardroom

Read all 114 reviews on Skool →

★ 4.9 out of 5 from 114 member reviews on Skool

Yosuke Sato, AI Profit Boardroom member review (5 stars, paying member for 17 months): The best Skool course you get for AI! I have attended a few Skool courses before, but AI Profit Boardroom is the most active community and has great support from the management team, including Julian! You can get the most up-to-date information and practical implementations in this community. I highly recommend this!Mazzie T, AI Profit Boardroom member review (5 stars, paying member for 10 months): My mind is blown everyday with all the content and the incredible information. So much support and I love the personalised attention and quick responses. The community is great and people are very responsive and helpful.Rene Clayton, AI Profit Boardroom member review (5 stars, paying member for 9 months): The AI Profit Boardroom is a GAME CHANGER people. I could not be more thrilled that I joined!! So much support, more information than you can even consume, a huge value for what the small fee was. I have grown my business, my knowledge and my FUN!. SUCH a fantastic leader and group of humans. I am incredibly honored to be a part of this fantastic group of intelligent humans :-) THANK YOU!Arpiet Malpani, AI Profit Boardroom member review (5 stars, paying member for 8 months): Incredible learning from Julian and team. I started out in march and was able to build end to end SAAS with the help and learning from this group. I am incredibly thankful to Julian for crazy support and prompt response.James Grimm, AI Profit Boardroom member review (5 stars, paying member for 7 months): I asked Julian for help finding local business websites with little to no traffic. Julian was gracious in providing me my personal "how-to" video instruction. Wow, this is the pinnacle of customer service! I am just one of 3,400 subscribers and Julian took the time to help me personally. Thank you Julian for your assistance and outstanding custoemr support and care. James (Jim) GrimmNick Good, AI Profit Boardroom member review (5 stars, paying member for 4 months): I’m part of many AI groups and hands down, what Julian Goldie, has built here…the classroom trainings are amazing, but the community he has created is second to none (the community is worth the cost of joining!). If you’re looking to become the AI expert and build/create something meaningful, then AI Profit Boardroom is for YOU!Steven Gray, AI Profit Boardroom member review (5 stars, paying member for 4 months): Been in awhile. I’m a dad on a factory shift, not a tech bloke. Julian’s lot showed me how to use AI without drowning in tools. Avatar, voice, the workflow thinking — that’s what I needed. If you’re stuck on “where do I start,” this group’s worth it. I always read them.Steve Erickson, AI Profit Boardroom member review (5 stars, paying member for 4 months): I asked Julian a simple question, and he ended up creating a 10-minute video and a guide breaking down in extreme detail what I had asked. Amazing response!Rod Martin, AI Profit Boardroom member review (5 stars, paying member for 3 months): Julian is brilliant, and more prolific than I can keep up with (which is a great thing). His AgentOS is remarkable, and he is THE go-to guy for what you can really do (and not do) with the ever-increasing deluge of new AI upgrades and tools. HIGHLY recommend.Daniel Shulman, AI Profit Boardroom member review (5 stars, paying member for 4 months): Julian is very helpful. He doesn't try sell stuff you dont need and will give you a straight answer to your questions.Rj Herzog, AI Profit Boardroom member review (5 stars, paying member for 5 months): Top notch. If you are searching & searching thinking there is always something better out there.... You just found it. No need to look any further. Goldie covers it all & is super helpful. Can't beat the group either. Really can't recommend this group enough.Jermey Matthews, AI Profit Boardroom member review (5 stars, paying member for 6 months): While he has the capacity to do it, Julian is definitely doing the unscalable things to support his community.

Ready To Join The #1 AI Community?

Join 3,400+ entrepreneurs inside the AI Profit Boardroom. Get the Agent OS, 1,000+ plug-and-play AI agent workflows, daily Q&A with Julian and 4 live coaching calls every week.

Join The AI Profit Boardroom →

7-Day No-Questions Refund • 30-Day ROI Guarantee

← Back to all posts