Context and caching

Each request contains more than the latest message. nanocode arranges the model’s context so the stable parts come first, the active conversation follows, and frequently changing task state comes last. This keeps the agent informed while giving supported providers a long, reusable prompt prefix.

What the model receives

─ reused prefix ───────────────────────────────────── system instructions how the agent should operate project environment directory · OS · shell · commands skills index only when skills are installed MCP servers only when a server is connected conversation history messages · tool calls · results─ changes each turn ───────────────────────────────── current turn the latest messages and results task memory goal · plan · facts · checks

Tool definitions are sent beside this message stack: built-in tools, Skill when skills are installed, and MCP tools and resources from currently connected servers. Configured but disconnected servers do not consume context.

Keeping context manageable

Large tool results are shortened before they enter the conversation; the agent can retrieve the complete result later with Recall. Repeated skill instructions and MCP descriptions are replaced with references to their first full copy instead of being sent in full again.

Compaction

When the estimated context reaches runtime.max_context_tokens, nanocode compacts: the older part of the conversation is replaced by a short summary, and the most recent messages are kept as they are. The session continues in the same turn, so a long task does not have to stop.

Compaction is lossy — detail in the summarized messages is gone. Task memory (goal, plan, facts, checks) is carried across untouched, which is why decisions worth keeping belong there.

Run /compact to compact immediately rather than waiting for the threshold, for example before starting a large refactor. /status reports how many compactions a session has done.

Prompt caching

Prompt caching lets a provider reuse work for an unchanged beginning of a request. The next request usually begins with the same instructions, environment, tools, and earlier conversation, so only the new tail needs to be processed. Connecting an MCP server, changing installed skills, switching models, or otherwise changing an early section can reduce the next request’s cache hit.

previous ████████████████████████░░░░░░next ████████████████████████░░░░░░░░░░ █ reused prefix ░ processed again

A request is reused only up to its first difference, so a change near the beginning — connecting an MCP server, installing a skill, switching models — shortens the reusable prefix. That is why the stable sections are placed first.

OpenAI-compatible providers may reuse matching prefixes automatically; nanocode supplies a stable cache key where the provider supports one. For Anthropic, nanocode explicitly marks the tools and system instructions as an ephemeral cacheable prefix. Provider support and accounting differ.

Checking the hit rate

/status reports the cached prompt tokens the provider counted — once for the whole session, and again for the most recent request:

usage calls 14; total 182304; cached 148992/162880 (91.5%); last 21120/22016 (95.9%) └─ whole session ─┘ └─ latest request ─┘

The ratio varies with the provider, model, prompt length, and conversation. When request prefixes line up, it can reach 90–99%. This is an observation, not a guaranteed rate.