Build a context ladder for your agent

An agent can answer incorrectly because required information was absent or because relevant information was buried inside unrelated context. Both are context-design failures. A context ladder makes placement explicit: stable policy stays near the prompt prefix, fresh state is loaded conditionally, and large or sensitive data is fetched through tools.

The ladder is not a fixed list of prompt sections. It is a classification system based on lifetime, freshness, sensitivity, and how often the data is needed.

Describe context as policy

This section defines metadata for each context source. The compiler can enforce these rules before text reaches the model.

The policy becomes easier to review when represented explicitly:

from dataclasses import dataclass
from enum import IntEnum, StrEnum


class Lifetime(IntEnum):
    DEPLOYMENT = 1
    USER = 2
    SESSION = 3
    TURN = 4
    ON_DEMAND = 5


class Sensitivity(StrEnum):
    PUBLIC = "public"
    INTERNAL = "internal"
    PERSONAL = "personal"
    RESTRICTED = "restricted"


@dataclass(frozen=True)
class ContextPolicy:
    name: str
    lifetime: Lifetime
    max_age_seconds: int | None
    sensitivity: Sensitivity
    max_tokens: int
    always_include: bool = False

Identity and safety policy normally use deployment lifetime and are always present. Stable profile data uses user lifetime with a refresh contract. Conversation state uses session lifetime. Routing signals use turn lifetime. Detailed records, search results, and long-term memory use on-demand lifetime.

Sensitivity can move data upward even when it is cheap. A small restricted field should not be copied into every prompt merely because it fits.

Compile the minimum permitted context

This section implements inclusion rules and a token budget. The compiler receives a turn intent and selects eligible sources.

from dataclasses import dataclass
from typing import Protocol


@dataclass(frozen=True)
class TurnIntent:
    name: str
    allowed_sources: frozenset[str]


class ContextSource(Protocol):
    policy: ContextPolicy

    async def render(self) -> str: ...


async def compile_context(
    sources: list[ContextSource],
    intent: TurnIntent,
    count_tokens,
    budget: int,
) -> str:
    blocks: list[str] = []
    used = 0

    for source in sources:
        allowed = (
            source.policy.always_include
            or source.policy.name in intent.allowed_sources
        )
        if not allowed:
            continue

        block = await source.render()
        tokens = count_tokens(block)
        if tokens > source.policy.max_tokens:
            raise ValueError(f"{source.policy.name} exceeded its source budget")
        if used + tokens > budget:
            raise ValueError("turn context exceeded its total budget")

        blocks.append(block)
        used += tokens

    return "\n\n".join(blocks)

Production compilers often need priority and graceful truncation. Apply those rules per source rather than slicing the final prompt at an arbitrary token boundary, which can remove closing delimiters or the field that establishes provenance.

Treat source rendering as untrusted-data handling. Escape delimiters, label provenance, and never let retrieved text append instructions outside its block.

Write a freshness contract

This section prevents the model from asserting expired state. Every source with mutable values needs a maximum age, an authoritative observation time, and an invalidation strategy.

A short time-to-live bounds staleness when events are delayed. Event-driven invalidation reduces the usual delay after an update. Using both creates a recovery path for missed events without polling every source on every turn.

Do not invent a custom freshness value for every field unless the product requires it. A small set of named policies is easier to test and monitor.

Move expensive data behind tools

This section handles sources that are large, changing, or sensitive. The model should receive a tool description and request data only when the turn intent requires it.

The tool boundary must still be deterministic. Code authorizes the request, applies tenant scope, limits result size, and records provenance. The model decides whether the user’s question requires the tool; it does not decide what records it may access.

Return a compact summary plus a continuation token when results exceed the per-source budget. Avoid storing the full result in session state because that turns an on-demand source into stale always-present context on the next turn.

Diagnose failures by source

This section turns vague answer failures into locatable defects. Record a context manifest beside each trace without storing the context content itself. Each entry needs source, version, observation time, token count, inclusion state, and bounded reason.

When a fact was missing, the manifest reveals whether its source was excluded, stale, over budget, or never fetched. When the wrong tool was selected, the trace points to routing rather than context assembly. This separation prevents prompt edits from becoming the default response to every agent bug.

A context ladder gives each data source a lifetime, freshness bound, sensitivity class, and token budget. The compiler enforces inclusion, tools provide expensive data on demand, and manifests explain what the model was allowed to see without copying private content into telemetry.