Engineering decisions, explained.
Field notes on AI systems, production architecture, reliability, data, and the judgment behind technical tradeoffs.
Deletion across data systems is a workflow, not a query
Coordinate erasure through an idempotent, checkpointed workflow when one subject's data spans operational, analytical, and AI stores.
How to measure the real cost of an agent evaluation
A practical accounting model for judge calls, agent calls, retries, and the unattributed work that per-case dashboards often hide.
Version the policy and evaluator separately
Record decision rules independently from measurement logic so AI behavior can be replayed, compared, and audited honestly.
Make your evaluation harness run the agent you ship
Use the production context path with synthetic data, relative dates, and structural fixtures so an eval measures real behavior without copying private state.
What should block a deploy for an LLM application?
Build a release gate around consequence, deterministic contracts, calibrated model judgments, missing evidence, and controlled baselines.
Optional intelligence needs hard failure boundaries
Keep semantic enrichment from taking down core processing by modeling dependencies, fallbacks, and degraded results explicitly.
How to find tests that assert nothing
Turn conditional validation into positive output contracts, reject invalid test declarations, and report cases that still lack expectations.
Freshness should choose the compute path
Route requests across batch, incremental, and live evidence according to measured coverage rather than a fixed data source.
Groundedness is not one property
How to scope a hallucination grader across user-specific claims, general knowledge, and interpretation without making the judge permissive.
Keeping agents fast with fewer hops and smaller context
A measurement-driven approach to removing unnecessary model calls, persistent state, prompt payloads, and post-tool narration.
Live and final are different data products
Model in-progress and completed reporting periods separately so partial evidence cannot overwrite settled history.
Move the model out of the request path
Precompute versioned AI artifacts so user requests read predictable results instead of waiting on variable inference.
Observability for multi-step agent systems
Instrument traces, model usage, tool calls, and evaluation results with one correlation model while keeping sensitive prompt data out of telemetry.
The model should generate content, not identity
Keep names, identifiers, ownership, and authorization outside model output so generated content cannot cross entity boundaries.
Memory is not context: a safe storage boundary for agents
Separate live state from durable user-stated facts, then add provenance, expiry, deletion, and intent-based retrieval.
Low confidence can be useful without being visible
Preserve uncertain observations for continuity while using a separate, deterministic policy to decide what reaches a user.
Facts, affordances, and pertinence in agent design
A concrete boundary for deciding which agent behavior belongs in deterministic code and which decisions should remain with the model.
Use models for ambiguity and code for consequence
Place probabilistic reasoning behind a typed boundary, then let deterministic code decide what the system stores, ranks, and shows.
Structured data is still untrusted input
Sanitize metadata and JSON-LD before storage, then render a bounded fact document instead of placing arbitrary page markup into an LLM prompt.
Build a context ladder for your agent
Classify agent data by lifetime, freshness, sensitivity, and retrieval cost, then compile only the context required for each turn.
A timestamp is not a time context
Store events in UTC, define one business calendar, and let deterministic code resolve phrases such as today before an agent queries data.
Agents do not see reality. They see data products.
Build an evidence pipeline that turns raw events into typed, traceable facts before an agent can retrieve or reason over them.