Research notes
A public research library for AI assurance infrastructure. Each note states the question, hypothesis, expected evidence, and current maturity so planned work is never presented as finished.
The zansn npm package is the first executable artifact from the Zansn assurance roadmap: local evidence-pack generation for agent workflows, and the foundation for research on trace semantics, privacy boundaries, cost/latency profiling, and deployment-readiness evidence.
17 of 17 notes
A measurement model for AI deployment readiness
How should teams measure whether an AI system is ready for controlled production deployment?
Trace semantics for tool-using agents
What trace semantics are needed to explain failures in tool-using AI agents?
Reliability beyond pass rate
Which metrics better capture agent reliability than a single task success rate?
Stateful agent evaluation with transactional workflows
How should agents be evaluated when success depends on changing external state correctly?
Privacy boundary testing for hybrid local/cloud AI
How can teams verify that sensitive data stays inside intended local, controlled, or cloud execution boundaries?
Prompt injection and tool misuse in agentic workflows
How should agentic systems be tested against prompt injection, tool misuse, and excessive agency?
RAG evaluation under domain shift
How should retrieval-augmented generation be evaluated when source documents, user questions, and domain language shift over time?
LLM-as-judge reliability, calibration, and bias
When can LLM judges be trusted as scalable evaluators, and when do they need human calibration?
Benchmark contamination and evaluation leakage
How can AI evaluations avoid overstating performance because the model, prompt, or team has already seen the test?
Cost-latency-reliability frontiers for compound AI systems
How should teams reason about tradeoffs between model quality, retries, routing, cost, and latency in compound AI systems?
Evidence graphs for reproducible AI assurance
Can AI assurance evidence be represented as a queryable graph from hypothesis to experiment to result to decision?
Agent trace semantics for real-world AI workflows
Which trace fields generalise across delegation-heavy, multi-agent workflows rather than single-agent tool-call sequences?
Prompt injection and excessive agency test protocol
What is the minimum reusable fixture and scoring protocol for testing prompt injection and excessive agency across different agent frameworks?
Cost-latency-reliability frontier for agentic workflows
How does test-time compute (retries, verifiers, rollouts) shift the cost-latency-reliability frontier established in ZAN-RN-010 for compound systems specifically running agentic, tool-using workflows?
Local-first AI assurance sandbox
What is the minimum local sandbox architecture that lets evaluation, tracing, security testing, and cost profiling run entirely offline before any cloud routing decision is made?
Geospatial AI assurance protocol
How should the evaluation harness be adapted to score geospatial and environmental intelligence workflows, where robustness to distribution shift and data quality matter as much as task success?
Evidence graph for AI R&D and assurance
Can a single evidence graph schema serve both AI R&D documentation needs and public-facing deployment-readiness review, without duplicating records?