SIMULATED TEST HARNESS • MILESTONE 1Live container clusters scheduled for Milestone 2
Evaluation Test Library
Standardized behavioral, grounding, and adversarial test suites for autonomous student agents.
functional10 test scenarios
Functional Capability Suite
Verifies multi-turn company research, tool dispatch, and structured financial extraction.
FN-001Company Overview Extraction
criticalFN-002Financial Table Parsing
criticalFN-003Competitor Multi-Hop Retrieval
majorFN-004Report Schema Compliance
majorgrounding5 test scenarios
RAG & Evidence Grounding Suite
Evaluates claim-to-document attribution and factual correctness.
GR-001Strict Attribution Check
criticalGR-002Hallucination Trap
criticalcitation5 test scenarios
Citation Veracity Suite
Validates all footnotes, URLs, and source anchors.
CI-001Citation Resolution
majoradversarial10 test scenarios
Adversarial & Injection Suite
Attacks agent with direct injections, indirect document triggers, and secret exfiltration.
PI-001Direct System Prompt Leakage
criticalPI-017Indirect Document Injection
criticalPI-018SSRF & Metadata Service Probe
criticalrepeatability5 test scenarios
Repeatability & Consistency Suite
Executes agent 5 times with identical seeds to evaluate consistency.
REL-001Output Variance Test
majorefficiency3 test scenarios
Compute & Cost Budget Suite
Enforces ceiling on latency, token volume, and dollar spend.
EFF-001Latency SLA (<5.0s)
minorEFF-002Token Ceiling (<20,000)
minor