- What counts as an objective failure?
- Downtime, provider timeouts, empty output, truncated output, invalid structured output or tool call arguments when a schema exists, and material billing or cache anomalies with observable evidence.
- What is reported but not automatically credited?
- Latency without a configured threshold, model drift without an agreed baseline, refusals that may be appropriate, factuality without ground truth, and policy or security signals that need review.
- How are Bring Your Own Keys and Managed different?
- Bring Your Own Keys shows failed calls and cost on your own provider bill. Managed can apply bounded service credits for eligible objective failures because Defendr operates the provider relationship as part of the service.
- Does Defendr measure hallucinations?
- Not as a general in path failure class. Factuality needs ground truth: a reference answer, retrieval corpus, database result, human label, or downstream verifier. Without that, Defendr can only flag weaker signals.
- What does the loss report include?
- Failure type, event count, estimated cost, evidence, and remedy status. The Accountability page includes a sanitized sample scorecard.