# Examples — Metric Anomaly Investigation Agent

## Conversion drop localized to one segment after a deploy → ranked hypothesis

Checkout conversion fell. The agent confirms it's real, localizes it to one platform + payment method, and correlates with a deploy — labeling it a strong association, not proven cause.

**Input**

```
Metric: checkout_conversion dropped from 0.31 to 0.22 on 2026-06-19. Investigate.
```

**Output**

```
{
  "metric": "checkout_conversion",
  "anomaly_confirmed": true,
  "decision": "REPORT",
  "magnitude": "-29% (0.31 -> 0.22), well below the 0.28–0.34 expected range",
  "localization": "Almost entirely iOS + 'Apple Pay' checkouts (that segment fell 0.30 -> 0.08 and accounts for ~85% of the total drop); web and card checkouts are normal.",
  "hypotheses": [
    { "explanation": "A regression in the Apple Pay flow introduced by mobile deploy v5.12", "evidence": "change_point at 14:10 on 06-19 aligns within minutes of v5.12 rollout; drop is confined to iOS+Apple Pay; error logs show a spike in 'payment_sheet_init_failed' on iOS post-deploy", "type": "likely_causal", "confidence": 0.8, "to_confirm": "Roll back v5.12 in a canary or check the Apple Pay init error rate before/after" },
    { "explanation": "Apple Pay outage (external)", "evidence": "no corresponding Apple system-status incident found in the window", "type": "correlation", "confidence": 0.15, "to_confirm": "Check Apple PassKit status / third-party reports" }
  ],
  "data_quality_notes": "Data fresh and complete; not seasonality (weekday, within normal traffic).",
  "recommendation": "Treat v5.12 Apple Pay regression as the leading hypothesis: check the iOS payment-sheet error and consider a rollback/canary to confirm.",
  "escalation": { "needed": false, "reason": "" }
}
```

**Note:** Strong investigation: it confirms the drop is real (not seasonality/data quality), localizes ~85% of it to iOS+Apple Pay, and ties the change point to a deploy with corroborating error logs. Crucially, even the leading hypothesis is labeled 'likely_causal' at 0.8 with a concrete way to confirm — it doesn't declare proof from observational data.

## 'Spike' is a double-count → flag data quality, don't explain a non-event

Signups appear to spike. Before theorizing about a viral moment, the agent catches a pipeline double-count and reports that instead.

**Input**

```
Metric: daily_signups spiked from ~1,200 to ~2,400 on 2026-06-20. Explain the growth.
```

**Output**

```
{
  "metric": "daily_signups",
  "anomaly_confirmed": false,
  "decision": "DATA_QUALITY",
  "magnitude": "Apparent +100% (1,200 -> 2,400)",
  "localization": "The 'extra' signups are exact duplicates: each user_id appears twice on 06-20 with identical timestamps from two pipeline sources.",
  "hypotheses": [
    { "explanation": "Double-counting from a duplicate event stream after the 06-20 ingestion change", "evidence": "100% of the surplus rows are duplicate (user_id, timestamp) pairs; distinct user_ids = ~1,210, in line with normal", "type": "data_quality", "confidence": 0.95, "to_confirm": "De-duplicate on user_id; check the 06-20 ingestion/pipeline change" }
  ],
  "data_quality_notes": "Likely caused by a duplicate ingestion path introduced 06-20; distinct-user count is normal (~1,210). The 'spike' is not real growth.",
  "recommendation": "Do not celebrate growth — fix the duplicate ingestion and de-dupe. True signups are ~1,210, consistent with prior days.",
  "escalation": { "needed": false, "reason": "" }
}
```

**Note:** The discipline that makes this trustworthy: before explaining a 'doubling' with some exciting narrative, the agent checks integrity and finds the surplus is exact duplicates. It reports a data-quality issue and explicitly says the spike isn't real — preventing a celebratory but false conclusion.

## Real drop, but under-determined → ranked hypotheses + honest 'insufficient'

A metric genuinely dropped, but nothing correlates cleanly and the segment data is noisy. The agent refuses to manufacture a single cause.

**Input**

```
Metric: weekly_active_users down 6% week-over-week. No deploys, no campaign changes, drop is spread across all segments. Explain it.
```

**Output**

```
{
  "metric": "weekly_active_users",
  "anomaly_confirmed": true,
  "decision": "INSUFFICIENT",
  "magnitude": "-6% WoW, modestly outside normal weekly variance",
  "localization": "Broad-based: the decline is spread roughly evenly across regions, platforms, and cohorts — no single segment drives it.",
  "hypotheses": [
    { "explanation": "Normal variance / a soft week (holiday-adjacent)", "evidence": "drop is broad and within ~2x typical weekly swing; a regional holiday overlapped part of the week", "type": "correlation", "confidence": 0.4, "to_confirm": "Compare to the same week last year and holiday-adjusted baseline" },
    { "explanation": "Early sign of engagement decay", "evidence": "no event correlates, but prior 2 weeks were flat-to-down", "type": "correlation", "confidence": 0.3, "to_confirm": "Watch next 1–2 weeks; check retention curves by cohort" }
  ],
  "data_quality_notes": "Data complete; no integrity issues found.",
  "recommendation": "No single cause is supported by the data. Most likely a soft/holiday-adjacent week; monitor next week and compare to a holiday-adjusted YoY baseline before treating it as a trend.",
  "escalation": { "needed": true, "reason": "Real but under-determined — no correlating event and a broad-based drop; needs more time/data to distinguish noise from trend." }
}
```

**Note:** The epistemic-humility example: the drop is real but broad-based with nothing correlating. Instead of inventing a tidy cause, the agent offers two low-confidence hypotheses, declines to pick a single explanation, and escalates with exactly what would resolve it. Knowing when NOT to claim a cause is what separates a trustworthy analyst from a plausible-sounding one.
