# User Feedback Synthesis Agent

> A feedback-synthesis agent that clusters user feedback into themes with cited quotes and honest frequency, separating signal from small-sample noise. Never fabricates.

- **Category:** Product Management
- **Difficulty:** Intermediate
- **Setup time:** 40 min
- **Stacks:** Claude, LangGraph, OpenAI
- **Version:** 2.0.0 (updated 2026-06-21)
- **Source:** https://agent-kits.com/kit/feedback-synthesizer

## Overview

Clusters raw feedback (tickets, reviews, surveys) into clear themes with representative quotes.

Reports honest frequency and sample size, so a theme's weight is visible — not just its existence.

Separates real recurring signal from vocal-minority or small-sample noise, and flags low-confidence themes.

Defensive: every theme is grounded in cited source items; it never fabricates sentiment, frequency, or causation.

## When to use it

- You have more user feedback than anyone can read and need it distilled into themes.
- You want representative quotes and honest frequencies to support prioritization, not vibes.
- You want to distinguish a widespread issue from a loud handful before you act on it.
- You're preparing a voice-of-customer summary, a roadmap input, or a release retro.

## When not to use it

- You want it to declare priorities or make roadmap decisions — it synthesizes evidence, humans decide.
- Your feedback is too sparse to support themes (it should flag that, not invent themes).
- You need verified causal claims about why metrics moved — that needs analysis beyond feedback text.
- You can't let it handle user text with appropriate PII care.

## Architecture

- **Feedback ingestion** — Pulls the feedback corpus from its sources (tickets, reviews, surveys, notes) and records the total volume and mix.
- **Dedupe & normalize** — Removes duplicates and near-duplicates and normalizes text so frequency counts reflect distinct feedback, not repeats.
- **Theme clustering** — Clusters items by the underlying issue or request rather than surface wording, forming coherent candidate themes.
- **Evidence & frequency** — For each theme, counts supporting items, extracts real representative quotes, and computes honest frequency against the total.
- **Confidence & signal check** — Assesses confidence from sample size and clarity, separates widespread signal from vocal-minority noise, and surfaces counter-themes.
- **Synthesis output** — Produces ranked themes with quotes, frequencies, sentiment, confidence, and caveats — plus what was too sparse to assert.
- **Review & export** — Hands a faithful, cited summary to the product team for prioritization, with PII kept out of the output.

## Files in this bundle

- `system-prompt.md` — the agent's system prompt
- `tools.json` — tool/function definitions the agent expects
- `workflow.md` — step-by-step run loop
- `.env.example` — environment variables to fill in
- `kit.json` — full structured metadata
- `starters/` — minimal starter snippets
- `LICENSE` — Apache-2.0 (code, configuration, and schema)
- `LICENSE-CONTENT` — CC BY 4.0 (specification text and documentation)
- `NOTICE` — attribution and trademark notice
- `run.py` — **runnable demo**: real agent loop with mock tools (`pip install anthropic`, set `ANTHROPIC_API_KEY`, `python run.py`)
- `agentaz.json` / `agentaz.schema.json` — AgentAz™ governance spec + schema

## Implementation notes

- Require real, cited quotes for every theme and block fabricated ones; a synthesis you can't trace to source items is worse than no synthesis.
- Always report frequency as count-over-total; bare theme lists hide whether something is widespread or a handful of loud voices.
- Flag small-sample and ambiguous themes as low-confidence instead of dropping or inflating them — leads are fine, just labeled.
- Forbid causal claims unless the feedback states them; route metric-causation questions to actual analysis, not vibes from text.
- Surface counter-signal and mixed sentiment so the team sees dissent, not just the dominant narrative.
- Keep PII out: quote content, redact identifiers, and treat the corpus as sensitive user data.
- Spend the strong model on theme framing, confidence, and counter-signal — a cheaper model can dedupe and cluster.

## Variations

- **Basic — Theme summarizer:** Clusters a feedback corpus into themes with representative quotes and counts for a quick read. Single source, on demand.
- **Advanced — Evidence-graded synthesis:** Adds honest frequency, confidence grading, signal-vs-noise separation, counter-signal, and causation guardrails across multiple sources.
- **Enterprise — Continuous voice-of-customer:** Adds scheduled multi-source synthesis, segmentation, trend tracking over time, PII governance, and integration into roadmap and research workflows.

## FAQ

**Will it make up themes or quotes?**

No — that's its core constraint. Every theme must be grounded in cited source items, and quotes must be real excerpts from the corpus. If it can't cite something, it doesn't claim it.

**How does it avoid over-weighting a loud minority?**

It reports honest frequency (count over total) for every theme and flags small-sample or ambiguous themes as low-confidence, so a handful of emphatic items can't masquerade as a top issue.

**Can it tell me why a metric like churn changed?**

Only what users actually said. It won't claim feedback caused a metric change unless the feedback states it — those causal questions need cohort analysis, and the agent will say so rather than invent a story.

**Does it decide what to build?**

No. It produces a faithful, evidence-graded synthesis for prioritization; the product team makes the roadmap decisions.

**How does it handle user privacy?**

It treats the corpus as sensitive, quotes content rather than personal data, and redacts identifiers so the synthesis doesn't leak PII.

**What about themes with very little data?**

It surfaces them as low-confidence leads with their small frequency rather than dropping them silently or inflating them, so genuinely narrow issues stay visible without distorting priorities.


---

© 2026 AgentKits · www.agent-kits.com

**Open source.** Code, configuration, and the AgentAz™ schema are licensed under
**Apache-2.0**; specification text and documentation under **CC BY 4.0** (with
attribution to AgentKits, link to https://www.agent-kits.com). See LICENSE, LICENSE-CONTENT,
and NOTICE in this bundle. "AgentAz" and "AgentKits" are trademarks of AgentKits.

**Flagship reference blueprint.** View the source, AgentAz™ spec, and frozen v1.0 schema
in the public registry: https://github.com/agent-kits/agentaz
Original: https://agent-kits.com/kit/feedback-synthesizer
