Turning customer feedback into product decisions

Agent Swarm can collect the support records and customer meeting notes your team is authorized to use, group the possible recurring problems it finds, and attach the source excerpt behind each one, but a product owner verifies every theme and decides what to investigate. That split answers the question this page exists for. How can a product team find recurring customer problems across support records and conversations without treating an AI summary as a roadmap decision? By separating theme gathering from theme trust. An agent proposes, the evidence travels with the proposal, and a named person owns the call.

A company with 20 to 100 engineers already sits on years of support tickets and call notes. The feedback itself is rarely in short supply. Reading it the same way twice is what is missing.

---

TL;DR:

• A theme an agent produces is a hypothesis with evidence attached, not a roadmap item. The product owner verifies it, corrects it, merges or splits it, then decides what to investigate.
• Collect with a deterministic script that emits JSON, so the model never chooses its sources and the same records produce the same payload.
• Every theme carries excerpts, dates, record counts and the limits of its sample. A count with no window and no segment is not a signal.
• Duplicate records, unrepresentative samples, conflicting accounts and privacy boundaries decide whether the output is usable.
• Use a dedicated feedback tool when it already holds the records, or manual review when volume is small or the records are too sensitive for an agent.

---

Table of Contents

• When something else fits better
• An illustrative workflow
• Failure modes and how to handle them
• What Agent Swarm provides today, and what you build
• Who decides
• Where to go next
• References

When something else fits better

Start with when not to use it.

If a dedicated feedback or support analytics tool already holds your records and its tagging answers the question, use it. Those tools do one job well and cost less than a pipeline you maintain. The gap opens when the records sit in three or four systems the tool cannot see, and the tagging stops at whichever fields the support team fills in.

Manual review is the right answer more often than vendors admit. When a month of feedback fits in an afternoon, a product manager reading forty tickets learns which complaints are the same complaint in different words, which no summary shows. Manual review also wins when the records are too sensitive for an agent at all, for example clinical notes your legal team has not cleared for processing outside its system of record.

Agent Swarm fits the middle case. Your evidence is spread across a support desk and a call recorder, with older material in a shared drive. You want the synthesis self-hosted on infrastructure you control, you want to see which records went in, and a person should own the conclusion.

Two limits belong here. This page describes a pattern built from documented primitives, not a shipped feedback feature, and no customer result is quoted because none is published for it. The workflow produces an internal document and sends nothing to a customer. For a managed deployment, use the contact address on agent-swarm.dev; this page covers no cloud specifics.

An illustrative workflow

Illustrative means a design assembled from documented parts, not a product feature, a customer deployment or a measured outcome. The five steps below are the shape we would build for a 60-person product team whose support desk and call notes live somewhere other than its email.

(a) Collect only records the team is authorized to use

Two inputs: support records and customer meeting notes. Not the whole CRM, not every internal channel where someone mentions a customer. The source list is a decision your team makes in writing, and the collection step encodes it.

The collection is a script, not a prompt. It fetches each source over HTTP or SQL, normalizes the fields, and emits one JSON payload per run, with no model in that step. The same records produce the same payload every time. A broken source is an error with a line number rather than a paragraph you have to distrust. Months later, "which records did the model read" has a file for an answer.

(b) An agent groups candidate recurring problems

A Researcher agent reads the payload and proposes themes. It merges candidates across records and accounts, names each theme in language close to what customers wrote, and keeps the accounts that contributed.

Ask for candidate themes, not a ranked list. A ranking is a judgment about priority, and priority belongs to the person who owns the roadmap. The agent is good at the mechanical middle: reading two thousand records and noticing that eleven describe the same export failure in different words.

(c) Every theme carries its evidence and its limits

A theme with no evidence is a rumor with good grammar. The minimum an agent should attach to each theme:

Field	What it holds
Excerpts	Two or three verbatim quotes, each with the record id and date
Record count	How many records support the theme, and how the count was reached
Accounts	Which accounts appear, and whether one account dominates
Window	The date range of the collected records
Gaps	What the sample cannot show, such as a segment with no records

The gaps row is the one teams skip and the one that keeps the output honest. With a six-week window and enterprise-only records, the owner knows the SMB voice is missing instead of assuming it agrees.

(d) A product owner verifies each theme

This step separates a synthesis from a summary. One named product owner works the list in one sitting and does four things: checks the excerpts against the records, corrects a theme whose name overstates what the quotes say, merges two themes that describe one problem, and splits a theme that hides two.

A theme that survives that check is a hypothesis with a name and an owner, dated. That is enough to put on a list of candidates for investigation. It is not enough to put on a roadmap.

(e) The output is a report the team reviews

The run ends in a Page or a Markdown report the team reads together: the themes with their evidence, plus the owner's notes on the gaps. A page beats a chat message here, because the review needs to open the excerpts next to the theme.

Failure modes and how to handle them

Four failures decide whether the output is usable. None disappear on their own.

Duplicate records

The same issue reaches you three times: a ticket, a call where the customer explains it, and an email that repeats the call. A naive count triples it and the theme looks more urgent than it is.

Deduplicate before anything ranks. Key on account plus an issue window rather than message id, because the analyst who opens the ticket restates the customer's words rather than creating an independent report. Treat every count as a lower bound and say so: four accounts beat eleven records.

Unrepresentative samples

Three patterns cause this. A loud account generates most of the volume, so the theme is one customer with a support habit. One segment dominates because a single integration or plan tier files tickets through the channel the collector reads. And one window catches an incident, not a trend.

Handle all three the same way. Name the window and the segment in every theme, compare two windows before calling something recurring, and cap how much of a theme one account can supply. Past the cap, report a single-account signal instead of deleting it. A large customer's voice is information. It is not a pattern.

Conflicting feedback across accounts

Two accounts ask for opposite behavior, and both are right for their own workflow. Averaging them produces a design nobody wanted. Reporting the conflict is the useful answer: name both sides, name the accounts, and write the trade-off as the decision it forces.

This is where an agent earns its place and then has to stop. It can find the disagreement across a payload of records, which a person will not read end to end. It cannot pick a side, because that depends on contract value and on what the team can maintain. The owner writes the resolution and the reason.

Privacy boundaries

Support records contain personal data, and some customers never agreed to have their conversations processed by a model.

Four habits handle most of it. Give the collection step scoped credentials and the narrowest read access each source allows. Exclude free-text fields you do not need, so a billing export carries amounts rather than support narratives. Keep payloads and reports in your own environment. And set a retention window on the working set, so old conversations do not pile up in a folder nobody reviews.

The harder case has no clean technical answer. If a contract limits how a customer's conversations are used, exclude those records and say the sample is partial. An audit trail that names what was left out beats a sample that quietly includes it.

What Agent Swarm provides today, and what you build

Everything below is documented in the repository or the docs site. The next subsection names what these parts do not cover.

Per-account working directories and meeting-note ingestion. Each account can have a persistent agent-fs folder where notes and transcripts accumulate across months, and a post-call step can append a new transcript. The proactive customer support playbook describes that loop, and the per-customer working directories pattern explains why files beat agent memory for account context. One honest detail: the meeting-note ingestion there runs through the granola-api skill, which is custom and swarm-managed in our deployment, not shipped in the box.

Deterministic collection, agent synthesis, and reports as Pages. The multi-source reporting playbook covers the pattern this page reuses: swarm-script nodes fetch each source over HTTP or SQL into one unified JSON payload, a Researcher turns that payload into narrative, and multi-chart reports render as auto-hosted Pages. The model stays out of the fetch, and one agent holds the scoped cross-source credentials.

Human approval gates. A human-in-the-loop workflow node pauses the run, posts to Slack, and resumes only when a person responds, with a defined branch for approve and for reject. The gate pattern says to gate only steps that cannot be undone, which is why the report needs no gate and any customer message would.

Credentials outside agent context. Registered connections and the scripts credential broker let a script authenticate outbound calls with a [REDACTED:<configKey>] placeholder that the runtime swaps for the real value only when the destination host is allowlisted. The raw secret never enters the script. Bindings are scoped to a level, so a credential registered for one agent does not reach another.

A security posture you configure. The security page separates what the repository implements from what stays with the operator: a self-hostable deployment where the database and files live on your volume, one container per worker with a private workspace, and config-store secrets encrypted at rest with AES-256-GCM.

What you build

No support-desk integration is documented, so reading support records is custom work: the endpoints, the fields you keep, the fields you exclude. Write that fetcher with scoped, read-only permissions where the provider offers them, and give it an owner who is not the product manager, because someone has to notice when a field changes.

The rest is also yours: the theme schema, the deduplication rule, the single-account cap and the verification checklist. Write them next to the workflow, so the third person to run this does not invent a different rule set.

Who decides

Agents propose themes and evidence. People verify themes and set priorities, and any customer contact is their call.

An agent in this workflow does not decide roadmap priorities and does not contact customers. It reads records and groups them, then writes a report addressed to a person who can be asked why. That boundary makes the output safe to circulate: nobody wonders whether a model already made a commitment on the company's behalf.

The adjacent case needs the same discipline. The customer support playbook puts a value report in front of a human before any direct-to-customer email leaves, and its rule is unambiguous: a human signs off, even when the drafting is good. Feedback synthesis is the internal half of that loop. When a theme becomes a message to the customer who inspired it, a person approves it first.

Where to go next

The multi-source reporting playbook and the customer support playbook are the two sources this design borrows from.

If you are still deciding which recurring work deserves an agent, the field guide for small companies covers the operating model this page sits inside. Shared context for AI agents covers how much context agents should share and how it fails. The task delegation architecture explains the delegation shape behind the Researcher step. The examples page shows the same primitives on other jobs.

References

1. Proactive Customer Support. https://docs.agent-swarm.dev/docs/playbooks/proactive-customer-support
2. Reports from Multiple Sources. https://docs.agent-swarm.dev/docs/playbooks/reports-multiple-sources
3. Pattern: Per-Customer Working Directories. https://docs.agent-swarm.dev/docs/playbooks/patterns/per-customer-working-directories
4. Pattern: HITL Gates. https://docs.agent-swarm.dev/docs/playbooks/patterns/hitl-gates
5. Scripts credential broker. https://docs.agent-swarm.dev/docs/guides/scripts-credential-broker
6. Agent Swarm security and trust. https://www.agent-swarm.dev/security