Triage Bot

Challenge

Across a portfolio of consumer products, a solo operator cannot read, classify (bug or feature request), file, and answer all user feedback in each product’s voice. Most teams use people for that judgment. The goal: one agent doing it safely, autonomous where mistakes are cheap and behind a hard human gate otherwise.

Process

Triage Bot is a persistent Python service (asyncio, discord.py) on one AWS EC2 instance. Feedback from Sentry widgets in each product fans out to per-project Discord channels and passes a moderation pre-filter, a mandatory PII scrub (Microsoft Presidio) before any model sees text, and a Gemini 2.5 Flash-Lite classifier returning a schema-validated label, confidence, and severity.

Then it acts on its own: structured GitHub issues for bugs, auto-fix PRs for high-confidence low-severity ones, feature requests cross-posted to the right forum, and replies drafted in each product’s brand voice, retried on banned-phrase hits. Sensitive topics (refunds, legal, security, pricing) go to a human, never auto-drafted. Adding a product is one YAML entry, no code change.

Safety is hardcoded: draft-only auto-fix PRs, no automation for high and critical bugs, prompt injection classified as spam, and public replies gated to my Discord ID. Gemini 2.5 Flash-Lite replaced Claude Haiku for roughly 10x lower per-message cost; I disabled thinking_budget after a measured token-starvation finding.

Result

Live in production. One service triages feedback for every product in the portfolio, each with its own enforced voice and escalation routing, at a target cost under $10 per month. It’s the portfolio’s clearest exhibit of an agent taking consequential action (real issues, real PRs) within hard, code-level safety boundaries. Unattended auto-reply waits on a sustained approval-rate threshold, not yet reached.