← Back to Work

Data engineering: LLM extraction pipelines over a unified entity graph

PodcastPal

A podcast discovery app built on one unified graph linking live events, host-and-guest relationships, and the merch those people sell.

Role
Sole engineer, designer, and operator
Year
2026
Status
In Development
  • React 19 / Vite / TypeScript
  • Python / FastAPI
  • Gemini 2.5 Flash (extraction)
  • PostgreSQL + PostGIS (Neon)
  • Lambda / EventBridge / SQS ingestion
  • D3.js / Leaflet
  • AWS CDK / Caddy on EC2

5,500+

Podcast episodes ingested and graphed

2026-04 · RSS ingestion + LLM extraction pipeline backfill

Challenge

Podcast events, host-and-guest relationships, and the merch those people endorse are usually separate features. PodcastPal makes them three views of one entity graph that enrich each other. The hard part: building it cheaply and correctly from messy third-party RSS, without a graph database or runaway per-request LLM cost.

Process

Ingestion is off the request path. A scheduled Lambda scrapes RSS into an S3 bronze tier; SQS workers run Gemini 2.5 Flash extraction into entities (podcasts, people, venues, events, merch); pages serve from Postgres with no per-request model cost. The graph lives in PostgreSQL with PostGIS by design: recursive queries and junction tables handle the relationship web at expected scale for roughly nothing, versus a graph database’s idle cost. A per-call cost tracker and budget ceiling cap extraction spend. D3 renders the lineage graph; Leaflet maps events.

Before beta, a dual-model adversarial audit had two independent AI reviewers work the same brief in isolation. I merged their findings into a fix queue and closed it out. Authorization, JWT validation, and SQL parameterization came back clean.

Result

Deployed to production at podcastpal.io with over 5,500 episodes ingested and graphed. Pre-launch, not open to customers. An entity-resolution pass for graph quality is designed and partly underway. The unified graph is the product spine.