RecipeWalk

Challenge

Cooking is hands-busy work. The phone sits two or three feet away, hands are wet or messy, and the kitchen is loud. Recipe blogs bury each step under ads and life stories, and a general chatbot loses track of where you are. RecipeWalk structures the recipe, then a real-time voice assistant guides the cook one step at a time without losing its place.

Process

I redesigned the interface prototype-first: a click-through prototype of every screen, reviewed one page at a time, then a sequenced plan for rebuilding the React frontend. The rules came from the cook’s situation. Glanceable from three feet. Voice leads and the screen reassures, so every action has a no-touch path. The current step is always the loudest thing on the screen. The cook screen shows a single step in large type, the ingredients for that step, and running timers, with a voice dock where the navigation would be.

Gemini Flash turns a pasted link or a photo of a recipe card into ingredients, steps, and timers, and the cook confirms the draft before voice ever reads a quantity aloud. Voice connects client-direct to the Gemini Live API over WebSocket. A server-side state machine owns the current step: each voice session is scoped to that step’s instructions and exact amounts, so the assistant stays on the current step until the cook moves it. Saying “next” or “set a timer for four minutes” fires a tool call that moves the state machine or starts a timer, and the cook can talk over the assistant to cut it off.

My voice-eval harness runs scripted turns and synthesized speech through the exact production config. A second model grades each turn: the right tool, the right step state, the right answer. It caught a silent failure before launch: the provider’s session-token path was dropping the app’s tools, so the assistant said it was moving to the next step while nothing happened. I built the tools into the token, and the probe went from fail to pass.

Other projects copy RecipeWalk’s reference implementations for secrets loading, deployer IAM scoping, and gift-code security.

Result

Deployed to production at recipewalk.com, all tests green (282 backend, 120 frontend). A real-money buy, redeem, refund, and revoke cycle proved the gift flow end-to-end. Not yet open to customers; a paid beta waits only on a go-live decision I am holding on purpose.