Dallas Crilley
← All case studies

AI automation · In progress

CoHost AI Studio

Nothing publishes until it clears the gate.

One instrumented pipeline replaces the five-to-eight-tool relay independent producers run today. An episode publishes only after it clears a measured quality bar.

Who it helps: Built for independent podcast producers and studio operators. Fewer listener-facing episodes go out with avoidable technical defects, because publishing waits on the quality checks.

At a glance

languages
Python 3.11, TypeScript / React 19, Rust (Tauri)
pipeline
23-step DAG, 4 profile variants
quality gate
11 programmatic scorers + 5 LLM evals
tests
pytest suite gates every publish
history
built Feb to May 2026 (15 weeks)
repo
Private

The problem

Independent podcast producers run every episode through five to eight disconnected tools, with a manual handoff at each step and no signal when a bad master or a thin transcript slips through to publish.

How it’s built

  1. A DAG pipeline with idempotency and fault isolation

    Twenty-three steps run as a topologically sorted DAG on a thread pool. Each step has an idempotency predicate, so partial runs resume cleanly, and per-step cascade behavior isolates a failure (a stitch timeout does not block show-notes generation). The graph is validated for cycles at build time, before the first episode runs.

  2. An instrumented quality gate, not a vibe check

    Eleven steps are scored against programmatic metrics: LUFS delta for audio mastering, word confidence and speaker resolution for transcripts, artifact completeness for video. A weighted composite plus a per-step hard-fail threshold feeds straight into the publish decision; below the bar, the pipeline halts before distribution.

  3. LLM evals layered over deterministic scores

    Five content-heavy steps add opt-in LLM evaluation via OpenRouter, each returning a criterion, score, and reasoning that merge into the scorecard alongside the programmatic metrics. Fast deterministic checks always run; the slower qualitative ones are opt-in.

By the numbers

steps in the podcast pipeline
23
programmatic scorers plus LLM evals
11 + 5
from first commit to a working gated pipeline
15 wks
pipeline profile variants from one 23-step DAG
4

Where this stands today

Live, pre-launch. Four of five P0 launch criteria are validated; YouTube OAuth and multi-show isolation remain open. Functional and test-validated for single-show local operation.

Get in touch

Want the parts of CoHost AI Studio that are not in a public repo? Email me at dallas@dallascrilley.com.