Advanced workshop: Mastering AI Observability
Official Schedule Context
- Date/time: 2026-06-29 · 9:00am-11:00am
- Track/room: track TBD · Track 9
- Speaker(s): Doug Guthrie
- Session type/status: session · confirmed
Official Description
Your AI is in production, but is it actually good? In this hands-on workshop, you'll learn how to
uncover patterns in your production traces using Braintrust Topics, build custom scorers to target
real issues, and systematically improve your agent. By the end, you'll have a repeatable eval
workflow and trace-backed evidence that your AI is actually doing what you think it is.
Related YouTube Video
Evals 101 — Doug Guthrie, Braintrust (speaker-match related prior/adjacent AI Engineer video; captions: English auto-captions).
Transcript Status
Related video transcript availability: English auto-captions. Treat this as supporting context, not a recording of this exact scheduled session unless later confirmed. Cached at raw/sources/youtube-transcripts/bk0TmxoZlUY.txt (9,125 words).
People
Notes
- Pending transcript synthesis when an official recording or confirmed matching video is available.
Supporting Slides
- youtube bk0TmxoZlUY slides — extracted from the related public AI Engineer video.
Slide Evidence
- Slide-only cropped deck: youtube bk0TmxoZlUY dense slides (1 viable slide images).
- Related slide/OCR pages:
- youtube bk0TmxoZlUY dense slides
- youtube bk0TmxoZlUY reconstructed slides
- youtube bk0TmxoZlUY slides
- Slide-derived terms:
evals,microsoft,prompt,braintrust,scorer,improve,eval,scorers,smol,code,zapier,model,quality,test,dataset,input,rome,datasets