Your Agent Is Lying to You About Whether It Worked
Official Schedule Context
- Date/time: 2026-06-29 · 12:05pm-12:25pm
- Track/room: track TBD · Expo Stage 1 NE
- Speaker(s): Dat Ngo
- Session type/status: session · confirmed
Official Description
Every span is green, every tool call returned cleanly, and the agent still regenerated the same plan
27 times before giving up invisible to any outcome metric, obvious in the trajectory. We pull up a
real trace where the outcome looks healthy and the path is a disaster, then show Signal, our agent,
surfacing it automatically: sweeping the project, ranking it above the noise, and linking straight
to the offending trace with debugging evidence attached. The live version of the trajectory-over-
outcomes argument, with a one-click path from "something's wrong" to "here's exactly where."
Related YouTube Video
LLM Observability, Evaluation, Experimentation Platform — Dat Ngo, Arize (speaker-match related prior/adjacent AI Engineer video; captions: English auto-captions).
Transcript Status
Related video transcript availability: English auto-captions. Treat this as supporting context, not a recording of this exact scheduled session unless later confirmed. Not fetched yet.
People
Notes
- Pending transcript synthesis when an official recording or confirmed matching video is available.
Supporting Slides
- youtube JsCCrBF7F1g slides — extracted from the related public AI Engineer video.
Slide Evidence
- Slide-only cropped deck: youtube JsCCrBF7F1g dense slides (1 viable slide images).
- Related slide/OCR pages:
- youtube JsCCrBF7F1g dense slides
- youtube JsCCrBF7F1g reconstructed slides
- youtube JsCCrBF7F1g slides
- Slide-derived terms:
observability,evaluation,teams,engineer,europe,braintrust,workos,openal,experimentation,genal,hard,tackling,know,sevan,arize,platform,google,deepmind