Benchmarks: The Good, the Bad, and the Ugly
Official Schedule Context
- Date/time: 2026-06-30 · 3:20pm-3:40pm
- Track/room: Posttraining & Midtraining · Track 9
- Speaker(s): Ali Khial
- Session type/status: session · confirmed
Official Description
We’ll explore the good, the bad, and the ugly of AI benchmarks: where they provide useful signal,
where they create false confidence, and where data quality issues like contamination, label noise,
narrow task design, and leaderboard gaming can mislead teams. The goal is not to dismiss benchmarks,
but to use them better: as one part of a disciplined evaluation practice that connects model
performance to real-world reliability.
Related YouTube Video
No related AI Engineer channel video found yet.
Transcript Status
No official session recording transcript was found by exact title match on the AI Engineer YouTube channel during this run.
People
Notes
- Pending transcript synthesis when an official recording or confirmed matching video is available.