Markdown source

Slides: Frontier results, on device - RL Nabors, Arize

Source Video

Frontier results, on device - RL Nabors, Arize

Relationship To World's Fair 2026

These slides are extracted from a public AI Engineer YouTube video connected to World's Fair 2026. Speaker-matched clips are supporting context unless later confirmed as exact session recordings; official livestream recordings are day-level/event-level source material.

Related Scheduled Sessions

Extracted Slides

slide-002.jpg

Slide text:

arize

You have agents.

We can test them.

slide-003.jpg

Slide text:

THE COST OF ONE-SIZE-FITS-ALL INFERENCE

slide-004.jpg

Slide text:

“[l]atency above 4 seconds degrades quality of experience...”

Mitigating Response Delays in Free-Form Conversations with LLM-powered Intelligent Virtual Agents July 7, 2025

slide-005.jpg

Slide text:

Couldn't connect to Claude

ERR_INTERNET_DISCONNECTED

Refresh

slide-006.jpg

Slide text:

Chief Product Officers should not confuse the deflation of commodity tokens with the democratization of frontier reasoning.

— Gartner, March 2026

slide-007.jpg

Slide text:

TASK-SPECIFIC MODELS CHEAT SHEET

Is a camera pointing at something?

Use vision models like MobileNet, YOLO, MediaPipe

slide-008.jpg

Slide text:

SLMs

LLMS BUT SMALL

slide-009.jpg

Slide text:

Small(er) Language Models (SLMs)

Smaller versions of LLMs containing several million to several billion parameters (LLMs may have hundreds of billions or even a trillion parameters)

slide-010.jpg

Slide text:

Energy consumption comparison

LLMs

SLMs

Task-specific Models

proportional energy consumed

slide-012.jpg

Slide text:

Rachel-Lee Nabors (they/them) nearestnabors.com Ichthy 00 todo

How lke! Successfully activated extension

RN How likely is It that Ichthysaurs had echolocatlon?

Looking at the evidence foric memory Successfully activated extension

How canlheipyou today narlgate messa!

rStisuuos 中0000

slide-013.jpg

Slide text:

Ichthyosaur echolocation capabilities

slide-014.jpg

Slide text:

Rachel-Lee Nabors (they/them) nearestnabors.com 00o Ichthyosaur echolocation capablites 8.8 pue poods ynm pooorei ea ppoo uopiu spodoeudao pue usy jo 00

it's moderately plauslble but far from certainprobably Lookdng at the cyidence Ior Ichthyosaur ccholocaton, Id say. What They DidUsea agility rather than echolocation

number on it. somewhere Inthe30-50%lkelihoodrange if I had toputa Visione Their large ey Lahts oven tn low

Here's my reasonlng: Evidence suggesting they might have had echolocatlon: 1Anatomical similariues: Some ichthyosaurs had eniarged Conclusion? Anbush Tacties They Ikely reled on stcalun snd rapid bursts o speed Tactile Sonsitivity; Someo specios may havo used lateras Hne systems (common In flsh) to detect prey movements: to capture prey.

2. Ecological niche Many ichthyosaurs were deep-diving stapedial bones (part of the ear structure) slmilar to modern cetaceans that echolocate. This couldIndlcate enhanced sound processing capabilitiess there is no evidence to support its presence kn ichthysaurs Thelr anatomy sound-basednavigation. to dm

predators hunting In low-llght conditlons:exactly the? Into that!

like in toothed whales and r g hins today. scenariowhere echolocatlon would be advantageous, Just

3.Convergent evolutlon:Ichthyosaurs show remarkable

How canh he p you today? Sdessalu oiebybu ot1t/lx

1 Sonnct4.SV 131 1.0000

slide-015.jpg

Slide text:

RIGHT-SIZING AI

slide-016.jpg

Slide text:

Rachel-Lee Nabors (they/them) nearestnabors.com Mima Nows口 Conversations Justnow G Sync

iversations Settings

Accounts APICredentials AI Feed Muted Dangerous

digests. Configure howMima usesAl to summarizeyour social feeds and generate topic Current Configuration Configure mima.social

UsingOpenAl erAnthropicAPI

VAPI Key-Ready

AboutAl Features

bsic keyordbaed summareFor bet eultusn alargr mcdel like LorQwn 2. 78 Alisusd tociuster smiatpost nto topics and generate summanesWitout At.topicswl show

Golden Dataset

@rachelnabors@edittrameThatisa fantasticusecase

Carl Assmann,SecFault & 16 others X

slide-017.jpg

Slide text:

Rachel-Lee Nabors (they/them) nearestnabors.com Mimalocal-modeleval

Goldendataset at a glance

28 Examples Threads 14 2-17 Messages/thread Annotated 100%

eachviewedlist+modal median5 hand-written[ref:N]

By platform By view context

22 Bluesky. O list. 14 14 modal.

Sourced from real Mima threads,anonymized. Each example pairs a thread input with a gold-standard summary and citation p

5models evaluatedagainst this set with 3repetitions each (420 runs total).

slide-018.jpg

Slide text:

Rachel-ee Nabors(they/them) nearestnabors.co Measures of Success

Dimension What it asks How it's measured

JSON validity Does the output parse?. Try JsoN. parse, count successes

Reference structural validity Do [ref:N] tokens point to messages that actually exist? message_count] Regex extract refs, check each N E [ 1,

Factual consistency Does the summary stay faithful to the thread, or invent claims? summary against its source LLM-as-judge -- Claude scores each

Length compliance Does it stay in the target word band? Word count vs context-specific limits (8-12 for list view, 19-46 for modal)

p50 latency: Typical TT summary Median across the eval set

p95 latency Worst-case wait 95th percentile across the e

Each evaluator returms a per-example scort. Average across the 28-exarmple golden datasel, with 3 repetitions per erampl?.

slide-019.jpg

Slide text:

Arize Phoenix

Trace the Exponential

The open-source platform for agent development and evaluation

slide-020.jpg

Slide text:

Capability eval

Asks, “What can this agent do well?” They should start at a low pass rate, targeting tasks the agent struggles with and giving teams a hill to climb

—Anthropic, Demystifying Evals for AI Agen

slide-021.jpg

Slide text:

Rachel-Lee Nabors (they/them) nearestnabors.com Datasots>goldon-summarios

golden-summaries Labet

Experiments Examples28 Evaluators Versions

ExperimentsAnalysis

0.25 075 0.5 0.0c 75s 5.0s 25s $O.0s

Coumna

name length_complianoe_oval roference_aocuracy_eval cemantio_cimiarity_eval mg latency totalcost totaltokons errorrate

5 lama-32- 30 1.00 100 0.89 0.58 01.3s 40.798 0.00% Tences

口 02b gemmy-4- 1.00 100 0.95 0.58 8.4s 63.742 0.00% Tences

qw3-170 0.89 100 Q76 0.51 07.26 ②81.293 0.00% Tracos

口 2 qwen25. 15b 0.92 0.96 0.72 0.49 010 36.930 0.00% Trocee

soonet- baselne daus- 100 100 0.96 aro 02.9s $0.22 42.931 0.00%

®

slide-022.jpg

Slide text:

“Small and Good Enough” Model

SAGE Model

slide-023.jpg

Slide text:

Reference accuracy (%) — higher is better

p50 latency (ms) — lower is better

Claude Sonnet (ceiling) Qwen 2.5 1.5B Qwen 3.1 7B Gemma 4 E2B Llama 3.2 3B (recommended)

slide-024.jpg

Slide text:

Rachel-Lee Nabors. Qrtdwoo souewans-uopooy seselea @ Lm1-32-36

mrerater)

am+-3.2-3b Avd 0 1.35 @ 405.69 Awa' 0 0.41 0 996.93 AvG'0 2.9s D 511.0s $0.01 claude-sonwt-basdr

yno $iu4p1 55922 05.70 2,160 01.sa c 925 c$+.01

(they/them)near hsircuts"! 0 cexpeeted_sum coeparing Sunil: Joking about Y.tPlyful Bodhldharma, expected_refere. bater! Pa1.to! Yeoyoops urorus srushy ue, puly Bodhidharza' avatar, pro=oting sumil'to consider'a haircut; Thrccpolntonc becoees Bodhidharms avatar, proapting sunll to conslder 4 halrcut'snd 'n, Agentic. Shaolin'schoole)": "summary"ilSm Jokingly, suggests,Threepointone becones Fretereaceso0h. Hsasaosns: Abugyof esbWtVurns-Vasuodsader Cison 'ale.trve. "proept: tokens:278, Ftotal tokens-a:314. coapletion tokens"l36. Estart ing on Agentic Shaotin Kung-tu schoot. Cgentie, Shaolin Kung-fu schoot. 1-injin' \"oiscuss ing' an' avatar, and' start ing 'an' Yreterencen g115. oravresponsejson\nAnVsuary tsuemsry'., -oiscussiog an avatar., sod! Cson_valial tne T6s2:: susxoy1daoud. coplet 1on tokens"1,455,) Ctotal_tokensas7it? Bodhhrmh1rcut appearance conparing to Bodhidhana? qthrecpointone's appearansekicoaparirg oiutodesyio snoge Surxoca(ixiemnst "reterences"U. ison vaud":'truen Erav_respoase". prompttokens.283, Fcoapletioctokens":31, "totaletokens"ti314 waryAJokt

1.00 AYo 0o'I Naeer eered uos 1.06 AvG 7

Largth_complaroe_t 1.09 1.89 AYG Cergth_conpllnoe erl 1.93 1.06 AVG Length_compdance

1.eo AYG eo:T N.s- Aoe.nooe-ecuereye] 1.06 Ay3 6

snantic_almRerity_svat 6.58 OY ts'o Cwmantic_ emllrity_era 鲁. 3? 0.38 AY3 tmmte_t

-.1 nca oltt. 02.10 01,696 09.5@3,430 03. 0

YtSan' Jokes that'Sunil·Pai Texpected_rta qetting'o'hircut Iret:3]'The"comersation'slso hints't modern Bodhiaharea'avatar (ref:2]f and a. response' about Pw response'e tn AYsunrAirTbiseusion., AThe discussion begsn with's'nention'ot. fono conceptual' petaphor'about' an watar'stoly [ret:lJ,:uhich'quickty'shifted Anto. 0..response" son\nfin A"summaryA"! conversarlon> Ebout'soncthing I. coaoarlson'ot threcpointore to bodhidhal rmresp. aede.

Hidden Non-Slide Evidence

Classification audit: raw/sources/slide-ai-classification/slides/fWXJM-J0ZB8/audit.json

Slide-Derived Subjects To Review

Subject extraction uses video title, related session titles/descriptions, transcript context, and OCR text when available. OCR is best-effort and should be reviewed against the embedded slide images.