---
title: "What latency and cost budget is right for agent systems?"
category: "questions"
status: "active"
sourceLabels: ["Official schedule", "Slide/OCR evidence", "Tool inventory", "Topic synthesis"]
---

# What latency and cost budget is right for agent systems?

## Why This Question Matters
Agent systems turn inference into a product constraint: latency, throughput, routing, local versus hosted execution, and token cost shape what can be shipped.

## Current Working Answer
This page is a first-pass research question, not a final recommendation. Use the linked evidence to refine the answer as more exact session recordings, transcripts, and reviewed slide readings become available.

## Source Evidence
- [[inference-engineering]] — Topic synthesis
- [[vllm]] — Tool inventory
- [[modal]] — Tool inventory
- [[openrouter]] — Tool inventory
- [[tokenspeed]] — Tool inventory
- [[2026-06-29-charles-frye-what-is-an-inference-engine-anyway]] — Official schedule
- [[2026-06-29-du-an-lightfoot-agents-that-own-their-inference-building-production-ai-agents-on-dedicated-gpus]] — Official schedule
- [[2026-07-01-qianru-lao-routing-llm-inference-in-production-from-engine-signals-to-policy]] — Official schedule
- [[2026-06-30-jesse-hall-latency-is-a-budget-humanlike-is-the-goal]] — Official schedule
- [[2026-07-01-sujee-maniyam-optimizing-open-models-for-production-grade-inference]] — Official schedule
- [[youtube-DeFF3J8T5Pk-slides]] — Slide/OCR evidence
- [[youtube-ESbWpPT_9-o-slides]] — Slide/OCR evidence
- [[youtube-sRpqPgKeXNk-slides]] — Slide/OCR evidence

## Follow-Up
- Extract specific claims from the linked source pages.
- Separate official schedule evidence from supporting YouTube, transcript, and OCR evidence.
- Convert stable answers into playbooks, harnesses, or evaluations where appropriate.