Markdown source

The Dark Arts of Web Automation: Teaching Agents to Use Websites Like Humans

Significance

This is a critical talk for the wiki's browser-agent and agentic-web coverage. It receives extra recording-search effort, richer company and tool context, and deeper synthesis because it connects web automation technique to repeatable engineering practice.

The talk matters because it argues for browser automation as durable infrastructure rather than disposable agent clicking. Corey Gallon's core claim is practical: if a browser task will happen more than once, an AI agent should be able to turn it into a repeatable program that drives Chrome through the Chrome DevTools Protocol, not burn tokens replaying the same step-by-step browsing sequence.

This talk sits at the fault line between three approaches to the web:

The important idea is not simply "use a browser." It is that browser work can become a programmable substrate for agents. Once an agent has discovered the right sequence, the workflow should graduate from improvisation to reusable automation: loops, pipes, scripts, scheduled runs, and evidence trails.

That makes the session especially relevant to agentic engineering. A production agent does not just need a model and a tool list. It needs a way to interact with the messy web, preserve state, observe what happened, and avoid paying reasoning tokens for mechanical repetition.

Conference Context

Session Description

Anything you can do in a browser, your agent can do too. Not by tiptoeing through an MCP server one polite, token-burning call at a time -- properly, programmatically, the way you'd drive any other tool. I'll show you how with chrome-agent, an open source wrapper over the Chrome DevTools Protocol that has become irreplaceable in my everyday work. If you'll ever do a browser task more than once, step-by-step MCP browsing is slow, brittle, and bills you tokens for every single click. A CLI straight onto CDP makes the whole browser programmable: loop it, pipe it, script it, walk away. Write it Tuesday, run it a thousand times Wednesday, all without a second of AI agent babysitting. We'll dispel the MCP hype and myths, with successful demonstrations of cheeky things like: the power of CLI-based browsing and how its so much more capable than mere MCP; reaching through those oh-so-clever cross-origin iframes to clear the verify you're human checkboxes; showing that a JavaScript .click() is not a click, rather, just a function call in a costume that is banhammerable; ultimately, proving that a CDP browser operates just like a meatbag with a mouse and keyboard. You'll learn how to point your AI agents at real, messy, uncooperative websites and web applications and have them get things done exactly the way that you would.

Recording Search Status

No exact AI Engineer YouTube recording for this session has been found yet.

Searches performed for this page:

Interpretation: treat youtube JsKTQbT58BY as supporting speaker context, not as the recording of this session. If the official cut or track recording appears later, this page should be revisited first.

Technical Pattern

The talk centers on chrome agent, an open source wrapper over the Chrome DevTools Protocol.

The key technical pattern is CDP-first browser control:

This contrasts with generic browser-use patterns where each step can require a fresh model call, a fresh visual parse, and a fresh decision about the next click.

Applied Pattern

Use the Dark Arts pattern when:

Do not treat this as permission to bypass site rules, security boundaries, or user consent. The page should be read as an engineering argument for reliable browser automation, not as a recommendation to violate terms, evade protections, or run unreviewed actions against third-party systems.

Media Evidence

These are phone-photo slide captures from the Google Photos AIE Slides album. They are supporting slide evidence and do not override official schedule fields.

Connections

Sources

Evidence Graph

This evidence graph is generated from currently linked source material: official schedule text, related video pages, cached transcripts, visible slide text, dense/reconstructed slide pages, and AI slide-classification audits.

Media Signals

Agent Reading Notes

Use these signals to refine the synopsis, topic links, people/company context, and method notes. If a source is a related external video rather than an exact official recording, keep it framed as supporting evidence.

Evidence Boundary

Official schedule and speaker data are canonical for the talk title, speaker, date, track, and room. The recording has not been found as of this enrichment pass. Public sources about Rexmore, Corey Gallon, and chrome-agent provide supporting context for interpreting the talk, but they are not a substitute for a session transcript.

Synthesis

Synthesized Breakdown

The session explains MCP Apps as a way to add interactive UI to MCP-based workflows. It treats the MCP host as the place where tool data, model reasoning, and human controls meet.

Speaker And Company Context

Topics Covered

Derived Links And Source Material

Novel Concepts / Clever Methods

Evidence Boundary

This synthesis uses the official schedule plus cached video transcripts. Official AI Engineer World's Fair San Francisco 2026 livestreams and cut videos are primary event video sources for transcript/slide evidence; external, historical, or speaker-matched videos remain supporting context unless manually verified as exact official event recordings.