An open source web scraping agent that drives a real browser from a plain language instruction and returns validated JSON. Built as a deliberately small alternative to browser-use, with the browser exposed to the model through seven tools over the Model Context Protocol.
Role Sole author. Period August 2025. Scope Browser automation, agent design, tool protocol.
Miss Scraper is a public web scraping agent. You tell it in plain language what you want from a website, a language model drives a real Chrome browser to go and find it, and the answer comes back as JSON that has been validated against a schema rather than as prose you then have to parse.
The point of the project is not that an agent can browse. Several projects do that. The point is how little code it takes once the browser is exposed properly: 1,257 lines of Python in total, of which the tool layer the model actually talks to is 227. The README makes that comparison explicitly against browser-use, the project it borrows from and argues with. Whether a small implementation is the right trade is a real question, and this repository is my answer to it rather than a proof.
Two servers run side by side. The first is an MCP server. MCP, the Model Context Protocol, is the emerging standard for how a language model calls tools that live outside itself, and here it exposes seven of them: navigate, click, type, scroll, go back, fetch the page as markdown, and extract structured content. The second is a FastAPI service holding the agent, so an application can post a sentence to one endpoint and receive both a written answer and the extracted rows.
Browsers are pooled per session. Each MCP session identifier gets its own browser, reused across calls so a multi step task keeps its cookies and history, and a sweeper closes anything idle. The idle window is twenty seconds, short enough that a slow model turn can outlive its own browser, and an index error from the pool is one of two crashes listed as open in the repository.
The driver underneath is zendriver, which speaks to Chrome over its debugging protocol rather than through a conventional automation framework, so it presents fewer of the signals that bot detection systems look for. Page text is converted to clean markdown by crawl4ai before the model ever sees it, which strips navigation, advertising and boilerplate.
The one decision worth transplanting into another project is the separation of browser state from page content.
Every tool returns the same small object: the current URL and a numbered list of the elements you can interact with. It does not return the page. The model clicks element 4 or types into element 7, and only calls for the page body when it has decided it wants to read something. A browsing agent that ships the whole page back on every action spends most of its context window re-reading text it already saw, and the cost of a task grows with the number of clicks rather than with the amount of information actually needed.
There are two agents for the same reason. A navigation agent holds the conversation and the history, and a separate, stateless extraction agent is handed one page and one schema and asked only to fill it in. Extraction never pollutes the navigation history, and navigation never distracts the extractor.
Extraction takes a field specification, one entry per field with a type, a description, whether it is required, and constraints such as permitted values or numeric bounds. That specification is compiled at request time into a Pydantic model, the validation layer Python developers use to enforce the shape of data, and the extraction agent is then constrained to produce output that satisfies it.
The effect is that the caller declares what a record looks like and the system guarantees the shape, rather than the caller receiving whatever the model felt like emitting and writing defensive parsing around it.
The two processes start separately. The MCP server is FastMCP speaking streamable HTTP on port 8000, mounting one tool module; the agent service is a FastAPI application on 8080 that opens its connection to that endpoint once in the startup lifespan and holds it for the life of the process, with the tool call timeout read from the environment.
A request arrives as a sentence and an optional session identifier at POST /api/v1/chat. The navigation agent, Gemini 2.5 Flash under Agno, is the only stateful component: its turns are written to a SQLite table and replayed into the next prompt. Its tool calls travel to the MCP server, where every browser tool resolves the session identifier to a tab from a pool, so two callers never share a browser and one caller keeps its cookies across a multi step task. A background task sweeps the pool every twenty seconds and closes anything idle for twenty seconds.
Each navigation tool then follows the same order: act, wait for the network to go quiet by counting request and response events over the debugging protocol, sleep for a configured settle, re-inject the interactivity script, and return the compact state.
Extraction is the tool that breaks the pattern. It converts the page to markdown with crawl4ai, prepends the page metadata, compiles the field specification into a Pydantic response model, and constructs a fresh extraction agent with storage set to None for that one call. The validated rows are attached to the tab and returned, and the FastAPI layer walks the message history backwards to find that tool result and lift it into the response body.
Look for the dashed return path in the upper row: it carries browser state back to the navigator, and it never carries the page.
The hardest single piece of the system is the JavaScript that decides which elements on a page are genuinely interactive. That file is 1,396 lines and it comes from browser-use. It is credited in the README, and I want to credit it here too, because a claim of minimalism that quietly excludes the largest file in the repository is not a claim worth making.
Two other things deserve saying plainly. The README advertises stealth against named commercial bot detection systems and near total content completeness. Neither is measured anywhere in the repository; the only supporting artefact is a screenshot of a bot detection test page. Those are aspirations written in the voice of results, and I would write them differently today.
The test suite, meanwhile, is one file containing one asynchronous test that drives the agent through three instructions and asserts nothing about what comes back. For a project whose selling point is a small, readable core, that is the gap I would close first.
A working prototype, public, with no commits since 12 August 2025. All seven tools are implemented and the extraction path produces validated records end to end. It is not packaged for installation, has no continuous integration, and carries two crashes in its own issue list: a DOM tree that sometimes fails to build, and an index error when the pool returns a browser with no tab. Treat it as a reference implementation of an idea rather than a dependency.