Scraping and OCR infrastructure for the Indonesian regulatory corpus. Self-hosted document layout and vision models on owned GPUs cut the OCR budget by 92% across more than 300,000 documents.
Role Co-founder and founding engineer, Klaussa AI. Period 2026. Scope Scraping infrastructure, OCR pipeline, GPU operations.
Everything Klaussa does sits on a corpus of Indonesian regulation and court decisions, and that corpus arrives as scanned PDFs on government portals. Turning it into text is the precondition for all of it.
Doing that through a hosted document AI service at this volume is expensive enough to be a strategic problem on its own. Running document layout analysis and vision models on hardware we own brought the OCR cost down by 92 percent across more than 300,000 documents.
That is the same argument as the model training work, applied one layer down: a cost proportional to volume, replaced by a cost proportional to hardware we had already bought.
It is easy to underestimate this line item because per page prices look small. They stop looking small when the corpus is six figures of documents and many of them run to dozens of pages, and they never fall, because a page costs the same to process on the three hundred thousandth document as on the first.
The pipeline runs document layout analysis with PP-DocLayoutV3 to work out the structure of a page before any text extraction happens, then a vision language model for the extraction itself. Layout first matters more than it sounds: Indonesian regulations are full of tables, multi-column blocks, stamps and signature pages, and a text extractor with no structural model turns a table into a paragraph of scrambled numbers.
Before OCR there is a harder and less discussed problem, which is obtaining the source documents from portals that were not designed to be read in bulk.
For the Supreme Court decision corpus I built a distributed scraper writing into R2 for the documents and Supabase for the metadata, structured as a producer and consumer pipeline so that fetching and processing could stream rather than run in phases. The useful discovery during that work was an undocumented search endpoint on the portal, which turned an awkward crawl over paginated listings into direct queries.
The system is two programs that meet at storage rather than one long job, which matters because acquisition and interpretation fail for completely different reasons and at completely different rates.
The first program is the scraper. It runs as a producer and consumer pair so that fetching and processing stream past each other instead of running in phases: a producer walks the portal’s listings and pushes document references onto a queue, and a bounded pool of consumers fetches them. Documents land in Cloudflare R2, which stores files without charging for retrieval, and their metadata goes into Supabase, which is managed Postgres. The metadata row is what makes the whole thing restartable, since a document already recorded is never fetched twice.
The second program is the OCR pipeline, and it reads from storage rather than from the network. Every page goes through layout analysis with PP-DocLayoutV3 before any text is extracted, so the system knows where the tables, headers, columns and signature blocks are. Only then does a vision language model read the regions. Doing it in that order is the difference between a table becoming a table and a table becoming a paragraph of scrambled numbers, and Indonesian regulations are full of them.
Both halves are idempotent and both run on hardware we already own, which is what turns the OCR bill from a per page charge into a fixed monthly one.
The figure shows the two programs and the boundary between them. The parts worth looking at are the two failure paths drawn in red, because both were found in production rather than in testing.
Two things went wrong in ways I think are worth writing down, because they are the kind of thing that does not appear in a results section.
The scraper was too fast. Higher concurrency produced 503s from the portal, and the throughput curve is not monotonic: past a certain point you get less data per hour because you spend it being rejected. The setting that worked was four concurrent requests, which was arrived at empirically rather than by reasoning. A government portal is not a service with a published rate limit; it is a service you are a guest on, and the right posture is to find the limit gently and stay well inside it.
The OCR batch hit a database statement timeout. Processing in batches of 100,000 rows worked in development and failed at scale on a Supabase statement timeout, which is the ordinary way that a pipeline correct in every respect except batch sizing announces itself. The interesting part is not the fix, it is that the failure only exists above a volume threshold, so it cannot be found by testing on a sample.
The 16 RTX 5090 cluster is the piece people ask about, so it is worth being precise about what it does and does not give you.
It does not give you better models. It gives you a fixed monthly cost, and with a fixed cost the marginal document becomes free, which changes which experiments are worth running. Reprocessing the whole corpus with a better layout model is a decision you can make casually when the compute is already paid for and a budget approval when it is not.
The operational side is unglamorous: LoRA and QLoRA where models need adapting, mixed precision, gradient accumulation and checkpointing, data parallelism across the cluster, and enough VRAM monitoring to catch a job before it takes the node down. None of it is novel. All of it is the difference between owning hardware and having hardware.
In production. The scraping infrastructure and the OCR pipeline run continuously to keep the corpus current, since regulation that is six months stale is worse than useless in a legal product.