At document #1 they match. At #1,000 they don't.
Left: the agent you built on Claude. Right: the same agent running on Kolena.
ClaudeDocument #1Tenant: Koala AI, Inc.
Date of Lease: 12/07/2023
KolenaRuns| File | Landlord | Tenant | Date | |
|---|---|---|---|---|
| Multi_Tenant_Lease.pdf | Leonard Properties | Koala AI, Inc. | 12/07/2023 | ✓ |
ClaudeDocument #1,000KolenaRuns| File | Landlord | Tenant | Date | |
|---|---|---|---|---|
| Amendment_2.pdf | Leonard Properties | Koala AI, Inc. | 12/07/2023 | ✓ |
| #1,000 · Office_Lease.pdf | Leonard Properties | Koala AI, Inc. | 12/07/2023 | ✓ |
| Amendment_1.pdf | Leonard Properties | Koala AI, Inc. | 12/07/2023 | ✓ |
The paper is worse than your test set.
You tested on a clean PDF. The real stack is an old scan, a fax of a fax, a copier smear. A chat build reads what it can and quietly leaves the rest blank. No flag, no error.
THIS LEASE AGREEMENT is made as of February 14, 2024, by and between NORTHGATE COMMONS, LLC ("Landlord") and RIDGELINE PARTNERS, LLC ("Tenant").
1. PREMISES. Landlord leases to Tenant Suite 240, approximately 8,400 rentable square feet.
2. TERM. The initial term shall commence March 1, 2024 for eighty-four (84) months.
3. BASE RENT. Tenant shall pay base rent of $18,450.00 per month, subject to 3.0% annual escalation.
It “forgets” on the next batch.
Run the next batch, a different set of leases, and the numbers come back identical to the last batch. It never actually read the new documents; it echoed the old answers. No error, no flag. Just last batch's values on this batch's files.
Extraction is easy. Delivery is the hard part.
Claude can push results to your systems in any format you want. The catch is that you build and run that pipeline yourself: the integrations, the field-to-template mapping, the retries and error handling, and the upkeep every time the model or your workflow changes.
The last mile is built in.
Nothing to wire up. Results land in Drive, SharePoint, a webhook, or your warehouse, in your template, automatically on every run, with each value cited to its source.
agents.kolena.com › IntegrationsBoth start together. Only one finishes.
Build the same agent on both, then turn up the volume. The gap isn't in the build. It opens as throughput climbs. A prototype proves the concept once. Production means the ten-thousandth document is as right as the first.
Every value, cited to the page.
"Did you read the whole thing?" stops being a question. On Kolena, each extracted field links to the exact page and clause it came from. Auditable, defensible, every run.
agents.kolena.com › lease abstract › Runs › Second Amendment to Lease.pdfTHIS SECOND AMENDMENT is made as of April 18, 2025, by and between LEONARD PROPERTIES, LLC, a Nevada limited liability company ("Lessor"), and KOALA AI, INC. ("Lessee").
WHEREAS, Lessor and Lessee entered into that certain Standard Multi-Tenant Office Lease dated December 7, 2023 (the "Original Lease") for the premises at 7890 Smiley City Dr., Happy City, NV 12345;
The term shall commence December 15, 2023 and continue through the Expiration Date of December 14, 2026, unless sooner terminated as provided herein.
Renewal notice must be delivered no fewer than ninety (90) days prior to expiry per Section 12.3.
Even past every breakpoint, the ground keeps moving.
Say you cleared every breakpoint. You didn't build a tool. You signed up to run one. The model underneath drifts, gets deprecated, reprices, and goes down. None of it is in your prototype, and none of it stops.
A running agent is mostly below the waterline
Keeping it accurate in production is a full engineering discipline: the part enterprises staff at six figures, and the part a DIY build reinvents from scratch. Kolena runs it for you, and can show it.
“The Agent”
A prompt, a document, an answer. The 5% above the surface.
Model evaluation & selection
Continuously test frontier models for accuracy and cost; switch only when one clearly wins.
Model lifecycle management
Catch drift and deprecation; migrate ahead of provider changes before your output moves.
Live production monitoring
See failures the moment they happen, not three months later in an audit.
Consistent availability
Throughput that doesn't depend on one provider having a good day.
Change management & validation
Prompt version control, ground-truth validation, a citation on every field, and access control.
None of this is in the prototype. All of it is the difference between a demo and a workflow you can run a business on.
Building it yourself still needs a team.
Running this yourself isn't a line item for tokens. It's the people who build, host, evaluate, and maintain the agent as your workflows change. A recurring team cost on top of a subscription that itself scales with heavy agent-building usage.
And it never finishes onboarding: every time a workflow changes or a model updates, the work comes back.
Even if you can afford it, it's the wrong focus.
Grant it fully. An enterprise can hire, can pay, can build it. The question isn't whether you can. It's whether running an AI agent is what your team should spend its focus on. Every week and every hire spent on it is diverted from the work that makes you money, and you'd still be building competencies from scratch that a platform exists to provide.
Who should build, who should buy
This isn't “never use a chatbot.” It's about matching the tool to the stakes.
Use the assistant directly
A few documents a month. One-off analyses. A person reviewing every output. At that scale a chat tool is exactly right, and cheaper than anything else.
Run it on Kolena
Real throughput. Accuracy and timeliness that must hold. Cost and audit pressure. When the workflow is the business, you need a platform, not a prototype.
lease abstractKolena just keeps running.
Point it at the queue and walk away. It reads every page, holds accuracy, and lands results in your systems. A whole portfolio in one cycle, no one babysitting it.
AI for CRE › lease abstract › RunsSA| File | Landlord | Tenant | Date of Lease | Term Comm. | |
|---|---|---|---|---|---|
| Northgate_Commons_Lease.pdf | |||||
| Harbor_Point_Retail.pdf | |||||
| Cedar_Ridge_Office.pdf |
In production at a private-lending customer:
96% less document-review labor · five days → same-day funding.
Keep the model. Lose the babysitting.
Send us the document that broke your prototype: the 80-page lease, the bad scan, and we'll run it and show you what comes back, cited to the page.
Book a demo