Skip to content

Case study · Zillow · 2019–2026

From demo to production

Six years building Zillow's document intelligence platform end to end: from a reusable multi-tenant foundation, to digital leases at national scale, to multimodal LLM document understanding that professionals trust with compliance paperwork, to the AI-first product for brokers and agents built on top of it.

documents a year
1M+
automated accuracy, from ~85%
98%+
teams served, every Zillow business
20+
landlord activation
2x

01

Documents are the gates

In real estate, nothing moves until a document is signed. Listing agreements, disclosures, leases, purchase contracts: documents are the lynchpin of the transaction, and executing them is what moves it forward. When the signature lands, a house gets bought, sold, or rented. Someone gets a place to live. Someone gets paid.

Discovery started there: mapping the document workflows that gate every Zillow business, the same use cases an entire industry runs through DocuSign and dotloop. E-signature. Secure document storage. Compliance for documents and signatures. And preparing fillable documents from the templates and forms that MLSs and brokerages publish.

02

The platform

Each business had been solving those use cases separately, and every vendor handoff bounced users into someone else's product: another brand, another login, a UX Zillow could not shape. The thesis: documents are too central to the experience to outsource. We took the use cases and built the internal platform that became the foundation for all real estate documents at Zillow: e-sign, secure document storage, compliance, and template-to-fillable-form preparation, native in one multi-tenant platform.

What that bought was the experience, not just the capability. A renter signs a lease without ever leaving Zillow. An agent prepares disclosures inside the tools they already work in. Every business unit shapes the flow to its own users, consumers on one side and professionals on the other, instead of inheriting a vendor's one-size-fits-all UX. The platform rolled out team by team to every Zillow business and internal team that touched a document: Rentals, Home Loans, Zillow Offers, Zillow Closing Services, Premier Agent, external real estate brokerages, plus marketing and sales, legal, recruiting, and vendor contracts, 20+ teams in all, each with a VP or C-level sponsor, a baseline, a success metric, and an adoption playbook. Recurring asks became platform capabilities, like fillable forms from MLS templates and PII redaction across every document type, or were declined when they would have forked the platform. It processes 1M+ documents a year at 95%+ platform accuracy, and it retired $2M+ in annual licensing along the way.

The platform also became the foundation for an AI-first product for the brokers and agents who run transactions on it, covered in its own section below.

03

Digital leases at national scale

Zillow Rentals was the flagship proof at scale: digital leases shipped nationally in about 12 months. I instrumented the lease funnel (create, send, sign) end to end and analyzed it in SQL, and the team iterated with experiments at every step where landlords lost momentum. The analytics showed the drop-off was at lease creation, not at signing, so the work went into creation.

The biggest piece of that work was Zillow Leases, a customized digital lease generated for this property, this landlord, this tenant and this term, carrying only the clauses that apply. The results: the digital lease experience doubled landlord activation, with Zillow Leases the largest single contributor, and market share grew from 18% to 30%.

Zillow Rental Manager marketing page: edit and sign a lease anywhere, with the lease upload, e-sign, and lease documents product surfaces
Zillow Rentals lease signing, public product page

04

Multimodal LLMs with confidence routing

The hardest test is the purchase and sale agreement: the most important contract in a residential transaction, the moment a lead becomes a transaction, and the place where a wrong field costs the most. The OCR/NLP generation of the system topped out around 85% accuracy, and a human labeling and review operation closed the gap to 98%, at roughly $2M a year in operational cost that scaled linearly with volume. The business already had its quality bar. What it did not have was the economics.

Extraction was one use case on the platform, and the expected move was a better OCR vendor. It was the wrong target. A document that is 92% right is unusable if you cannot tell which 8% is wrong. So the shipping gate became per-field confidence rather than average accuracy. Multimodal LLM document understanding changed the ceiling, but only because of how it shipped: confidence-based routing with human-in-the-loop guardrails, built in deep partnership with engineering on model selection, thresholds, and cost/latency tradeoffs.

High-confidence extractions flow straight through. Anything below threshold routes to a human reviewer. Automation alone took accuracy from ~85% to 98%+ across 1M+ documents a year, absorbing most of that review operation in the process. The reviewers did not disappear; they repositioned to the compliance-critical fields where being wrong is expensive, and on those the machine-plus-human system delivers ~100%.

The extraction engine itself was a buy, not a build: a bake-off across three vendors and an internal foundation-model proof of concept, every candidate scored per field against the same golden sets. Integration took about four weeks against a six-month build estimate, avoiding roughly $2M a year of internal engineering, and staying model-agnostic means the platform rides each frontier improvement without lock-in. The judgment layer stayed ours: the golden sets, the gates, and the routing policy. Own the evals, rent the pipeline.

Illustration: a stylized residential purchase and sale agreement with invented, generic values and fictional parties named A. Buyer and S. Seller. The extraction model draws bounding boxes around fields with confidence scores. Purchase price reads 98 percent and closing date 94 percent, both high confidence. Financing contingency reads 73 percent, below threshold, so it routes to a human review panel with an approve action. The accuracy ticker climbs from 85 percent to 98 percent and above, which is what automation achieves alone; a review stamp then appears reading approximately 100 percent on review-gated compliance fields, which is what the machine-plus-human system delivers.

Human review

Financing contingency

21 days

73% confidence, below threshold. Routed for review.

the editor layer

98%+

automated field accuracy, from an 85% baseline, measured in production

with human review

~100% on review-gated compliance fields

Illustration only: the document, the parties and the values are invented to show the routing pattern, not measured calibration.

05

The eval practice

No model rollout shipped without passing the eval practice: golden datasets drawn from millions of human-reviewed documents across 100+ document types, thresholds calibrated per field and set by the cost of a wrong answer, human labeling on compliance-critical fields and code checks on structural ones, regression gates on every model, prompt, and vendor change, and production monitoring with drift alerts on new templates and tenants. This is the system that turns AI that demos well into AI that professionals trust with compliance documents.

It is also the receipt behind the headline number. 98%+ automated is not a demo metric; it is measured, gated, and monitored in production. And on review-gated compliance fields, the machine-plus-human system reached ~100%.

The gates had teeth. A vendor update that raised average accuracy but hallucinated values into fields that were blank on the source document was caught on the golden set and held until the field-presence check shipped.

06

Guardrails and human review

The human-in-the-loop layer was a product decision, not a fallback. Compliance-critical fields always earn review; confidence thresholds decide everything else. Reviewer corrections flow back into quality monitoring, so the system gets more trustworthy with use.

The model is never the product. The evals, guardrails, and review systems that earn a professional's trust are the product. That pattern, agents draft and the human reviews, is the same one that now runs a real business.

07

The product for brokers and agents

Zillow's B2B business sells software and services to brokers and agents, from single-agent shops to brokerages with offices in every state. On top of the document platform I built the AI-first product for them, and it started in the field: I toured brokerage offices and sat with the owners, agents, office admins and transaction coordinators, mapping how a deal actually moved through a working day spread across 20-plus tools. The same picture came back from every office: no single source of truth, data re-keyed by hand between systems, extra staff hired to shepherd paperwork, and no instant answer to the question “where is this transaction?”

The product followed the workflow in the order it had to. First a source of truth: a CRM that holds the conversations, the homes, the documents and the clients. Then integrations with every tool the office already used, through APIs, so nobody had to switch on day one. Then native replacements for those tools, one at a time, as the CRM earned the right to absorb them. Then the workflow itself turned proactive and automated, surfacing the next action instead of waiting to be asked. The last step was an agentic admin pilot: the system does whatever an API or an automation can do and hands the person only what still needs a person, a call to a handyman about a repair, a call to another agent for access. AI decides, the person approves; a large vendor invoice still gets a human looking at it before it is paid.

Prototypes were tested in the field and deployed office by office, configured through gates, policies and workflow steps. Early pilots showed roughly 60% less admin time per deal and about $1M in annual savings for a mid-size brokerage. When I left, the product was out of beta and signing brokerages.

08

Results

  • 1M+ documents a year

    across the multi-tenant platform

  • ~85% to 98%+ automated accuracy

    and ~100% on review-gated compliance fields, machine plus human

  • Every Zillow business, 20+ teams

    Rentals, Home Loans, Offers, Closing Services, Premier Agent, plus internal teams, at 95%+ platform accuracy

  • 2x landlord activation

    with market share from 18% to 30%

  • ~60% less admin time per deal

    in early pilots of the AI-first workflow product

  • ~$6M a year in cost impact

    ~$2M of e-sign licensing retired by the platform, ~$2M of review operations absorbed by automation, ~$2M of engineering avoided by build-vs-buy

The platform earned its place by being boring in the best way: measured, gated, reviewed. That is what “from demo to production” means.