Triage at the Loan-Tape Boundary
Private mortgage lenders run servicing on spreadsheets and inboxes. Two jobs eat the most staff time: chasing NSF — insufficient-funds — payment failures, and managing loan renewals. Both start the same way. A loan tape lands as a CSV, the lender emails borrowers, and the borrowers reply in free-form prose. The actual work is reading those replies and deciding what happens next. It is exactly the kind of high-volume, low-glory judgment worth automating — if you can do it without acting wrongly on someone's mortgage.
What we built
We built ServiceTape, a focused, deployed slice of that workflow. You upload a loan tape, open a case, paste a borrower's email reply, and the system classifies it into structured JSON — a category (NSF, renewal, dispute, or other), an urgency, a suggested next action, and a confidence score — then records that classification as an immutable event on the case's activity log. It is the exact loop a servicing team runs by hand, with the reading-and-deciding step done by a model and the deciding-to-act step gated by a human.
The hard part: getting the model to say "I don't know"
The naive version classifies everything with false confidence. In a servicing context a confident wrong answer is worse than no answer at all — it can trigger the wrong borrower contact against a real mortgage. So the classifier is built around an abstain path: a strict JSON schema with a required confidence field, and a threshold below which the case routes to human review instead of an automated action.
That is the difference between a production LLM feature and a wrapper around a prompt. You design for the failure mode first, then build the happy path inside it. The same discipline shows up in our streaming work at demo.levelbrook.com/relay — there the failure mode is a flaky provider, handled with 503/429 retry and model fallback; here it is an ambiguous borrower reply, handled with an abstain gate. Different failure, identical instinct.
The interesting engineering is never the call to the model. It is everything that decides whether you are allowed to trust the answer.
Architecture
The correctness seams are where the work lives:
- Import. A loan-tape CSV becomes an idempotent upsert keyed on
(lender_id, loan_id), so re-importing the same tape is a no-op rather than a pile of duplicates. Every row also writes an audit record, so the import itself is traceable. - Multi-tenant by default. Every table carries a
lender_id, and queries are tenant-scoped from the start — not bolted on after the fact, which is the only way that ever works. - Event log. The case activity feed is append-only and derived from events; it is never mutated in place. "What happened to this case, and when" is always answerable from the log.
- LLM. The classify call runs entirely server-side, so the API key never reaches the team, and the model is forced to a JSON schema whose output is validated before it is trusted.
What this example is really about
ServiceTape is a single-lender demo, not a hardened multi-tenant product, and we say so plainly. But the seams that make it trustworthy — idempotent import, immutable audit, and an abstain path — are the real ones, built deliberately. Swap "borrower replies" for support tickets, insurance claims, or invoices and the playbook holds: force the model to structured output, validate before you trust, and give it a first-class way to decline. The feature that looks like a safety net is actually the product.