Handwritten textile packing lists in, structured rows out — with two models checking each other, because one was never going to be trusted.
Every dispatch produced a handwritten packing list. Someone then retyped it into the ERP — roll numbers, colours, quantities, buyer — line by line, every day. It was the single largest manual cost in the operation and the single largest source of bad inventory data, because a typo at that step propagated into stock, orders and invoicing.
The obvious answer was OCR. The obvious answer did not work. Mixed Hindi and English handwriting on creased carbon-copy paper defeated every single-model pipeline we tried — not by failing loudly, which would have been fine, but by returning confident, plausible, wrong numbers.
A single vision model was cheaper and simpler, and it was good enough that the demo looked finished. It was not good enough that anyone would stop checking its output. A tool people double-check has saved nobody any time — it has just moved the work from typing to proofreading. I killed it, because the whole value of the module was people trusting it enough to walk away.
GPT-4 Vision and Claude read the same document independently and their structured output is compared field by field. Agreement is accepted. Disagreement becomes a review item with both readings shown. This does not make either model better; it makes their failures visible, which is the only property that mattered.
Extraction runs as a BullMQ job rather than in the request. The operator photographs the sheet and carries on; results land in a review queue. Making it synchronous would have tied the warehouse's pace to inference latency and to whether an upstream API was having a good afternoon.
Both models return high confidence on text they have hallucinated. Per-field confidence scores were useless as a trust signal on their own — cross-model agreement turned out to be the only reliable one.
Carbon copies, folds, smudges, and sheets photographed at an angle in bad warehouse light. A meaningful share of the engineering was normalising the input before any model saw it.
The first month ran with full human review on every document even where the models agreed, purely to build the evidence that agreement meant correct. Skipping that would have saved a month and cost the rollout.
The cheapest model that people do not trust costs more than the expensive one they do.
Design for the failure you can see. Two models disagreeing is a workflow; one model being wrong is a silent liability.
Automation that removes 90% of a task and leaves the rest visible beats automation that claims 100% and hides its mistakes.