Why 78% of AI underwriting pilots stall and what accuracy has to do with it?

The pilot looks great. Then reality arrives.

Most carriers have tested AI for underwriting. A 2025 MIT report found that 95% of generative AI pilots fail to reach production. In insurance specifically, only 22% of carriers have fully deployed AI solutions, despite more than 90% reporting active exploration or testing. That is a staggering gap between intent and execution.

The usual explanations, budget constraints, legacy systems, organizational resistance, are real but incomplete. In conversations with underwriting leaders across dozens of commercial P&C carriers, we hear a simpler, more damaging pattern: underwriters stopped trusting the output.

Here is how it typically plays out. A carrier runs a 60-day proof of concept. The AI vendor processes a curated sample of submissions, clean PDFs, standard ACORD forms, well-structured loss runs. Accuracy hits 94%, maybe 96%. The executive sponsor signs off. IT schedules the production rollout.

Then the real submission flow arrives. Scanned documents with handwriting. Broker-specific Excel templates that vary across 200 agencies. Loss runs from legacy carriers formatted in ways no one anticipated. SOVs with merged cells, missing headers, and inconsistent column ordering. A single commercial property submission can include 200-plus pages of this.

Accuracy drops to 88%. Then 85%. Underwriters start checking every extracted field manually. Within three months, the tool sits unused, and the team is back to copy-paste.

This is not a hypothetical. It is the pattern we see in carriers who come to Pibit after a failed first vendor, implementations that plateaued at 95-96% accuracy, internal OCR builds that broke every time a new broker format appeared.

Why 95% accuracy is not good enough

The insurance industry has a peculiar relationship with the number 95%. Vendors use it as a badge of excellence. Procurement teams put it in RFPs as the threshold. But 95% accuracy in commercial underwriting is not a success metric. It is a failure mode.

Consider the math. A commercial lines submission workflow involves extracting data from multiple documents, validating against appetite guidelines, cross-referencing external data sources, and populating the rater. If each step in a 20-step process is 95% accurate, the probability of a fully correct end-to-end result is 0.95 raised to the 20th power, roughly 36%. Two out of three submissions contain at least one error.

Underwriters discover this within weeks. Not through formal audits, but through the slow accumulation of catches, a wrong SIC code here, a misread policy limit there, a loss run total that does not add up. Each catch erodes trust. After enough catches, the underwriter builds a personal rule: always verify the AI output. At that point, the automation has created more work, not less.

This is why 75% of insurance professionals in a recent Sedgwick survey said AI needs human oversight. They are not wrong. But the question is whether that oversight means checking everything (which destroys the ROI case) or reviewing only the exceptions (which requires the AI to know what it does not know).

The three traits of carriers who ship AI to production

The 22% of carriers who have successfully deployed AI in underwriting production share structural similarities. None of them treated accuracy as a single number on a vendor scorecard.

Trait 1: Contractual accuracy, not aspirational accuracy.

Production-grade underwriting AI requires accuracy guarantees backed by SLAs, not demo-day benchmarks. The difference between "our model achieves 95% on test data" and "we guarantee 99.9% accuracy with financial penalties for errors" is the difference between a pilot and a production system. Contractual accountability changes vendor behavior, it forces investment in validation layers, edge case handling, and continuous monitoring that demo-optimized systems skip.

Trait 2: Every data point links back to its source.

Underwriters do not need AI to be perfect. They need AI to be verifiable. When an extracted premium figure comes with a citation showing exactly which page, which table, and which cell it was pulled from, the underwriter makes a trust decision in seconds rather than minutes. This is the difference between provenance. a complete chain of evidence from source document to extracted value, and an audit trail, which often just logs that the AI made a decision without showing how.

Carriers who deploy successfully require this level of transparency from day one. Not as a compliance checkbox, but as a workflow design principle. When the AI shows its work, underwriters use it. When it does not, they route around it.

Trait 3: Human expert QA sits inside the system, not outside it.

The most common architecture for AI underwriting tools is a two-layer system: the AI extracts, and then a human reviews. The problem is that "human review" in most implementations means the underwriter, the person whose time you are trying to save.

Carriers succeeding with AI in production use a three-layer approach: AI extraction, followed by automated confidence scoring that routes low-confidence items to specialized QA analysts (not underwriters), followed by the underwriter reviewing only the final, validated output. This is the Centaur Underwriter model, the human expert augmented by AI and supported by a quality layer, not burdened by it.

The underwriter's judgment stays focused on risk assessment, pricing, and broker relationships. The data accuracy problem is solved before it reaches their desk.

What this means for your 2026 AI strategy

The market is moving fast. The vendor landscape is maturing, and carriers still running manual submission intake face a widening competitive gap.

Seventy-eight percent of brokers surveyed now say an insurer's use of technology "strongly influences" placement decisions. That number was considerably lower two years ago. Brokers are routing business to carriers who respond faster, and AI-driven submission processing is the primary driver of response speed.

But speed without accuracy is a liability. The carriers who will win broker preference in 2026 are not the ones who process submissions fastest. They are the ones who process submissions fastest and deliver data that underwriters trust without re-checking.

The distinction matters because it determines whether your AI investment creates operating leverage (more premium per underwriter, 32% GWP growth in Pibit.AI's customer base) or operating overhead (another tool underwriters work around while the subscription fee keeps hitting the P&L).

The compounding error problem nobody talks about

There is a deeper issue that the 95%-accuracy conversation misses entirely. Errors in underwriting data extraction do not just cause rework, they compound downstream.

A misclassified SIC code changes the appetite match. A wrong experience modification factor shifts the premium calculation. An incorrectly extracted loss total alters the loss ratio analysis. Each of these cascading errors looks like an underwriting judgment call, not a data entry mistake. They hide in the portfolio, surfacing months later as unexpected loss development or pricing inadequacy.

This is why deep learning approaches to document processing matter more than template-based OCR. Template-dependent systems work until the template changes. Insurance-native AI that understands document context, knowing that "Total Incurred" in a loss run means something specific regardless of where it appears on the page, maintains accuracy across the format variability that real submission flows produce.

The carriers deploying AI successfully treat accuracy not as a feature but as a foundation. Everything else, speed, efficiency gains, capacity improvement, loss run automation, depends on it.

From pilot to production: A practical checklist

If your carrier is evaluating or re-evaluating AI for underwriting, here is what to look for beyond the demo:

Ask for accuracy on your documents, not theirs. Any vendor performs well on curated test sets. Require a proof of value using your actual submission flow, the messy, multi-format, multi-broker reality your underwriters face daily.

Require provenance, not just output. Every extracted data point should link to its source location in the original document. If the vendor cannot demonstrate this in a live demo, their production system will not have it either.

Demand contractual SLAs. If the vendor will not guarantee accuracy with financial accountability, they are not confident in their own system. Production-grade accuracy is a commitment, not an aspiration.

Evaluate the QA architecture. Who reviews the AI output before it reaches the underwriter? If the answer is "the underwriter," you are buying a pre-population tool, not an automation platform. Look for systems with built-in confidence scoring and dedicated QA workflows.

Check the integration model. The fastest path to production is an API-first system that works inside your existing policy admin platform. Rip-and-replace implementations stall because they require underwriter retraining and IT migration simultaneously. Carriers on Guidewire, Duck Creek, or Insurity should expect their AI layer to integrate without workflow disruption. Data security and compliance should be verified through SOC 2 and ISO 27001 certifications, not just vendor claims.

Start narrow, prove value, then expand. Carriers who succeed typically begin with a single document type or LOB, loss runs for workers' compensation, SOVs for commercial property, demonstrate measurable improvement, and then extend across the submission workflow.