How can teams avoid acceptance disputes on AI projects?

Acceptance criteria must be defined before the project starts, and they cannot be vague phrases like "features completed and tests passed." Be specific: accuracy threshold, recall floor, response time ceiling, fallback behavior on errors. Define three tiers: P0 (must-pass, no acceptance without it), P1 (should-pass, deviation acceptable), P2 (nice-to-have). This aligns expectations on both sides from day one.

How do you run an effective AI PoC?

A PoC is not about "building something that works" — it is about answering the riskiest question at minimal cost. Before starting, ask: what is the biggest uncertainty in this project? Model accuracy, data availability, or user adoption? The PoC should validate only that one assumption; assume everything else holds. A PoC that tries to validate three assumptions simultaneously will likely validate none.

Why Both Clients and Vendors Feel Shortchanged After AI Projects

Q: Why do both clients and vendors feel shortchanged after AI project delivery?

The core mismatch: clients buy "outcomes" (business metric improvements), while vendors contract to deliver "features" (passing acceptance tests). The client expected a customer service AI to reduce complaint rates; the vendor delivered an API that answers questions. Those are different things. The fix: define "what good looks like" as quantifiable business metrics at project kickoff, not as a feature checklist.

Clients feel "it is not worth the price" while vendors feel "they changed too many requirements." The root cause: clients buy business outcomes, vendors deliver features. This article examines the gap from both sides and offers three preventive actions: PoC validation, quantifiable criteria, and warranty period. [See the acceptance criteria template →]

An Interesting Observation

At project acceptance, client reactions typically fall into three categories:

A: Satisfied sign-off, looking forward to continued collaboration. B: Reluctant sign-off, thinking “it is okay, but not worth the price.” C: No sign-off, disputes, protracted negotiations.

A is rare. B and C are common. Why?

The Core Conflict: Clients Buy Outcomes; Vendors Deliver Features

This is the root cause of nearly every AI project dispute.

Client expectation: “I spend ¥200K on an AI customer service system. It replaces 3 agents and saves me ¥40K per month.” Vendor delivery: “We built an AI customer service system with knowledge Q&A, multi-turn dialogue, and ticket routing. It passes acceptance testing.”

The client sees “features work” during acceptance but realizes “it is nowhere close to replacing agents.”

The vendor did not cut corners. The gap between “the system can answer 80% of questions” and “the system can reliably replace customer service agents” is a wide river — knowledge base coverage, answer accuracy, edge case handling, user satisfaction. None of these were quantified in the contract.

Lesson: Before the project starts, both sides must agree on one thing — what does “done” mean? “Code works” or “metrics are met”?

The Vendor Perspective: “Too Many Requirement Changes”

Vendors have their own pain. Mid-project, the client says:

“Can we add an export report feature?” “Can the model be more accurate?” “Can we launch two weeks earlier?”

Each seems like a “small request,” but together they blow up the timeline and budget. The vendor grudgingly agrees to maintain the relationship. The result: delayed delivery, over budget, both sides unhappy.

The fix is a “requirement change rule”:

Define MVP scope at project kickoff — core features in the contract, everything else labeled “Phase 2”
Every new request goes through a formal change process — assess effort, both sides confirm impact (delay or price adjustment)
Adding a feature means removing another — keep total scope constant

The Client Perspective: “If I Had Known It Was This Complex, I Would Not Have Started”

Clients have blind spots too. Two are most common:

Underestimating data. They think AI projects are about “having the model.” In reality, data cleaning, labeling, and validation often exceed model development effort. A RAG project spends 70% of the time on knowledge base preparation and only 30% on the model and code.

Underestimating ongoing operations. Going live is not the finish line — it is the starting line. Models need continuous evaluation and fine-tuning. Knowledge bases need continuous updates. User feedback needs continuous tracking. Many clients think “after acceptance, we are done.” Three months later, model accuracy drops by half and no one knows why.

Three Preventive Actions for a Win-Win Project

1. Run a PoC Before Full Development

Spend 1-2 weeks and ¥20-30K on a minimal viable prototype. Run the core scenarios together. A PoC exposes 80% of requirement misalignment — much cheaper than discovering them during formal development.

2. Make Acceptance Criteria Quantifiable

Do not say “the model should answer accurately.” Say “accuracy on the test set ≥ 90%, P99 response time ≤ 3 seconds.” Measurable criteria give both sides objective ground for acceptance — no more “I think it is not good enough, and you think it is done.”

3. Define a Post-Launch Warranty Period

First week: vendor responds within 4 hours. First month: weekly health check. After three months: SLA-based response. Put the warranty period in the contract. Both sides know what to expect.

Related reading:

How to Find a Reliable Software Development Partner — a buyer’s guide to evaluating technical teams
Why AI Projects Never Go Live
AI for SMEs: Three Cases and Three Ideas
Capacity Planning & Performance Testing