AI Customer Support Agent Development Guide

How much can an AI customer support agent deflect?

This guide covers AI customer support agent development: building an agent that answers tickets from your own documentation, calls tools where it must, and hands over to a person when it should. It runs through realistic deflection rates, ticket types, retrieval, handover, measurement, run cost and rollout order. Deflection comes first because it sets the case for everything after it.

Public evidence supports a wide band, not one number. Deflection depends on your ticket mix and documentation quality, and published figures usually come from best-case queues. Plan per category and treat any blended percentage as a hypothesis.

What do public numbers actually show?

Klarna announced in February 2024 that its AI assistant handled two-thirds of customer service chats in its first month. Klarna's chats skew towards repetitive payments and order questions, which is the easy end of the spectrum.

Bloomberg later reported that Klarna's chief executive said cost-cutting had hurt quality and the company was rehiring human agents. Both facts matter. High automation is achievable on narrow queues, and over-rotating on it costs trust.

Why does a blended percentage mislead?

A single "40% deflection" figure hides a distribution. Password and shipping-status tickets may deflect at high rates while billing disputes deflect at almost none. If your queue is mostly the second kind, a vendor's headline number tells you nothing. We have no deployed per-category figures to publish here, so this guide gives you the method to produce your own.

Which ticket types deflect, and which never will?

Tickets that have one correct, documented answer and need no account action deflect well. Tickets involving money, judgement, emotion or system state do not. Sort your last 90 days of tickets into these buckets before writing any code.

The list below is a planning heuristic drawn from how retrieval-based agents behave, not measured data.

• How-to and feature questions (high): the answer lives in your docs, and the risk is low if the agent is wrong.
• Policy lookups such as shipping and returns windows (high): there is a single source of truth, and it is easy to cite.
• Order or shipment status (medium to high): this needs an API call, not just retrieval.
• Account changes and password resets (medium): these need authenticated tool use and an audit trail.
• Refunds and billing disputes (low): they involve money, exceptions and liability.
• Bug reports and outages (low): they need triage and engineering context.
• Complaints and churn risk (never): the customer wants a person.

Status and account tickets are not retrieval problems. They need the agent to call your order system or identity provider safely, ideally with read-only access first and narrowly scoped credentials. That moves the work into integration, which our AI agent development practice treats as a separate workstream from the knowledge base.

AI customer support agent development starts with retrieval over your docs

Most of the effort in AI customer support agent development goes into content and retrieval, not prompts. If the right passage is not retrieved, the model cannot answer correctly, however good it is.

Why is document preparation the first job?

Help centres contain duplicates, stale articles, and policies contradicted by internal macros. The agent will quote whichever it finds. Audit sources, assign an owner and a last-reviewed date to each, and retire conflicting versions. Scanned PDFs and tables need proper extraction; our document AI work covers that step.

How do you make retrieval good enough?

Retrieval-augmented generation, introduced in a 2020 paper by Lewis and colleagues on arXiv, pairs a retriever with a generator so answers are grounded in fetched text. In practice, chunking and ranking decide quality.

Anthropic's contextual retrieval write-up reported that contextual embeddings alone cut failed retrievals by 35%. Adding contextual BM25 raised that to 49%, and adding reranking on top raised it to 67%. Those are Anthropic's benchmark results on its test sets, not a promise for your corpus.

Build a labelled set of 100 to 200 real questions with the correct source passage, and score retrieval before you score answers. Re-run it whenever you change chunking, embeddings or the document set, so regressions show up before customers see them.

Handover design and escalation rules

Handover is where customers decide whether the agent helped or trapped them. A good design escalates early, passes full context, and never makes the customer repeat themselves.

When should the agent escalate?

Escalate on any of these triggers:

• an explicit request for a person
• low retrieval confidence
• sentiment signals such as frustration
• any money or legal topic
• two failed attempts at the same question

Refusing to answer is a valid outcome. Gartner's July 2024 survey found 64% of customers would prefer companies did not use AI for customer service. A visible route to a person is therefore part of the product, not a fallback.

Accountability matters here too. Ars Technica's report on the Air Canada ruling is a reminder that a company is responsible for what its agent says.

What should the human receive?

Pass the transcript, the retrieved sources, the agent's proposed answer and the customer's identified account. The agent becomes a first-line drafter.

A National Bureau of Economic Research study by Brynjolfsson, Li and Raymond (2023) found an AI assistant raised issues resolved per hour by 14% on average and 34% for novice agents. That is assist, not deflection, but it shows value in the handover itself.

How do you measure deflection honestly?

Measure whether the customer's problem was solved, not whether the chat ended. Use repeat-contact windows, per-category rates and sampled transcript review. Raw containment overstates success because frustrated customers also leave.

Which metrics hold up?

• Resolved deflection: no new ticket or call on the same issue within seven days.
• Containment: the conversation never reached a human. Useful, but inflated by abandonment.
• Escalation quality: share of handovers where the human needed no clarification.
• Wrong-answer rate: from weekly human review of a random sample, split by category.

Ticket volume also moves for reasons unrelated to the agent: releases, outages, seasonality. Hold out a slice of traffic, or compare matched weeks, before claiming savings. Any support ticket deflection AI project that skips this ends up crediting the agent for a quiet month. Report deflection per category with sample sizes, and keep the sampled-transcript review running permanently.

What does it cost to build and run per 10k tickets?

Running cost scales with tokens per ticket and is small next to build and maintenance effort. We do not publish Viithiisys prices; scope is fixed after a discovery call. Here is how to model the run cost yourself.

How do you estimate run cost for a RAG support bot?

Assume each ticket sends 4,000 input tokens (system prompt, conversation, retrieved passages) and receives 500 output tokens.

Ten thousand tickets is then roughly 40 million input tokens and 5 million output tokens. Multiply each by your provider's current per-million-token rate from its pricing page.

Agents that loop through several tool calls multiply the input figure, so measure real traces rather than guessing. Prompt caching, documented by Anthropic, reduces the cost of repeated prompt prefixes.

What else belongs in a RAG support bot cost model?

• Vector store and embedding refresh when documents change.
• Observability and evaluation runs, which are a recurring cost.
• Human review time for the weekly transcript sample.
• Content ownership: someone must keep the docs correct.

In our experience content upkeep is often the largest ongoing line, and it is a headcount question, not an infrastructure one. Your own numbers may differ, so put a named owner and a budget against it from the start.

Rollout sequence for AI customer support agent development

Roll out in stages that earn trust: shadow mode, agent-assist, then limited customer-facing deflection on proven categories. Expanding only when measured quality holds protects both customers and the internal team's confidence.

1. Shadow mode. The agent drafts answers; nothing reaches customers. Score against what humans actually sent.
2. Agent-assist. Humans see drafts and sources, and edit or send. This builds your evaluation set for free.
3. One category live. Pick a high-potential category such as how-to questions, with a clear escape hatch.
4. Expand by evidence. Add categories only when wrong-answer and repeat-contact rates are acceptable.

Keep the first release narrow: one channel, one language, a few categories. A fixed-scope build like our Moonship MVP fits this shape. Narrow releases are easier to evaluate and to roll back, which is why we recommend them, and broad ones tend to stall while nobody can tell which part is failing.

For wider context on where an agent fits across your operations, see our AI development services and the solutions we build around helpdesk and workflow automation. Teams in the US, UK and Canada work with our engineers in Mohali and our office in Markham, Ontario.

Next step

The pattern across this guide is consistent. Plan deflection per category, invest in documents and retrieval before prompts, design the handover first, and measure resolved outcomes against a control. If you want help working out which of your tickets are worth automating, get a free workflow audit and we will map your queue against the categories above.