Ultimate’s Training Center: Introducing Our Confusion Matrix

James Longbotham, one of our Solution Consultants at Ultimate.
play Watch

Find out how our new Confusion Matrix can optimize support team workflows, keep deflection rates up, and elevate your customer experience.

Advanced technology, advanced expectations

As automated customer service enters the early mainstream phase, you should expect more from a bot than a fancily designed assortment of FAQs. In 2021, any virtual agent worth its salt can handle a returned order, shipment tracking, or faulty item claim without batting a virtual eyelash.

But as support technology has advanced, so have customer expectations.

In fact, your business’ ability to harness automated support can make or break your customer experience.

How can you make sure your support team comes up on top?

The first step is realizing that automation is a journey, not an event. There’s no setting and forgetting a virtual agent.

In conversational AI, the ability to learn and improve through training is what sets the cream of the crop apart from the rest, and it’s what will end up paying dividends for your customer relationships in the long run.

Proactive problem-solving with historical data

The most valuable source of knowledge for humans and bots alike? Mistakes and irritations.

And since troubleshooting is any customer support team’s raison d’etre, your customers’ past grievances will be your most prized possessions when it comes to proactively predicting solutions to their future problems.

In other words, your historical support data is your key to success when it comes to solving the following most common pain points in conversational AI:

  • unstructured intents, prompting
  • inaccurate replies by the bot
  • a drop in conversational quality, leading to
  • increased escalations to your human agents, and culminating in
  • a poor overall customer experience

With our new Training Center, you can capitalize on your very own customers’ messages to refine intent structures and optimize expressions through training.

Our AI will use every single incoming message as fodder to tweak and refine your virtual agent’s vocabulary, so your support technology can mature as your dataset grows in real time. Because you are using data specific to your business and your industry, the results will be tailor-made for your customers’ most important demographics.

The results?

  • higher quality conversations,
  • higher deflection rates,
  • more efficient use of your support team’s time and efforts,
  • reduced costs,
  • and improved TCO.

You can get started with the Training Center by running our CS Automation Explorer, and then continue to train your AI model using Message and Intent Training.

“My favorite experiences have been those where our customers realized just how accurately our Training Center reveals the most frequent topics their agents face daily, and how effortless, hassle-free intent training and discovery is thanks to our Training Center!”

- Marika Svennblad, Our Senior Solution Architect

Elevate your customer experience with our Confusion Matrix

Customer: I sent this order back last week. Where’s my refund?

Virtual agent: Sounds like you’re asking about the status of a return.

Customer: Yes.

Virtual agent: Please enter your order number, and I’ll look into the status of your return straight away.

The above example is business usual for a run-of-the-mill bot: It’s a straightforward, repetitive, and general request that can be solved with a basic intent structure and minimal upfront information from the customer. It works across industries and few virtual agents on the market would have any trouble handling it.


Customer: Can I return two different orders at the same time?

Virtual agent: Sounds like you’re asking about the status of a return.

Customer: Not really...

This example is slightly more complex, but certainly not uncommon. It can even be highly relevant for specific contexts and industries, such as ecommerce during peak season. In this case, the virtual agent has mixed up a question about the status of a return with a question about return policy.


So what can you do when intent confusion renders your virtual agent speechless?

Easy — upskill it at our new Training Center using the Confusion Matrix.

Available as both a list and matrix view, the Confusion Matrix leaves no room for human error when fixing intent inaccuracies:

It spots overlapping intents in real time, highlights them and suggests immediate action to solve the overlaps, or “confusions”.

In the example above, the Confusion Matrix would recognize an intent confusion between “return status” and “return policy” based on previous intent structures and a database of confidence and quality metrics. It would signal the confusion in your dashboard and suggest moving the expression “Can I return two different orders at the same time?” from intent A, “return status”, to intent B, “return policy”.

Your bot would be able to solve the problem by answering appropriately, and you’d easily prevent future escalations. Confusion solved.

“Thanks to AI-suggested training, I’ve been able to reduce my training efforts. I especially like the Confusion Matrix which shows exactly where the virtual agent can be confused and which intents are causing this confusion.”

- Lavinia Pricopie, Call Center Manager for Retail, Superbet


Find out how to elevate customer experience through more accurate AI