So Virtual Agents CAN Provide True Customer Service?!

Intelligent Virtual Agent

The future of virtual agents?

We may actually be closer than I thought to smart virtual agents that can perform complex customer service transactions. I recently watched a pretty amazing video of Interactions Corporation‘s virtual assistant Interactive Voice Response (IVR) system. I was impressed by how well the agent could understand the intent of the caller’s questions and requests. What really surprised me, though, was that the virtual agent could not only answer questions, but it could perform transactions too. Here’s a virtual agent that crossed over into the realm of true customer service rep. Who knew that this sort of advanced capability already existed?

If you’re interested in virtual agents and the future of IVR, you really need to watch this video from FinovateFall 2013. Finovate is a conference where companies showcase the latest technologies supporting financial institutions. The demo by Interactions Corporation is either one of the best rigged demos I’ve ever seen (and I guess that’s possible), or the company’s virtual agent technology goes way beyond what I thought was currently possible. There are also several audio demos available on the Interactions website.

Based on an article by Opus Research, the brains behind the Interactions smart virtual assistant are borrowed from AT&T WATSON(sm) speech and language engine. That’s right; it’s AT&T WATSON, not IBM Watson. Heh? Well, apparently AT&T named its speech processing technology WATSON a really long time ago and they’ve been using it in IVR systems ever since. They named the technology after Thomas A. Watson, the famous assistant of Alexander Graham Bell who’s name is forever memorialized in the silly utterance “Mr. Watson–come here–I want to see you.” IBM’s Watson is of course named after the first IBM President, Thomas J. Watson. So it seems there’s more than one Watson that’ll be honored by having some groundbreaking technology named after him. It does make things confusing though.

If the Interactions demo is any proof, AT&T’s WATSON seems to have come a long way. Will this technology, or others like it, become a standard feature of IVR systems and web-based virtual support agents? And what might the possibilities be if AT&T and IBM got their Watsons together?

I really like the idea of being able to call up a smart automated system that can quickly execute the transactions I need taken care of, such as booking me on a flight or transferring money from one bank account to another. Apparently the artificial intelligence exists to construct virtual assistants that can do these kinds of tasks and understand what we really want, even if we’re not that great at articulating it.

Virtual Human Toolkit – A Treasure Trove for Virtual Agent Developers

Virtual Human ToolkitThe University of Southern California’s Institute for Creative Technologies offers a Virtual Human Toolkit for constructing animated conversational characters. I ran across the Virtual Human Toolkit while browsing through the official proceedings from the 13th International Conference, IVA 2013, Edinburgh, UK, August 29-31, 2013. A team from the USC Institute for Creative Technologies wrote a paper titled “All Together Now: Introducing the Virtual Human Toolkit” that was presented at  IVA 2013.

The goal of the Virtual Human Toolkit is to provide a suite of ready-made components that developers can use to more quickly build well-rounded virtual agents. Virtual agent characters can have many benefits, but they are comprised of numerous complex technical components. Most teams don’t have access to all the knowledge and skills needed to build virtual characters with a broad range of capabilities. A versatile virtual human would ideally be able to simulate human behavior, perceive and adequately react to the behavior of others, and respond appropriately to questions or statements. Virtual humans are costly to develop, so the toolkit from USC’s Institute for Creative Technologies should be a great help to small teams looking to experiment with the technology.

Based on the documentation available, the virtual human toolkit currently consists of the following components:

  • Speech Recognition
  • Natural Language Understanding
  • Nonverbal Behavior Understanding
  • Natural Language Generation
  • Nonverbal Behavior Generation

These capabilities are embedded within individual modules that are all connected via an underlying messaging platform. A core module is called Multisense. This module enables the virtual human to track and interpret the non-verbal behavior of its human conversational partner. The virtual human can track facial expressions and body language using various input devices and then analyze the input to make generalizations about the human conversational partner’s emotional state.

The NPCEditor module understands incoming dialog and then determines an appropriate response. Currently the Virtual Human Toolkit uses chatbot-like pattern matching technology to engage in dialog. The editor does appear to have the ability to use statistical models to find the best perceived response if it encounters an utterance that doesn’t match, so this capability would put it ahead of basic pattern matching scripts.

The NonVerbal Behavior Generator helps the Virtual Human plan out its nonverbal responses, which can consist of things like nodding, gesturing with arms and hands, and so on. Other components work to synchronize behaviors associated with conversational speech, which include speech, gaze, gesturing and head movements.

In their IVA 2013 article, the Institute for Creative Technologies team suggests a number of practical applications for the Virtual Human Toolkit. Among the types of uses for the technology are: Question-Answering Characters, Virtual Listeners, Virtual Interviewers,  and Virtual Role-Players.

The toolkit is available free of charge for the academic research community and for U.S. Government uses. There’s an email address to use if you’d like to contact the team about using the toolkit for commercial purposes.

Is Intel Looking to Add Voice to Perceptual Computing?

IndisysTechcrunch ran an article last week scooping the fact that Intel acquired a Spanish natural language startup back in May of this year. The acquired company was called Indisys and they specialized in computational linguistics and virtual agent (or “intelligence assistant”) technologies. Ingrid Lunden of Techcrunch speculates that Intel will use the Indisys technology to continue building out its “perceptual computing” framework.

Perceptual computing is the term that Intel seems to have coined for software than can sense a user’s motions and gestures to control the user interface. Intel offers a perceptual computing software developer kit (SDK) that developers can use in conjunction with a special camera to create gesture-based games and other interactive software.

So how does natural language fit into the vision for perceptual computing? There’s an obvious link between gesturing and speaking. One can imagine that besides just motioning at a game to get the onscreen character to move, a player would like to be able to give verbal commands as well. Interacting with software by gesturing and talking has implications beyond gaming platforms. In her Techcrunch article, Lunden mentions that Intel has demonstrated multiple devices that showcase their “gesture and natural language recognition business.”

Now that Intel has purchased Indisys, they’ll have at least the basis for advanced language recognition and even virtual agent technologies to incorporate into their product set. It remains to be seen how perceptual computing and conversational software will intersect.

Hoaloha Robotics Building a Conversational Robot Caregiver

HoalohaRobotics Trends recently reported on an interesting conversational robot currently under development by Hoaloha Robotics. Hoaloha was founded by Tandy Trower, former robotics lead at Microsoft. The goal of the company is to make a robot that can act as a well-rounded assistant to older adults and that can be purchased from somewhere between $5,000 and $10,000. Trower made a post to the company’s blog recently that provided some insights into the status of the team’s development activities and to the overall design of their robot.

Trower writes about the fact that his team was primarily focused on building software for the assistive robot. They’d expected to be able to use hardware built by another company, but it turned out that they couldn’t find anything that met their specifications and cost constraints. As a result, they ended up building their own hardware.

As for the software, Trower’s design originally foresaw a robot that wasn’t conversational at all. He’s so skeptical of the current state of speech recognition and conversational software that he didn’t want users to talk to the robot. Instead, he wanted them to interact with it via a touchscreen. But in early trials, users kept trying to control the robot by speaking to it! So he eventually gave in.

Trower writes in his blog that he is very disappointed with the “lack of a true conversational experience”  for all speech interfaces, even those with supposedly more sophisticated search-based conversational agents such as Google Now and Siri. For the Hoaloha robot, he plans to includes several features that can help to increase its ability to interact with human dialog partners more effectively. Their robot will be able to know that someone is nearby, for example, and will have the ability track when the last conversation took place, what it was about, and the history of other conversations during the day.

Based on Trower’s discussion about the Hoaloha assistive robot, it sounds like he’s trying to develop a truly conversational companion. He’s going where most robotics manufacturers are still afraid to venture. The closest competitor to Trower’s vision that I’ve seen is Kirobo, the talking astronaut companion. I’m not sure of the range of Kirobo’s conversational abilities though, and how they compare to the Hoaloha assistant.

This is most definitely a company to keep an eye on.  It’ll be interested to see how their research progresses and when they’re able to bring a working prototype to market.