I ran across an incredibly detailed and interesting article on the technology behind IBM’s Watson, the cognitive computer that outplayed several human Jeopardy champions back in early 2011. (By the way, do you know who Watson is named after? I thought I knew too, but I was wrong. I’ll give you the answer a little later).
I highly recommend the article for anyone interested in virtual agent technology, cognitive computing, or artificial intelligence. I won’t do it justice, but here’s my cliff notes version of what makes Watson work. Watson is an outgrowth of the IBM DeepQA project, where QA stands for Question Answering. IBM and other research groups founded an open source initiative called Open Advancement of Question Answering (OAQA), and OAQA continues to work on building and improving upon a common methodology and architecture for automatic question answering systems.
The article describes DeepQA as utilizing a “massively parallel probabilistic evidence-based” architecture. To break that down into something less than geek speak, DeepQA has the ability to quickly categorize and understand a question and access lots of varied data sources to pick out many instances of information that might answer the question. It can then sift through all the answer candidates to rank them in terms of which answers have the highest probability of being correct. Only when it concludes that the possible answer set has a high enough probably of being right, will it attempt to answer the question.
How DeepQA navigates its way through each of these steps involves many complex processes. It first has to analyze and understand what’s being asked. It uses techniques such as identifying the lexical answer type, which means recognizing an important word in the question or clue that points to the type of answer. For example, in the jeopardy clue “The only President not to live in the White House,” the word “President” identifies the type of answer. (The correct response, by the way, is “Who is George Washington.”) The DeepQA system analyzes the clue in detail using knowledge of specific patterns, key words, lexical answer types, semantic relationships, and many other factors. Once the system has grasped the meaning of the clue, it uses shallow and deep searching techniques to comb through copious stores of content to identify possible answers. Interestingly, some of the most complex behaviors that DeepQA performs occur after it has gathered up all the potential solutions. It gathers supporting evidence and uses sophisticated ranking algorithms to determine how likely each answer is to be the correct one. See the full article for a much more in depth view of the core technologies that power DeepQA.
These abilities will certainly influence the future of virtual agent technologies. Customer-facing virtual agent support apps are, after all, basically question answering systems. A customer asks a question and the virtual agent needs to be able to quickly provide the best possible answer. The beauty of the DeepQA and OAQA framework is that it has a much better chance of understanding the customer’s question than a more limited and simplistic pattern matching system. The OAQA architecture should also easily outperform virtual agents that search website using key words in order to respond to user inquiries.
Will the technology of DeepQA be available for customer support and enterprise virtual agents in the near future? We’ll be watching the marketplace to find out. Also see our recent post on IBM’s Watson Goes to Med School. Oh yes… and the origin of Watson’s name has nothing to do with Sherlock Holmes. This cognitive computer with a knack for solving tough puzzlers was named after the founder of IBM, Thomas J. Watson.