What Makes IBM’s Watson Tick? And Is Watson the Future of Virtual Agents?

WatsonI ran across an incredibly detailed and interesting article on the technology behind IBM’s Watson, the cognitive computer that outplayed several human Jeopardy champions back in early 2011. (By the way, do you know who Watson is named after? I thought I knew too, but I was wrong. I’ll give you the answer a little later).

I highly recommend the article for anyone interested in virtual agent technology, cognitive computing, or artificial intelligence. I won’t do it justice, but here’s my cliff notes version of what makes Watson work. Watson is an outgrowth of the IBM DeepQA project, where QA stands for Question Answering. IBM and other research groups founded an open source initiative called Open Advancement of Question Answering (OAQA), and OAQA continues to work on building and improving upon a common methodology and architecture for automatic question answering systems.

The article describes DeepQA as utilizing a “massively parallel probabilistic evidence-based” architecture. To break that down into something less than geek speak, DeepQA has the ability to quickly categorize and understand a question and access lots of varied data sources to pick out many instances of information that might answer the question. It can then sift through all the answer candidates to rank them in terms of which answers have the highest probability of being correct. Only when it concludes that the possible answer set has a high enough probably of being right, will it attempt to answer the question.

How DeepQA navigates its way through each of these steps involves many complex processes. It first has to analyze and understand what’s being asked. It uses techniques such as identifying the lexical answer type, which means recognizing an important word in the question or clue that points to the type of answer. For example, in the jeopardy clue “The only President not to live in the White House,” the word “President” identifies the type of answer. (The correct response, by the way, is “Who is George Washington.”) The DeepQA system analyzes the clue in detail using knowledge of specific patterns, key words, lexical answer types, semantic relationships, and many other factors. Once the system has grasped the meaning of the clue, it uses shallow and deep searching techniques to comb through copious stores of content to identify possible answers. Interestingly, some of the most complex behaviors that DeepQA performs occur after it has gathered up all the potential solutions. It gathers supporting evidence and uses sophisticated ranking algorithms to determine how likely each answer is to be the correct one. See the full article for a much more in depth view of the core technologies that power DeepQA.

These abilities will certainly influence the future of virtual agent technologies. Customer-facing virtual agent support apps are, after all, basically question answering systems. A customer asks a question and the virtual agent needs to be able to quickly provide the best possible answer. The beauty of the DeepQA and OAQA framework is that it has a much better chance of understanding the customer’s question than a more limited and simplistic pattern matching system. The OAQA architecture should also easily outperform virtual agents that search website using key words in order to respond to user inquiries.

Will the technology of DeepQA be available for customer support and enterprise virtual agents in the near future? We’ll be watching the marketplace to find out. Also see our recent post on IBM’s Watson Goes to Med School. Oh yes… and the origin of Watson’s name has nothing to do with Sherlock Holmes. This cognitive computer with a knack for solving tough puzzlers was named after the founder of IBM, Thomas J. Watson.

IBM’s Watson Goes to Medical School

WatsonIBM’s Watson is not only smart enough to win at Jeopardy, it’s got what it takes to be a valuable expert resource for physicians responsible for making complex healthcare decisions. According to a recent article in Robotics Trends entitled The Robot Will See You Now, the IBM Watson team has demonstrated how Watson can utilize the same skills it employed for its Jeopardy win, to now aid doctors in diagnosing illnesses and recommending treatments. These breakthroughs in the area of cognitive computing are providing humans with a new partner for helping with complex knowledge processing tasks.

Watson has enormous processing power. According to sources, Watson is capable of processing the equivalent of a million books per seconds. Combine Watson’s processing horsepower with its language processing and question analyzing and answering skills, and it can search through data and make associations far more quickly than a human can. According to the Robotics Trends article, Memorial Sloan-Kettering, a top institution in the area of cancer research and treatment, is working with IBM to teach Watson the latest information from medical research, symptom analysis, and treatment options. Medical documentation from hundreds of sources is growing exponentially. All of the data is valuable, but a human physician has no way of keeping up with the deluge of information.

That’s where Watson comes in. With its tireless ability to ingest unstructured data and understand relationships and trends within the information,  Watson can sort through whole libraries in seconds and recommend possible matches between symptoms, diagnoses, and treatments. Watson also has the ability to learn, which means that its understanding of medical literature and real life medical cases is constantly expanding. And though none of the articles discuss this point, Watson is not subject to unconscious human bias and anchoring, which can cause physicians to make a diagnosis based on only one or two symptoms, while ignoring other possibly important inputs.

An InformationWeek article provides a more in-depth look at how Memorial Sloan-Kettering is working with Watson.

In an upcoming post, I’ll write  about the technology behind Watson that’s an offshoot of IBM’s DeepQA and the Open Advancement of Question Answering (OAQA) Initiative.