Each year since 1991, botmasters from around the world submit their chatbots to compete for the coveted Loebner Prize. I’ve written a lot recently about customer-facing intelligent assistants used by enterprises to augment call centers, improve web search, and perform routine business processes. Chatbots rely on some of the same concepts as enterprise intelligent assistants. They are conversational and respond to questions by leveraging a database of matching responses. But chatbots are fundamentally different.
The sole purpose of chatbots is to engage so effectively in small talk, that their human dialog partners don’t realize they’re speaking with software programs. During a presentation at this year’s SpeechTek, Dr. Richard Wallace of Pandorabots expressed the opinion that creative types–especially those with a background in the humanities and storytelling–create the most convincing chatbots. Being a successful botmaster is more about creating a character, apparently, than about programming. But I bet programming is still required. It’s not easy to pass the first round of testing, and even a convincing character isn’t likely to score well on the no-nonsense test.
This year 20 chatbots were entered into the contest and a fews weeks ago, the top 4 entrants were announced. These 4 will move on to the actual Turing Test at Bletchley Park on November 15th. Each entrant was asked 20 questions. Here’s a sample:
- The car couldn’t fit into the parking space because it was too small. What was too small?
- Which drink do you prefer, coffee, tea, or hot chocolate?
- What do I do with a spade?
- How many letters are there in perambulate?
As you can see, these aren’t normal chitchat questions. They’re questions designed to trip up a chatbot. Amazingly, the top chatbots were able to answer most of these questions convincingly. You can look at all the actual transcripts from each entrant’s test by going to the official AISB (The Society for the Study of Artificial Intelligence and Simulation Behavior) website. It’s interesting to see how the chatbots responded. The top 4 bots and their creators for 2014 are:
If you’re not easily offended by crude language, you can also check out the transcript of Trollbot’s responses to the questions. They’re not exactly accurate, but they’re highly amusing. Trollbot is a classic chatbot, in that he/it relies completely on his personality to fake it through the conversation. He doesn’t mince words!
It’ll be interesting to see which chatbot wins the Loebner Prize competition this year, and whether another bot can “pass” the Turing Test. In the meantime, if you’re an aspiring botmaster, take a look at my earlier post on what classic mistakes to avoid when scripting your bot’s answers.