Virtual Humans to the Rescue

Ada and GraceForget about the Singularity for a moment. While we may be able to map our brains into software one day or benefit from machines that outsmart us, those are dreams for the future. While we’re at it, forget about humanoid robots or clever C-3PO clones. In the here and now, a technology already exists that can give us human-like beings capable of helping us in amazing ways. Virtual humans are digitally rendered human actors that have already proven their potential. The only thing holding virtual humans back is our ability to create them and learn how to leverage their possibilities.

That’s where Arno Hartholt and the team at the University of Southern California’s Institute for Creative Technologies (ICT) come in. After speaking with Hartholt recently on the phone, I sensed that he and the ICT team are on a mission to get virtual humans into the mainstream where they can work their magic. The ICT has been working on virtual humans for a decade and they’ve had great success. Sergeant Star is a virtual human that’s used by the Army to answer questions from prospective recruits. Ada and Grace are engaging twins who interact with children at Boston’s Museum of Science. Other ICT virtual humans help soldiers prepare for situations in battle, train health workers to interact effectively with patients, and teach battlefield arbitrators the skills of negotiation. But these first generations of virtual humans barely scratch the surface of the possibilities.

Hartholt indicated that a big hurdle to the progress of virtual humans is their inherent complexity. Not many people have the expertise or resources to build a complete, well-rounded virtual human. An effective virtual human has to be able to understand language and meaning, speak intelligently, perceive and react to other people, and exhibit realistic emotions and movements. To create all those capabilities from scratch would require a huge investment in time and resources.

So what did the ICT decide to do? Over the past decade, they’ve been working on a Virtual Human Toolkit. The toolkit contains all the components that a builder of virtual humans might need to construct their own fully capable character. Since users of the toolkit are able to leverage these pre-built pieces, they can focus on the personality and functions of the virtual human, instead of sweating the technical details. In our conversation, Hartholt stated the importance of including people from a broad range of disciplines and interests in the development of virtual humans. The toolkit can enable psychologists, educators, medical professionals and others to develop and experiment with virtual humans. The more people and perspectives applied to discovering the potential benefits of virtual humans, the better.

Virtual Human ToolkitOne question I pondered with Hartholt was: what makes a virtual human work? The current generation of intelligent digital assistants like Apple’s Siri, Google Now and others offer a lot of useful features. Yet while they can provide helpful answers, they aren’t truly engaging personalities. Few people develop a real connection with these assistants. So why would someone be more drawn in by a virtual human? Hartholt postulates that part of the deficiency of current mobile personal assistants is simply their lack of a physical presence. They are disembodied voices. These assistants also don’t have any real sense of who we are or how we might be feeling when we push the button that activates their digital ears. They can’t express empathy.

A convincing virtual human needs a general awareness of the person it’s interacting with. You’re not going to spend a lot of time with a pretend human if it doesn’t acknowledge any cues you give it. The ICT has developed technology that it calls SimSensei that can detect facial expressions and body language in real time. When the technology is paired with a virtual interviewer, it results in a virtual character who can engage effectively with a real person. The SimSensei character can respond convincingly to the interviewee’s smiles, body language, and voice tones, for example. The virtual interviewer can even be trained to detect stress in the subject and adjust its responses accordingly.

Now that we live in a connected world, we could surround ourselves with smart, helpful virtual humans. Imagine a world where children had a set of virtual teachers or smart virtual playmates available to interact with them whenever they were bored or had a question. They could ask as many questions as they wanted and their virtual human would never get frustrated or tell them to go watch TV. They could even turn to their virtual human for empathy and encouragement, in the sad event that they weren’t able to find these critical acts of reinforcement from the real humans in their lives. Imagine a world where an ill person could have his personal virtual human contact his doctor and send in all his relevant vital statistics and either get a prescription or make an appointment. What about a companion virtual human for a senior living alone, or a virtual human coach for someone learning a new language, or a virtual human tutor for a child struggling with math?

The possibilities for virtual humans are as limitless as their potential benefits. Now all we need to do is get busy building them.

Voicesphere Points to Personalized Digital Assistants

VoicesphereAt Techcrunch Disrupt Berlin 2013 today, two recent high school grads from Germany presented Voicesphere, an android app that let’s you add voice control to other apps. The app uses Google technology for voice recognition and a custom built natural language processing engine. While Voicesphere isn’t a virtual agent, its goal is to allow the user to voice enable common functions available from mobile devices. I could see this type of tool being expanded so that users could easily cobble together their own personal assistant that they can control with voice commands.

Some of the apps available out of the box on Voicesphere are Facebook, SoundCloud, Instagram, Dropbox, Techcrunch, and Weather apps. The duo demoed a user interface that allows you to voice enable new apps that aren’t part of the platform. What if you could set up Voicesphere so that a voice prompt could bring back your key info, like updates to your timeline, stock price updates, songs you want played, photos you’re trying to find on Instagram?

The future of virtual agent technology might lie in tools like Voicesphere that help us to integrate all the key apps and functions we use into a single voice controlled interface. We’d basically be building and then training our our personalized intelligent assistant.

Gartner Predicts Smart Machines Will Be Highly Disruptive

Gartner’s Top 10 Strategic Technology Trends for 2014 have gotten a lot of press since they were released at their recent Symposium/ITxpo 2013. In a list that was dominated by mobile and cloud technologies, I was interested to see that “Smart Machines” were included.

Gartner describes smart machines as: “contextually aware, intelligent personal assistants, smart advisors (such as IBM Watson), advanced global industrial systems and [ ] autonomous vehicles.”  They go on to predict that the “smart machine era will be the most disruptive in the history of IT.”

Based on the rest of their short trend description, I don’t have a good sense of why they believe smart machines will be so disruptive. They indicate that smart machines will continue the trend of ‘consumerization,” which I interpret to mean that they believe smart machine demand will be driven, or at least influenced, more by the consumer than by enterprise demand. I can see how that  might hold true for personal digital assistants, but not so much for IBM Watson-type cognitive computing systems or smart web-self service virtual agents. Those technologies are cost-prohibitive to consumers, at least at present.

I found this prediction to be of interest: “Gartner expects individuals will invest in, control and use their own smart machines to become more successful.” Again, that’s a pretty high-level statement that I’d like to dig into further. Perhaps we’ll see more  on this topic in future reports.

It’ll be interesting to keep an eye on how Gartner expands its research into the area smart machines and artificial intelligence as these trends impact enterprise IT.

Will Cue Make Apple’s Siri Creepier In a Good Way?

CueA really good and useful personal digital assistant has to understand you. Like the proverbial butler of victorian literature, your personal assistant needs to know you and your quirks and it has to have insight into your daily routines, your current appointment calendar, your upcoming planned trips, and so forth.

Lots of folks praise Google Now over Apple’s Siri because of Google Now’s deep reach into the places where you store your information. Google Now can scan the contents of your Google email addresses and pick out information related to items you’ve purchased and flights you’ve booked. It can track packages for you or alert you about flight delays before you’ve even thought to ask about such things. This type of access to your personal data sphere is a little creepy, but it makes Google Now a more useful and capable personal assistant.

A company called Cue, formerly known as Greplin, offered a service that mimicked the victorian butler of whom we spoke earlier. Cue’s application acted as a an accumulator of important information from all of your social media identities. The app could sift through your email, Facebook, Twitter, LinkedIN and other accounts to create a unified index and a central inbox. Having all this knowledge about you in one place gave it the ability to predict what you might be looking for, based on context.

Now Apple has acquired Cue and the speculation abounds about how Apple will use the Cue predictive reasoning abilities to amp up Siri. If Siri has the ability to scrape data from our social and email treasure troves, it’ll certainly make it (her?) a more capable companion. Of course, there’s always the concern that we’re giving up some privacy in exchange for this added functionality. But can you ever really have any secrets from your butler? Probably not. Former Cue users will have to wait and see how it all turns out, since the Cue service is no longer available.

AIIDE ’13 Brings Together AI Researchers and Game Builders

AIIDE 13The 9th Annual AAAI Conference on Artificial intelligence and Interactive Digital Entertainment (AIIDE) is coming up from October 14-15. What’s the conference all about? It’s about bringing together artificial intelligence researchers with gaming developers. Some of the topics to be covered include AI and game aesthetics, AI in the game design process, AI and narrative technologies, and musical metacreation.

I did a little reading about the conference and some of the topic areas. I was interested to find out more about the narrative technologies domain. It turns out there’s a whole line of study that focuses on how software can be taught to generate believable stories (“narrative generation”). Research has been done, for example, on the use of Monte Carlo Tree Search techniques (a kind of search algorithm) to automate the creation of interactive story lines. These types of narratives can be used to improve user experiences inside a virtual gaming environment, or even for educational or training purposes.

Chances are good that there will also be presentations related to conversational agents in games and other virtual environments. Let’s hope that the AIIDE ’13 proceedings will be published and accessible to the public. In the meantime, it looks like you can follow some of the action on Twitter under the hashtag #AIIDE13.

Spambot or Mechanical Turk? We’d Rather Have the Spambot!

Twitter SpambotBianca Bosker recently wrote an interesting piece for Huffington Post on the alleged Twitterbot @Horse_ebooks. This isn’t your typical twitterbot tale. It turns out that almost everyone believed the tweeter behind @Horse_ebooks was a spambot, and yet the innocent, uncanny cleverness and wit of the automated tweets drew in lots of followers. But then it was revealed that the bot was actually a real person, or two real people. Followers were devastated.

Bosker draws comparisons between the @Horse_ebooks scam and the mechanical Turk robot of 1770. Wolfgang von Kempelen, a Hungarian inventor, developed a turban-wearing robot that could supposedly play chess. The inventor traveled around with the mechanical Turk and delighted crowds with the machine’s game-winning prowess. For forty years or so, the robot had crowds fooled. But eventually the robot was revealed to be a hoax, having a hidden compartment where a human could operate the machine without being seen.

Bosker postulates that people are disappointed when such supposed technological advances are shown to be fake, because we want to believe that machines can transcend their limits and become more human. Bosker also sites chatbots, such as Joseph Weizenbaum’s Eliza and the AOL Instant Messenger bot SmarterChild as evidence that people are willing to be hoodwinked by machines posing as humans. She doesn’t mention dating chatroom bots that prey on lonely singles, but I suppose such date bots fall into the same category.

In another article on the people behind the @Horse_ebooks hoax, a Gawker report reveals the team’s odd behavior and other bizarre projects they were involved with that played on user expectations and emotions.

What does all this really say about the future of virtual agent technology? Does it indicate that our conversational software doesn’t really have to be all that great to make people happy? I suppose that depends on what we’re expecting. If we’re interacting with a mobile personal assistant like Siri, we want it to be adept at understanding our questions and giving us the right answers. But we really don’t expect it to have a sense of humor. If it utters anything that even hints at being funny or quirky, it delights us. So maybe robots designed to help seniors, such as those under development by Hoaloha Robotics, don’t have to be all that talented at conversing. They need to understand commands, but even the most marginal capabilities in the areas of humor and empathy could be enough to win over hearts. At least to start out with…