Virtual Humans to the Rescue

Ada and GraceForget about the Singularity for a moment. While we may be able to map our brains into software one day or benefit from machines that outsmart us, those are dreams for the future. While we’re at it, forget about humanoid robots or clever C-3PO clones. In the here and now, a technology already exists that can give us human-like beings capable of helping us in amazing ways. Virtual humans are digitally rendered human actors that have already proven their potential. The only thing holding virtual humans back is our ability to create them and learn how to leverage their possibilities.

That’s where Arno Hartholt and the team at the University of Southern California’s Institute for Creative Technologies (ICT) come in. After speaking with Hartholt recently on the phone, I sensed that he and the ICT team are on a mission to get virtual humans into the mainstream where they can work their magic. The ICT has been working on virtual humans for a decade and they’ve had great success. Sergeant Star is a virtual human that’s used by the Army to answer questions from prospective recruits. Ada and Grace are engaging twins who interact with children at Boston’s Museum of Science. Other ICT virtual humans help soldiers prepare for situations in battle, train health workers to interact effectively with patients, and teach battlefield arbitrators the skills of negotiation. But these first generations of virtual humans barely scratch the surface of the possibilities.

Hartholt indicated that a big hurdle to the progress of virtual humans is their inherent complexity. Not many people have the expertise or resources to build a complete, well-rounded virtual human. An effective virtual human has to be able to understand language and meaning, speak intelligently, perceive and react to other people, and exhibit realistic emotions and movements. To create all those capabilities from scratch would require a huge investment in time and resources.

So what did the ICT decide to do? Over the past decade, they’ve been working on a Virtual Human Toolkit. The toolkit contains all the components that a builder of virtual humans might need to construct their own fully capable character. Since users of the toolkit are able to leverage these pre-built pieces, they can focus on the personality and functions of the virtual human, instead of sweating the technical details. In our conversation, Hartholt stated the importance of including people from a broad range of disciplines and interests in the development of virtual humans. The toolkit can enable psychologists, educators, medical professionals and others to develop and experiment with virtual humans. The more people and perspectives applied to discovering the potential benefits of virtual humans, the better.

Virtual Human ToolkitOne question I pondered with Hartholt was: what makes a virtual human work? The current generation of intelligent digital assistants like Apple’s Siri, Google Now and others offer a lot of useful features. Yet while they can provide helpful answers, they aren’t truly engaging personalities. Few people develop a real connection with these assistants. So why would someone be more drawn in by a virtual human? Hartholt postulates that part of the deficiency of current mobile personal assistants is simply their lack of a physical presence. They are disembodied voices. These assistants also don’t have any real sense of who we are or how we might be feeling when we push the button that activates their digital ears. They can’t express empathy.

A convincing virtual human needs a general awareness of the person it’s interacting with. You’re not going to spend a lot of time with a pretend human if it doesn’t acknowledge any cues you give it. The ICT has developed technology that it calls SimSensei that can detect facial expressions and body language in real time. When the technology is paired with a virtual interviewer, it results in a virtual character who can engage effectively with a real person. The SimSensei character can respond convincingly to the interviewee’s smiles, body language, and voice tones, for example. The virtual interviewer can even be trained to detect stress in the subject and adjust its responses accordingly.

Now that we live in a connected world, we could surround ourselves with smart, helpful virtual humans. Imagine a world where children had a set of virtual teachers or smart virtual playmates available to interact with them whenever they were bored or had a question. They could ask as many questions as they wanted and their virtual human would never get frustrated or tell them to go watch TV. They could even turn to their virtual human for empathy and encouragement, in the sad event that they weren’t able to find these critical acts of reinforcement from the real humans in their lives. Imagine a world where an ill person could have his personal virtual human contact his doctor and send in all his relevant vital statistics and either get a prescription or make an appointment. What about a companion virtual human for a senior living alone, or a virtual human coach for someone learning a new language, or a virtual human tutor for a child struggling with math?

The possibilities for virtual humans are as limitless as their potential benefits. Now all we need to do is get busy building them.