How the Apple Watch Might Benefit Siri

Dan Miller of Opus Research wrote a post earlier this month about how the Apple Watch is a perfect match for Siri. Miller points out that Siri has some apparent limitations on the Apple Watch, being that she doesn’t speak back to the wearer, but just listens and carries out spoken commands.

Apple WatchYet Miller views the Apple Watch as the perfect extension for speech-enabled use cases that the typical iPhone user is already accustomed to and comfortable with. iPhone users have come to depend on Siri’s reliability for controlling the clock, setting alarms, leaving reminders, or making calendar entries. Miller thinks these types of commands are closely integrated with watch functionality and that buyers of the Apple Watch will naturally use Siri to perform these operations.

Miller also points out that Siri’s range of capabilities is extensible. Siri is standing by to help execute a large number of apps and the inventory of apps with Siri integration will steadily increase.

In a post I wrote last September, I noted that the form factor of the Apple Watch might actually lead to a renewed interest in using Siri, even among the large number of iPhone users who rarely interact with Apple’s digital personal assistant (now that the novelty has long since worn off). When I saw the demo of the Apple Watch following the #SpringForward reveal, the concerns about the form factor resurfaced.

There are so many tiny app icons loaded onto the watch face that it’s difficult to tap on the app you really want to open. Wouldn’t it be a ton easier to just say “Hey Siri, open Uber” or “Hey Siri, open Facebook?” The constraints of the wearables form factor may provide a renewed raison d’etre for voice interfaces in general, and intelligent assistants in particular.

I agree with Dan Miller that Apple Watch and Siri are a natural pair. It’ll be interesting to observe how this next generation of wearables impacts the intelligent personal assistant market and whether wearables force voice interfaces to the forefront.


Nuance Wintermute and the Growing Competition in Voice Recognition

WintermuteBack in August, Parmy Olson wrote an interesting article in Forbes on some of the challenges that Nuance faces with its digital assistant Wintermute. I wrote about Wintermute in an earlier post. Olson makes the observation that Nuance rolled out its Wintermute personal assistant technology just as Google and Apple might be catching up and arguably even surpassing Nuance at its own game.

Nuance acquired Dragon Naturally Speaking software technology via a round about means from inventors James and Janet Baker. The Dragon dictation software has been a big component of Nuance’s product line.  Nuance also licensed its speech recognition technology to Apple for use in Siri. But with speech recognition becoming such a core capability for today’s smart phone apps, Google and Apple have been investing heavily in developing their own homegrown solutions.

Olson points out that Google’s voice recognition is based on deep learning technology, whereas Nuance’s approach to speech technology relies on statistical inference that analyzes syllable sounds to identify words. The jury is still out on which technical approach has the most promise, but Google’s implementation of voice recognition has been working. What’s even more threatening to competitors is the fact that Google offers its technology free to Android developers. A case in point are the recent German high school grades behind Voicesphere, which we wrote about a few weeks ago.

Apple recently established a research center in Boston where it’s been pursuing speech technology projects. Many of the team members are former employees of a speech software company that was once acquired by none other than Nuance. Observers speculate that Apple is developing its own voice recognition software that will displace the Nuance components from Siri in upcoming versions.

None of these facts proven with any certainty that Nuance is being overtaken by the competition (and current partners). Wintermute’s mission is to learn as much about your preference and habits as possible, store this knowledge in the cloud, and use that data to infer what you want and what you mean. In other words, Nuance has trained Wintermute to read your mind, which is what a really helpful digital companion needs to be able to do. So far Wintermute still seems to be more of a project than a fully fleshed out commercial offering. It’ll be interested to see how this technology pans out for Nuance as the competition in speech technology and personal digital assistants continues to heat up.

Will Cue Make Apple’s Siri Creepier In a Good Way?

CueA really good and useful personal digital assistant has to understand you. Like the proverbial butler of victorian literature, your personal assistant needs to know you and your quirks and it has to have insight into your daily routines, your current appointment calendar, your upcoming planned trips, and so forth.

Lots of folks praise Google Now over Apple’s Siri because of Google Now’s deep reach into the places where you store your information. Google Now can scan the contents of your Google email addresses and pick out information related to items you’ve purchased and flights you’ve booked. It can track packages for you or alert you about flight delays before you’ve even thought to ask about such things. This type of access to your personal data sphere is a little creepy, but it makes Google Now a more useful and capable personal assistant.

A company called Cue, formerly known as Greplin, offered a service that mimicked the victorian butler of whom we spoke earlier. Cue’s application acted as a an accumulator of important information from all of your social media identities. The app could sift through your email, Facebook, Twitter, LinkedIN and other accounts to create a unified index and a central inbox. Having all this knowledge about you in one place gave it the ability to predict what you might be looking for, based on context.

Now Apple has acquired Cue and the speculation abounds about how Apple will use the Cue predictive reasoning abilities to amp up Siri. If Siri has the ability to scrape data from our social and email treasure troves, it’ll certainly make it (her?) a more capable companion. Of course, there’s always the concern that we’re giving up some privacy in exchange for this added functionality. But can you ever really have any secrets from your butler? Probably not. Former Cue users will have to wait and see how it all turns out, since the Cue service is no longer available.

Apple’s Siri and the Unfulfilled Potential of Task Completion

SiriIn a recent Techcrunch article, Dan Kaplan laments the unfulfilled potential of Apple’s personal assistant Siri. Kaplan makes the point that the vision of the original Siri creators was for the mobile personal assistant to become  a “task-completion engine.” Siri was about disrupting traditional search and replacing it with an artificially intelligent assistant that could understand and then actually execute tasks for us. Instead of just looking up answers on the Internet and feeding them back to us, Siri’s destiny was supposed to be that of an indispensable concierge in a box.

Earlier pre-Apple versions of the assistant, Kaplan points out, could order you a cab, make restaurant reservations, or even find out what bands were playing around town. Siri’s creators envisioned even more task-completion functions, such as the ability to rebook an airline ticket for you if your flight got canceled or redirecting a package delivery to your office if it arrived at your house while you were out.

Kaplan complains that Siri has stagnated. The mobile assistant remains mired in the technology of search. Google Now has surpassed Siri in terms of its predictive powers. Google Now, for example, shows you cards with information based on tidbits it gleams from your Gmail accounts and other sources. That means that Google Now can show you the status of an airline flight you’re on without you asking, or update you on the score of a recent game played by a sports team you follow.

Kaplan’s point about the potential power of mobile task-completion is a good one. Most virtual agent technology today is focused on understanding language and intent and providing a good response. Even super sophisticated cognitive computing systems, like IBM Watson (aka DeepQA) are designed to provide answers to questions. Task-oriented virtual agents might be just around the corner, and it will be interesting to observe their impact on the marketplace. What will it mean when we have virtual agents that are not only able to converse with customers, but that can actually process customer transactions? We’re likely to find out one of these days.

Is Your Mobile Personal Assistant Spying on You?

SpyApple’s Siri, the iOS mobile personal assistant, presumably engages in thousands of conversations a day. Recent articles indicate that Apple stores the Siri voice data files from these conversations for up to two years. The data is stored in an anonymous way that doesn’t link the conversation to a specific user. At some point, the data files are purged completely. Why does Apple archive Siri conversations? Presumably to learn how people interact with the mobile personal assistant and improve its performance.

What are the implications of Apple’s policy for holding on to our conversations with Siri? First of all, for most of us, it just feels creepy to think that our conversations with a virtual software agent are being tracked and stored. Should it bother us? We all know that the text messages we exchange with friends, family, and whomever are kept on servers in the cloud. We know that these text messages could be used against us as evidence if we were ever to be charged with a crime or civil offense, but we mostly choose to ignore these possible transgressions against our privacy. What about privacy concerns when it comes to how we engage with mobile and web-based virtual agents? Are we in need of a whole new ethics model to help us deal with privacy in the age of non-human conversational partners?

I recently wrote about IBM’s DeepQA (Watson) technology and how it is being trained to analyze medical documentation and assist physicians in diagnosing patient illnesses. Intelligent cognitive computing systems that can quickly process and interpret vast amounts of structured and unstructured medical data can potentially save lives and reduce medical costs. But will patients have to sacrifice privacy to benefit from artificially intelligent systems? A virtual medical agent will undoubtedly store patient data so that it can be referred to for improving future diagnoses. If that data is not linked to a patient’s identity, the patient has little to be concerned about. What happens, though, when the conversation becomes more like a human-to-human interaction? Advances in technology may bring us a fully conversational AI as a doctor, therapist, or other trusted medical service provider. Can we be sure that the virtual agent we confide in will keep our discussions confidential? A virtual agent is not an autonomous being. It’s a piece of software technology that can be controlled by humans and corporations that may see value in the information they collect about us.

Surely the time is fast approaching when virtual conversational agents will be able to talk to us about anything, including providing companionship when we’re lonely or need an attentive ear. Do we want our conversations with surrogate virtual companions to be snatched up by advertisers so that they can target us for products? What if you spoke to a virtual therapist bot about being lonely and were bombarded a few minutes later by ads for dating sites and singles cruises?

Let’s not allow privacy concerns to curb our appetite for virtual agent technologies or stop us from pursuing research and new products in this area. We do, however, need to start thinking through the implications of the data that is generated from these human to machine interactions. How will it be handled and stored, and can we adequately protect the privacy of the people who will come to rely ever more heavily on mobile personal assistants and other virtual agents?