Macy’s On Call Offers Location-Based Intelligent Assistance

Macy's On CallMacy’s is experimenting with a new form of location-aware intelligent assistance. Leveraging technology from Satisfi and IBM Watson, Macy’s is rolling out what it calls “Macy’s On Call” in a select number of stores. The service acts like an intelligent assistant that understands the customer’s current location without having to ask.

Customers shopping in a Macy’s store that supports the On Call service can enter questions in natural language in a mobile web user interface. For example, a customer searching for a new toaster oven might ask where the kitchen appliances are located and the On Call automated assistant will direct the customer to right spot.

The new service combines two technologies: Satisfi offers a platform that can ascertain a user’s location from their smartphone data and that can respond to the user’s natural language requests. IBM Watson provides a cognitive computing platform that understands natural language inquiries and searches through complex knowledge sources to find information with the highest probability of answering a specific question.

The combination of location-based awareness, natural language understanding, and the ability to find answers about products, product locations within specific stores, inventory, pricing, etc. enables Macy’s to offer an innovative and powerful new type of intelligent assistance to its shoppers.

During the recent MobileBeat 2016 event there was lots of discussion about engaging with customers while they’re inside the store. Nichele Lindstrom, director of digital with Whole Foods, noted in a presentation that over 50% of online recipe searches happen in the grocery store aisle. Whole Foods decided to launch a Facebook Messenger chatbot to help its shoppers with recipes and other questions.

Macy’s On Call is an example of another natural language-based self-service offering that helps customers when and where they need it most: onsite at a retail location in the direct path to purchase. Now that the technology supports this type of assistance, we’re likely to see more brands extend the reach of self-service to follow customers wherever they go.

This post was originally published at Opus Research.

Niki and WhatsBot – Smart Assistants to Chat With

Virtual digital assistants that you interact with through SMS or messaging apps are all the rage. A group at Techcrunch’s London Hackathon jumped onto the texting-bot bandwagon with the creation of WhatsBot.

Text BubblesAs a hackathon creation, WhatsBot’s capabilities were understandably limited. By adding the bot’s phone number to your contact list, users could include WhatsBot in a group chat. The bot would suggest convenient meeting spots based on the locations of all the group’s members. As an added bonus, the bot would chastise users if their texts contained profanity.

WhatsApp blocked the unofficial bot assistant soon after it was launched. Apparently it violated WhatsApp’s terms of use.

A team out of Bangalore, India has launched a completely different, but also chat-based assistant called Niki. Niki isn’t integrated with popular messaging apps yet, but that seems to be on the radar. Niki assists users with purchase transactions. Users in India can currently text with Niki to ask the assistant to add data to their mobile plans or to order a cab. Niki will continue to add new capabilities.

See my full story on Niki, the smart-purchasing assistant, on Opus Research.


Baidu Enters Intelligent Assistant Race with Duer

The intelligent assistant wars continue to heat up. As reported in Tech Times and other news sources, Baidu used their World Conference on September 8th to announce the launch of a new digital intelligent assistant called Duer. Duer is a voice-driven assistant that is currently integrated into Baidu’s Android search app. The company has plans to incorporate Duer into other services and products in the future.

Baidu DuerBased on descriptions of Duer, the assistant has core capabilities in search that you would expect. But Duer appears to be very task-oriented as well. Duer can respond to spoken or text instructions to execute tasks that include: buying movie tickets, making restaurant reservations, ordering food for takeout or delivery, booking a ride with a ride sharing service, and purchasing airline tickets. Future iterations of the assistant are expected to control devices within the connected home and integrate with Baidu shopping apps.

Andrew Ng is Baidu Research’s Chief Scientist in Silicon Valley. Ng is also an associate professor at Stanford University and an expert in machine learning and deep learning. Before joining Baidu, Ng founded the Google Brain project at Google. Ng’s work in machine learning is wide-ranging, but one of his notable areas of research has been in visual learning. It appears that some of Ng’s knowledge will be applied in Duer’s ability to scan user reviews to discern answers to questions such as “is the restaurant pet friendly?” Presumably the intelligent assistant will make the conclusion that a restaurant with lots of dogs on the patio is a good place to bring Fido.

Ng is one of the rock stars of deep learning, so it will be interesting to watch the developing battle of intelligent assistants powered by artificial intelligence that is shaping up between Google, Facebook, Apple, Baidu and, to some extent, Microsoft.


Facebook M and the Future of Intelligent Assistants

There’s been a lot of press coverage about Facebook’s launch of their new Messenger app virtual assistant M. M is a human-assisted artificial intelligence. Users communicate with it primarily via texting, not by voice. The feature that differentiates Facebook’s M from Apple’s Siri, Google’s Google Now, and Microsoft’s Cortana is the system’s reliance on humans.

Facebook MessengerThe humans who assist the M algorithms are called trainers. They’re given that designation, because their job is to do what the intelligent assistant should be able to do, but doesn’t yet know how to do. Every step the trainers execute to complete a task is recorded and goes into the vast database that will become new fodder for M’s deep learning algorithms. Eventually, when someone asks M to call the DMV to set up an appointment for a driving test, M will know the steps it needs to carry out and everything it needs to say along the way to get the task completed.

M is designed to offer assistance with a broad range of activities, from suggesting and buying the perfect birthday gift for a loved one, to planning and booking a vacation, to making dinner reservations.

In a recent article about M written by Cade Metz for Wired, Facebook’s Alex Lebrun is quoted using the term “bootstrapping.” The human trainers bootstrap M by filling in for the gaps in its knowledgebase. They are only present to help the AI grow smarter, until such time as their assistance is no longer required.

Are you scared yet?

In the Wired interview, Lebrun seems to be saying that you shouldn’t be, because humans will be required into the foreseeable future to aid the AI in learning how to carry out ever more complex tasks. The other comforting factor for those who are nervous about the future of humanity: we don’t know if Lebrun’s plan will work. There’s no certainty that the trainers will follow repeatable, or easily duplicated, steps for many tasks. Recommending a great birthday gift isn’t as easy as learning how to recognize cats after watching a million cat videos.

And who’s to say that DMV employees won’t just hang up when an intelligent assistant calls to schedule your driving test appointment. Unless, that is, the DMV employee is an AI too. Hmmm.

Another question people are asking: is the model of augmenting the AI’s weaknesses with human trainers scalable? If people start to rely on M and the number of users increases, how many trainers will Facebook need to hire? And how many people want the job of intercepting people’s text messages and pretending to be their virtual assistant? Apparently enough people do. A recent article in TechCrunch lists a slew of apps that let users text “expert shoppers” who make recommendations and purchases on a user’s behalf.

The strategy that Facebook is taking with M signals that a couple of trends have staying power. Firstly, texting is becoming ever more entrenched as the preferred way to communicate using mobile devices. Will wearables change that, forcing a shift to voice? That remains to be seen. Secondly, the novelty of mobile personal assistants that can tell us the weather, do math, recite facts from Wikipedia, and offer the occasional joke has passed. People want assistants to do more for them and pure AI isn’t up to the challenge yet.

Will Facebook’s experiment be successful? If it is, the more important question might be: what does it mean for the future of intelligent assistants?

Microsoft’s XiaoIce Chatbot – What Does It Mean for Our Future?

XiaoiceMicrosoft’s chatbot experiment XiaoIce (meaning “Little Ice” and pronounced Shao-ice) has garnered lots of
media attention
recently. There’s been speculation about why so many Chinese mobile device and social media users seem enthralled with Cortana’s more chatty “younger sibling.” Microsoft introduced the social conversational assistant exclusively in China over a year ago for Mandarin-speaking users.  It’s interesting that Microsoft chose to introduce the XiaoIce technology in China only.

Technology observers are interested in what capabilities XiaoIce has that make it/her an engaging conversational partner. According to a Microsoft blog post that provides a brief description of XiaoIce, and other news sources, there are at least three features that give XiaoIce a major advantage over your average chatbot. XiaoIce has the ability to:

  • Use Bing to mine real conversations to populate a database of question and answer combinations
  • Apply sentiment analysis tools to understand a person’s mood and adjust her communication style accordingly
  • Remember key facts from past conversations to provide continuity to interactions

Some of the media attention seems to poke fun at XiaoIce’s users. Others are concerned about potential downsides of people developing relationships with virtual assistants. A New York Times article cites MIT social scientist Sherry Turkle’s concerns. Turkle observes that “children are learning that it’s safer to talk to a computer than to another human.”

We may be too quick to write off the value and potential good that applications like XiaoIce can provide. Human beings need reassurance. They need to hear that they’re ok and that someone cares about what they’re going through, even if that someone is a software-driven chatbot.

And yes, people may confide in machines more readily than they would in other humans. Machines aren’t as likely to judge, criticize, or pressure with unwanted advice. Why do people love their pets so much? Unconditional acceptance. Michael Schulson, in an article that covers both the sad fate of HitchBOT and the kerfuffle over XiaoIce, also makes the pet comparison. We anthropomorphize pets, but nobody seems to think this impacts our ability to interact with other humans.

We’ll almost certainly have lots of opportunity to figure out how smart chatbots fit into our lives and what benefits and downsides they bring. XiaoIce may or may not conquer China’s mobile users, but smart conversational chatbots will eventually spread across the globe. 

Crystal – Intelligent Assistant for Better Emails

In my last post I alluded to the promise of intelligent assistants in the enterprise. I was envisioning a world in which natural language understanding combines algorithms and other technologies to give office workers the perfect personal assistant. Our assistants will be omnipresent with data, reminders, insights, and coaching tips to ramp up our effectiveness and propel our careers to the next level.

Crystal KnowsJust this week I stumbled upon an application that might be added to the future enterprise assistant’s repertoire. While listening to a podcast from Note to Self, I heard about an application called Crystal. Crystal is designed to help you write more effective emails by coaching you to tailor your email message for your intended recipient.

How does Crystal work? Based on the Note to Self broadcast, Crystal tries to build a profile of the recipient by searching for samples of their writing in their LinkedIn profile, tweets, and other publically available social media content. Crystal then applies an algorithm to this content to determine the intended recipient’s most likely personality style, based on the DISC personality model.

Using the DISC profile, the Crystal algorithm derives a plausible model of the recipient’s preferred communication style. It’s then able to coach you in drafting an effective email message. According to the Crystal website, the coaching solution is meant to help you communicate with empathy.

The demo on the Crystal website shows suggestions for emailing with Mark Cuban, who apparently prefers short and direct communications. As you write the email, Crystal actually recommends rephrasings that will help you get your point across more effectively to your audience. When emailing Cuban, Crystal suggests replacing longer sentences such as “I am afraid I won’t be able to make it this Friday,” with a shorter, more to the point sentence along the lines of “I won’t be able to make it this Friday.”

If you’re inclined to hem and haw about a possible new date for the meeting, Crystal coaches you to suggest a specific date for Mark to accept or decline, such as “Can we reschedule for next Tuesday?” By the time you’re finished writing your message, you feel confident that you have a good shot at connecting with Mark on his level and in the way he prefers.

I’ve had training on DISC personality styles, including how best to communicate with people depending on their particular personality profile. I’ve found it almost impossible to keep all this in mind when the time comes to actually talk or email people. My guess is that I slip back automatically into the communication style that I prefer.

If Crystal can truly help us improve the effectiveness of our communications, it’ll be a great addition to the enterprise productivity toolset of our future personal intelligent “enterprise” assistant. The Note to Self podcast brought up the topic of privacy concerns, as well as the worry that leveraging personality insights to tailor communications could lead to manipulation. These are valid concerns that need to be addressed. But Crystal provides a glimpse into the powerful assistance that intelligent software could soon provide those of us forced to navigate the slippery slopes within the enterprise.

SoundHound Now Provides Intelligent Voice-Driven App Solutions

SoundHound SoundHound, the company behind the music recognition app of the same name, recently unveiled enhanced features under the name Hound that bring it squarely into the intelligent personal assistant camp.

A Techcrunch article from last week quotes founder and CEO Keyvan Mohajer as saying the company was always working towards a much larger vision than just providing a music recognition app. The Hound engine performs both speech recognition and natural language processing simultaneously in real-time, instead of separating them into different tasks. This is a technological advancement that enables Hound to return query responses very rapidly.

Based on the demo, the Hound engine can also process complex queries. An example is: “show me pet friendly hotels in Chicago under $300 a night with 3 or more stars excluding bed and breakfasts.” That’s pretty impressive.

The Hound assistant is available as an invitation-only beta Android app and an iOS version is in the works.

Perhaps even more interesting is that SoundHound is making its technology available for developers and app owners that want to “houndify” their apps. Just within the last few weeks I’ve written about MindMeld from Expect Labs and IBM’s Bluemix, both of which offer platforms and tools for voice-enabling apps. It seems there’s a real trend afoot.

The Houndify solution advertises itself as a provider of the full spectrum of services for creating voice-driven apps, including the same fast speech recognition and natural language processing engine that supports the Hound assistant. The Houndify website indicates that all operating platforms are supported: iOS, Android, Windows, Unix, Raspberry Pi, and others. You need an invitation code to create a Houndify developer account and the website doesn’t currently list any pricing information.

Will SoundHound’s Hound succeed as a voice-driven intelligent personal assistant? Will Houndify thrive as a platform for voice-driven apps? Both markets are certainly filled with opportunity. Now it looks like there’s yet another dog in the hunt. (Sorry. Couldn’t resist).