Connected, Talking Toys: Striving to Exceed Expectations

ToysChildren have always imagined dolls and other toys that could talk to them like real people. The new generation of WiFi-enabled web-connected toys has the technology to get close to realizing that dream. The implications for both entertainment and education are huge.

Connected Toys Have the Technology to Hear, Understand, and Speak

The most advanced connected toys can be equipped with automatic speech recognition (ASR), natural language understanding (NLU), dialogue management, and text-to-speech (TTS) abilities. These toys can hear and understand what a child says, determine an appropriate response, and speak using a synthesized voice.

Although Amazon’s Echo product isn’t in the toy category, the Echo and its built-in voice assistant Alexa are getting many parents and their children comfortable talking to an interactive smart device. Amazon now even offers the technology powering the Echo–the Alexa Voice Service (AVS)–to third parties to embed into their own devices.

We’ve come a long way since the days of pullstring toys and Furbie. But in spite of all the technology, there are still challenges involved in realizing every child’s dream and meeting the expectations of parents. In this post I’ll address two of these challenges and potential solutions

Challenge 1: Mimicking Real-Life Conversations

Child on BeachFirst and foremost is the challenge of making connected toys fun and engaging for children. To be engaging, the toy needs to mimic real-life conversation. Unfortunately, creating a toy that can carry on completely lifelike conversations, including turn taking, remembering context, and inventing responses on the fly, is not currently possible.

Toymakers can’t anticipate what a child might want to talk about, so toys can’t be pre-programmed to respond to every imaginative remark or suggestion. To get around this constraint, toys such as Mattel’s Hello Barbie take the lead in the conversation. Hello Barbie suggests topics to discuss and games to play.

Potential Solution

Even with the doll setting the direction of the conversation, a child can still be encouraged to think and imagine. The toy can suggest topics and imaginary adventures and prompt the child for input. If developed in a skillful way, these toy-directed dialogues can be very engaging for the child, especially if the toy can understand and respond to the child’s input.

Parents will see that their child is being encouraged to imagine and even think about new things. Children will feel empowered and will enjoy getting positive reinforcement from their conversational partner.

Pullstring (formerly ToyTalk)–the company behind Hello Barbie’s dialogue–offers a platform to support any company in building conversational dialogue. There are other existing services that toymakers can leverage to build dialogue if they don’t want to create and maintain their own proprietary platform.

Challenge 2: Keeping the Content Pipeline Filled

Creating hours or even days-worth of original engaging content is time consuming, to say the least. Everything a talking toy says, all of its dialogue, every story it tells, every game it directs the child to play, must be scripted. Content is what makes a talking toy interesting and valuable. A toy that only has a few things to say will quickly be cast aside.

Since today’s talking toys are connected to the cloud, they can access databases full of content. There’s still a question of where the content will come from and whether or not parents will feel comfortable trusting it.

Potential Solution

As more talking toys enter the market, we can assume that more people will be motivated to create content for these devices. At we create interactive stories and other engaging content designed for conversational devices, including talking toys. Our goal is to widen the funnel and collect, curate, and distribute content from many diverse content contributors.

We’re just at the start of the journey to create engaging talking toys that meet all the expectations of both parents and children. But the journey will no doubt be an interesting and rewarding one.

How Do Talking Toys Measure Up So Far?

I recently gave a presentation on talking toys at SpeechTek 2016, a technology conference.  I had a Hello Barbie with me and gave a demonstration of it after connecting to a personal MiFi device. Following the demo, several audience members said they were pleasantly surprised at the positive conversational experience a child could have with the talking doll. In fact, one person admitted that prior to the demo, they had a negative impression of the product just based on press coverage.

While talking toys certainly aren’t perfect, the best connected toys can already offer children a rich and engaging experience. As those of us in the industry continue to find solutions to the inherent challenges, web-connected toys will continue to improve and exceed expectations.


This article was originally posted at

SpeechTek 2016 Offers Opportunities to Talk about Connected Toys and Conversational Robots

Next week at SpeechTek 2016 I’ll be joining others to discuss the exciting topic of connected, conversational devices.

I’m looking forward to participating on a keynote panel next Tuesday, May 24th on the topic of “Social Impact of Conversing Robots.” Peter Krogh of Jibo will moderate the panel and I’ll be joined by Leor Grebler of UCIC and Bruce Balentine of Enterprise Integration Group. Some of the areas we may cover include what people might want to talk to robots about, where the content told by robots will come from, how much robots will know about us and how that might drive the conversation, and what technological advances are needed to make robots better conversational partners.

Conversational ToysOn Wednesday, May 25th, I’ll be giving a presentation entitled “Talking Toys: Technology and Outlook.” There is a lot  going on in this field right now. Many toy makers and startups are experimenting with connected devices to both explore what these devices can offer consumers and to test out the market.

Internet-connected dolls that talk are still controversial. But there’s little doubt that conversational toys will be part of the future. There have been and will continue to be growing pains around security, privacy, and even conversational content for talking toys. We’re still in the early days of defining standards and understanding how regulations like COPPA factor into the development of safe and engaging connected devices designed for children. Even devices like the Amazon Echo, which aren’t considered “toys,” offer a glimpse into what types entertainment and educational content are possible with voice interfaces.

I’ll explore both the challenges and opportunities of talking toys in my presentation and I’m hoping for a lively discussion. If time permits, we’ll have some connected devices available to demo. We can also demo the work we’ve been up to at, where we’re building a marketplace of content designed for conversational toys and devices.

A Few Calming Words in Defense of Hello Barbie

Ever since Mattel and partner ToyTalk announced Hello Barbie, privacy advocates, parental groups, technology pundits, security gurus, and the media have been heaping vitriol upon the little doll with the push-to-talk button on her belt buckle. Now they’ve even mustered up a class action lawsuit. Based on the ferocity of the attacks, one would almost think Mattel was trying to sell toddlers a working replica of Jack the Ripper.

Hello BarbieHello BarbieI’m still trying to understand what it is about Hello Barbie that elicits such a gut-level aversion in people, because it’s clear the reaction is primarily emotional. The main complaints levied against the doll are that it violates a child’s privacy, is riddled with security holes, and is likely to crush a child’s innate creativity.

Oddly, I rarely if ever read complaints about children wearing Disney magic band tracking devices, spending hours immersed in iPad videos and apps, or having their essays scored by robo-graders.

Not that there’s anything innately bad about any of the above. We’re just living in a new world and it takes some getting used to.

Hello Barbie isn’t perfect, but it isn’t evil. Like her early precursor Chatty Cathy, Hello Barbie is Mattel’s attempt to push available technology as far as possible in the direction of giving children what they’ve always wanted: a doll that actually talks to them. It seems everyone is upset with the toymaker because this time they’ve come darn close to delivering on the promise.

Is that really such a bad thing? Youngsters lucky enough to be playing with Hello Barbie this Christmas will be of age to enter the workforce in 2027. Since there’s a high probability (99.999%?) that in 2027 employees in all sectors will need to understand how to interact seamlessly with language-capable intelligent devices, why not get an early start?

Children can learn from disappointments with Hello Barbie’s current conversational limitations and watch as future iterations of the doll mature in capability. As far as privacy concerns go, digital natives probably reveal more personal info about themselves in 20-second social media interactions than they will in hours of scripted play with a talking doll. These children will be adults in a world where they’re surrounded by recording devices and where everything they say may be searchable. As much as we may want to, we can’t stop this from happening. The best we can do is prepare our children to navigate successfully through this new digital landscape.

Are there real risks and concerns with talking toys? Absolutely. I’m more concerned about what the doll is scripted to say than I am about who’s listening to the conversation. But there’s a difference in discussing risks and blindly jumping on a bash Barbie bandwagon.

Take a deep breath and think before succumbing to the knee jerk reaction to retweet the latest alleged Hello Barbie mishap. Or put down Siri for a minute and ask Alexa to play you some relaxation music. There now. Feel better?

Trobo Storytelling Robot Toy

Last fall, the team at Skookoo, LLC ran a successful Kickstarter campaign to fund their concept of a storytelling plush toy called Trobo. After a year of work, the company is now shipping the product. Currently there are two plush toy characters named Newton and Curie available for sale. Their stories are designed to foster a love of STEM learning in children aged 2 – 5.

TroboThe Trobo concept consists of an iPad app and the talking plush toy. The iPad app contains an interactive scripted story that the child can follow. As the story progresses, the plush toy converts text to speech to narrate the story and any associated dialog. All of the Trobo stories have science or math themes. One of the first stories describes how bees make honey. The company currently offers about 5 other stories with many more in the works.

The plush toy sells for $69.99 and additional story content currently runs $4.99 for each story. Stories include interactive graphics. Trobo is more of a story narrator than a conversational toy. But Trobo’s speech technology, coupled with the scripted educational apps, is well-suited for the purpose of engaging edutainment.

I wrote about several conversational toys earlier this year and it seems that the trend towards talking toys will continue. Mattel’s controversial Hello Barbie is available on preorder now and set to ship in days. The CogniToys Dino is also available for preorder. As more of these toys ship and get into the hands of children, it will be interesting to observe what children like and perhaps dislike about them. Children the world over have always wished their toys could talk. Now that they can, the challenge is on the toy makers to ensure the toys have something interesting to say.

Hello Barbie, Chatbots, and the Challenges of Talking Toys

James Vlahos recently published an insightful article on Hello Barbie, the soon to be released talking edition of Mattel’s fashion doll. This past spring, when Hello Barbie was debuted at Toy Fair 2015 in New York, there was a flood of negative press. The primary objection to the doll, which Mattel is building in partnership with speech and AI company ToyTalk, was that the Toytalk technology represents an invasion of privacy. In a blog post I wrote back in February, I briefly addressed these concerns. Conversations children have with ToyTalk’s apps and devices are sent back to their servers for analysis and storage (just like conversations we have with Siri and GoogleNow).

Hello BarbieThough Vlahos’s article touches on the privacy issue, his main focus is on what makes the talking Barbie work. Vlahos was invited to Mattel’s Imagination Center in El Segundo, CA to observe firsthand how the newly conversational Barbie interacted with real girls from the area.

Hello Barbie, it turns out, shares a lot in common with chatbots. All of her dialog is scripted by a small team of writers, including dramatic author Sarah Wulfeck. The writers decide what words to put into Barbie’s mouth. More importantly, they must anticipate what girls will ask their doll companion.

Years ago when I built my first chatbot on the popular site Pandorabots using AIML, I didn’t have to start from scratch. I could import the open source A.L.I.C.E library into my newly created chatbot and take advantage of the fruits of many years worth of someone else’s dialog-writing labor. The ToyTalk authors started from a blank canvas. Based on Vlahos’s article, they even (re)invented a chatbot scripting language they call PullString, named after the cord you used to pull from the backs of dolls and animals to get them to talk.

But what do you have Barbie say? Guessing what children will ask and figuring out how to respond is a challenge every botmaster knows well. To make the challenge even greater, Barbie is targeted at girls from the ages of 3 to 9 and even older. That’s a wide age range to cover and a response that might work for a 5-year-old could very well backfire for a 9-year-old, and vice versa.

One way around this challenge is to create a framework that puts Barbie in charge. If Barbie can be the one leading the conversation, then the possible user inputs are narrowed. Vlahos discovered that the team of writers is trying to do just that by constructing games that Hello Barbie can play with her human friends. One example is a mock game show in which Barbie asks the girl to nominate a person as the winner of various humorous prizes.  Creating a game environment allows Hello Barbie to maintain control of the conversation.

How will Barbie’s conversational abilities hold up under real usage? That remains to be seen. Many of those who commented online about the article expressed skepticism that Hello Barbie will be engaging for children in the long term. Presumably the writing team can keep adding scripts to Barbie’s repertoire so that her topics of conversation expand over time.

A big advantage that Hello Barbie has over traditional chatbots is that the ToyTalk technology enables the doll to remember key facts from previous discussions. If Barbie can remember that Brittany’s favorite color is orange, that she loves to play soccer, and that she wants to be a paleontologist when she grows up, that should help keep Brittany engaged.

Regardless of the skepticism, the age of talking toys is upon us. Hello Barbie is scheduled to launch in time for this Christmas. Now it’s up to the creative writers and chatbot master-types of the world to prove to the skeptics that talking toys and conversational devices can be forces for good.