CNN recently ran a story about research on a smartphone app that can predict depression. The Purple Robot app was created by researchers at Northwestern University in Illinois. The app leverages data about a user’s location, movement, phone usage, and other activities.
The goal of Purple Robot is to analyze this smartphone usage data to objectively predict how likely the user is to have depression. Movement turns out to be a good indicator of a person’s mental wellness. GPS data can show how much a person has moved around between their regular locations. The more stationary a person is, the more likely they are to be depressed.
The Purple Robot app can also analyze the way a person uses his or her phone.The amount of time someone spends playing games or texting, but not talking on the phone, turns out to be another indicator that the person may be suffering from depression.
The accuracy of Purple Robot is still being determined and there’s certainly room for improvement. But the idea that a fairly simple smartphone app can objectively gauge a user’s risk for depression has lots of implications.
Will our intelligent assistants of the future have mental wellness trackers built-in, alongside daily fitness trackers? Instead of just prompting the user to stand or take more steps, could the intelligent assistant encourage the user to talk to a friend? Even an average assistant should know the user’s taste in music, sports teams, and perhaps hobbies. If the assistant notes the user has been spending a lot of time at home alone, could it start pushing recommendation cards of upcoming events that it predicts the user will be interested in? Could it be proactive and invite friends to join the fun?
Health trackers and workout apps are popular. Mindfulness and meditation apps are a growing trend. It seems only a matter of time before apps that track mental fitness enter the mainstream and eventually become integrated with our personal assistants.
Thomas Morrow, chief medical officer for Next IT, published an article in the American Journal of Managed Care (AMJC) entitled “Automated Intelligent Engagement Using a Virtual Health Assistant.” Morrow points out that human-to-human engagement is critical to improve patient outcomes, but this level of engagement is costly and hard to achieve in reality.
Virtual Health Assistants (VHAs) can fill the gap by standing in as proxies for health care providers and by helping patients with answers to questions, reminders, and motivational support. Morrow suggests that VHAs might even be better suited than human practitioners when it comes to following up on sensitive personal topics.
For example, he points out that a patient might be less inhibited about discussing a prescribed drug’s impact on sexual performance than if the patient were interacting with another human. In fact, I’ve written before about research that supports the view that people are generally more open with virtual humans / avatars than they are with other people (and this certainly holds true when the other person is an authority figure such as a physician).
Morrow sees huge promise for VHAs. He describes the technology as being in its teenage years. VHAs will be able to automate activities that currently reduce the amount of quality time doctors have to interact with their patients–such as record keeping in electronic health systems.
Morrow also envisions VHAs of the future as effective health coaches. The ideal VHA will get to know the patient and their lifestyle and then influence positive choices related to diet, exercise, and overall wellness. With the right motivational VHA as coach, and incentives and rewards, individuals might even avoid conditions that lead to disease and dependency on prescription medicines. Many of the activity trackers available in today’s wearables already allude to this capability.
Morrow lists the following companies as those that offer virtual assistant technologies specializing in health care.
I’ve written about Codebaby and Geppetto recently. Based on the AJMC article, Morrow plans to take a closer look at some of the companies in the list. I’ll be sure to keep an eye out for his observations.
Less than two weeks ago the co-pilot of the Germanwings Airbus intentionally steered the commercial airliner into a mountain, killing all on board. Tha plane’s automated flight control systems operated flawlessly. A mentally unstable human being, however, was able to easily override these systems and make the plane do something it was never intended to do.
In the future, will aeronautical autopilot systems be intelligent enough to know when not to give control to a human pilot? Our technology isn’t at a point where it can protect us from every bad or purposely evil human decision. But what if intelligent assistants could at least help people suffering from depression to get through their day more easily? This question is apparently being explored by researchers at Northwestern University.
A clinical trial began last month to test the efficacy of smartphones to help people with depression and anxiety. The program is referred to as IntelliCare and leverages smartphones to target personalized treatment material to participants. The participants receive messaging and also provide input back into the app. The apps help the patients by teaching them skills to assist with managing their mood. According to the documentation describing the clinical trial, the IntelliCare system also applies machine learning techniques to develop algorithms that will better tailor the app to the individual participant, based on collected data.
Principal Investigator David C Mohr of the Feinberg School of Medicine at Northwestern has done previous studies on the effects of specially-designed mobile apps for those suffering from psychological disorders. One such study sought to measure the results of mobile app intervention on patients with schizophrenia.
The study found that 90% of patients rated the app as beneficial and that after just one month of using the app, significant reductions in psychotic symptoms were noted.
Can tools such as these be integrated into healthcare intelligent assistants of the future? Will we be able to rely on our smartphones, or other connected devices, to help us tame our inner demons and learn techniques to manage our moods? The technology definitely seems worth exploring.
I recently ran across Geppetto Avatars, a company that creates intelligent virtual characters. The avatars have many potential use cases, but currently the company seems to be focusing on the health care space.
Sophie is a compelling 3D virtual character that interacts with patients in the role of physician’s assistant or medical advisor. The underlying technology for Sophie includes speech recognition and natural language processing that enables Sophie to understand what patients are saying. In the video demos on the Geppetto Avatars website, the virtual assistant appears on a web screen and engages in a realistic conversation with the patient.
Sophie has access to the patient’s medical records and asks prompting question to ascertain the patient’s current condition. Based on the patient response, Sophie can ask additional questions to understand details about improvements or problems with the patient’s medical situation. Sophie’s user interface includes a method for patients to take photos of problem areas, such as swollen hands or rashes, and submit them for physician examination.
Sophie can review the patient’s medications with them and ask if they are still providing the desired improvement. She can inquire about whether the patient is doing any physical therapy exercises that may have been assigned.
Sophie can also pick up on visual and intonation cues to assess the patient’s mood. If the patient is upset or unhappy, Sophie senses this and adjusts the conversation accordingly. In addition, she can issue questionnaires to the patient to gather more data about their current condition.
If Sophie works as well in real life as she does in the demo videos on the Geppetto Avatars website, the technology could be of huge benefit to health care practitioners and their patients. Typical physicians have so many patients in their practice that it’s hard for them to spend adequate time with each patient. Having a virtual assistant like Sophie would offer patients a way to get the attention they need, when they need it, while ensuring that the physician stays up to date on their condition.
Why is the company called Geppetto Avatars? I don’t know for sure, but it’s interesting that Geppetto was the name of the woodcarver in Pinocchio who made the wooden puppet boy that came to life. These intelligent avatars aren’t flesh and blood, but they can support humans by acting as a reliable proxy to their health care provider.
Derek Top of Opus Research recently wrote about intelligent assistants in the healthcare space. One of the technologies Top profiled was CodeBaby’s intelligent assistant Kyla, which supports Colorado’s health insurance marketplace. I gave Kyla a test drive to see how she works.
CodeBaby’s Kyla differentiates itself from other self-service virtual assistants in that there is no text-based interface. In fact, you can’t ask Kyla specific questions. Though this structure sounds like it would be very constraining for the potential health insurance customer, I found that Kyla actually works quite well.
Kyla appears as an animated image of a young woman. The animation is pleasant and doesn’t mimic a human image enough to be creepy. Kyla is more like a guide than a question-answering bot. She pops up at the lower right of the healthcare connect screen and provides a pre-recorded message to welcome the user to the site and give them a quick overview of what the site is about. All of Kyla’s statements are pre-recorded. She has a human voice and her statements are made with natural intonation and tonality, which is a big plus over computer generated speech.
Once Kyla has finished introducing the user to the site (or to a new web page), a pop up appears with a selection of other topics that Kyla can address. Some examples of topics the user can choose are:
- How long will it take me to enroll?
- I want to learn more about financial options
- What are the important deadlines I should know about?
When you select a topic, Kyla delivers her pre-recorded response. The text of her response isn’t displayed on the screen, so you must have your speakers turned on and you must be able to hear. You can start and stop Kyla’s recorded message and replay it as often as you like.
I think Kyla definitely works as a site guide and advisor. If a user has questions beyond those that have already been anticipated, Kyla won’t be able to assist. But she can refer the health insurance shopper to a webpage that helps them find a human broker. The intelligent assistant guide is a great fit for a website as complex and intimidating as a health insurance marketplace. It’ll be interesting to see if the intelligent assistant as guide gets expanded to other use cases
At Opus Research’s recent Intelligent Assistants Conference, I ran into Ann Thyme-Gobbel and Charles Jankowski from 22otters. 22otters offers an interesting mobile app that provides a virtual health coach for patients preparing for a routine medical procedure. Currently the app is designed to support people preparing for a colonoscopy. Anyone who’s had one knows that the prep is the worst part of the whole experience. So who wouldn’t want a health coach to help you get through it?
22otters can adapt their virtual coach to support a whole range of medical procedures. But let’s take the colonoscopy prep as an example. The health coach assistant provides a complete checklist of steps to guide you through the preparation process. Each step is plotted on a calendar based on your appointment date and time. The health coach proactively alerts you when each step in your regimen needs to begin and uses voice commands to explain what you need to do. For example, on the day prior to your procedure, the assistant tells you to drink only clear liquids from now on until the procedure is complete.
If you need specific instructions on how to prepare your laxative concoction, the health assistant will walk you through the steps using both voice explanations and helpful graphics. The app also provides alarms to remind you when to take doses of liquids or other medications. As you follow each of the steps, you can verify that you’ve completed them. Your doctor can validate your overall status by looking at a portal that shows exactly where you are in the prep process and how well you’ve kept to the instructions.
You can take your health coach with your wherever you go and ask it clarifying questions. Procedures such as a colonoscopy are complicated for the patient, but all the steps are routine. It’s extremely helpful to have all the information in an easy-to-use health coach app, rather than having it in static paper handouts. The proactive alerts help to ensure that you don’t miss a step or forget something important in the process. Having the virtual health coach also reduces your need to pick up the phone and call the doctor.
Thyme-Gobbel and Jankowski showed me another app called HerStory that offers breast cancer patients an opportunity to record audio clips about their own personal experiences. Users of the app can access these crowdsourced survivor stories to receive encouragement and helpful tips from other women who’ve gone through the same experience. This “share you story” feature could potentially be added to any of the health coach apps.
I can imagine a whole range of routine procedures where both patients and healthcare providers could benefit from a health coach. Detox protocols offered by practitioners of functional medicine come to mind, as well as procedures that patients need to follow after certain types of surgeries. As intelligent assistants become more specialized, the virtual health coach offered by 22otters shows how a focused use case can offer lots of compelling benefits, in this case to patients and healthcare providers.
VentureBeat ran a story by Kia Kokalitcheva about the health technology startup Wellframe. Wellframe is designing mobile apps that assist people to better manage chronic illnesses. It doesn’t appear that the Wellframe health apps are conversational at this point, but they seem to me to fall into the virtual assistant category. They aim to collect and understand data about the patient’s health situation and then design a program specifically tailored to the patient’s needs.
After a patient is diagnosed with a condition, like a cardiac issue, the patient is matched to a clinical program designed to assist him or her in having a successful rehabilitation. Knowing what to do and when to do it can be confusing for someone facing new health issues. And it’s difficult to stay in touch with busy physicians and get personalized attention. The Wellframe app is a kind of stand-in for a physician’s assistant, offering daily to-do lists that help the patient focus on things that will improve his or her health outcome. The app can also send data about the patient’s regime and physical statistics to a central server so that the patient’s doctor can keep an eye on things through a web dashboard.
The Wellframe app can’t be used without an official activation code provided by the patient’s physician. It’s about extending the reach of the physician / clinician, not about simply providing a self-directed health app.
Now imagine an app like those designed by Wellframe, but integrated into a general virtual assistant like Google Now, Siri, Dragon Mobile Assistant, Cortana, or others. How powerful would it be to have an intelligent assistant that you can talk to and that can speak to you as well, that can prompt you with speech or a text message to take your medication, or encourage you to take your daily walk and provide friendly kudos when you’ve done so. If you’re not sure if it’s okay to have that salty treat or eat another helping of eggs, you can just have a quick conversation with your assistant about it. If you’re doctor isn’t available when you’ve got some basic questions, you can talk to your assistant and either get the answers, or have the assistant update your doctor’s dashboard with details about your question and a request to get back to you.
It’ll be interesting to see if these specialized apps can become integrated into the basic layer of more general voice-enabled, conversational assistant apps. These types of super-deep domain expertise are what’s needed to take today’s virtual assistants to the next level.