Amazon has launched its own fitness wearable and health app, built not just to log a person’s physical activity but to also track their emotional well-being by scanning the tone of their voice day-to-day.
The Halo service and its wristband tracker—a slimmed-down version compared to devices offered by Apple and Fitbit, with no touchscreen or notifications—uses a combination of artificial intelligence-powered sensors and a smartphone camera to build a 3D model of the wearer, providing data on body fat percentage, temperature, heart rate and sleep quality. Along with a weeklong battery, the water-resistant device is designed to be worn 24/7.
Meanwhile, two microphones record the tone of a person’s voice, and use machine learning to note signs of energy and positivity, or even tiredness and stress. This could help the wearer log their emotions throughout the day, and help them understand how they may sound to others, Amazon said.
“Health is much more than just the number of steps you take in a day or how many hours you sleep,” said Maulik Majmudar, principal medical officer for Amazon’s Halo project.
“Despite the rise in digital health services and devices over the last decade, we have not seen a corresponding improvement in population health in the U.S.,” Majmudar added. “We are using Amazon’s deep expertise in artificial intelligence and machine learning to offer customers a new way to discover, adopt and maintain personalized wellness habits.”
RELATED: Secret Amazon research unit developing health tech with Fred Hutch: report
And while it’s not a medical device per se, data from Halo will link with Cerner’s electronic health records and participating care providers. “Integrating the revolutionary body fat percentage measurement from Amazon Halo directly into the EHR provides physicians an actionable and previously hard to obtain health metric without the need for a doctor’s visit or costly technology,” said Cerner Chairman and CEO Brent Shafer.
All the health data collected by the device are encrypted before being sent to Amazon’s cloud servers, with body scan images automatically deleted after processing, the company said. Voice samples, on the other hand, cannot be listened to; they are analyzed locally on a person’s smartphone and deleted thereafter.
The voice AI program, referred to as Tone, passively analyzes a person’s pitch, intensity, tempo and rhythm and provides a summary that can be used to identify emotional trends throughout the day.
“I have been working from home with three kids under the age of four, so I use Tone to gut check that I am not taking any stress out on my family or friends,” said Majmudar, who before joining Amazon was associate director of the Healthcare Transformation Lab at Massachusetts General Hospital and an assistant professor at Harvard Medical School.
“I check my Tone results so that I can be more intentional about how I communicate in these strange times—and have noticed it takes a burden off my wife, as she doesn’t have to be the one to tell me I am overly stressed,” he said.
RELATED: ‘Alexa, what’s my blood sugar?’—Amazon unveils HIPAA-compliant voice services
Amazon is also working with a host of healthcare providers, fitness companies and wellness developers to provide challenges, workouts and ways to link accounts. This includes the American Heart Association and the Mayo Clinic, as well as Orangetheory Fitness and WW, formerly known as Weight Watchers.
WW will also offer a free Band and six months of the Halo service as part of a new WW membership. “The Amazon Halo Body composition tool is an important complement to weight measurement, and we are thrilled our members will have access to this information on a regular basis from the comfort and privacy of their own homes,” WW CEO Mindy Grossman said.
Halo will also serve as the complimentary wearable for the life insurance provider John Hancock, as a strategic collaboration with its Vitality digital wellness program. In 2018, the company announced that it would only offer plans that incorporate active wellness tracking and health data.