Dive Brief:

• Chatbots hold real potential for how consumers and providers connect, but barriers remain to widespread adoption, a new report shows.
• UserTesting asked 500 consumers to assess chatbot apps based on four criteria: ease of use, speed, credibility, aesthetics and delight. Apps received a score on a scale of zero to 100, as well as qualitative insights.
• The results highlighted concerns around consumer trust, handling of complex medical conditions and basic usability of these devices.

Dive Insight:

The consumer platform company looked at five healthcare chatbots: Ada, HealthTap, Mediktor, Your.MD and Symptomate. Chatbot apps were especially challenged by complex conditions, with all experiencing sharp declines in scores when asked to diagnose complex symptoms, such as food poisoning — often leading to alarming results.

Consumers also had concerns about personal health information and HIPAA compliance, especially with less familiar brands. “Oversights in general usability also contributed to lack of trust,” according to the report. “Some of the key complaints mentioned spelling errors, and suggested links to providers or content that seemed untrustworthy or fake.”

There were also lots of basic usability issues. For example, loading issues with HealthTap forced many users to exit the app and start the diagnosis process all over again. 

Nearly all of the study participants had some prior experience using chatbots, and nearly half used them to research medical symptoms. Still, the report cites data showing 50% of people who download health-related mobile apps eventually abandon them because of usability problems.  

Overall, Ada fared the best at 84.3, followed by Mediktor (80.1), Your.MD (75.8), Symptomate (72.6) and HealthTap (40.5).

“Customers have high expectations when it comes to their digital experiences,” Janelle Estes, chief insights officer at UserTesting, said in a statement.

Despite none of the apps scoring high in all categories, 73% of participants felt they were helpful. Just 12% said they were not helpful, while 15% were neutral. 

Younger users were generally more forgiving of chatbots’ shortcomings, which, while not surprising given their greater comfort level with technology generally, could limit their usefulness in older age groups, which have more regular visits and interactions with doctors.

“Keep in mind these are nascent, very emergent technologies … and one thing we’re dealing with in healthcare that is different in terms of maybe other technologies is our very, very low fault tolerance,” Laura Craft, research director at IT consultancy Gartner, told Healthcare Dive in an interview last year.

Craft said the real vision is for chatbots and virtual assistants to act as avatars that not only diagnose symptoms or help patients manage chronic conditions, but also perform sentiment analysis.