Part 2: Are Digital Health Solutions Actually Solutions?

I asked this question around one year ago: are digital health solutions actually solutions? You can read my article here.

A year later, we may have a preliminary answer….

In this article, I’ll discuss the impetus to my question, highlight the findings from the Peterson Health Technology Institute‘s report on the efficacy of several digital health solutions, and tell you the impact of such findings.

The Deets

Trustworthiness will continue to be a key differentiator among digital health startups in a saturated, low-funding space. But how can digital health startups build trust?

Trustworthiness is a challenge in the digital health space, given the lack of published evidence showing the efficacy of digital health solutions.

For example, a cross-sectional observational analysis of the 20 digital health companies with the most funding in 2017 found these companies had not yet “demonstrated substantial impact on disease burden or cost in the US health care system.” Most studies published by these companies only evaluated their digital health solution in healthy patients, and a paucity of published studies evaluated clinical outcomes for these companies’ digital health solutions.

Furthermore, in Rock Health’s cross-sectional observational study assessing the clinical robustness of digital health startups, researchers found that many digital health companies lack clinical robustness. The researchers analyzed 200+ digital health companies across different clinical domains (prevention, diagnosing, treatment) and assigned a “clinical robustness score” based on the sum of clinical trials and regulatory filings (510(k), De Novo, pre-market approval).

Here’s the distribution of results:

So, around 60% of the companies analyzed had a clinical robustness score of 1 or less (the median score was 1). Not a good look….

In my original article, I noted the fast pace in which the digital health space was growing, outpacing robust regulatory measures or frameworks to ensure digital health solutions are actually solutions from technical, clinical, and system areas.

Either my original piece truly resonated with folks or it was just good timing, but shortly after I published my article, the Peterson Health Technology Institute launched, serving as an independent evaluator of the clinical effectiveness and economic impact of digital health solutions. This is what we (digital health) needed. Until now, researchers have published frameworks for the evaluation of digital health solutions, but nobody—as far as I know—has actually taken a framework and put it to use.

PHTI’s framework (see my analysis here) for evaluating digital health technologies revolves around three principle questions followed by a final recommendation:

1. What is the digital health technology?

2. Does it work and for whom does it work?

3. Is it worth it?

PHTI will include all of their quantitative and qualitative data into a single report, with an assigned recommendation essentially saying whether a purchase should adopt or not adopt digital health technology, or if the digital health technology needs further testing.

Part 2: Are Digital Health Solutions Actually Solutions?

PHTI released their first report on digital diabetes management solutions in the remote patient monitoring and behavior and lifestyle modification categories, finding no meaningful clinical benefits and increased net healthcare spending for purchasers.

To assess clinical effectiveness, PHTI researchers analyzed 49 studies, which included both published data and data provided by companies. 10 of these studies compared HbA1c changes from baseline in digital intervention to usual care groups. Below I highlight the findings for clinical and economic effectiveness by categories. Note that the threshold for meaningful clinical benefits was 0.5% points (studies showing a difference >0.5% points is good, <0.5% bad).

Clinical Effectiveness

• Remote Patient Monitoring: five of the 10 studies examined remote patient monitoring, finding HbA1c reductions of 0.34% points to 1.2% points greater than usual care. The largest effect size (1.2% points) was observed in a cohort study of 40 patients with an average starting HbA1c of 10.8% (quite high) who were started on insulin and followed for 10–14 weeks. This suggests RPM has small benefits compared to usual care, but may benefit those with high baseline HbA1c and insulin treatment.

• Behavior and Lifestyle Modification: four of the 10 studies examined this category, finding HbA1c reductions of 0.23% points to 0.37% points greater than usual care. This suggests behavior and lifestyle modifications have small benefits compared to usual care. None are clinically meaningful.

• Nutritional Ketosis: one of the 10 studies examined this category from Virta Health found an effect size of 1.3% points HbA1c reduction at year one and 1.2% points reduction at the two-year follow-up. This category provides the only evidence that supports sustained positive effects for digital diabetes management solutions compared to usual care.

The above evidence suggests these digital health solutions mainly benefit patients with uncontrolled diabetes (HbA1c >9%) who are initiating insulin for the first time. Outside of this population and the nutritional ketosis category, digital health solutions for diabetes monitoring do not have a clinically meaningful impact.

Economic Impact

I highlight the economic impacts for each category in the table below:

Net Savings Change Per User for Payers by Category

*Based on Vista’s analysis estimating year one annual savings of $3,094 (price per user in year one $3,838). See exhibit 16 in report.

As you can see, there’s a net increase in spending across the board. For Virta Health, they state savings will be incremental, meaning there may be negative savings in year one, but as time goes on and their patient population achieves targeted HbA1c, positive savings will be realized.

Overall, the above results suggest these digital solutions provide little to no meaningful clinical benefits yet lead to increased healthcare spending relative to usual care. However, Virta’s nutritional ketosis solution, and others, may be clinically beneficial. PHTI summarized this into the table below (please zoom in to really see the details).

Dashevsky’s Dissection

PHTI’s report answers my original question… digital solutions may be solutions, but much more robust evidence is warranted.

As a physician, this report sheds light on the digital health solutions from which my patients with diabetes would most benefit. Virta Health, for example, offers intense but personalized interventions that lead to sustainable improvements in diabetes management, although at an initial cost. Other solutions likely won’t provide clinically meaningful benefits at a larger population level and will be more costly (unless more data is published showing otherwise). However, as physicians, it’s important to remember we treat patients—not populations.

Payers, on the other hand, cover populations and frequently make purchasing decisions based on clinical and economic benefits at the population level. Again, it appears the population that would most benefit from these solutions are patients with uncontrolled diabetes starting insulin. So, payers may choose to restrict coverage of these digital health solutions only for this population with uncontrolled diabetes.

For startups, I don’t see this report as a “negative” finding. Rather, it’s an an impetus—or wake up call—to prioritize coupling solution building with evidence building. The two should go hand in hand. Evidence that disfavors a digital health solution should encourage builders to pivot or figure out why their solution isn’t driving clinically meaningful results.

Data gathering is easier said than done, though—to do it correctly requires time and money. With a saturated digital health market, getting to market quickly is advantageous and building evidence to support a solution may delay go-to-market. Additionally, testing the solution requires money, especially if a company wants to build the highest standard of evidence (randomized control trial). In a dried-up funding space, founders may deprioritize raising money to “study if a digital health solution is clinically meaningful.”

It shouldn’t be deprioritized though…

PHTI, in my opinion, is setting a new standard in digital health, where evidence supporting the clinical and economic efficacy of solutions will be prioritized by payers, clinicians, and investors. So much so that there will soon be a time where the question sequence is “what is your solution?” and “is there evidence showing it works?” instead of “how many customers do you have?” Perhaps investors can work with founders to build appropriate evidence for their solutions (in the least conflict-of-interest manner).

Overall, PHTI's comprehensive report reveals that digital health interventions in diabetes care might fall short of being true solutions, highlighting a stark absence of solid clinical and economic evidence. It's imperative for digital health companies to intertwine solution building and evidence building to outpace competition and attract investors.