Control Groups in Digital Health Studies

accounting administration books business
Photo by Pixabay on


Digital health has been around since 2010, but the newer focus on digital therapeutics is only a few years old. In essence, a digital therapeutic translates an evidence base into a digital user experience to diagnose, monitor, and treat a clinical issue to defined clinical endpoints. For example, cognitive-behavioral therapy apps that meet the bar of digital therapeutic reliably reduce symptoms of depression and anxiety, so the users are more likely to achieve a clinically significant reduction or remission of symptoms.

One of the challenges in the digital space is studying an app that is continually changing. How is an end-user meant to know what good looks like or if an app is helpful? A net promoter score (NPS) is not a valid metric in this instance. In the clinical realm, a randomized-controlled trial (RCT) is a gold standard, so far this has been a challenge in the digital space as companies are funded by venture capital firms or angel investors who expect a return on their investment. As these solutions have developed, it is highly unlikely an app company would develop a sham version of their product to conduct an RCT- we don’t have the sugar pill placebo for digital therapeutics, but should we?

For digital therapeutics to gain traction in health care, developing an evidence base will be crucial to longevity. It is likely not top of mind for investors in-app companies, but it will be any health care organization approached regarding a partnership. In time, consumers will also demand more evidence; an app or sensor is beneficial beyond the NPS number. Hype will only take you so far.

A new opinion piece in Nature Digital Medicine calls on the different kinds of controls that may be necessary for digital health to advance. Janosch Priebe and colleagues from the University of Munich, Germany, flesh out what control groups might look like in pain management. Digital health solutions will often use a waitlist control group, that being one group who receive the app or sensor as part of “treatment,” and another group will “wait in the wings” to get access to that solution. One challenge with this approach is the sequelae of a particular disease state might mean people will worsen or get better in the case of low back pain over that multi-week waiting period, which would confound the results. As in any RCT ethics of non-treatment will also apply as participants have the expectation of treatment.

The authors propose a second approach that compares the app or sensor to guidelines delivered face-to-face care. In this case, comparing an app to a live physician is an unfair comparison. The participant may bond with that doctor, and frankly matching a patient population of app users to compare with patients who are part of a multi-disciplinary treatment protocol may be difficult to do. Are we at a point in digital health where we can say severe cases of pain management can be solely management by an app? I think not- yet.

A third approach might be to compare an app or sensor to usual care- also standard in the published literature. The value of adding an app to routine care may tease out where current gaps in care lie and fill them with continuous monitoring and support. This model may also enable broader reach beyond usual care, and the participant is getting help and engaging in skill-building between office visits.

All approaches have their pros and cons- what is clear is the need for digital health, specifically digital therapeutics, to consider the development of an evidence base for their solution as part of their initial development. It is a delicate balance to innovate and also provides evidence an app works, but this, along with data use, privacy, and transparency will be table stakes for the next evolution of digital therapeutic companies. Many tools in use today deploy as part of existing models of care.-The Value, I believe, of digital health is it can also lead us to new models of care and can enable us to reach more people with an evidence-based standard of care. Being able to demonstrate efficacy and effectiveness is part of that next frontier of development.

Thanks for reading – Trina
(Opinions are my own)



Is there a right control condition in mHealth trials? A critical view on pain management.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Close Menu
%d bloggers like this: