About the authors: Isobel Rosenthal (@isobelrosenthal) is a psychiatry resident at Mount Sinai Hospital in New York City. She holds an MBA from Columbia Business School. Rosenthal was the 2019 winner of American Psychiatric Association Innovation Prize, and is a cohost of the podcast Well Rounded.

Sofia Noori (@sofianoori) is the chief resident of digital psychiatry at Yale Department of Psychiatry. She is the curriculum lead for Innovation to Impact, a substance use entrepreneurship program funded by the National Institute for Drug Abuse.

With the unfolding mental health crisis as a result of COVID-19, investors and entrepreneurs alike are pouring record time and resources into digital mental health. According to Rock Health, behavioral health tech raised $588 million in the first half of 2020. High-profile investments such as Optum’s infusion of capital into AbleTo continue to make headlines. Every day, there is increasingly more focus on behavioral health technology and its promise to deliver fast, seamless care at a time when usual treatment modalities are difficult to access.

Although more mental health services are urgently needed by the American population, there is increasing recognition that innovations in mental health care should be based on sound research or evidence. In the past week alone, concerns about evidence base have arisen at popular startups such as TalkSpace and Headspace.

Why do so many companies fail to be evidence-based? The answer is complicated, and context around the existing state of mental health treatment can provide some insight into its difficulties.

Moving towards collecting rigorous clinical outcome data poses an enormous challenge across digital mental health. Incorporating outcomes metrics within traditional psychiatric practice has long been difficult. Measurement-based care is defined as assessing the severity of disease symptoms to guide decisions in care. For example, a doctor may use blood pressure measurements to diagnose a patient with hypertension. Despite the existence of validated scales for measuring symptoms (such as in depression and anxiety), which are shown to improve patient outcomes, the use of measurement-based care in mental health is low. Only 18% of psychiatrists use them to measure patient symptoms over time. There are multiple reasons for this, including lack of time or training to administer these scales and a sense that the scales don’t capture the diverse phenotypes of mental illness.

Evidence-based medicine requires clinical practice to be guided by existing clinical data, but because mental health symptoms are not routinely measured, it is difficult to then collect evidence on whether a treatment actually works. Thus, multiple modalities in usual mental healthcare are routinely recommended and have no clear evidence base – such as 12-step programs for substance use.

There is concern that many digital mental health companies are bringing products to market without clinical pilots to demonstrate that their solutions work. Additionally, many of these same companies may claim they are evidence-based when they are based off of vague translations of existing clinical models.

Furthermore, even if companies are running pilots prior to launch, very few publish their raw data or publish for peer review. This absence reduces health professionals’ ability to review the data and recommend digital health treatments to patients. However, these issues reflect larger concerns regarding evidence-based treatments across the field of mental health.

Another problem is that in the world of digital health, attempting to develop rigorous clinical models does not guarantee success. Lantern, a digital cognitive-behavioral therapy (CBT) company, published promising results on the cost-effectiveness of digital CBT and teamed up with the University of Pittsburgh to design a non-randomized control trial using their platform in 2018. However, Lantern shut down its clinical operations later that year and licensed its technology to other companies.

Conversely, many mental health startups have gained traction in the market with unclear evidence for their products. Unlike clinical medicine, success for digital mental health companies isn’t just about effective treatments, but also about running an operationally and financially sound enterprise.

Ultimately, the fact that many companies lack clinical evidence does not fall on just their shoulders. Simply put, clinical research practices have not caught up to the speed of technology. Randomized controlled trials are the gold standards of clinical research but they are very expensive, time-consuming, and difficult to implement, making them nearly impossible for upstart digital health companies to achieve.

Academic medicine would benefit from partnering with digital health companies either to assist with clinical research or help develop new research frameworks specifically for digital health.

Multiple stakeholders need to collaborate to protect customers and encourage high-quality technology in the field of digital mental health.

National organizations such as the FDA, DHS and NIH must focus on innovating digital health assessment and regulation. New funding streams dedicated to digital clinical research can incentivize both academics and industry partners to focus on building the evidence base.

Taking the lead, Germany recently published the Digital Healthcare Act, which offers a national framework for reimbursement, prescription and regulation of digital health apps.

The disconnect between the existing mental healthcare community and the greater technology community is an important barrier to recognize. Because mental health professionals are not educated about digital health, they have difficulty in navigating evidence-based mental health solutions and do not know how to recommend or prescribe them to patients.

Slowly, inroads are being made. The American Psychiatric Association recently launched an app advisor to help psychiatrists navigate the space. Additionally, we should be training psychologists, psychiatrists, social workers and other clinical professionals on digital behavioral health.

If digital mental health is to succeed, tech companies need to include clinicians and researchers in the design of behavioral health technology from its inception. Doing so will enable companies to better consider the nuances of person-centered design in healthcare and to structure their data collection to existing clinical research standards.

In the U.S., opportunities for academic partnerships or other public-private partnerships to spur evidence-based digital healthcare are emerging, such as Hope Lab and Stanford Brainstorm. Consulting groups that specifically focus on helping with clinical research aspects of technology can also offload the difficulty of finding and affording tech-literate clinical talent.

Additionally, mental health treatment in general would benefit from widespread adoption of measurement and evidence-based care. It is not just the burden of digital mental health solutions to prove efficacy, but also mental health treatment at large. In 2018, the Joint Commission required the use of measurement-based care in accredited practices, spurring the field in the right direction.

The COVID-19 pandemic has catapulted digital health into the mainstream. The U.S. populations’ developing mental distress has pushed interest in digital mental health to new heights. As so much of existing care moves to a virtual setting, more work needs to be done to ensure that we are creating effective and helpful digital mental health products. Technologists must collaborate with healthcare researchers and clinicians to commit to a refrain every clinician understands: “Do no harm.”