The power of digital data: Using AI to predict mental health states based on digital phenotyping

The famous Dutch philosopher Desiderius Erasmus once said, “Prevention is better than cure.”

This timeless wisdom resonates with us in our daily lives. We clean our rooms to avoid big clutters. We wear sunscreen to prevent getting sunburned. And we brush our teeth every day to prevent cavities. Why do we adhere to these preventive rituals? It’s because we can predict what would happen without these precautions. Sure, there are “cures” available, but they often bring stress, financial strain, and risks. Let’s be honest, everybody dislikes going to the dentist, right?

Now, imagine if we could apply the same proactive approach to our mental health. What if we could foresee the escalation of mental health symptoms?

Early intervention of mental health conditions 

Mental health conditions often exhibit a chronic and recurring pattern. Consider, for instance, that nearly half of individuals receiving treatment for depression experience a relapse within the first six months of recovery. Similarly, around 20% of those with anxiety relapse within the first two years. 

Despite this alarming rate of relapse, access to essential mental health services remains an unmet need for a significant majority. Shockingly, more than 70% of individuals requiring such care find themselves without the necessary support. This is partly due to the internal nature of the symptoms. Consequently, many suffer in silence, receiving assistance far later than optimal, if at all.

Delaying treatment is problematic. Conditions often escalate, becoming much more frequent, severe, and unpredictable. It is imperative that we address this gap in identifying those at risk to provide early intervention before symptoms escalate. 

Digital phenotyping: passive data

Recently, there has been an exciting advancement in AI and mental health care to address this gap. Researchers have been using machine learning to monitor the development of mental health and build a predictive model for depression and anxiety. What is more even exciting is the use of AI has extended to using digital phenotyping (DP)! 

Digital phenotyping is the quantification of behavioral biomarkers using data from personal digital devices. Using passive sensors already existing in our smartphones to collect data is advantageous because it allows a continuous collection of data within naturalistic settings. In particular, mobility, physical activity, phone usage, and sleeping patterns are all key markers of depression and anxiety. 

What did we do?

I carried out a research project with Thrive Mental Wellbeing, an NHS-approved digital mental health company, to predict the development of depression and anxiety symptoms based on DP. 

Thrive collected questionnaire scores and passive data for 8 weeks from working professionals using the AWARE-Light app. For the questionnaire, Patient Health Questionnaire-9 (PHQ-9) scores were used for depression and Generalised Anxiety Disorder-7 (GAD-7) scores were used for anxiety.

For passive data, spatial entropy, physical activity, phone use, and sleep were measured using five sensors. Click here to see more details about the sensors. 

Spatial Entropy: Location

Physical activity: Accelerometer & Gyroscope

Phone Use: Keyboard & Touch

Sleep: Accelerometer & Gyroscope

(Note: Touch is not listed on the website, and is only available on the app itself.)

Because of the limitations in data collection, some of these measurements had to be inferred from the sensors. For example, an idle state of physical activity was used to measure hours of sleep each night. With this data corpus, we paired the passive data with the questionnaire scores and built a Random Forest Classifier model. It sounds complicated but it is basically a machine learning model that allows us to predict future scores based on passive data!

What did we find?

The accuracy of the models for both GAD-7 and PHQ-9 scores was low (both .286). They were able to predict the correct class of scores in less than one-third of the instances. 

The confusion matrices below show a clearer picture. 0 indicates scores below 10 (mild symptoms) and 1 indicates scores equal or above 10 (moderate and severe). The model very often predicts a score that is lower or higher than the actual score.  

What does this mean?

The results of the model are puzzling. It does not align with previous literature. Instead, our model indicates that DP in fact does not add any value in predicting future mental health states. There are several reasons why this might have happened. First, it could be the very low number of participants we had. It’s no secret that a larger sample size yields more reliable results. Second, it could be the limitations in feature extraction. As mentioned before, we had to extract certain features like sleep from passive data (i.e., idle physical activity). Our attempts to infer these features, while logical and well-intentioned, may have inadvertently resulted in misleading data.

Imagine this: participants engaging with other digital devices during the night while their phones remained motionless. Or picture them dozing off with their phones lying on the bed, resulting in a phone that appears to be on the move. It’s like a real-life game of digital hide-and-seek, adding an unexpected twist to our quest for reliable data.

What should we do next?

Disappointing as it may be, our model’s inability to predict future mental health states does not signify a wasted effort. On the contrary, this research project has opened our eyes to an important realization: the immense challenge of capturing accurate passive data. In this tale of trial and error, we have paved the way for future researchers, providing them with valuable insights on what to avoid and paving the path toward more precise and impactful research.

References

Kessler, D., Lloyd, K., Lewis, G., & Gray, D. P. (1999). Cross sectional study of symptom attribution and recognition of depression and anxiety in primary care. BMJ : British Medical Journal, 318(7181), 436–440.

Kohn, R., Saxena, S., Levav, I., & Saraceno, B. (2004). The treatment gap in mental health care. Bulletin of the World Health Organization, 82(11), 858–866.

Moriarty, A. S., Paton, L. W., Snell, K. I. E., Riley, R. D., Buckman, J. E. J., Gilbody, S., Chew-Graham, C. A., Ali, S., Pilling, S., Meader, N., Phillips, B., Coventry, P. A., Delgadillo, J., Richards, D. A., Salisbury, C., & McMillan, D. (2021). The development and validation of a prognostic model to PREDICT Relapse of depression in adult patients in primary care: Protocol for the PREDICTR study. Diagnostic and Prognostic Research, 5(1), 12. https://doi.org/10.1186/s41512-021-00101-x

Nemesure, M. D., Heinz, M. V., Huang, R., & Jacobson, N. C. (2021). Predictive modeling of depression and anxiety using electronic health records and a novel machine learning approach with artificial intelligence. Scientific Reports, 11(1), Article 1. https://doi.org/10.1038/s41598-021-81368-4

Scholten, W. D., Batelaan, N. M., van Balkom, A. JLM., WJH. Penninx, B., Smit, J. H., & van Oppen, P. (2013). Recurrence of anxiety disorders and its predictors. Journal of Affective Disorders, 147(1), 180–185. https://doi.org/10.1016/j.jad.2012.10.031

Sensors – AWARE. (n.d.). Retrieved May 28, 2023, from https://awareframework.com/sensors/

Spinazze, P., Rykov, Y., Bottle, A., & Car, J. (2019). Digital phenotyping for assessment and prediction of mental health outcomes: A scoping review protocol. BMJ Open, 9(12), e032255. https://doi.org/10.1136/bmjopen-2019-032255

Torous, J., & Keshavan, M. (2018). A new window into psychosis: The rise digital phenotyping, smartphone assessment, and mobile monitoring. Schizophrenia Research, 197, 67–68. https://doi.org/10.1016/j.schres.2018.01.005

Wang, P. S., Berglund, P. A., Olfson, M., & Kessler, R. C. (2004). Delays in Initial Treatment Contact after First Onset of a Mental Disorder. Health Services Research, 39(2), 393–416. https://doi.org/10.1111/j.1475-6773.2004.00234.x