Dec 14, 2020 8:00 AM

An AI Used Facebook Data to Predict Mental Illness

Volunteers let an AI scan their messages from more than a year before they received a psychiatric diagnosis. It was able to flag signs of their conditions.

Faces overlapping on top of a figure with a phone.

It’s easy to do bad things with Facebook data. From targeting ads for bizarrely specific T-shirts to manipulating an electorate, the questionable purposes to which the social media behemoth can be put are numerous. But there are also some people out there trying to use Facebook for good—or, at least, to improve the diagnosis of mental illness. On December 3, a group of researchers reported that they had managed to predict psychiatric diagnoses with Facebook data—using messages sent up to 18 months before a user received an official diagnosis.

The team worked with 223 volunteers, who all gave the researchers access to their personal Facebook messages. Using an artificial intelligence algorithm, the researchers leveraged attributes extracted from these messages, as well as the Facebook photos each participant had posted, to predict whether they had a mood disorder (like bipolar or depression), a schizophrenia spectrum disorder, or no mental health issues. According to their results, swear words were indicative of mental illness in general, and perception words (like see, feel, hear) and words related to negative emotions were indicative of schizophrenia. And in photos, more bluish colors were associated with mood disorders.

To evaluate how successful their algorithm was, the researchers used a common metric in artificial intelligence that measures the trade-off between false positives and false negatives. As the algorithm categorizes more and more participants as positive (say, as having a schizophrenia spectrum disorder), it will miss fewer participants who really do have schizophrenia (a low false negative rate), but it will mislabel some healthy participants as having schizophrenia (a high false positive rate). A perfect algorithm can have no false positives and no false negatives at the same time; such an algorithm would be assigned a score of 1. An algorithm that guessed randomly would have a score of 0.5. The research team achieved scores ranging from 0.65 to 0.77, depending on the specific predictions they asked the algorithm to make. Even when the researchers restricted themselves to messages from over a year before the subjects received a diagnosis, they could make these predictions substantially better than would have been expected by chance.

According to H. Andrew Schwartz, an assistant professor of computer science at Stony Brook University who was not involved in the study, these scores are comparable to those achieved by the PHQ-9, a standard, 10-question survey used to screen for depression. This result raises the possibility that Facebook data could be used for mental illness screening—potentially long before a patient would otherwise have received a diagnosis.

Michael Birnbaum, an assistant professor at the Feinstein Institutes for Medical Research in Manhasset, New York, who led the study, believes that this sort of AI tool could make an enormous difference in the treatment of psychiatric illnesses. “We now understand this idea that cancer has many different stages,” Birnbaum says. “If you catch cancer at Stage I, it’s drastically different than if you catch it once it metastasizes. In psychiatry, we have a tendency to start working with people once it’s already metastasized. But there’s the potential to catch people earlier.”

Birnbaum is far from the first researcher to have used social media data to predict the presence of mental illness. Previously, researchers have used Facebook statuses, tweets, and Reddit posts to identify diagnoses ranging from depression to attention deficit hyperactivity disorder. But he and his team broke new ground by working directly with patients who had existing psychiatric diagnoses. Other researchers haven’t, in general, been able to work off of clinically confirmed diagnoses—they have taken subjects’ word for their diagnoses, asked them for self-diagnoses, or had them take questionnaires like the PHQ-9 as a proxy for diagnosis. Everyone in Birnbaum’s study, in contrast, had an official diagnosis from a psychiatric professional. And since the researchers had definitive dates for when these diagnoses were made, they could try to make predictions from messages sent before the patients knew about their mental illnesses.

Sharath Guntuku, an assistant professor of computer science at the University of Pennsylvania who was not involved in the research, cautions that, even if these algorithms achieve impressive results, they are nowhere near replacing the role of clinicians in diagnosing patients. “I don’t think there’ll be a time, at least in my lifetime, where just social media data is used to diagnose a person. It’s just not going to happen,” Guntuku says. But algorithms like the one designed by Birnbaum and his team could still play a crucial role in mental health care. “What we are increasingly looking at is using these as a complimentary data source to flag people at risk and to see if they need additional care or additional contact from the clinician,” Guntuku says.

Schwartz notes that diagnosing mental illness is an inexact science, one that could be improved with the addition of more data sources. “The idea is, you’re triangulating mental health,” he says. “Assessing mental health is an exercise that can’t just rely on one single tool.” And since social media provides a continuous record of a person’s thoughts and actions across a substantial period of time, it could effectively complement the hour-long clinical interviews that are typically used to make diagnoses. In such an interview, says Schwartz, “you’re still relying on a patient to recollect everything, to recollect things about themselves. The clinician has to determine when they’re being influenced by desirability biases”—that is, the patient telling their clinician what they think they want to hear. Perhaps, then, social media data could provide a less skewed impression of a patient’s mental state.

Munmun de Choudhury, a professor of interactive computing at Georgia Tech who has previously worked with Birnbaum but was not involved in this particular study, envisions an opt-in social media plugin that could warn users when they may be at risk of mental illness. But such a plugin immediately raises privacy concerns—data about an individual’s psychiatric state, if leaked, could be misused by insurance companies or employers, or force an individual to reveal their mental illness status before they are ready to do so. To work at all, de Choudhury says, the makers of the plugin would have to be entirely transparent about how it handles and secures user data. But, if such an algorithm could detect symptoms of mental illness a year and a half before a patient would typically be diagnosed, it could make an enormous difference in people’s lives. “If we catch these symptoms much earlier on, there could be other mechanisms to alleviate these concerns that don’t necessarily need a trip to the doctor,” she says.

There is already precedent for using social media to prevent mental health crises. “Facebook and Google, they’re already doing this at some level,” Guntuku says. If a user searches for suicide-related terms on Google, the National Suicide Prevention Lifeline number appears before all other results; Facebook uses artificial intelligence to detect posts that may indicate suicide risk and sends them to human moderators for review. If the moderators agree that the post indicates a real risk, Facebook can send suicide prevention resources to the user or even contact law enforcement. But suicide presents a clear and imminent danger, whereas the mere act of receiving a mental health diagnosis often does not—social media users may be willing to sacrifice more privacy to prevent suicide than to catch the onset of schizophrenia a bit earlier. “Any sort of public, large-scale mental health detection, at the level of individuals, is very tricky and very ethically risky,” Guntuku says.

For his own part, Birnbaum sees a less grand, but nevertheless impactful, use case for this research. A clinician himself, he thinks that social media data could not only help therapists triangulate diagnoses but also aid them in monitoring patients as they progress through long-term treatment. “Thoughts, feelings, actions—they’re dynamic, and they change all the time. Unfortunately, in psychiatry, we get a snapshot once a month, at best,” he says. “Incorporating this type of information really allows us to get a more comprehensive, more contextual understanding of somebody’s life.”

Researchers still have a long way to go in designing these algorithms and figuring out how to implement them ethically. But Birnbaum is hopeful that, in the next five to 10 years, social media data could become a normal part of psychiatric practice. “One day, digital data and mental health will really combine,” he says. “And this will be our X-ray into somebody’s mind. This will be our blood test to help support the diagnoses and the interventions that we recommend.”