Researchers Harness the Power of Smartphones to Identify Symptoms of Early Dementia
Digital technology in development at Dartmouth may slow the progress of diseases such as Alzheimer’s with earlier intervention.
Billions of people around the world stare at their smartphones every day. Is it possible for smartphones to look back at their users, watching their faces and following the movement of their fingers on the screen, to flag possible symptoms of Alzheimer’s, Parkinson’s, and other serious neurological disorders?
Karen Fortuna, an assistant professor of psychiatry at the Geisel School of Medicine, and Julia Hill ’24 are spearheading development of a smartphone app that will do just that. The app, called RealVision, tracks how users interact with their phones, noting such changes as disoriented eye movement, difficulty with typing, or needing more time to respond to prompts.
Fortuna emphasizes that the RealVision app can’t determine whether an individual is suffering from a neurodegenerative disease, but the app’s ability to track how a person interacts with technology may identify symptoms before they, or their doctors, realize the symptoms. With earlier intervention, there are pharmacological and behavioral modifications that can potentially slow the progression of these diseases.
“The technology collects data passively. From that, we’re creating a score and correlating that with risks for diseases such as Alzheimer’s,” says Fortuna. “The long-term goal is to partner with spouses, daughters, sons, and caregivers—the people most likely to observe a loved one’s changing behavior—and tap into the power of these relationships to encourage someone to download the app and potentially identify pre-clinical Alzheimer’s early on. We’re developing AI-powered strategies around different types of digital markers. For example, we’re looking at facial cues as an early predictor.
“Again, all this is to identify risk. The technology can’t say, ‘You have this or you don’t have this.’ But, this technology can get people into the clinic to begin testing and start treatment, if needed.”
Dartmouth fosters the creation of innovative technology such as RealVision, as multiple Dartmouth faculty at Geisel, Thayer School of Engineering, and the Faculty of Arts and Sciences are national leaders in the development of mobile digital technology that can improve human health. Fortuna has partnered with faculty member Elizabeth Murnane at Thayer, whom she met through their work with the Dartmouth Innovation Accelerator for Digital Health, a partnership between the Magnuson Center for Entrepreneurship and the Center for Technology and Behavioral Health.
“I can come up with the idea and tap into the power of humans to promote engagement with technology, but I can’t create the technology. I need help to bring it to market. I need engineers and business thinkers. Fortunately, we have some of the brightest people at Thayer, the Magnuson Center, and the Tuck School of Business,” she says.
The concept for RealVision emerged from Fortuna’s participation in a discovery fellowship funded by the New York Academy of Sciences. Fortuna, a social worker, was teamed with a neurologist and physicist and challenged to “create the next greatest discovery.” Her team developed the idea for what would become RealVision. At the conclusion of the exercise, Fortuna’s two teammates chose not to pursue the project any further, and she decided to run with it.
Fortuna, Hill, and colleagues created an early version of the technology, gathered patient partners and caregivers of people with dementia, and asked them to interact with the app. Their feedback led to modifications and research into ethical issues such as confidentiality—“We have no interest in selling data to a third party, RealVision is for people who need this technology,” says Fortuna. That work was funded through a Sayles Research Grant awarded to Hill by the Dartmouth’s Ethics Institute, which led to more grant applications and conversations with investors.
As part of their RealVision research, Fortuna and Hill have had papers published in leading academic journals, including an article in the Journal of Technology in Behavioral Science with Hill as the lead author.
Another major step forward came in the form of a 2023 pitch competition sponsored by the Dartmouth Innovation Accelerator for Digital Health. Fortuna couldn’t attend the final competition due to an engagement in Washington, D.C., so she trained Hill to give the final pitch.
Following that success, Fortuna developed a partnership with Murnane's Empower Lab, with engineering students building out AI algorithms based on predictive digital behaviors.
Real-world testing involving racially diverse populations will begin soon, says Fortuna.
“What is unique about RealVision is that we also have partnerships with caregivers, patients, advocacy groups, and a large payer system who are helping us determine what is important to them,” she says. “As we collect data to develop the AI algorithms, we will engage in a continuous improvement cycle to improve the algorithms and RealVision’s predictive validity of Alzheimer’s disease and related dementias. If we can identify these symptoms early using RealVision, we can start helping people and their families.”
In Conversation with Karen Fortuna and Julia Hill
Working with partners across the Dartmouth campus, Fortuna and Hill are developing an app called RealVision to detect symptoms of disorders such as Alzheimer’s disease and related dementias by tracking how individuals use their smartphones.