Thousands of people do not know they have Parkinson’s disease. Eric Horvitz wants them to be able to find out — before the incurable neurodegenerative disorder progresses to its later stages.
In his perfect world, they wouldn’t have to interrupt their daily routines. They could stay in their homes and offices, working on their computers, and their online activity would eventually trigger a message: A visit to the doctor is in order.
In March, Horvitz, PhD ’91, MD ’94, and his colleagues reported for the first time how the digital tracks left by a computer mouse may reveal the telltale involuntary tremors characteristic of Parkinson’s. This information, when analyzed with other data gleaned from an individual’s web searches, could alert that person that she has the disease, enabling her to seek treatment that could improve her quality of life and perhaps extend it.
“It’s a new tool in epidemiology, a formal tool,” says Horvitz. “To me, it’s mind‑bending.”
An intense but amiable 60-year-old computer scientist who in 2017 became the head of Microsoft’s worldwide network of research labs, Horvitz believes that every time people go online, they leave clues that could lead to earlier diagnoses of many serious health conditions. His studies of web searches have yielded insights into identifying pancreatic, lung and breast cancers, detecting dangerous drug interactions and gauging how your sleep is affecting your performance.
Horvitz’s diagnostic insights are rooted in his expertise in artificial intelligence. Over the course of his career, he has been a leader in developing AI to assist people with decision-making. It is this capability that is enabling his use of the web for what he calls “health-related sensing at large scale.” He applies machine learning, a branch of AI, to sift ginormous sets of data and identify patterns that provide a basis for diagnosing diseases. While others are making advances in this area, Horvitz was among the first to recognize its potential, and he remains at its forefront.
There are huge challenges to overcome. Among them: Will these methods hold up when tried beyond online simulations, in real-world settings? And are people ready to share private health information with systems that some liken to Big Brother?
Horvitz knows better than most how AI has overpromised and underdelivered, but he believes that the vast troves of data available on the web and elsewhere have brought us to an inflection point. He and Microsoft colleague Ryen White, along with co-investigators from Stanford and other universities, have examined anonymized data from hundreds of millions of users of Bing, Microsoft’s search engine. They initially looked at query terms — what you search for — and the time and date of searches. Then, they added IP addresses — your computer’s unique identifier — and other location information. Most recently, they have focused on motor movements such as keystrokes, clicks and mouse activity.
This data can reveal critical diagnostic evidence. People confide intimate secrets about their health — yellow skin, odd-looking stools and other curious symptoms — to their search engine that they do not share with others, even physicians, Horvitz says. And the biometric and geographic information picked up by search engines may uncover secrets of which even users are unaware.
Horvitz wants to tap into that well. He describes it as trying “to listen to the whispering of millions of minds,” and there is much to hear.
An AI Pioneer
The idea that someone could discover personal health details through online eavesdropping gives many people pause. “We are in the Wild West of research and operating without clear consensus on how to move forward,” says Camille Nebeker, an assistant professor of behavioral medicine at UC–San Diego who studies the ethics of online medical research. There are complicated questions of what constitutes informed consent, how this health information should be shared and protected, and whether companies should be constrained from profiting from this data and the systems analyzing them.
Horvitz could not agree more. “Eric is one of the leading voices who’s calling out and saying, ‘Hey, we’ve got to think about the impact of what we’re doing,’” says Alan Mackworth, a computer science professor at the University of British Columbia and a leading AI researcher known for designing a soccer-playing robot. “He was really prescient in anticipating the need for studying and getting to policy makers on the impact of AI on society.”
As president of the Association for the Advancement of Artificial Intelligence from 2007 to 2009, Horvitz convened a panel of AI scientists, roboticists, and ethical and legal scholars to report on the risks posed by AI and to propose measures to avert harms. Aware of the need to continue the discussion about AI’s future, Horvitz and his wife, Mary, endowed the One Hundred Year Study on Artificial Intelligence, a program at Stanford in 2014 to appraise AI’s social impact every five years for the next century. At Microsoft he is chair of a committee, AETHER, or AI and Ethics in Engineering and Research, that advises senior leadership on keeping the company’s development and sale of AI consistent with its human rights policy. “I am happy to say this committee has teeth,” he says. Based on the committee’s work, he says, Microsoft has turned down “significant sales” and, in other cases, has written specific limitations into usage agreements.
Horvitz has worked as an artificial intelligence researcher on the Microsoft campus in Redmond, Wash., since finishing his pediatrics rotation at Stanford in 1993. He recalls that final evening in September, driving from the School of Medicine to his new life at Microsoft and “thinking with sadness that I would never be back as a physician — and that I had deeply enjoyed the experience as a medical student.”
While still a student, Horvitz had co-founded Knowledge Industries, where he worked with David Heckerman, MS ’85, PhD ’90, MD ’92, and Jack Breese, PhD ’87, to apply a novel AI approach to assist in such decisions as how best to triage trauma patients or to diagnose problems in jet engines and locomotives. They caught the eye of Nathan Myhrvold, who was then establishing the Microsoft Research Lab and thought they were doing “super interesting” work.
To get them aboard, he acquired the technology of their company. The three were skeptical about making the move to Seattle, but “none more so than Eric,” Myhrvold says. Horvitz begrudgingly agreed to try it for up to six months.
Twenty-five years later, it remains a perfect fit. “Being in a research lab puts you in a world where you are . . . always kind of surfing this wave of the unknown,” Horvitz said in a 2017 podcast about his career. “I’ve always been the kind of person that never got to the end of my ‘whys,’” he said. “My mind is driven to ask questions, and when I come to an answer that I didn’t expect, I get such a burst of pleasure.”
Horvitz grew up in Merrick, on Long Island, the son of two public school teachers. By fifth grade, he knew that he was going to become a scientist. “I remember exactly where I was when I said, ‘Yes. You’ll be doing science,’” he says. “And that was like a done deal.” While majoring in biophysics at SUNY–Binghamton, he became fascinated by neuroscience. He wanted to understand how a tangle of brain cells could produce thought and consciousness.
He chose to move west to pursue his MD/PhD at Stanford; he was drawn by its reputation for innovative thinking, liked how the medical school was part of the main campus, and knew of the university’s excellence in artificial intelligence and, more broadly, computer science. “I had more than an inkling that I’d likely be headed down that affiliated path,” he says.
A friend from those days, Carol Rose, ’83, who now is executive director of the Massachusetts affiliate of the American Civil Liberties Union, recalls how Horvitz zipped around campus in an old MG convertible with his shock of red hair (now gray and receding) going up in every direction. “He talked a million miles an hour and was one of the most creative people I’d ever met,” she says. “His intellect is huge, but he doesn’t take himself too seriously. He was great fun.” Rose now serves on the board of the Partnership on Artificial Intelligence, a consortium of leading AI players Horvitz established in 2017. Their shared concern for ethical use of powerful new technologies “has been a basis of our friendship all of these years,” she says.
At Stanford, Horvitz became interested in using computing to assist with medical diagnoses and decision-making in general. He was among the pioneers of an approach to AI and machine learning built on Bayesian statistics. The Bayesian approach deals explicitly with uncertainty by using probability calculations to weigh the appropriateness of different responses to multiple decisions and to incorporate new information as it becomes available. This was a marked contrast to the prevailing systems that relied on fixed “if-then-else” statements and presumed a closed world.
“The other approaches to AI were considered much sexier and more people worked on them, in part because they were easier to do,” says Myhrvold. “The Bayesian thing was quite difficult.”
Horvitz’s work at the end of the 1980s coincided with the arrival of an “AI winter.” Funding for research in the field disappeared when its proponents failed to deliver the thinking machines they had promised.
Conventional approaches to AI and machine learning were too brittle to deal with the many uncertainties that arise in a real-world setting. But the Bayesian approach was ahead of its time. Data sets weren’t yet big enough for machine learning and computers weren’t yet sophisticated enough to handle the complexity of reasoning required to account for uncertainty.
“Eric was brave enough to keep working on it through that period,” says Myhrvold. “And, of course, now it’s hotter than hot.”
Horvitz’s achievements have garnered him two preeminent honors in the field of artificial intelligence: the Feigenbaum Prize and the Allen Newell Award. He holds nearly 300 patents, and a recent company video describes him as “Microsoft’s top inventor.” His work has been integrated into Microsoft Office, Windows and cloud services. He designed a new way to predict cholera epidemics, contributed to a system that helps characterize galaxies, applied machine learning to forecast traffic congestion, invented a robot that could host Jeopardy! and much, much more.
Disease Detectives
An unexpected phone call in 2004 spurred Horvitz to pursue a long-held interest: looking for a way to warn people who are unaware they are ill. A childhood friend, Ron Nadel, called Horvitz after seeing him interviewed on Charlie Rose about AI. As they talked, Nadel complained about itching everywhere.
Horvitz asked, “Any yellowing in your eyes?” Nadel said there was a bit. Abdominal pain? Yes. Horvitz encouraged his friend to go to the doctor immediately and convey those three symptoms. Within the month, Nadel was diagnosed with the disease Horvitz had feared: pancreatic cancer. Nadel died in less than a year.
Pancreatic cancer is almost always fatal within five years. It’s difficult to detect in the initial stages, but survival rates improve slightly when patients are diagnosed early. Horvitz mulled how to help people find out sooner, and gradually he and Microsoft colleague White, along with a graduate student, hatched a study.
The first step was to create a data set of Bing users whose search queries strongly suggested that they had recently been diagnosed with pancreatic cancer. (Bing’s service terms advise users that their anonymized log data may be included in research projects.) To identify which of the 6.4 million users had the disease, the researchers sifted through their search logs for queries like “I was just diagnosed with pancreatic cancer” and “pancreatic cancer, how long will I live.” By looking at the sorts of queries that followed — end-of-life plans, pathology reports, therapy side effects — they had evidence that certain people had the disease.
The researchers then went back several months in those users’ search logs, applying machine learning to identify patterns of symptoms they searched for before the moment of diagnosis. The computation and analysis weighed how different factors — including the number and frequency of relevant queries, the type of symptoms and demographic risk factors — influenced the likelihood of a diagnosis. The prediction model that emerged from this work, published in the Journal of Oncology Practice, identified 5 percent to 15 percent of those whose searches eventually revealed a diagnosis while making very few errors.
Horvitz and White have since extended this approach in several other studies. Later in 2016, they devised a prediction model for lung cancer. This one added geographic data, drawing from the search log information about where a user’s signal originated, allowing the researchers to examine such possible risks as in-home radon exposure and frequent air travel.
The next year, the researchers, along with Stanford computer science graduate student Tim Althoff and Stanford professor of psychiatry and behavioral sciences Jamie Zeitzer, conducted what they describe as “the largest prospective study of real-world human performance and sleep to date”: 31,000 participants who produced 75 million keystrokes and clicks, and logged more than 3 million nights of sleep over 18 months. The project combined data from Bing search logs with information on sleep from consenting users’ Microsoft Band wearable fitness devices. Instead of looking at search query terms, it analyzed changes in the speed of an individual’s keystrokes and clicks, down to the submillisecond.
Zeitzer, who specializes in sleep medicine, recalls when he was approached to be a co-author. “Honestly, my first response was a little incredulous: Why would they have stored all of that data?” he says. He was stunned when the results demonstrated a strong relationship between hours of sleep, hours since waking up, and the speed of typing and clicking. One dramatic finding: Those who slept less than six hours on two consecutive nights were sluggish for the next six days.
“It provides an unparalleled window into people’s health that we’ve never had before,” Zeitzer says. “I do think the future in medicine is this longitudinal, passive monitoring where you’re looking at people’s behavior and health patterns and where you can identify changes in trajectory early on.”
During a meeting with Althoff and White about the sleep study, Horvitz was hit with a breakthrough idea: going beyond keystrokes and clicks to examine other motor movements. “I’ve got to sit down — this is big!” Althoff recalls Horvitz saying. What emerged was a plan to gather a new type of web signal — cursor-movement data, stored on Microsoft servers, from Bing searchers. The researchers posited that certain cursor trails could be used as evidence of tremors.
In the study that followed, Horvitz and White reviewed 18 months of anonymized Bing logs from more than 31 million searchers to see if these signals could someday enable physicians to more easily diagnose Parkinson’s disease and other neurodegenerative disorders. They define as a digital proxy for a tremor “horizontal or vertical oscillations in cursor position up to 20 pixels in each direction.” The researchers tallied the number of tremors, the average tremor frequency and several measures of cursor activity. They pooled this data with an analysis of search queries similar to those used in previous papers.
A report of the findings by Horvitz, White and another co-author was published in April in NPJ Digital Medicine. The key finding: While they could detect Parkinson’s disease cases without using the data about cursor movements, they could do so more effectively if they included it.
Horvitz and his colleagues have been studying how this approach can be applied to other neurodegenerative diseases such as Alzheimer’s, and their preliminary analysis suggests that it has promise.
The Foreseeable Future
Walk into Building 99, the home of the research labs on the Microsoft campus, and the elevator senses your approach, opening automatically like the doors on the Starship Enterprise. Exit on the third floor, and a robot is there to give directions. Arrive at Horvitz’s office, and an AI personal assistant with an animated face may greet you, advise you as to Horvitz’s availability and help you schedule an appointment.
Despite the futuristic elements around him, Horvitz says AI has yet to be adopted in modern medical practice. “If you told me in 1988, when I was a grad student, that, ‘Here’s 2018, and there’s very little AI medicine,’ and so on, I’d say, ‘What’s going on?’ I’d be shocked,” he says.
The day before, Horvitz was in Denver, delivering the keynote Malcolm Peterson Lecture before an audience of about 3,000 physicians at the annual meeting of the Society of General Internal Medicine. The message of his talk, “AI Aspirations, Healthcare Futures,” is that the new technologies are meant to complement a physician’s work, not replace it. He cites one finding that expert pathologists performed better than a computer in diagnosing metastatic breast cancer. But working together, man and machine reduced errors from 3.4 percent to 0.5 percent, a decrease of about 85 percent.
Not everyone in the audience responds enthusiastically. “Beware of the hype,” Gordon Schiff, a general internist and quality and safety director for the Harvard Medical School Center for Primary Care, says in the Q&A that follows. His criticism turns in large part on the way electronic records have been integrated into health care — which is within neither the realm of Microsoft products nor Horvitz’s research, but which reflects a larger skepticism of the solutions that the tech industry is providing to doctors. “The life of a primary care physician has become immeasurably worse,” Schiff says. “There is insufficient attention to the things that really matter to our lives and our patients.”
Horvitz is unfazed. “Short-term frustration is, I think, to be expected,” he responds. “Everybody in this room I’m sure believes in their hearts that in the long term that’s the way to go.”
Back at Microsoft, Horvitz is brimming with enthusiasm. “I got a chance to talk to doctors, which was fabulous!” he says. “It was really a great day.” He believes that if doctors embrace the coming innovations, they will be able to focus more on connecting with their patients and less on reviewing medical tests and analyzing how the case fits into existing knowledge.
But technologies that rely on search data to improve diagnosis are not ready for prime time. As Horvitz and his colleagues acknowledge in their pancreatic cancer study, they don’t actually know that someone who searches for “just diagnosed with pancreatic cancer” indeed has the disease: “We lack explicit ground truth about diagnoses and rely on implicit self-reporting in queries.” Equally important, they have not been able to confirm whether their method for estimating false positives is accurate. “The question is, do we know enough yet to not send tens of thousands of people running to their doctor for a rule‑out?” Horvitz says.
He is in talks with oncologists and cancer researchers about opening a clinical trial. Only after the researchers’ method has been rigorously tested and reviewed, with confirmed successes, could they seriously undertake the next stage: Making it widely available.
That step may not prove as difficult.
Horvitz envisions an opt-in service that could run on your computer or mobile device as a personal detector. If it discovered that you had searched for symptoms of a particular disease, it could either create talking points for you to take to your doctor or alert your physician directly.
“If we built a detector with our web logs and web data — private, you know, with [human subjects] approval and proper de‑identification and anonymization — could I take that filter and run it on your cell phone?” Horvitz asks. “So it’s completely running in the privacy of your own device? And doing screening for you?
“That’s all doable,” he says. “Totally feasible.”
Jonathan Rabinovitz is a writer and an editor who lives in the Seattle area.
