Not her first rodeo

“They don’t trust the system”

Take, for example, an AI tool that uses racial identity as a parameter. Tell the AI that a patient is Latinx, or speaks a language other than English, and the AI will factor that into its risk prediction—but the original training data included so few Latinx patients that this data point can have outsized and unpredictable implications.  

At the root of this problem, Hernandez-Boussard says, is the fact that race is a social construct without any firm genetic basis. As she puts it, “There’s more diversity within a race than there are between races.”

At the same time, our socially constructed racial identities have real impact on people’s lives, such that social determinants of health, like income and education, tend to cluster around them. So, when an AI model produces medical treatment outcome probabilities based in part on racial identity, it is really using racial identity as a vague and unreliable proxy for something that hasn’t actually been measured—and for which there is no data.

To illustrate this point, Hernandez-Boussard describes an AI-driven risk calculator for adverse outcomes in Cesarean birth among women who have previously given birth vaginally. This calculator included being Black as a risk factor—even though it wasn’t. The real risk factor, for which race merely served as a proxy, was chronic hypertension. Chronic hypertension data hadn’t been collected, though. Race had.

And so race, instead of hypertension, became the parameter. This meant that the calculator might generate an inaccurate—and excessively high—risk of complications from Cesarean delivery for a Black mother who did not have hypertension. It might also generate a misleadingly low risk for a white mother who did have hypertension, putting both women in danger.

There is a very real concern, Hernandez-Boussard explains, that such problems will worsen as AI moves further into health care—especially in rural areas with marginalized populations like Bishop. These areas often operate as sort of “health care deserts” that put huge burdens on the providers who do serve them. Hernandez-Boussard worries that, in that context, the allure of AI to save time and money will mean it’s deployed for populations most likely to be underrepresented in the training data—and, therefore, most likely to receive harmfully inaccurate diagnoses and outcome predictions. 

That means that one of the great social justice challenges in AI research lies in Hernandez-Boussard’s field. Researchers have long known that medical datasets come overwhelmingly from the white majority population most likely to fill out questionnaires. Previous efforts to correct this have used the same data collection methods—and failed.

“They don’t trust the system,” Hernandez-Boussard says, explaining why people in marginalized populations resist sharing data. “We’ve harmed these populations many, many times.” She cited the infamous Tuskegee Trials of the mid-20th century, when the U.S. Public Health Service studied 399 Black men with syphilis by deliberately not treating them, despite the availability of effective medication.

What it takes

Turning things around demands a new approach—for which Hernandez-Boussard is uniquely qualified. In her latest role as associate dean of research for the School of Medicine, Hernandez-Boussard focuses on educating faculty across the university in the equitable application of AI. She works with assorted teams at the School of Medicine to ensure appropriate application of existing AI diagnostics for patients from marginalized populations. Finally, Hernandez-Boussard has become a thought leader in the national conversation about equitable AI, working with diverse stakeholders to develop standards and guidelines in the future development of AI health care tools.

Throughout this work, Hernandez-Boussard draws frequently on lessons learned in Bishop.  

Hernandez-Boussard cites research showing that the process of collecting data from marginalized populations often works best in places like barbershops and churches, where people naturally congregate and feel comfortable. That has her thinking of her hometown—places like the Bishop Paiute and Shoshone Indian Reservation, restaurants like Whiskey Creek and El Charro, or the city park. 

Most of all, when Hernandez-Boussard thinks about gathering data in rural communities like her own, she thinks about a big annual tradition in Bishop called Mule Days, held every summer with country music bands, rodeo, and more than 700 mules and trainers competing in cattle working, mule jumping, and chariot racing. 

“Mule Days is something I would go to,” she says. “Because everyone comes out and there’s Indian tacos—that’s the big thing in Bishop, like Indian fry bread made into a taco. You get out to a place like that, in person, and make those human connections.”

Hernandez-Boussard has already begun. “We did a program trying to engage with Native Americans,” she says, “so I just called one of my girlfriends at the Indian Health Center. I'm like, ‘Want to do this with me?’ And she said, ‘Absolutely!’ That's the way to get engaged, knowing somebody who will let you into the community—rather than just sending out emails that say, ‘Come down to my office and I’ll give you $25 to take a survey.’ They’re never going to do that. You have to show that you want more than just their data and you’re going to do good with it.”

The stakes are high, says Hernandez-Boussard, when it comes to the equitable engineering of health care AI.

“If we get this right,” she says, “some of these underrepresented populations can benefit the most. We can really close the racial gap in health outcomes. If we don’t, we’re widening that gap. It's all about building the trust to make responsible use of AI.”