arXiv subjects

Emma Pierson

Publications and source records attributed to Emma Pierson.

47 records · Page 3Linked to original sources

arXiv2020

Concept Bottleneck Models

We seek to learn models that we can interact with using high-level concepts: if the model did not think there was a bone spur in the x-ray, would it still predict severe arthritis? State-of-the-art models today do not typically support the manipulation of concepts like "the existence of bone spurs", as they are trained end-to-end to go directly from raw input (e.g., pixels) to output (e.g., arthritis severity). We revisit the classic idea of first predicting concepts that are provided at training time, and then using these concepts to predict the label. By construction, we can intervene on these concept bottleneck models by editing their predicted concept values and propagating these changes to the final prediction. On x-ray grading and bird identification, concept bottleneck models achieve competitive accuracy with standard end-to-end models, while enabling interpretation in terms of high-level clinical concepts ("bone spurs") or bird attributes ("wing color"). These models also allow for richer human-model interaction: accuracy improves significantly if we can correct model mistakes on concepts at test time.

cs.LG↗

arXiv2020

Assessing racial inequality in COVID-19 testing with Bayesian threshold tests

There are racial disparities in the COVID-19 test positivity rate, suggesting that minorities may be under-tested. Here, drawing on the literature on statistically assessing racial disparities in policing, we 1) illuminate a statistical flaw, known as infra-marginality, in using the positivity rate as a metric for assessing racial disparities in under-testing; 2) develop a new type of Bayesian threshold test to measure disparities in COVID-19 testing and 3) apply the test to measure racial disparities in testing thresholds in a real-world COVID-19 dataset.

stat.AP↗

arXiv2020

Ethical Machine Learning in Health Care

The use of machine learning (ML) in health care raises numerous ethical concerns, especially as models can amplify existing health inequities. Here, we outline ethical considerations for equitable ML in the advancement of health care. Specifically, we frame ethics of ML in health care through the lens of social justice. We describe ongoing efforts and outline challenges in a proposed pipeline of ethical ML in health, ranging from problem selection to post-deployment considerations. We close by summarizing recommendations to address these challenges.

cs.CY↗

arXiv2019

Predicting pregnancy using large-scale data from a women's health tracking mobile application

Predicting pregnancy has been a fundamental problem in women's health for more than 50 years. Previous datasets have been collected via carefully curated medical studies, but the recent growth of women's health tracking mobile apps offers potential for reaching a much broader population. However, the feasibility of predicting pregnancy from mobile health tracking data is unclear. Here we develop four models -- a logistic regression model, and 3 LSTM models -- to predict a woman's probability of becoming pregnant using data from a women's health tracking app, Clue by BioWink GmbH. Evaluating our models on a dataset of 79 million logs from 65,276 women with ground truth pregnancy test data, we show that our predicted pregnancy probabilities meaningfully stratify women: women in the top 10% of predicted probabilities have a 89% chance of becoming pregnant over 6 menstrual cycles, as compared to a 27% chance for women in the bottom 10%. We develop a technique for extracting interpretable time trends from our deep learning models, and show these trends are consistent with previous fertility research. Our findings illustrate the potential that women's health tracking data offers for predicting pregnancy on a broader population; we conclude by discussing the steps needed to fulfill this potential.

stat.AP↗

arXiv2019

Inferring Multidimensional Rates of Aging from Cross-Sectional Data

Modeling how individuals evolve over time is a fundamental problem in the natural and social sciences. However, existing datasets are often cross-sectional with each individual observed only once, making it impossible to apply traditional time-series methods. Motivated by the study of human aging, we present an interpretable latent-variable model that learns temporal dynamics from cross-sectional data. Our model represents each individual's features over time as a nonlinear function of a low-dimensional, linearly-evolving latent state. We prove that when this nonlinear function is constrained to be order-isomorphic, the model family is identifiable solely from cross-sectional data provided the distribution of time-independent variation is known. On the UK Biobank human health dataset, our model reconstructs the observed data while learning interpretable rates of aging associated with diseases, mortality, and aging risk factors.

cs.LG↗

arXiv2018

Modeling Individual Cyclic Variation in Human Behavior

Cycles are fundamental to human health and behavior. However, modeling cycles in time series data is challenging because in most cases the cycles are not labeled or directly observed and need to be inferred from multidimensional measurements taken over time. Here, we present CyHMMs, a cyclic hidden Markov model method for detecting and modeling cycles in a collection of multidimensional heterogeneous time series data. In contrast to previous cycle modeling methods, CyHMMs deal with a number of challenges encountered in modeling real-world cycles: they can model multivariate data with discrete and continuous dimensions; they explicitly model and are robust to missing data; and they can share information across individuals to model variation both within and between individual time series. Experiments on synthetic and real-world health-tracking data demonstrate that CyHMMs infer cycle lengths more accurately than existing methods, with 58% lower error on simulated data and 63% lower error on real-world data compared to the best-performing baseline. CyHMMs can also perform functions which baselines cannot: they can model the progression of individual features/symptoms over the course of the cycle, identify the most variable features, and cluster individual time series into groups with distinct characteristics. Applying CyHMMs to two real-world health-tracking datasets -- of menstrual cycle symptoms and physical activity tracking data -- yields important insights including which symptoms to expect at each point during the cycle. We also find that people fall into several groups with distinct cycle patterns, and that these groups differ along dimensions not provided to the model. For example, by modeling missing data in the menstrual cycles dataset, we are able to discover a medically relevant group of birth control users even though information on birth control is not given to the model.

cs.SI↗

arXiv2018

Fast Threshold Tests for Detecting Discrimination

Threshold tests have recently been proposed as a useful method for detecting bias in lending, hiring, and policing decisions. For example, in the case of credit extensions, these tests aim to estimate the bar for granting loans to white and minority applicants, with a higher inferred threshold for minorities indicative of discrimination. This technique, however, requires fitting a complex Bayesian latent variable model for which inference is often computationally challenging. Here we develop a method for fitting threshold tests that is two orders of magnitude faster than the existing approach, reducing computation from hours to minutes. To achieve these performance gains, we introduce and analyze a flexible family of probability distributions on the interval [0, 1] -- which we call discriminant distributions -- that is computationally efficient to work with. We demonstrate our technique by analyzing 2.7 million police stops of pedestrians in New York City.

stat.ML↗

arXiv2018

Demographics and discussion influence views on algorithmic fairness

The field of algorithmic fairness has highlighted ethical questions which may not have purely technical answers. For example, different algorithmic fairness constraints are often impossible to satisfy simultaneously, and choosing between them requires value judgments about which people may disagree. Achieving consensus on algorithmic fairness will be difficult unless we understand why people disagree in the first place. Here we use a series of surveys to investigate how two factors affect disagreement: demographics and discussion. First, we study whether disagreement on algorithmic fairness questions is caused partially by differences in demographic backgrounds. This is a question of interest because computer science is demographically non-representative. If beliefs about algorithmic fairness correlate with demographics, and algorithm designers are demographically non-representative, decisions made about algorithmic fairness may not reflect the will of the population as a whole. We show, using surveys of three separate populations, that there are gender differences in beliefs about algorithmic fairness. For example, women are less likely to favor including gender as a feature in an algorithm which recommends courses to students if doing so would make female students less likely to be recommended science courses. Second, we investigate whether people's views on algorithmic fairness can be changed by discussion and show, using longitudinal surveys of students in two computer science classes, that they can.

cs.CY↗

arXiv2018

SIMLR: A Tool for Large-Scale Genomic Analyses by Multi-Kernel Learning

We here present SIMLR (Single-cell Interpretation via Multi-kernel LeaRning), an open-source tool that implements a novel framework to learn a sample-to-sample similarity measure from expression data observed for heterogenous samples. SIMLR can be effectively used to perform tasks such as dimension reduction, clustering, and visualization of heterogeneous populations of samples. SIMLR was benchmarked against state-of-the-art methods for these three tasks on several public datasets, showing it to be scalable and capable of greatly improving clustering performance, as well as providing valuable insights by making the data more interpretable via better a visualization. Availability and Implementation SIMLR is available on GitHub in both R and MATLAB implementations. Furthermore, it is also available as an R package on http://bioconductor.org.

q-bio.GN↗

arXiv2017

A large-scale analysis of racial disparities in police stops across the United States

To assess racial disparities in police interactions with the public, we compiled and analyzed a dataset detailing over 60 million state patrol stops conducted in 20 U.S. states between 2011 and 2015. We find that black drivers are stopped more often than white drivers relative to their share of the driving-age population, but that Hispanic drivers are stopped less often than whites. Among stopped drivers -- and after controlling for age, gender, time, and location -- blacks and Hispanics are more likely to be ticketed, searched, and arrested than white drivers. These disparities may reflect differences in driving behavior, and are not necessarily the result of bias. In the case of search decisions, we explicitly test for discrimination by examining both the rate at which drivers are searched and the likelihood searches turn up contraband. We find evidence that the bar for searching black and Hispanic drivers is lower than for searching whites. Finally, we find that legalizing recreational marijuana in Washington and Colorado reduced the total number of searches and misdemeanors for all race groups, though a race gap still persists. We conclude by offering recommendations for improving data collection, analysis, and reporting by law enforcement agencies.

stat.AP↗

arXiv2017

Algorithmic decision making and the cost of fairness

Algorithms are now regularly used to decide whether defendants awaiting trial are too dangerous to be released back into the community. In some cases, black defendants are substantially more likely than white defendants to be incorrectly classified as high risk. To mitigate such disparities, several techniques recently have been proposed to achieve algorithmic fairness. Here we reformulate algorithmic fairness as constrained optimization: the objective is to maximize public safety while satisfying formal fairness constraints designed to reduce racial disparities. We show that for several past definitions of fairness, the optimal algorithms that result require detaining defendants above race-specific risk thresholds. We further show that the optimal unconstrained algorithm requires applying a single, uniform threshold to all defendants. The unconstrained algorithm thus maximizes public safety while also satisfying one important understanding of equality: that all individuals are held to the same standard, irrespective of race. Because the optimal constrained and unconstrained algorithms generally differ, there is tension between improving public safety and satisfying prevailing notions of algorithmic fairness. By examining data from Broward County, Florida, we show that this trade-off can be large in practice. We focus on algorithms for pretrial release decisions, but the principles we discuss apply to other domains, and also to human decision makers carrying out structured decision rules.

cs.CY↗