arXiv subjects

Richard A. Berk

Publications and source records attributed to Richard A. Berk.

12 recordsLinked to original sources

arXiv2025

Forecasting Extreme High Summer Temperatures in Paris and Cairo Using Gradient Boosting and Conformal Prediction Regions

In this paper, gradient boosting is used to forecast the Q(.95) values of air temperature and the Steadman Heat Index. Paris, France during late the spring and summer months is the major focus. Predictors and responses are drawn from the Paris-Montsouris weather station for the years 2018 through 2024. Q(.95) values are used because of interest in summer heat that is statistically rare and extreme. The data are curated as a multiple time series for each year. Predictors include seven routinely collected indicators of weather conditions. They each are lagged by 14 days such that temperature and heat index forecasts are provided two weeks in advance. Forecasting uncertainty is addressed with conformal prediction regions. Forecasting accuracy is promising. Cairo, Egypt is a second location using data from the weather station at the Cairo Internal Airport over the same years and months. Cairo is a more challenging setting for temperature forecasting because its desert climate can create abrupt and erratic temperature changes. Yet, there is some progress forecasting record-setting hot days.

stat.AP↗

arXiv2025

Forecasting Extreme Temperatures in Siberia Using Supervised Learning and Conformal Prediction Regions

In this paper, we step back from a variety of competing heat wave definitions and forecast directly unusually high temperatures. Our testbed is the Russian Far East in the summers of 2022 and 2023. Remotely sensed data from NASA's Aqua spacecraft are organized into a within-subject design that can reduce nuisance variation in forecasted temperatures. Spatial grid cells are the study units. Each is exposed to precursors of a faux heat wave in 2022 and to precursors of a reported heat wave in 2023. The precursors are used to forecast temperatures two weeks in the future for each of 31 consecutive days. Algorithmic fitting procedures produce forecasts with promise and relatively small conformal prediction regions having a coverage probability of at least .75. Spatial and temporal dependence are manageable. At worst, there is weak dependence such that conformal prediction inference is only asymptotically valid.

stat.AP↗

arXiv2024

Algorithmic Forecasting of Extreme Heat Waves

This paper provides some foundations for valid forecasting of rare and extreme heat waves through a better understanding of the similarities and differences between several consecutive hot days under normal circumstances and rare, extreme heat waves. We analyze AIRS data from the American Pacific Northwest and AIRS data from the Phoenix, Arizona region. A genetic algorithm is used to help determine the most promising predictors. Classification accuracy with supervised learning is excellent for the Pacific Northwest and is replicated for Phoenix. Conformal prediction sets are considered as a way to represent forecasting uncertainty. Complications caused by endogenous sampling are discussed.

stat.AP↗

arXiv2022

Improving Fairness in Criminal Justice Algorithmic Risk Assessments Using Optimal Transport and Conformal Prediction Sets

In the United States and elsewhere, risk assessment algorithms are being used to help inform criminal justice decision-makers. A common intent is to forecast an offender's ``future dangerousness.'' Such algorithms have been correctly criticized for potential unfairness, and there is an active cottage industry trying to make repairs. In this paper, we use counterfactual reasoning to consider the prospects for improved fairness when members of a less privileged group are treated by a risk algorithm as if they are members of a more privileged group. We combine a machine learning classifier trained in a novel manner with an optimal transport adjustment for the relevant joint probability distributions, which together provide a constructive response to claims of bias-in-bias-out. A key distinction is between fairness claims that are empirically testable and fairness claims that are not. We then use confusion tables and conformal prediction sets to evaluate achieved fairness for projected risk. Our data are a random sample of 300,000 offenders at their arraignments for a large metropolitan area in the United States during which decisions to release or detain are made. We show that substantial improvement in fairness can be achieved consistent with a Pareto improvement for protected groups.

stat.AP↗

arXiv2021

Post-Model-Selection Statistical Inference with Interrupted Time Series Designs: An Evaluation of an Assault Weapons Ban in California

There have been many claims in the media and a bit of respectable research about the causes of variation in firearm sales. The challenges for causal inference can be quite daunting. This paper reports an analysis of daily handgun sales in California from 1996 through 2018 using an interrupted time series design and analysis. The design was introduced to social scientists in 1963 by Campbell and Stanley, analysis methods were proposed by Box and Tiao in 1975, and more recent treatments are easily found (Box et al., 2016). But this approach to causal inference can be badly overmatched by the data on handgun sales, especially when the causal effects are estimated. More important for this paper are fundamental oversights in the standard statistical methods employed. Test multiplicity problems are introduced by adaptive model selection built into recommended practice. The challenges are computational and conceptual. Some progress is made on both problems that arguably improves on past research, but the take-home message may be to reduce aspirations about what can be learned.

stat.ME↗

arXiv2021

Improving Fairness in Criminal Justice Algorithmic Risk Assessments Using Conformal Prediction Sets

Risk assessment algorithms have been correctly criticized for potential unfairness, and there is an active cottage industry trying to make repairs. In this paper, we adopt a framework from conformal prediction sets to remove unfairness from risk algorithms themselves and the covariates used for forecasting. From a sample of 300,000 offenders at their arraignments, we construct a confusion table and its derived measures of fairness that are effectively free any meaningful differences between Black and White offenders. We also produce fair forecasts for individual offenders coupled with valid probability guarantees that the forecasted outcome is the true outcome. We see our work as a demonstration of concept for application in a wide variety of criminal justice decisions. The procedures provided can be routinely implemented in jurisdictions with the usual criminal justice datasets used by administrators. The requisite procedures can be found in the scripting software R. However, whether stakeholders will accept our approach as a means to achieve risk assessment fairness is unknown. There also are legal issues that would need to be resolved although we offer a Pareto improvement.

stat.AP↗

arXiv2021

Nested Conformal Prediction Sets for Classification with Applications to Probation Data

Risk assessments to help inform criminal justice decisions have been used in the United States since the 1920s. Over the past several years, statistical learning risk algorithms have been introduced amid much controversy about fairness, transparency and accuracy. In this paper, we focus on accuracy for a large department of probation and parole that is considering a major revision of its current, statistical learning risk methods. Because the content of each offender's supervision is substantially shaped by a forecast of subsequent conduct, forecasts have real consequences. Here we consider the probability that risk forecasts are correct. We augment standard statistical learning estimates of forecasting uncertainty (i.e., confusion tables) with uncertainty estimates from nested conformal prediction sets. In a demonstration of concept using data from the department of probation and parole, we show that the standard uncertainty measures and uncertainty measures from nested conformal prediction sets can differ dramatically in concept and output. We also provide a modification of nested conformal called the localized conformal method to match confusion tables more closely when possible. A strong case can be made favoring the nested and localized conformal approach. As best we can tell, our formulation of such comparisons and consequent recommendations is novel.

stat.ME↗

arXiv2019

Almost Politically Acceptable Criminal Justice Risk Assessment

In criminal justice risk forecasting, one can prove that it is impossible to optimize accuracy and fairness at the same time. One can also prove that it is impossible optimize at once all of the usual group definitions of fairness. In the policy arena, one is left with tradeoffs about which many stakeholders will adamantly disagree. In this paper, we offer a different approach. We do not seek perfectly accurate and fair risk assessments. We seek politically acceptable risk assessments. We describe and apply to data on 300,000 offenders a machine learning approach that responds to many of the most visible charges of "racial bias." Regardless of whether such claims are true, we adjust our procedures to compensate. We begin by training the algorithm on White offenders only and computing risk with test data separately for White offenders and Black offenders. Thus, the fitted algorithm structure is exactly the same for both groups; the algorithm treats all offenders as if they are White. But because White and Black offenders can bring different predictors distributions to the white-trained algorithm, we provide additional adjustments if needed. Insofar are conventional machine learning procedures do not produce accuracy and fairness that some stakeholders require, it is possible to alter conventional practice to respond explicitly to many salient stakeholder claims even if they are unsupported by the facts. The results can be a politically acceptable risk assessment tools.

stat.AP↗

arXiv2019

An Algorithmic Approach to Forecasting Rare Violent Events: An Illustration Based in IPV Perpetration

Mass violence, almost no matter how defined, is (thankfully) rare. Rare events are very difficult to study in a systematic manner. Standard statistical procedures can fail badly and usefully accurate forecasts of rare events often are little more than an aspiration. We offer an unconventional approach for the statistical analysis of rare events illustrated by an extensive case study. We report research whose goal is to learn about the attributes of very high risk IPV perpetrators and the circumstances associated with their IPV incidents reported to the police. Very high risk is defined as having a high probability of committing a repeat IPV assault in which the victim is injured. Such individuals represent a very small fraction of all IPV perpetrators; these acts of violence are relatively rare. To learn about them nevertheless, we apply in a novel fashion three algorithms sequentially to data collected from a large metropolitan police department: stochastic gradient boosting, a genetic algorithm inspired by natural selection, and agglomerative clustering. We try to characterize not just perpetrators who on balance are predicted to re-offend, but who are very likely to re-offend in a manner that leads to victim injuries. With this strategy, we learn a lot. We also provide a new way to estimate the importance of risk predictors. There are lessons for the study of other rare forms of violence especially when instructive forecasts are sought. In the absence of sufficiently accurate forecasts, scarce prevention resources cannot be allocated where they are most needed.

stat.AP↗

arXiv2018

Using Recursive Partitioning to Find and Estimate Heterogenous Treatment Effects In Randomized Clinical Trials

Heterogeneous treatment effects can be very important in the analysis of randomized clinical trials. Heightened risks or enhanced benefits may exist for particular subsets of study subjects. When the heterogeneous treatment effects are specified as the research is being designed, there are proper and readily available analysis techniques. When the heterogeneous treatment effects are inductively obtained as an experiment's data are analyzed, significant complications are introduced. There can be a need for special loss functions designed to find local average treatment effects and for techniques that properly address post selection statistical inference. In this paper, we tackle both while undertaking a recursive partitioning analysis of a randomized clinical trial testing whether individuals on probation, who are low risk, can be minimally supervised with no increase in recidivism.

stat.ME↗

arXiv2018

Assumption Lean Regression

It is well known that models used in conventional regression analysis are commonly misspecified. A standard response is little more than a shrug. Data analysts invoke Box's maxim that all models are wrong and then proceed as if the results are useful nevertheless. In this paper, we provide an alternative. Regression models are treated explicitly as approximations of a true response surface that can have a number of desirable statistical properties, including estimates that are asymptotically unbiased. Valid statistical inference follows. We generalize the formulation to include regression functionals, which broadens substantially the range of potential applications. An empirical application is provided to illustrate the paper's key concepts.

stat.ME↗

arXiv2017

Fairness in Criminal Justice Risk Assessments: The State of the Art

Objectives: Discussions of fairness in criminal justice risk assessments typically lack conceptual precision. Rhetoric too often substitutes for careful analysis. In this paper, we seek to clarify the tradeoffs between different kinds of fairness and between fairness and accuracy. Methods: We draw on the existing literatures in criminology, computer science and statistics to provide an integrated examination of fairness and accuracy in criminal justice risk assessments. We also provide an empirical illustration using data from arraignments. Results: We show that there are at least six kinds of fairness, some of which are incompatible with one another and with accuracy. Conclusions: Except in trivial cases, it is impossible to maximize accuracy and fairness at the same time, and impossible simultaneously to satisfy all kinds of fairness. In practice, a major complication is different base rates across different legally protected groups. There is a need to consider challenging tradeoffs.

stat.ML↗