arXiv subjects

Zhenbang Jiao

Publications and source records attributed to Zhenbang Jiao.

6 recordsLinked to original sources

arXiv2025

Policy-Oriented Binary Classification: Improving (KD-)CART Final Splits for Subpopulation Targeting

Policymakers often use recursive binary split rules to partition populations based on binary outcomes and target subpopulations whose probability of the binary event exceeds a threshold. We call such problems Latent Probability Classification (LPC). Practitioners typically employ Classification and Regression Trees (CART) for LPC. We prove that in the context of LPC, classic CART and the knowledge distillation method, whose student model is a CART (referred to as KD-CART), are suboptimal. We propose Maximizing Distance Final Split (MDFS), which generates split rules that strictly dominate CART/KD-CART under the unique intersect assumption. MDFS identifies the unique best split rule, is consistent, and targets more vulnerable subpopulations than CART/KD-CART. To relax the unique intersect assumption, we additionally propose Penalized Final Split (PFS) and weighted Empirical risk Final Split (wEFS). Through extensive simulation studies, we demonstrate that the proposed methods predominantly outperform CART/KD-CART. When applied to real-world datasets, MDFS generates policies that target more vulnerable subpopulations than the CART/KD-CART.

stat.ML↗

arXiv2025

Balancing Efficiency and Equity in Classroom Assignment under Endogenous Peer Effects

This paper presents a three-step empirical framework for optimizing classroom assignments under endogenous peer effects, using data from the China Education Panel Survey (CEPS). We design \textit{PeerNN}, a neural network that mimics endogenous network formation as a discrete choice model, generating a friendship-intensity matrix ($Ω$) that captures student popularity. \textbf{Step 2: Estimating Peer Effects.} We measure the peer effect friends' average 6th-grade class rank weighted by $Ω$ on 8th-grade cognitive test score. Incorporating $Ω$ into the linear-in-means model induces endogeneity. Using quasi-random classroom assignments, we instrument friends' average 6th-grade class rank with the average classmates' 6th-grade class rank (unweighted by $Ω$). Our main regression result shows that a 10\% improvement in friends' 6th-grade class rank raises 8th-grade cognitive test scores by 0.13 SD. Positive $β$ implies maximizing (minimizing) the popularity of high (low) achievers optimizes outcomes. \textbf{Step 3: Simulating Policy Trade-offs.} We use estimates from Step 1 and Step 2 to simulate optimal classroom assignments. We first implement a genetic algorithm (GA) to maximize average peer effect and observe a 1.9\% improvement. However, serious inequity issues arise: low-achieving students are hurt the most in the pursuit of the higher average peer effect. We propose an \textit{Algorithmically Fair GA} (AFGA), achieving a 1.2\% gain while ensuring more equitable educational outcomes. These results underscore that efficiency-focused classroom assignment policies can exacerbate inequality. We recommend incorporating fairness considerations when designing classroom assignment policies that account for endogenous spillovers.

econ.GN↗

arXiv2024

Assessment of Case Influence in the Lasso with a Case-weight Adjusted Solution Path

We study case influence in the Lasso regression using Cook's distance which measures overall change in the fitted values when one observation is deleted. Unlike in ordinary least squares regression, the estimated coefficients in the Lasso do not have a closed form due to the nondifferentiability of the $\ell_1$ penalty, and neither does Cook's distance. To find the case-deleted Lasso solution without refitting the model, we approach it from the full data solution by introducing a weight parameter ranging from 1 to 0 and generating a solution path indexed by this parameter. We show that the solution path is piecewise linear with respect to a simple function of the weight parameter under a fixed penalty. The resulting case influence is a function of the penalty and weight, and it becomes Cook's distance when the weight is 0. As the penalty parameter changes, selected variables change, and the magnitude of Cook's distance for the same data point may vary with the subset of variables selected. In addition, we introduce a case influence graph to visualize how the contribution of each data point changes with the penalty parameter. From the graph, we can identify influential points at different penalty levels and make modeling decisions accordingly. Moreover, we find that case influence graphs exhibit different patterns between underfitting and overfitting phases, which can provide additional information for model selection.

stat.ME↗

arXiv2020

Interpreting LSTM Prediction on Solar Flare Eruption with Time-series Clustering

We conduct a post hoc analysis of solar flare predictions made by a Long Short Term Memory (LSTM) model employing data in the form of Space-weather HMI Active Region Patches (SHARP) parameters calculated from data in proximity to the magnetic polarity inversion line where the flares originate. We train the the LSTM model for binary classification to provide a prediction score for the probability of M/X class flares to occur in next hour. We then develop a dimension-reduction technique to reduce the dimensions of SHARP parameter (LSTM inputs) and demonstrate the different patterns of SHARP parameters corresponding to the transition from low to high prediction score. Our work shows that a subset of SHARP parameters contain the key signals that strong solar flare eruptions are imminent. The dynamics of these parameters have a highly uniform trajectory for many events whose LSTM prediction scores for M/X class flares transition from very low to very high. The results demonstrate the existence of a few threshold values of SHARP parameters that when surpassed indicate a high probability of the eruption of a strong flare. Our method has distilled the knowledge of solar flare eruption learnt by deep learning model and provides a more interpretable approximation, which provides physical insight to processes driving solar flares.

astro-ph.SR↗

arXiv2020

Solar Flare Intensity Prediction with Machine Learning Models

We develop a mixed Long Short Term Memory (LSTM) regression model to predict the maximum solar flare intensity within a 24-hour time window 0$\sim$24, 6$\sim$30, 12$\sim$36 and 24$\sim$48 hours ahead of time using 6, 12, 24 and 48 hours of data (predictors) for each Helioseismic and Magnetic Imager (HMI) Active Region Patch (HARP). The model makes use of (1) the Space-weather HMI Active Region Patch (SHARP) parameters as predictors and (2) the exact flare intensities instead of class labels recorded in the Geostationary Operational Environmental Satellites (GOES) data set, which serves as the source of the response variables. Compared to solar flare classification, the model offers us more detailed information about the exact maximum flux level, i.e. intensity, for each occurrence of a flare. We also consider classification models built on top of the regression model and obtain better results in solar flare classifications. Our results suggest that the most efficient time period for predicting the solar activity is within 24 hours before the prediction time using the SHARP parameters and the LSTM model.

astro-ph.SR↗

arXiv2020

Predicting solar flares with machine learning: investigating solar cycle dependence

A deep learning network, Long-Short Term Memory (LSTM) network, is used in this work to predict whether the maximum flare class an active region (AR) will produce in the next 24 hours is class $Γ$. We considered $Γ$ are $\ge M$, $\ge C$ and any flare class. The essence of using LSTM, which is a recurrent neural network, is its capability to capture temporal information of the data samples. The input features are time sequences of 20 magnetic parameters from SHARPs - Space-weather HMI Active Region Patches. We analyzed active regions from June 2010 to Dec 2018, using the Geostationary Operational Environmental Satellite (GOES) X-ray flare catalogs and label the data samples with identified ARs in the GOES X-ray flare catalogs. Our results (i) shows consistent skill scores with recently published results using LSTMs and better than the previous work using single time input (eg. DeFN) (ii) The skill scores from the model show essential differences when different years of data was chosen for training and testing.

astro-ph.SR↗