arXiv subjects

James P. Long

Publications and source records attributed to James P. Long.

At least 19 recordsLinked to original sources

arXiv2026

Multi-Objective Composite Longitudinal Biomarker Scores for Improved Cancer Risk Assessment

Repeated blood-based biomarker measurements can improve cancer risk assessment by capturing longitudinal changes missed by single-time-point analyses. Parametric Empirical Bayes (PEB) incorporates prior measurements to estimate individualized reference values, but existing implementations do not account for the time between measurements and rely on predefined panels with fixed combination rules. We developed improved Parametric Empirical Bayes (iPEB), which accounts for the intervals between serial measurements, adjusts for covariates, and performs feature selection and optimized biomarker combination. iPEB optimizes biomarker weights for specific clinical objectives, such as maximizing sensitivity at a prespecified specificity or diagnostic lead time. We evaluated iPEB through simulations and a real-world application using six protein biomarkers (pro-SFTPB, CEA, CA125, CYFRA 21-1, osteopontin, and HE4) from a case-control study nested within the Prostate, Lung, Colorectal, and Ovarian (PLCO) Cancer Screening Trial. The analysis included 324 lung cancer cases and 1,674 controls with at least two serial measurements; six centers were used for model development and four for independent validation. Optimized for sensitivity at 99% specificity, iPEB achieved 24.2% sensitivity in the independent test set, compared with 18.2% for conventional PEB applied to the same four-marker panel. Optimized instead for lead time, iPEB added approximately 50 days of lead time at that stringent operating point. iPEB improved lung cancer risk assessment in independent PLCO data, supporting objective-driven optimization of longitudinal biomarkers for early detection.

stat.ME↗

arXiv2026

Partial Causal Structure Learning for Valid Selective Conformal Inference under Interventions

Selective conformal prediction can yield substantially tighter uncertainty sets when we can identify calibration examples that are exchangeable with the test example. In interventional settings, such as perturbation experiments in genomics, exchangeability often holds only within subsets of interventions that leave a target variable "unaffected" (e.g., non-descendants of an intervened node in a causal graph). We study the practical regime where this invariance structure is unknown and must be estimated from data. Our main result quantifies how coverage degrades when the estimated safe calibration set accidentally includes interventions that affect the target, and gives a conservative correction when an upper bound on this error is available. Rather than learning a full causal graph, we learn only the intervention-target relationships needed to choose calibration interventions. We give algorithms for this partial learning task and evaluate them on synthetic structural equation models and Replogle K562 CRISPR-interference data, where the experiments illustrate synthetic gains from selective calibration and finite-sample tradeoffs on real perturbation screens.

cs.LG↗

arXiv2025

Causal Discovery with Mixed Latent Confounding via Precision Decomposition

We study causal discovery from observational data in linear Gaussian systems affected by \emph{mixed latent confounding}, where some unobserved factors act broadly across many variables while others influence only small subsets. This setting is common in practice and poses a challenge for existing methods: differentiable and score-based DAG learners can misinterpret global latent effects as causal edges, while latent-variable graphical models recover only undirected structure. We propose \textsc{DCL-DECOR}, a modular, precision-led pipeline that separates these roles. The method first isolates pervasive latent effects by decomposing the observed precision matrix into a structured component and a low-rank component. The structured component corresponds to the conditional distribution after accounting for pervasive confounders and retains only local dependence induced by the causal graph and localized confounding. A correlated-noise DAG learner is then applied to this deconfounded representation to recover directed edges while modeling remaining structured error correlations, followed by a simple reconciliation step to enforce bow-freeness. We provide identifiability results that characterize the recoverable causal target under mixed confounding and show how the overall problem reduces to well-studied subproblems with modular guarantees. Synthetic experiments that vary the strength and dimensionality of pervasive confounding demonstrate consistent improvements in directed edge recovery over applying correlated-noise DAG learning directly to the confounded data.

cs.LG↗

arXiv2025

DAG DECORation: Continuous Optimization for Structure Learning under Hidden Confounding

We study structure learning for linear Gaussian SEMs in the presence of latent confounding. Existing continuous methods excel when errors are independent, while deconfounding-first pipelines rely on pervasive factor structure or nonlinearity. We propose \textsc{DECOR}, a single likelihood-based and fully differentiable estimator that jointly learns a DAG and a correlated noise model. Our theory gives simple sufficient conditions for global parameter identifiability: if the mixed graph is bow free and the noise covariance has a uniform eigenvalue margin, then the map from $(\B,\OmegaMat)$ to the observational covariance is injective, so both the directed structure and the noise are uniquely determined. The estimator alternates a smooth-acyclic graph update with a convex noise update and can include a light bow complementarity penalty or a post hoc reconciliation step. On synthetic benchmarks that vary confounding density, graph density, latent rank, and dimension with $n<p$, \textsc{DECOR} matches or outperforms strong baselines and is especially robust when confounding is non-pervasive, while remaining competitive under pervasiveness.

cs.LG↗

arXiv2025

Bias in Meta-Analytic Modeling of Surrogate Endpoints in Cancer Screening Trials

In meta-analytic modeling, the functional relationship between a primary and surrogate endpoint is estimated using summary data from a set of completed clinical trials. Parameters in the meta-analytic model are used to assess the quality of the proposed surrogate. Recently, meta-analytic models have been employed to evaluate whether late-stage cancer incidence can serve as a surrogate for cancer mortality in cancer screening trials. A major challenge in meta-analytic models is that uncertainty of trial-level estimates affects the evaluation of surrogacy, since each trial provides only estimates of the primary and surrogate endpoints rather than their true parameter values. In this work, we show via simulation and theory that trial-level estimate uncertainty may bias the results of meta-analytic models towards positive findings of the quality of the surrogate. We focus on cancer screening trials and the late stage incidence surrogate. We reassess correlations between primary and surrogate endpoints in Ovarian cancer screening trials. Our findings indicate that completed trials provide limited information regarding quality of the late-stage incidence surrogate. These results support restricting meta-analytic regression usage to settings where trial-level estimate uncertainty is incorporated into the model.

stat.ME↗

arXiv2025

Robust Learning on Noisy Graphs via Latent Space Constraints with External Knowledge

Graph Neural Networks (GNNs) often struggle with noisy edges. We propose Latent Space Constrained Graph Neural Networks (LSC-GNN) to incorporate external "clean" links and guide embeddings of a noisy target graph. We train two encoders--one on the full graph (target plus external edges) and another on a regularization graph excluding the target's potentially noisy links--then penalize discrepancies between their latent representations. This constraint steers the model away from overfitting spurious edges. Experiments on benchmark datasets show LSC-GNN outperforms standard and noise-resilient GNNs in graphs subjected to moderate noise. We extend LSC-GNN to heterogeneous graphs and validate it on a small protein-metabolite network, where metabolite-protein interactions reduce noise in protein co-occurrence data. Our results highlight LSC-GNN's potential to boost predictive performance and interpretability in settings with noisy relational structures.

cs.LG↗

arXiv2023

Estimating Causal Effects with Hidden Confounding using Instrumental Variables and Environments

Recent works have proposed regression models which are invariant across data collection environments. These estimators often have a causal interpretation under conditions on the environments and type of invariance imposed. One recent example, the Causal Dantzig (CD), is consistent under hidden confounding and represents an alternative to classical instrumental variable estimators such as Two Stage Least Squares (TSLS). In this work we derive the CD as a generalized method of moments (GMM) estimator. The GMM representation leads to several practical results, including 1) creation of the Generalized Causal Dantzig (GCD) estimator which can be applied to problems with continuous environments where the CD cannot be fit 2) a Hybrid (GCD-TSLS combination) estimator which has properties superior to GCD or TSLS alone 3) straightforward asymptotic results for all methods using GMM theory. We compare the CD, GCD, TSLS, and Hybrid estimators in simulations and an application to a Flow Cytometry data set. The newly proposed GCD and Hybrid estimators have superior performance to existing methods in many settings.

stat.ME↗

arXiv2022

Causal Models, Prediction, and Extrapolation in Cell Line Perturbation Experiments

In cell line perturbation experiments, a collection of cells is perturbed with external agents (e.g. drugs) and responses such as protein expression measured. Due to cost constraints, only a small fraction of all possible perturbations can be tested in vitro. This has led to the development of computational (in silico) models which can predict cellular responses to perturbations. Perturbations with clinically interesting predicted responses can be prioritized for in vitro testing. In this work, we compare causal and non-causal regression models for perturbation response prediction in a Melanoma cancer cell line. The current best performing method on this data set is Cellbox which models how proteins causally effect each other using a system of ordinary differential equations (ODEs). We derive a closed form solution to the Cellbox system of ODEs in the linear case. These analytic results facilitate comparison of Cellbox to regression approaches. We show that causal models such as Cellbox, while requiring more assumptions, enable extrapolation in ways that non-causal regression models cannot. For example, causal models can predict responses for never before tested drugs. We illustrate these strengths and weaknesses in simulations. In an application to the Melanoma cell line data, we find that regression models outperform the Cellbox causal model.

stat.AP↗

arXiv2021

Sample Selection Bias in Evaluation of Prediction Performance of Causal Models

Causal models are notoriously difficult to validate because they make untestable assumptions regarding confounding. New scientific experiments offer the possibility of evaluating causal models using prediction performance. Prediction performance measures are typically robust to violations in causal assumptions. However, prediction performance does depend on the selection of training and test sets. Biased training sets can lead to optimistic assessments of model performance. In this work, we revisit the prediction performance of several recently proposed causal models tested on a genetic perturbation data set of Kemmeren. We find that sample selection bias is likely a key driver of model performance. We propose using a less-biased evaluation set for assessing prediction performance and compare models on this new set. In this setting, the causal models have similar or worse performance compared to standard association-based estimators such as Lasso. Finally, we compare the performance of causal estimators in simulation studies that reproduce the Kemmeren structure of genetic knockout experiments but without any sample selection bias. These results provide an improved understanding of the performance of several causal models and offer guidance on how future studies should use Kemmeren.

stat.ML↗

arXiv2020

A Framework for Mediation Analysis with Multiple Exposures, Multivariate Mediators, and Non-Linear Response Models

Mediation analysis seeks to identify and quantify the paths by which an exposure affects an outcome. Intermediate variables which are effected by the exposure and which effect the outcome are known as mediators. There exists extensive work on mediation analysis in the context of models with a single mediator and continuous and binary outcomes. However these methods are often not suitable for multi-omic data that include highly interconnected variables measuring biological mechanisms and various types of outcome variables such as censored survival responses. In this article, we develop a general framework for causal mediation analysis with multiple exposures, multivariate mediators, and continuous, binary, and survival responses. We estimate mediation effects on several scales including the mean difference, odds ratio, and restricted mean scale as appropriate for various outcome models. Our estimation method avoids imposing constraints on model parameters such as the rare disease assumption while accommodating continuous exposures. We evaluate the framework and compare it to other methods in extensive simulation studies by assessing bias, type I error and power at a range of sample sizes, disease prevalences, and number of false mediators. Using Kidney Renal Clear Cell Carcinoma data from The Cancer Genome Atlas, we identify proteins which mediate the effect of metabolic gene expression on survival. Software for implementing this unified framework is made available in an R package (https://github.com/longjp/mediateR).

stat.ME↗

arXiv2020

A Flexible Procedure for Mixture Proportion Estimation in Positive-Unlabeled Learning

Positive--unlabeled (PU) learning considers two samples, a positive set P with observations from only one class and an unlabeled set U with observations from two classes. The goal is to classify observations in U. Class mixture proportion estimation (MPE) in U is a key step in PU learning. Blanchard et al. [2010] showed that MPE in PU learning is a generalization of the problem of estimating the proportion of true null hypotheses in multiple testing problems. Motivated by this idea, we propose reducing the problem to one dimension via construction of a probabilistic classifier trained on the P and U data sets followed by application of a one--dimensional mixture proportion method from the multiple testing literature to the observation class probabilities. The flexibility of this framework lies in the freedom to choose the classifier and the one--dimensional MPE method. We prove consistency of two mixture proportion estimators using bounds from empirical process theory, develop tuning parameter free implementations, and demonstrate that they have competitive performance on simulated waveform data and a protein signaling problem.

stat.ME↗

arXiv2017

Statistical methods in astronomy

We present a review of data types and statistical methods often encountered in astronomy. The aim is to provide an introduction to statistical applications in astronomy for statisticians and computer scientists. We highlight the complex, often hierarchical, nature of many astronomy inference problems and advocate for cross-disciplinary collaborations to address these challenges.

physics.ed-ph↗

arXiv2017

Active Tuning of Surface Phonon Polariton Resonances via Carrier Photoinjection

Surface-phonon polaritons (SPhPs) are attractive alternatives to far-infrared plasmonics for sub-diffractional confinement of light. Localized SPhP resonances in semiconductor nanoresonators are very narrow, but that linewidth and the limited extent of the Reststrahlen band inherently limit spectral coverage. To address this limitation, we report active tuning of SPhP resonances in InP and 4H-SiC by photoinjecting free carriers into the nanoresonators, taking advantage of the coupling between the carrier plasma and optical phonons to blue-shift SPhP resonances. We demonstrate state-of-the-art tuning figures of merit upon continuous-wave (CW) excitation (in InP) or pulsed excitation (in 4H-SiC). Lifetime effects cause the tuning to saturate in InP, and carrier-redistribution leads to rapid (<50 ps) recovery of the tuning in 4H-SiC. This work opens the path toward actively tuned nanophotonic devices, such as modulators and beacons, in the infrared and identifies important implications of coupling between electronic and photonic excitations.

cond-mat.mtrl-sci↗

arXiv2017

A Note on Parameter Estimation for Misspecified Regression Models with Heteroskedastic Errors

Misspecified models often provide useful information about the true data generating distribution. For example, if $y$ is a non-linear function of $x$ the least squares estimator $\hatβ$ is an estimate of $β$, the slope of the best linear approximation to the non-linear function. Motivated by problems in astronomy, we study how to incorporate observation measurement error variances into fitting parameters of misspecified models. Our asymptotic theory focuses on the particular case of linear regression where often weighted least squares procedures are used to account for heteroskedasticity. We find that when the response is a non-linear function of the independent variable, the standard procedure of weighting by the inverse of the observation variances can be counter-productive. In particular, ordinary least squares may have lower asymptotic variance. We construct an adaptive estimator which has lower asymptotic variance than either OLS or standard WLS. We demonstrate our theory in a small simulation and apply these ideas to the problem of estimating the period of a periodic function using a sinusoidal model.

stat.ME↗

arXiv2016

A Study of Functional Depths

Functional depth is used for ranking functional observations from most outlying to most typical. The ranks produced by functional depth have been proposed as the basis for functional classifiers, rank tests, and data visualization procedures. Many of the proposed functional depths are invariant to domain permutation, an unusual property for a functional data analysis procedure. Essentially these depths treat functional data as if it were multivariate data. In this work, we compare the performance of several existing functional depths to a simple adaptation of an existing multivariate depth notion, $L^\infty$ depth ($L^{\infty}D$). On simulated and real data, we show $L^{\infty}D$ has performance comparable or superior to several existing notions of functional depth. In addition, we review how depth functions are evaluated and propose some improvements. In particular, we show that empirical depth function asymptotics can be mis--leading and instead propose a new method, the rank--rank plot, for evaluating empirical depth rank stability.

stat.ME↗

arXiv2015

Photoinduced tunability of the Reststrahlen band in 4H-SiC

Materials with a negative dielectric permittivity (e.g. metals) display high reflectance and can be shaped into nanoscale optical-resonators exhibiting extreme mode confinement, a central theme of nanophotonics. However, the ability to $actively$ tune these effects remains elusive. By photoexciting free carriers in 4H-SiC, we induce dramatic changes in reflectance near the "Reststrahlen band" where the permittivity is negative due to charge oscillations of the polar optical phonons in the mid-infrared. We infer carrier-induced changes in the permittivity required for useful tunability (~ 40 cm$^{-1}$) in nanoscale resonators, providing a direct avenue towards the realization of actively tunable nanophotonic devices in the mid-infrared to terahertz spectral range.

cond-mat.mtrl-sci↗

arXiv2015

A Multiband Generalization of the Analysis of Variance Period Estimation Algorithm and the Effect of Inter-band Observing Cadence on Period Recovery Rate

We present a new method of extending the single band Analysis of Variance period estimation algorithm to multiple bands. We use SDSS Stripe 82 RR Lyrae to show that in the case of low number of observations per band and non-simultaneous observations, improvements in period recovery rates of up to $\approx$60\% are observed. We also investigate the effect of inter-band observing cadence on period recovery rates. We find that using non-simultaneous observation times between bands is ideal for the multiband method, and using simultaneous multiband data is only marginally better than using single band data. These results will be particularly useful in planning observing cadences for wide-field astronomical imaging surveys such as LSST. They also have the potential to improve the extraction of transient data from surveys with few ($\lesssim 30$) observations per band across several bands, such as the Dark Energy Survey.

astro-ph.SR↗

arXiv2014

Estimating a Common Period for a Set of Irregularly Sampled Functions with Applications to Periodic Variable Star Data

We consider the estimation of a common period for a set of functions sampled at irregular intervals. The problem arises in astronomy, where the functions represent a star's brightness observed over time through different photometric filters. While current methods can estimate periods accurately provided that the brightness is well--sampled in at least one filter, there are no existing methods that can provide accurate estimates when no brightness function is well--sampled. In this paper we introduce two new methods for period estimation when brightnesses are poorly--sampled in all filters. The first, multiband generalized Lomb-Scargle (MGLS), extends the frequently used Lomb-Scargle method in a way that naïvely combines information across filters. The second, penalized generalized Lomb-Scargle (PGLS), builds on the first by more intelligently borrowing strength across filters. Specifically, we incorporate constraints on the phases and amplitudes across the different functions using a non--convex penalized likelihood function. We develop a fast algorithm to optimize the penalized likelihood by combining block coordinate descent with the majorization-minimization (MM) principle. We illustrate our methods on synthetic and real astronomy data. Both advance the state-of-the-art in period estimation; however, PGLS significantly outperforms MGLS when all functions are extremely poorly--sampled.

stat.AP↗