arXiv subjects

Martin T. Wells

Publications and source records attributed to Martin T. Wells.

At least 55 records · Page 3Linked to original sources

arXiv2021

An empirical Bayes approach to estimating dynamic models of co-regulated gene expression

Time-course gene expression datasets provide insight into the dynamics of complex biological processes, such as immune response and organ development. It is of interest to identify genes with similar temporal expression patterns because such genes are often biologically related. However, this task is challenging due to the high dimensionality of these datasets and the nonlinearity of gene expression time dynamics. We propose an empirical Bayes approach to estimating ordinary differential equation (ODE) models of gene expression, from which we derive a similarity metric between genes called the Bayesian lead-lag $R^2$ (LLR2). Importantly, the calculation of the LLR2 leverages biological databases that document known interactions amongst genes; this information is automatically used to define informative prior distributions on the ODE model's parameters. As a result, the LLR2 is a biologically-informed metric that can be used to identify clusters or networks of functionally-related genes with co-moving or time-delayed expression patterns. We then derive data-driven shrinkage parameters from Stein's unbiased risk estimate that optimally balance the ODE model's fit to both data and external biological information. Using real gene expression data, we demonstrate that our methodology allows us to recover interpretable gene clusters and sparse networks. These results reveal new insights about the dynamics of biological systems.

stat.ME↗

arXiv2021

Clustering Structure of Microstructure Measures

This paper builds the clustering model of measures of market microstructure features which are popular in predicting stock returns. In a 10-second time-frequency, we study the clustering structure of different measures to find out the best ones for predicting. In this way, we can predict more accurately with a limited number of predictors, which removes the noise and makes the model more interpretable.

q-fin.ST↗

arXiv2021

High-Dimensional Estimation, Basis Assets, and the Adaptive Multi-Factor Model

The paper proposes a new algorithm for the high-dimensional financial data -- the Groupwise Interpretable Basis Selection (GIBS) algorithm, to estimate a new Adaptive Multi-Factor (AMF) asset pricing model, implied by the recently developed Generalized Arbitrage Pricing Theory, which relaxes the convention that the number of risk-factors is small. We first obtain an adaptive collection of basis assets and then simultaneously test which basis assets correspond to which securities, using high-dimensional methods. The AMF model, along with the GIBS algorithm, is shown to have a significantly better fitting and prediction power than the Fama-French 5-factor model.

q-fin.ST↗

arXiv2021

Quantile Regression Modelling via Location and Scale Mixtures of Normal Distributions

We show that the estimating equations for quantile regression can be solved using a simple EM algorithm in which the M-step is computed via weighted least squares, with weights computed at the E-step as the expectation of independent generalized inverse-Gaussian variables. We compute the variance-covariance matrix for the quantile regression coefficients using a kernel density estimator that results in more stable standard errors than those produced by existing software. A natural modification of the EM algorithm that involves fitting a linear mixed model at the M-step extends the methodology to mixed effects quantile regression models. In this case, the fitting method can be justified as a generalized alternating minimization algorithm. Obtaining quantile regression estimates via the weighted least squares method enables model diagnostic techniques similar to the ones used in the linear regression setting. The computational approach is compared with existing software using simulated data, and the methodology is illustrated with several case studies.

stat.ME↗

arXiv2021

On Graphical Models and Convex Geometry

We introduce a mixture-model of beta distributions to identify significant correlations among $P$ predictors when $P$ is large. The method relies on theorems in convex geometry, which we use to show how to control the error rate of edge detection in graphical models. Our `betaMix' method does not require any assumptions about the network structure, nor does it assume that the network is sparse. The results in this article hold for a wide class of data generating distributions that include light-tailed and heavy-tailed spherically symmetric distributions.

stat.ME↗

arXiv2021

A News-based Machine Learning Model for Adaptive Asset Pricing

The paper proposes a new asset pricing model -- the News Embedding UMAP Selection (NEUS) model, to explain and predict the stock returns based on the financial news. Using a combination of various machine learning algorithms, we first derive a company embedding vector for each basis asset from the financial news. Then we obtain a collection of the basis assets based on their company embedding. After that for each stock, we select the basis assets to explain and predict the stock return with high-dimensional statistical methods. The new model is shown to have a significantly better fitting and prediction power than the Fama-French 5-factor model.

q-fin.ST↗

arXiv2021

Time-Invariance Coefficients Tests with the Adaptive Multi-Factor Model

The purpose of this paper is to test the time-invariance of the beta coefficients estimated by the Adaptive Multi-Factor (AMF) model. The AMF model is implied by the generalized arbitrage pricing theory (GAPT), which implies constant beta coefficients. The AMF model utilizes a Groupwise Interpretable Basis Selection (GIBS) algorithm to identify the relevant factors from among all traded ETFs. We compare the AMF model with the Fama-French 5-factor (FF5) model. We show that for nearly all time periods with length less than 6 years, the beta coefficients are time-invariant for the AMF model, but not for the FF5 model. This implies that the AMF model with a rolling window (such as 5 years) is more consistent with realized asset returns than is the FF5 model.

q-fin.ST↗

arXiv2021

The Low-volatility Anomaly and the Adaptive Multi-Factor Model

The paper provides a new explanation of the low-volatility anomaly. We use the Adaptive Multi-Factor (AMF) model estimated by the Groupwise Interpretable Basis Selection (GIBS) algorithm to find those basis assets significantly related to low and high volatility portfolios. These two portfolios load on very different factors, indicating that volatility is not an independent risk, but that it's related to existing risk factors. The out-performance of the low-volatility portfolio is due to the (equilibrium) performance of these loaded risk factors. The AMF model outperforms the Fama-French 5-factor model both in-sample and out-of-sample.

q-fin.ST↗

arXiv2021

Mixed Effect Modeling and Variable Selection for Quantile Regression

It is known that the estimating equations for quantile regression (QR) can be solved using an EM algorithm in which the M-step is computed via weighted least squares, with weights computed at the E-step as the expectation of independent generalized inverse-Gaussian variables. This fact is exploited here to extend QR to allow for random effects in the linear predictor. Convergence of the algorithm in this setting is established by showing that it is a generalized alternating minimization (GAM) procedure. Another modification of the EM algorithm also allows us to adapt a recently proposed method for variable selection in mean regression models to the QR setting. Simulations show the resulting method significantly outperforms variable selection in QR models using the lasso penalty. Applications to real data include a frailty QR analysis of hospital stays, and variable selection for age at onset of lung cancer and for riboflavin production rate using high-dimensional gene expression arrays for prediction.

stat.ME↗

arXiv2021

HALO: Learning to Prune Neural Networks with Shrinkage

Deep neural networks achieve state-of-the-art performance in a variety of tasks by extracting a rich set of features from unstructured data, however this performance is closely tied to model size. Modern techniques for inducing sparsity and reducing model size are (1) network pruning, (2) training with a sparsity inducing penalty, and (3) training a binary mask jointly with the weights of the network. We study different sparsity inducing penalties from the perspective of Bayesian hierarchical models and present a novel penalty called Hierarchical Adaptive Lasso (HALO) which learns to adaptively sparsify weights of a given network via trainable parameters. When used to train over-parametrized networks, our penalty yields small subnetworks with high accuracy without fine-tuning. Empirically, on image recognition tasks, we find that HALO is able to learn highly sparse network (only 5% of the parameters) with significant gains in performance over state-of-the-art magnitude pruning methods at the same level of sparsity. Code is available at https://github.com/skyler120/sparsity-halo.

cs.LG↗

arXiv2020

Tree Space Prototypes: Another Look at Making Tree Ensembles Interpretable

Ensembles of decision trees perform well on many problems, but are not interpretable. In contrast to existing approaches in interpretability that focus on explaining relationships between features and predictions, we propose an alternative approach to interpret tree ensemble classifiers by surfacing representative points for each class -- prototypes. We introduce a new distance for Gradient Boosted Tree models, and propose new, adaptive prototype selection methods with theoretical guarantees, with the flexibility to choose a different number of prototypes in each class. We demonstrate our methods on random forests and gradient boosted trees, showing that the prototypes can perform as well as or even better than the original tree ensemble when used as a nearest-prototype classifier. In a user study, humans were better at predicting the output of a tree ensemble classifier when using prototypes than when using Shapley values, a popular feature attribution method. Hence, prototypes present a viable alternative to feature-based explanations for tree ensembles.

stat.ML↗

arXiv2020

Robust Matrix Completion with Mixed Data Types

We consider the matrix completion problem of recovering a structured low rank matrix with partially observed entries with mixed data types. Vast majority of the solutions have proposed computationally feasible estimators with strong statistical guarantees for the case where the underlying distribution of data in the matrix is continuous. A few recent approaches have extended using similar ideas these estimators to the case where the underlying distributions belongs to the exponential family. Most of these approaches assume that there is only one underlying distribution and the low rank constraint is regularized by the matrix Schatten Norm. We propose a computationally feasible statistical approach with strong recovery guarantees along with an algorithmic framework suited for parallelization to recover a low rank matrix with partially observed entries for mixed data types in one step. We also provide extensive simulation evidence that corroborate our theoretical results.

stat.ML↗

arXiv2019

A Scalable Empirical Bayes Approach to Variable Selection in Generalized Linear Models

A new empirical Bayes approach to variable selection in the context of generalized linear models is developed. The proposed algorithm scales to situations in which the number of putative explanatory variables is very large, possibly much larger than the number of responses. The coefficients in the linear predictor are modeled as a three-component mixture allowing the explanatory variables to have a random positive effect on the response, a random negative effect, or no effect. A key assumption is that only a small (but unknown) fraction of the candidate variables have a non-zero effect. This assumption, in addition to treating the coefficients as random effects facilitates an approach that is computationally efficient. In particular, the number of parameters that have to be estimated is small, and remains constant regardless of the number of explanatory variables. The model parameters are estimated using a Generalized Alternating Maximization algorithm which is scalable, and leads to significantly faster convergence compared with simulation-based fully Bayesian methods.

stat.ME↗

arXiv2019

Fairness criteria through the lens of directed acyclic graphical models

A substantial portion of the literature on fairness in algorithms proposes, analyzes, and operationalizes simple formulaic criteria for assessing fairness. Two of these criteria, Equalized Odds and Calibration by Group, have gained significant attention for their simplicity and intuitive appeal, but also for their incompatibility. This chapter provides a perspective on the meaning and consequences of these and other fairness criteria using graphical models which reveals Equalized Odds and related criteria to be ultimately misleading. An assessment of various graphical models suggests that fairness criteria should ultimately be case-specific and sensitive to the nature of the information the algorithm processes.

cs.CY↗

arXiv2019

Tuning Free Rank-Sparse Bayesian Matrix and Tensor Completion with Global-Local Priors

Matrix and tensor completion are frameworks for a wide range of problems, including collaborative filtering, missing data, and image reconstruction. Missing entries are estimated by leveraging an assumption that the matrix or tensor is low-rank. Most existing Bayesian techniques encourage rank-sparsity by modelling factorized matrices and tensors with Normal-Gamma priors. However, the Horseshoe prior and other "global-local" formulations provide tuning-parameter-free solutions which may better achieve simultaneous rank-sparsity and missing-value recovery. We find these global-local priors outperform commonly used alternatives in simulations and in a collaborative filtering task predicting board game ratings.

stat.ME↗

arXiv2018

Ludometrics: Luck, and How to Measure It

Game theory is the study of tractable games which may be used to model more complex systems. Board games, video games and sports, however, are intractable by design, so "ludological" theories about these games as complex phenomena should be grounded in empiricism. A first "ludometric" concern is the empirical measurement of the amount of luck in various games. We argue against a narrow view of luck which includes only factors outside any player's control, and advocate for a holistic definition of luck as complementary to the variation in effective skill within a population of players. We introduce two metrics for luck in a game for a given population - one information theoretical, and one Bayesian, and discuss the estimation of these metrics using sparse, high-dimensional regression techniques. Finally, we apply these techniques to compare the amount of luck between various professional sports, between Chess and Go, and between two hobby board games: Race for the Galaxy and Seasons.

stat.AP↗

arXiv2018

Rank-based approach for estimating correlations in mixed ordinal data

High-dimensional mixed data as a combination of both continuous and ordinal variables are widely seen in many research areas such as genomic studies and survey data analysis. Estimating the underlying correlation among mixed data is hence crucial for further inferring dependence structure. We propose a semiparametric latent Gaussian copula model for this problem. We start with estimating the association among ternary-continuous mixed data via a rank-based approach and generalize the methodology to p-level-ordinal and continuous mixed data. Concentration rate of the estimator is also provided and proved. At last, we demonstrate the performance of the proposed estimator by extensive simulations and two case studies of real data examples of algorithmic risk score evaluation and cancer patients survival data.

stat.ME↗

arXiv2017

A Double Parametric Bootstrap Test for Topic Models

Non-negative matrix factorization (NMF) is a technique for finding latent representations of data. The method has been applied to corpora to construct topic models. However, NMF has likelihood assumptions which are often violated by real document corpora. We present a double parametric bootstrap test for evaluating the fit of an NMF-based topic model based on the duality of the KL divergence and Poisson maximum likelihood estimation. The test correctly identifies whether a topic model based on an NMF approach yields reliable results in simulated and real data.

stat.ML↗