arXiv subjects

Nicholas Polson

Publications and source records attributed to Nicholas Polson.

At least 19 recordsLinked to original sources

arXiv2026

Chess Signatures of Play

A game of chess is a stream: a time-ordered sequence of moves, each carrying an engine evaluation, a measure of accuracy, a measure of position complexity, and a clock reading. We model a game as a multivariate path and apply the signature transform of rough-path theory to obtain a reparametrization-invariant, graded feature set that records the order and interaction of in-game events without a parametric likelihood. We show that a player's law of play is identifiable from the expected signature up to tree-like equivalence, construct a signature-kernel two-sample test on path space, and recast cheating detection as an anytime-valid sequential test: a signature conformance score becomes an e-process whose error is controlled for every sample size at once by Ville's inequality, with fluctuations calibrated on the moderate-deviation scale. The discriminating information lives in the signature's Levy areas, which measure whether accuracy rises precisely when positions become hard--the fingerprint of engine assistance that aggregate match-rate statistics discard. In a controlled study the test holds exact type-I control and detection power rises from negligible for subtle assistance to 0.98 for blatant assistance, with a median detection time matching the growth-rate prediction. Calibrated to Magnus Carlsen's documented elite accuracy, the monitor does not flag world-champion-level play; and we exhibit cheating strategies that leave every aggregate statistic, including the best-move-frequency z-score of the Regan system, unchanged yet are caught cleanly by the signature--making precise how an order-aware, anytime-valid test strengthens the prevailing approach to chess anti-cheating.

stat.AP↗

arXiv2026

Horseshoe Priors for Spatial Small Area Estimation: Regular Variation, Tail Robustness, and Deep Learning

Small area estimation borrows strength across domains to repair the poor precision of direct survey estimators. Two philosophies dominate the area-level literature. The first, descending from Ghosh and Rao (1994), borrows strength through structured Gaussian smoothing: an intrinsic conditional autoregression or its BYM2 reparameterization pools each area towards its neighbours. The second borrows strength globally but acts locally through a heavy-tailed global-local prior on area effects, of which the horseshoe of Carvalho et al. (2010) is the canonical instance; Tang et al. (2018) first brought this idea to small area estimation. We study the horseshoe Fay-Herriot model with known unequal sampling variances and make four contributions. First, a tail-robustness theorem: through a heteroscedastic Tweedie identity the posterior mean leaves strongly signalled areas essentially unshrunk, bounding the influence of an outlying direct estimate, unlike Gaussian random-effect models. Second, standardizing by the known design variances transfers the minimax contraction and credible-set theory of the homoscedastic sequence model to the heteroscedastic Fay-Herriot problem; the posterior contracts at the nearly-black minimax rate, with a matching lower bound. Third, we give an account of when structured smoothing and global-local shrinkage each win. Fourth, an O(m) Gibbs sampler, simulations, and an analysis of the Scottish lip cancer data confirm the account: on strongly spatial data the smoother predicts held-out districts best, yet the horseshoe flags exceptional districts that smoothing suppresses. Throughout we argue, following the regular-variation theory of Bhadra et al. (2016), that these properties make the horseshoe a sound default prior for area effects: it borrows strength aggressively yet lets genuinely exceptional areas speak for themselves, with no tuning and no neighbourhood graph.

stat.ME↗

arXiv2026

Bayesian Methods for the Navier-Stokes Equations

We develop a Bayesian methodology for numerical solution of the incompressible Navier--Stokes equations with quantified uncertainty. The central idea is to treat discretized Navier--Stokes dynamics as a state-space model and to view numerical solution as posterior computation: priors encode physical structure and modeling error, and the solver outputs a distribution over states and quantities of interest rather than a single trajectory. In two dimensions, stochastic representations (Feynman--Kac and stochastic characteristics for linear advection--diffusion with prescribed drift) motivate Monte Carlo solvers and provide intuition for uncertainty propagation. In three dimensions, we formulate stochastic Navier--Stokes models and describe particle-based and ensemble-based Bayesian workflows for uncertainty propagation in spectral discretizations. A key computational advantage is that parameter learning can be performed stably via particle learning: marginalization and resample--propagate (one-step smoothing) constructions avoid the weight-collapse that plagues naive sequential importance sampling on static parameters. When partial observations are available, the same machinery supports sequential observational updating as an additional capability. We also discuss non-Gaussian (heavy-tailed) error models based on normal variance-mean mixtures, which yield conditionally Gaussian updates via latent scale augmentation.

stat.CO↗

arXiv2025

Generative AI for Validating Physics Laws

We propose generative learner for estimating heterogeneous treatment effects and characterizing the full distribution of causal effects. The learner takes the form of a multi-head feed-forward neural network with three jointly estimated subnetworks, propensity score, baseline outcome, and heterogeneous treatment effects, where the treatment-effect subnetwork parameterizes the conditional quantile function via a compositional architecture in which covariate features and cosine quantile embeddings are combined through element-wise multiplication. This quantile-based formulation recovers the conditional average treatment effect as an integral over quantile treatment effects while additionally characterizing the entire effect distribution. Under the classical assumptions in a Neyman--Rubin causal model, we demonstrate that the performance gains of the proposed generative learner are consistent across experimental designs and frequently exceed 70% in out-of-sample mean squared error when compared to the generalized random forest, double machine learning, and generative adversarial networks. These gains are particularly pronounced in small samples. As an empirical application, we formalize the Stefan--Boltzmann law as a unidirectional causal model and apply the method to Gaia DR3 stellar data. The method recovers the expected nonlinear temperature--luminosity relationship and quantifies heterogeneous effects across stellar radius and absolute magnitude.

astro-ph.SR↗

arXiv2023

The Value of Chess Squares

We propose a neural network-based approach to calculate the value of a chess square-piece combination. Our model takes a triplet (Color, Piece, Square) as an input and calculates a value that measures the advantage/disadvantage of having this piece on this square. Our methods build on recent advances in chess AI, and can accurately assess the worth of positions in a game of chess. The conventional approach assigns fixed values to pieces $(\symking=\infty, \symqueen=9, \symrook=5, \symbishop=3, \symknight=3, \sympawn=1)$. We enhance this analysis by introducing marginal valuations. We use deep Q-learning to estimate the parameters of our model. We demonstrate our method by examining the positioning of Knights and Bishops, and also provide valuable insights into the valuation of pawns. Finally, we conclude by suggesting potential avenues for future research.

cs.AI↗

arXiv2023

Generative Causal Inference

Generative Bayesian Computation (GBC) methods are developed for Casual Inference. Generative methods are simulation-based methods that use a large training dataset to represent posterior distributions as a map (a.k.a. optimal transport) to a base distribution. They avoid the use of MCMC by replacing the conditional posterior inference problem with a supervised learning problem. We further propose the use Quantile ReLU networks which are density free and hence apply in a variety of Econometric settings where data generating processes are specified by deterministic latent variables updates or as moment constraints. Generative approaches directly simulate large samples of observables and unobservable (parameters, latent variables) and then apply high-dimensional quantile regression to learn a nonlinear transport map from base distribution to parameter inference. We illustrate our methodology in the field of causal inference. Our approach can also handle nonlinearity and heterogeneity. Finally, we conclude with the directions for future research.

stat.ME↗

arXiv2022

Feature Selection for Personalized Policy Analysis

In this paper, we propose Forest-PLS, a feature selection method for analyzing policy effect heterogeneity in a more flexible and comprehensive manner than is typically available with conventional methods. In particular, our method is able to capture policy effect heterogeneity both within and across subgroups of the population defined by observable characteristics. To achieve this, we employ partial least squares to identify target components of the population and causal forests to estimate personalized policy effects across these components. We show that the method is consistent and leads to asymptotically normally distributed policy effects. To demonstrate the efficacy of our approach, we apply it to the data from the Pennsylvania Reemployment Bonus Experiments, which were conducted in 1988-1989. The analysis reveals that financial incentives can motivate some young non-white individuals to enter the labor market. However, these incentives may also provide a temporary financial cushion for others, dissuading them from actively seeking employment. Our findings highlight the need for targeted, personalized measures for young non-white male participants.

econ.EM↗

arXiv2022

Deep Partial Least Squares for Instrumental Variable Regression

In this paper, we propose deep partial least squares for the estimation of high-dimensional nonlinear instrumental variable regression. As a precursor to a flexible deep neural network architecture, our methodology uses partial least squares for dimension reduction and feature selection from the set of instruments and covariates. A central theoretical result, due to Brillinger (2012) shows that the feature selection provided by partial least squares is consistent and the weights are estimated up to a proportionality constant. We illustrate our methodology with synthetic datasets with a sparse and correlated network structure and draw applications to the effect of childbearing on the mother's labor supply based on classic data of Angrist and Evans (1996). The results on synthetic data as well as applications show that the deep partial least squares method significantly outperforms other related methods. Finally, we conclude with directions for future research.

stat.ME↗

arXiv2022

Inverse Probability Weighting: from Survey Sampling to Evidence Estimation

We consider the class of inverse probability weight (IPW) estimators, including the popular Horvitz-Thompson and Hajek estimators used routinely in survey sampling, causal inference and evidence estimation for Bayesian computation. We focus on the 'weak paradoxes' for these estimators due to two counterexamples by Basu [1988] and Wasserman [2004] and investigate the two natural Bayesian answers to this problem: one based on binning and smoothing : a 'Bayesian sieve' and the other based on a conjugate hierarchical model that allows borrowing information via exchangeability. We compare the mean squared errors for the two Bayesian estimators with the IPW estimators for Wasserman's example via simulation studies on a broad range of parameter configurations. We also prove posterior consistency for the Bayes estimators under missing-completely-at-random assumption and show that it requires fewer assumptions on the inclusion probabilities. We also revisit the connection between the different problems where improved or adaptive IPW estimators will be useful, including survey sampling, evidence estimation strategies such as Conditional Monte Carlo, Riemannian sum, Trapezoidal rules and vertical likelihood, as well as average treatment effect estimation in causal inference.

stat.ME↗

arXiv2021

Gambits: Theory and Evidence

Gambits are central to human decision-making. Our goal is to provide a theory of Gambits. A Gambit is a combination of psychological and technical factors designed to disrupt predictable play. Chess provides an environment to study gambits and behavioral game theory. Our theory is based on the Bellman optimality path for sequential decision-making. This allows us to calculate the $Q$-values of a Gambit where material (usually a pawn) is sacrificed for dynamic play. On the empirical side, we study the effectiveness of a number of popular chess Gambits. This is a natural setting as chess Gambits require a sequential assessment of a set of moves (a.k.a. policy) after the Gambit has been accepted. Our analysis uses Stockfish 14.1 to calculate the optimal Bellman $Q$ values, which fundamentally measures if a position is winning or losing. To test whether Bellman's equation holds in play, we estimate the transition probabilities to the next board state via a database of expert human play. This then allows us to test whether the \emph{Gambiteer} is following the optimal path in his decision-making. Our methodology is applied to the popular Stafford and reverse Stafford (a.k.a. Boden-Kieretsky-Morphy) Gambit and other common ones including the Smith-Morra, Goring, Danish and Halloween Gambits. We build on research in human decision-making by proving an irrational skewness preference within agents in chess. We conclude with directions for future research.

econ.TH↗

arXiv2021

Deep Learning Partial Least Squares

High dimensional data reduction techniques are provided by using partial least squares within deep learning. Our framework provides a nonlinear extension of PLS together with a disciplined approach to feature selection and architecture design in deep learning. This leads to a statistical interpretation of deep learning that is tailor made for predictive problems. We can use the tools of PLS, such as scree-plot, bi-plot to provide model diagnostics. Posterior predictive uncertainty is available using MCMC methods at the last layer. Thus we achieve the best of both worlds: scalability and fast predictive rule construction together with uncertainty quantification. Our key construct is to employ deep learning within PLS by predicting the output scores as a deep learner of the input scores. As with PLS our X-scores are constructed using SVD and applied to both regression and classification problems and are fast and scalable. Following Frank and Friedman 1993, we provide a Bayesian shrinkage interpretation of our nonlinear predictor. We introduce a variety of new partial least squares models: PLS-ReLU, PLS-Autoencoder, PLS-Trees and PLS-GP. To illustrate our methodology, we use simulated examples and the analysis of preferences of orange juice and predicting wine quality as a function of input characteristics. We also illustrate Brillinger's estimation procedure to provide the feature selection and data dimension reduction. Finally, we conclude with directions for future research.

stat.ME↗

arXiv2021

Bayesian Inference for Gamma Models

We use the theory of normal variance-mean mixtures to derive a data augmentation scheme for models that include gamma functions. Our methodology applies to many situations in statistics and machine learning, including Multinomial-Dirichlet distributions, Negative binomial regression, Poisson-Gamma hierarchical models, Extreme value models, to name but a few. All of those models include a gamma function which does not admit a natural conjugate prior distribution providing a significant challenge to inference and prediction. To provide a data augmentation strategy, we construct and develop the theory of the class of Exponential Reciprocal Gamma distributions. This allows scalable EM and MCMC algorithms to be developed. We illustrate our methodology on a number of examples, including gamma shape inference, negative binomial regression and Dirichlet allocation. Finally, we conclude with directions for future research.

stat.ME↗

arXiv2018

Deep Learning: Computational Aspects

In this article we review computational aspects of Deep Learning (DL). Deep learning uses network architectures consisting of hierarchical layers of latent variables to construct predictors for high-dimensional input-output models. Training a deep learning architecture is computationally intensive, and efficient linear algebra libraries is the key for training and inference. Stochastic gradient descent (SGD) optimization and batch sampling are used to learn from massive data sets.

cs.LG↗

arXiv2018

Posterior Concentration for Sparse Deep Learning

Spike-and-Slab Deep Learning (SS-DL) is a fully Bayesian alternative to Dropout for improving generalizability of deep ReLU networks. This new type of regularization enables provable recovery of smooth input-output maps with unknown levels of smoothness. Indeed, we show that the posterior distribution concentrates at the near minimax rate for $\alpha$-H\"older smooth maps, performing as well as if we knew the smoothness level $\alpha$ ahead of time. Our result sheds light on architecture design for deep neural networks, namely the choice of depth, width and sparsity level. These network attributes typically depend on unknown smoothness in order to be optimal. We obviate this constraint with the fully Bayes construction. As an aside, we show that SS-DL does not overfit in the sense that the posterior concentrates on smaller networks with fewer (up to the optimal number of) nodes and links. Our results provide new theoretical justifications for deep ReLU networks from a Bayesian point of view.

stat.ML↗

arXiv2017

Statistical sparsity

The main contribution of this paper is a mathematical definition of statistical sparsity, which is expressed as a limiting property of a sequence of probability distributions. The limit is characterized by an exceedance measure~$H$ and a rate parameter~$\rho > 0$, both of which are unrelated to sample size. The definition is sufficient to encompass all sparsity models that have been suggested in the signal-detection literature. Sparsity implies that $\rho$~is small, and a sparse approximation is asymptotic in the rate parameter, typically with error $o(\rho)$ in the sparse limit $\rho \to 0$. To first order in sparsity, the sparse signal plus Gaussian noise convolution depends on the signal distribution only through its rate parameter and exceedance measure. This is one of several asymptotic approximations implied by the definition, each of which is most conveniently expressed in terms of the zeta-transformation of the exceedance measure. One implication is that two sparse families having the same exceedance measure are inferentially equivalent, and cannot be distinguished to first order. A converse implication for methodological strategy is that it may be more fruitful to focus on the exceedance measure, ignoring aspects of the signal distribution that have negligible effect on observables and on inferences. From this point of view, scale models and inverse-power measures seem particularly attractive.

stat.ME↗

arXiv2017

Sparse Regularization in Marketing and Economics

Sparse alpha-norm regularization has many data-rich applications in Marketing and Economics. Alpha-norm, in contrast to lasso and ridge regularization, jumps to a sparse solution. This feature is attractive for ultra high-dimensional problems that occur in demand estimation and forecasting. The alpha-norm objective is nonconvex and requires coordinate descent and proximal operators to find the sparse solution. We study a typical marketing demand forecasting problem, grocery store sales for salty snacks, that has many dummy variables as controls. The key predictors of demand include price, equivalized volume, promotion, flavor, scent, and brand effects. By comparing with many commonly used machine learning methods, alpha-norm regularization achieves its goal of providing accurate out-of-sample estimates for the promotion lift effects. Finally, we conclude with directions for future research.

stat.ML↗

arXiv2017

Deep Learning: A Bayesian Perspective

Deep learning is a form of machine learning for nonlinear high dimensional pattern matching and prediction. By taking a Bayesian probabilistic perspective, we provide a number of insights into more efficient algorithms for optimisation and hyper-parameter tuning. Traditional high-dimensional data reduction techniques, such as principal component analysis (PCA), partial least squares (PLS), reduced rank regression (RRR), projection pursuit regression (PPR) are all shown to be shallow learners. Their deep learning counterparts exploit multiple deep layers of data reduction which provide predictive performance gains. Stochastic gradient descent (SGD) training optimisation and Dropout (DO) regularization provide estimation and variable selection. Bayesian regularization is central to finding weights and connections in networks to optimize the predictive bias-variance trade-off. To illustrate our methodology, we provide an analysis of international bookings on Airbnb. Finally, we conclude with directions for future research.

stat.ML↗

arXiv2016

Sequential Bayesian Learning for Merton's Jump Model with Stochastic Volatility

Jump stochastic volatility models are central to financial econometrics for volatility forecasting, portfolio risk management, and derivatives pricing. Markov Chain Monte Carlo (MCMC) algorithms are computationally unfeasible for the sequential learning of volatility state variables and parameters, whereby the investor must update all posterior and predictive densities as new information arrives. We develop a particle filtering and learning algorithm to sample posterior distribution in Merton's jump stochastic volatility. This allows to filter spot volatilities and jump times, together with sequentially updating (learning) of jump and volatility parameters. We illustrate our methodology on Google's stock return. We conclude with directions for future research.

stat.AP↗