arXiv subjects

Petar Djuric

Publications and source records attributed to Petar Djuric.

7 recordsLinked to original sources

arXiv2026

Higher-Order Feature Attribution: Bridging Statistics, Explainable AI, and Topological Signal Processing

Feature attributions are post-training analysis methods that assess how various input features of a machine learning model contribute to an output prediction. Their interpretation is straightforward when features act independently, but it becomes less clear when the predictive model involves interactions, such as multiplicative relationships or joint feature contributions. In this work, we propose a general theory of higher-order feature attribution, which we develop on the foundation of Integrated Gradients (IG). This work extends existing frameworks in the literature on explainable AI. When using IG as the method of feature attribution, we discover natural connections to statistics and topological signal processing. We provide several theoretical results that establish the theory, and we validate our theory on a few examples.

cs.LG↗

arXiv2025

Trustworthy Prediction with Gaussian Process Knowledge Scores

Probabilistic models are often used to make predictions in regions of the data space where no observations are available, but it is not always clear whether such predictions are well-informed by previously seen data. In this paper, we propose a knowledge score for predictions from Gaussian process regression (GPR) models that quantifies the extent to which observing data have reduced our uncertainty about a prediction. The knowledge score is interpretable and naturally bounded between 0 and 1. We demonstrate in several experiments that the knowledge score can anticipate when predictions from a GPR model are accurate, and that this anticipation improves performance in tasks such as anomaly detection, extrapolation, and missing data imputation. Source code for this project is available online at https://github.com/KurtButler/GP-knowledge.

stat.ML↗

arXiv2024

Sequential Estimation of Gaussian Process-based Deep State-Space Models

We consider the problem of sequential estimation of the unknowns of state-space and deep state-space models that include estimation of functions and latent processes of the models. The proposed approach relies on Gaussian and deep Gaussian processes that are implemented via random feature-based Gaussian processes. In these models, we have two sets of unknowns, highly nonlinear unknowns (the values of the latent processes) and conditionally linear unknowns (the constant parameters of the random feature-based Gaussian processes). We present a method based on particle filtering where the parameters of the random feature-based Gaussian processes are integrated out in obtaining the predictive density of the states and do not need particles. We also propose an ensemble version of the method, with each member of the ensemble having its own set of features. With several experiments, we show that the method can track the latent processes up to a scale and rotation.

cs.LG↗

arXiv2024

Gaussian Process-Gated Hierarchical Mixtures of Experts

In this paper, we propose novel Gaussian process-gated hierarchical mixtures of experts (GPHMEs). Unlike other mixtures of experts with gating models linear in the input, our model employs gating functions built with Gaussian processes (GPs). These processes are based on random features that are non-linear functions of the inputs. Furthermore, the experts in our model are also constructed with GPs. The optimization of the GPHMEs is performed by variational inference. The proposed GPHMEs have several advantages. They outperform tree-based HME benchmarks that partition the data in the input space, and they achieve good performance with reduced complexity. Another advantage is the interpretability they provide for deep GPs, and more generally, for deep Bayesian neural networks. Our GPHMEs demonstrate excellent performance for large-scale data sets, even with quite modest sizes.

cs.LG↗

arXiv2021

Bayesian Nonparametric Dimensionality Reduction of Categorical Data for Predicting Severity of COVID-19 in Pregnant Women

The coronavirus disease (COVID-19) has rapidly spread throughout the world and while pregnant women present the same adverse outcome rates, they are underrepresented in clinical research. We collected clinical data of 155 test-positive COVID-19 pregnant women at Stony Brook University Hospital. Many of these collected data are of multivariate categorical type, where the number of possible outcomes grows exponentially as the dimension of data increases. We modeled the data within the unsupervised Bayesian framework and mapped them into a lower-dimensional space using latent Gaussian processes. The latent features in the lower dimensional space were further used for predicting if a pregnant woman would be admitted to a hospital due to COVID-19 or would remain with mild symptoms. We compared the prediction accuracy with the dummy/one-hot encoding of categorical data and found that the latent Gaussian process had better accuracy.

stat.AP↗

arXiv2015

A Bayesian Approach to Data Fusion in Sensor Networks

In this paper, we address the fusion problem in wireless sensor networks, where the cross-correlation between the estimates is unknown. To solve the problem within the Bayesian framework, we assume that the covariance matrix has a prior distribution. We also assume that we know the covariance of each estimate, i.e., the diagonal block of the entire covariance matrix (of the random vector consisting of the two estimates). We then derive the conditional distribution of the off-diagonal blocks, which is the cross-correlation of our interest. We show that when there are two nodes, the conditional distribution happens to be the inverted matrix variate $t$-distribution, from which we can readily sample. For more than two nodes, the conditional distribution is no longer the inverted matrix variate $t$-distribution. But we show that we can decompose it into several sampling problems, each of which is the inverted matrix variate $t$-distribution and therefore we can still sample from it. Since we can sample from this distribution, it enables us to use the Monte Carlo method to compute the minimum mean square error estimate for the fusion problem. We use two models to generate experiment data and demonstrate the generality of our method. Simulation results show that the proposed method works better than the popular covariance intersection method.

cs.IT↗

arXiv2014

A Search-free DOA Estimation Algorithm for Coprime Arrays

Recently, coprime arrays have been in the focus of research because of their potential in exploiting redundancy in spanning large apertures with fewer elements than suggested by theory. A coprime array consists of two uniform linear subarrays with inter-element spacings $Mλ/2$ and $Nλ/2$, where $M$ and $N$ are coprime integers and $λ$ is the wavelength of the signal. In this paper, we propose a fast search-free method for direction-of-arrival (DOA) estimation with coprime arrays. It is based on the use of methods that operate on the uniform linear subarrays of the coprime array and that enjoy many processing advantages. We first estimate the DOAs for each uniform linear subarray separately and then combine the estimates from the subarrays. For combining the estimates, we propose a method that projects the estimated point in the two-dimensional plane onto one-dimensional line segments that correspond to the entire angular domain. By doing so, we avoid the search step and consequently, we greatly reduce the computational complexity of the method. We demonstrate the performance of the method with computer simulations and compare it with that of the FD-root MUSIC method.

cs.IT↗