arXiv subjects

Alexey Miroshnikov

Publications and source records attributed to Alexey Miroshnikov.

At least 19 recordsLinked to original sources

arXiv2025

Explainable post-training bias mitigation with distribution-based fairness metrics

We develop a novel bias mitigation framework with distribution-based fairness constraints suitable for producing demographically blind and explainable machine-learning models across a wide range of fairness levels. This is accomplished through post-processing, allowing fairer models to be generated efficiently without retraining the underlying model. Our framework, which is based on stochastic gradient descent, can be applied to a wide range of model types, with a particular emphasis on the post-processing of gradient-boosted decision trees. Additionally, we design a broad family of global fairness metrics, along with differentiable and consistent estimators compatible with our framework, building on previous work. We empirically test our methodology on a variety of datasets and compare it with alternative post-processing approaches, including Bayesian search, optimal transport projection, and direct neural network training.

cs.LG↗

arXiv2024

Stability theory of game-theoretic group feature explanations for machine learning models

In this article, we study feature attributions of Machine Learning (ML) models originating from linear game values and coalitional values defined as operators on appropriate functional spaces. The main focus is on random games based on the conditional and marginal expectations. The first part of our work formulates a stability theory for these explanation operators by establishing certain bounds for both marginal and conditional explanations. The differences between the two games are then elucidated, such as showing that the marginal explanations can become discontinuous on some naturally-designed domains, while the conditional explanations remain stable. In the second part of our work, group explanation methodologies are devised based on game values with coalition structure, where the features are grouped based on dependencies. We show analytically that grouping features this way has a stabilizing effect on the marginal operator on both group and individual levels, and allows for the unification of marginal and conditional explanations. Our results are verified in a number of numerical experiments where an information-theoretic measure of dependence is used for grouping.

cs.GT↗

arXiv2024

On marginal feature attributions of tree-based models

Due to their power and ease of use, tree-based machine learning models, such as random forests and gradient-boosted tree ensembles, have become very popular. To interpret them, local feature attributions based on marginal expectations, e.g. marginal (interventional) Shapley, Owen or Banzhaf values, may be employed. Such methods are true to the model and implementation invariant, i.e. dependent only on the input-output function of the model. We contrast this with the popular TreeSHAP algorithm by presenting two (statistically similar) decision trees that compute the exact same function for which the "path-dependent" TreeSHAP yields different rankings of features, whereas the marginal Shapley values coincide. Furthermore, we discuss how the internal structure of tree-based models may be leveraged to help with computing their marginal feature attributions according to a linear game value. One important observation is that these are simple (piecewise-constant) functions with respect to a certain grid partition of the input space determined by the trained model. Another crucial observation, showcased by experiments with XGBoost, LightGBM and CatBoost libraries, is that only a portion of all features appears in a tree from the ensemble. Thus, the complexity of computing marginal Shapley (or Owen or Banzhaf) feature attributions may be reduced. This remains valid for a broader class of game values which we shall axiomatically characterize. A prime example is the case of CatBoost models where the trees are oblivious (symmetric) and the number of features in each of them is no larger than the depth. We exploit the symmetry to derive an explicit formula, with improved complexity and only in terms of the internal model parameters, for marginal Shapley (and Banzhaf and Owen) values of CatBoost models. This results in a fast, accurate algorithm for estimating these feature attributions.

cs.LG↗

arXiv2024

Approximation of group explainers with coalition structure using Monte Carlo sampling on the product space of coalitions and features

In recent years, many Machine Learning (ML) explanation techniques have been designed using ideas from cooperative game theory. These game-theoretic explainers suffer from high complexity, hindering their exact computation in practical settings. In our work, we focus on a wide class of linear game values, as well as coalitional values, for the marginal game based on a given ML model and predictor vector. By viewing these explainers as expectations over appropriate sample spaces, we design a novel Monte Carlo sampling algorithm that estimates them at a reduced complexity that depends linearly on the size of the background dataset. We set up a rigorous framework for the statistical analysis and obtain error bounds for our sampling methods. The advantage of this approach is that it is fast, easily implementable, and model-agnostic. Furthermore, it has similar statistical accuracy as other known estimation techniques that are more complex and model-specific. We provide rigorous proofs of statistical convergence, as well as numerical experiments whose results agree with our theoretical findings.

cs.LG↗

arXiv2023

Asymptotic properties and approximation of Bayesian logspline density estimators for communication-free parallel computing methods

In this article we perform an asymptotic analysis of parallel Bayesian logspline density estimators. Such estimators are useful for the analysis of datasets that are partitioned into subsets and stored in separate databases without the capability of accessing the full dataset from a single computer. The parallel estimator we introduce is in the spirit of a kernel density estimator introduced in recent studies. We provide a numerical procedure that produces the normalized density estimator itself in place of the sampling algorithm. We then derive an error bound for the mean integrated squared error of the full dataset posterior estimator. The error bound depends upon the parameters that arise in logspline density estimation and the numerical approximation procedure. In our analysis, we identify the choices for the parameters that result in the error bound scaling optimally in relation to the number of samples. This provides our method with increased estimation accuracy, while also minimizing the computational cost.

math.ST↗

arXiv2022

Wasserstein-based fairness interpretability framework for machine learning models

The objective of this article is to introduce a fairness interpretability framework for measuring and explaining the bias in classification and regression models at the level of a distribution. In our work, we measure the model bias across sub-population distributions in the model output using the Wasserstein metric. To properly quantify the contributions of predictors, we take into account the favorability of both the model and predictors with respect to the non-protected class. The quantification is accomplished by the use of transport theory, which gives rise to the decomposition of the model bias and bias explanations to positive and negative contributions. To gain more insight into the role of favorability and allow for additivity of bias explanations, we adapt techniques from cooperative game theory.

cs.LG↗

arXiv2021

Model-agnostic bias mitigation methods with regressor distribution control for Wasserstein-based fairness metrics

This article is a companion paper to our earlier work Miroshnikov et al. (2021) on fairness interpretability, which introduces bias explanations. In the current work, we propose a bias mitigation methodology based upon the construction of post-processed models with fairer regressor distributions for Wasserstein-based fairness metrics. By identifying the list of predictors contributing the most to the bias, we reduce the dimensionality of the problem by mitigating the bias originating from those predictors. The post-processing methodology involves reshaping the predictor distributions by balancing the positive and negative bias explanations and allows for the regressor bias to decrease. We design an algorithm that uses Bayesian optimization to construct the bias-performance efficient frontier over the family of post-processed models, from which an optimal model is selected. Our novel methodology performs optimization in low-dimensional spaces and avoids expensive model retraining.

cs.LG↗

arXiv2017

Computing the joint distribution of the total tree length across loci in populations with variable size

In recent years, a number of methods have been developed to infer complex demographic histories, especially historical population size changes, from genomic sequence data. Coalescent Hidden Markov Models have proven to be particularly useful for this type of inference. Due to the Markovian structure of these models, an essential building block is the joint distribution of local genealogical trees, or statistics of these genealogies, at two neighboring loci in populations of variable size. Here, we present a novel method to compute the marginal and the joint distribution of the total length of the genealogical trees at two loci separated by at most one recombination event for samples of arbitrary size. To our knowledge, no method to compute these distributions has been presented in the literature to date. We show that they can be obtained from the solution of certain hyperbolic systems of partial differential equations. We present a numerical algorithm, based on the method of characteristics, that can be used to efficiently and accurately solve these systems and compute the marginal and the joint distributions. We demonstrate its utility to study the properties of the joint distribution. Our flexible method can be straightforwardly extended to handle an arbitrary fixed number of recombination events, to include the distributions of other statistics of the genealogies as well, and can also be applied in structured populations.

q-bio.PE↗

arXiv2017

On the construction and properties of weak solutions describing dynamic cavitation

We consider the problem of dynamic cavity formation in isotropic compressible nonlinear elastic media. For the equations of radial elasticity we construct self-similar weak solutions that describe a cavity emanating from a state of uniform deformation. For dimensions $d =2, 3$ we show that cavity formation is necessarily associated with a unique precursor shock. We also study the bifurcation diagram and do a detailed analysis of the singular asymptotics associated to cavity initiation as a function of the cavity speed of the self-similar profiles. We show that for stress free cavities the critical stretching associated with dynamically cavitating solutions coincides with the critical stretching in the bifurcation diagram of equilibrium elasticity. Our analysis treats both stress-free cavities and cavities with contents.

math.AP↗

arXiv2017

Cellulose Biodegradation Models; An Example of Cooperative Interactions in Structured Populations

We introduce various models for cellulose bio-degradation by micro-organisms. Those models rely on complex chemical mechanisms, involve the structure of the cellulose chains and are allowed to depend on the phenotypical traits of the population of micro-organisms. We then use the corresponding models in the context of multiple-trait populations. This leads to classical, logistic type, reproduction rates limiting the growth of large populations but also, and more surprisingly, limiting the growth of populations which are too small in a manner similar to the effects seen in populations requiring cooperative interactions (or sexual reproduction). This study hence offers a striking example of how some mechanisms resembling cooperation can occur in structured biological populations, even in the absence of any actual cooperation.

math.DS↗

arXiv2017

Weak* Solutions II: The Vacuum in Lagrangian Gas Dynamics

We develop a framework in which to make sense of solutions containing the vacuum in Lagrangian gas dynamics. At and near vacuum, the specific volume becomes infinite and enclosed vacuums are represented by Dirac masses, so they cannot be treated in the usual weak sense. However, the weak* solutions recently introduced by the authors can be extended to include solutions containing vacuums. We present a definition of these natural vacuum solutions and provide explicit examples which demonstrate some of their features. Our examples are isentropic for clarity, and we briefly discuss the extension to the full $3\times3$ system of gas dynamics. We also extend our methods to one-dimensional dynamic elasticity to show that fractures cannot form in an entropy solution.

math.AP↗

arXiv2017

Stability and Convergence of Relaxation Schemes to Hyperbolic Balance Laws via a Wave Operator

This article deals with relaxation approximations of nonlinear systems of hyperbolic balance laws. We introduce a class of relaxation schemes and establish their stability and convergence to the solution of hyperbolic balance laws before the formation of shocks, provided that we are within the framework of the compensated compactness method. Our analysis treats systems of hyperbolic balance laws with source terms satisfying a special mechanism which induces weak dissipation in the spirit of Dafermos [C.M. Dafermos J. Hyp. Diff. Equations, 3, 505-527, 2006], as well as hyperbolic balance laws with more general source terms. The rate of convergence of the relaxation system to a solution of the balance laws in the smooth regime is established. Our work follows in spirit the analysis presented in [S. Jin, X. Xin, Comm. Pure. Appl. Math. (1995), 48] and [Ch. Arvanitis, Ch. Makridakis, and A.E. Tzavaras, SIAM J. on Num. Anal. (2005), 42-4] for systems of hyperbolic conservation laws without source terms.

math.AP↗

arXiv2017

The problem of dynamic cavitation in nonlinear elasticity

The notion of singular limiting induced from continuum solutions (slic-solutions) is applied to the problem of cavitation in nonlinear elasticity, in order to re-assess an example of non-uniqueness of entropic weak solutions (with polyconvex energy) due to a forming cavity.

math.AP↗

arXiv2017

Asymptotic properties of parallel Bayesian kernel density estimators

In this article we perform an asymptotic analysis of Bayesian parallel kernel density estimators introduced by Neiswanger, Wang and Xing (2014). We derive the asymptotic expansion of the mean integrated squared error for the full data posterior estimator and investigate the properties of asymptotically optimal bandwidth parameters. Our analysis demonstrates that partitioning data into subsets requires a non-trivial choice of bandwidth parameters that optimizes the estimation error.

math.ST↗

arXiv2017

Weak* solutions I: A new perspective on solutions to systems of conservation laws

We introduce a new notion of solution, which we call weak* solutions, for systems of conservation laws. These solutions can be used to handle singular situations that standard weak solutions cannot, such as vacuums in Lagrangian gas dynamics or cavities in elasticity. Our framework allows us to treat the systems as ODEs in Banach space. Starting with the observation that solutions act linearly on test functions $α\in X$, we require solutions to take values in the dual space $X^*$ of $X$. Moreover, we weaken the usual requirement of measurability of solutions. In order to do this, we develop the calculus of the Gelfand integral, which is appropriate for weak* measurable functions. We then use the Gelfand calculus to define weak* solutions, and show that they are stronger than the usual notion of weak solution, although for $BV$ solutions the notions are equivalent. It is expected that these solutions will also shed light on vexing issues of ill-posedness for multi-dimensional systems.

math.AP↗

arXiv2016

Stability of fully discrete variational schemes for elastodynamics with a polyconvex stored energy

In this article we develop a fully discrete variational scheme that approximates the equations of three dimensional elastodynamics with polyconvex stored energy. The fully discrete scheme is based on a time-discrete variational scheme developed by S.~Demoulini, D.~M.~A.~Stuart and A.~E.~Tzavaras (2001). We show that the fully discrete scheme is unconditionally stable. The proof of stability is based on a relative entropy estimation for the fully discrete approximates.

math.AP↗

arXiv2015

Parallel Markov Chain Monte Carlo for Non-Gaussian Posterior Distributions

Recent developments in big data and analytics research have produced an abundance of large data sets that are too big to be analyzed in their entirety, due to limits on computer memory or storage capacity. To address these issues, communication-free parallel Markov chain Monte Carlo (MCMC) methods have been developed for Bayesian analysis of big data. These methods partition data into manageable subsets, perform independent Bayesian MCMC analysis on each subset, and combine the subset posterior samples to estimate the full data posterior. Current approaches to combining subset posterior samples include sample averaging, weighted averaging, and kernel smoothing techniques. Although these methods work well for Gaussian posteriors, they are not well-suited to non-Gaussian posterior distributions. Here, we develop a new direct density product method for combining subset marginal posterior samples to estimate full data marginal posterior densities. Using a commonly-implemented distance metric, we show in simulation studies of Bayesian models with non-Gaussian posteriors that our method outperforms the existing methods in approximating the full data marginal posteriors. Since our method estimates only marginal densities, there is no limitation on the number of model parameters analyzed. Our procedure is suitable for Bayesian models with unknown parameters with fixed dimension in continuous parameter spaces.

stat.ME↗

arXiv2015

BayesSummaryStatLM: An R package for Bayesian Linear Models for Big Data and Data Science

Recent developments in data science and big data research have produced an abundance of large data sets that are too big to be analyzed in their entirety, due to limits on either computer memory or storage capacity. Here, we introduce our R package 'BayesSummaryStatLM' for Bayesian linear regression models with Markov chain Monte Carlo implementation that overcomes these limitations. Our Bayesian models use only summary statistics of data as input; these summary statistics can be calculated from subsets of big data and combined over subsets. Thus, complete data sets do not need to be read into memory in full, which removes any physical memory limitations of a user. Our package incorporates the R package 'ff' and its functions for reading in big data sets in chunks while simultaneously calculating summary statistics. We describe our Bayesian linear regression models, including several choices of prior distributions for unknown model parameters, and illustrate capabilities and features of our R package using both simulated and real data sets.

stat.AP↗