SearcharxivSearch

arXiv subjects

Darinka Dentcheva

Publications and source records attributed to Darinka Dentcheva.

17 recordsLinked to original sources

Fairness in multi-class multi-group classification problems via contextial coherent risk measures

We propose a new design of fair classifiers for multi-class classification problems in the presence of vector-valued sensitive attributes. In that scenario each sensitive attribute has multiple values and forms several groups relevant to the fairness consideration. Naturally those groups are overlapping and one should also analyze the interaction of factors. Additionally, the decision makers aided by the classification should not violate individual rights at the expense of satisfying fairness metrics at the group level. We propose an approach using the theory and methods of coherent measures of risk aiming at resolving the fairness challenges. Further, we propose a specialized numerical method for solving the resulting optimization problem. The method scales well with the increase of the number of observations. Additionally, we note that the obtained classifier is robust with respect to corrupted data or to situation when data is scarce. We demonstrate the advantages of the proposed framework in comparison to the support-vector machine framework and other methods handling fairness.

stat.ML

Risk-averse Fair Multi-class Classification

We develop a new classification framework based on the theory of coherent risk measures and systemic risk. The proposed approach is suitable for multi-class problems when the data is noisy, scarce (relative to the dimension of the problem), and the labeling might be unreliable. In the first part of our paper, we provide the foundation of the use of systemic risk models and show how to apply it in the context of linear and kernel-based multi-class problems. More advanced formulation via a system-theoretic approach with non-linear aggregation is proposed, which leads to a two-stage stochastic programming problem. A risk-averse regularized decomposition method is designed to solve the problem. We use a popular multi-class method as a benchmark in the performance analysis of the proposed classification methods. We illustrate our ideas by proposing several generalization of that method by the use of coherent measures of risk. The viability of the proposed risk-averse methods are supported theoretically and numerically. Additionally, we demonstrate that the application of systemic risk measures facilitates enforcing fairness in classification. Analysis and experiments regarding the fairness of the proposed models are carefully conducted. For all methods, our numerical experiments demonstrate that they are robust in the presence of unreliable training data and perform better on unknown data than the methods minimizing expected classification errors. Furthermore, the performance improves when the number of classes increases.

stat.ML

An Axiomatic Risk-Reward Framework for Sustainable Investing

Continued interest in sustainable investing calls for an axiomatic approach to measures of risk and reward that focus not only on financial returns, but also on measures of environmental and social sustainability, i.e. environmental, social, and governance (ESG) scores. We propose definitions for ESG-coherent risk measures and ESG reward-risk ratios based on functions of bivariate random variables that are applied to financial returns and real-time ESG scores, extending the traditional univariate measures to the ESG case. We provide examples and present an empirical analysis in which the ESG-coherent risk measures and ESG reward-risk ratios are used to rank stocks.

q-fin.MF

Subdifferentials of Convex Operators Valued in the Space of Integrable Functions with Application to Risk-Averse Optimization

We study differentiability properties of convex operators defined on a Banach space with values in an $\Lc_p$ space and of their compositions with monotonic convex functionals on this space. We develop new tools for operators enjoying an additional feature known as the local property. The new approach and results go beyond the classical theory of normal integrands and lattice-valued operators. We further describe the subdifferentials of compositions of such operators with convex monotonic functionals. The new results are applied to obtain novel optimality conditions in the subdifferential form for a broad class of risk-averse stochastic optimization problems with risk functionals as objectives, with partial information, and with stochastic dominance constraints. While our analysis is motivated by the theory and methods of risk-averse optimization, it addresses problems of a more general structure and has potential for further applications.

math.OC

Fair Risk Optimization of Distributed Systems

The paper provides a framework for the assessment and optimization of the total risk of complex distributed systems. The framework takes into account the risk of each agent, which may arise from heterogeneous sources, as well as the risk associated with the efficient operation of the system as a whole. The challenges posed by this task are associated with the lack of additivity of risk, the need to evaluate the risk of every agent (unit) using confidential or proprietary information, and the requirement of fair risk allocation to agents (units). We analyze systemic risk measures that are based on a sound axiomatic foundation while at the same time facilitate risk-averse sequential decision-making by distributed numerical methods, which allow the agents to operate autonomously with minimal exchange of information. We formulate a two-stage decision problem for a distributed system using systemic measures of risk and devise a decomposition method for solving the problem. The method is applied to a disaster management problem. We have paid attention to maintain fair risk allocation to all areas in the course of the relief operation. Our numerical results show the efficiency of the proposed methodology.

math.OC

Relaxation of stochastic dominance constraints via optimal mass transport

Optimization problems with stochastic dominance constraints provide a possibility to shape risk by selecting a benchmark random outcome with a desired distribution. The comparison of the relevant random outcomes to the respective benchmarks requires functional inequalities between the distribution functions or their transforms. A difficulty arises when no feasible decision results in a distribution that dominates the benchmark. Our paper addresses the problem of choosing a tight relaxation of the stochastic dominance constraint by selecting a feasible distribution, which is closest to those dominating the benchmark in terms of mass transportation distance. For the second-order stochastic dominance in a standard atomless space, we obtain new explicit formulae for the Monge-Kantorovich transportation distance of a given distribution to the set of dominating distributions. We use our results to construct a numerical method for solving the relaxation problem. Under an additional assumption, we also construct the associated projection of the distribution of interest onto the set of distributions dominating the benchmark. For the stochastic dominance relations of order $r\in[1,\infty)$, we show a lower bound for the relevant mass transportation distance. Our numerical experience illustrates the efficiency of the proposed approach.

math.OC

Asset liability management under sequential stochastic dominance constraints

We consider a financial intermediary managing assets and liabilities exposed to several risk sources and seeking an optimal portfolio strategy to minimise the initial capital invested and the total risk associated with investment losses and financial debt. We formulate the problem as a multistage stochastic programming model, with a time-consistent dynamic risk measure in the objective function to control the investment risk. To ensure that the intermediary's financial equilibrium is preserved, we introduce a funding constraint in the model by enforcing in a time-consistent manner a sequential second-order stochastic dominance (SSD) of the portfolio return distribution over the liability distribution. We demonstrate that imposing the SSD constraint at the last-but-one stage is sufficient to enforce the SSD ordering at each stage. To deal with the computational burden of associated MSP, we develop a novel decomposition scheme integrating, for the first time in the literature, time-consistent dynamic risk measures and sequential stochastic dominance constraints. The proposed methodology is computationally validated on a case study developed on a property and casualty ALM problem.

math.OC

Central limit theorems for vector-valued composite functionals with smoothing and applications

This paper focuses on vector-valued composite functionals, which may be nonlinear in probability. Our primary goal is to establish central limit theorems for these functionals when mixed estimators are employed. Our study is relevant to the evaluation and comparison of risk in decision-making contexts and extends to functionals that arise in machine learning methods. A generalized family of composite risk functionals is presented, which encompasses most of the known coherent risk measures including systemic measures of risk. The paper makes two main contributions. First, we analyze vector-valued functionals, providing a framework for evaluating high-dimensional risks. This framework facilitates the comparison of multiple risk measures, as well as the estimation and asymptotic analysis of systemic risk and its optimal value in decision-making problems. Second, we derive novel central limit theorems for optimized composite functionals when mixed types of estimators: empirical and smoothed estimators are used. We provide verifiable sufficient conditions for the central limit formulae and show their applicability to several popular measures of risk.

math.ST

On Risk Evaluation and Control of Distributed Multi-Agent Systems

In this paper, we deal with risk evaluation and risk-averse optimization of complex distributed systems with general risk functionals. We postulate a novel set of axioms for the functionals evaluating the total risk of the system. We derive a dual representation for the systemic risk measures and propose a way to construct non-trivial families of measures by using either a collection of linear scalarizations or non-linear risk aggregation. The new framework facilitates risk-averse sequential decision-making by distributed methods. The proposed approach is compared theoretically and numerically to some of the systemic risk measurements in the existing literature. We formulate a two-stage decision problem with monotropic structure and systemic measure of risk. The structure is typical for distributed systems arising in energy networks, robotics, and other practical situations. A distributed decomposition method for solving the two-stage problem is proposed and it is applied to a problem arising in communication networks. We have used this problem to compare the methods of systemic risk evaluation. We show that the proposed risk aggregation leads to less conservative risk evaluation and results in a substantially better solution of the problem at hand as compared to an aggregation of the risk of individual agents and other methods.

math.OC

Stability and Sample-based Approximations of Composite Stochastic Optimization Problems

Optimization under uncertainty and risk is indispensable in many practical situations. Our paper addresses stability of optimization problems using composite risk functionals which are subjected to measure perturbations. Our main focus is the asymptotic behavior of data-driven formulations with empirical or smoothing estimators such as kernels or wavelets applied to some or to all functions of the compositions. We analyze the properties of the new estimators and we establish strong law of large numbers, consistency, and bias reduction potential under fairly general assumptions. Our results are germane to risk-averse optimization and to data science in general.

math.OC

Bias Reduction in Sample-Based Optimization

We consider stochastic optimization problems which use observed data to estimate essential characteristics of the random quantities involved. Sample average approximation (SAA) or empirical (plug-in) estimation are very popular ways to use data in optimization. It is well known that sample average optimization suffers from downward bias. We propose to use smooth estimators rather than empirical ones in optimization problems. We establish consistency results for the optimal value and the set of optimal solutions of the new problem formulation. The performance of the proposed approach is compared to SAA theoretically and numerically. We analyze the bias of the new problems and identify sufficient conditions for ensuring less biased estimation of the optimal value of the true problem. At the same time, the error of the new estimator remains controlled. We show that those conditions are satisfied for many popular statistical problems such as regression models, classification problems, and optimization problems with Average (Conditional) Value-at-Risk. We have observed that smoothing the least-squares objective in a regression problem by a normal kernel leads to a ridge regression. Our numerical experience shows that the new estimators frequently exhibit also smaller variance and smaller mean-square error than those of SAA.

math.ST

Subregular Recourse in Nonlinear Multistage Stochastic Optimization

We consider nonlinear multistage stochastic optimization problems in the spaces of integrable functions. We allow for nonlinear dynamics and general objective functionals, including dynamic risk measures. We study causal operators describing the dynamics of the system and derive the Clarke subdifferential for a penalty function involving such operators. Then we introduce the concept of subregular recourse in nonlinear multistage stochastic optimization and establish subregularity of the resulting systems in two formulations: with built-in nonanticipativity and with explicit nonanticipativity constraints. Finally, we derive optimality conditions for both formulations and study their relations.

math.OC

Risk Forms: Representation, Disintegration, and Application to Partially Observable Two-Stage Systems

We introduce the concept of a risk form, which is a real functional of two arguments: a measurable function on a Polish space and a measure on that space. We generalize the duality theory and the Kusuoka representation to this setting. For a risk form acting on a product of Polish spaces, we define marginal and conditional forms and we prove a disintegration formula, which represents a risk form as a composition of its marginal and conditional forms. We apply the proposed approach to two-stage stochastic programming problems with partial information and decision-dependent observation distribution.

math.OC

Risk-Averse Classification

We develop a new approach to solving classification problems, which is bases on the theory of coherent measures of risk and risk sharing ideas. The proposed approach aims at designing a risk-averse classifier. The new approach allows for associating distinct risk functional to each classes. The risk may be measured by different (non-linear in probability) measures, We analyze the structure of the new classifier design problem and establish its theoretical relation to known risk-neutral design problems. In particular, we show that the risk-sharing classification problem is equivalent to an implicitly defined optimization problem with unequal, implicitly defined but unknown, weights for each data point. We implement our methodology in a binary classification scenario on several different data sets and carry out numerical comparison with classifiers which are obtained using the Huber loss function and other loss functions known in the literature. We formulate specific risk-averse support vector machines in order to demonstrate the viability of our method.

stat.ML

Time-Consistent Risk Measures for Continuous-Time Markov Chains

We develop an approach to time-consistent risk evaluation of continuous-time processes in Markov systems. Our analysis is based on dual representation of coherent risk measures, differentiability concepts for multivalued mappings, and a refined concept of time consistency. We prove that the risk measures are defined by a family of risk evaluation functionals (transition risk mappings), which depend on state, time, and the transition function. Their dual representations are risk multikernels of the Markov system. We introduce the concept of a semi-derivative of a risk multikernel and use it to generalize the concept of a generator of a Markov process. Using these semi-derivatives, we derive a system of ordinary differential equations that the risk evaluation must satisfy, which generalize the classical backward Kolmogorov equations for Markov processes. Additionally, we construct convergent discrete-time approximations to the continuous-time risk measures.

math.OC

Statistical Estimation of Composite Risk Functionals and Risk Optimization Problems

We address the statistical estimation of composite functionals which may be nonlinear in the probability measure. Our study is motivated by the need to estimate coherent measures of risk, which become increasingly popular in finance, insurance, and other areas associated with optimization under uncertainty and risk. We establish central limit formulae for composite risk functionals. Furthermore, we discuss the asymptotic behavior of optimization problems whose objectives are composite risk functionals and we establish a central limit formula of their optimal values when an estimator of the risk functional is used. While the mathematical structures accommodate commonly used coherent measures of risk, they have more general character, which may be of independent interest.

math.ST

Common Mathematical Foundations of Expected Utility and Dual Utility Theories

We show that the main results of the expected utility and dual utility theories can be derived in a unified way from two fundamental mathematical ideas: the separation principle of convex analysis, and integral representations of continuous linear functionals from functional analysis. Our analysis reveals the dual character of utility functions. We also derive new integral representations of dual utility models.

math.FA