arXiv subjects

Alexander Shapiro

Publications and source records attributed to Alexander Shapiro.

At least 37 records · Page 2Linked to original sources

arXiv2024

Rectangularity and duality of distributionally robust Markov Decision Processes

The main goal of this paper is to discuss several approaches to formulation of distributionally robust counterparts of Markov Decision Processes, where the transition kernels are not specified exactly but rather are assumed to be elements of the corresponding ambiguity sets. The intent is to clarify some connections between the game and static formulations of distributionally robust MDPs, and delineate the role of rectangularity associated with ambiguity sets in determining these connections.

math.OC↗

arXiv2023

Small-scale Dynamo in Cool Stars: I. Changes in stratification and near-surface convection for main-sequence spectral types

Some of the small-scale solar magnetic flux can be attributed to a small-scale dynamo (SSD) operating in the near-surface convection. The SSD fields have consequences for solar granular convection, basal flux, as well as chromospheric heating. A similar SSD mechanism is expected to be active in the near-surface convection of other cool main-sequence stars, but this has never been investigated. We aim to investigate changes in stratification and convection due to inclusion of SSD fields for F3V, G2V, K0V and M0V spectral types in the near-surface convection. 3D magnetohydrodynamic (MHD) models of the four stellar boxes, covering the subsurface convection zone up to the lower photosphere in a small cartesian box, are studied using the \textit{MURaM} radiative-MHD simulation code. The SSD runs are compared against reference hydrodynamic runs. An SSD is found to efficiently produce magnetic field with energies ranging between 5\% to 80\% of the plasma kinetic energy at different depths. This ratio tends to be larger for larger $T_{\mathrm{eff}}$. The relative change in density and gas pressure stratification for the deeper convective layers due to SSD magnetic fields is negligible, except for the F-star. For the F-star, there is a substantial reduction in convective velocities due to Lorentz force feedback from magnetic fields, which, in turn, reduces the turbulent pressure. SSD in near-surface convection for cool main-sequence stars introduces small but significant changes in thermodynamic stratification (especially for the F-star) due to reduction in convective velocities.

astro-ph.SR↗

arXiv2023

Risk-Averse Stochastic Optimal Control: an efficiently computable statistical upper bound

In this paper, we discuss an application of the SDDP type algorithm to nested risk-averse formulations of Stochastic Optimal Control (SOC) problems. We propose a construction of a statistical upper bound for the optimal value of risk-averse SOC problems. This outlines an approach to a solution of a long standing problem in that area of research. The bound holds for a large class of convex and monotone conditional risk mappings. Finally, we show the validity of the statistical upper bound to solve a real-life stochastic hydro-thermal planning problem.

math.OC↗

arXiv2023

Numerical Methods for Convex Multistage Stochastic Optimization

Optimization problems involving sequential decisions in a stochastic environment were studied in Stochastic Programming (SP), Stochastic Optimal Control (SOC) and Markov Decision Processes (MDP). In this paper we mainly concentrate on SP and SOC modelling approaches. In these frameworks there are natural situations when the considered problems are convex. Classical approach to sequential optimization is based on dynamic programming. It has the problem of the so-called ``Curse of Dimensionality", in that its computational complexity increases exponentially with increase of dimension of state variables. Recent progress in solving convex multistage stochastic problems is based on cutting planes approximations of the cost-to-go (value) functions of dynamic programming equations. Cutting planes type algorithms in dynamical settings is one of the main topics of this paper. We also discuss Stochastic Approximation type methods applied to multistage stochastic optimization problems. From the computational complexity point of view, these two types of methods seem to be complimentary to each other. Cutting plane type methods can handle multistage problems with a large number of stages, but a relatively smaller number of state (decision) variables. On the other hand, stochastic approximation type methods can only deal with a small number of stages, but a large number of decision variables.

math.OC↗

arXiv2023

Statistical Limit Theorems in Distributionally Robust Optimization

The goal of this paper is to develop methodology for the systematic analysis of asymptotic statistical properties of data driven DRO formulations based on their corresponding non-DRO counterparts. We illustrate our approach in various settings, including both phi-divergence and Wasserstein uncertainty sets. Different types of asymptotic behaviors are obtained depending on the rate at which the uncertainty radius decreases to zero as a function of the sample size and the geometry of the uncertainty sets.

math.OC↗

arXiv2023

Bayesian Distributionally Robust Optimization

We introduce a new framework, Bayesian Distributionally Robust Optimization (Bayesian-DRO), for data-driven stochastic optimization where the underlying distribution is unknown. Bayesian-DRO contrasts with most of the existing DRO approaches in the use of Bayesian estimation of the unknown distribution. To make computation of Bayesian updating tractable, Bayesian-DRO first assumes the underlying distribution takes a parametric form with unknown parameter and then computes the posterior distribution of the parameter. To address the model uncertainty brought by the assumed parametric distribution, Bayesian-DRO constructs an ambiguity set of distributions with the assumed parametric distribution as the reference distribution and then optimizes with respect to the worst case in the ambiguity set. We show the consistency of the Bayesian posterior distribution and subsequently the convergence of objective functions and optimal solutions of Bayesian-DRO. Our consistency result of the Bayesian posterior requires simpler assumptions than the classical literature on Bayesian consistency. We also consider several approaches for selecting the ambiguity set size in Bayesian-DRO and compare them numerically. Our numerical experiments demonstrate the out-of-sample performance of Bayesian-DRO in comparison with Kullback-Leibler-based (KL-) and Wasserstein-based empirical DRO as well as risk-neutral Bayesian Risk Optimization. Our numerical results shed light on how to choose the modeling framework (Bayesian-DRO, KL-DRO, Wasserstein-DRO) for specific problems, but the choice for general problems still remains an important and open question.

math.OC↗

arXiv2022

Multiwavelength Mitigation of Stellar Activity in Astrometric Planet Detection

Astrometry has long been a promising technique for exoplanet detection. At the theoretical limits, astrometry would allow for the detection of smaller planets than previously seen by current exoplanet search methods, but stellar activity may make these theoretical limits unreachable. Astrometric jitter of a Sun-like star due to magnetic activity in its photosphere induces apparent variability in the photocenter of order $0.5\ \textrm{m}R_\odot$. This jitter creates a fundamental astrophysical noise floor preventing detection of lower-mass planets in a single spectral band. By injecting planet orbits into simulated solar data at five different passbands, we investigate mitigation of this fundamental astrometric noise using correlations across passbands. For a true solar analog and a planet at 1 au semimajor axis, the $6σ$ detection limit set by stellar activity for an ideal telescope at the best single passband is $0.01$ Earth masses. We found that pairs of passbands with highly correlated astrometric jitter due to stellar activity, but with less motion in the redder band, enable higher-precision measurements of the common signal from the planet. Using this method improves detectable planet masses at 1 au by up to a factor of $10$, corresponding to at best $0.005$ Earth masses for a Sun-like star with a perfect telescope. Given these results, we recommend that future astrometry missions consider proceeding with two or more passbands to reduce noise due to stellar activity.

astro-ph.EP↗

arXiv2021

Mathematical Foundations of Distributionally Robust Multistage Optimization

Distributionally robust optimization involves various probability measures in its problem formulation. They can be bundled to constitute a risk functional. For this equivalence, risk functionals constitute a fundamental building block in distributionally robust stochastic programming. Multistage programming requires conditional versions of risk functionals to re-assess future risk after partial realizations and after preceding decisions. This paper discusses a construction of the conditional counterpart of a risk functional by passing its genuine characteristics to its conditional counterparts. The conditional risk functionals turn out to be different from the nested analogues of the original (law invariant) risk measure. It is demonstrated that the initial measure and its nested decomposition can be used in a distributionally robust multistage setting.

math.OC↗

arXiv2021

Quantum decorated character stacks

We initiate the study of decorated character stacks and their quantizations using the framework of stratified factorization homology. We thereby extend the construction by Fock and Goncharov of (quantum) decorated character varieties to encompass also the stacky points, in a way that is both compatible with cutting and gluing and equivariant with respect to canonical actions of the modular group of the surface. In the cases $G=SL_2,PGL_2$ we construct a system of categorical charts and flips on the quantum decorated character stacks which generalize the well--known cluster structures on the Fock--Goncharov moduli spaces.

math.QA↗

arXiv2021

Irradiance Variations of the Sun and Sun-Like Stars -- Overview of Topical Collection

This topical collection summarizes recent advances in observing and modeling irradiance variations of the Sun and Sun-like stars, emphasizing the links between surface magnetic fields and the resulting solar and stellar variability. In particular, the articles composing this collection summarize recent progress in i) solar-irradiance measurements; ii) modeling of solar- and stellar-irradiance variability; and iii) understanding of the effects of such variability on Earth's climate and exoplanet environments. This topical-collection overview article gives background and more details on these aspects of variability.

astro-ph.SR↗

arXiv2021

Testing Rank of Incomplete Unimodal Matrices

Several statistics-based detectors, based on unimodal matrix models, for determining the number of sources in a field are designed. A new variance ratio statistic is proposed, and its asymptotic distribution is analyzed. The variance ratio detector is shown to outperform the alternatives. It is shown that further improvements are achievable via optimally selected rotations. Numerical experiments demonstrate the performance gains of our detection methods over the baseline approach.

stat.AP↗

arXiv2020

On characteristic rank for matrix and tensor completion

In this lecture note, we discuss a fundamental concept, referred to as the {\it characteristic rank}, which suggests a general framework for characterizing the basic properties of various low-dimensional models used in signal processing. Below, we illustrate this framework using two examples: matrix and three-way tensor completion problems, and consider basic properties include identifiability of a matrix or tensor, given partial observations. In this note, we consider cases without observation noise to illustrate the principle.

math.ST↗

arXiv2020

Goodness-of-fit tests on manifolds

We develop a general theory for the goodness-of-fit test to non-linear models. In particular, we assume that the observations are noisy samples of a submanifold defined by a \yao{sufficiently smooth non-linear map}. The observation noise is additive Gaussian. Our main result shows that the "residual" of the model fit, by solving a non-linear least-square problem, follows a (possibly noncentral) $χ^2$ distribution. The parameters of the $χ^2$ distribution are related to the model order and dimension of the problem. We further present a method to select the model orders sequentially. We demonstrate the broad application of the general theory in machine learning and signal processing, including determining the rank of low-rank (possibly complex-valued) matrices and tensors from noisy, partial, or indirect observations, determining the number of sources in signal demixing, and potential applications in determining the number of hidden nodes in neural networks.

stat.ML↗

arXiv2020

Duality and sensitivity analysis of multistage linear stochastic programs

We investigate the dual of a Multistage Stochastic Linear Program (MSLP) to study two questions for this class of problems. The first of these questions is the study of the optimal value of the problem as a function of the involved parameters. For this sensitivity analysis problem, we provide formulas for the derivatives of the value function with respect to the parameters and illustrate their application on an inventory problem. Since these formulas involve optimal dual solutions, we need an algorithm that computes such solutions to use them. In this context, the second question we address is the study of solution methods for the dual problem. Writing Dynamic Programming (DP) equations for the dual, we can use an SDDP type method, called Dual SDDP, which solves these DP equations. However, applying this method will only be possible if the Relatively Complete Recourse (RCR) holds for the dual. Since the RCR assumption may fail to hold, we design two variants of Dual SDDP, namely Dual SDDP with penalizations and Dual SDDP with feasibility cuts. The study of this second question allows us to take a fresh look at the notoriously difficult to solve class of MSLP with interstage dependent cost coefficients. Indeed, for this class of problems, cost-to-go functions are non-convex and solution methods were so far using SDDP for a Markov chain approximation of the cost coefficients process. For these problems, we propose to apply Dual SDDP with penalizations to the cost-to-go functions of the dual which are concave. This algorithm converges to the optimal value of the problem. Finally, as a proof of concept of the tools developed, we present the results of numerical experiments computing the sensitivity of the optimal value of an inventory problem as a function of parameters of the demand process and compare Primal and Dual SDDP on the inventory and a hydro-thermal planning problems.

math.OC↗

arXiv2020

Rank one tensor completion problem

In this paper, we consider the rank-one tensor completion problem. We address the question of existence and uniqueness of the rank-one solution. In particular we show that the global uniqueness over the field of real numbers can be verified in a polynomial time. We give examples showing that there is an essential difference between the question of global uniqueness over the fields of real and complex numbers. Finally we briefly discuss the rank-one approximation problem for noisy observations.

math.NA↗

arXiv2020

Risk Neutral Reformulation Approach to Risk Averse Stochastic Programming

The aim of this paper is to show that in some cases risk averse multistage stochastic programming problems can be reformulated in a form of risk neutral setting. This is achieved by a change of the reference probability measure making ``bad" (extreme) scenarios more frequent. As a numerical example we demonstrate advantages of such change-of-measure approach applied to the Brazilian Interconnected Power System operation planning problem.

math.OC↗

arXiv2019

Risk averse stochastic programming: time consistency and optimal stopping

Bellman formulated a vague principle for optimization over time, which characterizes optimal policies by stating that a decision maker should not regret previous decisions retrospectively. This paper addresses time consistency in stochastic optimization. The problem is stated in generality first. The paper discusses time consistent decision-making by addressing risk measures which are recursive, nested, dynamically or time consistent and introduces stopping time risk measures. It turns out that the paradigm of time consistency is in conflict with various desirable, classical properties of general risk measures.

math.OC↗

arXiv2019

A cluster realization of $U_q(\mathfrak{sl_n})$ from quantum character varieties

We construct an injective algebra homomorphism of the quantum group $U_q(\mathfrak{sl}_{n+1})$ into a quantum cluster algebra $\mathbf{L}_n$ associated to the moduli space of framed $PGL_{n+1}$-local systems on a marked punctured disk. We obtain a description of the coproduct of $U_q(\mathfrak{sl}_{n+1})$ in terms of the corresponding quantum cluster algebra associated to the marked twice punctured disk, and express the action of the $R$-matrix in terms of a mapping class group element corresponding to the half-Dehn twist rotating one puncture about the other. As a consequence, we realize the algebra automorphism of $U_q(\mathfrak{sl}_{n+1})^{\otimes 2}$ given by conjugation by the $R$-matrix as an explicit sequence of cluster mutations, and derive a refined factorization of the $R$-matrix into quantum dilogarithms of cluster monomials.

math.QA↗