Aleksandrina Goeva | Searcharxiv

arXiv2019

Loss Landscapes of Regularized Linear Autoencoders

Autoencoders are a deep learning model for representation learning. When trained to minimize the distance between the data and its reconstruction, linear autoencoders (LAEs) learn the subspace spanned by the top principal directions but cannot learn the principal directions themselves. In this paper, we prove that $L_2$-regularized LAEs are symmetric at all critical points and learn the principal directions as the left singular vectors of the decoder. We smoothly parameterize the critical manifold and relate the minima to the MAP estimate of probabilistic PCA. We illustrate these results empirically and consider implications for PCA algorithms, computational neuroscience, and the algebraic topology of learning.

cs.LG↗

arXiv2018

Optimization-based Calibration of Simulation Input Models

Studies on simulation input uncertainty often built on the availability of input data. In this paper, we investigate an inverse problem where, given only the availability of output data, we nonparametrically calibrate the input models and other related performance measures of interest. We propose an optimization-based framework to compute statistically valid bounds on input quantities. The framework utilizes constraints that connect the statistical information of the real-world outputs with the input-output relation via a simulable map. We analyze the statistical guarantees of this approach from the view of data-driven robust optimization, and show how the guarantees relate to the function complexity of the constraints arising in our framework. We investigate an iterative procedure based on a stochastic quadratic penalty method to approximately solve the resulting optimization. We conduct numerical experiments to demonstrate our performance in bounding the input models and related quantities.

math.OC↗