arXiv subjects

Peng Zheng

Publications and source records attributed to Peng Zheng.

47 records · Page 3Linked to original sources

arXiv2019

An inexact augmented Lagrangian method for nonsmooth optimization on Riemannian manifold

We consider a nonsmooth optimization problem on Riemannian manifold, whose objective function is the sum of a differentiable component and a nonsmooth convex function. We propose a manifold inexact augmented Lagrangian method (MIALM) for the considered problem. The problem is reformulated to a separable form. By utilizing the Moreau envelope, we get a smoothing subproblem at each iteration of the proposed method. Theoretically, under suitable assumptions, the convergence to critical point of the proposed method is established. In particular, under the condition of that the approximate global minimizer of the iteration subproblem could be obtained, we prove the convergence to global minimizer of the origin problem. Numerical experiments show that, the MIALM is a competitive method compared to some existing methods.

math.OC↗

arXiv2019

Efficient Robust Parameter Identification in Generalized Kalman Smoothing Models

Dynamic inference problems in autoregressive (AR/ARMA/ARIMA), exponential smoothing, and navigation are often formulated and solved using state-space models (SSM), which allow a range of statistical distributions to inform innovations and errors. In many applications the main goal is to identify not only the hidden state, but also additional unknown model parameters (e.g. AR coefficients or unknown dynamics). We show how to efficiently optimize over model parameters in SSM that use smooth process and measurement losses. Our approach is to project out state variables, obtaining a value function that only depends on the parameters of interest, and derive analytical formulas for first and second derivatives that can be used by many types of optimization methods. The approach can be used with smooth robust penalties such as Hybrid and the Student's t, in addition to classic least squares. We use the approach to estimate robust AR models and long-run unemployment rates with sudden changes.

math.OC↗

arXiv2019

Learning Nonlinear Brain Dynamics: van der Pol Meets LSTM

Many real-world data sets, especially in biology, are produced by complex nonlinear dynamical systems. In this paper, we focus on brain calcium imaging (CaI) of different organisms (zebrafish and rat), aiming to build a model of joint activation dynamics in large neuronal populations, including the whole brain of zebrafish. We propose a new approach for capturing dynamics of temporal SVD components that uses the coupled (multivariate) van der Pol (VDP) oscillator, a nonlinear ordinary differential equation (ODE) model describing neural activity, with a new parameter estimation technique that combines variable projection optimization and stochastic search. We show that the approach successfully handles nonlinearities and hidden state variables in the coupled VDP. The approach is accurate, achieving 0.82 to 0.94 correlation between the actual and model-generated components, and interpretable, as VDP's coupling matrix reveals anatomically meaningful positive (excitatory) and negative (inhibitory) interactions across different brain subsystems corresponding to spatial SVD components. Moreover, VDP is comparable to (or sometimes better than) recurrent neural networks (LSTM) for (short-term) prediction of future brain activity; VDP needs less parameters to train, which was a plus on our small training data. Finally, the overall best predictive method, greatly outperforming both VDP and LSTM in short- and long-term predictive settings on both datasets, was the new hybrid VDP-LSTM approach that used VDP to simulate large domain-specific dataset for LSTM pretraining; note that simple LSTM data-augmentation via noisy versions of training data was much less effective.

stat.ML↗

arXiv2019

M-estimation with the Trimmed l1 Penalty

We study high-dimensional estimators with the trimmed $\ell_1$ penalty, which leaves the $h$ largest parameter entries penalty-free. While optimization techniques for this nonconvex penalty have been studied, the statistical properties have not yet been analyzed. We present the first statistical analyses for $M$-estimation and characterize support recovery, $\ell_\infty$ and $\ell_2$ error of the trimmed $\ell_1$ estimates as a function of the trimming parameter $h$. Our results show different regimes based on how $h$ compares to the true support size. Our second contribution is a new algorithm for the trimmed regularization problem, which has the same theoretical convergence rate as the difference of convex (DC) algorithms, but in practice is faster and finds lower objective values. Empirical evaluation of $\ell_1$ trimming for sparse linear regression and graphical model estimation indicate that trimmed $\ell_1$ can outperform vanilla $\ell_1$ and non-convex alternatives. Our last contribution is to show that the trimmed penalty is beneficial beyond $M$-estimation, and yields promising results for two deep learning tasks: input structures recovery and network sparsification.

math.ST↗

arXiv2019

Object Detection with Deep Learning: A Review

Due to object detection's close relationship with video analysis and image understanding, it has attracted much research attention in recent years. Traditional object detection methods are built on handcrafted features and shallow trainable architectures. Their performance easily stagnates by constructing complex ensembles which combine multiple low-level image features with high-level context from object detectors and scene classifiers. With the rapid development in deep learning, more powerful tools, which are able to learn semantic, high-level, deeper features, are introduced to address the problems existing in traditional architectures. These models behave differently in network architecture, training strategy and optimization function, etc. In this paper, we provide a review on deep learning based object detection frameworks. Our review begins with a brief introduction on the history of deep learning and its representative tool, namely Convolutional Neural Network (CNN). Then we focus on typical generic object detection architectures along with some modifications and useful tricks to improve detection performance further. As distinct specific detection tasks exhibit different characteristics, we also briefly survey several specific tasks, including salient object detection, face detection and pedestrian detection. Experimental analyses are also provided to compare various methods and draw some meaningful conclusions. Finally, several promising directions and tasks are provided to serve as guidelines for future work in both object detection and relevant neural network based learning systems.

cs.CV↗

arXiv2018

A Unified Framework for Sparse Relaxed Regularized Regression: SR3

Regularized regression problems are ubiquitous in statistical modeling, signal processing, and machine learning. Sparse regression in particular has been instrumental in scientific model discovery, including compressed sensing applications, variable selection, and high-dimensional analysis. We propose a broad framework for sparse relaxed regularized regression, called SR3. The key idea is to solve a relaxation of the regularized problem, which has three advantages over the state-of-the-art: (1) solutions of the relaxed problem are superior with respect to errors, false positives, and conditioning, (2) relaxation allows extremely fast algorithms for both convex and nonconvex formulations, and (3) the methods apply to composite regularizers such as total variation (TV) and its nonconvex variants. We demonstrate the advantages of SR3 (computational efficiency, higher accuracy, faster convergence rates, greater flexibility) across a range of regularized regression problems with synthetic and real data, including applications in compressed sensing, LASSO, matrix completion, TV regularization, and group sparsity. To promote reproducible research, we also provide a companion MATLAB package that implements these examples.

stat.ML↗

arXiv2018

Relax-and-split method for nonsmooth nonconvex problems

We develop and analyze a new `relax-and-split' (RS) approach for compositions of separable nonconvex nonsmooth functions with linear maps. RS uses a relaxation technique together with partial minimization, and brings classic techniques including direct factorization, matrix decompositions, and fast iterative methods to bear on nonsmooth nonconvex problems. We also extend the approach to trimmed nonconvex-composite formulations; the resulting Trimmed RS (TRS) can fit models while detecting outliers in the data. We then test RS and TRS on a diverse set of applications: (1) phase retrieval, (2) stochastic shortest path problems, (3) semi-supervised classification, and (4) new clustering approaches. RS/TRS can be applied to models with very weak functional assumptions, are easy to implement, competitive with existing methods, and enable a new level of modeling formulations to be put forward to address emerging challenges in the mathematical sciences.

math.OC↗

arXiv2018

Computer Assisted Localization of a Heart Arrhythmia

We consider the problem of locating a point-source heart arrhythmia using data from a standard diagnostic procedure, where a reference catheter is placed in the heart, and arrival times from a second diagnostic catheter are recorded as the diagnostic catheter moves around within the heart. We model this situation as a nonconvex feasibility problem, where given a set of arrival times, we look for a source location that is consistent with the available data. We develop a new optimization approach and fast algorithm to obtain online proposals for the next location to suggest to the operator as she collects data. We validate the procedure using a Monte Carlo simulation based on patients' electrophysiological data. The proposed procedure robustly and quickly locates the source of arrhythmias without any prior knowledge of heart anatomy.

q-bio.TO↗

arXiv2017

Robust and scalable methods for the dynamic mode decomposition

The dynamic mode decomposition (DMD) is a broadly applicable dimensionality reduction algorithm that approximates a matrix containing time-series data by the outer product of a matrix of exponentials, representing Fourier-like time dynamics, and a matrix of coefficients, representing spatial structures. This interpretable spatio-temporal decomposition is commonly computed using linear algebraic techniques in its simplest formulation or a nonlinear optimization procedure within the variable projection framework. For data with sparse outliers or data which are not well-represented by exponentials in time, the standard Frobenius norm fit of the data creates significant biases in the recovered time dynamics. As a result, practitioners are left to clean such defects from the data manually or to use a black-box cleaning approach like robust PCA. As an alternative, we propose a framework and a set of algorithms for incorporating robust features into the nonlinear optimization used to compute the DMD itself. The algorithms presented are flexible, allowing for regu- larizers and constraints on the optimization, and scalable, using a stochastic approach to decrease the computational cost for data in high dimensional space. Both synthetic and real data examples are provided.

math.OC↗

arXiv2017

Learning Robust Representations for Computer Vision

Unsupervised learning techniques in computer vision often require learning latent representations, such as low-dimensional linear and non-linear subspaces. Noise and outliers in the data can frustrate these approaches by obscuring the latent spaces. Our main goal is deeper understanding and new development of robust approaches for representation learning. We provide a new interpretation for existing robust approaches and present two specific contributions: a new robust PCA approach, which can separate foreground features from dynamic background, and a novel robust spectral clustering method, that can cluster facial images with high accuracy. Both contributions show superior performance to standard methods on real-world test sets.

stat.ML↗

arXiv2012

Determining Seebeck coefficient of heavily doped La:SrTiO3 from density functional calculations

A new approach is developed to calculate temperature dependent Seebeck coefficient of heavily doped systems by using Boltzmann transport theory and electron density of states (DOS) obtained from density functional calculations. This approach is applied to heavily doped La:STO with DOS from tetrahedral method and Fermi energy using Fermi integrals. The calculated Seebeck coefficient agrees with the experimental data nearly quantitatively, which proved the accuracy of this approach. The influence of the Fermi energy and asymmetry of DOS on the Seebeck coefficient is analyzed.

cond-mat.mtrl-sci↗