arXiv subjects

Hongchao Zhang

Publications and source records attributed to Hongchao Zhang.

53 records · Page 3Linked to original sources

arXiv2020

On the acceleration of the Barzilai-Borwein method

The Barzilai-Borwein (BB) gradient method is efficient for solving large-scale unconstrained problems to the modest accuracy and has a great advantage of being easily extended to solve a wide class of constrained optimization problems. In this paper, we propose a new stepsize to accelerate the BB method by requiring finite termination for minimizing two-dimensional strongly convex quadratic function. Combing with this new stepsize, we develop gradient methods which adaptively take the nonmonotone BB stepsizes and certain monotone stepsizes for minimizing general strongly convex quadratic function. Furthermore, by incorporating nonmonotone line searches and gradient projection techniques, we extend these new gradient methods to solve general smooth unconstrained and bound constrained optimization. Extensive numerical experiments show that our strategies of properly inserting monotone gradient steps into the nonmonotone BB method could significantly improve its performance and the new resulted methods can outperform the most successful gradient decent methods developed in the recent literature.

math.OC↗

arXiv2019

On the asymptotic convergence and acceleration of gradient methods

We consider the asymptotic behavior of a family of gradient methods, which include the steepest descent and minimal gradient methods as special instances. It is proved that each method in the family will asymptotically zigzag between two directions. Asymptotic convergence results of the objective value, gradient norm, and stepsize are presented as well. To accelerate the family of gradient methods, we further exploit spectral properties of stepsizes to break the zigzagging pattern. In particular, a new stepsize is derived by imposing finite termination on minimizing two-dimensional strictly convex quadratic function. It is shown that, for the general quadratic function, the proposed stepsize asymptotically converges to the reciprocal of the largest eigenvalue of the Hessian. Furthermore, based on this spectral property, we propose a periodic gradient method by incorporating the Barzilai-Borwein method. Numerical comparisons with some recent successful gradient methods show that our new method is very promising.

math.OC↗

arXiv2019

Late-time acceleration and inflation in a Poincaré gauge cosmological model

A self-consistent model which can unify Starobinsky inflation and the $Λ$CDM model in the framework of Poincaré gauge cosmology (PGC) is studied in this work, without extra inflaton and ``dark energy''. We start from the general nine-parameter PGC Lagrangian and get two Friedmann-like analytical solutions with the certain ghost- and tachyon-free conditions on the Lagrangian parameters. The scalar torsion $h$-determined solution is consistent with Starobinsky cosmology in the early-time and the pseudo-scalar torsion $f$-determined solution contains naturally a constant ``dark energy'' density, $3(α-1)^2/8B_0$, which covers the $Λ$CDM model in the late-time. According to the latest observations, we estimate the magnitudes of parameters: $α-1\simeq 8.07\times10^{-56}$, $B_0\simeq5.76\times10^{-28}GeV^{-2}$ in the natural units.

gr-qc↗

arXiv2019

Inflation in the general Poincaré gauge cosmology

The general Poincaré gauge cosmology given by a nine-parameter gravitational Lagrangian with ghost- and tachyon-free conditions is studied from the perspective of field theory. By introducing new variables for replacing two (pseudo-) scalar torsions, the Poincaré gauge cosmological system can be recast into a gravitational system coupled to two-scalar fields with a potential up to quartic-order. We discussed the possibility of this system producing two types of inflation without any extra inflatons. The hybrid inflation with a first-order phase transition can be ruled out, while the slow rollover can be achieved. The numerical analysis shows that the two-scalar fields system evolved in a potential well processes spontaneously four stages: "pre-inflation", slow-roll inflation with large enough e-folds, "pre-reheating" and reheating. We also studied the stableness of this system by setting large values of initial kinetic energies. The results show that even if the system evolves past the highest point of the potential well, the scalar fields can still return to the potential well and cause inflation. The general Poincaré gauge cosmology provides us with a self-consistent candidate of inflation.

gr-qc↗

arXiv2019

Gradient methods exploiting spectral properties

We propose a new stepsize for the gradient method. It is shown that this new stepsize will converge to the reciprocal of the largest eigenvalue of the Hessian, when Dai-Yang's asymptotic optimal gradient method (Computational Optimization and Applications, 2006, 33(1): 73-88) is applied for minimizing quadratic objective functions. Based on this spectral property, we develop a monotone gradient method that takes a certain number of steps using the asymptotically optimal stepsize by Dai and Yang, and then follows by some short steps associated with this new stepsize. By employing one step retard of the asymptotic optimal stepsize, a nonmonotone variant of this method is also proposed. Under mild conditions, $R$-linear convergence of the proposed methods is established for minimizing quadratic functions. In addition, by combining gradient projection techniques and adaptive nonmonotone line search, we further extend those methods for general bound constrained optimization. Two variants of gradient projection methods combining with the Barzilai-Borwein stepsizes are also proposed. Our numerical experiments on both quadratic and bound constrained optimization indicate that the new proposed strategies and methods are very effective.

math.OC↗

arXiv2018

Generalized Symmetric ADMM for Separable Convex Optimization

The Alternating Direction Method of Multipliers (ADMM) has been proved to be effective for solving separable convex optimization subject to linear constraints. In this paper, we propose a Generalized Symmetric ADMM (GS-ADMM), which updates the Lagrange multiplier twice with suitable stepsizes, to solve the multi-block separable convex programming. This GS-ADMM partitions the data into two group variables so that one group consists of $p$ block variables while the other has $q$ block variables, where $p \ge 1$ and $q \ge 1$ are two integers. The two grouped variables are updated in a {\it Gauss-Seidel} scheme, while the variables within each group are updated in a {\it Jacobi} scheme, which would make it very attractive for a big data setting. By adding proper proximal terms to the subproblems, we specify the domain of the stepsizes to guarantee that GS-ADMM is globally convergent with a worst-case $O(1/t)$ ergodic convergence rate. It turns out that our convergence domain of the stepsizes is significantly larger than other convergence domains in the literature. Hence, the GS-ADMM is more flexible and attractive on choosing and using larger stepsizes of the dual variable. Besides, two special cases of GS-ADMM, which allows using zero penalty terms, are also discussed and analyzed. Compared with several state-of-the-art methods, preliminary numerical experiments on solving a sparse matrix minimization problem in the statistical learning show that our proposed method is effective and promising.

math.OC↗

arXiv2018

A parameterized proximal point algorithm for separable convex optimization

In this paper, we develop a parameterized proximal point algorithm (P-PPA) for solving a class of separable convex programming problems subject to linear and convex constraints. The proposed algorithm is provable to be globally convergent with a worst-case O(1/t) convergence rate, wheret denotes the iteration number. By properly choosing the algorithm parameters, numerical experiments on solving a sparse optimization problem arising from statistical learning show that our P-PPA could perform significantly better than other state-of-the-art methods, such as the alternating direction method of multipliers and the relaxed proximal point algorithm.

math.OC↗

arXiv2018

Probing the Neutrino Mass Hierarchy beyond $Λ$CDM Model

Taking the neutrino oscillation data into consideration, a dimensionless parameter $Δ= (m_3-m_1)/(m_3+m_1)$ is adopted to parameterize the three neutrino mass eigenstates and the normal (positive $Δ$) or inverted (negative $Δ$) mass hierarchies in three typical cosmological models. Using the currently available cosmic observational data, several Markov Chain Monte Carlo chains are obtained with uniform priors on the free parameters at first. Applying importance sampling the results are compared with three new priors, i.e., logarithmic prior on $|Δ|$, linear and logarithmic priors on $Σm_ν$. It turns out that the three new priors increase the upper limits of neutrino mass, but do not change the tendency towards different model's preference for different hierarchies, i.e., the normal hierarchy tends to be favored by $Λ$CDM and $w$CDM, which, however, disappears in the $w_0 w_a$CDM model. In addition, the almost symmetrical contours in the $w-Δ$, $w_0-Δ$, $w_a-Δ$ planes indicate that the normal and inverted hierarchy have strong degeneracy. Finally, we perform a Bayesian model comparison analysis, finding that flat linear prior on $Δ$ and $w_0 w_a$CDM are the most preferred prior and model, respectively.

astro-ph.CO↗

arXiv2017

Poincaré Gauge Gravity Cosmology

In this work, we construct the logical framework of the Poincaré gauge gravity cosmology based on five postulations, and introduce the modified redshift relation within this framework. Then we solve a system with quadratic action and some other assumptions to get an analytic solution on background level. The evolution of the Universe on background can be reproduced from this solution without hypothesizing dark energy. Further, we use the type Ia supernova data set JLA to test the effect of the modified redshift relation under the constraints of system parameters. The results show that the constraint on some parameters are compact.

gr-qc↗

arXiv2016

Distinguish three time-dependent dark energy models using statefinder pairs with error bars and Bayesian evidence

In this work, two completely different approaches, statefinder with error bars and Bayesian evidence, are used to distinguish and judge three time-dependent dark energy models. The parameters constrain for the three dark energy models are given using the current cosmic observational data sets : $Planck$ 2015, SNIa, BAO and OHD. Using the statefinder pairs with error bars, we find that the error region of the dark energy model, whose equation of state parameter is given by $w(a) = w_0+w_3\frac{1-a}{a^2+(1-a)^2}$, is relativity compact than the other two models during the all the evolving history. Meanwhile, the Bayesian evidence also provide that this model is significantly better than the other two models and the other two models are inconclusive. Then, there are reasons for believing that this model is a preferential candidate in dark energy investigation rather than the other two.

gr-qc↗

arXiv2016

An Active Set Algorithm for Nonlinear Optimization with Polyhedral Constraints

A polyhedral active set algorithm PASA is developed for solving a nonlinear optimization problem whose feasible set is a polyhedron. Phase one of the algorithm is the gradient projection method, while phase two is any algorithm for solving a linearly constrained optimization problem. Rules are provided for branching between the two phases. Global convergence to a stationary point is established, while asymptotically PASA performs only phase two when either a nondegeneracy assumption holds, or the active constraints are linearly independent and a strong second-order sufficient optimality condition holds.

math.OC↗

arXiv2016

Inexact alternating direction multiplier methods for separable convex optimization

Inexact alternating direction multiplier methods (ADMMs) are developed for solving general separable convex optimization problems with a linear constraint and with an objective that is the sum of smooth and nonsmooth terms. The approach involves linearized subproblems, a back substitution step, and either gradient or accelerated gradient techniques. Global convergence is established. The methods are particularly useful when the ADMM subproblems do not have closed form solution or when the solution of the subproblems is expensive. Numerical experiments based on image reconstruction problems show the effectiveness of the proposed methods.

math.OC↗

arXiv2015

Generalized Uniformly Optimal Methods for Nonlinear Programming

In this paper, we present a generic framework to extend existing uniformly optimal convex programming algorithms to solve more general nonlinear, possibly nonconvex, optimization problems. The basic idea is to incorporate a local search step (gradient descent or Quasi-Newton iteration) into these uniformly optimal convex programming methods, and then enforce a monotone decreasing property of the function values computed along the trajectory. Algorithms of these types will then achieve the best known complexity for nonconvex problems, and the optimal complexity for convex ones without requiring any problem parameters. As a consequence, we can have a unified treatment for a general class of nonlinear programming problems regardless of their convexity and smoothness level. In particular, we show that the accelerated gradient and level methods, both originally designed for solving convex optimization problems only, can be used for solving both convex and nonconvex problems uniformly. In a similar vein, we show that some well-studied techniques for nonlinear programming, e.g., Quasi-Newton iteration, can be embedded into optimal convex optimization algorithms to possibly further enhance their numerical performance. Our theoretical and algorithmic developments are complemented by some promising numerical results obtained for solving a few important nonconvex and nonlinear data analysis problems in the literature.

math.OC↗

arXiv2013

Mini-batch Stochastic Approximation Methods for Nonconvex Stochastic Composite Optimization

This paper considers a class of constrained stochastic composite optimization problems whose objective function is given by the summation of a differentiable (possibly nonconvex) component, together with a certain non-differentiable (but convex) component. In order to solve these problems, we propose a randomized stochastic projected gradient (RSPG) algorithm, in which proper mini-batch of samples are taken at each iteration depending on the total budget of stochastic samples allowed. The RSPG algorithm also employs a general distance function to allow taking advantage of the geometry of the feasible region. Complexity of this algorithm is established in a unified setting, which shows nearly optimal complexity of the algorithm for convex stochastic programming. A post-optimization phase is also proposed to significantly reduce the variance of the solutions returned by the algorithm. In addition, based on the RSPG algorithm, a stochastic gradient free algorithm, which only uses the stochastic zeroth-order information, has been also discussed. Some preliminary numerical results are also provided.

math.OC↗

arXiv2010

A nonmonotone spectral projected gradient method for large-scale topology optimization problems

An efficient gradient-based method to solve the volume constrained topology optimization problems is presented. Each iterate of this algorithm is obtained by the projection of a Barzilai-Borwein step onto the feasible set consisting of box and one linear constraints (volume constraint). To ensure the global convergence, an adaptive nonmonotone line search is performed along the direction that is given by the current and projection point. The adaptive cyclic reuse of the Barzilai-Borwein step is applied as the initial stepsize. The minimum memory requirement, the guaranteed convergence property, and almost only one function and gradient evaluations per iteration make this new method very attractive within common alternative methods to solve large-scale optimal design problems. Efficiency and feasibility of the presented method are supported by numerical experiments.

math.OC↗

arXiv2009

Gradient-based methods for sparse recovery

The convergence rate is analyzed for the SpaSRA algorithm (Sparse Reconstruction by Separable Approximation) for minimizing a sum $f (\m{x}) + ψ(\m{x})$ where $f$ is smooth and $ψ$ is convex, but possibly nonsmooth. It is shown that if $f$ is convex, then the error in the objective function at iteration $k$, for $k$ sufficiently large, is bounded by $a/(b+k)$ for suitable choices of $a$ and $b$. Moreover, if the objective function is strongly convex, then the convergence is $R$-linear. An improved version of the algorithm based on a cycle version of the BB iteration and an adaptive line search is given. The performance of the algorithm is investigated using applications in the areas of signal processing and image reconstruction.

math.OC↗

arXiv2009

An exact algorithm for graph partitioning

An exact algorithm is presented for solving edge weighted graph partitioning problems. The algorithm is based on a branch and bound method applied to a continuous quadratic programming formulation of the problem. Lower bounds are obtained by decomposing the objective function into convex and concave parts and replacing the concave part by an affine underestimate. It is shown that the best affine underestimate can be expressed in terms of the center and the radius of the smallest sphere containing the feasible set. The concave term is obtained either by a constant diagonal shift associated with the smallest eigenvalue of the objective function Hessian, or by a diagonal shift obtained by solving a semidefinite programming problem. Numerical results show that the proposed algorithm is competitive with state-of-the-art graph partitioning codes.

math.OC↗