arXiv subjects

Liping Tang

Publications and source records attributed to Liping Tang.

27 records · Page 2Linked to original sources

arXiv2023

Learning to optimize by multi-gradient for multi-objective optimization

The development of artificial intelligence (AI) for science has led to the emergence of learning-based research paradigms, necessitating a compelling reevaluation of the design of multi-objective optimization (MOO) methods. The new generation MOO methods should be rooted in automated learning rather than manual design. In this paper, we introduce a new automatic learning paradigm for optimizing MOO problems, and propose a multi-gradient learning to optimize (ML2O) method, which automatically learns a generator (or mappings) from multiple gradients to update directions. As a learning-based method, ML2O acquires knowledge of local landscapes by leveraging information from the current step and incorporates global experience extracted from historical iteration trajectory data. By introducing a new guarding mechanism, we propose a guarded multi-gradient learning to optimize (GML2O) method, and prove that the iterative sequence generated by GML2O converges to a Pareto critical point. The experimental results demonstrate that our learned optimizer outperforms hand-designed competitors on training multi-task learning (MTL) neural network.

cs.LG↗

arXiv2023

Barzilai-Borwein Descent Methods for Multiobjective Optimization Problems with Variable Trade-off Metrics

The imbalances and conditioning of the objective functions influence the performance of first-order methods for multiobjective optimization problems (MOPs). The latter is related to the metric selected in the direction-finding subproblems. Unlike single-objective optimization problems, capturing the curvature of all objective functions with a single Hessian matrix is impossible. On the other hand, second-order methods for MOPs use different metrics for objectives in direction-finding subproblems, leading to a high per-iteration cost. To balance per-iteration cost and better curvature exploration, we propose a Barzilai-Borwein descent method with variable metrics (BBDMO\_VM). In the direction-finding subproblems, we employ a variable metric to explore the curvature of all objectives. Subsequently, Barzilai-Borwein's method relative to the variable metric is applied to tune objectives, which mitigates the effect of imbalances. We investigate the convergence behaviour of the BBDMO\_VM, confirming fast linear convergence for well-conditioned problems relative to the variable metric. In particular, we establish linear convergence for problems that involve some linear objectives. These convergence results emphasize the importance of metric selection, motivating us to approximate the trade-off of Hessian matrices to better capture the geometry of the problem. Comparative numerical results confirm the efficiency of the proposed method, even when applied to large-scale and ill-conditioned problems.

math.OC↗

arXiv2023

On the Convergence of Newton-type Proximal Gradient Method for Multiobjective Optimization Problems

In a recent study, Ansary (Optim Methods Softw 38(3):570-590,2023) proposed a Newton-type proximal gradient method for nonlinear multiobjective optimization problems (NPGMO). However, the favorable convergence properties typically associated with Newton-type methods were not established for NPGMO in Ansary's work. In response to this gap, we develop a straightforward framework for analyzing the convergence behavior of the NPGMO. Specifically, under the assumption of strong convexity, we demonstrate that the NPGMO enjoys quadratic termination, superlinear convergence, and quadratic convergence for problems that are quadratic, twice continuously differentiable and twice Lipschitz continuously differentiable, respectively.

math.OC↗

arXiv2023

Barzilai-Borwein Proximal Gradient Methods for Multiobjective Composite Optimization Problems with Improved Linear Convergence

When minimizing a multiobjective optimization problem (MOP) using multiobjective gradient descent methods, the imbalances among objective functions often decelerate the convergence. In response to this challenge, we propose two types of the Barzilai-Borwein proximal gradient method for multi-objective composite optimization problems (BBPGMO). We establish convergence rates for BBPGMO, demonstrating that it achieves rates of $O(\frac{1}{\sqrt{k}})$, $O(\frac{1}{k})$, and $O(r^{k})(0<r<1)$ for non-convex, convex, and strongly convex problems, respectively. Furthermore, we show that BBPGMO exhibits linear convergence for MOPs with several linear objective functions. Interestingly, the linear convergence rate of BBPGMO surpasses the existing convergence rates of first-order methods for MOPs, which indicates its enhanced performance and its ability to effectively address imbalances from theoretical perspective. Finally, we provide numerical examples to illustrate the efficiency of the proposed method and verify the theoretical results.

math.OC↗

arXiv2022

Robust Unsupervised Cross-Lingual Word Embedding using Domain Flow Interpolation

This paper investigates an unsupervised approach towards deriving a universal, cross-lingual word embedding space, where words with similar semantics from different languages are close to one another. Previous adversarial approaches have shown promising results in inducing cross-lingual word embedding without parallel data. However, the training stage shows instability for distant language pairs. Instead of mapping the source language space directly to the target language space, we propose to make use of a sequence of intermediate spaces for smooth bridging. Each intermediate space may be conceived as a pseudo-language space and is introduced via simple linear interpolation. This approach is modeled after domain flow in computer vision, but with a modified objective function. Experiments on intrinsic Bilingual Dictionary Induction tasks show that the proposed approach can improve the robustness of adversarial models with comparable and even better precision. Further experiments on the downstream task of Cross-Lingual Natural Language Inference show that the proposed model achieves significant performance improvement for distant language pairs in downstream tasks compared to state-of-the-art adversarial and non-adversarial models.

cs.CL↗

arXiv2022

A Modification Piecewise Convexification Method for Box-Constrained Non-Convex Optimization Programs

This paper presents a piecewise convexification method to approximate the whole approximate optimal solution set of non-convex optimization problems with box constraints. In the process of box division, we first classify the sub-boxes and only continue to divide only some sub-boxes in the subsequent division. At the same time, applying the $α$-based Branch-and-Bound ({\rm$α$BB}) method, we construct a series of piecewise convex relax sub-problems, which are collectively called the piecewise convexification problem of the original problem. Then, we define the (approximate) solution set of the piecewise convexification problem based on the classification result of sub-boxes. Subsequently, we derive that these sets can be used to approximate the global solution set with a predefined quality. Finally, a piecewise convexification algorithm with a new selection rule of sub-box for the division and two new termination tests is proposed. Several instances verify that these techniques are beneficial to improve the performance of the algorithm.

math.OC↗

arXiv2022

Convergence rates analysis of Interior Bregman Gradient Method for Vector Optimization Problems

In recent years, by using Bregman distance, the Lipschitz gradient continuity and strong convexity were lifted and replaced by relative smoothness and relative strong convexity. Under the mild assumptions, it was proved that gradient methods with Bregman regularity converge linearly for single-objective optimization problems (SOPs). In this paper, we extend the relative smoothness and relative strong convexity to vector-valued functions and analyze the convergence of an interior Bregman gradient method for vector optimization problems (VOPs). Specifically, the global convergence rates are $\mathcal{O}(\frac{1}{k})$ and $\mathcal{O}(r^{k})(0<r<1)$ for convex and relative strongly convex VOPs, respectively. Moreover, the proposed method converges linearly for VOPs that satisfy a vector Bregman-PL inequality.

math.OC↗

arXiv2022

A Barzilai-Borwein Descent Method for Multiobjective Optimization Problems

The steepest descent method proposed by Fliege et al. motivates the research on descent methods for multiobjective optimization, which has received increasing attention in recent years. However, empirical results show that the Armijo line search often gives a very small stepsize along the steepest direction, which decelerates the convergence seriously. This paper points out that the issue is mainly due to the imbalances among objective functions. To address this issue, we propose a Barzilai-Borwein descent method for multiobjective optimization (BBDMO) that dynamically tunes gradient magnitudes using Barzilai-Borwein's rule in direction-finding subproblem. With monotone and nonmonotone line search techniques, it is proved that accumulation points generated by BBDMO are Pareto critical points, respectively. Furthermore, theoretical results indicate the Armijo line search can achieve a better stepsize in BBDMO. Finally, comparative results of numerical experiments are reported to illustrate the efficiency of BBDMO and verify the theoretical results.

math.OC↗

arXiv2015

Centimeter-deep tissue fluorescence microscopic imaging with high signal-to-noise ratio and picomole sensitivity

Fluorescence microscopic imaging in centimeter-deep tissue has been highly sought-after for many years because much interesting in vivo micro-information, such as microcirculation, tumor angiogenesis, and metastasis, may deeply locate in tissue. In this study, for the first time this goal has been achieved in 3-centimeter deep tissue with high signal-to-noise ratio (SNR) and picomole sensitivity under radiation safety thresholds. These results are demonstrated not only in tissue-mimic phantoms but also in actual tissues, such as porcine muscle, ex vivo mouse liver, ex vivo spleen, and in vivo mouse tissue. These results are achieved based on three unique technologies: excellent near infrared ultrasound-switchable fluorescence (USF) contrast agents, a sensitive USF imaging system, and an effective correlation method. Multiplex USF fluorescence imaging is also achieved. It is useful to simultaneously image multiple targets and observe their interactions. This work opens the door for future studies of centimeter-deep tissue fluorescence microscopic imaging.

physics.optics↗