arXiv subjects

Xudong Li

Publications and source records attributed to Xudong Li.

At least 73 records · Page 4Linked to original sources

arXiv2023

Less is More: Learning Reference Knowledge Using No-Reference Image Quality Assessment

Image Quality Assessment (IQA) with reference images have achieved great success by imitating the human vision system, in which the image quality is effectively assessed by comparing the query image with its pristine reference image. However, for the images in the wild, it is quite difficult to access accurate reference images. We argue that it is possible to learn reference knowledge under the No-Reference Image Quality Assessment (NR-IQA) setting, which is effective and efficient empirically. Concretely, by innovatively introducing a novel feature distillation method in IQA, we propose a new framework to learn comparative knowledge from non-aligned reference images. And then, to achieve fast convergence and avoid overfitting, we further propose an inductive bias regularization. Such a framework not only solves the congenital defects of NR-IQA but also improves the feature extraction framework, enabling it to express more abundant quality information. Surprisingly, our method utilizes less input while obtaining a more significant improvement compared to the teacher models. Extensive experiments on eight standard NR-IQA datasets demonstrate the superior performance to the state-of-the-art NR-IQA methods, i.e., achieving the PLCC values of 0.917 (vs. 0.884 in LIVEC) and 0.686 (vs. 0.661 in LIVEFB).

cs.CV↗

arXiv2023

Data-Driven Minimax Optimization with Expectation Constraints

Attention to data-driven optimization approaches, including the well-known stochastic gradient descent method, has grown significantly over recent decades, but data-driven constraints have rarely been studied, because of the computational challenges of projections onto the feasible set defined by these hard constraints. In this paper, we focus on the non-smooth convex-concave stochastic minimax regime and formulate the data-driven constraints as expectation constraints. The minimax expectation constrained problem subsumes a broad class of real-world applications, including two-player zero-sum game and data-driven robust optimization. We propose a class of efficient primal-dual algorithms to tackle the minimax expectation-constrained problem, and show that our algorithms converge at the optimal rate of $\mathcal{O}(\frac{1}{\sqrt{N}})$. We demonstrate the practical efficiency of our algorithms by conducting numerical experiments on large-scale real-world applications.

cs.LG↗

arXiv2023

MixPath: A Unified Approach for One-shot Neural Architecture Search

Blending multiple convolutional kernels is proved advantageous in neural architecture design. However, current two-stage neural architecture search methods are mainly limited to single-path search spaces. How to efficiently search models of multi-path structures remains a difficult problem. In this paper, we are motivated to train a one-shot multi-path supernet to accurately evaluate the candidate architectures. Specifically, we discover that in the studied search spaces, feature vectors summed from multiple paths are nearly multiples of those from a single path. Such disparity perturbs the supernet training and its ranking ability. Therefore, we propose a novel mechanism called Shadow Batch Normalization (SBN) to regularize the disparate feature statistics. Extensive experiments prove that SBNs are capable of stabilizing the optimization and improving ranking performance. We call our unified multi-path one-shot approach as MixPath, which generates a series of models that achieve state-of-the-art results on ImageNet.

cs.LG↗

arXiv2023

On Geometric Connections of Embedded and Quotient Geometries in Riemannian Fixed-rank Matrix Optimization

In this paper, we propose a general procedure for establishing the geometric landscape connections of a Riemannian optimization problem under the embedded and quotient geometries. By applying the general procedure to the fixed-rank positive semidefinite (PSD) and general matrix optimization, we establish an exact Riemannian gradient connection under two geometries at every point on the manifold and sandwich inequalities between the spectra of Riemannian Hessians at Riemannian first-order stationary points (FOSPs). These results immediately imply an equivalence on the sets of Riemannian FOSPs, Riemannian second-order stationary points (SOSPs), and strict saddles of fixed-rank matrix optimization under the embedded and the quotient geometries. To the best of our knowledge, this is the first geometric landscape connection between the embedded and the quotient geometries for fixed-rank matrix optimization and it provides a concrete example of how these two geometries are connected in Riemannian optimization. In addition, the effects of the Riemannian metric and quotient structure on the landscape connection are discussed. We also observe an algorithmic connection between two geometries with some specific Riemannian metrics in fixed-rank matrix optimization: there is an equivalence between gradient flows under two geometries with shared spectra of Riemannian Hessians. A number of novel ideas and technical ingredients including a unified treatment for different Riemannian metrics, novel metrics for the Stiefel manifold, and new horizontal space representations under quotient geometries are developed to obtain our results. The results in this paper deepen our understanding of geometric and algorithmic connections of Riemannian optimization under different Riemannian geometries and provide a few new theoretical insights to unanswered questions in the literature.

math.OC↗

arXiv2023

An active-set based recursive approach for solving convex isotonic regression with generalized order restrictions

This paper studies the convex isotonic regression with generalized order restrictions induced by a directed tree. The proposed model covers various intriguing optimization problems with shape or order restrictions, including the generalized nearly isotonic optimization and the total variation on a tree. Inspired by the success of the pool-adjacent-violator algorithm and its active-set interpretation, we propose an active-set based recursive approach for solving the underlying model. Unlike the brute-force approach that traverses an exponential number of possible active-set combinations, our algorithm has a polynomial time computational complexity under mild assumptions.

math.OC↗

arXiv2022

Recursive Importance Sketching for Rank Constrained Least Squares: Algorithms and High-order Convergence

In this paper, we propose {\it \underline{R}ecursive} {\it \underline{I}mportance} {\it \underline{S}ketching} algorithm for {\it \underline{R}ank} constrained least squares {\it \underline{O}ptimization} (RISRO). The key step of RISRO is recursive importance sketching, a new sketching framework based on deterministically designed recursive projections, which significantly differs from the randomized sketching in the literature \citep{mahoney2011randomized,woodruff2014sketching}. Several existing algorithms in the literature can be reinterpreted under this new sketching framework and RISRO offers clear advantages over them. RISRO is easy to implement and computationally efficient, where the core procedure in each iteration is to solve a dimension-reduced least squares problem. We establish the local quadratic-linear and quadratic rate of convergence for RISRO under some mild conditions. We also discover a deep connection of RISRO to the Riemannian Gauss-Newton algorithm on fixed rank matrices. The effectiveness of RISRO is demonstrated in two applications in machine learning and statistics: low-rank matrix trace regression and phase retrieval. Simulation studies demonstrate the superior numerical performance of RISRO.

math.OC↗

arXiv2022

Influence of Pre-bottleneck Diversion Devices on Pedestrian Flow

The existence of bottlenecks often leads to the stagnation of pedestrian gatherings, which seriously affects the efficiency of traffic and reduces the flow of pedestrians. Some studies have shown that setting devices in front of bottlenecks can promote pedestrian evacuation under certain conditions. In this paper, the effect of setting diversion devices in front of the exit on pedestrian flow is studied. From our observation, these diversion devices can form a buffer zone before the exit and affect pedestrian behaviors. The evacuation times are found to decrease as the devices became farther away from the exit. In our experiments, it is found that the effect of shunt piles on evacuation is better than in the case of safety barriers and without device conditions. Under the condition of setting up safety barriers approximately 1m and 3m in front of the exit, the evacuation times are extended by 0.88% and 2.67%. For shunt piles, the evacuation times are 11.53% and 14.96% shorter than that of those without a device regarding the different distances to exit (1m and 3m, respectively). In addition, setting up shunt piles reduces the time interval between two consecutive pedestrians. To sum up, in our experimental settings, the diversion devices can effectively improve the average speed ahead of the exit and promote evacuation to become more orderly, which reduces the congestion in the later period of evacuation. In other words, this study demonstrates that a reasonable layout of facilities can not only meet the daily functional requirements but also improve the efficient use of space in emergencies, reducing the probability of crowd conventions and jams.

physics.soc-ph↗

arXiv2022

A dynamic programming approach for generalized nearly isotonic optimization

Shape restricted statistical estimation problems have been extensively studied, with many important practical applications in signal processing, bioinformatics, and machine learning. In this paper, we propose and study a generalized nearly isotonic optimization (GNIO) model, which recovers, as special cases, many classic problems in shape constrained statistical regression, such as isotonic regression, nearly isotonic regression and unimodal regression problems. We develop an efficient and easy-to-implement dynamic programming algorithm for solving the proposed model whose recursion nature is carefully uncovered and exploited. For special $\ell_2$-GNIO problems, implementation details and the optimal ${\cal O}(n)$ running time analysis of our algorithm are discussed. Numerical experiments, including the comparisons among our approach, the powerful commercial solver Gurobi, and existing fast algorithms for solving $\ell_1$-GNIO and $\ell_2$-GNIO problems, on both simulated and real data sets, are presented to demonstrate the high efficiency and robustness of our proposed algorithm in solving large scale GNIO problems.

math.OC↗

arXiv2022

Physical interpretation of nonlocal quantum correlation through local description of subsystems

Characterization and categorization of quantum correlations are both fundamentally and practically important in quantum information science. Although quantum correlations such as non-separability, steerability, and non-locality can be characterized by different theoretical models in different scenarios with either known (trusted) or unknown (untrusted) knowledge of the associated systems, such characterization sometimes lacks unambiguous to experimentalist. In this work, we propose the physical interpretation of nonlocal quantum correlation between two systems. In the absence of {\it complete local description} of one of the subsystems quantified by the {\it local uncertainty relation}, the correlation between subsystems becomes nonlocal. Remarkably, different nonlocal quantum correlations can be discriminated from a single uncertainty relation derived under local hidden state (LHS)-LHS model only. We experimentally characterize the two-qubit Werner state in different scenarios.

quant-ph↗

arXiv2022

Robust Security Analysis Based on Random Geometry Theory for Satellite-Terrestrial-Vehicle Network

Driven by B5G and 6G technologies, multi-network fusion is an indispensable tendency for future communications. In this paper, we focus on and analyze the \emph{security performance} (SP) of the \emph{satellite-terrestrial downlink transmission} (STDT). Here, the STDT is composed of a satellite network and a vehicular network with a legitimate mobile receiver and an mobile eavesdropper distributing. To theoretically analyze the SP of this system from the perspective of mobile terminals better, the random geometry theory is adopted, which assumes that both terrestrial vehicles are distributed stochastically in one beam of the satellite. Furthermore, based on this theory, the closed-form analytical expressions for two crucial and specific indicators in the STDT are derived, respectively, the secrecy outage probability and the ergodic secrecy capacity. Additionally, several related variables restricting the SP of the STDT are discussed, and specific schemes are presented to enhance the SP. Then, the asymptotic property is investigated in the high signal-to-noise ratio scenario, and accurate and asymptotic closed-form expressions are given. Finally, simulation results show that, under the precondition of guaranteeing the reliability of the STDT, the asymptotic solutions outperform the corresponding accurate results significantly in the effectiveness.

cs.IT↗

arXiv2022

Solving Stackelberg Prediction Game with Least Squares Loss via Spherically Constrained Least Squares Reformulation

The Stackelberg prediction game (SPG) is popular in characterizing strategic interactions between a learner and an attacker. As an important special case, the SPG with least squares loss (SPG-LS) has recently received much research attention. Although initially formulated as a difficult bi-level optimization problem, SPG-LS admits tractable reformulations which can be polynomially globally solved by semidefinite programming or second order cone programming. However, all the available approaches are not well-suited for handling large-scale datasets, especially those with huge numbers of features. In this paper, we explore an alternative reformulation of the SPG-LS. By a novel nonlinear change of variables, we rewrite the SPG-LS as a spherically constrained least squares (SCLS) problem. Theoretically, we show that an $ε$ optimal solution to the SCLS (and the SPG-LS) can be achieved in $\tilde{O}(N/\sqrtε)$ floating-point operations, where $N$ is the number of nonzero entries in the data matrix. Practically, we apply two well-known methods for solving this new reformulation, i.e., the Krylov subspace method and the Riemannian trust region method. Both algorithms are factorization free so that they are suitable for solving large scale problems. Numerical results on both synthetic and real-world datasets indicate that the SPG-LS, equipped with the SCLS reformulation, can be solved orders of magnitude faster than the state of the art.

math.OC↗

arXiv2022

Copositivity for a class of fourth order symmetric tensors given by scalar dark matter

The mathematical model of multiple microscopic particles potentials corresponds to a fourth order symmetric tensor with a particular structure in particle physics. In this paper, we mainly dedicate to the study of copositivity for a class of tensors defined by the scalar dark matter with the standard model Higgs and an inert doublet and a complex singlet. With the help of its structure, we obtain the necessary and sufficient conditions, which attains the analytic conditions required by the physical problems. At the same time, this analytic expression provides how to determine a unique solution of the corresponding tensor complementarity problem with a parameter.

math-ph↗

arXiv2022

QPPAL: A two-phase proximal augmented Lagrangian method for high dimensional convex quadratic programming problems

In this paper, we aim to solve high dimensional convex quadratic programming (QP) problems with a large number of quadratic terms, linear equality and inequality constraints. In order to solve the targeted {\bf QP} problems to a desired accuracy efficiently, we develop a two-phase {\bf P}roximal {\bf A}ugmented {\bf L}agrangian method {(QPPAL)}, with Phase I to generate a reasonably good initial point to warm start Phase II to obtain an accurate solution efficiently. More specifically, in Phase I, based on the recently developed symmetric Gauss-Seidel (sGS) decomposition technique, we design a novel sGS based semi-proximal augmented Lagrangian method for the purpose of finding a solution of low to medium accuracy. Then, in Phase II, a proximal augmented Lagrangian algorithm is proposed to obtain a more accurate solution efficiently. Extensive numerical results evaluating the performance of {QPPAL} against {existing state-of-the-art solvers Gurobi, OSQP and QPALM} are presented to demonstrate the high efficiency and robustness of our proposed algorithm for solving various classes of large-scale convex QP problems. {The MATLAB implementation of the software package QPPAL is available at: \url{https://blog.nus.edu.sg/mattohkc/softwares/qppal/}.

math.OC↗

arXiv2021

Design of Power-Imbalanced SCMA Codebook

Sparse code multiple access (SCMA) is a promising multiuser communication technique for the enabling of future massive machine-type networks. Unlike existing codebook design schemes assuming uniform power allocation, we present a novel class of SCMA codebooks which display power imbalance among different users for downlink transmission. Based on the Star-QAM mother constellation structure and with the aid of genetic algorithm, we optimize the minimum Euclidean distance (MED) and the minimum product distance (MPD) of the proposed codebooks. Numerical simulation results show that our proposed codebooks lead to significantly improved error rate performances over Gaussian channels and Rayleigh fading channels.

cs.IT↗

arXiv2021

SCARLET-NAS: Bridging the Gap between Stability and Scalability in Weight-sharing Neural Architecture Search

To discover powerful yet compact models is an important goal of neural architecture search. Previous two-stage one-shot approaches are limited by search space with a fixed depth. It seems handy to include an additional skip connection in the search space to make depths variable. However, it creates a large range of perturbation during supernet training and it has difficulty giving a confident ranking for subnetworks. In this paper, we discover that skip connections bring about significant feature inconsistency compared with other operations, which potentially degrades the supernet performance. Based on this observation, we tackle the problem by imposing an equivariant learnable stabilizer to homogenize such disparities. Experiments show that our proposed stabilizer helps to improve the supernet's convergence as well as ranking performance. With an evolutionary search backend that incorporates the stabilized supernet as an evaluator, we derive a family of state-of-the-art architectures, the SCARLET series of several depths, especially SCARLET-A obtains 76.9% top-1 accuracy on ImageNet. Code is available at https://github.com/xiaomi-automl/ScarletNAS.

cs.LG↗

arXiv2021

Fast Algorithms for Stackelberg Prediction Game with Least Squares Loss

The Stackelberg prediction game (SPG) has been extensively used to model the interactions between the learner and data provider in the training process of various machine learning algorithms. Particularly, SPGs played prominent roles in cybersecurity applications, such as intrusion detection, banking fraud detection, spam filtering, and malware detection. Often formulated as NP-hard bi-level optimization problems, it is generally computationally intractable to find global solutions to SPGs. As an interesting progress in this area, a special class of SPGs with the least squares loss (SPG-LS) have recently been shown polynomially solvable by a bisection method. However, in each iteration of this method, a semidefinite program (SDP) needs to be solved. The resulted high computational costs prevent its applications for large-scale problems. In contrast, we propose a novel approach that reformulates a SPG-LS as a single SDP of a similar form and the same dimension as those solved in the bisection method. Our SDP reformulation is, evidenced by our numerical experiments, orders of magnitude faster than the existing bisection method. We further show that the obtained SDP can be reduced to a second order cone program (SOCP). This allows us to provide real-time response to large-scale SPG-LS problems. Numerical results on both synthetic and real world datasets indicate that the proposed SOCP method is up to 20,000+ times faster than the state of the art.

math.OC↗

arXiv2021

Few-Shot Meta-Learning on Point Cloud for Semantic Segmentation

The promotion of construction robots can solve the problem of human resource shortage and improve the quality of decoration. To help the construction robots obtain environmental information, we need to use 3D point cloud, which is widely used in robotics, autonomous driving, and so on. With a good understanding of environmental information, construction robots can work better. However, the dynamic changes of 3D point cloud data may bring difficulties for construction robots to understand environmental information, such as when construction robots renovate houses. The paper proposes a semantic segmentation method for point cloud based on meta-learning. The method includes a basic learning module and a meta-learning module. The basic learning module is responsible for learning data features and evaluating the model, while the meta-learning module is responsible for updating the parameters of the model and improving the model generalization ability. In our work, we pioneered the method of producing datasets for meta-learning in 3D scenes, as well as demonstrated that the Model-Agnostic Meta-Learning (MAML) algorithm can be applied to process 3D point cloud data. At the same time, experiments show that our method can allow the model to be quickly applied to new environments with a few samples. Our method has important applications.

cs.RO↗

arXiv2020

Learning Markov models via low-rank optimization

Modeling unknown systems from data is a precursor of system optimization and sequential decision making. In this paper, we focus on learning a Markov model from a single trajectory of states. Suppose that the transition model has a small rank despite of having a large state space, meaning that the system admits a low-dimensional latent structure. We show that one can estimate the full transition model accurately using a trajectory of length that is proportional to the total number of states. We propose two maximum likelihood estimation methods: a convex approach with nuclear-norm regularization and a nonconvex approach with rank constraint. We explicitly derive the statistical rates of both estimators in terms of the Kullback-Leiber divergence and the $\ell_2$ error and also establish a minimax lower bound to assess the tightness of these rates. For computing the nonconvex estimator, we develop a novel DC (difference of convex function) programming algorithm that starts with the convex M-estimator and then successively refines the solution till convergence. Empirical experiments demonstrate consistent superiority of the nonconvex estimator over the convex one.

stat.ME↗