arXiv subjects

Huiping Li

Publications and source records attributed to Huiping Li.

At least 19 recordsLinked to original sources

arXiv2026

Knowledge-Distilled End-to-End Reinforcement Learning for Smooth 6-DOF Thrust Control and Rapid Adaptation to Ocean Currents in Remotely Operated Vehicles

With the continuous improvement of computational capabilities, end-to-end reinforcement learning has been rapidly developed for remotely operated vehicles control. Nevertheless, existing end-to-end reinforcement-learningbased methods still face challenges in achieving optimal control under oceancurrent disturbances. In particular, there remains a lack of a unified control framework that can simultaneously achieve low steady-state tracking error, rapid transient response, energy-efficient operation, and smooth controlforce outputs under disturbances. To address the issue, this paper proposes the thrust smoothness rapid current adaptation proximal policy optimization (TSRCA-PPO) method which learns a near-optimal strategy by a twostage distillation learning framework. The core innovations of this work lie in the reward-function design and the privileged multi-encoder architecture. Ablation studies validate the effectiveness of each module. Simulation results demonstrate that the proposed TSRCA-PPO method consistently outperforms the conventional cascaded P-PID controller across all evaluation metrics. Specifically, TSRCA-PPO reduces the steady-state position error, steady-state attitude error, settling time, energy index, and thrustsmoothness index to 42.7%, 76.5%, 10.6%, 93.5%, and 15.9% of the corresponding P-PID values, respectively.

cs.RO↗

arXiv2026

Weighted Riemannian Optimization for Solving Quadratic Equations from Gaussian Magnitude Measurements

This paper explores the problem of generalized phase retrieval, which involves reconstructing a length-$n$ signal $\bm{x}$ from its $m$ phaseless samples $y_k = \left|\langle \bm{a}_k,\bm{x}\rangle\right|^2$, where $k = 1,2,...,m$, and $\bm{a}_k$ are the measurement vectors. This problem can be reformulated into recovering a positive semidefinite rank-$1$ matrix $\bm{X}=\bm{x}\bm{x}^*$ from linear samples $\bm{y}=\mathcal{A}(\bm{X})\in\mathbb{R}^m$, thereby requiring us to find a rank-$1$ solution of the linear equations. We demonstrate that several existing phase retrieval algorithms, including Wirtinger Flow (WF) and the canonical Riemannian gradient descent (RGD), actually solve the least-squares fitting of this linear equation on the Riemannian manifold of rank-$1$ matrices, but utilize different metrics on this manifold. Nevertheless, these metrics only allow for a stable and far-apart-from-isometric embedding of rank-$1$ matrices to $\mathbb{R}^m$ by $\mathcal{A}$, resulting in a linear convergence with a considerably large convergence factor. To expedite the convergence, we establish a new metric on the rank-$1$ matrix manifold that facilitates the nearly isometric embedding of rank-$1$ matrices into $\mathbb{R}^m$ through $\mathcal{A}$. A RGD algorithm under this new metric, termed Weighted RGD (WRGD), is proposed to tackle the phase retrieval problem. Owing to the near isometry, we prove that our WRGD algorithm, initialized by spectral methods, can linearly converge to the underlying signal $\bm{x}$ with a small convergence factor. Empirical experiments strongly validate the efficiency and resilience of our algorithms compared to the truncated Wirtinger Flow (TWF) algorithm and the canonical RGD algorithm.

cs.IT↗

arXiv2025

TIMRL: A Novel Meta-Reinforcement Learning Framework for Non-Stationary and Multi-Task Environments

In recent years, meta-reinforcement learning (meta-RL) algorithm has been proposed to improve sample efficiency in the field of decision-making and control, enabling agents to learn new knowledge from a small number of samples. However, most research uses the Gaussian distribution to extract task representation, which is poorly adapted to tasks that change in non-stationary environment. To address this problem, we propose a novel meta-reinforcement learning method by leveraging Gaussian mixture model and the transformer network to construct task inference model. The Gaussian mixture model is utilized to extend the task representation and conduct explicit encoding of tasks. Specifically, the classification of tasks is encoded through transformer network to determine the Gaussian component corresponding to the task. By leveraging task labels, the transformer network is trained using supervised learning. We validate our method on MuJoCo benchmarks with non-stationary and multi-task environments. Experimental results demonstrate that the proposed method dramatically improves sample efficiency and accurately recognizes the classification of the tasks, while performing excellently in the environment.

cs.LG↗

arXiv2024

Fast and Accurate Multi-Agent Trajectory Prediction For Crowded Unknown Scenes

This paper studies the problem of multi-agent trajectory prediction in crowded unknown environments. A novel energy function optimization-based framework is proposed to generate prediction trajectories. Firstly, a new energy function is designed for easier optimization. Secondly, an online optimization pipeline for calculating parameters and agents' velocities is developed. In this pipeline, we first design an efficient group division method based on Frechet distance to classify agents online. Then the strategy on decoupling the optimization of velocities and critical parameters in the energy function is developed, where the the slap swarm algorithm and gradient descent algorithms are integrated to solve the optimization problems more efficiently. Thirdly, we propose a similarity-based resample evaluation algorithm to predict agents' optimal goals, defined as the target-moving headings of agents, which effectively extracts hidden information in observed states and avoids learning agents' destinations via the training dataset in advance. Experiments and comparison studies verify the advantages of the proposed method in terms of prediction accuracy and speed.

cs.RO↗

arXiv2023

Directly Attention Loss Adjusted Prioritized Experience Replay

Prioritized Experience Replay (PER) enables the model to learn more about relatively important samples by artificially changing their accessed frequencies. However, this non-uniform sampling method shifts the state-action distribution that is originally used to estimate Q-value functions, which brings about the estimation deviation. In this article, an novel off policy reinforcement learning training framework called Directly Attention Loss Adjusted Prioritized Experience Replay (DALAP) is proposed, which can directly quantify the changed extent of the shifted distribution through Parallel Self-Attention network, so as to accurately compensate the error. In addition, a Priority-Encouragement mechanism is designed simultaneously to optimize the sample screening criterion, and further improve the training efficiency. In order to verify the effectiveness and generality of DALAP, we integrate it with the value-function based, the policy-gradient based and multi-agent reinforcement learning algorithm, respectively. The multiple groups of comparative experiments show that DALAP has the significant advantages of both improving the convergence rate and reducing the training variance.

cs.LG↗

arXiv2023

QFree: A Universal Value Function Factorization for Multi-Agent Reinforcement Learning

Centralized training is widely utilized in the field of multi-agent reinforcement learning (MARL) to assure the stability of training process. Once a joint policy is obtained, it is critical to design a value function factorization method to extract optimal decentralized policies for the agents, which needs to satisfy the individual-global-max (IGM) principle. While imposing additional limitations on the IGM function class can help to meet the requirement, it comes at the cost of restricting its application to more complex multi-agent environments. In this paper, we propose QFree, a universal value function factorization method for MARL. We start by developing mathematical equivalent conditions of the IGM principle based on the advantage function, which ensures that the principle holds without any compromise, removing the conservatism of conventional methods. We then establish a more expressive mixing network architecture that can fulfill the equivalent factorization. In particular, the novel loss function is developed by considering the equivalent conditions as regularization term during policy evaluation in the MARL algorithm. Finally, the effectiveness of the proposed method is verified in a nonmonotonic matrix game scenario. Moreover, we show that QFree achieves the state-of-the-art performance in a general-purpose complex MARL benchmark environment, Starcraft Multi-Agent Challenge (SMAC).

cs.AI↗

arXiv2023

Attention Loss Adjusted Prioritized Experience Replay

Prioritized Experience Replay (PER) is a technical means of deep reinforcement learning by selecting experience samples with more knowledge quantity to improve the training rate of neural network. However, the non-uniform sampling used in PER inevitably shifts the state-action space distribution and brings the estimation error of Q-value function. In this paper, an Attention Loss Adjusted Prioritized (ALAP) Experience Replay algorithm is proposed, which integrates the improved Self-Attention network with Double-Sampling mechanism to fit the hyperparameter that can regulate the importance sampling weights to eliminate the estimation error caused by PER. In order to verify the effectiveness and generality of the algorithm, the ALAP is tested with value-function based, policy-gradient based and multi-agent reinforcement learning algorithms in OPENAI gym, and comparison studies verify the advantage and efficiency of the proposed training framework.

cs.LG↗

arXiv2022

Policy Learning for Nonlinear Model Predictive Control with Application to USVs

The unaffordable computation load of nonlinear model predictive control (NMPC) has prevented it for being used in robots with high sampling rates for decades. This paper is concerned with the policy learning problem for nonlinear MPC with system constraints, and its applications to unmanned surface vehicles (USVs), where the nonlinear MPC policy is learned offline and deployed online to resolve the computational complexity issue. A deep neural networks (DNN) based policy learning MPC (PL-MPC) method is proposed to avoid solving nonlinear optimal control problems online. The detailed policy learning method is developed and the PL-MPC algorithm is designed. The strategy to ensure the practical feasibility of policy implementation is proposed, and it is theoretically proved that the closed-loop system under the proposed method is asymptotically stable in probability. In addition, we apply the PL-MPC algorithm successfully to the motion control of USVs. It is shown that the proposed algorithm can be implemented at a sampling rate up to $5 Hz$ with high-precision motion control. The experiment video is available via:\url{https://v.youku.com/v_show/id_XNTkwMTM0NzM5Ng==.html

cs.RO↗

arXiv2022

Accelerated Dual Averaging Methods for Decentralized Constrained Optimization

In this work, we study decentralized convex constrained optimization problems in networks. We focus on the dual averaging-based algorithmic framework that is well-documented to be superior in handling constraints and complex communication environments simultaneously. Two new decentralized dual averaging (DDA) algorithms are proposed. In the first one, a second-order dynamic average consensus protocol is tailored for DDA-type algorithms, which equips each agent with a provably more accurate estimate of the global dual variable than conventional schemes. We rigorously prove that the proposed algorithm attains $\mathcal{O}(1/t)$ convergence for general convex and smooth problems, for which existing DDA methods were only known to converge at $\mathcal{O}(1/\sqrt{t})$ prior to our work. In the second one, we use the extrapolation technique to accelerate the convergence of DDA. Compared to existing accelerated algorithms, where typically two different variables are exchanged among agents at each time, the proposed algorithm only seeks consensus on local gradients. Then, the extrapolation is performed based on two sequences of primal variables which are determined by the accumulations of gradients at two consecutive time instants, respectively. The algorithm is proved to converge at $\mathcal{O}(1)\left(\frac{1}{t^2}+\frac{1}{t(1-β)^2}\right)$, where $β$ denotes the second largest singular value of the mixing matrix. We remark that the condition for the algorithmic parameter to guarantee convergence does not rely on the spectrum of the mixing matrix, making itself easy to satisfy in practice. Finally, numerical results are presented to demonstrate the efficiency of the proposed methods.

math.OC↗

arXiv2021

Economic MPC-based planning for marine vehicles: Tuning safety and energy efficiency

Energy efficiency and safety are two critical objectives for marine vehicles operating in environments with obstacles, and they generally conflict with each other. In this paper, we propose a novel online motion planning method of marine vehicles which can make trade-offs between the two design objectives based on the framework of economic model predictive control (EMPC). Firstly, the feasible trajectory with the most safety margin is designed and utilized as tracking reference. Secondly, the EMPC-based receding horizon motion planning algorithm is designed, in which the practical consumed energy and safety measure (i.e., the distance between the planning trajectory and the reference) are considered. Experimental results verify the effectiveness and feasibility of the proposed method.

eess.SY↗

arXiv2021

Phonon-related monochromatic THz radiation and its magneto-modulation in 2D ferromagnetic Cr2Ge2Te6

Searching multiple types of terahertz (THz) irradiation source is crucial for the THz technology. Here, by utilizing a two-dimensional (2D) ferromagnetic Cr2Ge2Te6 crystal, we firstly demonstrate a magneto-tunable monochromatic THz irradiation source. With a low-photonic-energy broadband THz pump, a strong THz irradiation with frequency ~0.9 THz and bandwidth ~0.25 THz can be generated and its conversion efficiency could even reach 2.1% at 160 K. Moreover, it is intriguing to find that such monochromatic THz irradiation can be efficiently modulated by the magnetic field below 160 K. According to both experimental and theoretical analyses, the emergent THz irradiation is identified as the emission from the phonon-polariton and its temperature and magnetic field dependent behaviors confirmed the large spin-lattice coupling in this 2D ferromagnetic crystal. These observations provide a new route for the creation of tunable monochromatic THz source which may have great practical interests in future applications in photonic and spintronic devices.

physics.optics↗

arXiv2021

Stochastic Model Predictive Control for Linear Systems with Unbounded Additive Uncertainties

This paper presents two stochastic model predictive control methods for linear time-invariant systems subject to unbounded additive uncertainties. The new methods are developed by formulating the chance constraints into deterministic form, which are treated in analogy with robust constraints, by using the probabilistic reachable set. Firstly, the probabilistically resolvable time-varying tube-based stochastic model predictive control algorithm is designed by employing the time-varying probabilistic reachable sets as the tubes. Secondly, by utilizing the probabilistic positively invariant set, the probabilistically resolvable constant tube-based stochastic model predictive control algorithm is developed by employing the constantly tightened constraints in the entire prediction horizons. In addition, to enhance the feasibility of the algorithms, the soft constraints are imposed to the state initializations. The algorithm feasibility and closed-loop stability results are provided. The efficacy of the approaches are demonstrated by means of numerical simulations.

eess.SY↗

arXiv2020

Kinetics-Limited Two-Step Growth of van der Waals Puckered Honeycomb Sb Monolayer

Puckered honeycomb Sb monolayer, the structural analog of black phosphorene, has been recently successfully grown by means of molecular beam epitaxy. However, little is known to date about the growth mechanism for such puckered honeycomb monolayer. In this study, by using scanning tunneling microscopy in combination with first-principles density functional theory calculations, we unveil that the puckered honeycomb Sb monolayer takes a kinetics-limited two-step growth mode. As the coverage of Sb increases, the Sb atoms firstly form the distorted hexagonal lattice as the half layer, and then the distorted hexagonal half-layer transforms into the puckered honeycomb lattice as the full layer. These results provide the atomic-scale insight in understanding the growth mechanism of puckered honeycomb monolayer, and can be instructive to the direct growth of other monolayers with the same structure.

cond-mat.mtrl-sci↗

arXiv2020

Tuning the electronic structure of α-antimonene monolayer through interface engineering

The interfacial charge transfer from the substrate may influence the electronic structure of the epitaxial van der Waals (vdW) monolayers and thus their further technological applications. For instance, the freestanding Sb monolayer in puckered honeycomb phase (α-antimonene), the structural analog of black phosphorene, was predicted to be a semiconductor, but the epitaxial one behaves as a gapless semimetal when grown on the Td-WTe2 substrate. Here, we demonstrate that interface engineering can be applied to tune the interfacial charge transfer and thus the electron band of epitaxial monolayer. As a result, the nearly freestanding (semiconducting) α-antimonene monolayer with a band gap of ~170 meV was successfully obtained on the SnSe substrate. Furthermore, a semiconductor-semimetal crossover is observed in the bilayer α-antimonene. This study paves the way towards modifying the electron structure in two-dimensional vdW materials through interface engineering.

cond-mat.mtrl-sci↗

arXiv2020

Towards an $O(\frac{1}{t})$ convergence rate for distributed dual averaging

Recently, distributed dual averaging has received increasing attention due to its superiority in handling constraints and dynamic networks in multiagent optimization. However, all distributed dual averaging methods reported so far considered nonsmooth problems and have a convergence rate of $O(\frac{1}{\sqrt{t}})$. To achieve an improved convergence guarantee for smooth problems, this work proposes a second-order consensus scheme that assists each agent to locally track the global dual variable more accurately. This new scheme in conjunction with smoothness of the objective ensures that the accumulation of consensus error over time caused by incomplete global information is bounded from above. Then, a rigorous investigation of dual averaging with inexact gradient oracles is carried out to compensate the consensus error and achieve an $O(\frac{1}{t})$ convergence rate. The proposed method is examined in a large-scale LASSO problem.

math.OC↗

arXiv2020

Resource-aware Exact Decentralized Optimization Using Event-triggered Broadcasting

This work addresses the decentralized optimization problem where a group of agents with coupled private objective functions work together to exactly optimize the summation of local interests. Upon modeling the decentralized problem as an equality-constrained centralized one, we leverage the linearized augmented Lagrangian method (LALM) to design an event-triggered decentralized algorithm that only requires light local computation at generic time instants and peer-to-peer communication at sporadic triggering time instants. The triggering time instants for each agent are locally determined by comparing the deviation between true and broadcast primal variables with certain triggering thresholds. Provided that the threshold is summable over time, we established a new upper bound for the effect of triggering behavior on the primal-dual residual. Based on this, the same convergence rate $O(\frac{1}{k})$ with periodic algorithms is secured for nonsmooth convex problems. Stronger convergence results have been further established for strongly convex and smooth problems, that is, the iterates linearly converge with exponentially decaying triggering thresholds.} We examine the developed strategy in two common optimization problems; comparison results illustrate its performance and superiority in exploiting communication resources.

math.OC↗

arXiv2020

A unitary distributed subgradient method for multi-agent optimization with different coupling sources

In this work, we first consider distributed convex constrained optimization problems where the objective function is encoded by multiple local and possibly nonsmooth objectives privately held by a group of agents, and propose a distributed subgradient method with double averaging (abbreviated as ${\rm DSA_2}$) that only requires peer-to-peer communication and local computation to solve the global problem. The algorithmic framework builds on dual methods and dynamic average consensus; the sequence of test points is formed by iteratively minimizing a local dual model of the overall objective where the coefficients, i.e., approximated subgradients of the objective, are supplied by the dynamic average consensus scheme. We theoretically show that ${\rm DSA_2}$ enjoys non-ergodic convergence properties, i.e., the local minimizing sequence itself is convergent, a distinct feature that cannot be found in existing results. Specifically, we establish a convergence rate of $O(\frac{1}{\sqrt{t}})$ in terms of objective function error. Then, extensions are made to tackle distributed optimization problems with coupled functional constraints by combining ${\rm DSA_2}$ and dual decomposition. This is made possible by Lagrangian relaxation that transforms the coupling in constraints of the primal problem into that in cost functions of the dual, thus allowing us to solve the dual problem via ${\rm DSA_2}$. Both the dual objective error and the quadratic penalty for the coupled constraint are proved to converge at a rate of $O(\frac{1}{\sqrt{t}})$, and the primal objective error asymptotically vanishes. Numerical experiments and comparisons are conducted to illustrate the advantage of the proposed algorithms and validate our theoretical findings.

math.OC↗

arXiv2019

Van der Waals heteroepitaxial growth of monolayer Sb in puckered honeycomb structure

Atomically thin two-dimensional (2D) crystals have gained tremendous attentions owing to their potential impacts to the future electronics technologies, as well as the exotic phenomena emerging in these materials. Monolayer of α phase Sb (α-antimonene) that shares the same puckered structure as black phosphorous, has been predicted to be stable with precious properties. However, the experimental realization still remains challenging. Here, we successfully grow high-quality monolayer α-antimonene, with the thickness finely controlled. The α-antimonene exhibits great stability upon exposure to air. Combining scanning tunneling microscope, density functional theory calculations and transport measurement, it is found that the electron band crossing the Fermi level exhibits a linear dispersion with a fairly small effective mass, and thus a good electrical conductivity. All of these properties make the α-antimonene promising in the future electronic applications.

cond-mat.mtrl-sci↗