arXiv subjects

Hongyi Li

Publications and source records attributed to Hongyi Li.

48 records · Page 3Linked to original sources

arXiv2024

Event-Triggered Active Disturbance Rejection Control for Uncertain Random Nonlinear Systems

In this paper, event-triggered active disturbance rejection control (ADRC) is first addressed for a class of uncertain random nonlinear systems driven by bounded noise and colored noise. The event-triggered extended state observer (ESO) and ADRC controller are designed, where two respective event-triggering mechanisms with a fixed positive lower bound for the inter-execution times are proposed. The random total disturbance representing the coupling of nonlinear unmodeled dynamics, external deterministic disturbance, bounded noise, and colored noise is estimated in real time by the event-triggered ESO and compensated in the event-triggered feedback loop. Both the mean square and almost surely practical convergence of the closed-loop systems is shown with rigorous theoretical analysis. Finally, some numerical simulations are implemented to validate the proposed control scheme and theoretical results.

math.OC↗

arXiv2024

i-Octree: A Fast, Lightweight, and Dynamic Octree for Proximity Search

Establishing the correspondences between newly acquired points and historically accumulated data (i.e., map) through nearest neighbors search is crucial in numerous robotic applications. However, static tree data structures are inadequate to handle large and dynamically growing maps in real-time. To address this issue, we present the i-Octree, a dynamic octree data structure that supports both fast nearest neighbor search and real-time dynamic updates, such as point insertion, deletion, and on-tree down-sampling. The i-Octree is built upon a leaf-based octree and has two key features: a local spatially continuous storing strategy that allows for fast access to points while minimizing memory usage, and local on-tree updates that significantly reduce computation time compared to existing static or dynamic tree structures. The experiments show that i-Octree outperforms contemporary state-of-the-art approaches by achieving, on average, a 19% reduction in runtime on realworld open datasets.

cs.RO↗

arXiv2023

Fast gradient method for Low-Rank Matrix Estimation

Projected gradient descent and its Riemannian variant belong to a typical class of methods for low-rank matrix estimation. This paper proposes a new Nesterov's Accelerated Riemannian Gradient algorithm by efficient orthographic retraction and tangent space projection. The subspace relationship between iterative and extrapolated sequences on the low-rank matrix manifold provides a computational convenience. With perturbation analysis of truncated singular value decomposition and two retractions, we systematically analyze the local convergence of gradient algorithms and Nesterov's variants in the Euclidean and Riemannian settings. Theoretically, we estimate the exact rate of local linear convergence under different parameters using the spectral radius in a closed form and give the optimal convergence rate and the corresponding momentum parameter. When the parameter is unknown, the adaptive restart scheme can avoid the oscillation problem caused by high momentum, thus approaching the optimal convergence rate. Extensive numerical experiments confirm the estimations of convergence rate and demonstrate that the proposed algorithm is competitive with first-order methods for matrix completion and matrix sensing.

math.OC↗

arXiv2022

Towards Quantized Model Parallelism for Graph-Augmented MLPs Based on Gradient-Free ADMM Framework

While Graph Neural Networks (GNNs) are popular in the deep learning community, they suffer from several challenges including over-smoothing, over-squashing, and gradient vanishing. Recently, a series of models have attempted to relieve these issues by first augmenting the node features and then imposing node-wise functions based on Multi-Layer Perceptron (MLP), which are widely referred to as GA-MLP models. However, while GA-MLP models enjoy deeper architectures for better accuracy, their efficiency largely deteriorates. Moreover, popular acceleration techniques such as stochastic-version or data-parallelism cannot be effectively applied due to the dependency among samples (i.e., nodes) in graphs. To address these issues, in this paper, instead of data parallelism, we propose a parallel graph deep learning Alternating Direction Method of Multipliers (pdADMM-G) framework to achieve model parallelism: parameters in each layer of GA-MLP models can be updated in parallel. The extended pdADMM-G-Q algorithm reduces communication costs by introducing the quantization technique. Theoretical convergence to a (quantized) stationary point of the pdADMM-G algorithm and the pdADMM-G-Q algorithm is provided with a sublinear convergence rate $o(1/k)$, where $k$ is the number of iterations. Extensive experiments demonstrate the convergence of two proposed algorithms. Moreover, they lead to a more massive speedup and better performance than all state-of-the-art comparison methods on nine benchmark datasets. Last but not least, the proposed pdADMM-G-Q algorithm reduces communication overheads by up to $45\%$ without loss of performance. Our code is available at \url{https://github.com/xianggebenben/pdADMM-G}.

cs.LG↗

arXiv2022

Edge Graph Neural Networks for Massive MIMO Detection

Massive Multiple-Input Multiple-Out (MIMO) detection is an important problem in modern wireless communication systems. While traditional Belief Propagation (BP) detectors perform poorly on loopy graphs, the recent Graph Neural Networks (GNNs)-based method can overcome the drawbacks of BP and achieve superior performance. Nevertheless, direct use of GNN ignores the importance of edge attributes and suffers from high computation overhead using a fully connected graph structure. In this paper, we propose an efficient GNN-inspired algorithm, called the Edge Graph Neural Network (EGNN), to detect MIMO signals. We first compute graph edge weights through channel correlation and then leverage the obtained weights as a metric to evaluate the importance of neighbors of each node. Moreover, we design an adaptive Edge Drop (ED) scheme to sparsify the graph such that computational cost can be significantly reduced. Experimental results demonstrate that our proposed EGNN achieves better or comparable performance to popular MIMO detection methods for different modulation schemes and costs the least detection time compared to GNN-based approaches.

cs.IT↗

arXiv2022

Accelerated Gradient-free Neural Network Training by Multi-convex Alternating Optimization

In recent years, even though Stochastic Gradient Descent (SGD) and its variants are well-known for training neural networks, it suffers from limitations such as the lack of theoretical guarantees, vanishing gradients, and excessive sensitivity to input. To overcome these drawbacks, alternating minimization methods have attracted fast-increasing attention recently. As an emerging and open domain, however, several new challenges need to be addressed, including 1) Convergence properties are sensitive to penalty parameters, and 2) Slow theoretical convergence rate. We, therefore, propose a novel monotonous Deep Learning Alternating Minimization (mDLAM) algorithm to deal with these two challenges. Our innovative inequality-constrained formulation infinitely approximates the original problem with non-convex equality constraints, enabling our convergence proof of the proposed mDLAM algorithm regardless of the choice of hyperparameters. Our mDLAM algorithm is shown to achieve a fast linear convergence by the Nesterov acceleration technique. Extensive experiments on multiple benchmark datasets demonstrate the convergence, effectiveness, and efficiency of the proposed mDLAM algorithm. Our code is available at \url{https://github.com/xianggebenben/mDLAM}.

math.OC↗

arXiv2021

A Convergent ADMM Framework for Efficient Neural Network Training

As a well-known optimization framework, the Alternating Direction Method of Multipliers (ADMM) has achieved tremendous success in many classification and regression applications. Recently, it has attracted the attention of deep learning researchers and is considered to be a potential substitute to Gradient Descent (GD). However, as an emerging domain, several challenges remain unsolved, including 1) The lack of global convergence guarantees, 2) Slow convergence towards solutions, and 3) Cubic time complexity with regard to feature dimensions. In this paper, we propose a novel optimization framework to solve a general neural network training problem via ADMM (dlADMM) to address these challenges simultaneously. Specifically, the parameters in each layer are updated backward and then forward so that parameter information in each layer is exchanged efficiently. When the dlADMM is applied to specific architectures, the time complexity of subproblems is reduced from cubic to quadratic via a dedicated algorithm design utilizing quadratic approximations and backtracking techniques. Last but not least, we provide the first proof of convergence to a critical point sublinearly for an ADMM-type method (dlADMM) under mild conditions. Experiments on seven benchmark datasets demonstrate the convergence, efficiency, and effectiveness of our proposed dlADMM algorithm.

cs.LG↗

arXiv2021

Community-based Layerwise Distributed Training of Graph Convolutional Networks

The Graph Convolutional Network (GCN) has been successfully applied to many graph-based applications. Training a large-scale GCN model, however, is still challenging: Due to the node dependency and layer dependency of the GCN architecture, a huge amount of computational time and memory is required in the training process. In this paper, we propose a parallel and distributed GCN training algorithm based on the Alternating Direction Method of Multipliers (ADMM) to tackle the two challenges simultaneously. We first split GCN layers into independent blocks to achieve layer parallelism. Furthermore, we reduce node dependency by dividing the graph into several dense communities such that each of them can be trained with an agent in parallel. Finally, we provide solutions for all subproblems in the community-based ADMM algorithm. Preliminary results demonstrate that our proposed community-based ADMM training algorithm can lead to more than triple speedup while achieving the best performance compared with state-of-the-art methods.

cs.LG↗

arXiv2021

Commentaries on the publication entitled Evidence for continuity of interstitial spaces across tissue and organ boundaries in humans

Interstitial connective tissues are one of the four basic types of animal tissue and continuously distributed throughout the body. Illustrated by the movements of the micron-sized particles across the defined tissue boundaries and the continuously distributions of the fluorescently stained hyaluronic acid within the fibrous tissues around blood vessels and nerves, the authors of this paper suggested that interstitial fluid (ISF) is not fixed locally but would flow through fibrous matrices for a long-distance, even between more distant parts of the body. However,the fluid-filled interstitial spaces are similar with the interstitial tissue channel, lacking a distinct spatial structure for "free" interstitial fluid flow, which might be the key to comprehend the whereabouts and directions of a systemic fluid flow and circulation.

q-bio.TO↗

arXiv2021

The Impact of COVID-19 on Urban Energy Consumption of the Commercial Tourism City

In 2020, the COVID-19 pandemic spreads all over the world. In order to alleviate the spread of the epidemic, various blockade policies have been implemented in many areas. In order to formulate a better epidemic prevention policy for urban energy consumption of the commercial tourism cities, this paper first analyses the energy characteristics of Macao during the epidemic period from two aspects, based on the energy consumption data of Macao. On this basis, the power consumption characteristics of commercial tourism cities during the epidemic were analyzed. Then, this paper provides analysis of the characteristics of the energy consumption in different fields of commercial tourism cities from the aspects of hotel, transportation, tourism culture and public utilities. Finally, a detailed analysis of the energy consumption characteristics of commercial tourism cities represented by Macao during the epidemic period is provided, by comparing with some typical countries.

physics.soc-ph↗

arXiv2020

Relational Communication

We study a communication game between an informed sender and an uninformed receiver with repeated interactions and voluntary transfers. Transfers motivate the receiver's decision-making and signal the sender's information. Although full separation can always be supported in equilibrium, partial or complete pooling is optimal if the receiver's decision-making is highly responsive to information. In this case, the receiver's decision-making is disciplined by pooling extreme states where she is most tempted to defect.

econ.TH↗

arXiv2020

A multilayer interstitial fluid flow along vascular adventitia

Objective: Interstitial fluid flow through vascular adventitia has been disclosed recently. However, its kinetic pattern was unclear. Methods and Results: We used histological and topographical identifications to observe ISF flow along venous vessels in rabbits. By MRI in alive subjects, the inherent ISF flow pathways in legs, abdomen and thorax were enhanced by paramagnetic contrast from ankle dermis. By fluorescence stereomicroscopy and layer-by-layer dissection after the rabbits were sacrificed, the perivascular and adventitial connective tissues (PACT) along the saphenous veins and inferior vena cava were found to be stained by sodium fluorescein from ankle dermis, which coincided with the findings by MRI. By confocal microscopy and histological analysis, the stained PACT pathways were verified to be the fibrous connective tissues and consisted of longitudinally assembled fibers. By usages of nanoparticles and surfactants, a PACT pathway was found to be accessible for a nanoparticle under 100nm and contain two parts: a tunica channel part and an absorptive part. In real-time observations, the calculated velocity of a continuous ISF flow along fibers of a PACT pathway was 3.6-15.6 mm/sec. Conclusion: These data further revealed more kinetic features of a continuous ISF flow along vascular vessel. A multiscale, multilayer, and multiform interstitial/interfacial fluid flow throughout perivascular and adventitial connective tissues was suggested as one of kinetic and dynamic mechanisms for ISF flow, which might be another principal fluid dynamic pattern besides convective/vascular and diffusive transport in biological system.

q-bio.TO↗