arXiv subjects

Tao Li

Publications and source records attributed to Tao Li.

At least 361 records · Page 20Linked to original sources

arXiv2023

PIEKF-VIWO: Visual-Inertial-Wheel Odometry using Partial Invariant Extended Kalman Filter

Invariant Extended Kalman Filter (IEKF) has been successfully applied in Visual-inertial Odometry (VIO) as an advanced achievement of Kalman filter, showing great potential in sensor fusion. In this paper, we propose partial IEKF (PIEKF), which only incorporates rotation-velocity state into the Lie group structure and apply it for Visual-Inertial-Wheel Odometry (VIWO) to improve positioning accuracy and consistency. Specifically, we derive the rotation-velocity measurement model, which combines wheel measurements with kinematic constraints. The model circumvents the wheel odometer's 3D integration and covariance propagation, which is essential for filter consistency. And a plane constraint is also introduced to enhance the position accuracy. A dynamic outlier detection method is adopted, leveraging the velocity state output. Through the simulation and real-world test, we validate the effectiveness of our approach, which outperforms the standard Multi-State Constraint Kalman Filter (MSCKF) based VIWO in consistency and accuracy.

cs.RO↗

arXiv2023

Sampling Attacks on Meta Reinforcement Learning: A Minimax Formulation and Complexity Analysis

Meta reinforcement learning (meta RL), as a combination of meta-learning ideas and reinforcement learning (RL), enables the agent to adapt to different tasks using a few samples. However, this sampling-based adaptation also makes meta RL vulnerable to adversarial attacks. By manipulating the reward feedback from sampling processes in meta RL, an attacker can mislead the agent into building wrong knowledge from training experience, which deteriorates the agent's performance when dealing with different tasks after adaptation. This paper provides a game-theoretical underpinning for understanding this type of security risk. In particular, we formally define the sampling attack model as a Stackelberg game between the attacker and the agent, which yields a minimax formulation. It leads to two online attack schemes: Intermittent Attack and Persistent Attack, which enable the attacker to learn an optimal sampling attack, defined by an $ε$-first-order stationary point, within $\mathcal{O}(ε^{-2})$ iterations. These attack schemes freeride the learning progress concurrently without extra interactions with the environment. By corroborating the convergence results with numerical experiments, we observe that a minor effort of the attacker can significantly deteriorate the learning performance, and the minimax approach can also help robustify the meta RL algorithms.

cs.LG↗

arXiv2023

Self-Adaptive Driving in Nonstationary Environments through Conjectural Online Lookahead Adaptation

Powered by deep representation learning, reinforcement learning (RL) provides an end-to-end learning framework capable of solving self-driving (SD) tasks without manual designs. However, time-varying nonstationary environments cause proficient but specialized RL policies to fail at execution time. For example, an RL-based SD policy trained under sunny days does not generalize well to rainy weather. Even though meta learning enables the RL agent to adapt to new tasks/environments, its offline operation fails to equip the agent with online adaptation ability when facing nonstationary environments. This work proposes an online meta reinforcement learning algorithm based on the \emph{conjectural online lookahead adaptation} (COLA). COLA determines the online adaptation at every step by maximizing the agent's conjecture of the future performance in a lookahead horizon. Experimental results demonstrate that under dynamically changing weather and lighting conditions, the COLA-based self-adaptive driving outperforms the baseline policies in terms of online adaptability. A demo video, source code, and appendixes are available at {\tt https://github.com/Panshark/COLA}

cs.RO↗

arXiv2023

Scenario-Agnostic Zero-Trust Defense with Explainable Threshold Policy: A Meta-Learning Approach

The increasing connectivity and intricate remote access environment have made traditional perimeter-based network defense vulnerable. Zero trust becomes a promising approach to provide defense policies based on agent-centric trust evaluation. However, the limited observations of the agent's trace bring information asymmetry in the decision-making. To facilitate the human understanding of the policy and the technology adoption, one needs to create a zero-trust defense that is explainable to humans and adaptable to different attack scenarios. To this end, we propose a scenario-agnostic zero-trust defense based on Partially Observable Markov Decision Processes (POMDP) and first-order Meta-Learning using only a handful of sample scenarios. The framework leads to an explainable and generalizable trust-threshold defense policy. To address the distribution shift between empirical security datasets and reality, we extend the model to a robust zero-trust defense minimizing the worst-case loss. We use case studies and real-world attacks to corroborate the results.

cs.AI↗

arXiv2023

Multi-Arm Robot Task Planning for Fruit Harvesting Using Multi-Agent Reinforcement Learning

The emergence of harvesting robotics offers a promising solution to the issue of limited agricultural labor resources and the increasing demand for fruits. Despite notable advancements in the field of harvesting robotics, the utilization of such technology in orchards is still limited. The key challenge is to improve operational efficiency. Taking into account inner-arm conflicts, couplings of DoFs, and dynamic tasks, we propose a task planning strategy for a harvesting robot with four arms in this paper. The proposed method employs a Markov game framework to formulate the four-arm robotic harvesting task, which avoids the computational complexity of solving an NP-hard scheduling problem. Furthermore, a multi-agent reinforcement learning (MARL) structure with a fully centralized collaboration protocol is used to train a MARL-based task planning network. Several simulations and orchard experiments are conducted to validate the effectiveness of the proposed method for a multi-arm harvesting robot in comparison with the existing method.

cs.RO↗

arXiv2023

How Surface Roughness Affects the Interparticle Interactions at a Liquid Interface

Shapes of colloids matter at liquid interfaces. We explore the interactions between rough-surfaced nanocolloids at the air--water interface through the compaction of monolayers experimentally and numerically. Sufficiently rough systems exhibit a non-trivial intermediate state between a gas-like state and a close-packed jamming state due to roughness-induced capillary attraction. We also find that roughness-induced friction lowers the jamming point, and the tangential contact force owing to surface asperities can cause a gradual off-plane collapse of the compressed monolayer.

cond-mat.soft↗

arXiv2023

A numerical simulation method of fish adaption behavior based on deep reinforcement learning and fluid-structure coupling-realization of some lateral line functions

Improving the numerical method of fish autonomous swimming behavior in complex environments is of great significance to the optimization of bionic controller,the design of fish passing facilities and the study of fish behavior.This work has built a fish autonomous swimming simulation platform,which adapts the high-precision IB-LBM to simulate the dynamic process of the interaction between the fish and the flow field in real time,and realizes the fish brain motion control through the SAC deep reinforcement learning algorithm.More importantly,in view of the poor generalization of the existing simulation platform,a method to simulate the fish's lateral line function is proposed.By adding the Lateral-line machine and designing the Macro-action system,the intelligent fish has the ability to recognize,classify,memorize and transplant the corresponding swimming strategy in the unsteady field.Using this method,the training and simulation of point-to-point predation swimming and Kamangait test under different inlet velocities are carried out.In the example of point-to-point predation swimming,the fish in random position can adjust the swimming posture and speed autonomously to catch the fast moving food,and has a certain prediction ability on the movement trajectory of the food.In the Kaman-gait test,the trained fish are placed in three different Kamangait flow fields,to study its ability to recognize the flow field and select swimming strategies through experience.The results of numerical experiments show that,comparing with the other value function networks,the SAC algorithm based on maximum entropy has more advantages in convergence speed and training efficiency when simulating fish brain decision-making.The use of the Lateral line Machine and Macro-action system can avoid the waste of experience and improve the adaptability of intelligent fish in the new complex flow field environment.

physics.flu-dyn↗

arXiv2022

Artificial gauge field enabled low-crosstalk, broadband, half-wavelength-pitched waveguide arrays

Dense waveguide arrays with half-wavelength-pitch, low-crosstalk, broadband, and flexible routing capability are essential for integrated photonics. However, achieving such performance is challenging due to the relatively weaker confinement of dielectric waveguides and the increased interactions among densely packed waveguides. Here, leveraging the artificial gauge field mechanism, we demonstrate half-wavelength-pitched dense waveguide arrays, consisting of 64 waveguides, in silicon with -30dB crosstalk suppression from 1480nm to 1550nm. The waveguide array features negligible insertion loss for 90-degree bending. Our approach enables flexibly routing a large-scale dense waveguide array that significantly reduces on-chip estate, leading to a high-density photonic integrated circuit, and may open up opportunities for important device performance improvement, such as half-wavelength-pitch OPA and ultra-dense space-division multiplexing

physics.optics↗

arXiv2022

Cell Biomechanical Modeling Based on Membrane Theory with Considering Speed Effect of Microinjection

As an effective method to deliver external materials into biological cells, microinjection has been widely applied in the biomedical field. However, the cognition of cell mechanical property is still inadequate, which greatly limits the efficiency and success rate of injection. Thus, a new rate-dependent mechanical model based on membrane theory is proposed for the first time. In this model, an analytical equilibrium equation between the injection force and cell deformation is established by considering the speed effect of microinjection. Different from the traditional membrane-theory-based model, the elastic coefficient of the constitutive material in the proposed model is modified as a function of the injection velocity and acceleration, effectively simulating the influence of speeds on the mechanical responses and providing a more generalized and practical model. Using this model, other mechanical responses at different speeds can be also accurately predicted, including the distribution of membrane tension and stress and the deformed shape. To verify the validity of the model, numerical simulations and experiments are carried out. The results show that the proposed model can match the real mechanical responses well at different injection speeds.

cs.RO↗

arXiv2022

Efficient Generalization Improvement Guided by Random Weight Perturbation

To fully uncover the great potential of deep neural networks (DNNs), various learning algorithms have been developed to improve the model's generalization ability. Recently, sharpness-aware minimization (SAM) establishes a generic scheme for generalization improvements by minimizing the sharpness measure within a small neighborhood and achieves state-of-the-art performance. However, SAM requires two consecutive gradient evaluations for solving the min-max problem and inevitably doubles the training time. In this paper, we resort to filter-wise random weight perturbations (RWP) to decouple the nested gradients in SAM. Different from the small adversarial perturbations in SAM, RWP is softer and allows a much larger magnitude of perturbations. Specifically, we jointly optimize the loss function with random perturbations and the original loss function: the former guides the network towards a wider flat region while the latter helps recover the necessary local information. These two loss terms are complementary to each other and mutually independent. Hence, the corresponding gradients can be efficiently computed in parallel, enabling nearly the same training speed as regular training. As a result, we achieve very competitive performance on CIFAR and remarkably better performance on ImageNet (e.g. $\mathbf{ +1.1\%}$) compared with SAM, but always require half of the training time. The code is released at https://github.com/nblt/RWP.

cs.CV↗

arXiv2022

Profit-Driven Team Grouping in Social Networks

In this paper, we investigate the profit-driven team grouping problem in social networks. We consider a setting in which people possess different skills, and the compatibility between these individuals is captured by a social network. Moreover, there is a collection of tasks, where each task requires a specific set of skills and yields a profit upon completion. Individuals may collaborate with each other as \emph{teams} to accomplish a set of tasks. We aim to find a group of teams to maximize the total profit of the tasks that they can complete. Any feasible grouping must satisfy the following conditions: (i) each team possesses all the skills required by the task assigned to it, (ii) individuals belonging to the same team are socially compatible, and (iii) no individual is overloaded. We refer to this as the \textsc{TeamGrouping} problem. We analyze the computational complexity of this problem and then propose a linear program-based approximation algorithm to address it and its variants. Although we focus on team grouping, our results apply to a broad range of optimization problems that can be formulated as cover decomposition problems.

cs.SI↗

arXiv2022

Distributed Machine Learning with Strategic Network Design: A Game-Theoretic Perspective

This paper considers a game-theoretic framework for distributed machine learning problems over networks where the information acquisition at a node is modeled as a rational choice of a player. In the proposed game, players decide both the learning parameters and the network structure. The Nash equilibrium characterizes the tradeoff between the local performance and the global agreement of the learned classifiers. We first introduce a commutative approach which features a joint learning process that integrates the iterative learning at each node and the network formation. We show that our game is equivalent to a generalized potential game in the setting of undirected networks. We study the convergence of the proposed commutative algorithm, analyze the network structures determined by our game, and show the improvement of the social welfare in comparison with standard distributed learning over fixed networks. To adapt our framework to streaming data, we derive a distributed Kalman filter. A concurrent algorithm based on the online mirror descent algorithm is also introduced for solving for Nash equilibria in a holistic manner. In the case study, we use telemonitoring of Parkinson's disease to corroborate the results.

cs.GT↗

arXiv2022

Decentralized Policy Gradient for Nash Equilibria Learning of General-sum Stochastic Games

We study Nash equilibria learning of a general-sum stochastic game with an unknown transition probability density function. Agents take actions at the current environment state and their joint action influences the transition of the environment state and their immediate rewards. Each agent only observes the environment state and its own immediate reward and is unknown about the actions or immediate rewards of others. We introduce the concepts of weighted asymptotic Nash equilibrium with probability 1 and in probability. For the case with exact pseudo gradients, we design a two-loop algorithm by the equivalence of Nash equilibrium and variational inequality problems. In the outer loop, we sequentially update a constructed strongly monotone variational inequality by updating a proximal parameter while employing a single-call extra-gradient algorithm in the inner loop for solving the constructed variational inequality. We show that if the associated Minty variational inequality has a solution, then the designed algorithm converges to the k^{1/2}-weighted asymptotic Nash equilibrium. Further, for the case with unknown pseudo gradients, we propose a decentralized algorithm, where the G(PO)MDP gradient estimator of the pseudo gradient is provided by Monte-Carlo simulations. The convergence to the k^{1/4} -weighted asymptotic Nash equilibrium in probability is achieved.

eess.SY↗

arXiv2022

On the Resilience of Traffic Networks under Non-Equilibrium Learning

We investigate the resilience of learning-based \textit{Intelligent Navigation Systems} (INS) to informational flow attacks, which exploit the vulnerabilities of IT infrastructure and manipulate traffic condition data. To this end, we propose the notion of \textit{Wardrop Non-Equilibrium Solution} (WANES), which captures the finite-time behavior of dynamic traffic flow adaptation under a learning process. The proposed non-equilibrium solution, characterized by target sets and measurement functions, evaluates the outcome of learning under a bounded number of rounds of interactions, and it pertains to and generalizes the concept of approximate equilibrium. Leveraging finite-time analysis methods, we discover that under the mirror descent (MD) online-learning framework, the traffic flow trajectory is capable of restoring to the Wardrop non-equilibrium solution after a bounded INS attack. The resulting performance loss is of order $\tilde{\mathcal{O}}(T^β)$ ($-\frac{1}{2} \leq β< 0 )$), with a constant dependent on the size of the traffic network, indicating the resilience of the MD-based INS. We corroborate the results using an evacuation case study on a Sioux-Fall transportation network.

eess.SY↗

arXiv2022

MUG: Interactive Multimodal Grounding on User Interfaces

We present MUG, a novel interactive task for multimodal grounding where a user and an agent work collaboratively on an interface screen. Prior works modeled multimodal UI grounding in one round: the user gives a command and the agent responds to the command. Yet, in a realistic scenario, a user command can be ambiguous when the target action is inherently difficult to articulate in natural language. MUG allows multiple rounds of interactions such that upon seeing the agent responses, the user can give further commands for the agent to refine or even correct its actions. Such interaction is critical for improving grounding performances in real-world use cases. To investigate the problem, we create a new dataset that consists of 77,820 sequences of human user-agent interaction on mobile interfaces in which 20% involves multiple rounds of interactions. To establish our benchmark, we experiment with a range of modeling variants and evaluation strategies, including both offline and online evaluation-the online strategy consists of both human evaluation and automatic with simulators. Our experiments show that allowing iterative interaction significantly improves the absolute task completion by 18% over the entire test dataset and 31% over the challenging subset. Our results lay the foundation for further investigation of the problem.

cs.CL↗

arXiv2022

Event-Triggered Extended State Observer Based Distributed Control of Nonlinear Vehicle Platoons

We study the platoon control of vehicles with third-order nonlinear dynamics under the constant spacing policy. We consider a vehicle model with parameter uncertainties and external disturbances and propose a distributed control law based on an event-triggered extended state observer (ESO). First, an event-triggered ESO is designed to estimate the unmodeled dynamics in the vehicle model. Then based on the estimate of the unmodeled dynamics, a distributed control law is designed by using a modified dynamic surface control method. The control law of each follower vehicle only uses the information obtained by on-board sensors, including its own velocity, acceleration, the velocity of the preceding vehicle and the inter-vehicle distance. Finally, we give the range of the control parameters to ensure the stability of the vehicle platoon system. It is shown that the control parameters can be properly designed to make the observation errors of the ESOs bounded and ensure the string stability and closed-loop stability. We prove that the Zeno behavior is avoided under the designed event-triggered mechanism. The joint simulations of CarSim and MATLAB are given to demonstrate the effectiveness of the proposed control law.

eess.SY↗

arXiv2022

Animating collider processes with Event-time-frame Format

High Energy Physics processes, such as hard scattering, parton shower, and hadronization, occur at colliders around the world, e.g., the Large Hadron Collider in Europe. The various steps are also components within corresponding Monte-Carlo simulations. They are usually considered to occur in an instant and displayed in MC simulations as intricate paths hard-coded with the HepMC format. We recently developed a framework to convert HEP event records into online 3D animations, aiming for visual Monte-Carlo studies and science popularization, where the most difficult parts are about designing an event timeline and particles' movement. As a by-product, we propose here an event-time-frame format for animation data exchanging and persistence, which is potentially helpful in other visualization works. The code is maintained at https://github.com/lyazj/hepani, and the web service is available at https://ppnp.pku.edu.cn/hepani/index.html.

hep-ph↗

arXiv2022

A Scattering Matrix Formalism to Model Periodic Heat Diffusion in Stratified Solid Media

The transfer matrix formalism is widely used in modeling heat diffusion in layered structures.Due to an intrinsic numerical instability issue, which has not yet drawn enough attention to the heat transfer community,this formalism fails at high heating frequencies and/or in thick structures. Inspired by its success in modeling wave propagation, we develop a numerically-stable scattering matrix framework to model periodic heat diffusion in stratified solid media.As a concreate example, we apply this scattering matrix methodology to the three omega method.We first validate our framework using various well-known solutions.Next, we demonstrate the numerical stability of the framework using a configuration that resembles the three-dimensional stacked architecture for chip packing. Last, we propose synthetic experiments to exhibit, under certain circumstances, the merits of the scattering matrix formalism in extracting thermal properties.

physics.app-ph↗