arXiv subjects

Li Li

Publications and source records attributed to Li Li.

At least 433 records · Page 24Linked to original sources

arXiv2023

Unified Emulation-Simulation Training Environment for Autonomous Cyber Agents

Autonomous cyber agents may be developed by applying reinforcement and deep reinforcement learning (RL/DRL), where agents are trained in a representative environment. The training environment must simulate with high-fidelity the network Cyber Operations (CyOp) that the agent aims to explore. Given the complexity of net-work CyOps, a good simulator is difficult to achieve. This work presents a systematic solution to automatically generate a high-fidelity simulator in the Cyber Gym for Intelligent Learning (CyGIL). Through representation learning and continuous learning, CyGIL provides a unified CyOp training environment where an emulated CyGIL-E automatically generates a simulated CyGIL-S. The simulator generation is integrated with the agent training process to further reduce the required agent training time. The agent trained in CyGIL-S is transferrable directly to CyGIL-E showing full transferability to the emulated "real" network. Experimental results are presented to demonstrate the CyGIL training performance. Enabling offline RL, the CyGIL solution presents a promising direction towards sim-to-real for leveraging RL agents in real-world cyber networks.

cs.LG↗

arXiv2023

Enabling A Network AI Gym for Autonomous Cyber Agents

This work aims to enable autonomous agents for network cyber operations (CyOps) by applying reinforcement and deep reinforcement learning (RL/DRL). The required RL training environment is particularly challenging, as it must balance the need for high-fidelity, best achieved through real network emulation, with the need for running large numbers of training episodes, best achieved using simulation. A unified training environment, namely the Cyber Gym for Intelligent Learning (CyGIL) is developed where an emulated CyGIL-E automatically generates a simulated CyGIL-S. From preliminary experimental results, CyGIL-S is capable to train agents in minutes compared with the days required in CyGIL-E. The agents trained in CyGIL-S are transferrable directly to CyGIL-E showing full decision proficiency in the emulated "real" network. Enabling offline RL, the CyGIL solution presents a promising direction towards sim-to-real for leveraging RL agents in real-world cyber networks.

cs.AI↗

arXiv2023

A Multiagent CyberBattleSim for RL Cyber Operation Agents

Hardening cyber physical assets is both crucial and labor-intensive. Recently, Machine Learning (ML) in general and Reinforcement Learning RL) more specifically has shown great promise to automate tasks that otherwise would require significant human insight/intelligence. The development of autonomous RL agents requires a suitable training environment that allows us to quickly evaluate various alternatives, in particular how to arrange training scenarios that pit attackers and defenders against each other. CyberBattleSim is a training environment that supports the training of red agents, i.e., attackers. We added the capability to train blue agents, i.e., defenders. The paper describes our changes and reports on the results we obtained when training blue agents, either in isolation or jointly with red agents. Our results show that training a blue agent does lead to stronger defenses against attacks. In particular, training a blue agent jointly with a red agent increases the blue agent's capability to thwart sophisticated red agents.

cs.CR↗

arXiv2023

Milestones in Autonomous Driving and Intelligent Vehicles: Survey of Surveys

Interest in autonomous driving (AD) and intelligent vehicles (IVs) is growing at a rapid pace due to the convenience, safety, and economic benefits. Although a number of surveys have reviewed research achievements in this field, they are still limited in specific tasks, lack of systematic summary and research directions in the future. Here we propose a Survey of Surveys (SoS) for total technologies of AD and IVs that reviews the history, summarizes the milestones, and provides the perspectives, ethics, and future research directions. To our knowledge, this article is the first SoS with milestones in AD and IVs, which constitutes our complete research work together with two other technical surveys. We anticipate that this article will bring novel and diverse insights to researchers and abecedarians, and serve as a bridge between past and future.

cs.RO↗

arXiv2023

Less is More: Reducing Task and Model Complexity for 3D Point Cloud Semantic Segmentation

Whilst the availability of 3D LiDAR point cloud data has significantly grown in recent years, annotation remains expensive and time-consuming, leading to a demand for semi-supervised semantic segmentation methods with application domains such as autonomous driving. Existing work very often employs relatively large segmentation backbone networks to improve segmentation accuracy, at the expense of computational costs. In addition, many use uniform sampling to reduce ground truth data requirements for learning needed, often resulting in sub-optimal performance. To address these issues, we propose a new pipeline that employs a smaller architecture, requiring fewer ground-truth annotations to achieve superior segmentation accuracy compared to contemporary approaches. This is facilitated via a novel Sparse Depthwise Separable Convolution module that significantly reduces the network parameter count while retaining overall task performance. To effectively sub-sample our training data, we propose a new Spatio-Temporal Redundant Frame Downsampling (ST-RFD) method that leverages knowledge of sensor motion within the environment to extract a more diverse subset of training data frame samples. To leverage the use of limited annotated data samples, we further propose a soft pseudo-label method informed by LiDAR reflectivity. Our method outperforms contemporary semi-supervised work in terms of mIoU, using less labeled data, on the SemanticKITTI (59.5@5%) and ScribbleKITTI (58.1@5%) benchmark datasets, based on a 2.3x reduction in model parameters and 641x fewer multiply-add operations whilst also demonstrating significant performance improvement on limited training data (i.e., Less is More).

cs.CV↗

arXiv2023

HandNeRF: Neural Radiance Fields for Animatable Interacting Hands

We propose a novel framework to reconstruct accurate appearance and geometry with neural radiance fields (NeRF) for interacting hands, enabling the rendering of photo-realistic images and videos for gesture animation from arbitrary views. Given multi-view images of a single hand or interacting hands, an off-the-shelf skeleton estimator is first employed to parameterize the hand poses. Then we design a pose-driven deformation field to establish correspondence from those different poses to a shared canonical space, where a pose-disentangled NeRF for one hand is optimized. Such unified modeling efficiently complements the geometry and texture cues in rarely-observed areas for both hands. Meanwhile, we further leverage the pose priors to generate pseudo depth maps as guidance for occlusion-aware density learning. Moreover, a neural feature distillation method is proposed to achieve cross-domain alignment for color optimization. We conduct extensive experiments to verify the merits of our proposed HandNeRF and report a series of state-of-the-art results both qualitatively and quantitatively on the large-scale InterHand2.6M dataset.

cs.CV↗

arXiv2023

Amos-SLAM: An Anti-Dynamics Two-stage SLAM Approach

The traditional Simultaneous Localization And Mapping (SLAM) systems rely on the assumption of a static environment and fail to accurately estimate the system's location when dynamic objects are present in the background. While learning-based dynamic SLAM systems have difficulties in handling unknown moving objects, geometry-based methods have limited success in addressing the residual effects of unidentified dynamic objects on location estimation. To address these issues, we propose an anti-dynamics two-stage SLAM approach. Firstly, the potential motion regions of both prior and non-prior dynamic objects are extracted and pose estimates for dynamic discrimination are quickly obtained using optical flow tracking and model generation methods. Secondly, dynamic points in each frame are removed through dynamic judgment. For non-prior dynamic objects, we present a approach that uses super-pixel extraction and geometric clustering to determine the potential motion regions based on color and geometric information in the image. Evaluations on multiple low and high dynamic sequences in a public RGB-D dataset show that our proposed method outperforms state-of-the-art dynamic SLAM methods.

cs.RO↗

arXiv2023

An Adaptive Plug-and-Play Network for Few-Shot Learning

Few-shot learning (FSL) requires a model to classify new samples after learning from only a few samples. While remarkable results are achieved in existing methods, the performance of embedding and metrics determines the upper limit of classification accuracy in FSL. The bottleneck is that deep networks and complex metrics tend to induce overfitting in FSL, making it difficult to further improve the performance. Towards this, we propose plug-and-play model-adaptive resizer (MAR) and adaptive similarity metric (ASM) without any other losses. MAR retains high-resolution details to alleviate the overfitting problem caused by data scarcity, and ASM decouples the relationship between different metrics and then fuses them into an advanced one. Extensive experiments show that the proposed method could boost existing methods on two standard dataset and a fine-grained datasets, and achieve state-of-the-art results on mini-ImageNet and tiered-ImageNet.

cs.CV↗

arXiv2023

Probing QCD critical point and induced gravitational wave by black hole physics

Locating the critical endpoint of QCD and the region of a first-order phase transition at finite baryon chemical potential is an active research area for QCD matter. We provide a gravitational dual description of QCD matter at finite baryon chemical potential and finite temperature using the non-perturbative approach from gauge/gravity duality. After fixing all model parameters using state-of-the-art lattice QCD data at zero chemical potential, the predicted equations of state and QCD trace anomaly relation are in quantitative agreement with the latest lattice results. We then give the exact location of the critical endpoint as well as the first-order transition line, which is within the coverage of many upcoming experimental measurements. Moreover, using the data from our model at finite baryon chemical potential, we calculate the spectrum of the stochastic gravitational wave background associated with the first-order QCD transition in the early universe, which could be observable via pulsar timing in the future.

hep-th↗

arXiv2023

Unconditional and robust quantum metrological advantage beyond NOON states

Quantum metrology employs quantum resources to enhance the measurement sensitivity beyond that can be achieved classically. While multi-photon entangled NOON states can in principle beat the shot-noise limit and reach the Heisenberg limit, high NOON states are difficult to prepare and fragile to photon loss which hinders it from reaching unconditional quantum metrological advantages. Here, we combine the idea of unconventional nonlinear interferometers and stimulated emission of squeezed light, previously developed for photonic quantum computer Jiuzhang, to propose and realize a new scheme that achieves a scalable, unconditional, and robust quantum metrological advantage. We observe a 5.8(1)-fold enhancement above the shot-noise limit in the Fisher information extracted per photon, without discounting for photon loss and imperfections, which outperforms ideal 5-NOON states. The Heisenberg-limited scaling, the robustness to external photon loss, and the ease-to-use of our method make it applicable in practical quantum metrology at low photon flux regime.

quant-ph↗

arXiv2023

Denoising and Prompt-Tuning for Multi-Behavior Recommendation

In practical recommendation scenarios, users often interact with items under multi-typed behaviors (e.g., click, add-to-cart, and purchase). Traditional collaborative filtering techniques typically assume that users only have a single type of behavior with items, making it insufficient to utilize complex collaborative signals to learn informative representations and infer actual user preferences. Consequently, some pioneer studies explore modeling multi-behavior heterogeneity to learn better representations and boost the performance of recommendations for a target behavior. However, a large number of auxiliary behaviors (i.e., click and add-to-cart) could introduce irrelevant information to recommenders, which could mislead the target behavior (i.e., purchase) recommendation, rendering two critical challenges: (i) denoising auxiliary behaviors and (ii) bridging the semantic gap between auxiliary and target behaviors. Motivated by the above observation, we propose a novel framework-Denoising and Prompt-Tuning (DPT) with a three-stage learning paradigm to solve the aforementioned challenges. In particular, DPT is equipped with a pattern-enhanced graph encoder in the first stage to learn complex patterns as prior knowledge in a data-driven manner to guide learning informative representation and pinpointing reliable noise for subsequent stages. Accordingly, we adopt different lightweight tuning approaches with effectiveness and efficiency in the following stages to further attenuate the influence of noise and alleviate the semantic gap among multi-typed behaviors. Extensive experiments on two real-world datasets demonstrate the superiority of DPT over a wide range of state-of-the-art methods. The implementation code is available online at https://github.com/zc-97/DPT.

cs.IR↗

arXiv2023

An inverse problem for the fractional porous medium equation

We consider a time-independent variable coefficients fractional porous medium equation and formulate an associated inverse problem. We determine both the conductivity and the absorption coefficient from exterior partial measurements of the Dirichlet-to-Neumann map. Our approach relies on a time-integral transform technique as well as the unique continuation property of the fractional operator.

math.AP↗

arXiv2023

Temporal Context Mining for Learned Video Compression

We address end-to-end learned video compression with a special focus on better learning and utilizing temporal contexts. For temporal context mining, we propose to store not only the previously reconstructed frames, but also the propagated features into the generalized decoded picture buffer. From the stored propagated features, we propose to learn multi-scale temporal contexts, and re-fill the learned temporal contexts into the modules of our compression scheme, including the contextual encoder-decoder, the frame generator, and the temporal context encoder. Our scheme discards the parallelization-unfriendly auto-regressive entropy model to pursue a more practical decoding time. We compare our scheme with x264 and x265 (representing industrial software for H.264 and H.265, respectively) as well as the official reference software for H.264, H.265, and H.266 (JM, HM, and VTM, respectively). When intra period is 32 and oriented to PSNR, our scheme outperforms H.265--HM by 14.4% bit rate saving; when oriented to MS-SSIM, our scheme outperforms H.266--VTM by 21.1% bit rate saving.

cs.CV↗

arXiv2023

Phase diagram of holographic thermal dense QCD matter with rotation

We study the rotation effects of the hot and dense QCD matter in a non-perturbative regime by the gauge/gravity duality. We use the gravitational model that is designated to match the state-of-the-art lattice data on the thermal properties of (2+1)-flavor QCD and predict the location of the critical endpoint and the first-order phase transition line at large baryon chemical potential without rotation. After introducing the angular velocity via a local Lorentz boost, we investigate the thermodynamic quantities for the system under rotation in a self-consistent way. We find that the critical temperature and baryon chemical potential associated with the QCD phase transition decrease as the angular velocity increases. Moreover, some interesting phenomena are observed near the critical endpoint. We then construct the 3-dimensional phase diagram of the QCD matter in terms of temperature, baryon chemical potential, and angular velocity. As a parallel investigation, we also consider the gravitational model of $SU(3)$ pure gluon system, for which the 2-dimensional phase diagram associated with temperature and angular velocity has been predicted. The corresponding thermodynamic quantities with rotation are investigated.

hep-ph↗

arXiv2022

Bring Your Own View: Graph Neural Networks for Link Prediction with Personalized Subgraph Selection

Graph neural networks (GNNs) have received remarkable success in link prediction (GNNLP) tasks. Existing efforts first predefine the subgraph for the whole dataset and then apply GNNs to encode edge representations by leveraging the neighborhood structure induced by the fixed subgraph. The prominence of GNNLP methods significantly relies on the adhoc subgraph. Since node connectivity in real-world graphs is complex, one shared subgraph is limited for all edges. Thus, the choices of subgraphs should be personalized to different edges. However, performing personalized subgraph selection is nontrivial since the potential selection space grows exponentially to the scale of edges. Besides, the inference edges are not available during training in link prediction scenarios, so the selection process needs to be inductive. To bridge the gap, we introduce a Personalized Subgraph Selector (PS2) as a plug-and-play framework to automatically, personally, and inductively identify optimal subgraphs for different edges when performing GNNLP. PS2 is instantiated as a bi-level optimization problem that can be efficiently solved differently. Coupling GNNLP models with PS2, we suggest a brand-new angle towards GNNLP training: by first identifying the optimal subgraphs for edges; and then focusing on training the inference model by using the sampled subgraphs. Comprehensive experiments endorse the effectiveness of our proposed method across various GNNLP backbones (GCN, GraphSage, NGCF, LightGCN, and SEAL) and diverse benchmarks (Planetoid, OGB, and Recommendation datasets). Our code is publicly available at \url{https://github.com/qiaoyu-tan/PS2}

cs.IR↗

arXiv2022

Thermoelectric Transport in Holographic Quantum Matter under Shear Strain

We study the thermoelectric transport under shear strain in two spatial dimensional quantum matter using the holographic duality. General analytic formulae for the DC thermoelectric conductivities subjected to finite shear strain are obtained in terms of the black hole horizon data. Off-diagonal terms in the conductivity matrix appear also at zero magnetic field, resembling an emergent electronic nematicity which cannot nevertheless be identified with the presence of an anomalous Hall effect. For an explicit model study, we numerically construct a family of strained black holes and obtain the corresponding nonlinear stress-strain curves. We then compute all electric, thermoelectric, and thermal conductivities and discuss the effects of strain. While the shear elastic deformation does not affect the temperature dependence of thermoelectric and thermal conductivities quantitatively, it can strongly change the behavior of the electric conductivity. For both shear hardening and softening cases, we find a clear metal-insulator transition driven by the shear deformation. Moreover, the violation of the previously conjectured thermal conductivity bound is observed for large shear deformation.

hep-th↗

arXiv2022

Observation of size-dependent boundary effects in non-Hermitian electric circuits

The non-Hermitian systems with the non-Hermitian skin effect (NHSE) are very sensitive to the imposed boundary conditions and lattice size, which leads to size-dependent non-Hermitian skin effects. Here, we report the experimental observation of NHSE with different boundary conditions and different lattice size in a unidirectional hopping model based on a circuit platform. The circuit admittance spectra and corresponding eigenstates are very sensitive to the presence of the boundary. Meanwhile, our experimental results show how the lattice size and boundary terms together affect the strength of NHSE. Therefore, our electric circuit provides a good platform to observe size-dependent boundary effects in non-Hermitian systems.

quant-ph↗

arXiv2022

Distributed Policy Gradient with Variance Reduction in Multi-Agent Reinforcement Learning

This paper studies a distributed policy gradient in collaborative multi-agent reinforcement learning (MARL), where agents over a communication network aim to find the optimal policy to maximize the average of all agents' local returns. Due to the non-concave performance function of policy gradient, the existing distributed stochastic optimization methods for convex problems cannot be directly used for policy gradient in MARL. This paper proposes a distributed policy gradient with variance reduction and gradient tracking to address the high variances of policy gradient, and utilizes importance weight to solve the {distribution shift} problem in the sampling process. We then provide an upper bound on the mean-squared stationary gap, which depends on the number of iterations, the mini-batch size, the epoch size, the problem parameters, and the network topology. We further establish the sample and communication complexity to obtain an $ε$-approximate stationary point. Numerical experiments are performed to validate the effectiveness of the proposed algorithm.

cs.MA↗