arXiv subjects

Chen Zhong

Publications and source records attributed to Chen Zhong.

At least 37 records · Page 2Linked to original sources

arXiv2024

An End-to-End Reinforcement Learning Based Approach for Micro-View Order-Dispatching in Ride-Hailing

Assigning orders to drivers under localized spatiotemporal context (micro-view order-dispatching) is a major task in Didi, as it influences ride-hailing service experience. Existing industrial solutions mainly follow a two-stage pattern that incorporate heuristic or learning-based algorithms with naive combinatorial methods, tackling the uncertainty of both sides' behaviors, including emerging timings, spatial relationships, and travel duration, etc. In this paper, we propose a one-stage end-to-end reinforcement learning based order-dispatching approach that solves behavior prediction and combinatorial optimization uniformly in a sequential decision-making manner. Specifically, we employ a two-layer Markov Decision Process framework to model this problem, and present \underline{D}eep \underline{D}ouble \underline{S}calable \underline{N}etwork (D2SN), an encoder-decoder structure network to generate order-driver assignments directly and stop assignments accordingly. Besides, by leveraging contextual dynamics, our approach can adapt to the behavioral patterns for better performance. Extensive experiments on Didi's real-world benchmarks justify that the proposed approach significantly outperforms competitive baselines in optimizing matching efficiency and user experience tasks. In addition, we evaluate the deployment outline and discuss the gains and experiences obtained during the deployment tests from the view of large-scale engineering implementation.

cs.LG↗

arXiv2024

Large Language Models Powered Context-aware Motion Prediction in Autonomous Driving

Motion prediction is among the most fundamental tasks in autonomous driving. Traditional methods of motion forecasting primarily encode vector information of maps and historical trajectory data of traffic participants, lacking a comprehensive understanding of overall traffic semantics, which in turn affects the performance of prediction tasks. In this paper, we utilized Large Language Models (LLMs) to enhance the global traffic context understanding for motion prediction tasks. We first conducted systematic prompt engineering, visualizing complex traffic environments and historical trajectory information of traffic participants into image prompts -- Transportation Context Map (TC-Map), accompanied by corresponding text prompts. Through this approach, we obtained rich traffic context information from the LLM. By integrating this information into the motion prediction model, we demonstrate that such context can enhance the accuracy of motion predictions. Furthermore, considering the cost associated with LLMs, we propose a cost-effective deployment strategy: enhancing the accuracy of motion prediction tasks at scale with 0.7\% LLM-augmented datasets. Our research offers valuable insights into enhancing the understanding of traffic scenes of LLMs and the motion prediction performance of autonomous driving. The source code is available at \url{https://github.com/AIR-DISCOVER/LLM-Augmented-MTR} and \url{https://aistudio.baidu.com/projectdetail/7809548}.

cs.CV↗

arXiv2023

Anomaly Detection via Learning-Based Sequential Controlled Sensing

In this paper, we address the problem of detecting anomalies among a given set of binary processes via learning-based controlled sensing. Each process is parameterized by a binary random variable indicating whether the process is anomalous. To identify the anomalies, the decision-making agent is allowed to observe a subset of the processes at each time instant. Also, probing each process has an associated cost. Our objective is to design a sequential selection policy that dynamically determines which processes to observe at each time with the goal to minimize the delay in making the decision and the total sensing cost. We cast this problem as a sequential hypothesis testing problem within the framework of Markov decision processes. This formulation utilizes both a Bayesian log-likelihood ratio-based reward and an entropy-based reward. The problem is then solved using two approaches: 1) a deep reinforcement learning-based approach where we design both deep Q-learning and policy gradient actor-critic algorithms; and 2) a deep active inference-based approach. Using numerical experiments, we demonstrate the efficacy of our algorithms and show that our algorithms adapt to any unknown statistical dependence pattern of the processes.

cs.LG↗

arXiv2023

The temporal concentration of travel demand in an urban transport network

Suppose $A$ and $B$ are two stations within the mass rapid transit network of a city. Both stations see approximately the same average daily number of passengers entering and exiting their gates. However, passengers are evenly distributed at $A$, whereas activity is concentrated mainly during peak hours at $B$. Although the daily travel demand is the same for both stations, $B$ requires more resources since the number of vehicles, station dimensions and staffing level must be tailored to meet the demands of peak hours. This hypothetical scenario underscores the need to quantify the concentration of travel demand for optimising resource allocation and planning efficiency in an urban transport network. To this end, we introduce a novel metric for assessing the temporal concentration of travel demand at different locations in a generic transport network. Our approach is validated using granular data sourced from smart travel cards, encompassing 272 London Underground (LU) stations. Additionally, we present a methodological framework based on Random Forests to identify attributes of the locations of interest within the transport network that contribute to varying levels of temporal concentration of travel demand. Our case study unveils that LU stations located in areas characterised by low residential, retail, and employment density, predominantly situated in outer London, exhibit the most pronounced temporal concentration of travel demand. Conversely, within inner London, stations servicing high-density employment zones, especially around the City of London, experience a greater temporal concentration of travel demand compared to those catering to commercial and residential districts, typically situated in West London.

physics.soc-ph↗

arXiv2022

Inferring urban polycentricity from the variability in human mobility patterns

The polycentric city model has gained popularity in spatial planning policy, since it is believed to overcome some of the problems often present in monocentric metropolises, ranging from congestion to difficult accessibility to jobs and services. However, the concept 'polycentric city' has a fuzzy definition and as a result, the extent to which a city is polycentric cannot be easily determined. Here, we leverage the fine spatio-temporal resolution of smart travel card data to infer urban polycentricity by examining how a city departs from a well-defined monocentric model. In particular, we analyse the human movements that arise as a result of sophisticated forms of urban structure by introducing a novel probabilistic approach which captures the complexity of these human movements. We focus on London (UK) and Seoul (South Korea) as our two case studies, and we specifically find evidence that London displays a higher degree of monocentricity than Seoul, suggesting that Seoul is likely to be more polycentric than London.

physics.soc-ph↗

arXiv2022

Performance Optimization and Parameters Estimation for MIMO-OFDM Dual-functional Communication-radar Systems

In dual-functional communication-radar systems, common radio frequency (RF) signals are used for both communication and detection. For better compatibility with existing communication systems, we adopt multiple-input multiple-output (MIMO) orthogonal frequency division multiplexing (OFDM) signals as integrated signals and investigate the estimation performance of MIMO-OFDM signals. We first analyze the Cramer-Rao lower bound (CRLB) of parameters estimation. Then, transmit powers over different subcarriers are optimized to achieve the best tradeoff between transmission rate and estimation performance. Finally, we propose a more accurate estimation method which utilizes canonical polyadic decomposition (CPD) of three-order tensor to obtain the parameter matrices. Due to the characteristic of the column structure of the parameter matrices, we just need to use DFT / IDFT to recover the parameters of multiple targets. The simulation results show that the estimation method based on tensor can achieve performance close to CRLB and the estimation performance can be improved by optimizing the transmit powers.

cs.IT↗

arXiv2021

Dynamic Channel Access and Power Control in Wireless Interference Networks via Multi-Agent Deep Reinforcement Learning

Due to the scarcity in the wireless spectrum and limited energy resources especially in mobile applications, efficient resource allocation strategies are critical in wireless networks. Motivated by the recent advances in deep reinforcement learning (DRL), we address multi-agent DRL-based joint dynamic channel access and power control in a wireless interference network. We first propose a multi-agent DRL algorithm with centralized training (DRL-CT) to tackle the joint resource allocation problem. In this case, the training is performed at the central unit (CU) and after training, the users make autonomous decisions on their transmission strategies with only local information. We demonstrate that with limited information exchange and faster convergence, DRL-CT algorithm can achieve 90% of the performance achieved by the combination of weighted minimum mean square error (WMMSE) algorithm for power control and exhaustive search for dynamic channel access. In the second part of this paper, we consider distributed multi-agent DRL scenario in which each user conducts its own training and makes its decisions individually, acting as a DRL agent. Finally, as a compromise between centralized and fully distributed scenarios, we consider federated DRL (FDRL) to approach the performance of DRL-CT with the use of a central unit in training while limiting the information exchange and preserving privacy of the users in the wireless system. Via simulation results, we show that proposed learning frameworks lead to efficient adaptive channel access and power control policies in dynamic environments.

cs.IT↗

arXiv2021

A Predictive Online Transient Stability Assessment with Hierarchical Generative Adversarial Networks

Online transient stability assessment (TSA) is essential for secure and stable power system operations. The growing number of Phasor Measurement Units (PMUs) brings about massive sources of data that can enhance online TSA. However, conventional data-driven methods require large amounts of transient data to correctly assess the transient stability state of a system. In this paper, a new data-driven TSA approach is developed for TSA with fewer data compared to the conventional methods. The data reduction is enabled by learning the dynamic behaviors of the historical transient data using generative and adversarial networks (GAN). This knowledge is used online to predict the voltage time series data after a transient event. A classifier embedded in the generative network deploys the predicted post-contingency data to determine the stability of the system following a fault. The developed GAN-based TSA approach preserves the spatial and temporal correlations that exist in multivariate PMU time series data. Hence, in comparison with the state-of-the-art TSA methods, it achieves a higher assessment accuracy using only one sample of the measured data and a shorter response time. Case studies conducted on the IEEE 118-bus system demonstrate the superior performance of the GAN-based method compared to the conventional data-driven techniques.

eess.SY↗

arXiv2021

Scalable and Decentralized Algorithms for Anomaly Detection via Learning-Based Controlled Sensing

We address the problem of sequentially selecting and observing processes from a given set to find the anomalies among them. The decision-maker observes a subset of the processes at any given time instant and obtains a noisy binary indicator of whether or not the corresponding process is anomalous. In this setting, we develop an anomaly detection algorithm that chooses the processes to be observed at a given time instant, decides when to stop taking observations, and declares the decision on anomalous processes. The objective of the detection algorithm is to identify the anomalies with an accuracy exceeding the desired value while minimizing the delay in decision making. We devise a centralized algorithm where the processes are jointly selected by a common agent as well as a decentralized algorithm where the decision of whether to select a process is made independently for each process. Our algorithms rely on a Markov decision process defined using the marginal probability of each process being normal or anomalous, conditioned on the observations. We implement the detection algorithms using the deep actor-critic reinforcement learning framework. Unlike prior work on this topic that has exponential complexity in the number of processes, our algorithms have computational and memory requirements that are both polynomial in the number of processes. We demonstrate the efficacy of these algorithms using numerical experiments by comparing them with state-of-the-art methods.

cs.LG↗

arXiv2021

Anomaly Detection via Controlled Sensing and Deep Active Inference

In this paper, we address the anomaly detection problem where the objective is to find the anomalous processes among a given set of processes. To this end, the decision-making agent probes a subset of processes at every time instant and obtains a potentially erroneous estimate of the binary variable which indicates whether or not the corresponding process is anomalous. The agent continues to probe the processes until it obtains a sufficient number of measurements to reliably identify the anomalous processes. In this context, we develop a sequential selection algorithm that decides which processes to be probed at every instant to detect the anomalies with an accuracy exceeding a desired value while minimizing the delay in making the decision and the total number of measurements taken. Our algorithm is based on active inference which is a general framework to make sequential decisions in order to maximize the notion of free energy. We define the free energy using the objectives of the selection policy and implement the active inference framework using a deep neural network approximation. Using numerical experiments, we compare our algorithm with the state-of-the-art method based on deep actor-critic reinforcement learning and demonstrate the superior performance of our algorithm.

cs.LG↗

arXiv2020

Anomaly Detection and Sampling Cost Control via Hierarchical GANs

Anomaly detection incurs certain sampling and sensing costs and therefore it is of great importance to strike a balance between the detection accuracy and these costs. In this work, we study anomaly detection by considering the detection of threshold crossings in a stochastic time series without the knowledge of its statistics. To reduce the sampling cost in this detection process, we propose the use of hierarchical generative adversarial networks (GANs) to perform nonuniform sampling. In order to improve the detection accuracy and reduce the delay in detection, we introduce a buffer zone in the operation of the proposed GAN-based detector. In the experiments, we analyze the performance of the proposed hierarchical GAN detector considering the metrics of detection delay, miss rates, average cost of error, and sampling ratio. We identify the tradeoffs in the performance as the buffer zone sizes and the number of GAN levels in the hierarchy vary. We also compare the performance with that of a sampling policy that approximately minimizes the sum of average costs of sampling and error given the parameters of the stochastic process. We demonstrate that the proposed GAN-based detector can have significant performance improvements in terms of detection delay and average cost of error with a larger buffer zone but at the cost of increased sampling rates.

cs.LG↗

arXiv2020

Adversarial jamming attacks and defense strategies via adaptive deep reinforcement learning

As the applications of deep reinforcement learning (DRL) in wireless communications grow, sensitivity of DRL based wireless communication strategies against adversarial attacks has started to draw increasing attention. In order to address such sensitivity and alleviate the resulting security concerns, we in this paper consider a victim user that performs DRL-based dynamic channel access, and an attacker that executes DRLbased jamming attacks to disrupt the victim. Hence, both the victim and attacker are DRL agents and can interact with each other, retrain their models, and adapt to opponents' policies. In this setting, we initially develop an adversarial jamming attack policy that aims at minimizing the accuracy of victim's decision making on dynamic channel access. Subsequently, we devise defense strategies against such an attacker, and propose three defense strategies, namely diversified defense with proportional-integral-derivative (PID) control, diversified defense with an imitation attacker, and defense via orthogonal policies. We design these strategies to maximize the attacked victim's accuracy and evaluate their performances.

eess.SP↗

arXiv2020

Clustering structure effect on Hanbury-Brown-Twiss correlation in $^{12}$C + $^{197}$Au collisions at 200 GeV}

Through $^{12}$C + $^{197}$Au collisions at $\sqrt{s_{NN}} =$ 200 GeV using a multiphase transport (AMPT) model, the azimuthal angle dependences of the Hanbury Brown-Twiss (HBT) radii relative to the second- and third-order participant plane from $π$-$π$ correlations are discussed. Three initial geometric configurations of $^{12}$C, namely three-$α$-cluster triangle, three-$α$-cluster chain and Woods-Saxon distribution of nucleons, are taken into account, and their effects on the correlations are investigated. The ratio of the third- to the second-order HBT radii $R_{o(s),3}^2/R_{o(s),2}^2$ is shown to be a clear probe for three configurations. In addition, this work presents the hadronic rescattering time evolution of the azimuthally dependent HBT radii. From the present study, one can learn that the HBT correlation from identical particles at freeze-out is able to provide the information of different initial configurations as collective flow proposed before.

nucl-th↗

arXiv2019

Charge asymmetry dependence of flow and a novel correlator to detect the chiral magnetic wave in a multiphase transport model

In a multiphase transport model with the initial electric quadrupole moment, we studied and discussed the charge asymmetry ($A_{\rm ch}$) dependence of flow at varied kinematic windows in semi-central Au+Au collisions at $\sqrt{s_{\rm NN}}$ = 200 GeV. We then proposed a novel correlator $W$ which specially focuses on the difference of elliptic flow between positively and negatively charged hadrons induced by the chiral magnetic wave and, more importantly, is irrelevant to the ambiguous $A_{\rm ch}$. We found that the distribution of the second order correlator $W_2$ displays a convex structure in the absence of the quadrupole and a concave shape in the presence of the quadrupole. We also studied the response of $W_n$ for both signal and resonance background in a toy model and in analytical calculation. Such a method provides a new way to detect the chiral magnetic wave.

hep-ph↗

arXiv2019

Electromagnetic field from asymmetric to symmetric heavy ion collision at 200 GeV/c

Electromagnetic fields produced in relativistic heavy-ion collisions are affected by the asymmetry of the projectile-target combination as well as the different initial configurations of the nucleus. In this study, the results of the electric and magnetic fields produced for different combinations of ions, namely $^{12}$C + $^{197}$Au, $^{24}$Mg + $^{197}$Au, $^{64}$Cu + $^{197}$Au, and $^{197}$Au + $^{197}$Au at $\sqrt{s_{NN}} = 200$ GeV are demonstrated with a multi-phase transport model (AMPT). The configuration of the distribution of nucleons of $^{12}$C is initialized by a Woods-Saxon spherical structure, a three-$α$-clustering triangular structure or a three-$α$-clustering chain structure. It was observed that the electric and magnetic fields display different behavioral patterns for asymmetric combinations of the projectile and target nuclei as well as for different initial configurations of the carbon nucleus. The major features of the process are discussed.

nucl-th↗

arXiv2019

Deep Actor-Critic Reinforcement Learning for Anomaly Detection

Anomaly detection is widely applied in a variety of domains, involving for instance, smart home systems, network traffic monitoring, IoT applications and sensor networks. In this paper, we study deep reinforcement learning based active sequential testing for anomaly detection. We assume that there is an unknown number of abnormal processes at a time and the agent can only check with one sensor in each sampling step. To maximize the confidence level of the decision and minimize the stopping time concurrently, we propose a deep actor-critic reinforcement learning framework that can dynamically select the sensor based on the posterior probabilities. We provide simulation results for both the training phase and testing phase, and compare the proposed framework with the Chernoff test in terms of claim delay and loss.

cs.LG↗

arXiv2019

A Deep Actor-Critic Reinforcement Learning Framework for Dynamic Multichannel Access

To make efficient use of limited spectral resources, we in this work propose a deep actor-critic reinforcement learning based framework for dynamic multichannel access. We consider both a single-user case and a scenario in which multiple users attempt to access channels simultaneously. We employ the proposed framework as a single agent in the single-user case, and extend it to a decentralized multi-agent framework in the multi-user scenario. In both cases, we develop algorithms for the actor-critic deep reinforcement learning and evaluate the proposed learning policies via experiments and numerical results. In the single-user model, in order to evaluate the performance of the proposed channel access policy and the framework's tolerance against uncertainty, we explore different channel switching patterns and different switching probabilities. In the case of multiple users, we analyze the probabilities of each user accessing channels with favorable channel conditions and the probability of collision. We also address a time-varying environment to identify the adaptive ability of the proposed framework. Additionally, we provide comparisons (in terms of both the average reward and time efficiency) between the proposed actor-critic deep reinforcement learning framework, Deep-Q network (DQN) based approach, random access, and the optimal policy when the channel dynamics are known.

cs.LG↗

arXiv2019

Deep Multi-Agent Reinforcement Learning Based Cooperative Edge Caching in Wireless Networks

The growing demand on high-quality and low-latency multimedia services has led to much interest in edge caching techniques. Motivated by this, we in this paper consider edge caching at the base stations with unknown content popularity distributions. To solve the dynamic control problem of making caching decisions, we propose a deep actor-critic reinforcement learning based multi-agent framework with the aim to minimize the overall average transmission delay. To evaluate the proposed framework, we compare the learning-based performance with three other caching policies, namely least recently used (LRU), least frequently used (LFU), and first-in-first-out (FIFO) policies. Through simulation results, performance improvements of the proposed framework over these three caching algorithms have been identified and its superior ability to adapt to varying environments is demonstrated.

cs.IT↗