Searcharxiv⌕ Search

arXiv subjects

Jian Sun

Publications and source records attributed to Jian Sun.

At least 199 records · Page 11Linked to original sources
arXiv2023

Soft Decomposed Policy-Critic: Bridging the Gap for Effective Continuous Control with Discrete RL

Discrete reinforcement learning (RL) algorithms have demonstrated exceptional performance in solving sequential decision tasks with discrete action spaces, such as Atari games. However, their effectiveness is hindered when applied to continuous control problems due to the challenge of dimensional explosion. In this paper, we present the Soft Decomposed Policy-Critic (SDPC) architecture, which combines soft RL and actor-critic techniques with discrete RL methods to overcome this limitation. SDPC discretizes each action dimension independently and employs a shared critic network to maximize the soft $Q$-function. This novel approach enables SDPC to support two types of policies: decomposed actors that lead to the Soft Decomposed Actor-Critic (SDAC) algorithm, and decomposed $Q$-networks that generate Boltzmann soft exploration policies, resulting in the Soft Decomposed-Critic Q (SDCQ) algorithm. Through extensive experiments, we demonstrate that our proposed approach outperforms state-of-the-art continuous RL algorithms in a variety of continuous control tasks, including Mujoco's Humanoid and Box2d's BipedalWalker. These empirical results validate the effectiveness of the SDPC architecture in addressing the challenges associated with continuous control.

cs.LG↗
arXiv2023

MTD-GPT: A Multi-Task Decision-Making GPT Model for Autonomous Driving at Unsignalized Intersections

Autonomous driving technology is poised to transform transportation systems. However, achieving safe and accurate multi-task decision-making in complex scenarios, such as unsignalized intersections, remains a challenge for autonomous vehicles. This paper presents a novel approach to this issue with the development of a Multi-Task Decision-Making Generative Pre-trained Transformer (MTD-GPT) model. Leveraging the inherent strengths of reinforcement learning (RL) and the sophisticated sequence modeling capabilities of the Generative Pre-trained Transformer (GPT), the MTD-GPT model is designed to simultaneously manage multiple driving tasks, such as left turns, straight-ahead driving, and right turns at unsignalized intersections. We initially train a single-task RL expert model, sample expert data in the environment, and subsequently utilize a mixed multi-task dataset for offline GPT training. This approach abstracts the multi-task decision-making problem in autonomous driving as a sequence modeling task. The MTD-GPT model is trained and evaluated across several decision-making tasks, demonstrating performance that is either superior or comparable to that of state-of-the-art single-task decision-making models.

cs.RO↗
arXiv2023

Data-driven Polytopic Output Synchronization of Heterogeneous Multi-agent Systems from Noisy Data

This paper proposes a novel approach to addressing the output synchronization problem in unknown heterogeneous multi-agent systems (MASs) using noisy data. Unlike existing studies that focus on noiseless data, we introduce a distributed data-driven controller that enables all heterogeneous followers to synchronize with a leader's trajectory. To handle the noise in the state-input-output data, we develop a data-based polytopic representation for the MAS. We tackle the issue of infeasibility in the set of output regulator equations caused by the noise by seeking approximate solutions via constrained fitting error minimization. This method utilizes measured data and a noise-matrix polytope to ensure near-optimal output synchronization. Stability conditions in the form of data-dependent semidefinite programs are derived, providing stabilizing controller gains for each follower. The proposed distributed data-driven control protocol achieves near-optimal output synchronization by ensuring the convergence of the tracking error to a bounded polytope, with the polytope size positively correlated with the noise bound. Numerical tests validate the practical merits of the proposed data-driven design and theory.

eess.SY↗
arXiv2023

A Unified Distributed Method for Constrained Networked Optimization via Saddle-Point Dynamics

This paper develops a unified distributed method for solving two classes of constrained networked optimization problems, i.e., optimal consensus problem and resource allocation problem with non-identical set constraints. We first transform these two constrained networked optimization problems into a unified saddle-point problem framework with set constraints. Subsequently, two projection-based primal-dual algorithms via Optimistic Gradient Descent Ascent (OGDA) method and Extra-gradient (EG) method are developed for solving constrained saddle-point problems. It is shown that the developed algorithms achieve exact convergence to a saddle point with an ergodic convergence rate $O(1/k)$ for general convex-concave functions. Based on the proposed primal-dual algorithms via saddle-point dynamics, we develop unified distributed algorithm design and convergence analysis for these two networked optimization problems. Finally, two numerical examples are presented to demonstrate the theoretical results.

math.OC↗
arXiv2023

Flexible Job Shop Scheduling via Dual Attention Network Based Reinforcement Learning

Flexible manufacturing has given rise to complex scheduling problems such as the flexible job shop scheduling problem (FJSP). In FJSP, operations can be processed on multiple machines, leading to intricate relationships between operations and machines. Recent works have employed deep reinforcement learning (DRL) to learn priority dispatching rules (PDRs) for solving FJSP. However, the quality of solutions still has room for improvement relative to that by the exact methods such as OR-Tools. To address this issue, this paper presents a novel end-to-end learning framework that weds the merits of self-attention models for deep feature extraction and DRL for scalable decision-making. The complex relationships between operations and machines are represented precisely and concisely, for which a dual-attention network (DAN) comprising several interconnected operation message attention blocks and machine message attention blocks is proposed. The DAN exploits the complicated relationships to construct production-adaptive operation and machine features to support high-quality decisionmaking. Experimental results using synthetic data as well as public benchmarks corroborate that the proposed approach outperforms both traditional PDRs and the state-of-the-art DRL method. Moreover, it achieves results comparable to exact methods in certain cases and demonstrates favorable generalization ability to large-scale and real-world unseen FJSP tasks.

cs.LG↗
arXiv2023

A Macro-Micro Approach to Reconstructing Vehicle Trajectories on Multi-Lane Freeways with Lane Changing

Vehicle trajectories can offer the most precise and detailed depiction of traffic flow and serve as a critical component in traffic management and control applications. Various technologies have been applied to reconstruct vehicle trajectories from sparse fixed and mobile detection data. However, existing methods predominantly concentrate on single-lane scenarios and neglect lane-changing (LC) behaviors that occur across multiple lanes, which limit their applicability in practical traffic systems. To address this research gap, we propose a macro-micro approach for reconstructing complete vehicle trajectories on multi-lane freeways, wherein the macro traffic state information and micro driving models are integrated to overcome the restrictions imposed by lane boundary. Particularly, the macroscopic velocity contour maps are established for each lane to regulate the movement of vehicle platoons, meanwhile the velocity difference between adjacent lanes provide valuable criteria for guiding LC behaviors. Simultaneously, the car-following models are extended from micro perspective to supply lane-based candidate trajectories and define the plausible range for LC positions. Later, a two-stage trajectory fusion algorithm is proposed to jointly infer both the car-following and LC behaviors, in which the optimal LC positions is identified and candidate trajectories are adjusted according to their weights. The proposed framework was evaluated using NGSIM dataset, and the results indicated a remarkable enhancement in both the accuracy and smoothness of reconstructed trajectories, with performance indicators reduced by over 30% compared to two representative reconstruction methods. Furthermore, the reconstruction process effectively reproduced LC behaviors across contiguous lanes, adding to the framework's comprehensiveness and realism.

math.OC↗
arXiv2023

Correlating sparse sensing for large-scale traffic speed estimation: A Laplacian-enhanced low-rank tensor kriging approach

Traffic speed is central to characterizing the fluidity of the road network. Many transportation applications rely on it, such as real-time navigation, dynamic route planning, and congestion management. Rapid advances in sensing and communication techniques make traffic speed detection easier than ever. However, due to sparse deployment of static sensors or low penetration of mobile sensors, speeds detected are incomplete and far from network-wide use. In addition, sensors are prone to error or missing data due to various kinds of reasons, speeds from these sensors can become highly noisy. These drawbacks call for effective techniques to recover credible estimates from the incomplete data. In this work, we first identify the issue as a spatiotemporal kriging problem and propose a Laplacian enhanced low-rank tensor completion (LETC) framework featuring both lowrankness and multi-dimensional correlations for large-scale traffic speed kriging under limited observations. To be specific, three types of speed correlation including temporal continuity, temporal periodicity, and spatial proximity are carefully chosen and simultaneously modeled by three different forms of graph Laplacian, named temporal graph Fourier transform, generalized temporal consistency regularization, and diffusion graph regularization. We then design an efficient solution algorithm via several effective numeric techniques to scale up the proposed model to network-wide kriging. By performing experiments on two public million-level traffic speed datasets, we finally draw the conclusion and find our proposed LETC achieves the state-of-the-art kriging performance even under low observation rates, while at the same time saving more than half computing time compared with baseline methods. Some insights into spatiotemporal traffic data modeling and kriging at the network level are provided as well.

stat.ML↗
arXiv2023

Cooperative Driving of Connected Autonomous Vehicles in Heterogeneous Mixed Traffic: A Game Theoretic Approach

High-density, unsignalized intersection has always been a bottleneck of efficiency and safety. The emergence of Connected Autonomous Vehicles (CAVs) results in a mixed traffic condition, further increasing the complexity of the transportation system. Against this background, this paper aims to study the intricate and heterogeneous interaction of vehicles and conflict resolution at the high-density, mixed, unsignalized intersection. Theoretical insights about the interaction between CAVs and Human-driven Vehicles (HVs) and the cooperation of CAVs are synthesized, based on which a novel cooperative decision-making framework in heterogeneous mixed traffic is proposed. Normalized Cooperative game is concatenated with Level-k game (NCL game) to generate a system optimal solution. Then Lattice planner generates the optimal and collision-free trajectories for CAVs. To reproduce HVs in mixed traffic, interactions from naturalistic human driving data are extracted as prior knowledge. Non-cooperative game and Inverse Reinforcement Learning (IRL) are integrated to mimic the decision making of heterogeneous HVs. Finally, three cases are conducted to verify the performance of the proposed algorithm, including the comparative analysis with different methods, the case study under different Rates of Penetration (ROP) and the interaction analysis with heterogeneous HVs. It is found that the proposed cooperative decision-making framework is beneficial to the driving conflict resolution and the traffic efficiency improvement of the mixed unsignalized intersection. Besides, due to the consideration of driving heterogeneity, better human-machine interaction and cooperation can be realized in this paper.

cs.MA↗
arXiv2023

Efficient and Robust Time-Optimal Trajectory Planning and Control for Agile Quadrotor Flight

Agile quadrotor flight relies on rapidly planning and accurately tracking time-optimal trajectories, a technology critical to their application in the wild. However, the computational burden of computing time-optimal trajectories based on the full quadrotor dynamics (typically on the order of minutes or even hours) can hinder its ability to respond quickly to changing scenarios. Additionally, modeling errors and external disturbances can lead to deviations from the desired trajectory during tracking in real time. This letter proposes a novel approach to computing time-optimal trajectories, by fixing the nodes with waypoint constraints and adopting separate sampling intervals for trajectories between waypoints, which significantly accelerates trajectory planning. Furthermore, the planned paths are tracked via a time-adaptive model predictive control scheme whose allocated tracking time can be adaptively adjusted on-the-fly, therefore enhancing the tracking accuracy and robustness. We evaluate our approach through simulations and experimentally validate its performance in dynamic waypoint scenarios for time-optimal trajectory replanning and trajectory tracking.

cs.RO↗
arXiv2023

Learning Robust Data-based LQG Controllers from Noisy Data

This paper addresses the joint state estimation and control problems for unknown linear time-invariant systems subject to both process and measurement noise. The aim is to redesign the linear quadratic Gaussian (LQG) controller based solely on data. The LQG controller comprises a linear quadratic regulator (LQR) and a steady-state Kalman observer; while the data-based LQR design problem has been previously studied, constructing the Kalman gain and the LQG controller from noisy data presents a novel challenge. In this work, a data-based formulation for computing the steady-state Kalman gain is proposed based on semi-definite programming (SDP) using some noise-free input-state-output data. Additionally, a data-based LQG controller is developed, which is shown to be equivalent to the model-based LQG controller. For cases where offline data are corrupted by noise, a robust data-based observer gain is constructed by tackling a relaxed SDP. The proposed controllers are proven to achieve robust global exponential stability (RGES) for state estimation and input-to-state practical stability (ISpS) under standard conditions. Finally, numerical tests are conducted to validate the proposed controllers' correctness and effectiveness.

eess.SY↗
arXiv2023

DBQ-SSD: Dynamic Ball Query for Efficient 3D Object Detection

Many point-based 3D detectors adopt point-feature sampling strategies to drop some points for efficient inference. These strategies are typically based on fixed and handcrafted rules, making it difficult to handle complicated scenes. Different from them, we propose a Dynamic Ball Query (DBQ) network to adaptively select a subset of input points according to the input features, and assign the feature transform with a suitable receptive field for each selected point. It can be embedded into some state-of-the-art 3D detectors and trained in an end-to-end manner, which significantly reduces the computational cost. Extensive experiments demonstrate that our method can increase the inference speed by 30%-100% on KITTI, Waymo, and ONCE datasets. Specifically, the inference speed of our detector can reach 162 FPS on KITTI scene, and 30 FPS on Waymo and ONCE scenes without performance degradation. Due to skipping the redundant points, some evaluation metrics show significant improvements. Codes will be released at https://github.com/yancie-yjr/DBQ-SSD.

cs.CV↗
arXiv2023

Causal Inference with Confounders MNAR under Treatment-independent Missingness Assumption

Causal inference in observational studies can be challenging when confounders are subject to missingness. Generally, the identification of causal effects is not guaranteed even under restrictive parametric model assumptions when confounders are missing not at random. To address this, We propose a general framework to establish the identification of causal effects when confounders are subject to treatment-independent missingness, which means that the missing data mechanism is independent of the treatment, given the outcome and possibly missing confounders. We give special consideration to commonly-used models for continuous and binary outcomes and provide counterexamples when identification fails. For estimation, we provide a weighted estimation equation estimating method for model parameters and purpose three estimators for the average causal effect based on the estimated models. We evaluate the finite-sample performance of the estimators via simulations. We further illustrate the proposed method with real data sets from the National Health and Nutrition Examination Survey.

stat.ME↗
arXiv2023

Generalized Semantic Segmentation by Self-Supervised Source Domain Projection and Multi-Level Contrastive Learning

Deep networks trained on the source domain show degraded performance when tested on unseen target domain data. To enhance the model's generalization ability, most existing domain generalization methods learn domain invariant features by suppressing domain sensitive features. Different from them, we propose a Domain Projection and Contrastive Learning (DPCL) approach for generalized semantic segmentation, which includes two modules: Self-supervised Source Domain Projection (SSDP) and Multi-level Contrastive Learning (MLCL). SSDP aims to reduce domain gap by projecting data to the source domain, while MLCL is a learning scheme to learn discriminative and generalizable features on the projected data. During test time, we first project the target data by SSDP to mitigate domain shift, then generate the segmentation results by the learned segmentation network based on MLCL. At test time, we can update the projected data by minimizing our proposed pixel-to-pixel contrastive loss to obtain better results. Extensive experiments for semantic segmentation demonstrate the favorable generalization capability of our method on benchmark datasets.

cs.CV↗
arXiv2023

Record-high-Tc elemental superconductivity in scandium

Elemental materials provide clean and fundamental platforms for studying superconductivity. However, the highest superconducting critical temperature (Tc) yet observed in elements has not exceeded 30 K. Discovering elemental superconductors with a higher Tc is one of the most fundamental and challenging tasks in condensed matter physics. In this study, by applying high pressure up to approximately 260 GPa, we demonstrate that the superconducting transition temperature of elemental scandium (Sc) can be increased to 36 K, which is a record-high Tc for superconducting elements. The pressure dependence of Tc implies the occurrence of multiple phase transitions in Sc, which is in agreement with previous X-ray diffraction results. Optimization of Tc is achieved in the Sc-V phase, which can be attributed to the strong coupling between d electrons and moderate-frequency phonons, as suggested by our first-principles calculations. This study provides insights for exploring new high-Tc elemental metals.

cond-mat.supr-con↗
arXiv2023

EqCo: Equivalent Rules for Self-supervised Contrastive Learning

In this paper, we propose EqCo (Equivalent Rules for Contrastive Learning) to make self-supervised learning irrelevant to the number of negative samples in the contrastive learning framework. Inspired by the InfoMax principle, we point that the margin term in contrastive loss needs to be adaptively scaled according to the number of negative pairs in order to keep steady mutual information bound and gradient magnitude. EqCo bridges the performance gap among a wide range of negative sample sizes, so that for the first time, we can use only a few negative pairs (e.g., 16 per query) to perform self-supervised contrastive training on large-scale vision datasets like ImageNet, while with almost no accuracy drop. This is quite a contrast to the widely used large batch training or memory bank mechanism in current practices. Equipped with EqCo, our simplified MoCo (SiMo) achieves comparable accuracy with MoCov2 on ImageNet (linear evaluation protocol) while only involves 16 negative pairs per query instead of 65536, suggesting that large quantities of negative samples is not a critical factor in contrastive learning frameworks.

cs.CV↗
arXiv2023

EdgeYOLO: An Edge-Real-Time Object Detector

This paper proposes an efficient, low-complexity and anchor-free object detector based on the state-of-the-art YOLO framework, which can be implemented in real time on edge computing platforms. We develop an enhanced data augmentation method to effectively suppress overfitting during training, and design a hybrid random loss function to improve the detection accuracy of small objects. Inspired by FCOS, a lighter and more efficient decoupled head is proposed, and its inference speed can be improved with little loss of precision. Our baseline model can reach the accuracy of 50.6% AP50:95 and 69.8% AP50 in MS COCO2017 dataset, 26.4% AP50:95 and 44.8% AP50 in VisDrone2019-DET dataset, and it meets real-time requirements (FPS>=30) on edge-computing device Nvidia Jetson AGX Xavier. We also designed lighter models with less parameters for edge computing devices with lower computing power, which also show better performances. Our source code, hyper-parameters and model weights are all available at https://github.com/LSH9832/edgeyolo.

cs.CV↗
arXiv2023

Self-triggered Resilient Stabilization of Linear Systems with Quantized Output

This paper studies the problem of stabilizing a self-triggered control system with quantized output. Employing a standard observer-based state feedback control law, a self-triggering mechanism that dictates the next sampling time based on quantized output is co-developed with an output encoding scheme. If, in addition, the transmission protocols at the controller-to-actuator (C-A) and sensor-to-controller (S-C) channels can be adapted, the self-triggered control architecture can be considerably simplified, leveraging a delicate observer-based deadbeat controller to eliminate the need for running the controller in parallel at the encoder side. To account for denial-of-service (DoS) in the S-C channel, the proposed output encoding and self-triggered control schemes are further made resilient. It is shown that a linear time-invariant system can be exponentially stabilized if some conditions on the average DoS duration time are met. There is a trade-off between the maximum inter-sampling time and the resilience against DoS attacks. Finally, a numerical example is presented to demonstrate the practical merits of the proposed self-triggered control schemes and associated theory.

eess.SY↗
arXiv2023

Time-attenuating Twin Delayed DDPG Reinforcement Learning for Trajectory Tracking Control of Quadrotors

Continuous trajectory tracking control of quadrotors is complicated when considering noise from the environment. Due to the difficulty in modeling the environmental dynamics, tracking methodologies based on conventional control theory, such as model predictive control, have limitations on tracking accuracy and response time. We propose a Time-attenuating Twin Delayed DDPG, a model-free algorithm that is robust to noise, to better handle the trajectory tracking task. A deep reinforcement learning framework is constructed, where a time decay strategy is designed to avoid trapping into local optima. The experimental results show that the tracking error is significantly small, and the operation time is one-tenth of that of a traditional algorithm. The OpenAI Mujoco tool is used to verify the proposed algorithm, and the simulation results show that, the proposed method can significantly improve the training efficiency and effectively improve the accuracy and convergence stability.

cs.RO↗