arXiv subjects

Sen Li

Publications and source records attributed to Sen Li.

At least 37 records · Page 2Linked to original sources

arXiv2026

One Step is Enough: Multi-Agent Reinforcement Learning based on One-Step Policy Optimization for Order Dispatch on Ride-Sharing Platforms

Order dispatch is a critical task in ride-sharing systems with Autonomous Vehicles (AVs), directly influencing efficiency and profits. Recently, Multi-Agent Reinforcement Learning (MARL) has emerged as a promising solution to this problem by decomposing the large state and action spaces among individual agents, effectively addressing the Curse of Dimensionality (CoD) in transportation market, which is caused by the substantial number of vehicles, passengers, and orders. However, conventional MARL-based approaches heavily rely on accurate estimation of the value function, which becomes problematic in large-scale, highly uncertain environments. To address this issue, we propose two novel methods that bypass value function estimation, leveraging the homogeneous property of AV fleets. First, we draw an analogy between AV fleets and groups in Group Relative Policy Optimization (GRPO), adapting it to the order dispatch task. By replacing the Proximal Policy Optimization (PPO) baseline with the group average reward-to-go, GRPO eliminates critic estimation errors and reduces training bias. Inspired by this baseline replacement, we further propose One-Step Policy Optimization (OSPO), demonstrating that the optimal policy can be trained using only one-step group rewards under a homogeneous fleet. Experiments on a real-world ride-hailing dataset show that both GRPO and OSPO achieve promising performance across all scenarios, efficiently optimizing pickup times and the number of served orders using simple Multilayer Perceptron (MLP) networks. Furthermore, OSPO outperforms GRPO in all scenarios, attributed to its elimination of bias caused by the bounded time horizon of GRPO. Our code, trained models, and processed data are provided at https://github.com/RS2002/OSPO .

cs.AI↗

arXiv2026

Invertible Diffusion for Low-Memory Channel Gain Map Construction in Wireless Communication Networks

Channel gain maps (CGMs) enable propagation-aware services in edge-intelligent wireless communication networks, while diffusion-based CGM construction is memory intensive for on-device training or adaptation. This letter proposes InvDiff-CGM, an invertible diffusion framework that constructs CGMs from sparse measurements and environmental priors. By adopting invertible architectures in both the diffusion process and the U-Net noise estimator, InvDiff-CGM achieves near-constant training memory consumption. A prior-informed multi-scale injector further integrates environmental priors with sparse measurements to improve physical consistency and detail preservation. Experiments on RadioMap3DSeer show about an 85\% reduction in peak training memory and a PSNR of 38.02~dB, outperforming representative recent baselines. This validates the practicality of InvDiff-CGM for high-fidelity CGM construction under edge resource constraints.

eess.SP↗

arXiv2026

Field-Trial Quantum Key Distribution with Qubit-Based Frame Synchronization

Quantum key distribution (QKD) is a cryptographic technique that uses quantum mechanical principles to enable secure key exchange. Practical deployment of QKD requires robust, cost-effective systems that can operate in challenging field environments. A major challenge is achieving reliable clock synchronization without adding hardware complexity. Conventional approaches often use separate classical light signals, which increase costs and introduce noise that degrades quantum channel performance. To address this limitation, we demonstrate a QKD system incorporating a recently proposed qubit-based distributed frame synchronization method, deployed over a metropolitan fiber network in Nanning, China. Using the polarization-encoded one-decoy-state BB84 protocol and the recently proposed qubit-based distributed frame synchronization method, our system achieves synchronization directly from the quantum signal, eliminating the need for dedicated synchronization hardware. Furthermore, to counteract dynamic polarization disturbances in urban fibers, the system integrates qubit-based polarization feedback control, enabling real-time polarization compensation through an automated polarization controller using data recovered from the qubit-based synchronization signals. During 12 hours of continuous operation, the system maintained a low average quantum bit error rate (QBER) of 1.12/%, achieving a secure key rate of 26.6 kbit/s under 18 dB channel loss. Even under a high channel loss of 40 dB, a finite-key secure rate of 115 bit/s was achieved. This study represents the first successful long-term validation of a frame-synchronization based QKD scheme in a real urban environment, demonstrating exceptional stability and high-loss tolerance, and offering an alternative for building practical, scalable, and cost-efficient quantum-secure communication networks.

quant-ph↗

arXiv2026

Car-1000: A New Large Scale Fine-Grained Visual Categorization Dataset

Fine-grained visual categorization (FGVC) is a challenging but significant task in computer vision, which aims to recognize different sub-categories of birds, cars, airplanes, etc. Among them, recognizing models of different cars has significant application value in autonomous driving, traffic surveillance and scene understanding, which has received considerable attention in the past few years. However, Stanford-Car, the most widely used fine-grained dataset for car recognition, only has 196 different categories and only includes vehicle models produced earlier than 2013. Due to the rapid advancements in the automotive industry during recent years, the appearances of various car models have become increasingly intricate and sophisticated. Consequently, the previous Stanford-Car dataset fails to capture this evolving landscape and cannot satisfy the requirements of automotive industry. To address these challenges, in our paper, we introduce Car-1000, a large-scale dataset designed specifically for fine-grained visual categorization of diverse car models. Car-1000 encompasses vehicles from 166 different automakers, spanning a wide range of 1000 distinct car models. Additionally, we have reproduced several state-of-the-art FGVC methods on the Car-1000 dataset, establishing a new benchmark for research in this field. We hope that our work will offer a fresh perspective for future FGVC researchers. Our dataset is available at https://github.com/toggle1995/Car-1000.

cs.CV↗

arXiv2026

BMG-Q: Localized Bipartite Match Graph Attention Q-Learning for Ride-Pooling Order Dispatch

This paper introduces Localized Bipartite Match Graph Attention Q-Learning (BMG-Q), a novel Multi-Agent Reinforcement Learning (MARL) algorithm framework tailored for ride-pooling order dispatch. BMG-Q advances ride-pooling decision-making process with the localized bipartite match graph underlying the Markov Decision Process, enabling the development of novel Graph Attention Double Deep Q Network (GATDDQN) as the MARL backbone to capture the dynamic interactions among ride-pooling vehicles in fleet. Our approach enriches the state information for each agent with GATDDQN by leveraging a localized bipartite interdependence graph and enables a centralized global coordinator to optimize order matching and agent behavior using Integer Linear Programming (ILP). Enhanced by gradient clipping and localized graph sampling, our GATDDQN improves scalability and robustness. Furthermore, the inclusion of a posterior score function in the ILP captures the online exploration-exploitation trade-off and reduces the potential overestimation bias of agents, thereby elevating the quality of the derived solutions. Through extensive experiments and validation, BMG-Q has demonstrated superior performance in both training and operations for thousands of vehicle agents, outperforming benchmark reinforcement learning frameworks by around 10% in accumulative rewards and showing a significant reduction in overestimation bias by over 50%. Additionally, it maintains robustness amidst task variations and fleet size changes, establishing BMG-Q as an effective, scalable, and robust framework for advancing ride-pooling order dispatch operations.

cs.MA↗

arXiv2025

Triple-BERT: Do We Really Need MARL for Order Dispatch on Ride-Sharing Platforms?

On-demand ride-sharing platforms, such as Uber and Lyft, face the intricate real-time challenge of bundling and matching passengers-each with distinct origins and destinations-to available vehicles, all while navigating significant system uncertainties. Due to the extensive observation space arising from the large number of drivers and orders, order dispatching, though fundamentally a centralized task, is often addressed using Multi-Agent Reinforcement Learning (MARL). However, independent MARL methods fail to capture global information and exhibit poor cooperation among workers, while Centralized Training Decentralized Execution (CTDE) MARL methods suffer from the curse of dimensionality. To overcome these challenges, we propose Triple-BERT, a centralized Single Agent Reinforcement Learning (MARL) method designed specifically for large-scale order dispatching on ride-sharing platforms. Built on a variant TD3, our approach addresses the vast action space through an action decomposition strategy that breaks down the joint action probability into individual driver action probabilities. To handle the extensive observation space, we introduce a novel BERT-based network, where parameter reuse mitigates parameter growth as the number of drivers and orders increases, and the attention mechanism effectively captures the complex relationships among the large pool of driver and orders. We validate our method using a real-world ride-hailing dataset from Manhattan. Triple-BERT achieves approximately an 11.95% improvement over current state-of-the-art methods, with a 4.26% increase in served orders and a 22.25% reduction in pickup times. Our code, trained model parameters, and processed data are publicly available at the repository https://github.com/RS2002/Triple-BERT .

cs.LG↗

arXiv2025

CitySeeker: How Do VLMS Explore Embodied Urban Navigation With Implicit Human Needs?

Vision-Language Models (VLMs) have made significant progress in explicit instruction-based navigation; however, their ability to interpret implicit human needs (e.g., "I am thirsty") in dynamic urban environments remains underexplored. This paper introduces CitySeeker, a novel benchmark designed to assess VLMs' spatial reasoning and decision-making capabilities for exploring embodied urban navigation to address implicit needs. CitySeeker includes 6,440 trajectories across 8 cities, capturing diverse visual characteristics and implicit needs in 7 goal-driven scenarios. Extensive experiments reveal that even top-performing models (e.g., Qwen2.5-VL-32B-Instruct) achieve only 21.1% task completion. We find key bottlenecks in error accumulation in long-horizon reasoning, inadequate spatial cognition, and deficient experiential recall. To further analyze them, we investigate a series of exploratory strategies-Backtracking Mechanisms, Enriching Spatial Cognition, and Memory-Based Retrieval (BCR), inspired by human cognitive mapping's emphasis on iterative observation-reasoning cycles and adaptive path optimization. Our analysis provides actionable insights for developing VLMs with robust spatial intelligence required for tackling "last-mile" navigation challenges.

cs.AI↗

arXiv2025

Inference of $B$-mode polarization in the presence of non-Gaussian foregrounds

The inflationary $B$-mode signals encode invaluable information about the origin of our Universe and searching for potential signatures of primordial gravitational waves (PGWs) is one of the major science goals for future precision observations of cosmic microwave background (CMB) polarization. However, dominant $B$-mode signals of both Galactic foreground contamination and gravitational lensing effects prevent direct measurements of the PGW $B$-mode signals. There are existing proposals which can effectively eliminate these two contaminants but issues remain for future high-sensitivity and multifrequency CMB polarization observations, such as spatially-varying spectral energy distribution (SED) of polarized foreground and cosmological $B$-mode signals due to primordial magnetic fields (PMFs). In this work, we investigate inference of PGW $B$-mode signals in the presence of both complexities. We employ a constrained moment internal linear combination (cMILC) method to remove polarization signals arising from spatially varying SEDs. Also, we employ a power-spectrum-based approach to extracting both the Galactic and cosmological $B$-mode components. Two methods have been validated by mock data and different consistency tests have been performed. We apply these two methods to end-to-end simulations for future high-sensitivity and multifrequency polarization observations and investigate the detectability of different $B$-mode signals in the presence of non-Gaussian polarized foregrounds under different scenarios. This study will be important for new physics studies with $B$-mode signatures.

astro-ph.CO↗

arXiv2025

QSTAformer: A Quantum-Enhanced Transformer for Robust Short-Term Voltage Stability Assessment against Adversarial Attacks

Short-term voltage stability assessment (STVSA) is critical for secure power system operation. While classical machine learning-based methods have demonstrated strong performance, they still face challenges in robustness under adversarial conditions. This paper proposes QSTAformer-a tailored quantum-enhanced Transformer architecture that embeds parameterized quantum circuits (PQCs) into attention mechanisms-for robust and efficient STVSA. A dedicated adversarial training strategy is developed to defend against both white-box and gray-box attacks. Furthermore, diverse PQC architectures are benchmarked to explore trade-offs between expressiveness, convergence, and efficiency. To the best of our knowledge, this is the first work to systematically investigate the adversarial vulnerability of quantum machine learning-based STVSA. Case studies on the IEEE 39-bus system demonstrate that QSTAformer achieves competitive accuracy, reduced complexity, and stronger robustness, underscoring its potential for secure and scalable STVSA under adversarial conditions.

eess.SY↗

arXiv2025

Pre-emptive parametric kill switch for evaporative atomic sources in vacuum

A robust pre-emptive kill switch for cold atom experiments is introduced to significantly reduce costly system reassembly or replacement. The design incorporates upper (alarm) and lower (evaporation) event detection mechanisms based on predefined thresholds. Meanwhile, a duty cycle timing methodology is used to avert unintentional activation of the dispenser in circumstances where pulse signals occur. The circuit employs generic components, a modular design, and formalized logic, ensuring cost-effectiveness, making the design suitable for school laboratories and other research environments. This design is highly versatile and can be applied to other sensitive devices beyond dispensers, such as heating filaments, titanium sublimation pumps, tungsten lamps, and comparable systems.

physics.atom-ph↗

arXiv2025

LLMs as Sparse Retrievers:A Framework for First-Stage Product Search

Product search is a crucial component of modern e-commerce platforms, with billions of user queries every day. In product search systems, first-stage retrieval should achieve high recall while ensuring efficient online deployment. Sparse retrieval is particularly attractive in this context due to its interpretability and storage efficiency. However, sparse retrieval methods suffer from severe vocabulary mismatch issues, leading to suboptimal performance in product search scenarios. With their potential for semantic analysis, large language models (LLMs) offer a promising avenue for mitigating vocabulary mismatch issues and thereby improving retrieval quality. Directly applying LLMs to sparse retrieval in product search exposes two key challenges:(1)Queries and product titles are typically short and highly susceptible to LLM-induced hallucinations, such as generating irrelevant expansion terms or underweighting critical literal terms like brand names and model numbers;(2)The large vocabulary space of LLMs leads to difficulty in initializing training effectively, making it challenging to learn meaningful sparse representations in such ultra-high-dimensional spaces.To address these challenges, we propose PROSPER, a framework for PROduct search leveraging LLMs as SParsE Retrievers. PROSPER incorporates: (1)A literal residual network that alleviates hallucination in lexical expansion by reinforcing underweighted literal terms through a residual compensation mechanism; and (2)A lexical focusing window that facilitates effective training initialization via a coarse-to-fine sparsification strategy.Extensive offline and online experiments show that PROSPER significantly outperforms sparse baselines and achieves recall performance comparable to advanced dense retrievers, while also achieving revenue increments online.

cs.IR↗

arXiv2025

C2T-ID: Converting Semantic Codebooks to Textual Document Identifiers for Generative Search

Designing document identifiers (docids) that carry rich semantic information while maintaining tractable search spaces is a important challenge in generative retrieval (GR). Popular codebook methods address this by building a hierarchical semantic tree and constraining generation to its child nodes, yet their numeric identifiers cannot leverage the large language model's pretrained natural language understanding. Conversely, using text as docid provides more semantic expressivity but inflates the decoding space, making the system brittle to early-step errors. To resolve this trade-off, we propose C2T-ID: (i) first construct semantic numerical docid via hierarchical clustering; (ii) then extract high-frequency metadata keywords and iteratively replace each numeric label with its cluster's top-K keywords; and (iii) an optional two-level semantic smoothing step further enhances the fluency of C2T-ID. Experiments on Natural Questions and Taobao's product search demonstrate that C2T-ID significantly outperforms atomic, semantic codebook, and pure-text docid baselines, demonstrating its effectiveness in balancing semantic expressiveness with search space constraints.

cs.IR↗

arXiv2025

MetaOcc: Spatio-Temporal Fusion of Surround-View 4D Radar and Camera for 3D Occupancy Prediction with Dual Training Strategies

Robust 3D occupancy prediction is essential for autonomous driving, particularly under adverse weather conditions where traditional vision-only systems struggle. While the fusion of surround-view 4D radar and cameras offers a promising low-cost solution, effectively extracting and integrating features from these heterogeneous sensors remains challenging. This paper introduces MetaOcc, a novel multi-modal framework for omnidirectional 3D occupancy prediction that leverages both multi-view 4D radar and images. To address the limitations of directly applying LiDAR-oriented encoders to sparse radar data, we propose a Radar Height Self-Attention module that enhances vertical spatial reasoning and feature extraction. Additionally, a Hierarchical Multi-scale Multi-modal Fusion strategy is developed to perform adaptive local-global fusion across modalities and time, mitigating spatio-temporal misalignments and enriching fused feature representations. To reduce reliance on expensive point cloud annotations, we further propose a pseudo-label generation pipeline based on an open-set segmentor. This enables a semi-supervised strategy that achieves 90% of the fully supervised performance using only 50% of the ground truth labels, offering an effective trade-off between annotation cost and accuracy. Extensive experiments demonstrate that MetaOcc under full supervision achieves state-of-the-art performance, outperforming previous methods by +0.47 SC IoU and +4.02 mIoU on the OmniHD-Scenes dataset, and by +1.16 SC IoU and +1.24 mIoU on the SurroundOcc-nuScenes dataset. These results demonstrate the scalability and robustness of MetaOcc across sensor domains and training conditions, paving the way for practical deployment in real-world autonomous systems. Code and data are available at https://github.com/LucasYang567/MetaOcc.

cs.CV↗

arXiv2025

From General Reasoning to Domain Expertise: Uncovering the Limits of Generalization in Large Language Models

Recent advancements in Large Language Models (LLMs) have demonstrated remarkable capabilities in various domains. However, effective decision-making relies heavily on strong reasoning abilities. Reasoning is the foundation for decision-making, providing the analytical and logical framework to make sound choices. Reasoning involves analyzing information, drawing inferences, and reaching conclusions based on logic or evidence. Decision-making builds on this foundation by applying the insights from reasoning to select the best course of action among alternatives. Together, these processes create a continuous cycle of thought and action aimed at achieving goals effectively. As AI technology evolves, there is a growing trend to train LLMs to excel in general reasoning. This study explores how the general reasoning capabilities of LLMs connect to their performance in domain-specific reasoning tasks.

cs.CL↗

arXiv2025

Coordinating Ride-Pooling with Public Transit using Reward-Guided Conservative Q-Learning: An Offline Training and Online Fine-Tuning Reinforcement Learning Framework

This paper introduces a novel reinforcement learning (RL) framework, termed Reward-Guided Conservative Q-learning (RG-CQL), to enhance coordination between ride-pooling and public transit within a multimodal transportation network. We model each ride-pooling vehicle as an agent governed by a Markov Decision Process (MDP) and propose an offline training and online fine-tuning RL framework to learn the optimal operational decisions of the multimodal transportation systems, including rider-vehicle matching, selection of drop-off locations for passengers, and vehicle routing decisions, with improved data efficiency. During the offline training phase, we develop a Conservative Double Deep Q Network (CDDQN) as the action executor and a supervised learning-based reward estimator, termed the Guider Network, to extract valuable insights into action-reward relationships from data batches. In the online fine-tuning phase, the Guider Network serves as an exploration guide, aiding CDDQN in effectively and conservatively exploring unknown state-action pairs. The efficacy of our algorithm is demonstrated through a realistic case study using real-world data from Manhattan. We show that integrating ride-pooling with public transit outperforms two benchmark cases solo rides coordinated with transit and ride-pooling without transit coordination by 17% and 22% in the achieved system rewards, respectively. Furthermore, our innovative offline training and online fine-tuning framework offers a remarkable 81.3% improvement in data efficiency compared to traditional online RL methods with adequate exploration budgets, with a 4.3% increase in total rewards and a 5.6% reduction in overestimation errors. Experimental results further demonstrate that RG-CQL effectively addresses the challenges of transitioning from offline to online RL in large-scale ride-pooling systems integrated with transit.

cs.LG↗

arXiv2025

Joint Infrastructure Planning and Order Assignment for On-Demand Food-Delivery Services with Coordinated Drones and Human Couriers

This paper investigates the optimal infrastructure planning and order assignment problem of an on-demand food-delivery platform with a mixed fleet of drones and human couriers. The platform has two delivery modes: (a) ground delivery and (b) drone-assisted delivery (i.e., air delivery). In ground delivery, couriers directly collect and transport orders from restaurants to destinations. For air delivery, the delivery process involves three legs: initially, a human courier picks up the order from the restaurant and transports it to a nearby launchpad, where personnel load the orders onto drones and replace batteries as needed. The loaded drone then transports the order from the launchpad to a kiosk, where another courier retrieves the order from the kiosk for final delivery. The platform must determine the optimal locations for launchpads and kiosks within a transportation network, and devise an order assignment strategy that allocates food-delivery orders between ground and air delivery considering the bundling probabilities of ground deliveries and the waiting times at launchpads and kiosks. We formulate the platform's problem as a mixed-integer nonlinear program and develop a novel neural network-assisted optimization method to obtain high-quality solutions. A case study in Hong Kong validates our model and algorithm, revealing that drone delivery reduces operational costs, minimizes courier fleet size, and increases order bundling opportunities. We also find that the expansion of air delivery services may entail larger delivery times due to the trade-off between the travel time savings induced by the faster air delivery and the associated detours incurred by intermodal transfer and extra waiting times at launchpads and kiosks, which crucially depends on the distance of the orders and the sequence of activating long-distance air delivery routes versus short-distance ones.

eess.SY↗

arXiv2024

Temporal Action Localization with Cross Layer Task Decoupling and Refinement

Temporal action localization (TAL) involves dual tasks to classify and localize actions within untrimmed videos. However, the two tasks often have conflicting requirements for features. Existing methods typically employ separate heads for classification and localization tasks but share the same input feature, leading to suboptimal performance. To address this issue, we propose a novel TAL method with Cross Layer Task Decoupling and Refinement (CLTDR). Based on the feature pyramid of video, CLTDR strategy integrates semantically strong features from higher pyramid layers and detailed boundary-aware boundary features from lower pyramid layers to effectively disentangle the action classification and localization tasks. Moreover, the multiple features from cross layers are also employed to refine and align the disentangled classification and regression results. At last, a lightweight Gated Multi-Granularity (GMG) module is proposed to comprehensively extract and aggregate video features at instant, local, and global temporal granularities. Benefiting from the CLTDR and GMG modules, our method achieves state-of-the-art performance on five challenging benchmarks: THUMOS14, MultiTHUMOS, EPIC-KITCHENS-100, ActivityNet-1.3, and HACS. Our code and pre-trained models are publicly available at: https://github.com/LiQiang0307/CLTDR-GMG.

cs.CV↗

arXiv2024

A conditional normalizing flow for domain decomposed uncertainty quantification

In this paper we present a conditional KRnet (cKRnet) based domain decomposed uncertainty quantification (CKR-DDUQ) approach to propagate uncertainties across different physical domains in models governed by partial differential equations (PDEs) with random inputs. This approach is based on the domain decomposed uncertainty quantification (DDUQ) method presented in [Q. Liao and K. Willcox, SIAM J. Sci. Comput., 37 (2015), pp. A103--A133], which suffers a bottleneck of density estimation for local joint input distributions in practice. In this work, we reformulate the required joint distributions as conditional distributions, and propose a new conditional normalizing flow model, called cKRnet, to efficiently estimate the conditional probability density functions. We present the general framework of CKR-DDUQ, conduct its convergence analysis, validate its accuracy and demonstrate its efficiency with numerical experiments.

math.NA↗