arXiv subjects

Yaosheng Xu

Publications and source records attributed to Yaosheng Xu.

9 recordsLinked to original sources

arXiv2025

Explicit Steady-State Approximations for Parallel Server Systems with Heterogeneous Servers

We study the steady-state performance of parallel-server systems under an immediate routing architecture with two sources of heterogeneity: servers and job classes, subject to compatibility constraints. We focus on the weighted-workload-task-allocation (WWTA) policy, a load-balancing scheme known to be throughput-optimal for such systems. Under a relaxed complete-resource-pooling (CRP) condition, we prove a "strong form" of state-space collapse in heavy traffic and that the scaled workload of each server converges in distribution to an exponential random variable, whose parameter is explicitly given by system primitives. Our analysis yields three main insights. First, the conventional heavy-traffic requirement of a unique static allocation plan can be dropped; a relaxed CRP condition suffices. Second, the limiting workload distribution is shown to be independent of local scheduling policy on server side, allowing substantial flexibility. Third, the inefficient (non-basic) activities prescribed by static allocation plan is proved to receive an asymptotically negligible fraction of routing and service, even though WWTA has no prior knowledge of which activities are basic, highlighting its robustness to changing arrival rates.

math.PR↗

arXiv2025

Asymptotic product-form steady-state for generalized Jackson networks in multi-scale heavy traffic

We prove that under a multi-scale heavy traffic condition, the stationary distribution of the scaled queue length vector process in any generalized Jackson network has a product-form limit. Each component in the product form follows an exponential distribution, corresponding to the Brownian approximation of a single station queue. The ``single station'' can be constructed precisely and its parameters have a good intuitive interpretation.

math.PR↗

arXiv2025

Dynamic control of stochastic matching systems in heavy traffic: An effective computational method for high-dimensional problems

Bipartite matching systems arise in many settings where agents or tasks from two distinct sets must be paired dynamically under compatibility constraints. We consider a high-dimensional bipartite matching system under uncertainty and seek an effective dynamic control policy that maximizes the expected discounted total value generated by the matches minus the congestion-related costs. To derive a tractable approximation, we focus attention on balanced, high-volume systems, i.e., the heavy-traffic regime, and derive an approximating Brownian control problem. We then develop a computational method that relies on deep neural network technology for solving this problem. To show the effectiveness of the policy derived from our computational method, we compare it to the benchmark policies available in the extant literature in the context of the original matching problem. In the test problems attempted thus far, our proposed policy outperforms the benchmarks, and its derivation is computationally feasible for dimensions up to 100 or more.

math.OC↗

arXiv2025

NeuralMOVES: A lightweight and microscopic vehicle emission estimation model based on reverse engineering and surrogate learning

The transportation sector significantly contributes to greenhouse gas emissions, necessitating accurate emission models to guide mitigation strategies. Despite its field validation and certification, the industry-standard Motor Vehicle Emission Simulator (MOVES) faces challenges related to complexity in usage, high computational demands, and its unsuitability for microscopic real-time applications. To address these limitations, we present NeuralMOVES, a comprehensive suite of high-performance, lightweight surrogate models for vehicle CO2 emissions. Developed based on reverse engineering and Neural Networks, NeuralMOVES achieves a remarkable 6.013% Mean Average Percentage Error relative to MOVES across extensive tests spanning over two million scenarios with diverse trajectories and the factors regarding environments and vehicles. NeuralMOVES is only 2.4 MB, largely condensing the original MOVES and the reverse engineered MOVES into a compact representation, while maintaining high accuracy. Therefore, NeuralMOVES significantly enhances accessibility while maintaining the accuracy of MOVES, simplifying CO2 evaluation for transportation analyses and enabling real-time, microscopic applications across diverse scenarios without reliance on complex software or extensive computational resources. Moreover, this paper provides, for the first time, a framework for reverse engineering industrial-grade software tailored specifically to transportation scenarios, going beyond MOVES. The surrogate models are available at https://github.com/edgar-rs/neuralMOVES.

cs.LG↗

arXiv2024

Steady-State Convergence of the Continuous-Time Routing System with General Distributions in Heavy Traffic

This paper examines a continuous-time routing system with general interarrival and service time distributions, operating under the join-the-shortest-queue and power-of-two-choices policies. Under a weaker set of assumptions than those commonly found in the literature, we prove that the scaled steady-state queue length at each station converges weakly to an identical exponential random variable in heavy traffic. Specifically, our results hold under the assumption of the $(2 + δ_0)$th moment for the interarrival and service distributions with some $δ_0 > 0$. The proof leverages the Palm version of the basic adjoint relationship (BAR) as a key technique.

math.PR↗

arXiv2022

Reducing Variance in Temporal-Difference Value Estimation via Ensemble of Deep Networks

In temporal-difference reinforcement learning algorithms, variance in value estimation can cause instability and overestimation of the maximal target value. Many algorithms have been proposed to reduce overestimation, including several recent ensemble methods, however none have shown success in sample-efficient learning through addressing estimation variance as the root cause of overestimation. In this paper, we propose MeanQ, a simple ensemble method that estimates target values as ensemble means. Despite its simplicity, MeanQ shows remarkable sample efficiency in experiments on the Atari Learning Environment benchmark. Importantly, we find that an ensemble of size 5 sufficiently reduces estimation variance to obviate the lagging target network, eliminating it as a source of bias and further gaining sample efficiency. We justify intuitively and empirically the design choices in MeanQ, including the necessity of independent experience sampling. On a set of 26 benchmark Atari environments, MeanQ outperforms all tested baselines, including the best available baseline, SUNRISE, at 100K interaction steps in 16/26 environments, and by 68% on average. MeanQ also outperforms Rainbow DQN at 500K steps in 21/26 environments, and by 49% on average, and achieves average human-level performance using 200K ($\pm$100K) interaction steps. Our implementation is available at https://github.com/indylab/MeanQ.

cs.LG↗

arXiv2021

Target Entropy Annealing for Discrete Soft Actor-Critic

Soft Actor-Critic (SAC) is considered the state-of-the-art algorithm in continuous action space settings. It uses the maximum entropy framework for efficiency and stability, and applies a heuristic temperature Lagrange term to tune the temperature $α$, which determines how "soft" the policy should be. It is counter-intuitive that empirical evidence shows SAC does not perform well in discrete domains. In this paper we investigate the possible explanations for this phenomenon and propose Target Entropy Scheduled SAC (TES-SAC), an annealing method for the target entropy parameter applied on SAC. Target entropy is a constant in the temperature Lagrange term and represents the target policy entropy in discrete SAC. We compare our method on Atari 2600 games with different constant target entropy SAC, and analyze on how our scheduling affects SAC.

cs.LG↗

arXiv2021

Temporal-Difference Value Estimation via Uncertainty-Guided Soft Updates

Temporal-Difference (TD) learning methods, such as Q-Learning, have proven effective at learning a policy to perform control tasks. One issue with methods like Q-Learning is that the value update introduces bias when predicting the TD target of a unfamiliar state. Estimation noise becomes a bias after the max operator in the policy improvement step, and carries over to value estimations of other states, causing Q-Learning to overestimate the Q value. Algorithms like Soft Q-Learning (SQL) introduce the notion of a soft-greedy policy, which reduces the estimation bias via soft updates in early stages of training. However, the inverse temperature $β$ that controls the softness of an update is usually set by a hand-designed heuristic, which can be inaccurate at capturing the uncertainty in the target estimate. Under the belief that $β$ is closely related to the (state dependent) model uncertainty, Entropy Regularized Q-Learning (EQL) further introduces a principled scheduling of $β$ by maintaining a collection of the model parameters that characterizes model uncertainty. In this paper, we present Unbiased Soft Q-Learning (UQL), which extends the work of EQL from two action, finite state spaces to multi-action, infinite state space Markov Decision Processes. We also provide a principled numerical scheduling of $β$, extended from SQL and using model uncertainty, during the optimization process. We show the theoretical guarantees and the effectiveness of this update method in experiments on several discrete control environments.

cs.LG↗

arXiv2021

Adaptive Estimation in Multivariate Response Regression with Hidden Variables

This paper studies the estimation of the coefficient matrix $\Ttheta$ in multivariate regression with hidden variables, $Y = (\Ttheta)^TX + (B^*)^TZ + E$, where $Y$ is a $m$-dimensional response vector, $X$ is a $p$-dimensional vector of observable features, $Z$ represents a $K$-dimensional vector of unobserved hidden variables, possibly correlated with $X$, and $E$ is an independent error. The number of hidden variables $K$ is unknown and both $m$ and $p$ are allowed but not required to grow with the sample size $n$. Since only $Y$ and $X$ are observable, we provide necessary conditions for the identifiability of $\Ttheta$. The same set of conditions are shown to be sufficient when the error $E$ is homoscedastic. Our identifiability proof is constructive and leads to a novel and computationally efficient estimation algorithm, called HIVE. The first step of the algorithm is to estimate the best linear prediction of $Y$ given $X$ in which the unknown coefficient matrix exhibits an additive decomposition of $\Ttheta$ and a dense matrix originated from the correlation between $X$ and the hidden variable $Z$. Under the row sparsity assumption on $\Ttheta$, we propose to minimize a penalized least squares loss by regularizing $\Ttheta$ via a group-lasso penalty and regularizing the dense matrix via a multivariate ridge penalty. Non-asymptotic deviation bounds of the in-sample prediction error are established. Our second step is to estimate the row space of $B^*$ by leveraging the covariance structure of the residual vector from the first step. In the last step, we remove the effect of hidden variable by projecting $Y$ onto the complement of the estimated row space of $B^*$. Non-asymptotic error bounds of our final estimator are established. The model identifiability, parameter estimation and statistical guarantees are further extended to the setting with heteroscedastic errors.

math.ST↗