arXiv subjects

I-Hong Hou

Publications and source records attributed to I-Hong Hou.

At least 19 recordsLinked to original sources

arXiv2026

TSDM: A Scheduling Policy for Joint Throughput-AoI Optimization in Multichannel Wireless Networks

Optimizing for both low Age of Information (AoI) and high throughput is critical for remote sensing applications that rely on multichannel wireless networks. However, jointly optimizing these two metrics is an analytically challenging problem, particularly in systems with heterogeneous and unreliable channels. To address this challenge, we propose TSDM, a Two-Stage Deficit Matching scheduling framework. TSDM is based on a second-order approach that characterizes the performance of each data flow by its mean and temporal variance. In the first stage, TSDM translates the high-level utility maximization objective into a concrete set of target mean and temporal variance statistics for transmissions over each node-channel pair. In the second stage, a low-complexity Weighted Matching Deficit (WMD) rule performs real-time channel assignment. We theoretically prove that TSDM achieves the desired mean and temporal variance for each flow. Furthermore, we conduct extensive simulations on two open joint throughput-AoI optimization problems. In both cases, TSDM significantly outperforms existing scheduling policies.

eess.SY↗

arXiv2026

Serving Chain-structured Jobs with Large Memory Footprints with Application to Large Foundation Model Serving

As a current trend in Artificial Intelligence (AI), large foundation models are increasingly employed as the core of AI services. However, even after training, serving such models at scale remains a challenging task due to their heavy resource footprints, particularly in terms of GPU memory. While recent works revealed unique characteristics of systems serving foundation models that distinguish them from traditional distributed computing systems, there is still a lack of fundamental understanding of the underlying system management problems. This work aims at addressing this gap by extracting a novel problem of "server chain composition" via block placement and cache allocation for serving chainstructured jobs with large memory footprints, which models a fundamental problem in serving large foundation models through pipeline parallelism. After showing the NP-hardness of the optimal solution, the focus is turned to developing scalable algorithms with guaranteed performance under state-of-the-art load balancing. Application of the proposed solution to a distributed large language model (LLM) serving system shows significant reduction of response times compared to state-of-the-art solutions.

cs.DC↗

arXiv2026

Restless Bandits with Individual Penalty Constraints: Near-Optimal Indices and Deep Reinforcement Learning

This paper investigates the Restless Multi-Armed Bandit (RMAB) framework under individual penalty constraints to address resource allocation challenges in dynamic wireless networked environments. Unlike conventional RMAB models, our model allows each user (arm) to have distinct and stringent performance constraints, such as energy limits, activation limits, or age of information minimums, enabling the capture of diverse objectives including fairness and efficiency. To find the optimal resource allocation policy, we propose a new Penalty-Optimal Whittle (POW) index policy. The POW index of an user only depends on the user's transition kernel and penalty constraints, and remains invariable to system-wide features such as the number of users present and the amount of resource available. This makes it computationally tractable to calculate the POW indices offline without any need for online adaptation. Moreover, we theoretically prove that the POW index policy is asymptotically optimal while satisfying all individual penalty constraints. We also introduce a deep reinforcement learning algorithm to efficiently learn the POW index on the fly. Simulation results across various applications and system configurations further demonstrate that the POW index policy not only has near-optimal performance but also significantly outperforms other existing policies.

cs.LG↗

arXiv2025

Understanding the Fundamental Trade-Off Between Age of Information and Throughput in Unreliable Wireless Networks

This paper characterizes the fundamental trade-off between throughput and Age of Information (AoI) in wireless networks where multiple devices transmit status updates to a central base station over unreliable channels. To address the complexity introduced by stochastic transmission successes, we propose the throughput-AoI capacity region, which defines all feasible throughput-AoI pairs achievable under any scheduling policy. Using a second-order approximation that incorporates both mean and temporal variance, we derive an outer bound and a tight inner bound for the throughput-AoI capacity region. Furthermore, we propose a simple and low complexity scheduling policy and prove that it achieves every interior point within the tight inner bound. This establishes a systematic and theoretically grounded framework for the joint optimization of throughput and information freshness in practical wireless communication scenarios. To validate our theoretical framework and demonstrate the utility of the throughput-AoI capacity region, extensive simulations are implemented. Simulation results demonstrate that our proposed policy significantly outperforms conventional methods across various practical network optimization scenarios. The findings highlight our approach's effectiveness in optimizing both throughput and AoI, underscoring its applicability and robustness in practical wireless networks.

eess.SY↗

arXiv2024

Network Optimization in Dynamic Systems: Fast Adaptation via Zero-Shot Lagrangian Update

This paper addresses network optimization in dynamic systems, where factors such as user composition, service requirements, system capacity, and channel conditions can change abruptly and unpredictably. Unlike existing studies that focus primarily on optimizing long-term performance in steady states, we develop online learning algorithms that enable rapid adaptation to sudden changes. Recognizing that many current network optimization algorithms rely on dual methods to iteratively learn optimal Lagrange multipliers, we propose zero-shot updates for these multipliers using only information available at the time of abrupt changes. By combining Taylor series analysis with complementary slackness conditions, we theoretically derive zero-shot updates applicable to various abrupt changes in two distinct network optimization problems. These updates can be integrated with existing algorithms to significantly improve performance during transitory phases in terms of total utility, operational cost, and constraint violations. Simulation results demonstrate that our zero-shot updates substantially improve transitory performance, often achieving near-optimal outcomes without additional learning, even under severe system changes.

cs.NI↗

arXiv2024

Second-Order Analysis of CSMA Protocols for Age-of-Information Minimization

This paper introduces a general framework to analyze and optimize age-of-information (AoI) in CSMA protocols for distributed uplink transmissions. The proposed framework combines two theoretical approaches. First, it employs second-order analysis that characterizes all random processes by their respective means and temporal variances and approximates AoI as a function of the mean and temporal variance of the packet delivery process. Second, it employs mean-field approximation to derive the mean and temporal variance of the packet delivery process for one node in the presence of interference from others. To demonstrate the utility of this framework, this paper applies it to the age-threshold ALOHA policy and identifies parameter settings that outperform those previously suggested as optimal in the original work that introduced this policy. Simulation results demonstrate that our framework provides precise AoI approximations and achieves significantly better performance, even in networks with a small number of users.

cs.IT↗

arXiv2024

Deep Index Policy for Multi-Resource Restless Matching Bandit and Its Application in Multi-Channel Scheduling

Scheduling in multi-channel wireless communication system presents formidable challenges in effectively allocating resources. To address these challenges, we investigate a multi-resource restless matching bandit (MR-RMB) model for heterogeneous resource systems with an objective of maximizing long-term discounted total rewards while respecting resource constraints. We have also generalized to applications beyond multi-channel wireless. We discuss the Max-Weight Index Matching algorithm, which optimizes resource allocation based on learned partial indexes. We have derived the policy gradient theorem for index learning. Our main contribution is the introduction of a new Deep Index Policy (DIP), an online learning algorithm tailored for MR-RMB. DIP learns the partial index by leveraging the policy gradient theorem for restless arms with convoluted and unknown transition kernels of heterogeneous resources. We demonstrate the utility of DIP by evaluating its performance for three different MR-RMB problems. Our simulation results show that DIP indeed learns the partial indexes efficiently.

cs.LG↗

arXiv2024

Distributed No-Regret Learning for Multi-Stage Systems with End-to-End Bandit Feedback

This paper studies multi-stage systems with end-to-end bandit feedback. In such systems, each job needs to go through multiple stages, each managed by a different agent, before generating an outcome. Each agent can only control its own action and learn the final outcome of the job. It has neither knowledge nor control on actions taken by agents in the next stage. The goal of this paper is to develop distributed online learning algorithms that achieve sublinear regret in adversarial environments. The setting of this paper significantly expands the traditional multi-armed bandit problem, which considers only one agent and one stage. In addition to the exploration-exploitation dilemma in the traditional multi-armed bandit problem, we show that the consideration of multiple stages introduces a third component, education, where an agent needs to choose its actions to facilitate the learning of agents in the next stage. To solve this newly introduced exploration-exploitation-education trilemma, we propose a simple distributed online learning algorithm, $ε-$EXP3. We theoretically prove that the $ε-$EXP3 algorithm is a no-regret policy that achieves sublinear regret. Simulation results show that the $ε-$EXP3 algorithm significantly outperforms existing no-regret online learning algorithms for the traditional multi-armed bandit problem.

cs.LG↗

arXiv2024

AoI, Timely-Throughput, and Beyond: A Theory of Second-Order Wireless Network Optimization

This paper introduces a new theoretical framework for optimizing second-order behaviors of wireless networks. Unlike existing techniques for network utility maximization, which only consider first-order statistics, this framework models every random process by its mean and temporal variance. The inclusion of temporal variance makes this framework well-suited for modeling Markovian fading wireless channels and emerging network performance metrics such as age-of-information (AoI) and timely-throughput. Using this framework, we sharply characterize the second-order capacity region of wireless access networks. We also propose a simple scheduling policy and prove that it can achieve every interior point in the second-order capacity region. To demonstrate the utility of this framework, we apply it to an unsolved network optimization problem where some clients wish to minimize AoI while others wish to maximize timely-throughput. We show that this framework accurately characterizes AoI and timely-throughput. Moreover, it leads to a tractable scheduling policy that outperforms other existing work.

cs.NI↗

arXiv2024

Learning and Communications Co-Design for Remote Inference Systems: Feature Length Selection and Transmission Scheduling

In this paper, we consider a remote inference system, where a neural network is used to infer a time-varying target (e.g., robot movement), based on features (e.g., video clips) that are progressively received from a sensing node (e.g., a camera). Each feature is a temporal sequence of sensory data. The inference error is determined by (i) the timeliness and (ii) the sequence length of the feature, where we use Age of Information (AoI) as a metric for timeliness. While a longer feature can typically provide better inference performance, it often requires more channel resources for sending the feature. To minimize the time-averaged inference error, we study a learning and communication co-design problem that jointly optimizes feature length selection and transmission scheduling. When there is a single sensor-predictor pair and a single channel, we develop low-complexity optimal co-designs for both the cases of time-invariant and time-variant feature length. When there are multiple sensor-predictor pairs and multiple channels, the co-design problem becomes a restless multi-arm multi-action bandit problem that is PSPACE-hard. For this setting, we design a low-complexity algorithm to solve the problem. Trace-driven evaluations demonstrate the potential of these co-designs to reduce inference error by up to 10000 times.

cs.IT↗

arXiv2024

Timely Communications for Remote Inference

In this paper, we analyze the impact of data freshness on remote inference systems, where a pre-trained neural network blue infers a time-varying target (e.g., the locations of vehicles and pedestrians) based on features (e.g., video frames) observed at a sensing node (e.g., a camera). One might expect that the performance of a remote inference system degrades monotonically as the feature becomes stale. Using an information-theoretic analysis, we show that this is true if the feature and target data sequence can be closely approximated as a Markov chain, whereas it is not true if the data sequence is far from being Markovian. Hence, the inference error is a function of Age of Information (AoI), where the function could be non-monotonic. To minimize the inference error in real-time, we propose a new "selection-from-buffer" model for sending the features, which is more general than the "generate-at-will" model used in earlier studies. In addition, we design low-complexity scheduling policies to improve inference performance. For single-source, single-channel systems, we provide an optimal scheduling policy. In multi-source, multi-channel systems, the scheduling problem becomes a multi-action restless multi-armed bandit problem. For this setting, we design a new scheduling policy by integrating Whittle index-based source selection and duality-based feature selection-from-buffer algorithms. This new scheduling policy is proven to be asymptotically optimal. These scheduling results hold for minimizing general AoI functions (monotonic or non-monotonic). Data-driven evaluations demonstrate the significant advantages of our proposed scheduling policies.

cs.NI↗

arXiv2024

Optimizing Age of Information in Random Access Networks: A Second-Order Approach for Active/Passive Users

In this paper, we study the moments of the Age of Information (AoI) for both active and passive users in a random access network. In this network, active users broadcast sensing data, while passive users detect in-band radio activities from out-of-network devices, such as jammers. Collisions occur when multiple active users transmit simultaneously. Passive users can detect radio activities only when no active user transmits. Each active user's transmission behavior follows a Markov process. We aim to minimize the weighted sum of any moments of AoI for both user types. To achieve this, we employ a second-order analysis of system behavior. Specifically, we characterize an active user's transmission Markov process using its mean and temporal variance. We show that any moment of the AoI can be approximated by a function of these two parameters. This insight enables us to analyze and optimize the transmission Markov process for active users. We apply this strategy to two different random access models. Simulation results show that policies derived from this strategy outperform other baseline policies.

cs.NI↗

arXiv2024

Goal-Oriented Sensor Reporting Scheduling for Non-linear Dynamic System Monitoring

Goal-oriented communication (GoC) is a form of semantic communication where the effectiveness of information transmission is measured by its impact on achieving the desired goal. In Internet-of-Things (IoT) networks, GoC can enable sensors to selectively transmit data relevant to the intended goals of the receiver, thereby facilitating timely decision-making, reducing network congestion, and enhancing spectral efficiency. In this paper, we consider an IoT scenario where an edge node polls sensors monitoring the state of a non-linear dynamic system (NLDS) to respond to the queries of several clients. This work delves into the foregoing GoC problem and solution, which we termed goal-oriented scheduling (GoS). The latter utilizes deep reinforcement learning (DRL) with meticulously devised action space, state space, and reward function. A long short-term memory network is used to estimate the inter-query duration and the corresponding estimation standard deviation. This empowers the proposed DRL scheduler to make judicious decisions, even when no queries are posed. Numerical analysis demonstrates that the proposed GoS reduces the mean square error (MSE) of the query responses compared to the benchmark scheduling methods even as the number of clients and DRL action space increase, which proves its scalability. Moreover, this is attained without polling sensors during $70\% - 87\%$ of the testing phase, thus promoting energy efficiency. Lastly, the complexity of the proposed GoS is relatively lower than the benchmark scheduling methods.

eess.SY↗

arXiv2024

Contextual Restless Multi-Armed Bandits with Application to Demand Response Decision-Making

This paper introduces a novel multi-armed bandits framework, termed Contextual Restless Bandits (CRB), for complex online decision-making. This CRB framework incorporates the core features of contextual bandits and restless bandits, so that it can model both the internal state transitions of each arm and the influence of external global environmental contexts. Using the dual decomposition method, we develop a scalable index policy algorithm for solving the CRB problem, and theoretically analyze the asymptotical optimality of this algorithm. In the case when the arm models are unknown, we further propose a model-based online learning algorithm based on the index policy to learn the arm models and make decisions simultaneously. Furthermore, we apply the proposed CRB framework and the index policy algorithm specifically to the demand response decision-making problem in smart grids. The numerical simulations demonstrate the performance and efficiency of our proposed CRB approaches.

cs.AI↗

arXiv2023

Minimizing Moments of AoI for Both Active and Passive Users through Second-Order Analysis

In this paper, we address the optimization problem of moments of Age of Information (AoI) for active and passive users in a random access network. In this network, active users broadcast sensing data while passive users only receive signals. Collisions occur when multiple active users transmit simultaneously, and passive users are unable to receive signals while any active user is transmitting. Each active user follows a Markov process for their transmissions. We aim to minimize the weighted sum of any moments of AoI for both active and passive users in this network. To achieve this, we employ a second-order analysis to analyze the system. Specifically, we characterize an active user's transmission Markov process by its mean and temporal process. We show that any moment of the AoI can be expressed a function of the mean and temporal variance, which, in turn, enables us to derive the optimal transmission Markov process. Our simulation results demonstrate that this proposed strategy outperforms other baseline policies that use different active user transmission models.

cs.IT↗

arXiv2023

Dynamic Regret of Randomized Online Service Caching in Edge Computing

This paper studies an online service caching problem, where an edge server, equipped with a prediction window of future service request arrivals, needs to decide which services to host locally subject to limited storage capacity. The edge server aims to minimize the sum of a request forwarding cost (i.e., the cost of forwarding requests to remote data centers to process) and a service instantiating cost (i.e., that of retrieving and setting up a service). Considering request patterns are usually non-stationary in practice, the performance of the edge server is measured by dynamic regret, which compares the total cost with that of the dynamic optimal offline solution. To solve the problem, we propose a randomized online algorithm with low complexity and theoretically derive an upper bound on its expected dynamic regret. Simulation results show that our algorithm significantly outperforms other state-of-the-art policies in terms of the runtime and expected total cost.

cs.NI↗

arXiv2022

DeepTOP: Deep Threshold-Optimal Policy for MDPs and RMABs

We consider the problem of learning the optimal threshold policy for control problems. Threshold policies make control decisions by evaluating whether an element of the system state exceeds a certain threshold, whose value is determined by other elements of the system state. By leveraging the monotone property of threshold policies, we prove that their policy gradients have a surprisingly simple expression. We use this simple expression to build an off-policy actor-critic algorithm for learning the optimal threshold policy. Simulation results show that our policy significantly outperforms other reinforcement learning algorithms due to its ability to exploit the monotone property. In addition, we show that the Whittle index, a powerful tool for restless multi-armed bandit problems, is equivalent to the optimal threshold policy for an alternative problem. This observation leads to a simple algorithm that finds the Whittle index by learning the optimal threshold policy in the alternative problem. Simulation results show that our algorithm learns the Whittle index much faster than several recent studies that learn the Whittle index through indirect means.

cs.LG↗

arXiv2022

Online Service Caching and Routing at the Edge with Unknown Arrivals

This paper studies a problem of jointly optimizing two important operations in mobile edge computing without knowing future requests, namely service caching, which determines which services to be hosted at the edge, and service routing, which determines which requests to be processed locally at the edge. We aim to address several practical challenges, including limited storage and computation capacities of edge servers and unknown future request arrival patterns. To this end, we formulate the problem as an online optimization problem, in which the objective function includes costs of forwarding requests, processing requests, and reconfiguring edge servers. By leveraging a natural timescale separation between service routing and service caching, namely, the former happens faster than the latter, we propose an online two-stage algorithm and its randomized variant. Both algorithms have low complexity, and our fractional solution achieves sublinear regret. Simulation results show that our algorithms significantly outperform other state-of-the-art online policies.

cs.NI↗