arXiv subjects

Manali Dutta

Publications and source records attributed to Manali Dutta.

6 recordsLinked to original sources

arXiv2026

Optimal Scheduling for Remote State Estimation over Hybrid Channels

We study optimal scheduling for remote state estimation over a network with two heterogeneous communication channels: a fast but unreliable channel and a slow but reliable channel. To capture temporal correlations in packet losses, we model the unreliable channel as a Gilbert-Elliott (GE) channel. The remote estimation setup consists of a source, a sensor, and a remote estimator. The source evolves as a discrete-time autoregressive (AR) process, and the sensor decides at each time whether to use the fast unreliable channel or the slow reliable channel. We formulate the scheduling problem faced by the sensor as a Markov decision process (MDP) with a continuous state-space and consider minimizing the infinite horizon average cost criterion, where the cost consists of the squared estimation error and the transmission energy consumed. We establish the existence of an optimal stationary policy. We then characterize the structure of an optimal policy, and show that it has a threshold structure with respect to the estimation error. An optimal policy chooses from amongst the two channels based on whether the error exceeds certain thresholds, where the threshold value depends upon the GE channel state. When the system parameters are unknown, we propose an actor-critic (AC) learning algorithm that exploits the threshold structure of an optimal policy. Numerical results demonstrate that the proposed AC algorithm learns the policy structure effectively and achieves performance close to that of the optimal policy computed using the relative value iteration (RVI).

math.OC↗

arXiv2025

Jointly Optimal Policies for Remote Estimation of Autoregressive Markov Processes over Time-Correlated Fading Channel

We study a remote estimation setup with an autoregressive (AR) Markov process, a sensor, and a remote estimator. The sensor observes the process and sends encoded observations to the estimator as packets over an unreliable communication channel modeled as the Gilbert-Elliot (GE) channel. We assume that the sensor gets to observe the channel state by the ACK/NACK feedback mechanism only when it attempts a transmission while it does not observe the channel state when no transmission attempt is made. The objective is to design a transmission scheduling strategy for the sensor, and an estimation strategy for the estimator that are jointly optimal, i.e., they minimize the expected value of an infinite-horizon cumulative discounted cost defined as the sum of squared estimation error over time and the sensor's transmission power. Since the sensor and the estimator have access to different information sets, this constitutes a decentralized stochastic control problem. We formulate this problem as a partially observed Markov decision process (POMDP) and show the existence of jointly optimal transmission and estimation strategies that have a simple structure. More specifically, an optimal transmission strategy exhibits a threshold structure, i.e., the sensor attempts a transmission only when its belief about the channel being in a good state exceeds a threshold that depends on a certain error. Moreover, an optimal estimation strategy follows a `Kalman-like' update rule. When the channel parameters are unknown, we exploit this structure to design an actor-critic reinforcement learning algorithm that converges to a locally optimal policy. Simulations show the learned policy performs close to a globally optimal one, with about a 5.5% average relative gap across evaluated parameters.

math.OC↗

arXiv2025

Delay-Optimal Transmission Scheduling Policies for Time-Correlated Fading Channels

Millimeter-wave (mmWave) networks have the potential to support high throughput and low-latency requirements of 5G-and-beyond communication standards. But transmissions in this band are highly vulnerable to attenuation and blockages from humans, buildings, and foliage, which increase end-to-end packet delays. This work designs dynamic scheduling policies that minimize end-to-end packet delays while keeping packet transmission costs low. Specifically, we consider a mmWave network that consists of a transmitter that transmits data packets over an unreliable communication channel modeled as a Gilbert-Elliott channel.The transmitter operates under an ACK/NACK feedback model and does not observe the channel state unless it attempts a transmission. The objective is to minimize a weighted average cost consisting of end-to-end packet delays and packet transmission costs. We pose this dynamic optimization problem as a partially observable Markov decision process (POMDP). To the best of our knowledge, this is the first POMDP formulation for mmWave network with partial channel state information that considers delay minimization. We show that the POMDP admits a solution that has a threshold structure, i.e., for each queue length, the belief (the conditional probability that the channel is in a good state) is partitioned into intervals, and the transmitter sends j packets when the belief lies in the j-th interval. We then consider the case when the system parameters such as the packet arrival rate, and the transition probabilities of the channel are not known, and leverage these structural results in order to use the actor-critic algorithm to efficiently search for a policy that is locally optimal.

math.OC↗

arXiv2024

Optimal Scheduling of Uplink-Downlink Networked Control Systems with Energy Harvesting Sensor

In this work, we consider a wireless networked control system (WNCS) consisting of a plant, a battery-operated sensor, a controller, and an actuator. The battery in the sensor harvests energy from the environment. The sensor then uses this energy for packet transmissions. There are two types of wireless communication channels, (i) sensor--controller channel (also called uplink channel), and (ii) controller--actuator channel (also called downlink channel). The controller is \emph{half-duplex}, and this prevents it from simultaneously receiving an update from the sensor, and also transmitting a control packet to the actuator. Though frequent transmissions via uplink channel improve controller's estimate of the plant state, but this also reduces the timely control of the plant. Hence, in order to strike a balance between these two, we consider the problem of designing an optimal scheduling policy that minimizes the expected cumulative infinite horizon discounted cost, where the instantaneous cost is equal to the square of the plant state. At each time $t$, the scheduler at the sensor has to decide whether it should activate the uplink channel, or downlink. We pose this dynamic optimization problem as a Markov decision process (MDP), in which the state at time $t$ is composed of (i) the plant state $x(t)$, (ii) the age of the data packet available at the controller, denoted by $\tau(t)$, (iii) a binary variable $y(t)$ which indicates the availability of a control packet at the controller, and (iv) the energy level of the battery at the sensor $b(t)$. We show that there exists an optimal scheduling policy that exhibits a threshold structure, meaning that for each time $t$, if there is a control packet available with the controller, then the sensor activates the downlink channel in case $|x(t)|$ exceeds a threshold $x\ust(\tau(t),b(t))$.

math.OC↗

arXiv2024

Optimal Risk-Sensitive Scheduling Policies for Remote Estimation of Autoregressive Markov Processes

We design scheduling policies that minimize a risk-sensitive cost criterion for a remote estimation setup. Since risk-sensitive cost objective takes into account not just the mean value of the cost, but also higher order moments of its probability distribution, the resulting policy is robust to changes in the underlying system's parameters. The setup consists of a sensor that observes a discrete-time autoregressive Markov process, and at each time $t$ decides whether or not to transmit its observations to a remote estimator using an unreliable wireless communication channel after encoding these observations into data packets. We model the communication channel as a Gilbert-Elliott channel \cite{10384144}. Sensor probes the channel \cite{laourine2010betting} and hence knows the channel state at each time $t$ before making scheduling decision. The scheduler has to minimize the expected value of the exponential of the finite horizon cumulative cost that is sum of the following two quantities (i) the cumulative transmission power consumed, (ii) the cumulative squared estimator error. We pose this dynamic optimization problem as a Markov decision process (MDP), in which the system state at time $t$ is composed of (i) the instantaneous error $\Delta(t):= x(t)-a\hat{x}(t-1)$, where $x(t),\hat{x}(t-1)$ are the system state and the estimate at time $t,t-1$ respectively, and (ii) the channel state $c(t)$. We show that there exists an optimal policy that has a threshold structure, i.e., at each time $t$, for each possible channel state $c$, there is a threshold $\D\ust(c)$ such that if the current channel state is $c$, then it transmits only when the error $\D(t)$ exceeds $\D\ust(c)$.

math.OC↗

arXiv2023

Optimal Scheduling Policies for Remote Estimation of Autoregressive Markov Processes over Time-Correlated Fading Channel

We consider the problem of transmission scheduling for the remote estimation of a discrete-time autoregressive Markov process that is driven by white Gaussian noise. A sensor observes this process, and then decides to either encode the current state of this process into a data packet and attempts to transmit it to the estimator over an unreliable wireless channel modeled as a Gilbert-Elliott channel, or does not send any update. Each transmission attempt consumes $\lambda$ units of transmission power, and the remote estimator is assumed to be linear. The channel state is revealed only via the feedback (ACK\slash NACK) of a transmission, and hence the channel state is not revealed if no transmission occurs. The goal of the scheduler is to minimize the expected value of an infinite-horizon cumulative discounted cost, in which the instantaneous cost is composed of the following two quantities: (i)~squared estimation error, (ii) transmission power. We show that this problem can equivalently be posed as a partially observable Markov decision process (POMDP), in which the scheduler maintains a belief about the current state of the channel, and makes decisions on the basis of the current value of the estimation error, and the belief state.~We then show that the optimal policy is of threshold-type, i.e. for each value of the estimation error $e$, there is a threshold $b\ust(e)$ such that when the error is equal to $e$, then it is optimal to transmit only when the current belief state is greater than $b\ust(e)$.

math.OC↗