Optimal Sampling and Actuation for Real-Time Monitoring of Markov Sources

Abstract

This paper studies efficient data management and timely information dissemination for real-time monitoring of an N-state Markov process, with the objective of enabling accurate state estimation and reliable actuation decisions. We analyze the real-time reconstruction error and the Age of Incorrect Information (AoII), and derive closed-form expressions for their time-averaged values under several sampling and transmission policies. We then formulate and solve constrained optimization problems to minimize the time-averaged reconstruction error and the average AoII under a time-averaged sampling frequency constraint. The resulting optimal sampling and transmission policies are compared to identify the conditions under which each policy is most effective. We further show that directly using the reconstructed state for actuation can degrade system performance, especially when the receiver is uncertain about the state estimate or when actuation is costly. These findings reveal that accurate state estimation alone does not necessarily lead to effective actuation, highlighting the importance of incorporating uncertainty into the decision-making process. To address this issue, we introduce a cost function, termed the Cost of Actions under Uncertainty (CoAU), which characterizes correct and incorrect actuation decisions under receiver-side uncertainty. We propose a randomized actuation policy and derive a closed-form expression for the probability of a correct actuation decision, defined as the event in which the CoAU equals zero. Finally, we formulate an optimization problem to find the optimal randomized actuation policy that maximizes this probability. The results show that the resulting policy substantially reduces incorrect actuator actions.