Pseudo Closed-Loop Bootstrapping for Reinforcement Learning Based AO Control
High contrast imaging with ground-based telescopes requires extremely precise wavefront control. The performance of the controller ultimately depends on the quality and efficiency of the wavefront sensing technique, which has led to the development of ever more sensitive wavefront sensors (WFS). However, the increase in sensitivity comes at the cost of greater nonlinearity, which poses challenges for conventional linear reconstructors and controllers. Reinforcement Learning (RL) is a branch of machine learning in which control policies are learned by interacting with the environment. RL has recently attracted interest in the field of XAO, and previous studies have demonstrated that a model-based RL variant, the Policy Optimization for Adaptive Optics (PO4AO), can effectively compensate for temporal delays, misregistration errors, and moderate WFS nonlinearities. PO4AO learns an initial control strategy by collecting closed-loop data from the integrator controller with a linear reconstructor. However, in cases of highly non-linear WFS and challenging conditions, closing the loop with the integrator is difficult, and high-fidelity data cannot be collected. Consequently, learning robust control with PO4AO is challenging. We propose a robust learning strategy in which PO4AO is pre-calibrated using an internal light source and a DM, enabling direct on-sky closed-loop operation without a stable integrator controller for bootstrapping the initial models.