Towards Reliable Neural Optimizers: A Permutation Equivariant Neural Approximation for Information Processing Applications

Abstract

The complexities of information processing across Dynamic Data Driven Applications Systems drive the development and adoption of Artificial Intelligence-based optimization solutions. Traditional solvers often suffer from slow response times and an inability to adapt swiftly to real-time input variations. To address these deficiencies, we will expand on our previous research in neural-based optimizers by introducing a machine learning-enabled neural approximation model called LOOP-PE (Learning to Optimize the Optimization Process -- Permutation Equivariance version). This model not only enhances decision-making efficiency but also dynamically adapts to variations of data collections from sensor networks. In this work, we focus on mitigating the heterogeneity issues of data collection from sensor networks, including sensor dropout and failures, communication delays, and the complexities involved in integrating new sensors during system scaling. The proposed LOOP-PE model specifically overcomes these issues with a unique structure that is permutation equivariant, allowing it to accommodate inputs from a varying number of sensors and directly linking these inputs to their optimal operational outputs. This design significantly boosts the system's flexibility and adaptability, especially in scenarios characterized by unordered, distributed, and asynchronous data collections. Moreover, our approach increases the robustness of decision-making by integrating physical constraints through the generalized gauge map method, which theoretically ensures the decisions' practical feasibility and operational viability under dynamic conditions. We use a DDDAS case study to demonstrate that LOOP-PE model reliably delivers near-optimal and adaptable solutions, significantly outperforming traditional methods in managing the complexities of multi-sensor environments for real-time deployments.