arXiv subjects

C G Krishnanunni

Publications and source records attributed to C G Krishnanunni.

4 recordsLinked to original sources

arXiv2026

An optimal control approach for neural network architecture adaptation with a posteriori error estimation

This work presents a novel approach for adapting neural network architecture along the depth based on a posteriori error estimation. By formulating neural network training as a continuous-time optimal control problem, we derive rigorous error estimates that quantify how approximation error distributes across network layers. This error decomposition enables a principled depth adaptation strategy: new layers are inserted at locations of maximum estimated error, allowing the network to efficiently capture complex, nonlinear variations in the underlying problem. Our framework introduces a novel network architecture that treats weights and biases as piecewise linear functions varying across layers, with the error estimator bounding the discrepancy between this discrete representation and the true continuous optimal control solution. The approach leverages dual weighted residual methodology from finite element analysis to derive computable upper bounds on the functional error. A key theoretical contribution is the derivation of explicit error bounds that decompose the total approximation error into interval-wise contributions, providing a rigorous basis for targeted architecture refinement. We demonstrate the effectiveness of our method on scientific datasets, including learning the observable-to-parameter map for the Navier-Stokes equation. Numerical results reveal that our approach consistently outperforms existing architecture adaptation methods in terms of generalization performance.

cs.LG↗

arXiv2026

Topological derivative approach for deep neural network architecture adaptation

This work presents a novel algorithm for progressively adapting neural network architecture along the depth. In particular, we attempt to address the following questions in a mathematically principled way: i) Where to add a new capacity (layer) during the training process? ii) How to initialize the new capacity? At the heart of our approach are two key ingredients: i) the introduction of a ``shape functional" to be minimized, which depends on neural network topology, and ii) the introduction of a topological derivative of the shape functional with respect to the neural network topology. Using an optimal control viewpoint, we show that the network topological derivative exists under certain conditions, and its closed-form expression is derived. In particular, we explore, for the first time, the connection between the topological derivative from a topology optimization framework with the Hamiltonian from optimal control theory. Further, we show that the optimality condition for the shape functional leads to an eigenvalue problem for deep neural architecture adaptation. Our approach thus determines the most sensitive location along the depth where a new layer needs to be inserted during the training phase and the associated parametric initialization for the newly added layer. We also demonstrate that our layer insertion strategy can be derived from an optimal transport viewpoint as a solution to maximizing a topological derivative in $p$-Wasserstein space, where $p>= 1$. Numerical investigations with fully connected network, convolutional neural network, and vision transformer on various regression and classification problems demonstrate that our proposed approach can outperform an ad-hoc baseline network and other architecture adaptation strategies. Further, we also demonstrate other applications of topological derivative in fields such as transfer learning.

cs.LG↗

arXiv2026

A New Look at the Ensemble Kalman Filter for Inverse Problems: Duality, Non-Asymptotic Analysis and Convergence Acceleration

This work presents new results and understanding of the Ensemble Kalman filter (EnKF) for inverse problems. In particular, using a Lagrangian dual perspective we show that EnKF can be derived from the sample average approximation (SAA) of the Lagrangian dual function. The beauty of this new duality perspective is that it facilitates us to prove and numerically verify a novel non-asymptotic convergence result for the EnKF. Motivated by the new perspective, we also present a new convergence improvement strategy for the Ensemble Kalman Inversion Algorithm (EnKI), which is an iterative version of the EnKF for inverse problems. In particular, we propose an adaptive multiplicative correction to the sample covariance matrix at each iteration and we call this new algorithm as EnKI-MC (I). Based on the new duality perspective, we derive an expression for the optimal correction factor at each iteration of the EnKI algorithm to accelerate the convergence. In addition, we also consider an ensemble specific multiplicative covariance correction strategy (EnKI-MC (II)) where a different correction is employed for each ensemble. By viewing EnKI through the lens of fixed-point iteration, we also provide theoretical results that guarantees the convergence of EnKI-MC (I) algorithm. Numerical investigations for the deconvolution problem, initial condition inversion in advection-convection problem, initial condition inversion in a Lorenz 96 model, and inverse problem constrained by elliptic partial differential equation are conducted to verify the non-asymptotic results for EnKF and to assess the performance of convergence improvement strategies for EnKI. The numerical results suggest that the proposed strategies for EnKI not only led to faster convergence in comparison to the currently employed techniques but also better quality solutions at termination of the algorithm.

math.NA↗

arXiv2024

An Adaptive and Stability-Promoting Layerwise Training Approach for Sparse Deep Neural Network Architecture

This work presents a two-stage adaptive framework for progressively developing deep neural network (DNN) architectures that generalize well for a given training data set. In the first stage, a layerwise training approach is adopted where a new layer is added each time and trained independently by freezing parameters in the previous layers. We impose desirable structures on the DNN by employing manifold regularization, sparsity regularization, and physics-informed terms. We introduce a epsilon-delta stability-promoting concept as a desirable property for a learning algorithm and show that employing manifold regularization yields a epsilon-delta stability-promoting algorithm. Further, we also derive the necessary conditions for the trainability of a newly added layer and investigate the training saturation problem. In the second stage of the algorithm (post-processing), a sequence of shallow networks is employed to extract information from the residual produced in the first stage, thereby improving the prediction accuracy. Numerical investigations on prototype regression and classification problems demonstrate that the proposed approach can outperform fully connected DNNs of the same size. Moreover, by equipping the physics-informed neural network (PINN) with the proposed adaptive architecture strategy to solve partial differential equations, we numerically show that adaptive PINNs not only are superior to standard PINNs but also produce interpretable hidden layers with provable stability. We also apply our architecture design strategy to solve inverse problems governed by elliptic partial differential equations.

cs.LG↗