arXiv subjects

Lorenzo Sabug Jr.

Publications and source records attributed to Lorenzo Sabug Jr..

4 recordsLinked to original sources

arXiv2025

Rethinking Physics-Informed Regression Beyond Training Loops and Bespoke Architectures

We revisit the problem of physics-informed regression, and propose a method that directly computes the state at the prediction point, simultaneously with the derivative and curvature information of the existing samples. We frame each prediction as a constrained optimisation problem, leveraging multivariate Taylor series expansions and explicitly enforcing physical laws. Each individual query can be processed with low computational cost without any pre- or re-training, in contrast to global function approximator-based solutions such as neural networks. Our comparative benchmarks on a reaction-diffusion system show competitive predictive accuracy relative to a neural network-based solution, while completely eliminating the need for long training loops, and remaining robust to changes in the sampling layout.

math.OC↗

arXiv2025

SAGE: A Set-based Adaptive Gradient Estimator

A new paradigm to estimate the gradient of a black-box scalar function is introduced, considering it as a member of a set of admissible gradients that are computed using existing function samples. Results on gradient estimate accuracy, derived from a multivariate Taylor series analysis, are used to express the set of admissible gradients through linear inequalities. An approach to refine this gradient estimate set to a desired precision is proposed as well, using an adaptive sampling approach. The resulting framework allows one to estimate gradients from data sets affected by noise with finite bounds, to provide the theoretical best attainable gradient estimate accuracy, and the optimal sampling distance from the point of interest to achieve the best refinement of the gradient set estimates. Using these results, a new algorithm is proposed, named Set-based Adaptive Gradient Estimator (SAGE), which features both sample efficiency and robustness to noise. The performance of SAGE are demonstrated by comparing it with commonly-used and latest gradient estimators from literature and practice, in the context of numerical optimization with a first-order method. The results of an extensive statistical test show that SAGE performs competitively when faced with noiseless data, and emerges as the best method when faced with high noise bounds where other gradient estimators result in large errors.

math.OC↗

arXiv2022

SMGO-$Δ$: Balancing Caution and Reward in Global Optimization with Black-Box Constraints

In numerous applications across all science and engineering areas, there are optimization problems where both the objective function and the constraints have no closed-form expression or are too complex to be managed analytically, that they can only be evaluated through experiments. To address such issues, we design a global optimization technique for problems with black-box objective and constraints. Assuming Lipschitz continuity of the cost and constraint functions, a Set Membership framework is adopted to build a surrogate model of the optimization program, that is used for exploitation and exploration routines. The resulting algorithm, named Set Membership Global Optimization With Black-Box Constraints (SMGO-$Δ$), features one tunable risk parameter, which the user can intuitively adjust to trade-off safety, exploitation, and exploration. The theoretical properties of the algorithm are derived, and the optimization performance is compared with representative techniques from the literature in several benchmarks. Lastly, it is tested and compared with constrained Bayesian optimization in a case study pertaining to model predictive control tuning for a servomechanism with disturbances and plant uncertainties, addressing practically-motivated task-level constraints.

math.OC↗

arXiv2021

SMGO: A Set Membership Approach to Data-Driven Global Optimization

Many science and engineering applications feature non-convex optimization problems where the objective function can not be handled analytically, i.e. it is a black box. Examples include design optimization via experiments, or via costly finite elements simulations. To solve these problems, global optimization routines are used. These iterative techniques must trade-off exploitation close to the current best point with exploration of unseen regions of the search space. In this respect, a new global optimization strategy based on a Set Membership (SM) framework is proposed. Assuming Lipschitz continuity of the cost function, the approach employs SM concepts to decide whether to switch from an exploitation mode to an exploration one, and vice-versa. The resulting algorithm, named SMGO (Set Membership Global Optimization) is presented. Theoretical properties regarding convergence and computational complexity are derived, and implementation aspects are discussed. Finally, the SMGO performance is evaluated on a set of benchmark non-convex problems and compared with those of other global optimization approaches.

math.OC↗