arXiv subjects

Nataša Krklec Jerinkić

Publications and source records attributed to Nataša Krklec Jerinkić.

10 recordsLinked to original sources

arXiv2026

IPAS: An Adaptive Sample Size Method for Weighted Finite Sum Problems with Linear Equality Constraints

Optimization problems with the objective function in the form of weighted sum and linear equality constraints are considered. Given that the number of local cost functions can be large as well as the number of constraints, a stochastic optimization method is proposed. The method belongs to the class of variable sample size first order methods, where the sample size is adaptive and governed by the additional sampling technique earlier proposed in the unconstrained optimization framework. The resulting algorithm may be a mini-batch method, increasing sample size method, or even deterministic in a sense that it eventually reaches the full sample size, depending on the problem and similarity of the local cost functions. Regarding the constraints, the method uses controlled, but inexact projections on the feasible set, yielding possibly infeasible iterates. Almost sure convergence is proved under some standard assumptions for the stochastic framework, without imposing the convexity. Numerical results on relevant machine learning experiments, i.e., real-world data sets for logistic regression problems, show that the proposed algorithm is competitive with the state-of-the-art methods.

math.OC↗

arXiv2026

SMOP: Stochastic trust region method for multi-objective problems

The problem we consider is a multi-objective optimization problem, in which the goal is to find an optimal value of a vector function representing various criteria. The aim of this work is to develop an algorithm which utilizes the trust region framework with probabilistic model functions, able to cope with noisy problems, using inaccurate functions and gradients. The key novelty is approximation of each function in the multiobjective problem with probabilistically fully linear model which yields the composite model defined by max operator as a satisfactory approximation for the nonsmooth scalarized objective function. We prove the almost sure convergence of the proposed algorithm to a Pareto critical point. Numerical results demonstrate effectiveness of the probabilistic trust region by comparing it to competitive stochastic multi-objective solvers. The application in supervised machine learning is showcased by training non discriminatory Logistic Regression models on different size data groups. Additionally, we use several test examples with irregularly shaped fronts to exhibit the efficiency of the algorithm

math.OC↗

arXiv2026

ASMOP: Additional sampling stochastic trust region method for multi-objective problems

We consider unconstrained multi-criteria optimization problems with finite sum objective functions. The proposed algorithm belongs to a non-monotone trust region framework where additional sampling approach is used to govern the sample size and the acceptance of a candidate point. Depending on the problem, the method can yield a mini-batch or an increasing sample size behavior. This work can be viewed as an extension of additional sampling trust region method for scalar finite sum function minimization presented in the literature, requiring nontrivial modifications both in construction and in convergence analysis of the algorithm. Under assumptions standard for this framework, we prove stochastic convergence for twice continuously-differentiable, but possibly non-convex objective functions. The experiments on machine learning binary classification datasets show the efficiency of the proposed scheme and its competitiveness with the relevant state-of-the-art methods in both convex and non-convex setup.

math.OC↗

arXiv2025

AS-BOX: Additional Sampling Method for Weighted Sum Problems with Box Constraints

A class of optimization problems characterized by a weighted finite-sum objective function subject to box constraints is considered. We propose a novel stochastic optimization method, named AS-BOX (\text{A}ddi\-ti\-onal \text{S}ampling for \text{BOX} constraints), that combines projected gradient directions with adaptive variable sample size strategies and nonmonotone line search. The method dynamically adjusts the batch size based on progress with respect to the additional sampling function and on structural consistency of the projected direction, enabling practical adaptivity of AS-BOX, while ensuring theoretical support. We establish almost sure convergence under standard assumptions and provide complexity bounds. Numerical experiments demonstrate the efficiency and competitiveness of the proposed method compared to state-of-the-art algorithms.

math.OC↗

arXiv2025

ASPEN: An Additional Sampling Penalty Method for Finite-Sum Optimization Problems with Nonlinear Equality Constraints

We propose a novel algorithm for solving non-convex, nonlinear equality-constrained finite-sum optimization problems. The proposed algorithm incorporates an additional sampling strategy for sample size update into the well-known framework of quadratic penalty methods. Thus, depending on the problem at hand, the resulting method may exhibit a sample size strategy ranging from a mini-batch on one end, to increasing sample size that achieves the full sample eventually, on the other end of the spectrum. A non-monotone line search is used for the step size update, while the penalty parameter is also adaptive. The proposed algorithm avoids costly projections, which, together with the sample size update, may yield significant computational cost savings. Also, the proposed method can be viewed as a transition of an additional sampling approach for unconstrained and linear constrained problems, to a more general class with non-linear constraints. The almost sure convergence is proved under a standard set of assumptions for this framework, while numerical experiments on both academic and real-data based machine learning problems demonstrate the effectiveness of the proposed approach.

math.OC↗

arXiv2024

AN-SPS: Adaptive Sample Size Nonmonotone Line Search Spectral Projected Subgradient Method for Convex Constrained Optimization Problems

We consider convex optimization problems with a possibly nonsmooth objective function in the form of a mathematical expectation. The proposed framework (AN-SPS) employs Sample Average Approximations (SAA) to approximate the objective function, which is either unavailable or too costly to compute. The sample size is chosen in an adaptive manner, which eventually pushes the SAA error to zero almost surely (a.s.). The search direction is based on a scaled subgradient and a spectral coefficient, both related to the SAA function. The step size is obtained via a nonmonotone line search over a predefined interval, which yields a theoretically sound and practically efficient algorithm. The method retains feasibility by projecting the resulting points onto a feasible set. The a.s. convergence of AN-SPS method is proved without the assumption of a bounded feasible set or bounded iterates. Preliminary numerical results on Hinge loss problems reveal the advantages of the proposed adaptive scheme. In addition, a study of different nonmonotone line search strategies in combination with different spectral coefficients within AN-SPS framework is also conducted, yielding some hints for future work.

math.OC↗

arXiv2024

SLiSeS: Subsampled Line Search Spectral Gradient Method for Finite Sums

The spectral gradient method is known to be a powerful low-cost tool for solving large-scale optimization problems. In this paper, our goal is to exploit its advantages in the stochastic optimization framework, especially in the case of mini-batch subsampling that is often used in big data settings. To allow the spectral coefficient to properly explore the underlying approximate Hessian spectrum, we keep the same subsample for several iterations before subsampling again. We analyze the required algorithmic features and the conditions for almost sure convergence, and present initial numerical results that show the advantages of the proposed method.

math.OC↗

arXiv2022

LSOS: Line-search Second-Order Stochastic optimization methods for nonconvex finite sums

We develop a line-search second-order algorithmic framework for minimizing finite sums. We do not make any convexity assumptions, but require the terms of the sum to be continuously differentiable and have Lipschitz-continuous gradients. The methods fitting into this framework combine line searches and suitably decaying step lengths. A key issue is a two-step sampling at each iteration, which allows us to control the error present in the line-search procedure. Stationarity of limit points is proved in the almost-sure sense, while almost-sure convergence of the sequence of approximations to the solution holds with the additional hypothesis that the functions are strongly convex. Numerical experiments, including comparisons with state-of-the art stochastic optimization methods, show the efficiency of our approach.

math.OC↗

arXiv2022

Group pattern detection of longitudinal data using functional statistics

Estimations and evaluations of the main patterns of time series data in groups benefit large amounts of applications in various fields. Different from the classical auto-correlation time series analysis and the modern neural networks techniques, in this paper we propose a combination of functional analysis of variance (FANOVA) and permutation tests in a more intuitive manner for a limited sample size. First, FANOVA is applied in order to separate the common information and to dig out the additional categorical influence through paired group comparison, the results of which are secondly analyzed through permutation tests to identify the time zones where the means of the different groups differ significantly. Normalized kernel functions of different groups are able to reflect remarkable mean characteristics in grouped unities, also meaningful for deeper interpretation and group-wise classification. In order to learn whether and when the proposed method of FANOVA and permutation F-test works precisely and efficiently, we compare the estimated kernel results with the ground truth on simulated data. After the confirmation of the model's efficiency from simulation, we apply it also to the RAVDESS facial dataset to extract the emotional behaviors of humans based on facial muscles contractions (so-called action units (AU) technically in computer graphics), by comparing the neutral performances with emotional ones.

stat.AP↗

arXiv2019

Subsampled Nonmonotone Spectral Gradient Methods

This paper deals with subsampled spectral gradient methods for minimizing finite sum. Subsample function and gradient approximations are employed in order to reduce the overall computational cost of the classical spectral gradient methods. The global convergence is enforced by a nonmonotone line search procedure. Global convergence is proved when functions and gradients are approximated with increasing accuracy. R-linear convergence and worst-case iteration complexity is investigated in case of strongly convex objective function. Numerical results on well known binary classification problems are given to show the effectiveness of this framework and analyze the effect of different spectral coefficient approximations arising from variable sample nature of this procedure.

math.NA↗