arXiv subjects

Nikolaos P. Bakas

Publications and source records attributed to Nikolaos P. Bakas.

5 recordsLinked to original sources

arXiv2023

Algorithmic Solution for Systems of Linear Equations, in $\mathcal{O}(mn)$ time

We present a novel algorithm attaining excessively fast, the sought solution of linear systems of equations. The algorithm is short in its basic formulation and, by definition, vectorized, while the memory allocation demands are trivial, because, for each iteration, only one dimension of the given input matrix $\mathbf X$ is utilized. The execution time is very short compared with state-of-the-art methods, exhibiting $> \times 10^2$ speed-up and low memory allocation demands, especially for non-square Systems of Linear Equations, with ratio of equations versus features high (tall systems), or low (wide systems) accordingly. The accuracy is high and straightforwardly controlled, and the numerical results highlight the efficiency of the proposed algorithm, in terms of computation time, solution accuracy and memory demands. The paper also comprises a theoretical proof for the algorithmic convergence, and we extend the implementation of the proposed algorithmic rationale to feature selection tasks.

cs.LG↗

arXiv2020

A Gradient Free Neural Network Framework Based on Universal Approximation Theorem

We present a numerical scheme for computation of Artificial Neural Networks (ANN) weights, which stems from the Universal Approximation Theorem, avoiding laborious iterations. The proposed algorithm adheres to the underlying theory, is highly fast, and results in remarkably low errors when applied for regression and classification of complex data-sets, such as the Griewank function of multiple variables $\mathbf{x} \in \mathbb{R}^{100}$ with random noise addition, and MNIST database for handwritten digits recognition, with $7\times10^4$ images. The same mathematical formulation is found capable of approximating highly nonlinear functions in multiple dimensions, with low errors (e.g. $10^{-10}$) for the test-set of the unknown functions, their higher-order partial derivatives, as well as numerically solving Partial Differential Equations. The method is based on the calculation of the weights of each neuron in small neighborhoods of the data, such that the corresponding local approximation matrix is invertible. Accordingly, optimization of hyperparameters is not necessary, as the number of neurons stems directly from the dimensionality of the data, further improving the algorithmic speed. Under this setting, overfitting is inherently avoided, and the results are interpretable and reproducible. The complexity of the proposed algorithm is of class P with $\mathcal{O}(mn^2)+\mathcal{O}(\frac{m^3}{n^2})-\mathcal{O}(\log(n+1))$ computing time, with respect to the observations $m$ and features $n$, in contrast with the NP-Complete class of standard algorithms for ANN training. The performance of the method is high, irrespective of the size of the dataset, and the test-set errors are similar or smaller than the training errors, indicating the generalization efficiency of the algorithm.

cs.LG↗

arXiv2020

The Lagrangian remainder of Taylor's series, distinguishes $\mathcal{O}(f(x))$ time complexities to polynomials or not

The purpose of this letter is to investigate the time complexity consequences of the truncated Taylor series, known as Taylor Polynomials \cite{bakas2019taylor,Katsoprinakis2011,Nestoridis2011}. In particular, it is demonstrated that the examination of the $\mathbf{P=NP}$ equality, is associated with the determination of whether the $n^{th}$ derivative of a particular solution is bounded or not. Accordingly, in some cases, this is not true, and hence in general.

cs.CC↗

arXiv2020

ITSO: A novel Inverse Transform Sampling-based Optimization algorithm for stochastic search

Optimization algorithms appear in the core calculations of numerous Artificial Intelligence (AI) and Machine Learning methods, as well as Engineering and Business applications. Following recent works on the theoretical deficiencies of AI, a rigor context for the optimization problem of a \textit{black-box} objective function is developed. The algorithm stems directly from the theory of probability, instead of a presumed inspiration, thus the convergence properties of the proposed methodology are inherently stable. In particular, the proposed optimizer utilizes an algorithmic implementation of the $n$-dimensional inverse transform sampling as a search strategy. No control parameters are required to be tuned, and the trade-off among exploration and exploitation is by definition satisfied. A theoretical proof is provided, concluding that only falling into the proposed framework, either directly or incidentally, any optimization algorithm converges in the fastest possible time. The numerical experiments, verify the theoretical results on the efficacy of the algorithm apropos reaching the optimum, as fast as possible.

math.OC↗

arXiv2019

Taylor Polynomials in High Arithmetic Precision as Universal Approximators

Function approximation is a generic process in a variety of computational problems, from data interpolation to the solution of differential equations and inverse problems. In this work, a unified approach for such techniques is demonstrated, by utilizing partial sums of Taylor series in high arithmetic precision. In particular, the proposed method is capable of interpolation, extrapolation, numerical differentiation, numerical integration, solution of ordinary and partial differential equations, and system identification. The method is based on the utilization of Taylor polynomials, by exploiting some hundreds of computer digits, resulting in highly accurate calculations. Interestingly, some well-known problems were found to reason by calculations accuracy, and not methodological inefficiencies, as supposed. In particular, the approximation errors are precisely predictable, the Runge phenomenon is eliminated and the extrapolation extent may a-priory be anticipated. The attained polynomials offer a precise representation of the unknown system as well as its radius of convergence, which provide a rigor estimation of the prediction ability. The approximation errors have comprehensively been analyzed, for a variety of calculation digits and test problems.

math.NA↗