Searcharxiv⌕ Search

arXiv subjects

Martin Holeňa

Publications and source records attributed to Martin Holeňa.

7 recordsLinked to original sources
arXiv2026

Artificial Neural Networks as Surrogate Models in Black Box Optimization

Black-Box Optimization (BBO) is often applied in several engineering fields and can utilize an advancement of numerical measure- ments and simulation technologies. It deals with the optimization func- tions, where an analytical description is unavailable. It relies on meth- ods that require only an input point in the search space, paired with its corresponding objective function value, obtained through non-analytical means, e.g., sensors, experiments, or simulations. Common approaches include evolutionary optimization and other metaheuristics. Since BBO methods rely solely on objective function values, they typically require many evaluations, which becomes problematic when evaluating the ob- jective function is time-consuming or expensive. This leads to using surrogate-based optimization which evaluates selected true objective val- ues and trains a regression model to approximate the objective function across the search space. Surrogate-assisted black-box optimization is a small-data learning problem because the optimizer must approximate an expensive objective function from limited evaluations. Surrogate models act as data-efficient regressors, guiding the search toward promising or informative points under a restricted evaluation budget. In this paper, a new surrogate model using artificial neural networks, called Adaptive- Fidelity Nexus Covariance Matrix Adaptation Evolution Strategy (AFN- CMA-ES), is proposed for the selective evaluation of objective functions. The experimental results show its competitive performance compared to state-of-the-art surrogate-assisted BBO methods.

cs.NE↗
arXiv2024

On Difficulties of Attention Factorization through Shared Memory

Transformers have revolutionized deep learning in numerous fields, including natural language processing, computer vision, and audio processing. Their strength lies in their attention mechanism, which allows for the discovering of complex input relationships. However, this mechanism's quadratic time and memory complexity pose challenges for larger inputs. Researchers are now investigating models like Linear Unified Nested Attention (Luna) or Memory Augmented Transformer, which leverage external learnable memory to either reduce the attention computation complexity down to linear, or to propagate information between chunks in chunk-wise processing. Our findings challenge the conventional thinking on these models, revealing that interfacing with the memory directly through an attention operation is suboptimal, and that the performance may be considerably improved by filtering the input signal before communicating with memory.

cs.LG↗
arXiv2023

Video Scene Location Recognition with Neural Networks

This paper provides an insight into the possibility of scene recognition from a video sequence with a small set of repeated shooting locations (such as in television series) using artificial neural networks. The basic idea of the presented approach is to select a set of frames from each scene, transform them by a pre-trained singleimage pre-processing convolutional network, and classify the scene location with subsequent layers of the neural network. The considered networks have been tested and compared on a dataset obtained from The Big Bang Theory television series. We have investigated different neural network layers to combine individual frames, particularly AveragePooling, MaxPooling, Product, Flatten, LSTM, and Bidirectional LSTM layers. We have observed that only some of the approaches are suitable for the task at hand.

cs.CV↗
arXiv2023

Using Artificial Neural Networks to Determine Ontologies Most Relevant to Scientific Texts

This paper provides an insight into the possibility of how to find ontologies most relevant to scientific texts using artificial neural networks. The basic idea of the presented approach is to select a representative paragraph from a source text file, embed it to a vector space by a pre-trained fine-tuned transformer, and classify the embedded vector according to its relevance to a target ontology. We have considered different classifiers to categorize the output from the transformer, in particular random forest, support vector machine, multilayer perceptron, k-nearest neighbors, and Gaussian process classifiers. Their suitability has been evaluated in a use case with ontologies and scientific texts concerning catalysis research. From results we can say the worst results have random forest. The best results in this task brought support vector machine classifier.

cs.NE↗
arXiv2022

Landscape Analysis for Surrogate Models in the Evolutionary Black-Box Context

Surrogate modeling has become a valuable technique for black-box optimization tasks with expensive evaluation of the objective function. In this paper, we investigate the relationship between the predictive accuracy of surrogate models and features of the black-box function landscape. We also study properties of features for landscape analysis in the context of different transformations and ways of selecting the input data. We perform the landscape analysis of a large set of data generated using runs of a surrogate-assisted version of the Covariance Matrix Adaptation Evolution Strategy on the noiseless part of the Comparing Continuous Optimisers benchmark function testbed.

cs.NE↗
arXiv2016

On structure, family and parameter estimation of hierarchical Archimedean copulas

Research on structure determination and parameter estimation of hierarchical Archimedean copulas (HACs) has so far mostly focused on the case in which all appearing Archimedean copulas belong to the same Archimedean family. The present work addresses this issue and proposes a new approach for estimating HACs that involve different Archimedean families. It is based on employing goodness-of-fit test statistics directly into HAC estimation. The approach is summarized in a simple algorithm, its theoretical justification is given and its applicability is illustrated by several experiments, which include estimation of HACs involving up to five different Archimedean families.

stat.ME↗