arXiv subjects

Michael Hellwig

Publications and source records attributed to Michael Hellwig.

7 recordsLinked to original sources

arXiv2025

Benchmarking that Matters: Rethinking Benchmarking for Practical Impact

Benchmarking has driven scientific progress in Evolutionary Computation, yet current practices fall short of real-world needs. Widely used synthetic suites such as BBOB and CEC isolate algorithmic phenomena but poorly reflect the structure, constraints, and information limitations of continuous and mixed-integer optimization problems in practice. This disconnect leads to the misuse of benchmarking suites for competitions, automated algorithm selection, and industrial decision-making, despite these suites being designed for different purposes. We identify key gaps in current benchmarking practices and tooling, including limited availability of real-world-inspired problems, missing high-level features, and challenges in multi-objective and noisy settings. We propose a vision centered on curated real-world-inspired benchmarks, practitioner-accessible feature spaces and community-maintained performance databases. Real progress requires coordinated effort: A living benchmarking ecosystem that evolves with real-world insights and supports both scientific understanding and industrial use.

cs.NE↗

arXiv2025

A Benchmarking Suite for Flexible Job Shop Scheduling Problems with Worker Flexibility under Uncertainty

This paper addresses the Flexible Job Shop Scheduling Problem and its extension with Worker Flexibility, which integrates workforce assignment into machine-operation scheduling. Diverse solvers have been proposed across multiple optimization domains including Mathematical Programming, Constraint Programming, and Simulation-Based Optimization, or Simulation-based Optimization. These are often tailored to narrow use cases and validated on limited test problem sets, hindering cross-domain comparison. To overcome this, a comprehensive benchmarking environment built on 402 standardized Flexible Job Shop Scheduling Problem instances is introduced and systematically extended to include worker flexibility. This creates a hitherto unique collection of ready-to-use worker flexibility instances. The benchmark suite features several metrics for algorithm performance assessment, the visualization of algorithmic results, as well as state-of-the-art baseline results. This enables rigorous, reproducible, and comparable performance analysis between solvers and scheduling problem subdomains. Through the simulation-based integration of uncertainties in processing times as well as resource availabilities, the environment supports the development and evaluation of robust optimization strategies. The present work lays a foundation for targeted algorithm development and consistent performance evaluation in production scheduling research.

cs.NE↗

arXiv2024

Systematic review and characterisation of malicious industrial network traffic datasets

The adoption of the Industrial Internet of Things (IIoT) as a complementary technology to Operational Technology (OT) has enabled a new level of standardised data access and process visibility. This convergence of Information Technology (IT), OT, and IIoT has also created new cybersecurity vulnerabilities and risks that must be managed. Artificial Intelligence (AI) is emerging as a powerful tool to monitor OT/IIoT networks for malicious activity and is a highly active area of research. AI researchers are applying advanced Machine Learning (ML) and Deep Learning (DL) techniques to the detection of anomalous or malicious activity in network traffic. They typically use datasets derived from IoT/IIoT/OT network traffic captures to measure the performance of their proposed approaches. Therefore, there is a widespread need for datasets for algorithm testing. This work systematically reviews publicly available network traffic capture-based datasets, including categorisation of contained attack types, review of metadata, and statistical as well as complexity analysis. Each dataset is analysed to provide researchers with metadata that can be used to select the best dataset for their research question. This results in an added benefit to the community as researchers can select the best dataset for their research more easily and according to their specific Machine Learning goals.

cs.CR↗

arXiv2024

Analyzing design principles for competitive evolution strategies in constrained search spaces

In the context of the 2018 IEEE Congress of Evolutionary Computation, the Matrix Adaptation Evolution Strategy for constrained optimization turned out to be notably successful in the competition on constrained single objective real-parameter optimization. Across all considered instances the so-called $\epsilon$MAg-ES achieved the second rank. However, it can be considered to be the most successful participant in high dimensions. Unfortunately, the competition result does not provide any information about the modus operandi of a successful algorithm or its suitability for problems of a particular shape. To this end, the present paper is concerned with an extensive empirical analysis of the $\epsilon$MAg-ES working principles that is expected to provide insights about the performance contribution of specific algorithmic components. To avoid rankings with respect to insignificant differences within the algorithm realizations, the paper additionally introduces significance testing into the ranking process.

cs.NE↗

arXiv2018

Benchmarking Evolutionary Algorithms For Single Objective Real-valued Constrained Optimization - A Critical Review

Benchmarking plays an important role in the development of novel search algorithms as well as for the assessment and comparison of contemporary algorithmic ideas. This paper presents common principles that need to be taken into account when considering benchmarking problems for constrained optimization. Current benchmark environments for testing Evolutionary Algorithms are reviewed in the light of these principles. Along with this line, the reader is provided with an overview of the available problem domains in the field of constrained benchmarking. Hence, the review supports algorithms developers with information about the merits and demerits of the available frameworks.

cs.NE↗

arXiv2018

A Covariance Matrix Self-Adaptation Evolution Strategy for Optimization under Linear Constraints

This paper addresses the development of a covariance matrix self-adaptation evolution strategy (CMSA-ES) for solving optimization problems with linear constraints. The proposed algorithm is referred to as Linear Constraint CMSA-ES (lcCMSA-ES). It uses a specially built mutation operator together with repair by projection to satisfy the constraints. The lcCMSA-ES evolves itself on a linear manifold defined by the constraints. The objective function is only evaluated at feasible search points (interior point method). This is a property often required in application domains such as simulation optimization and finite element methods. The algorithm is tested on a variety of different test problems revealing considerable results.

cs.NE↗

arXiv2018

A Linear Constrained Optimization Benchmark For Probabilistic Search Algorithms: The Rotated Klee-Minty Problem

The development, assessment, and comparison of randomized search algorithms heavily rely on benchmarking. Regarding the domain of constrained optimization, the number of currently available benchmark environments bears no relation to the number of distinct problem features. The present paper advances a proposal of a scalable linear constrained optimization problem that is suitable for benchmarking Evolutionary Algorithms. By comparing two recent EA variants, the linear benchmarking environment is demonstrated.

cs.NE↗