arXiv subjects

Markus Götz

Publications and source records attributed to Markus Götz.

27 records · Page 2Linked to original sources

arXiv2023

Massively Parallel Genetic Optimization through Asynchronous Propagation of Populations

We present Propulate, an evolutionary optimization algorithm and software package for global optimization and in particular hyperparameter search. For efficient use of HPC resources, Propulate omits the synchronization after each generation as done in conventional genetic algorithms. Instead, it steers the search with the complete population present at time of breeding new individuals. We provide an MPI-based implementation of our algorithm, which features variants of selection, mutation, crossover, and migration and is easy to extend with custom functionality. We compare Propulate to the established optimization tool Optuna. We find that Propulate is up to three orders of magnitude faster without sacrificing solution accuracy, demonstrating the efficiency and efficacy of our lazy synchronization approach. Code and documentation are available at https://github.com/Helmholtz-AI-Energy/propulate

cs.NE↗

arXiv2022

Precise Energy Consumption Measurements of Heterogeneous Artificial Intelligence Workloads

With the rise of AI in recent years and the increase in complexity of the models, the growing demand in computational resources is starting to pose a significant challenge. The need for higher compute power is being met with increasingly more potent accelerators and the use of large compute clusters. However, the gain in prediction accuracy from large models trained on distributed and accelerated systems comes at the price of a substantial increase in energy demand, and researchers have started questioning the environmental friendliness of such AI methods at scale. Consequently, energy efficiency plays an important role for AI model developers and infrastructure operators alike. The energy consumption of AI workloads depends on the model implementation and the utilized hardware. Therefore, accurate measurements of the power draw of AI workflows on different types of compute nodes is key to algorithmic improvements and the design of future compute clusters and hardware. To this end, we present measurements of the energy consumption of two typical applications of deep learning models on different types of compute nodes. Our results indicate that 1. deriving energy consumption directly from runtime is not accurate, but the consumption of the compute node needs to be considered regarding its composition; 2. neglecting accelerator hardware on mixed nodes results in overproportional inefficiency regarding energy consumption; 3. energy consumption of model training and inference should be considered separately - while training on GPUs outperforms all other node types regarding both runtime and energy consumption, inference on CPU nodes can be comparably efficient. One advantage of our approach is that the information on energy consumption is available to all users of the supercomputer, enabling an easy transfer to other workloads alongside a raise in user-awareness of energy consumption.

cs.DC↗

arXiv2022

Learning Tree Structures from Leaves For Particle Decay Reconstruction

In this work, we present a neural approach to reconstructing rooted tree graphs describing hierarchical interactions, using a novel representation we term the Lowest Common Ancestor Generations (LCAG) matrix. This compact formulation is equivalent to the adjacency matrix, but enables learning a tree's structure from its leaves alone without the prior assumptions required if using the adjacency matrix directly. Employing the LCAG therefore enables the first end-to-end trainable solution which learns the hierarchical structure of varying tree sizes directly, using only the terminal tree leaves to do so. In the case of high-energy particle physics, a particle decay forms a hierarchical tree structure of which only the final products can be observed experimentally, and the large combinatorial space of possible trees makes an analytic solution intractable. We demonstrate the use of the LCAG as a target in the task of predicting simulated particle physics decay structures using both a Transformer encoder and a Neural Relational Inference encoder Graph Neural Network. With this approach, we are able to correctly predict the LCAG purely from leaf features for a maximum tree-depth of $8$ in $92.5\%$ of cases for trees up to $6$ leaves (including) and $59.7\%$ for trees up to $10$ in our simulated dataset.

physics.comp-ph↗

arXiv2022

HyDe: The First Open-Source, Python-Based, GPU-Accelerated Hyperspectral Denoising Package

As with any physical instrument, hyperspectral cameras induce different kinds of noise in the acquired data. Therefore, Hyperspectral denoising is a crucial step for analyzing hyperspectral images (HSIs). Conventional computational methods rarely use GPUs to improve efficiency and are not fully open-source. Alternatively, deep learning-based methods are often open-source and use GPUs, but their training and utilization for real-world applications remain non-trivial for many researchers. Consequently, we propose HyDe: the first open-source, GPU-accelerated Python-based, hyperspectral image denoising toolbox, which aims to provide a large set of methods with an easy-to-use environment. HyDe includes a variety of methods ranging from low-rank wavelet-based methods to deep neural network (DNN) models. HyDe's interface dramatically improves the interoperability of these methods and the performance of the underlying functions. In fact, these methods maintain similar HSI denoising performance to their original implementations while consuming nearly ten times less energy. Furthermore, we present a method for training DNNs for denoising HSIs which are not spatially related to the training dataset, i.e., training on ground-level HSIs for denoising HSIs with other perspectives including airborne, drone-borne, and space-borne. To utilize the trained DNNs, we show a sliding window method to effectively denoise HSIs which would otherwise require more than 40 GB. The package can be found at: \url{https://github.com/Helmholtz-AI-Energy/HyDe}.

cs.CV↗

arXiv2021

Accelerating Neural Network Training with Distributed Asynchronous and Selective Optimization (DASO)

With increasing data and model complexities, the time required to train neural networks has become prohibitively large. To address the exponential rise in training time, users are turning to data parallel neural networks (DPNN) to utilize large-scale distributed resources on computer clusters. Current DPNN approaches implement the network parameter updates by synchronizing and averaging gradients across all processes with blocking communication operations. This synchronization is the central algorithmic bottleneck. To combat this, we introduce the Distributed Asynchronous and Selective Optimization (DASO) method which leverages multi-GPU compute node architectures to accelerate network training. DASO uses a hierarchical and asynchronous communication scheme comprised of node-local and global networks while adjusting the global synchronization rate during the learning process. We show that DASO yields a reduction in training time of up to 34% on classical and state-of-the-art networks, as compared to other existing data parallel training methods.

cs.LG↗

arXiv2020

HeAT -- a Distributed and GPU-accelerated Tensor Framework for Data Analytics

To cope with the rapid growth in available data, the efficiency of data analysis and machine learning libraries has recently received increased attention. Although great advancements have been made in traditional array-based computations, most are limited by the resources available on a single computation node. Consequently, novel approaches must be made to exploit distributed resources, e.g. distributed memory architectures. To this end, we introduce HeAT, an array-based numerical programming framework for large-scale parallel processing with an easy-to-use NumPy-like API. HeAT utilizes PyTorch as a node-local eager execution engine and distributes the workload on arbitrarily large high-performance computing systems via MPI. It provides both low-level array computations, as well as assorted higher-level algorithms. With HeAT, it is possible for a NumPy user to take full advantage of their available resources, significantly lowering the barrier to distributed data analysis. When compared to similar frameworks, HeAT achieves speedups of up to two orders of magnitude.

cs.DC↗

arXiv2018

Conformational dynamics of a single protein monitored for 24 hours at video rate

We use plasmon rulers to follow the conformational dynamics of a single protein for up to 24 h at a video rate. The plasmon ruler consists of two gold nanospheres connected by a single protein linker. In our experiment, we follow the dynamics of the molecular chaperone heat shock protein 90, which is known to show open and closed conformations. Our measurements confirm the previously known conformational dynamics with transition times in the second to minute time scale and reveals new dynamics on the time scale of minutes to hours. Plasmon rulers thus extend the observation bandwidth 3/4 orders of magnitude with respect to single-molecule fluorescence resonance energy transfer and enable the study of molecular dynamics with unprecedented precision.

q-bio.QM↗

arXiv2017

Precision and accuracy of single-molecule FRET measurements - a worldwide benchmark study

Single-molecule Förster resonance energy transfer (smFRET) is increasingly being used to determine distances, structures, and dynamics of biomolecules in vitro and in vivo. However, generalized protocols and FRET standards ensuring both the reproducibility and accuracy of measuring FRET efficiencies are currently lacking. Here we report the results of a worldwide, comparative, blind study, in which 20 labs determined the FRET efficiencies of several dye-labeled DNA duplexes. Using a unified and straightforward method, we show that FRET efficiencies can be obtained with a standard deviation between $Δ$E = +-0.02 and +-0.05. We further suggest an experimental and computational procedure for converting FRET efficiencies into accurate distances. We discuss potential uncertainties in the experiment and the modelling. Our extensive quantitative assessment of intensity-based smFRET measurements and correction procedures serve as an essential step towards validation of distance networks with the ultimate aim to archive reliable structural models of biomolecular systems obtained by smFRET-based hybrid methods.

q-bio.QM↗

arXiv2016

Experiment-friendly kinetic analysis of single molecule data in and out of equilibrium

We present a simple and robust technique to extract kinetic rate models and thermodynamic quantities from single molecule time traces. SMACKS (Single Molecule Analysis of Complex Kinetic Sequences) is a maximum likelihood approach that works equally well for long trajectories as for a set of short ones. It resolves all statistically relevant rates and also their uncertainties. This is achieved by optimizing one global kinetic model based on the complete dataset, while allowing for experimental variations between individual trajectories. In particular, neither a priori models nor equilibrium have to be assumed. The power of SMACKS is demonstrated on the kinetics of the multi-domain protein Hsp90 measured by smFRET (single molecule Förster resonance energy transfer). Experiments in and out of equilibrium are analyzed and compared to simulations, shedding new light on the role of Hsp90's ATPase function. SMACKS pushes the boundaries of single molecule kinetics far beyond current methods.

q-bio.QM↗