arXiv subjects

Utz-Uwe Haus

Publications and source records attributed to Utz-Uwe Haus.

At least 19 recordsLinked to original sources

arXiv2026

Multi-tenant Kubernetes Use Cases for AI, Secure Computing and Data Services, and More

Kubernetes, as a container orchestration engine, has been widely used in cloud-native ecosystems for several years. In supercomputing ecosystems, especially where bare-metal performance for compute and network devices are considered, the adoption is somewhat limited. However, with the increasing diversity of use cases such as AI, secure and confidential computing for sensitive data, and mixed workload orchestration, a traditional, single-tenant batch computing system does not offer the flexibility and reproducibility to which public cloud users are accustomed. Note that Kubernetes is not considered a replacement for batch scheduling systems, which have powerful features for large-scale MPI jobs with thousands of network end points. Rather, it is a complementary service provided as part of a national AI Research Resource. We evaluate Kubernetes deployment on a Hewlett Packard Enterprise (HPE) Cray EX supercomputerwith HPE Slingshot interconnect, called Isambard-AI, with co-design use cases. One is a Trusted Research Environment used for medical and health sciences. The other combines KubeRay, Ray, and vLLM to provide a distributed, sandboxed, persistent AI model hosting service targeting multi-tenant confidential computing. We discuss challenges and lessons learned, and where further development is needed to offer a production Kubernetes-as-a-Service on HPE Cray EX (and later) platforms.

cs.DC↗

arXiv2026

A User-oriented Portable, Reproducible, and Scalable Software Ecosystem

It is normal for scientists to perform their research on a diverse set of hardware, ranging from laptops and workstations to supercomputers and cloud resources. The standard scenario requires a mix of these resources. In this paper we describe a software ecosystem that enables users to rely on the same development environment for running their workflows across the different computational resources. We describe a modular, unified command-line interface that allows for the interaction with a user-workflow across diverse hardware platform. The software ecosystem has been successfully tested as part of the plan4res EU H2020 project. It can be extended to other projects with similar requirements, so that they can benefit from the same approach for executing computational workflows.

cs.DC↗

arXiv2025

SIREN: Software Identification and Recognition in HPC Systems

HPC systems use monitoring and operational data analytics to ensure efficiency, performance, and orderly operations. Application-specific insights are crucial for analyzing the increasing complexity and diversity of HPC workloads, particularly through the identification of unknown software and recognition of repeated executions, which facilitate system optimization and security improvements. However, traditional identification methods using job or file names are unreliable for arbitrary user-provided names (a.out). Fuzzy hashing of executables detects similarities despite changes in executable version or compilation approach while preserving privacy and file integrity, overcoming these limitations. We introduce SIREN, a process-level data collection framework for software identification and recognition. SIREN improves observability in HPC by enabling analysis of process metadata, environment information, and executable fuzzy hashes. Findings from a first opt-in deployment campaign on LUMI show SIREN's ability to provide insights into software usage, recognition of repeated executions of known applications, and similarity-based identification of unknown applications.

cs.DC↗

arXiv2025

Closing the HPC-Cloud Convergence Gap: Multi-Tenant Slingshot RDMA for Kubernetes

Converged HPC-Cloud computing is an emerging computing paradigm that aims to support increasingly complex and multi-tenant scientific workflows. These systems require reconciliation of the isolation requirements of native cloud workloads and the performance demands of HPC applications. In this context, networking hardware is a critical boundary component: it is the conduit for high-throughput, low-latency communication and enables isolation across tenants. HPE Slingshot is a high-speed network interconnect that provides up to 200 Gbps of throughput per port and targets high-performance computing (HPC) systems. The Slingshot host software, including hardware drivers and network middleware libraries, is designed to meet HPC deployments, which predominantly use single-tenant access modes. Hence, the Slingshot stack is not suited for secure use in multi-tenant deployments, such as converged HPC-Cloud deployments. In this paper, we design and implement an extension to the Slingshot stack targeting converged deployments on the basis of Kubernetes. Our integration provides secure, container-granular, and multi-tenant access to Slingshot RDMA networking capabilities at minimal overhead.

cs.DC↗

arXiv2025

SLURM Heterogeneous Jobs for Hybrid Classical-Quantum Workflows

A method for efficient scheduling of hybrid classical-quantum workflows is presented, based on standard tools available on common supercomputer systems. Moderate interventions by the user are required, such as splitting a monolithic workflow in to basic building blocks and ensuring the data flow. This bares the potential to significantly reduce idle time of the quantum resource as well as overall wall time of co-scheduled workflows. Relevant pseudo-code samples and scripts are provided to demonstrate the simplicity and working principles of the method.

cs.DC↗

arXiv2024

Workflows Community Summit 2024: Future Trends and Challenges in Scientific Workflows

The Workflows Community Summit gathered 111 participants from 18 countries to discuss emerging trends and challenges in scientific workflows, focusing on six key areas: time-sensitive workflows, AI-HPC convergence, multi-facility workflows, heterogeneous HPC environments, user experience, and FAIR computational workflows. The integration of AI and exascale computing has revolutionized scientific workflows, enabling higher-fidelity models and complex, time-sensitive processes, while introducing challenges in managing heterogeneous environments and multi-facility data dependencies. The rise of large language models is driving computational demands to zettaflop scales, necessitating modular, adaptable systems and cloud-service models to optimize resource utilization and ensure reproducibility. Multi-facility workflows present challenges in data movement, curation, and overcoming institutional silos, while diverse hardware architectures require integrating workflow considerations into early system design and developing standardized resource management tools. The summit emphasized improving user experience in workflow systems and ensuring FAIR workflows to enhance collaboration and accelerate scientific discovery. Key recommendations include developing standardized metrics for time-sensitive workflows, creating frameworks for cloud-HPC integration, implementing distributed-by-design workflow modeling, establishing multi-facility authentication protocols, and accelerating AI integration in HPC workflow management. The summit also called for comprehensive workflow benchmarks, workflow-specific UX principles, and a FAIR workflow maturity model, highlighting the need for continued collaboration in addressing the complex challenges posed by the convergence of AI, HPC, and multi-facility research environments.

cs.DC↗

arXiv2024

OpenCUBE: Building an Open Source Cloud Blueprint with EPI Systems

OpenCUBE aims to develop an open-source full software stack for Cloud computing blueprint deployed on EPI hardware, adaptable to emerging workloads across the computing continuum. OpenCUBE prioritizes energy awareness and utilizes open APIs, Open Source components, advanced SiPearl Rhea processors, and RISC-V accelerator. The project leverages representative workloads, such as cloud-native workloads and workflows of weather forecast data management, molecular docking, and space weather, for evaluation and validation.

cs.DC↗

arXiv2024

Evaluating Versal AI Engines for option price discovery in market risk analysis

Whilst Field-Programmable Gate Arrays (FPGAs) have been popular in accelerating high-frequency financial workload for many years, their application in quantitative finance, the utilisation of mathematical models to analyse financial markets and securities, is less mature. Nevertheless, recent work has demonstrated the benefits that FPGAs can deliver to quantitative workloads, and in this paper, we study whether the Versal ACAP and its AI Engines (AIEs) can also deliver improved performance. We focus specifically on the industry standard Strategic Technology Analysis Center's (STAC) derivatives risk analysis benchmark STAC-A2. Porting a purely FPGA-based accelerator STAC-A2 inspired market risk (SIMR) benchmark to the Versal ACAP device by combining Programmable Logic (PL) and AIEs, we explore the development approach and techniques, before comparing performance across PL and AIEs. Ultimately, we found that our AIE approach is slower than a highly optimised existing PL-only version due to limits on both the AIE and PL that we explore and describe.

cs.DC↗

arXiv2024

Autonomy Loops for Monitoring, Operational Data Analytics, Feedback, and Response in HPC Operations

Many High Performance Computing (HPC) facilities have developed and deployed frameworks in support of continuous monitoring and operational data analytics (MODA) to help improve efficiency and throughput. Because of the complexity and scale of systems and workflows and the need for low-latency response to address dynamic circumstances, automated feedback and response have the potential to be more effective than current human-in-the-loop approaches which are laborious and error prone. Progress has been limited, however, by factors such as the lack of infrastructure and feedback hooks, and successful deployment is often site- and case-specific. In this position paper we report on the outcomes and plans from a recent Dagstuhl Seminar, seeking to carve a path for community progress in the development of autonomous feedback loops for MODA, based on the established formalism of similar (MAPE-K) loops in autonomous computing and self-adaptive systems. By defining and developing such loops for significant cases experienced across HPC sites, we seek to extract commonalities and develop conventions that will facilitate interoperability and interchangeability with system hardware, software, and applications across different sites, and will motivate vendors and others to provide telemetry interfaces and feedback hooks to enable community development and pervasive deployment of MODA autonomy loops.

cs.DC↗

arXiv2023

Fortran High-Level Synthesis: Reducing the barriers to accelerating HPC codes on FPGAs

In recent years the use of FPGAs to accelerate scientific applications has grown, with numerous applications demonstrating the benefit of FPGAs for high performance workloads. However, whilst High Level Synthesis (HLS) has significantly lowered the barrier to entry in programming FPGAs by enabling programmers to use C++, a major challenge is that most often these codes are not originally written in C++. Instead, Fortran is the lingua franca of scientific computing and-so it requires a complex and time consuming initial step to convert into C++ even before considering the FPGA. In this paper we describe work enabling Fortran for AMD Xilinx FPGAs by connecting the LLVM Flang front end to AMD Xilinx's LLVM back end. This enables programmers to use Fortran as a first-class language for programming FPGAs, and as we demonstrate enjoy all the tuning and optimisation opportunities that HLS C++ provides. Furthermore, we demonstrate that certain language features of Fortran make it especially beneficial for programming FPGAs compared to C++. The result of this work is a lowering of the barrier to entry in using FPGAs for scientific computing, enabling programmers to leverage their existing codebase and language of choice on the FPGA directly.

cs.DC↗

arXiv2021

The Boundary for Quantum Advantage in Gaussian Boson Sampling

Identifying the boundary beyond which quantum machines provide a computational advantage over their classical counterparts is a crucial step in charting their usefulness. Gaussian Boson Sampling (GBS), in which photons are measured from a highly entangled Gaussian state, is a leading approach in pursuing quantum advantage. State-of-the-art quantum photonics experiments that, once programmed, run in minutes, would require 600 million years to simulate using the best pre-existing classical algorithms. Here, we present substantially faster classical GBS simulation methods, including speed and accuracy improvements to the calculation of loop hafnians, the matrix function at the heart of GBS. We test these on a $\sim \! 100,000$ core supercomputer to emulate a range of different GBS experiments with up to 100 modes and up to 92 photons. This reduces the run-time of classically simulating state-of-the-art GBS experiments to several months -- a nine orders of magnitude improvement over previous estimates. Finally, we introduce a distribution that is efficient to sample from classically and that passes a variety of GBS validation methods, providing an important adversary for future experiments to test against.

quant-ph↗

arXiv2017

Scenario Aggregation using Binary Decision Diagrams for Stochastic Programs with Endogenous Uncertainty

Modeling decision-dependent scenario probabilities in stochastic programs is difficult and typically leads to large and highly non-linear MINLPs that are very difficult to solve. In this paper, we develop a new approach to obtain a compact representation of the recourse function using a set of binary decision diagrams (BDDs) that encode a nested cover of the scenario set. The resulting BDDs can then be used to efficiently characterize the decision-dependent scenario probabilities by a set of linear inequalities, which essentially factorizes the probability distribution and thus allows to reformulate the entire problem as a small mixed-integer linear program. The approach is applicable to a large class of stochastic programs with multivariate binary scenario sets, such as stochastic network design, network reliability, or stochastic network interdiction problems. Computational results show that the BDD-based scenario representation reduces the problem size, and hence the computation time, significant compared to previous approaches.

math.OC↗

arXiv2015

Model-Driven Automatic Tiling with Cache Associativity Lattices

Traditional compiler optimization theory distinguishes three separate classes of cache miss -- Cold, Conflict and Capacity. Tiling for cache is typically guided by capacity miss counts. Models of cache function have not been effectively used to guide cache tiling optimizations due to model error and expense. Instead, heuristic or empirical approaches are used to select tilings. We argue that conflict misses, traditionally neglected or seen as a small constant effect, are the only fundamentally important cache miss category, that they form a solid basis by which caches can become modellable, and that models leaning on cache associatvity analysis can be used to generate cache performant tilings. We develop a mathematical framework that expresses potential and actual cache misses in associative caches using Associativity Lattices. We show these lattices to possess two theoretical advantages over rectangular tiles -- volume maximization and miss regularity. We also show that to generate such lattice tiles requires, unlike rectangular tiling, no explicit, expensive lattice point counting. We also describe an implementation of our lattice tiling approach, show that it can be used to give speedups of over 10x versus unoptimized code, and despite currently only tiling for one level of cache, can already be competitive with the aggressive compiler optimizations used in general purposes compares such as GCC and Intel's ICC. We also show that the tiling approach can lead to reasonable automatic parallelism when compared to existing auto-threading compilers.

cs.PF↗

arXiv2014

Bounding Stochastic Dependence, Complete Mixability of Matrices, and Multidimensional Bottleneck Assignment Problems

We call a matrix completely mixable if the entries in its columns can be permuted so that all row sums are equal. If it is not completely mixable, we want to determine the smallest maximal and largest minimal row sum attainable. These values provide a discrete approximation of of minimum variance problems for discrete distributions, a problem motivated by the question how to estimate the $α$-quantile of an aggregate random variable with unknown dependence structure given the marginals of the constituent random variables. We relate this problem to the multidimensional bottleneck assignment problem and show that there exists a polynomial $2$-approximation algorithm if the matrix has only $3$ columns. In general, deciding complete mixability is $\mathcal{NP}$-complete. In particular the swapping algorithm of Puccetti et al. is not an exact method unless $\mathcal{NP}\subseteq\mathcal{ZPP}$. For a fixed number of columns it remains $\mathcal{NP}$-complete, but there exists a PTAS. The problem can be solved in pseudopolynomial time for a fixed number of rows, and even in polynomial time if all columns furthermore contain entries from the same multiset.

math.OC↗

arXiv2011

A Polynomial Time Approximation Algorithm for the Two-Commodity Splittable Flow Problem

We consider a generalization of the unsplittable maximum two-commodity flow problem on undirected graphs where each commodity $i\in{1,2}$ can be split into a bounded number $k_i$ of equally-sized chunks that can be routed on different paths. We show that in contrast to the single-commodity case this problem is NP-hard, and hard to approximate to within a factor of $α>1/2$. We present a polynomial time 1/2-approximation algorithm for the case of uniform chunk size over both commodities and show that for even $k_i$ and a mild cut condition it can be modified to yield an exact method. The uniform case can be used to derive a 1/4-approximation for the maximum concurrent $(k_1,k_2)$-splittable flow without chunk size restrictions for fixed demand ratios.

cs.DS↗

arXiv2011

Reconstructing biochemical cluster networks

Motivated by fundamental problems in chemistry and biology we study cluster graphs arising from a set of initial states $S\subseteq\Z^n_+$ and a set of transitions/reactions $M\subseteq\Z^n_+\times\Z^n_+$. The clusters are formed out of states that can be mutually transformed into each other by a sequence of reversible transitions. We provide a solution method from computational commutative algebra that allows for deciding whether two given states belong to the same cluster as well as for the reconstruction of the full cluster graph. Using the cluster graph approach we provide solutions to two fundamental questions: 1) Deciding whether two states are connected, e.g., if the initial state can be turned into the final state by a sequence of transition and 2) listing concisely all reactions processes that can accomplish that. As a computational example, we apply the framework to the permanganate/oxalic acid reaction.

math.AC↗

arXiv2009

Minimal Conflicting Sets for the Consecutive Ones Property in ancestral genome reconstruction

A binary matrix has the Consecutive Ones Property (C1P) if its columns can be ordered in such a way that all 1's on each row are consecutive. A Minimal Conflicting Set is a set of rows that does not have the C1P, but every proper subset has the C1P. Such submatrices have been considered in comparative genomics applications, but very little is known about their combinatorial structure and efficient algorithms to compute them. We first describe an algorithm that detects rows that belong to Minimal Conflicting Sets. This algorithm has a polynomial time complexity when the number of 1's in each row of the considered matrix is bounded by a constant. Next, we show that the problem of computing all Minimal Conflicting Sets can be reduced to the joint generation of all minimal true clauses and maximal false clauses for some monotone boolean function. We use these methods on simulated data related to ancestral genome reconstruction to show that computing Minimal Conflicting Set is useful in discriminating between true positive and false positive ancestral syntenies. We also study a dataset of yeast genomes and address the reliability of an ancestral genome proposal of the Saccahromycetaceae yeasts.

q-bio.GN↗

arXiv2008

Logic Integer Programming Models for Signaling Networks

We propose a static and a dynamic approach to model biological signaling networks, and show how each can be used to answer relevant biological questions. For this we use the two different mathematical tools of Propositional Logic and Integer Programming. The power of discrete mathematics for handling qualitative as well as quantitative data has so far not been exploited in Molecular Biology, which is mostly driven by experimental research, relying on first-order or statistical models. The arising logic statements and integer programs are analyzed and can be solved with standard software. For a restricted class of problems the logic models reduce to a polynomial-time solvable satisfiability algorithm. Additionally, a more dynamic model enables enumeration of possible time resolutions in poly-logarithmic time. Computational experiments are included.

q-bio.QM↗