arXiv subjects

Mark Klein

Publications and source records attributed to Mark Klein.

10 recordsLinked to original sources

arXiv2026

An Engineering Journey Training Large Language Models at Scale on Alps: The Apertus Experience

Large Language Models (LLMs) have surged as a transformative technology for science and society, prompting governments worldwide to pursue sovereign AI capabilities that ensure data compliance and cultural representation. However, the associated capital costs and engineering complexity required to train these models have largely restricted such capabilities to the private sector, leaving a significant gap for public institutions. This paper details the engineering journey behind training Apertus, a fully open multilingual foundation model, on the Alps supercomputer. Representing a first-of-its-kind achievement for academia at the 70B parameter scale, we successfully deployed a massive pre-training campaign on one of Europe's largest systems for open science, powered by NVIDIA GH200 Grace Hopper Superchips. We detail the challenges encountered in readying HPC infrastructure for training AI models, from overcoming storage bottlenecks to stabilizing large-scale interconnects, and the lessons learned in transforming a supercomputer into a resilient software-defined Machine Learning Platform. Finally, we discuss the post-training requirements and evolution of our Machine Learning platform, outlining how this initial release lays the groundwork for a sustained, iterative operational capability, in particular for fine tuning foundation models, that extends well beyond a single model training run.

cs.DC↗

arXiv2025

Apertus: Democratizing Open and Compliant LLMs for Global Language Environments

We present Apertus, a fully open suite of large language models (LLMs) designed to address two systemic shortcomings in today's open model ecosystem: data compliance and multilingual representation. Unlike many prior models that release weights without reproducible data pipelines or regard for content-owner rights, Apertus models are pretrained exclusively on openly available data, retroactively respecting `robots.txt` exclusions and filtering for non-permissive, toxic, and personally identifiable content. To mitigate risks of memorization, we adopt the Goldfish objective during pretraining, strongly suppressing verbatim recall of data while retaining downstream task performance. The Apertus models also expand multilingual coverage, training on 15T tokens from over 1800 languages, with ~40% of pretraining data allocated to non-English content. Released at 8B and 70B scales, Apertus approaches state-of-the-art results among fully open models on multilingual benchmarks, rivalling or surpassing open-weight counterparts. Beyond model weights, we release all scientific artifacts from our development cycle with a permissive license, including data preparation scripts, checkpoints, evaluation suites, and training code, enabling transparent audit and extension.

cs.CL↗

arXiv2025

Using utility graphs to search for Pareto-optimal outcomes in complex, interdependent issue negotiations

This paper studies how utility graphs decomposition algorithms can be used to effectively search for Pareto-efficient outcomes in complex automated negotiation. We propose a number of algorithms that can efficiently handle high-dimensional utility graphs, and test them on a variety of utility graph topologies, generated based on state of the art methods for analysing complex graphs. We show that we can achieve exponential speed-up, for many structures, even for very large utility graphs. To our knowledge, our approach can handle the largest utility spaces to date for complex interdependent negotiations, in terms of number of issues. Moreover, we examine the performance of our algorithms across two different types of elicitation queries from the literature: value and comparison queries, thus making a connection between automated negotiation and the preference elicitation literature.

cs.MA↗

arXiv2025

Alps, a versatile research infrastructure

The Swiss National Supercomputing Centre (CSCS) has a long-standing tradition of delivering top-tier high-performance computing systems, exemplified by the Piz Daint supercomputer. However, the increasing diversity of scientific needs has exposed limitations in traditional vertically integrated HPC architectures, which often lack flexibility and composability. To address these challenges, CSCS developed Alps, a next-generation HPC infrastructure designed with a transformative principle: resources operate as independent endpoints within a high-speed network. This architecture enables the creation of independent tenant-specific and platform-specific services, tailored to diverse scientific requirements. Alps incorporates heterogeneous hardware, including CPUs and GPUs, interconnected by a high-performance Slingshot network, and offers a modular storage system. A key innovation is the versatile software-defined cluster (vCluster) technology, which bridges cloud and HPC paradigms. By abstracting infrastructure, service management, and user environments into distinct layers, vClusters allow for customized platforms that support diverse workloads. Current platforms on Alps serve various scientific domains, including numerical weather prediction, and AI research.

cs.DC↗

arXiv2019

Dynamically Provisioning Cray DataWarp Storage

Complex applications and workflows needs are often exclusively expressed in terms of computational resources on HPC systems. In many cases, other resources like storage or network are not allocatable and are shared across the entire HPC system. By looking at the storage resource in particular, any workflow or application should be able to select both its preferred data manager and its required storage capability or capacity. To achieve such a goal, new mechanisms should be introduced. In this work, we introduce such a mechanism for dynamically provision a data management system on top of storage devices. We particularly focus our effort on deploying a BeeGFS instance across multiple DataWarp nodes on a Cray XC50 system. However, we also demonstrate that the same mechanism can be used to deploy BeeGFS on non-Cray system.

cs.DC↗

arXiv2019

REACT: REActive resilience for critical infrastructures using graph-Coloring Techniques

Nowadays society is more and more dependent on critical infrastructures. Critical network infrastructures (CNI) are communication networks whose disruption can create a severe impact. In this paper we propose REACT, a distributed framework for reactive network resilience, which allows networks to reconfigure themselves in the event of a security incidents so that the risk of further damage is mitigated. Our framework takes advantage of a risk model based on multilayer networks, as well as a graph-coloring problem conversion, to identify new, more resilient configurations for networks in the event of an attack. We propose two different solution approaches, and evaluate them from two different perspectives, with a number of centralized optimization techniques. Experiments show that our approaches outperform the reference approaches in terms of risk mitigation and performance.

cs.NI↗

arXiv2019

Nonlinear Negotiation Approaches for Complex-Network Optimization: A Study Inspired by Wi-Fi Channel Assignment

At the present time, Wi-Fi networks are everywhere. They operate in unlicensed radio-frequency spectrum bands (divided in channels), which are highly congested. The purpose of this paper is to tackle the problem of channel assignment in Wi-Fi networks. To this end, we have modeled the networks as multilayer graphs, in a way that frequency channel assignment becomes a graph coloring problem. For a high number and variety of scenarios, we have solved the problem with two different automated negotiation techniques: a hill-climber and a simulated annealer. As an upper bound reference for the performance of these two techniques, we have also solved the problem using a particle swarm optimizer. Results show that the annealer negotiator behaves as the best choice because it is able to obtain even better results than the particle swarm optimizer in the most complex scenarios under study, with running times one order of magnitude below. Finally, we study how different properties of the network layout affect to the performance gain that the annealer is able to obtain with respect to the particle swarm optimizer.

cs.NI↗

arXiv2018

A Class of Weighted TSPs with Applications

Motivated by applications to poaching and burglary prevention, we define a class of weighted Traveling Salesman Problems on metric spaces. The goal is to output an infinite (though typically periodic) tour that visits the n points repeatedly, such that no point goes unvisited for "too long." More specifically, we consider two objective functions for each point x. The maximum objective is simply the maximum duration of any absence from x, while the quadratic objective is the normalized sum of squares of absence lengths from x. For periodic tours, the quadratic objective captures the expected duration of absence from x at a uniformly random point in time during the tour. The overall objective is then the weighted maximum of the individual points' objectives. When a point has weight w_x, the absences under an optimal tour should be roughly a 1/w_x fraction of the absences from points of weight 1. Thus, the objective naturally encourages visiting high-weight points more frequently, and at roughly evenly spaced intervals. We give a polynomial-time combinatorial algorithm whose output is simultaneously an O(log n) approximation under both objectives. We prove that up to constant factors, approximation guarantees for the quadratic objective directly imply the same guarantees for a natural security patrol game defined in recent work.

cs.DS↗

arXiv2015

Design for Online Deliberative Processes and Technologies: Towards a Multidisciplinary Research Agenda

There has been rapidly growing interest in studying and designing online deliberative processes and technologies. This SIG aims at providing a venue for continuous and constructive dialogue between social, political and cognitive sciences as well as computer science, HCI, and CSCW. Through an online community and a modified version of world cafe discussions, we contribute to the definition of the theoretical building blocks, the identification of a research agenda for the CHI community, and the network of individuals from academia, industry, and the public sector who share interests in different aspects of online deliberation.

cs.HC↗

arXiv2003

Efficiency through disinformation

We study the impact of disinformation on a model of resource allocation with independent selfish agents: clients send requests to one of two servers, depending on which one is perceived as offering shorter waiting times. Delays in the information about the servers' state leads to oscillations in load. Servers can give false information about their state (global disinformation) or refuse service to individual clients (local disinformation). We discuss the tradeoff between positive effects of disinformation (attenuation of oscillations) and negative effects (increased fluctuations and reduced adaptability) for different parameter values.

cond-mat.dis-nn↗