Searcharxiv⌕ Search
Home/Archive/arXiv

arXiv · 2609.11946

Hyperion: An AI-powered HPC cluster for sciences and humanities research that utilizes ML for predicting job turnaround time

Jun Zhou·Nathan Elgar·Tawnee Benedetto·John Richards·Ming Hu·Greg Wilsbacher·Lawrence Miao·Paul Sagona
View original source ↗Read source document ↗Export metadata ↓

Abstract

Hyperion is an innovative high-performance computing (HPC) cluster developed for researchers in both science and humanities disciplines at the University of South Carolina (USC). Our approach involved constructing a HPC cluster designed to meet the current research needs while accommodating future expansion. Additionally, we developed and trained two machine learning (ML) models to predict turnaround time, including wait time and wall time, and seamlessly integrated them into the Slurm job submission. Finally, we showcase a variety of sample applications hosted on the Hyperion platform.

Explore related subjects

cs.DCcs.LG

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Jun Zhou, Nathan Elgar, Tawnee Benedetto, John Richards, Ming Hu, Greg Wilsbacher, Lawrence Miao, Paul Sagona. 2026-07-22. Hyperion: An AI-powered HPC cluster for sciences and humanities research that utilizes ML for predicting job turnaround time. https://arxiv.org/abs/2609.11946

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Konflux: Optimized Function Fusion for Serverless Applications

Function-as-a-Service (FaaS) has become a central paradigm in serverless cloud computing, yet optimizing FaaS deployments remains challenging. Using function fusion, multiple functions can be combined into a single deployment unit, which can be used to reduce cost and latency of complex serverless applications comprising multiple functions. Even in small-scale applications, the number of possible fusion configurations is vast, making brute-force benchmarking in production both cost- and time-prohibitive. In this paper, we present a system that can analyze every possible fusion setup of complex applications. By emulating the FaaS platform, our system enables local experimentation, eliminating the need to reconfigure the live platform and significantly reducing associated cost and time. We evaluate all fusion configurations across a number of example FaaS applications and resource limits. Our results reveal that, when analyzing cost and latency trade-offs, only a limited set of fusion configurations represent optimal solutions, which are strongly influenced by the specific pricing model in use.

cs.DC↗
arXiv2026

A Self-Defining Operator for Distributed Applications

To operate distributed applications with high availability, operators must rapidly diagnose and mitigate any incident. LLM-based agents can help, but unlike experienced human operators they do not improve as they handle more incidents. Each investigation starts from scratch, re-deriving likely causes and re-testing hypotheses even when a similar fault was resolved in the past. We present a self-defining operator (SDO) that improves with time by updating its operational behavior and memory. When SDO resolves an incident, it distills that experience into playbooks, which describe reusable diagnostic instructions for future incidents with similar faults, and detectors, scripts that take application telemetry as input and output relevant playbook(s). SDO continuously runs the detectors and invokes LLMs only when one fires, so its token cost scales with incident count rather than with operational time. Across 11 open-source microservice applications, SDO cuts resolution time by 52% on recurring faults while keeping per-incident cost low. SDO is open source at https://github.com/self-defining-systems/sdo.

cs.DC↗
arXiv2026

Native Dual Pools or Shared Paging? Memory Organization for Concurrent Hybrid LLM Serving

Hybrid attention-SSM serving combines growing attention KV with fixed-size recurrent and convolution state whose allocation units, reuse boundaries, and lifetimes differ. We implement four memory organizations in a modified vLLM runtime: a coarse shared-page baseline (S0), checkpoint-internal recurrent paging (S1), paging with convolution-slot packing (S2), and native dual pools (D0). Nine evaluation questions cover storage costs and serving behavior. The cumulative S0-S1-S2 refinement improves completed-output throughput and tail completion time at the main operating point. D0-S2 performance varies with memory budget, offered load, and admission headroom. Shared-prefix workloads yield larger stepwise gains; gains also persist in a no-sharing control with equal recorded input-work totals. State traces identify recurrent-checkpoint availability as the limit on joint reuse in the tested S2 runs. Extending producer-side checkpoint lifetime increases reusable depth at a fixed page budget. A fresh-seed SHARE/ISOLATE comparison estimates a positive retention interaction, with a primary confidence interval that includes zero. Retained-state continuation correctness remains unresolved. These findings motivate evaluating physical state representation together with request admission and checkpoint lifetime when designing memory management for concurrent hybrid-model serving.

cs.DC↗