arXiv subjects

Pei-Hung Lin

Publications and source records attributed to Pei-Hung Lin.

18 recordsLinked to original sources

arXiv2026

Closer in the Gap: Towards Portable Performance on RISC-V Vector Processors

The RISC-V Vector Extension~(RVV) is a cornerstone for supporting compute throughout in scientific and machine learning workloads. Yet compiler support and performance monitoring on real RVV~1.0 hardware are still evolving. In this work, we design a suite of assembly microbenchmarks to establish performance ceilings and calibrate performance counters on RVV hardware. Leveraging the assembly benchmarks, we find that predication overhead and stride load pose performance challenges that current compiler cost models do not yet fully address. Moreover, we present the first evaluation of GCC~15 and LLVM~21 autovectorization in HPC and ML proxy applications. GCC~15 outperforms LLVM~21 in four out of six applications. LLVM~21 only outperforms GCC~15 in SGEMM and DGEMM, driven by more aggressive instruction reduction confirmed through validated \texttt{perf} counters on the RVV hardware. We further show that the default LMUL selection in compilers performs close to the optimal. To study the RVV support for product-level application, we also evaluate the state-vector quantum simulator, Google's Qsim, with both manual RVV intrinsics and compiler auto-vectorization, revealing immaturity in current RVV compiler for complicated memory access pattern.

cs.DC↗

arXiv2026

High-performance Vector-length Agnostic Quantum Circuit Simulations on ARM Processors

ARM SVE and RISC-V RVV are emerging vector architectures in high-end processors that support vectorization of flexible vector length. In this work, we leverage an important workload for quantum computing, quantum state-vector simulations, to understand whether high-performance portability can be achieved in a vector-length agnostic (VLA) design. We propose a VLA design and optimization techniques critical for achieving high performance, including VLEN-adaptive memory layout adjustment, load buffering, fine-grained loop control, and gate fusion-based arithmetic intensity adaptation. We provide an implementation in Google's Qsim and evaluate five quantum circuits of up to 36 qubits on three ARM processors, including NVIDIA Grace, AWS Graviton3, and Fujitsu A64FX. By defining new metrics and PMU events to quantify vectorization activities, we draw generic insights for future VLA designs. Our single-source implementation of VLA quantum simulations achieves up to 4.5x speedup on A64FX, 2.5x speedup on Grace, and 1.5x speedup on Graviton.

cs.DC↗

arXiv2025

Beyond Code Pairs: Dialogue-Based Data Generation for LLM Code Translation

Large language models (LLMs) have shown remarkable capabilities in code translation, yet their performance deteriorates in low-resource programming domains such as Fortran and emerging frameworks like CUDA, where high-quality parallel data are scarce. We present an automated dataset generation pipeline featuring a dual-LLM Questioner-Solver design that incorporates external knowledge from compilers and runtime feedback. Beyond traditional source-target code pair datasets, our approach additionally generates (1) verified translations with unit tests for assessing functional consistency and (2) multi-turn dialogues that capture the reasoning process behind translation refinement. Applied to Fortran-to-C++ and C++-to-CUDA, the pipeline yields 3.64k and 3.93k dialogues, respectively. Fine-tuning on this data yields dramatic improvements in functional correctness, boosting unit test success rates by over 56% on the challenging C++-to-CUDA task. We show that the generated data enables a 7B open-weight model to significantly outperform larger proprietary systems on key metrics like compilation success.

cs.PL↗

arXiv2025

ARM SVE Unleashed: Performance and Insights Across HPC Applications on Nvidia Grace

Vector architectures are essential for boosting computing throughput. ARM provides SVE as the next-generation length-agnostic vector extension beyond traditional fixed-length SIMD. This work provides a first study of the maturity and readiness of exploiting ARM and SVE in HPC. Using selected performance hardware events on the ARM Grace processor and analytical models, we derive new metrics to quantify the effectiveness of exploiting SVE vectorization to reduce executed instructions and improve performance speedup. We further propose an adapted roofline model that combines vector length and data elements to identify potential performance bottlenecks. Finally, we propose a decision tree for classifying the SVE-boosted performance in applications.

cs.DC↗

arXiv2024

Fortran2CPP: Automating Fortran-to-C++ Translation using LLMs via Multi-Turn Dialogue and Dual-Agent Integration

Translating legacy Fortran code into C++ is a crucial step in modernizing high-performance computing (HPC) applications. However, the scarcity of high-quality, parallel Fortran-to-C++ datasets and the limited domain-specific expertise in large language models (LLMs) present significant challenges for automated translation. In this paper, we introduce Fortran2CPP, a multi-turn dialogue dataset generated by a novel LLM agent-based approach that integrates a dual-LLM Questioner-Solver module to enhance translation accuracy. Our dataset comprises 11.7k dialogues capturing iterative feedback-decision workflows including code translation, compilation, execution, unit testing, and error-fixing. Using this dataset, we fine-tune several open-weight LLMs and achieve up to a 3.31x improvement in CodeBLEU scores and a 92\% increase in compilation success rate, demonstrating enhanced syntactic accuracy and functional reliability. Our findings highlight the value of dialogue-based LLM training for complex code translation tasks. The dataset and model have been open-sourced and are available on our public GitHub repository\footnote{\url{https://github.com/HPC-Fortran2CPP/Fortran2Cpp}}.

cs.LG↗

arXiv2023

HPC-GPT: Integrating Large Language Model for High-Performance Computing

Large Language Models (LLMs), including the LLaMA model, have exhibited their efficacy across various general-domain natural language processing (NLP) tasks. However, their performance in high-performance computing (HPC) domain tasks has been less than optimal due to the specialized expertise required to interpret the model responses. In response to this challenge, we propose HPC-GPT, a novel LLaMA-based model that has been supervised fine-tuning using generated QA (Question-Answer) instances for the HPC domain. To evaluate its effectiveness, we concentrate on two HPC tasks: managing AI models and datasets for HPC, and data race detection. By employing HPC-GPT, we demonstrate comparable performance with existing methods on both tasks, exemplifying its excellence in HPC-related scenarios. Our experiments on open-source benchmarks yield extensive results, underscoring HPC-GPT's potential to bridge the performance gap between LLMs and HPC-specific tasks. With HPC-GPT, we aim to pave the way for LLMs to excel in HPC domains, simplifying the utilization of language models in complex computing applications.

cs.DC↗

arXiv2023

DataRaceBench V1.4.1 and DataRaceBench-ML V0.1: Benchmark Suites for Data Race Detection

Data races pose a significant threat in multi-threaded parallel applications due to their negative impact on program correctness. DataRaceBench, an open-source benchmark suite, is specifically crafted to assess these data race detection tools in a systematic and measurable manner. Machine learning techniques have recently demonstrated considerable potential in high-performance computing (HPC) program analysis and optimization. However, these techniques require specialized data formats for training and refinement. This paper presents the latest update to DataRaceBench, incorporating new data race contributions from Wu et al. \cite{wu2023model}, and introduces a derived dataset named DataRaceBench-ML (DRB-ML) \cite{drbml}. DRB-ML aligns with the emerging trend of machine learning and large language models. Originating from DataRaceBench, this dataset includes detailed labels that denote the presence of a data race and provides comprehensive details of associated variables, such as variable names, line numbers, and the operation (read/write). Unique to DRB-ML, we have also integrated a series of tailored prompt-response pairs specifically designed for LLM fine-tuning.

cs.SE↗

arXiv2023

Towards Zero Memory Footprint Spiking Neural Network Training

Biologically-inspired Spiking Neural Networks (SNNs), processing information using discrete-time events known as spikes rather than continuous values, have garnered significant attention due to their hardware-friendly and energy-efficient characteristics. However, the training of SNNs necessitates a considerably large memory footprint, given the additional storage requirements for spikes or events, leading to a complex structure and dynamic setup. In this paper, to address memory constraint in SNN training, we introduce an innovative framework, characterized by a remarkably low memory footprint. We \textbf{(i)} design a reversible SNN node that retains a high level of accuracy. Our design is able to achieve a $\mathbf{58.65\times}$ reduction in memory usage compared to the current SNN node. We \textbf{(ii)} propose a unique algorithm to streamline the backpropagation process of our reversible SNN node. This significantly trims the backward Floating Point Operations Per Second (FLOPs), thereby accelerating the training process in comparison to current reversible layer backpropagation method. By using our algorithm, the training time is able to be curtailed by $\mathbf{23.8\%}$ relative to existing reversible layer architectures.

cs.NE↗

arXiv2023

Creating a Dataset for High-Performance Computing Code Translation using LLMs: A Bridge Between OpenMP Fortran and C++

In this study, we present a novel dataset for training machine learning models translating between OpenMP Fortran and C++ code. To ensure reliability and applicability, the dataset is created from a range of representative open-source OpenMP benchmarks. It is also refined using a meticulous code similarity test. The effectiveness of our dataset is assessed using both quantitative (CodeBLEU) and qualitative (human evaluation) methods. We showcase how this dataset significantly elevates the translation competencies of large language models (LLMs). Specifically, models without prior coding knowledge experienced a boost of $\mathbf{\times~5.1}$ in their CodeBLEU scores, while models with some coding familiarity saw an impressive $\mathbf{\times~9.9}$-fold increase. The best fine-tuned model using our dataset outperforms GPT-4. It is also reaching human-level accuracy. This work underscores the immense potential of our dataset in propelling advancements in the domain of code translation for high-performance computing. The dataset is accessible at \href{https://github.com/bin123apple/Fortran-CPP-HPC-code-translation-dataset}{OpenMP-Fortran-CPP-Translation}.

cs.SE↗

arXiv2023

LM4HPC: Towards Effective Language Model Application in High-Performance Computing

In recent years, language models (LMs), such as GPT-4, have been widely used in multiple domains, including natural language processing, visualization, and so on. However, applying them for analyzing and optimizing high-performance computing (HPC) software is still challenging due to the lack of HPC-specific support. In this paper, we design the LM4HPC framework to facilitate the research and development of HPC software analyses and optimizations using LMs. Tailored for supporting HPC datasets, AI models, and pipelines, our framework is built on top of a range of components from different levels of the machine learning software stack, with Hugging Face-compatible APIs. Using three representative tasks, we evaluated the prototype of our framework. The results show that LM4HPC can help users quickly evaluate a set of state-of-the-art models and generate insightful leaderboards.

cs.LG↗

arXiv2023

Structured Thoughts Automaton: First Formalized Execution Model for Auto-Regressive Language Models

In recent months, Language Models (LMs) have become a part of daily discourse, with focus on OpenAI and the potential of Artificial General Intelligence (AGI). Furthermore, the leaking of LLama's weights to the public has led to an influx of innovations demonstrating the impressive capabilities of generative LMs. While we believe that AGI is still a distant goal, we recognize the potential of LMs in solving tasks such as searching complex documents, compiling reports with basic analysis, and providing assistance in problem-solving. In this paper, we propose formalizing the execution model of language models. We investigate current execution models, to find that this formalism has received little attention, and present our contribution: the first formalized execution model for LMs. We introduce a new algorithm for sampling the predictions of LMs, which we use to build a reliable and inspectable execution model. We introduce a low-level language to write "cognitive program" for this execution model. We hope to shed light on the need for execution models for LMs and encourage further research in this area.

cs.CL↗

arXiv2022

Towards Seamless Management of AI Models in High-Performance Computing

With the increasing prevalence of artificial intelligence (AI) in diverse science/engineering communities, AI models emerge on an unprecedented scale among various domains. However, given the complexity and diversity of the software and hardware environments, reusing AI artifacts (models and datasets) is extremely challenging, especially with AI-driven science applications. Building an ecosystem to run and reuse AI applications/datasets at scale efficiently becomes increasingly essential for diverse science and engineering and high-performance computing (HPC) communities. In this paper, we innovate over an HPC-AI ecosystem -- HPCFair, which enables the Findable, Accessible, Interoperable, and Reproducible (FAIR) principles. HPCFair enables the collection of AI models/datasets allowing users to download/upload AI artifacts with authentications. Most importantly, our proposed framework provides user-friendly APIs for users to easily run inference jobs and customize AI artifacts to their tasks as needed. Our results show that, with HPCFair API, users irrespective of technical expertise in AI, can easily leverage AI artifacts to their tasks with minimal effort.

cs.DC↗

arXiv2022

Making Machine Learning Datasets and Models FAIR for HPC: A Methodology and Case Study

The FAIR Guiding Principles aim to improve the findability, accessibility, interoperability, and reusability of digital content by making them both human and machine actionable. However, these principles have not yet been broadly adopted in the domain of machine learning-based program analyses and optimizations for High-Performance Computing (HPC). In this paper, we design a methodology to make HPC datasets and machine learning models FAIR after investigating existing FAIRness assessment and improvement techniques. Our methodology includes a comprehensive, quantitative assessment for elected data, followed by concrete, actionable suggestions to improve FAIRness with respect to common issues related to persistent identifiers, rich metadata descriptions, license and provenance information. Moreover, we select a representative training dataset to evaluate our methodology. The experiment shows the methodology can effectively improve the dataset and model's FAIRness from an initial score of 19.1% to the final score of 83.0%.

cs.LG↗

arXiv2022

Finding Reusable Machine Learning Components to Build Programming Language Processing Pipelines

Programming Language Processing (PLP) using machine learning has made vast improvements in the past few years. Increasingly more people are interested in exploring this promising field. However, it is challenging for new researchers and developers to find the right components to construct their own machine learning pipelines, given the diverse PLP tasks to be solved, the large number of datasets and models being released, and the set of complex compilers or tools involved. To improve the findability, accessibility, interoperability and reusability (FAIRness) of machine learning components, we collect and analyze a set of representative papers in the domain of machine learning-based PLP. We then identify and characterize key concepts including PLP tasks, model architectures and supportive tools. Finally, we show some example use cases of leveraging the reusable components to construct machine learning pipelines to solve a set of PLP tasks.

cs.LG↗

arXiv2018

Simulating 3-D Stellar Hydrodynamics using PPM and PPB Multifluid Gas Dynamics on CPU and CPU+GPU Nodes

The special computational challenges of simulating 3-D hydrodynamics in deep stellar interiors are discussed, and numerical algorithmic responses described. Results of recent simulations carried out at scale on the NSF's Blue Waters machine at the University of Illinois are presented, with a special focus on the computational challenges they address. Prospects for future work using GPU-accelerated nodes such as those on the DoE's new Summit machine at Oak Ridge National Laboratory are described, with a focus on numerical algorithmic accommodations that we believe will be necessary.

astro-ph.SR↗

arXiv2014

Hydrodynamic simulations of H entrainment at the top of He-shell flash convection

We present the first 3-dimensional, fully compressible gas-dynamics simulations in $4π$ geometry of He-shell flash convection with proton-rich fuel entrainment at the upper boundary. This work is motivated by the insufficiently understood observed consequences of the H-ingestion flash in post-AGB stars (Sakurai's object) and metal-poor AGB stars. Our investigation is focused on the entrainment process at the top convection boundary and on the subsequent advection of H-rich material into deeper layers, and we therefore ignore the burning of the proton-rich fuel in this study. We find that, for our deep convection zone, coherent convective motions of near global scale appear to dominate the flow. At the top boundary convective shear flows are stable against Kelvin-Helmholtz instabilities. However, such shear instabilities are induced by the boundary-layer separation in large-scale, opposing flows. This links the global nature of thick shell convection with the entrainment process. We establish the quantitative dependence of the entrainment rate on grid resolution. With our numerical technique simulations with $1024^3$ cells or more are required to reach a numerical fidelity appropriate for this problem. However, only the result from the $1536^3$ simulation provides a clear indication that we approach convergence with regard to the entrainment rate. Our results demonstrate that our method, which is described in detail, can provide quantitative results related to entrainment and convective boundary mixing in deep stellar interior environments with veryvstiff convective boundaries. For the representative case we study in detail, we find an entrainment rate of $4.38 \pm 1.48 \times 10^{-13}M_\odot \mathrm{/s}$.

astro-ph.SR↗

arXiv2014

Global non-spherical oscillations in 3-D $4π$ simulations of the H-ingestion flash

We performed 3-D simulations of proton-rich material entrainment into \czw-rich He-shell flash convection and the subsequent H-ingestion flash that took place in the post-AGB star Sakurai's object. Observations of the transient nature and anomalous abundance features are available to validate our method and assumptions, with the aim to apply them to very low metallicity stars in the future. We include nuclear energy feedback from H burning and cover the full $4π$ geometry of the shell. Runs on $768^3$ and $1536^3$ grids agree well with each other and have been followed for $1500\mathrm{min}$ and $1200\mathrm{min}$. After a $850\mathrm{min}$ long quiescent entrainment phase the simulations enter into a global non-spherical oscillation that is launched and sustained by individual ignition events of H-rich fluid pockets. Fast circumferential flows collide at the antipode and cause the formation and localized ignition of the next H-overabundant pocket. The cycle repeats for more than a dozen times while its amplitude decreases. During the global oscillation the entrainment rate increases temporarily by a factor $\approx 100$. Entrained entropy quenches convective motions in the upper layer until the burning of entrained H establishes a separate convection zone. The lower-resolution run hints at the possibility that another global oscillation, perhaps even more violent will follow. The location of the H-burning convection zone agrees with a 1-D model in which the mixing efficiency is calibrated to reproduce the light curve. The simulations have been performed at the NSF Blue Waters supercomputer at NCSA.

astro-ph.SR↗

arXiv2009

The Hydrodynamic Environment for the s Process in the He-Shell Flash of AGB Stars

The He-shell flash convection in AGB stars is the site for the high-temperature component of the s-process in low- and intermediate mass giants, driven by the Ne22 neutron source. [...] The upper convection boundary plays a critical role during the H-ingestion episode that may lead to neutron-bursts in the most metal-poor AGB stars. We address these problems through global 3-dimensional hydrodynamic simulations including the entire spherical He-shell flash convection zone (as oposed to the 3D box-in-a-star simulations). An important aspect of our current effort is to establish the feasibility of our appoach. We explain why we favour the explicit treatment over the anelastic approximation for this problem. The simulations presented in this paper use a Cartesian grid of 512^3 cells and have been run on four 8-core workstations for four days to simulate ~5000s, which corresponds to almost ten convective turn-over times. The convection layer extends radially at the simulated point in the flash evolution over 7 H_p pressure scale-heights and exceeds the size of the underlying core. Convection is dominated by large convective cells that fill more than an entire octant. [...]

astro-ph.SR↗