arXiv subjects

Jing Gong

Publications and source records attributed to Jing Gong.

At least 19 recordsLinked to original sources

arXiv2026

Multi-Partner Project: Multi-GPU Performance Portability Analysis for CFD Simulations at Scale

As heterogeneous supercomputing architectures leveraging GPUs become increasingly central to high-performance computing (HPC), it is crucial for computational fluid dynamics (CFD) simulations, a de-facto HPC workload, to efficiently utilize such hardware. One of the key challenges of HPC codes is performance portability, i.e. the ability to maintain near-optimal performance across different accelerators. In the context of the \textbf{REFMAP} project, which targets scalable, GPU-enabled multi-fidelity CFD for urban airflow prediction, this paper analyzes the performance portability of SOD2D, a state-of-the-art Spectral Elements simulation framework across AMD and NVIDIA GPU architectures. We first discuss the physical and numerical models underlying SOD2D, highlighting its computational hotspots. Then, we examine its performance and scalability in a multi-level manner, i.e. defining and characterizing an extensive full-stack design space spanning across application, software and hardware infrastructure related parameters. Single-GPU performance characterization across server-grade NVIDIA and AMD GPU architectures and vendor-specific compiler stacks, show the potential as well as the diverse effect of memory access optimizations, i.e. 0.69$\times$ - 3.91$\times$ deviations in acceleration speedup. Performance variability of SOD2D at scale is further examined on the LUMI multi-GPU cluster, where profiling reveals similar throughput variations, highlighting the limits of performance projections and the need for multi-level, informed tuning.

cs.DC↗

arXiv2025

HybridMamba: A Dual-domain Mamba for 3D Medical Image Segmentation

In the domain of 3D biomedical image segmentation, Mamba exhibits the superior performance for it addresses the limitations in modeling long-range dependencies inherent to CNNs and mitigates the abundant computational overhead associated with Transformer-based frameworks when processing high-resolution medical volumes. However, attaching undue importance to global context modeling may inadvertently compromise critical local structural information, thus leading to boundary ambiguity and regional distortion in segmentation outputs. Therefore, we propose the HybridMamba, an architecture employing dual complementary mechanisms: 1) a feature scanning strategy that progressively integrates representations both axial-traversal and local-adaptive pathways to harmonize the relationship between local and global representations, and 2) a gated module combining spatial-frequency analysis for comprehensive contextual modeling. Besides, we collect a multi-center CT dataset related to lung cancer. Experiments on MRI and CT datasets demonstrate that HybridMamba significantly outperforms the state-of-the-art methods in 3D medical image segmentation.

cs.CV↗

arXiv2025

Polymath: A Self-Optimizing Agent with Dynamic Hierarchical Workflow

Large language models (LLMs) excel at solving complex tasks by executing agentic workflows composed of detailed instructions and structured operations. Yet, building general-purpose agents by manually embedding foundation models into agentic systems such as Chain-of-Thought, Self-Reflection, and ReACT through text interfaces limits scalability and efficiency. Recently, many researchers have sought to automate the generation and optimization of these workflows through code-based representations. However, existing methods often rely on labeled datasets to train and optimize workflows, making them ineffective and inflexible for solving real-world, dynamic problems where labeled data is unavailable. To address this challenge, we introduce Polymath, a self-optimizing agent with dynamic hierarchical workflow that leverages the flexibility of task flow graphs and the expressiveness of code-represented workflows to solve a wide range of real-world, dynamic problems. The proposed optimization methodology integrates multi-grid-inspired graph optimization with a self-reflection-guided evolutionary algorithm to refine workflows without labeled data. Experimental results on six benchmark datasets across coding, math, and multi-turn QA tasks show that Polymath achieves 8.1% average improvement over state-of-the-art baselines.

cs.AI↗

arXiv2025

RustEvo^2: An Evolving Benchmark for API Evolution in LLM-based Rust Code Generation

Large Language Models (LLMs) have become pivotal tools for automating code generation in software development. However, these models face significant challenges in producing version-aware code for rapidly evolving languages like Rust, where frequent Application Programming Interfaces (API) changes across versions lead to compatibility issues and correctness errors. Existing benchmarks lack systematic evaluation of how models navigate API transitions, relying on labor-intensive manual curation and offering limited version-specific insights. To address this gap, we present RustEvo, a novel framework for constructing dynamic benchmarks that evaluate the ability of LLMs to adapt to evolving Rust APIs. RustEvo automates dataset creation by synthesizing 588 API changes (380 from Rust standard libraries, 208 from 15 third-party crates) into programming tasks mirroring real-world challenges. These tasks cover four API evolution categories: Stabilizations, Signature Changes, Behavioral Changes, and Deprecations, reflecting their actual distribution in the Rust ecosystem. Experiments on state-of-the-art (SOTA) LLMs reveal significant performance variations: models achieve a 65.8% average success rate on stabilized APIs but only 38.0% on behavioral changes, highlighting difficulties in detecting semantic shifts without signature alterations. Knowledge cutoff dates strongly influence performance, with models scoring 56.1% on before-cutoff APIs versus 32.5% on after-cutoff tasks. Retrieval-Augmented Generation (RAG) mitigates this gap, improving success rates by 13.5% on average for APIs released after model training. Our findings underscore the necessity of our evolution-aware benchmarks to advance the adaptability of LLMs in fast-paced software ecosystems. The framework and the benchmarks are publicly released at https://github.com/SYSUSELab/RustEvo.

cs.SE↗

arXiv2025

Marco: Configurable Graph-Based Task Solving and Multi-AI Agents Framework for Hardware Design

Hardware design presents numerous challenges stemming from its complexity and advancing technologies. These challenges result in longer turn-around-time (TAT) for optimizing performance, power, area, and cost (PPAC) during synthesis, verification, physical design, and reliability loops. Large Language Models (LLMs) have shown remarkable capacity to comprehend and generate natural language at a massive scale, leading to many potential applications and benefits across various domains. Successful LLM-based agents for hardware design can drastically reduce TAT, leading to faster product cycles, lower costs, improved design reliability and reduced risk of costly errors. In this work, we propose a unified framework, Marco, that integrates configurable graph-based task solving with multi-modality and multi-AI agents for chip design by leveraging the natural language and reasoning abilities with collaborative toolkits. Lastly, we demonstrate promising performance, productivity, and efficiency of LLM agents by leveraging the Marco framework on layout optimization, Verilog/design rule checker (DRC) coding, and timing analysis tasks.

cs.AR↗

arXiv2024

Training Compute-Optimal Protein Language Models

We explore optimally training protein language models, an area of significant interest in biological research where guidance on best practices is limited. Most models are trained with extensive compute resources until performance gains plateau, focusing primarily on increasing model sizes rather than optimizing the efficient compute frontier that balances performance and compute budgets. Our investigation is grounded in a massive dataset consisting of 939 million protein sequences. We trained over 300 models ranging from 3.5 million to 10.7 billion parameters on 5 to 200 billion unique tokens, to investigate the relations between model sizes, training token numbers, and objectives. First, we observed the effect of diminishing returns for the Causal Language Model (CLM) and that of overfitting for the Masked Language Model~(MLM) when repeating the commonly used Uniref database. To address this, we included metagenomic protein sequences in the training set to increase the diversity and avoid the plateau or overfitting effects. Second, we obtained the scaling laws of CLM and MLM on Transformer, tailored to the specific characteristics of protein sequence data. Third, we observe a transfer scaling phenomenon from CLM to MLM, further demonstrating the effectiveness of transfer through scaling behaviors based on estimated Effectively Transferred Tokens. Finally, to validate our scaling laws, we compare the large-scale versions of ESM-2 and PROGEN2 on downstream tasks, encompassing evaluations of protein generation as well as structure- and function-related tasks, all within less or equivalent pre-training compute budgets.

cs.LG↗

arXiv2024

A Historical Trajectory Assisted Optimization Method for Zeroth-Order Federated Learning

Federated learning heavily relies on distributed gradient descent techniques. In the situation where gradient information is not available, the gradients need to be estimated from zeroth-order information, which typically involves computing finite-differences along isotropic random directions. This method suffers from high estimation errors, as the geometric features of the objective landscape may be overlooked during the isotropic sampling. In this work, we propose a non-isotropic sampling method to improve the gradient estimation procedure. Gradients in our method are estimated in a subspace spanned by historical trajectories of solutions, aiming to encourage the exploration of promising regions and hence improve the convergence. The proposed method uses a covariance matrix for sampling which is a convex combination of two parts. The first part is a thin projection matrix containing the basis of the subspace which is designed to improve the exploitation ability. The second part is the historical trajectories. We implement this method in zeroth-order federated settings, and show that the convergence rate aligns with existing ones while introducing no significant overheads in communication or local computation. The effectiveness of our proposal is verified on several numerical experiments in comparison to several commonly-used zeroth-order federated optimization algorithms.

cs.LG↗

arXiv2024

CoSQA+: Pioneering the Multi-Choice Code Search Benchmark with Test-Driven Agents

Semantic code search, retrieving code that matches a given natural language query, is an important task to improve productivity in software engineering. Existing code search datasets face limitations: they rely on human annotators who assess code primarily through semantic understanding rather than functional verification, leading to potential inaccuracies and scalability issues. Additionally, current evaluation metrics often overlook the multi-choice nature of code search. This paper introduces CoSQA+, pairing high-quality queries from CoSQA with multiple suitable codes. We develop an automated pipeline featuring multiple model-based candidate selections and the novel test-driven agent annotation system. Among a single Large Language Model (LLM) annotator and Python expert annotators (without test-based verification), agents leverage test-based verification and achieve the highest accuracy of 93.9%. Through extensive experiments, CoSQA+ has demonstrated superior quality over CoSQA. Models trained on CoSQA+ exhibit improved performance. We publicly release both CoSQA+_all, which contains 412,080 agent-annotated pairs, and CoSQA+_verified, which contains 1,000 human-verified pairs, at https://github.com/DeepSoftwareAnalytics/CoSQA_Plus.

cs.SE↗

arXiv2024

xTrimoGene: An Efficient and Scalable Representation Learner for Single-Cell RNA-Seq Data

Advances in high-throughput sequencing technology have led to significant progress in measuring gene expressions at the single-cell level. The amount of publicly available single-cell RNA-seq (scRNA-seq) data is already surpassing 50M records for humans with each record measuring 20,000 genes. This highlights the need for unsupervised representation learning to fully ingest these data, yet classical transformer architectures are prohibitive to train on such data in terms of both computation and memory. To address this challenge, we propose a novel asymmetric encoder-decoder transformer for scRNA-seq data, called xTrimoGene$^α$ (or xTrimoGene for short), which leverages the sparse characteristic of the data to scale up the pre-training. This scalable design of xTrimoGene reduces FLOPs by one to two orders of magnitude compared to classical transformers while maintaining high accuracy, enabling us to train the largest transformer models over the largest scRNA-seq dataset today. Our experiments also show that the performance of xTrimoGene improves as we scale up the model sizes, and it also leads to SOTA performance over various downstream tasks, such as cell type annotation, perturb-seq effect prediction, and drug combination prediction. xTrimoGene model is now available for use as a service via the following link: https://api.biomap.com/xTrimoGene/apply.

cs.LG↗

arXiv2024

xTrimoPGLM: Unified 100B-Scale Pre-trained Transformer for Deciphering the Language of Protein

Protein language models have shown remarkable success in learning biological information from protein sequences. However, most existing models are limited by either autoencoding or autoregressive pre-training objectives, which makes them struggle to handle protein understanding and generation tasks concurrently. We propose a unified protein language model, xTrimoPGLM, to address these two types of tasks simultaneously through an innovative pre-training framework. Our key technical contribution is an exploration of the compatibility and the potential for joint optimization of the two types of objectives, which has led to a strategy for training xTrimoPGLM at an unprecedented scale of 100 billion parameters and 1 trillion training tokens. Our extensive experiments reveal that 1) xTrimoPGLM significantly outperforms other advanced baselines in 18 protein understanding benchmarks across four categories. The model also facilitates an atomic-resolution view of protein structures, leading to an advanced 3D structural prediction model that surpasses existing language model-based tools. 2) xTrimoPGLM not only can generate de novo protein sequences following the principles of natural ones, but also can perform programmable generation after supervised fine-tuning (SFT) on curated sequences. These results highlight the substantial capability and versatility of xTrimoPGLM in understanding and generating protein sequences, contributing to the evolving landscape of foundation models in protein science.

q-bio.QM↗

arXiv2022

ApproxTrain: Fast Simulation of Approximate Multipliers for DNN Training and Inference

Edge training of Deep Neural Networks (DNNs) is a desirable goal for continuous learning; however, it is hindered by the enormous computational power required by training. Hardware approximate multipliers have shown their effectiveness for gaining resource-efficiency in DNN inference accelerators; however, training with approximate multipliers is largely unexplored. To build resource efficient accelerators with approximate multipliers supporting DNN training, a thorough evaluation of training convergence and accuracy for different DNN architectures and different approximate multipliers is needed. This paper presents ApproxTrain, an open-source framework that allows fast evaluation of DNN training and inference using simulated approximate multipliers. ApproxTrain is as user-friendly as TensorFlow (TF) and requires only a high-level description of a DNN architecture along with C/C++ functional models of the approximate multiplier. We improve the speed of the simulation at the multiplier level by using a novel LUT-based approximate floating-point (FP) multiplier simulator on GPU (AMSim). ApproxTrain leverages CUDA and efficiently integrates AMSim into the TensorFlow library, in order to overcome the absence of native hardware approximate multiplier in commercial GPUs. We use ApproxTrain to evaluate the convergence and accuracy of DNN training with approximate multipliers for small and large datasets (including ImageNet) using LeNets and ResNets architectures. The evaluations demonstrate similar convergence behavior and negligible change in test accuracy compared to FP32 and bfloat16 multipliers. Compared to CPU-based approximate multiplier simulations in training and inference, the GPU-accelerated ApproxTrain is more than 2500x faster. Based on highly optimized closed-source cuDNN/cuBLAS libraries with native hardware multipliers, the original TensorFlow is only 8x faster than ApproxTrain.

cs.AR↗

arXiv2022

Critical behavior in the Mn$_{5}$Ge$_{3}$ ferromagnet

High-Curie-temperature ferromagnets are promising candidates for designing new spintronic devices. Here we have successfully synthesized a single-crystal sample of the itinerant ferromagnet Mn$ _{5}$Ge$_{3}$ used flux method and its critical properties were investigated by means of bulk dc-magnetization at the boundary between the ferromagnetic (FM) and paramagnetic (PM) phase. Critical exponents $ β=0.336 \pm 0.001 $ with a critical temperature $ T_{c}=300.29 \pm 0.01 $ K and $ γ=1.193 \pm 0.003 $ with $ T_{c} = 300.15 \pm 0.05 $ K are obtained by the modified Arrott plot, whereas $ δ= 4.61 \pm 0.03 $ is deduced by a critical isotherm analysis at $ T_{c} = 300 $ K. The self-consistency and reliability of these critical exponents are verified by the Widom scaling law and the scaling equations. Further analysis reveals that the spin coupling in Mn$ _{5}$Ge$_{3}$ exhibits three-dimensional Ising-like behavior. The magnetic exchange is found to decay as $ J(r)\approx r^{-4.855} $ and the spin interactions are extended beyond the nearest neighbors, which may be related to different set of Mn--Mn interactions with unequal magnitude of exchange strengths. Additionally, the existence of noncollinear spin configurations in Mn$ _{5} $Ge$ _{3} $ results in a small deviation of obtained critical exponents from those for standard 3D-Ising model.

cond-mat.mtrl-sci↗

arXiv2022

Large anomalous Hall effect in layered antiferromagnet Co$_{0.29}$TaS$_2$

We present a study on the magnetization, anomalous Hall effect (AHE) and novel longitudinal resistivity in layered antiferromagnet Co$_{0.29}$TaS$_{2}$. Of particular interests in Co$_{0.29}$TaS$_{2}$ are abundant magnetic transitions, which show that the magnetic structures are tuned by temperature or magnetic field. With decreasing temperature, Co$_{0.29}$TaS$_{2}$ undergoes two transitions at T$_{t1}\sim$ 38.3 K and T$_{t2}\sim$ 24.3 K. Once the magnetic field is applied, another transition T$_{t3}\sim$ 34.3 K appears between 0.3 T and 5 T. At 2 K, an obvious ferromagnetic hysteresis loop within H$_{t1}\sim\pm$ 6.9 T is observed, which decreases with increasing temperature and eventually disappears at T$_{t2}$. Besides, Co$_{0.29}$TaS$_{2}$ displays step-like behavior as another magnetic transition around H$_{t2}\sim\pm$ 4 T, which exists until $\sim$ T$_{t1}$. These characteristic temperatures and magnetic fields mark complex magnetic phase transitions in Co$_{0.29}$TaS$_{2}$, which are also evidenced in transport results. Large AHE dominates in the Hall resistivity with the conspicuous value of R$_{s}$/R$_{0}\sim 10^{5}$, considering that the tiny net magnetization (0.0094$μ_{B}$/Co) alone would not lead to this value, thus the contribution of Berry curvature is necessary. The longitudinal resistivity illustrates a prominent irreversible behavior within H$_{t1}$. The abrupt change at H$_{t2}$ below T$_{t1}$, corresponding to the step-like magnetic transitions, is also observed. Synergy between the magnetism and topological properties, both playing a crucial role, may be the key factor of large AHE in antiferromagnet, which also offers a new perspective in magnetic topological materials with the platform of Co$_{0.29}$TaS$_{2}$.

cond-mat.mtrl-sci↗

arXiv2022

Quantum oscillations and weak anisotropic resistivity in the chiral Fermion semimetal PdGa

We perform a detailed analysis of the magnetotransport and de Haas-van Alphen (dHvA) oscillations in crystal PdGa which is predicted to be a typical chiral Fermion semimetal from CoSi family holding a large Chern number. The unsaturated quadratic magnetoresistance (MR) and nonlinear Hall resistivity indicate that PdGa is a multi-band system without electron-hole compensation. Angle-dependent resistivity in PdGa shows weak anisotropy with twofold or threefold symmetry when the magnetic field rotates within the (1$\bar{1}$0) or (111) plane perpendicular to the current. Nine or three frequencies are extracted after the fast Fourier-transform analysis (FFT) of the dHvA oscillations with B//[001] or B//[011], respectively, which is confirmed to be consistent with the Fermi surfaces (FSs) obtained from first-principles calculations with spin-orbit coupling (SOC) considered.

cond-mat.mtrl-sci↗

arXiv2021

Strong Scaling of OpenACC enabled Nek5000 on several GPU based HPC systems

We present new results on the strong parallel scaling for the OpenACC-accelerated implementation of the high-order spectral element fluid dynamics solver Nek5000. The test case considered consists of a direct numerical simulation of fully-developed turbulent flow in a straight pipe, at two different Reynolds numbers $Re_τ=360$ and $Re_τ=550$, based on friction velocity and pipe radius. The strong scaling is tested on several GPU-enabled HPC systems, including the Swiss Piz Daint system, TACC's Longhorn, Jülich's JUWELS Booster, and Berzelius in Sweden. The performance results show that speed-up between 3-5 can be achieved using the GPU accelerated version compared with the CPU version on these different systems. The run-time for 20 timesteps reduces from 43.5 to 13.2 seconds with increasing the number of GPUs from 64 to 512 for $Re_τ=550$ case on JUWELS Booster system. This illustrates the GPU accelerated version the potential for high throughput. At the same time, the strong scaling limit is significantly larger for GPUs, at about $2000-5000$ elements per rank; compared to about $50-100$ for a CPU-rank.

cs.DC↗

arXiv2020

Improving Machine Reading Comprehension with Single-choice Decision and Transfer Learning

Multi-choice Machine Reading Comprehension (MMRC) aims to select the correct answer from a set of options based on a given passage and question. Due to task specific of MMRC, it is non-trivial to transfer knowledge from other MRC tasks such as SQuAD, Dream. In this paper, we simply reconstruct multi-choice to single-choice by training a binary classification to distinguish whether a certain answer is correct. Then select the option with the highest confidence score. We construct our model upon ALBERT-xxlarge model and estimate it on the RACE dataset. During training, We adopt AutoML strategy to tune better parameters. Experimental results show that the single-choice is better than multi-choice. In addition, by transferring knowledge from other kinds of MRC tasks, our model achieves a new state-of-the-art results in both single and ensemble settings.

cs.CL↗

arXiv2020

Towards Scalable Distributed Training of Deep Learning on Public Cloud Clusters

Distributed training techniques have been widely deployed in large-scale deep neural networks (DNNs) training on dense-GPU clusters. However, on public cloud clusters, due to the moderate inter-connection bandwidth between instances, traditional state-of-the-art distributed training systems cannot scale well in training large-scale models. In this paper, we propose a new computing and communication efficient top-k sparsification communication library for distributed training. To further improve the system scalability, we optimize I/O by proposing a simple yet efficient multi-level data caching mechanism and optimize the update operation by introducing a novel parallel tensor operator. Experimental results on a 16-node Tencent Cloud cluster (each node with 8 Nvidia Tesla V100 GPUs) show that our system achieves 25%-40% faster than existing state-of-the-art systems on CNNs and Transformer. We finally break the record on DAWNBench on training ResNet-50 to 93% top-5 accuracy on ImageNet.

cs.DC↗

arXiv2020

Inelastic electron tunneling in 2H-Ta$_x$Nb$_{1-x}$Se$_2$ evidenced by scanning tunneling spectroscopy

We report a detailed study of tunneling spectra measured on 2H-Ta$_x$Nb$_{1-x}$Se$_2$ ($x=0\sim 0.1$) single crystals using a low-temperature scanning tunneling microscope. The prominent gap-like feature unintelligible for a long time was found to be accompanied by some "in-gap" fine structures. By investigating the second-derivative spectra and their temperature and magnetic field dependencies, we were able to prove that inelastic electron tunneling is the origin of these features and obtain the Eliashberg function of 2H-Ta$_x$Nb$_{1-x}$Se$_2$ at atomic scale, providing a potential way to study the local Eliashberg function and phonon spectra of the related transition-metal dichalcogenides.

cond-mat.mtrl-sci↗