arXiv subjects

Haojie Wang

Publications and source records attributed to Haojie Wang.

27 records · Page 2Linked to original sources

arXiv2025

TritonBench: Benchmarking Large Language Model Capabilities for Generating Triton Operators

Triton, a high-level Python-like language designed for building efficient GPU kernels, is widely adopted in deep learning frameworks due to its portability, flexibility, and accessibility. However, programming and parallel optimization still require considerable trial and error from Triton developers. Despite advances in large language models (LLMs) for conventional code generation, these models struggle to generate accurate, performance-optimized Triton code, as they lack awareness of its specifications and the complexities of GPU programming. More critically, there is an urgent need for systematic evaluations tailored to Triton. In this work, we introduce TritonBench, the first comprehensive benchmark for Triton operator generation. TritonBench features two evaluation channels: a curated set of 184 real-world operators from GitHub and a collection of operators aligned with PyTorch interfaces. Unlike conventional code benchmarks prioritizing functional correctness, TritonBench also profiles efficiency performance on widely deployed GPUs aligned with industry applications. Our study reveals that current state-of-the-art code LLMs struggle to generate efficient Triton operators, highlighting a significant gap in high-performance code generation. TritonBench will be available at https://github.com/thunlp/TritonBench.

cs.CL↗

arXiv2025

COMET:Combined Matrix for Elucidating Targets

Identifying the interaction targets of bioactive compounds is a foundational element for deciphering their pharmacological effects. Target prediction algorithms equip researchers with an effective tool to rapidly scope and explore potential targets. Here, we introduce the COMET, a multi-technological modular target prediction tool that provides comprehensive predictive insights, including similar active compounds, three-dimensional predicted binding modes, and probability scores, all within an average processing time of less than 10 minutes per task. With meticulously curated data, the COMET database encompasses 990,944 drug-target interaction pairs and 45,035 binding pockets, enabling predictions for 2,685 targets, which span confirmed and exploratory therapeutic targets for human diseases. In comparative testing using datasets from ChEMBL and BindingDB, COMET outperformed five other well-known algorithms, offering nearly an 80% probability of accurately identifying at least one true target within the top 15 predictions for a given compound. COMET also features a user-friendly web server, accessible freely at https://www.pdbbind-plus.org.cn/comet.

cs.LG↗

arXiv2024

A fast and efficient numerical method for computing the stress concentration between closely located stiff inclusions of general shapes

When two stiff inclusions are closely located, the gradient of the solution to the Lamé system, in other words the stress, may become arbitrarily large as the distance between two inclusions tends to zero. To compute the gradient of the solution in the narrow region, extremely fine meshes are required. It is a challenging problem to numerically compute the stress near the narrow region between two inclusions of general shapes as their distance goes to zero. A recent study [15] has shown that the major singularity of the gradient can be extracted in an explicit way for two general shaped inclusions. Thus the complexity of the computation can be greatly reduced by removing the singular term and it suffices to compute the residual term only using regular meshes. The goal of this paper is to numerically compute the stress concentration in a fast and efficient way. In this paper, we compute the value of the stress concentration factor, which is the normalized magnitude of the stress concentration, for general shaped domain as the distance between two inclusions tends to zero. We also compute the solution for two closely located inclusions of general shapes and show the convergence of the solution. Only regular meshes are used in our numerical computation and the results clearly show that the characterization of the singular term method can be efficiently used for computation of the stress concentration between two closely located inclusions of general shapes.

math.NA↗

arXiv2024

Optimal Kernel Orchestration for Tensor Programs with Korch

Kernel orchestration is the task of mapping the computation defined in different operators of a deep neural network (DNN) to the execution of GPU kernels on modern hardware platforms. Prior approaches optimize kernel orchestration by greedily applying operator fusion, which fuses the computation of multiple operators into a single kernel, and miss a variety of optimization opportunities in kernel orchestration. This paper presents Korch, a tensor program optimizer that discovers optimal kernel orchestration strategies for tensor programs. Instead of directly fusing operators, Korch first applies operator fission to decompose tensor operators into a small set of basic tensor algebra primitives. This decomposition enables a diversity of fine-grained, inter-operator optimizations. Next, Korch optimizes kernel orchestration by formalizing it as a constrained optimization problem, leveraging an off-the-shelf binary linear programming solver to discover an optimal orchestration strategy, and generating an executable that can be directly deployed on modern GPU platforms. Evaluation on a variety of DNNs shows that Korch outperforms existing tensor program optimizers by up to 1.7x on V100 GPUs and up to 1.6x on A100 GPUs. Korch is publicly available at https://github.com/humuyan/Korch.

cs.DS↗

arXiv2023

PowerFusion: A Tensor Compiler with Explicit Data Movement Description and Instruction-level Graph IR

Deep neural networks (DNNs) are of critical use in different domains. To accelerate DNN computation, tensor compilers are proposed to generate efficient code on different domain-specific accelerators. Existing tensor compilers mainly focus on optimizing computation efficiency. However, memory access is becoming a key performance bottleneck because the computational performance of accelerators is increasing much faster than memory performance. The lack of direct description of memory access and data dependence in current tensor compilers' intermediate representation (IR) brings significant challenges to generate memory-efficient code. In this paper, we propose IntelliGen, a tensor compiler that can generate high-performance code for memory-intensive operators by considering both computation and data movement optimizations. IntelliGen represent a DNN program using GIR, which includes primitives indicating its computation, data movement, and parallel strategies. This information will be further composed as an instruction-level dataflow graph to perform holistic optimizations by searching different memory access patterns and computation operations, and generating memory-efficient code on different hardware. We evaluate IntelliGen on NVIDIA GPU, AMD GPU, and Cambricon MLU, showing speedup up to 1.97x, 2.93x, and 16.91x(1.28x, 1.23x, and 2.31x on average), respectively, compared to current most performant frameworks.

cs.LG↗

arXiv2022

OLLIE: Derivation-based Tensor Program Optimizer

Boosting the runtime performance of deep neural networks (DNNs) is critical due to their wide adoption in real-world tasks. Existing approaches to optimizing the tensor algebra expression of a DNN only consider expressions representable by a fixed set of predefined operators, missing possible optimization opportunities between general expressions. We propose OLLIE, the first derivation-based tensor program optimizer. OLLIE optimizes tensor programs by leveraging transformations between general tensor algebra expressions, enabling a significantly larger expression search space that includes those supported by prior work as special cases. OLLIE uses a hybrid derivation-based optimizer that effectively combines explorative and guided derivations to quickly discover highly optimized expressions. Evaluation on seven DNNs shows that OLLIE can outperform existing optimizers by up to 2.73$\times$ (1.46$\times$ on average) on an A100 GPU and up to 2.68$\times$ (1.51$\times$) on a V100 GPU, respectively.

cs.LG↗

arXiv2022

2020 CATARACTS Semantic Segmentation Challenge

Surgical scene segmentation is essential for anatomy and instrument localization which can be further used to assess tissue-instrument interactions during a surgical procedure. In 2017, the Challenge on Automatic Tool Annotation for cataRACT Surgery (CATARACTS) released 50 cataract surgery videos accompanied by instrument usage annotations. These annotations included frame-level instrument presence information. In 2020, we released pixel-wise semantic annotations for anatomy and instruments for 4670 images sampled from 25 videos of the CATARACTS training set. The 2020 CATARACTS Semantic Segmentation Challenge, which was a sub-challenge of the 2020 MICCAI Endoscopic Vision (EndoVis) Challenge, presented three sub-tasks to assess participating solutions on anatomical structure and instrument segmentation. Their performance was assessed on a hidden test set of 531 images from 10 videos of the CATARACTS test set.

eess.IV↗

arXiv2020

ScalAna: Automating Scaling Loss Detection with Graph Analysis

Scaling a parallel program to modern supercomputers is challenging due to inter-process communication, Amdahl's law, and resource contention. Performance analysis tools for finding such scaling bottlenecks either base on profiling or tracing. Profiling incurs low overheads but does not capture detailed dependencies needed for root-cause analysis. Tracing collects all information at prohibitive overheads. In this work, we design ScalAna that uses static analysis techniques to achieve the best of both worlds - it enables the analyzability of traces at a cost similar to profiling. ScalAna first leverages static compiler techniques to build a Program Structure Graph, which records the main computation and communication patterns as well as the program's control structures. At runtime, we adopt lightweight techniques to collect performance data according to the graph structure and generate a Program Performance Graph. With this graph, we propose a novel approach, called backtracking root cause detection, which can automatically and efficiently detect the root cause of scaling loss. We evaluate ScalAna with real applications. Results show that our approach can effectively locate the root cause of scaling loss for real applications and incurs 1.73% overhead on average for up to 2,048 processes. We achieve up to 11.11% performance improvement by fixing the root causes detected by ScalAna on 2,048 processes.

cs.PF↗

arXiv2018

Deep Learning-Based BSDE Solver for Libor Market Model with Application to Bermudan Swaption Pricing and Hedging

The Libor market model is a mainstay term structure model of interest rates for derivatives pricing, especially for Bermudan swaptions, and other exotic Libor callable derivatives. For numerical implementation the pricing of derivatives with Libor market models is mainly carried out with Monte Carlo simulation. The PDE grid approach is not particularly feasible due to Curse of Dimensionality. The standard Monte Carlo method for American/Bermudan swaption pricing more or less uses regression to estimate expected value as a linear combination of basis functions (Longstaff and Schwartz). However, Monte Carlo method only provides the lower bound for American option price. Another complexity is the computation of the sensitivities of the option, the so-called Greeks, which are fundamental for a trader's hedging activity. Recently, an alternative numerical method based on deep learning and backward stochastic differential equations appeared in quite a few researches. For European style options the feedforward deep neural networks (DNN) show not only feasibility but also efficiency to obtain both prices and numerical Greeks. In this paper, a new backward DNN solver is proposed for Bermudan swaptions. Our approach is representing financial pricing problems in the form of high dimensional stochastic optimal control problems, FBSDEs, or equivalent PDEs. We demonstrate that using backward DNN the high-dimension Bermudan swaption pricing and hedging can be solved effectively and efficiently. A comparison between Monte Carlo simulation and the new method for pricing vanilla interest rate options manifests the superior performance of the new method. We then use this method to calculate prices and Greeks of Bermudan swaptions as a prelude for other Libor callable derivatives.

q-fin.CP↗