arXiv subjects
Publications and source records attributed to Le Chen.
We study the radius $R_T$ of a self-repellent fractional Brownian motion $\left\{B^H_t\right\}_{0\le t\le T}$ taking values in $\mathbb{R}^d$. Our sharpest result is for $d=1$, where we find that with high probability, \begin{equation*} R_T \asymp T^ν, \quad \text{with $ν=\frac{2}{3}\left(1+H\right)$.} \end{equation*} For $d>1$, we provide upper and lower bounds for the exponent $ν$, but these bounds do not match.
Large language models (LLMs) have become increasingly prominent in academia and industry due to their remarkable performance in diverse applications. As these models evolve with increasing parameters, they excel in tasks like sentiment analysis and machine translation. However, even models with billions of parameters face challenges in tasks demanding multi-step reasoning. Code generation and comprehension, especially in C and C++, emerge as significant challenges. While LLMs trained on code datasets demonstrate competence in many tasks, they struggle with rectifying non-compilable C and C++ code. Our investigation attributes this subpar performance to two primary factors: the quality of the training dataset and the inherent complexity of the problem which demands intricate reasoning. Existing "Chain of Thought" (CoT) prompting techniques aim to enhance multi-step reasoning. This approach, however, retains the limitations associated with the latent drawbacks of LLMs. In this work, we propose CompCodeVet, a compiler-guided CoT approach to produce compilable code from non-compilable ones. Diverging from the conventional approach of utilizing larger LLMs, we employ compilers as a teacher to establish a more robust zero-shot thought process. The evaluation of CompCodeVet on two open-source code datasets shows that CompCodeVet has the ability to improve the training dataset quality for LLMs.
As a promising fashion for visual localization, scene coordinate regression (SCR) has seen tremendous progress in the past decade. Most recent methods usually adopt neural networks to learn the mapping from image pixels to 3D scene coordinates, which requires a vast amount of annotated training data. We propose to leverage Neural Radiance Fields (NeRF) to generate training samples for SCR. Despite NeRF's efficiency in rendering, many of the rendered data are polluted by artifacts or only contain minimal information gain, which can hinder the regression accuracy or bring unnecessary computational costs with redundant data. These challenges are addressed in three folds in this paper: (1) A NeRF is designed to separately predict uncertainties for the rendered color and depth images, which reveal data reliability at the pixel level. (2) SCR is formulated as deep evidential learning with epistemic uncertainty, which is used to evaluate information gain and scene coordinate quality. (3) Based on the three arts of uncertainties, a novel view selection policy is formed that significantly improves data efficiency. Experiments on public datasets demonstrate that our method could select the samples that bring the most information gain and promote the performance with the highest efficiency.
We study the \textit{stochastic heat equation} (SHE) on $\R^d$ subject to a centered Gaussian noise that is white in time and colored in space.The drift term is assumed to satisfy an Osgood-type condition and the diffusion coefficient may have certain related growth. We show that there exists random field solution which do not explode in finite time. This complements and improves upon recent results on blow-up of solutions to stochastic partial differential equations.
Large language models (LLMs) are demonstrating significant promise as an alternate strategy to facilitate analyses and optimizations of high-performance computing programs, circumventing the need for resource-intensive manual tool creation. In this paper, we explore a novel LLM-based data race detection approach combining prompting engineering and fine-tuning techniques. We create a dedicated dataset named DRB-ML, which is derived from DataRaceBench, with fine-grain labels showing the presence of data race pairs and their associated variables, line numbers, and read/write information. DRB-ML is then used to evaluate representative LLMs and fine-tune open-source ones. Our experiment shows that LLMs can be a viable approach to data race detection. However, they still cannot compete with traditional data race detection tools when we need detailed information about variable pairs causing data races.
Large Language Models (LLMs), including the LLaMA model, have exhibited their efficacy across various general-domain natural language processing (NLP) tasks. However, their performance in high-performance computing (HPC) domain tasks has been less than optimal due to the specialized expertise required to interpret the model responses. In response to this challenge, we propose HPC-GPT, a novel LLaMA-based model that has been supervised fine-tuning using generated QA (Question-Answer) instances for the HPC domain. To evaluate its effectiveness, we concentrate on two HPC tasks: managing AI models and datasets for HPC, and data race detection. By employing HPC-GPT, we demonstrate comparable performance with existing methods on both tasks, exemplifying its excellence in HPC-related scenarios. Our experiments on open-source benchmarks yield extensive results, underscoring HPC-GPT's potential to bridge the performance gap between LLMs and HPC-specific tasks. With HPC-GPT, we aim to pave the way for LLMs to excel in HPC domains, simplifying the utilization of language models in complex computing applications.
In this study, we present a novel dataset for training machine learning models translating between OpenMP Fortran and C++ code. To ensure reliability and applicability, the dataset is created from a range of representative open-source OpenMP benchmarks. It is also refined using a meticulous code similarity test. The effectiveness of our dataset is assessed using both quantitative (CodeBLEU) and qualitative (human evaluation) methods. We showcase how this dataset significantly elevates the translation competencies of large language models (LLMs). Specifically, models without prior coding knowledge experienced a boost of $\mathbf{\times~5.1}$ in their CodeBLEU scores, while models with some coding familiarity saw an impressive $\mathbf{\times~9.9}$-fold increase. The best fine-tuned model using our dataset outperforms GPT-4. It is also reaching human-level accuracy. This work underscores the immense potential of our dataset in propelling advancements in the domain of code translation for high-performance computing. The dataset is accessible at \href{https://github.com/bin123apple/Fortran-CPP-HPC-code-translation-dataset}{OpenMP-Fortran-CPP-Translation}.
We construct an intrinsic family of Gaussian noises on $d$-dimensional flat torus $\mathbb{T}^d$. It is the analogue of the colored noise on $\mathbb{R}^d$, and allows us to study stochastic PDEs on torus in the Itô sense in high dimensions. With this noise, we consider the parabolic Anderson model (PAM) with measure-valued initial conditions and establish some basic properties of the solution, including a sharp upper and lower bound for the moments and Hölder continuity in space and time. The study of the toy model of $\mathbb{T}^d$ in the present paper is a first step towards our effort in understanding how geometry and topology play an role in the behavior of stochastic PDEs on general (compact) manifolds.
Data races pose a significant threat in multi-threaded parallel applications due to their negative impact on program correctness. DataRaceBench, an open-source benchmark suite, is specifically crafted to assess these data race detection tools in a systematic and measurable manner. Machine learning techniques have recently demonstrated considerable potential in high-performance computing (HPC) program analysis and optimization. However, these techniques require specialized data formats for training and refinement. This paper presents the latest update to DataRaceBench, incorporating new data race contributions from Wu et al. \cite{wu2023model}, and introduces a derived dataset named DataRaceBench-ML (DRB-ML) \cite{drbml}. DRB-ML aligns with the emerging trend of machine learning and large language models. Originating from DataRaceBench, this dataset includes detailed labels that denote the presence of a data race and provides comprehensive details of associated variables, such as variable names, line numbers, and the operation (read/write). Unique to DRB-ML, we have also integrated a series of tailored prompt-response pairs specifically designed for LLM fine-tuning.
We consider nonlinear parabolic stochastic PDEs on a bounded Lipschitz domain driven by a Gaussian noise that is white in time and colored in space, with Dirichlet or Neumann boundary condition. We establish existence, uniqueness and moment bounds of the random field solution under measure-valued initial data $ν$. We also study the two-point correlation function of the solution and obtain explicit upper and lower bounds. For $C^{1, α}$-domains with Dirichlet condition, the initial data $ν$ is not required to be a finite measure and the moment bounds can be improved under the weaker condition that the leading eigenfunction of the differential operator is integrable with respect to $|ν|$. As an application, we show that the solution is fully intermittent for sufficiently high level $λ$ of noise under the Dirichlet condition, and for all $λ> 0$ under the Neumann condition.
In recent years, language models (LMs), such as GPT-4, have been widely used in multiple domains, including natural language processing, visualization, and so on. However, applying them for analyzing and optimizing high-performance computing (HPC) software is still challenging due to the lack of HPC-specific support. In this paper, we design the LM4HPC framework to facilitate the research and development of HPC software analyses and optimizations using LMs. Tailored for supporting HPC datasets, AI models, and pipelines, our framework is built on top of a range of components from different levels of the machine learning software stack, with Hugging Face-compatible APIs. Using three representative tasks, we evaluated the prototype of our framework. The results show that LM4HPC can help users quickly evaluate a set of state-of-the-art models and generate insightful leaderboards.
In this paper, we investigate stochastic heat equation with sublinear diffusion coefficients. By assuming certain concavity of the diffusion coefficient, we establish non-trivial moment upper bounds and almost sure spatial asymptotic properties for the solutions. These results shed light on the smoothing intermittency effect under weak diffusion (i.e., sublinear growth) previously observed by Zeldovich et al. [Zel+87]. The sample-path spatial asymptotics obtained in this paper partially bridge a gap in earlier works of Conus et al. [CJK13; Con+13], which focused on two extreme scenarios: a linear diffusion coefficient and a bounded diffusion coefficient. Our approach is highly robust and applicable to a variety of stochastic partial differential equations, including the one-dimensional stochastic wave equation and the stochastic fractional diffusion equations.
Detecting parallelizable code regions is a challenging task, even for experienced developers. Numerous recent studies have explored the use of machine learning for code analysis and program synthesis, including parallelization, in light of the success of machine learning in natural language processing. However, applying machine learning techniques to parallelism detection presents several challenges, such as the lack of an adequate dataset for training, an effective code representation with rich information, and a suitable machine learning model to learn the latent features of code for diverse analyses. To address these challenges, we propose a novel graph-based learning approach called Graph2Par that utilizes a heterogeneous augmented abstract syntax tree (Augmented-AST) representation for code. The proposed approach primarily focused on loop-level parallelization with OpenMP. Moreover, we create an OMP\_Serial dataset with 18598 parallelizable and 13972 non-parallelizable loops to train the machine learning models. Our results show that our proposed approach achieves the accuracy of parallelizable code region detection with 85\% accuracy and outperforms the state-of-the-art token-based machine learning approach. These results indicate that our approach is competitive with state-of-the-art tools and capable of handling loops with complex structures that other tools may overlook.
Let $\{u(t\,, x)\}_{t >0, x \in\mathbb{R}}$ denote the solution to the parabolic Anderson model with initial condition $δ_0$ and driven by space-time white noise on $\mathbb{R}_+\times\mathbb{R}$, and let $p_t(x):= (2πt)^{-1/2}\exp\{-x^2/(2t)\}$ denote the standard Gaussian heat kernel on the line. We use a non-trivial adaptation of the methods in our companion papers \cite{CKNP,CKNP_b} in order to prove that the random field $x\mapsto u(t\,,x)/p_t(x)$ is ergodic for every $t >0$. And we establish an associated quantitative central limit theorem following the approach based on the Malliavin-Stein method introduced in Huang, Nualart, and Viitasaari \cite{HNV2018}.
In this paper, we tackle the problem of active robotic 3D reconstruction of an object. In particular, we study how a mobile robot with an arm-held camera can select a favorable number of views to recover an object's 3D shape efficiently. Contrary to the existing solution to this problem, we leverage the popular neural radiance fields-based object representation, which has recently shown impressive results for various computer vision tasks. However, it is not straightforward to directly reason about an object's explicit 3D geometric details using such a representation, making the next-best-view selection problem for dense 3D reconstruction challenging. This paper introduces a ray-based volumetric uncertainty estimator, which computes the entropy of the weight distribution of the color samples along each ray of the object's implicit neural representation. We show that it is possible to infer the uncertainty of the underlying 3D geometry given a novel view with the proposed estimator. We then present a next-best-view selection policy guided by the ray-based volumetric uncertainty in neural radiance fields-based representations. Encouraging experimental results on synthetic and real-world data suggest that the approach presented in this paper can enable a new research direction of using an implicit 3D object representation for the next-best-view problem in robot vision applications, distinguishing our approach from the existing approaches that rely on explicit 3D geometric modeling.
This paper deals with the long term behavior of the solution to the nonlinear stochastic heat equation $\partial u /\partial t - \frac{1}{2}Δu = b(u)\dot{W}$, where $b$ is assumed to be a globally Lipschitz continuous function and the noise $\dot{W}$ is a centered and spatially homogeneous Gaussian noise that is white in time. Using the moment formulas obtained in [9, 10], we identify a set of conditions on the initial data, the correlation measure and the weight function $ρ$, which will together guarantee the existence of an invariant measure in the weighted space $L^2_ρ(\mathbb{R}^d)$. In particular, our result includes the parabolic Anderson model (i.e., the case when $b(u) = λu$) starting from the Dirac delta measure.
In this paper, we study the stochastic heat equation (SHE) on $\mathbb{R}^d$ subject to a centered Gaussian noise that is white in time and colored in space. We establish the existence and uniqueness of the random field solution in the presence of locally Lipschitz drift and diffusion coefficients, which can have certain superlinear growth. This is a nontrivial extension of the recent work by Dalang, Khoshnevisan and Zhang (2019), where the one-dimensional SHE on $[0,1]$ subject to space-time white noise has been studied.
In this note, we consider the parabolic Anderson model on $\mathbb{R}_{+} \times \mathbb{R}$, driven by a Gaussian noise which is fractional in time with index $H_0>1/2$ and fractional in space with index $0 3/4$. Under a general condition on the initial data, we prove the existence and uniqueness of the mild solution and obtain its exponential upper bounds in time for all $p$-th moments with $p\ge 2$.