arXiv subjects

Zhenhao Li

Publications and source records attributed to Zhenhao Li.

At least 55 records · Page 3Linked to original sources

arXiv2025

A Milli-Kelvin Atomic Force Microscope Made of Glass

Milli-Kelvin atomic force microscopy (mK-AFM) presents an ongoing experimental challenge due to the intense vibrations in a cryogen-free dilution refrigerator and the low cooling power available at mK temperatures. A viable approach is to make the system exceptionally rigid and thermally insulating to decouple external vibrations and isolate heat dissipation from the piezo elements. Here, we present a low-cost and large scan-range mK-AFM that operates below 100 mK. All the essential parts of our mK-AFM, including the scanners, tip assembly, and microscope body, are custom-made of fused silica glass by taking advantage of its high specific modulus, extremely low thermal expansion coefficient, and excellent thermal insulation properties. We carefully balance the scan range (25 $μ$m $\times$ 25 $μ$m), heat dissipation, and stiffness of the system to reach optimal performance at mK temperatures.

physics.ins-det↗

arXiv2025

Internal waves in a 2D subcritical channel

We analyze the scattering of linear internal waves in a two dimensional channel with subcritical bottom topography. We construct the scattering matrix for the internal wave problem in a channel with straight ends, mapping incoming data to outgoing data; this operator turns out to differ by a smoothing operator from the pullback by the ``bounce map'' for boundary data obtained by ray-tracing. As a consequence we obtain unique solvability of the inhomogeneous stationary scattering problem subject to an appropriate outgoing radiation condition.

math.AP↗

arXiv2025

An Empirical Study of Retrieval-Augmented Code Generation: Challenges and Opportunities

Code generation aims to automatically generate code snippets of specific programming language according to natural language descriptions. The continuous advancements in deep learning, particularly pre-trained models, have empowered the code generation task to achieve remarkable performance. One main challenge of pre-trained models for code generation is the semantic gap between natural language requirements and source code. To address the issue, prior studies typically adopt a retrieval-augmented framework for the task, where the similar code snippets collected by a retrieval process can be leveraged to help understand the requirements and provide guidance for the generation process. However, there is a lack of systematic study on the application of this framework for code generation, including the impact of the final generated results and the specific usage of the framework. In this paper, we choose three popular pre-trained code models, namely CodeGen, UniXcoder, and CodeT5, to assess the impact of the quality and utilization of retrieved code on the retrieval-augmented framework. Our analysis shows that the retrieval-augmented framework is beneficial for improving the performance of the existing pre-trained models. We also provide suggestions on the utilization of the retrieval-augmented code generation framework: BM25 and Sequential Integration Fusion are recommended due to their convenience and superior performance. Sketch Filling Fusion, which extracts a sketch of relevant code, could help the model improve its performance further. Additionally, we conduct experiments to investigate the influence of the retrieval-augmented framework on large language models for code generation, showing the effectiveness of the framework, and we discuss the trade-off between performance improvement and computational costs in each phase within the framework.

cs.SE↗

arXiv2025

Synthetic notions of Ricci flow for metric measure spaces

We develop different synthetic notions of Ricci flow in the setting of time-dependent metric measure spaces based on ideas from optimal transport. They are formulated in terms of dynamic convexity and local concavity of the entropy along Wasserstein geodesics on the one hand and in terms of global and short-time asymptotic transport cost estimates for the heat flow on the other hand. We show that these properties characterise smooth (weighted) Ricci flows. Further, we investigate the relation between the different notions in the non-smooth setting of time-dependent metric measure spaces.

math.DG↗

arXiv2024

Discovery of Timeline and Crowd Reaction of Software Vulnerability Disclosures

Reusing third-party libraries increases productivity and saves time and costs for developers. However, the downside is the presence of vulnerabilities in those libraries, which can lead to catastrophic outcomes. For instance, Apache Log4J was found to be vulnerable to remote code execution attacks. A total of more than 35,000 packages were forced to update their Log4J libraries with the latest version. Although several studies have been conducted to predict software vulnerabilities, the prediction does not cover the vulnerabilities found in third-party libraries. Even if the developers are aware of the forthcoming issue, replicating a function similar to the libraries would be time-consuming and labour-intensive. Nevertheless, it is practically reasonable for software developers to update their third-party libraries (and dependencies) whenever the software vendors have released a vulnerable-free version. In this work, our manual study focuses on the real-world practices (crowd reaction) adopted by software vendors and developer communities when a vulnerability is disclosed. We manually investigated 312 CVEs and identified that the primary trend of vulnerability handling is to provide a fix before publishing an announcement. Otherwise, developers wait an average of 10 days for a fix if it is unavailable upon the announcement. Additionally, the crowd reaction is oblivious to the vulnerability severity. In particular, we identified Oracle as the most vibrant community diligent in releasing fixes. Their software developers also actively participate in the associated vulnerability announcements.

cs.SE↗

arXiv2024

Studying and Benchmarking Large Language Models For Log Level Suggestion

Large Language Models (LLMs) have become a focal point of research across various domains, including software engineering, where their capabilities are increasingly leveraged. Recent studies have explored the integration of LLMs into software development tools and frameworks, revealing their potential to enhance performance in text and code-related tasks. Log level is a key part of a logging statement that allows software developers control the information recorded during system runtime. Given that log messages often mix natural language with code-like variables, LLMs' language translation abilities could be applied to determine the suitable verbosity level for logging statements. In this paper, we undertake a detailed empirical analysis to investigate the impact of characteristics and learning paradigms on the performance of 12 open-source LLMs in log level suggestion. We opted for open-source models because they enable us to utilize in-house code while effectively protecting sensitive information and maintaining data security. We examine several prompting strategies, including Zero-shot, Few-shot, and fine-tuning techniques, across different LLMs to identify the most effective combinations for accurate log level suggestions. Our research is supported by experiments conducted on 9 large-scale Java systems. The results indicate that although smaller LLMs can perform effectively with appropriate instruction and suitable techniques, there is still considerable potential for improvement in their ability to suggest log levels.

cs.SE↗

arXiv2024

Uniqueness of excited states to $-Δu+u-u^3=0$ in three dimensions

We prove the uniqueness of several excited states to the ODE $\ddot y(t) + \frac{2}{t} \dot y(t) + f(y(t)) = 0$, $y(0) = b$, and $\dot y(0) = 0$ for the model nonlinearity $f(y) = y^3 - y$. The $n$-th excited state is a solution with exactly $n$ zeros and which tends to $0$ as $t \to \infty$. These represent all smooth radial nonzero solutions to the PDE $Δu + f(u)= 0$ in $H^1$. We interpret the ODE as a damped oscillator governed by a double-well potential, and the result is proved via rigorous numerical analysis of the energy and variation of the solutions. More specifically, the problem of uniqueness can be formulated entirely in terms of inequalities on the solutions and their variation, and these inequalities can be verified numerically.

math.CA↗

arXiv2024

Reasoning Runtime Behavior of a Program with LLM: How Far Are We?

Large language models for code (i.e., code LLMs) have shown strong code understanding and generation capabilities. To evaluate the capabilities of code LLMs in various aspects, many benchmarks have been proposed (e.g., HumanEval and ClassEval). Code reasoning is one of the most essential abilities of code LLMs, but existing benchmarks for code reasoning are not sufficient. Typically, they focus on predicting the input and output of a program, ignoring the evaluation of the intermediate behavior during program execution, as well as the logical consistency (e.g., the model should not give the correct output if the prediction of execution path is wrong) when performing the reasoning. To address these problems, in this paper, we propose a framework, namely REval, for evaluating code reasoning abilities and consistency of code LLMs with program execution. We utilize existing code benchmarks and adapt them to new benchmarks within our framework. A large-scale empirical study is conducted and most LLMs show unsatisfactory performance on both Runtime Behavior Reasoning (i.e., an average accuracy of 44.4%) and Incremental Consistency Evaluation (i.e., an average IC score of 10.3). Evaluation results of current code LLMs reflect the urgent need for the community to strengthen the code reasoning capability of code LLMs. Our code, data, and \newname leaderboard are available at https://r-eval.github.io.

cs.SE↗

arXiv2024

ThinkRepair: Self-Directed Automated Program Repair

Though many approaches have been proposed for Automated Program Repair (APR) and indeed achieved remarkable performance, they still have limitations in fixing bugs that require analyzing and reasoning about the logic of the buggy program. Recently, large language models (LLMs) instructed by prompt engineering have attracted much attention for their powerful ability to address many kinds of tasks including bug-fixing. However, the quality of the prompt will highly affect the ability of LLMs and manually constructing high-quality prompts is a costly endeavor. To address this limitation, we propose a self-directed LLM-based automated program repair, ThinkRepair, with two main phases: collection phase and fixing phase. The former phase automatically collects various chains of thoughts that constitute pre-fixed knowledge by instructing LLMs with the Chain-of-Thought (CoT) prompt. The latter phase targets fixing a bug by first selecting examples for few-shot learning and second automatically interacting with LLMs, optionally appending with feedback of testing information. Evaluations on two widely studied datasets (Defects4J and QuixBugs) by comparing ThinkRepair with 12 SOTA APRs indicate the priority of ThinkRepair in fixing bugs. Notably, ThinkRepair fixes 98 bugs and improves baselines by 27%-344.4% on Defects4J V1.2. On Defects4J V2.0, ThinkRepair fixes 12-65 more bugs than the SOTA APRs. Additionally, ThinkRepair also makes a considerable improvement on QuixBugs (31 for Java and 21 for Python at most).

cs.SE↗

arXiv2024

NLPerturbator: Studying the Robustness of Code LLMs to Natural Language Variations

Large language models (LLMs) achieve promising results in code generation based on a given natural language description. They have been integrated into open-source projects and commercial products to facilitate daily coding activities. The natural language description in the prompt is crucial for LLMs to comprehend users' requirements. Prior studies uncover that LLMs are sensitive to the changes in the prompts, including slight changes that look inconspicuous. However, the natural language descriptions often vary in real-world scenarios (e.g., different formats, grammar, and wording). Prior studies on the robustness of LLMs are often based on random perturbations and such perturbations may not actually happen. In this paper, we conduct a comprehensive study to investigate how are code LLMs robust to variations of natural language description in real-world scenarios. We summarize 18 categories of perturbations of natural language and 3 combinations of co-occurred categories based on our literature review and an online survey with practitioners. We propose an automated framework, NLPerturbator, which can perform perturbations of each category given a set of prompts. Through a series of experiments on code generation using six code LLMs, we find that the perturbed prompts can decrease the performance of code generation by a considerable margin (e.g., up to 21.2%, and 4.8% to 6.1% on average). Our study highlights the importance of enhancing the robustness of LLMs to real-world variations in the prompts, as well as the essentiality of attentively constructing the prompts.

cs.SE↗

arXiv2024

Deep Learning Based Code Generation Methods: Literature Review

This paper focuses on Code Generation task that aims at generating relevant code fragments according to given natural language descriptions. In the process of software development, developers often encounter two scenarios. One is requested to write a large amount of repetitive and low-technical code for implementing common functionalities. The other is writing code that depends on specific task requirements, which may necessitate the use of external resources such as documentation or other tools. Therefore, code generation has received a lot of attention among academia and industry for assisting developers in coding. In fact, it has also been one of the key concerns in the field of software engineering to make machines understand users' requirements and write programs on their own. The recent development of deep learning techniques especially pre-training models make the code generation task achieve promising performance. In this paper, we systematically review the current work on deep learning-based code generation and classify the current deep learning-based code generation methods into three categories: methods based on code features, methods incorporated with retrieval, and methods incorporated with post-processing. The first category refers to the methods that use deep learning algorithms for code generation based on code features, and the second and third categories of methods improve the performance of the methods in the first category. In this paper, the existing research results of each category of methods are systematically reviewed, summarized and commented. Besides, the paper summarizes and analyzes the corpus and the popular evaluation metrics used in the existing code generation work. Finally, the paper summarizes the overall literature review and provides a prospect on future research directions worthy of attention.

cs.SE↗

arXiv2024

Internal waves in aquariums with characteristic corners

We give a precise microlocal description of the singular profile that forms in the long-time propagation of internal waves in an effectively two-dimensional aquarium. We allow domains with corners, such as polygons appearing in the experimental set ups of Maas et al. This extends the previous work of Dyatlov--Wang--Zworski arXiv:2112.10191, which considered domains with smooth boundary. We show that in addition to singularities that correspond to attractors in the underlying classical dynamics, milder singularities propagate out of the corners as well.

math.AP↗

arXiv2024

Test-Time Personalization with Meta Prompt for Gaze Estimation

Despite the recent remarkable achievement in gaze estimation, efficient and accurate personalization of gaze estimation without labels is a practical problem but rarely touched on in the literature. To achieve efficient personalization, we take inspiration from the recent advances in Natural Language Processing (NLP) by updating a negligible number of parameters, "prompts", at the test time. Specifically, the prompt is additionally attached without perturbing original network and can contain less than 1% of a ResNet-18's parameters. Our experiments show high efficiency of the prompt tuning approach. The proposed one can be 10 times faster in terms of adaptation speed than the methods compared. However, it is non-trivial to update the prompt for personalized gaze estimation without labels. At the test time, it is essential to ensure that the minimizing of particular unsupervised loss leads to the goals of minimizing gaze estimation error. To address this difficulty, we propose to meta-learn the prompt to ensure that its updates align with the goal. Our experiments show that the meta-learned prompt can be effectively adapted even with a simple symmetry loss. In addition, we experiment on four cross-dataset validations to show the remarkable advantages of the proposed method. Code is available at https://github.com/hmarkamcan/TPGaze.

cs.CV↗

arXiv2023

Absolutely continuous and BV-curves in 1-Wasserstein spaces

We extend the result of Lisini (Calc Var Partial Differ Equ 28:85-120, 2007) on the superposition principle for absolutely continuous curves in $p$-Wasserstein spaces to the special case of $p=1$. In contrast to the case of $p>1$, it is not always possible to have lifts on absolutely continuous curves. Therefore, one needs to relax the notion of a lift by considering curves of bounded variation, or shortly BV-curves, and replace the metric speed by the total variation measure. We prove that any BV-curve in a 1-Wasserstein space can be represented by a probability measure on the space of BV-curves which encodes the total variation measure of the Wasserstein curve. In particular, when the curve is absolutely continuous, the result gives a lift concentrated on BV-curves which also characterizes the metric speed. The main theorem is then applied for the characterization of geodesics and the study of the continuity equation in a discrete setting.

math.MG↗

arXiv2023

Are They All Good? Studying Practitioners' Expectations on the Readability of Log Messages

Developers write logging statements to generate logs that provide run-time information for various tasks. The readability of log messages in the logging statements (i.e., the descriptive text) is rather crucial to the value of the generated logs. Immature log messages may slow down or even obstruct the process of log analysis. Despite the importance of log messages, there is still a lack of standards on what constitutes good readability in log messages and how to write them. In this paper, we conduct a series of interviews with 17 industrial practitioners to investigate their expectations on the readability of log messages. Through the interviews, we derive three aspects related to the readability of log messages, including Structure, Information, and Wording, along with several specific practices to improve each aspect. We validate our findings through a series of online questionnaire surveys and receive positive feedback from the participants. We then manually investigate the readability of log messages in large-scale open source systems and find that a large portion (38.1%) of the log messages have inadequate readability. Motivated by such observation, we further explore the potential of automatically classifying the readability of log messages using deep learning and machine learning models. We find that both deep learning and machine learning models can effectively classify the readability of log messages with a balanced accuracy above 80.0% on average. Our study provides comprehensive guidelines for composing log messages to further improve practitioners' logging practices.

cs.SE↗

arXiv2023

Internal waves in 2D domains with ergodic classical dynamics

We study a model of internal waves in an effectively 2D aquarium under periodic forcing. In the case when the underlying classical dynamics has sufficiently irrational rotation number, we prove that the energy of the internal waves remains bounded. This involves studying the spectrum of a related 0-th order pseudodifferential operator at spectral parameters corresponding to such dynamics. For the special cases of rectangular and elliptic domains, we give an explicit spectral description of that operator.

math.AP↗

arXiv2023

Did We Miss Something Important? Studying and Exploring Variable-Aware Log Abstraction

Due to the sheer size of software logs, developers rely on automated techniques for log analysis. One of the first and most important steps of automated log analysis is log abstraction, which parses the raw logs into a structured format. Prior log abstraction techniques aim to identify and abstract all the dynamic variables in logs and output a static log template for automated log analysis. However, these abstracted dynamic variables may also contain important information that is useful to different tasks in log analysis. In this paper, we investigate the characteristics of dynamic variables and their importance in practice, and explore the potential of a variable-aware log abstraction technique. Through manual investigations and surveys with practitioners, we find that different categories of dynamic variables record various information that can be important depending on the given tasks, the distinction of dynamic variables in log abstraction can further assist in log analysis. We then propose a deep learning based log abstraction approach, named VALB, which can identify different categories of dynamic variables and preserve the value of specified categories of dynamic variables along with the log templates (i.e., variable-aware log abstraction). Through the evaluation on a widely used log abstraction benchmark, we find that VALB outperforms other state-of-the-art log abstraction techniques on general log abstraction (i.e., when abstracting all the dynamic variables) and also achieves a high variable-aware log abstraction accuracy that further identifies the category of the dynamic variables. Our study highlights the potential of leveraging the important information recorded in the dynamic variables to further improve the process of log analysis.

cs.SE↗

arXiv2023

The Globalization Theorem for CD(K,N) on locally finite Spaces

We establish the local-to-global property of the synthetic curvature-dimension condition for essentially non-branching locally finite metric-measure spaces, extending the work [F. Cavalletti, E. Milman \textit{Invent. Math.} 226 (2021), no. 1, 1-137].

math.MG↗