Searcharxiv⌕ Search

arXiv subjects

Wei Tao

Publications and source records attributed to Wei Tao.

At least 55 records · Page 3Linked to original sources
arXiv2024

UNet--: Memory-Efficient and Feature-Enhanced Network Architecture based on U-Net with Reduced Skip-Connections

U-Net models with encoder, decoder, and skip-connections components have demonstrated effectiveness in a variety of vision tasks. The skip-connections transmit fine-grained information from the encoder to the decoder. It is necessary to maintain the feature maps used by the skip-connections in memory before the decoding stage. Therefore, they are not friendly to devices with limited resource. In this paper, we propose a universal method and architecture to reduce the memory consumption and meanwhile generate enhanced feature maps to improve network performance. To this end, we design a simple but effective Multi-Scale Information Aggregation Module (MSIAM) in the encoder and an Information Enhancement Module (IEM) in the decoder. The MSIAM aggregates multi-scale feature maps into single-scale with less memory. After that, the aggregated feature maps can be expanded and enhanced to multi-scale feature maps by the IEM. By applying the proposed method on NAFNet, a SOTA model in the field of image restoration, we design a memory-efficient and feature-enhanced network architecture, UNet--. The memory demand by the skip-connections in the UNet-- is reduced by 93.3%, while the performance is improved compared to NAFNet. Furthermore, we show that our proposed method can be generalized to multiple visual tasks, with consistent improvements in both memory consumption and network accuracy compared to the existing efficient architectures.

cs.CV↗
arXiv2024

Data Interpreter: An LLM Agent For Data Science

Large Language Model (LLM)-based agents have shown effectiveness across many applications. However, their use in data science scenarios requiring solving long-term interconnected tasks, dynamic data adjustments and domain expertise remains challenging. Previous approaches primarily focus on individual tasks, making it difficult to assess the complete data science workflow. Moreover, they struggle to handle real-time changes in intermediate data and fail to adapt dynamically to evolving task dependencies inherent to data science problems. In this paper, we present Data Interpreter, an LLM-based agent designed to automatically solve various data science problems end-to-end. Our Data Interpreter incorporates two key modules: 1) Hierarchical Graph Modeling, which breaks down complex problems into manageable subproblems, enabling dynamic node generation and graph optimization; and 2) Programmable Node Generation, a technique that refines and verifies each subproblem to iteratively improve code generation results and robustness. Extensive experiments consistently demonstrate the superiority of Data Interpreter. On InfiAgent-DABench, it achieves a 25% performance boost, raising accuracy from 75.9% to 94.9%. For machine learning and open-ended tasks, it improves performance from 88% to 95%, and from 60% to 97%, respectively. Moreover, on the MATH dataset, Data Interpreter achieves remarkable performance with a 26% improvement compared to state-of-the-art baselines. The code is available at https://github.com/geekan/MetaGPT.

cs.AI↗
arXiv2024

MAGIS: LLM-Based Multi-Agent Framework for GitHub Issue Resolution

In software development, resolving the emergent issues within GitHub repositories is a complex challenge that involves not only the incorporation of new code but also the maintenance of existing code. Large Language Models (LLMs) have shown promise in code generation but face difficulties in resolving Github issues, particularly at the repository level. To overcome this challenge, we empirically study the reason why LLMs fail to resolve GitHub issues and analyze the major factors. Motivated by the empirical findings, we propose a novel LLM-based Multi-Agent framework for GitHub Issue reSolution, MAGIS, consisting of four agents customized for software evolution: Manager, Repository Custodian, Developer, and Quality Assurance Engineer agents. This framework leverages the collaboration of various agents in the planning and coding process to unlock the potential of LLMs to resolve GitHub issues. In experiments, we employ the SWE-bench benchmark to compare MAGIS with popular LLMs, including GPT-3.5, GPT-4, and Claude-2. MAGIS can resolve 13.94% GitHub issues, significantly outperforming the baselines. Specifically, MAGIS achieves an eight-fold increase in resolved ratio over the direct application of GPT-4, the advanced LLM.

cs.SE↗
arXiv2024

Provable Acceleration of Nesterov's Accelerated Gradient Method over Heavy Ball Method in Training Over-Parameterized Neural Networks

Due to its simplicity and efficiency, the first-order gradient method has been extensively employed in training neural networks. Although the optimization problem of the neural network is non-convex, recent research has proved that the first-order method is capable of attaining a global minimum during training over-parameterized neural networks, where the number of parameters is significantly larger than that of training instances. Momentum methods, including the heavy ball (HB) method and Nesterov's accelerated gradient (NAG) method, are the workhorse of first-order gradient methods owning to their accelerated convergence. In practice, NAG often exhibits superior performance than HB. However, current theoretical works fail to distinguish their convergence difference in training neural networks. To fill this gap, we consider the training problem of the two-layer ReLU neural network under over-parameterization and random initialization. Leveraging high-resolution dynamical systems and neural tangent kernel (NTK) theory, our result not only establishes tighter upper bounds of the convergence rate for both HB and NAG, but also provides the first theoretical guarantee for the acceleration of NAG over HB in training neural networks. Finally, we validate our theoretical results on three benchmark datasets.

cs.LG↗
arXiv2024

BotanicGarden: A High-Quality Dataset for Robot Navigation in Unstructured Natural Environments

The rapid developments of mobile robotics and autonomous navigation over the years are largely empowered by public datasets for testing and upgrading, such as sensor odometry and SLAM tasks. Impressive demos and benchmark scores have arisen, which may suggest the maturity of existing navigation techniques. However, these results are primarily based on moderate structured scenario testing. When transitioning to challenging unstructured environments, especially in GNSS-denied, texture-monotonous, and dense-vegetated natural fields, their performance can hardly sustain at a high level and requires further validation and improvement. To bridge this gap, we build a novel robot navigation dataset in a luxuriant botanic garden of more than 48000m2. Comprehensive sensors are used, including Gray and RGB stereo cameras, spinning and MEMS 3D LiDARs, and low-cost and industrial-grade IMUs, all of which are well calibrated and hardware-synchronized. An all-terrain wheeled robot is employed for data collection, traversing through thick woods, riversides, narrow trails, bridges, and grasslands, which are scarce in previous resources. This yields 33 short and long sequences, forming 17.1km trajectories in total. Excitedly, both highly-accurate ego-motions and 3D map ground truth are provided, along with fine-annotated vision semantics. We firmly believe that our dataset can advance robot navigation and sensor fusion research to a higher level.

cs.RO↗
arXiv2024

Value-Driven Mixed-Precision Quantization for Patch-Based Inference on Microcontrollers

Deploying neural networks on microcontroller units (MCUs) presents substantial challenges due to their constrained computation and memory resources. Previous researches have explored patch-based inference as a strategy to conserve memory without sacrificing model accuracy. However, this technique suffers from severe redundant computation overhead, leading to a substantial increase in execution latency. A feasible solution to address this issue is mixed-precision quantization, but it faces the challenges of accuracy degradation and a time-consuming search time. In this paper, we propose QuantMCU, a novel patch-based inference method that utilizes value-driven mixed-precision quantization to reduce redundant computation. We first utilize value-driven patch classification (VDPC) to maintain the model accuracy. VDPC classifies patches into two classes based on whether they contain outlier values. For patches containing outlier values, we apply 8-bit quantization to the feature maps on the dataflow branches that follow. In addition, for patches without outlier values, we utilize value-driven quantization search (VDQS) on the feature maps of their following dataflow branches to reduce search time. Specifically, VDQS introduces a novel quantization search metric that takes into account both computation and accuracy, and it employs entropy as an accuracy representation to avoid additional training. VDQS also adopts an iterative approach to determine the bitwidth of each feature map to further accelerate the search process. Experimental results on real-world MCU devices show that QuantMCU can reduce computation by 2.2x on average while maintaining comparable model accuracy compared to the state-of-the-art patch-based inference methods.

cs.CV↗
arXiv2024

KADEL: Knowledge-Aware Denoising Learning for Commit Message Generation

Commit messages are natural language descriptions of code changes, which are important for software evolution such as code understanding and maintenance. However, previous methods are trained on the entire dataset without considering the fact that a portion of commit messages adhere to good practice (i.e., good-practice commits), while the rest do not. On the basis of our empirical study, we discover that training on good-practice commits significantly contributes to the commit message generation. Motivated by this finding, we propose a novel knowledge-aware denoising learning method called KADEL. Considering that good-practice commits constitute only a small proportion of the dataset, we align the remaining training samples with these good-practice commits. To achieve this, we propose a model that learns the commit knowledge by training on good-practice commits. This knowledge model enables supplementing more information for training samples that do not conform to good practice. However, since the supplementary information may contain noise or prediction errors, we propose a dynamic denoising training method. This method composes a distribution-aware confidence function and a dynamic distribution list, which enhances the effectiveness of the training process. Experimental results on the whole MCMD dataset demonstrate that our method overall achieves state-of-the-art performance compared with previous methods. Our source code and data are available at https://github.com/DeepSoftwareAnalytics/KADEL

cs.SE↗
arXiv2023

Discovery of a Single-Band Mott Insulator in a van der Waals Flat-Band Compound

The Mott insulator provides an excellent foundation for exploring a wide range of strongly correlated physical phenomena, such as high-temperature superconductivity, quantum spin liquid, and colossal magnetoresistance. A Mott insulator with the simplest degree of freedom is an ideal and highly desirable system for studying the fundamental physics of Mottness. In this study, we have unambiguously identified such an anticipated Mott insulator in a van der Waals layered compound Nb3Cl8. In the high-temperature phase, where interlayer coupling is negligible, density functional theory calculations for the monolayer of Nb3Cl8 suggest a half-filled flat band at the Fermi level, whereas angle-resolved photoemission spectroscopy experiments observe a large gap. This observation is perfectly reproduced by dynamical mean-field theory calculations considering strong electron correlations, indicating a correlation-driven Mott insulator state. Since this half-filled band derived from a single 2a1 orbital is isolated from all other bands, the monolayer of Nb3Cl8 is an ideal realization of the celebrated single-band Hubbard model. Upon decreasing the temperature, the bulk system undergoes a phase transition, where structural changes significantly enhance the interlayer coupling. This results in a bonding-antibonding splitting in the Hubbard bands, while the Mott gap remains dominant. Our discovery provides a simple and seminal model system for investigating Mott physics and other emerging correlated states.

cond-mat.str-el↗
arXiv2023

Decay constants of $c \bar b$ mesons involving the ten heavy flavor-changing currents at N$^3$LO QCD

Within the nonrelativistic QCD (NRQCD) framework, we complete the three-loop calculations of the NRQCD renormalization constants and the matching coefficients, for the heavy flavor-changing temporal vector, spatial-spatial tensor, spatial-temporal axial-tensor currents, which are coupled to the $P$-wave $c\bar b$ mesons. We further study the ten decay constants for the $c\bar b$ mesons ($B_c,B_c^*,B_{c0}^*,B_{c1}$) coupled with the ten heavy flavor-changing currents involving (pseudo-)scalar, (axial-)vector and (axial-)tensor up to the next-to-next-to-next-to-leading order (N$^3$LO) of $α_s$. We obtain the six ratios of decay constants by approximating them to the corresponding ratios of matching coefficients. We find the N$^3$LO QCD corrections to the six ratios of decay constants have good convergence and weak scale-dependence. We finally predict the hierarchical relationship among the ten decay constants for the $c\bar b$ mesons coupled with the ten currents.

hep-ph↗
arXiv2023

Convergent decay constants for beauty-charm mesons at N$^3$LO in QCD and phenomenological implications

We show the three-loop calculation of the matching coefficients $\mathcal{C}_J$ for the four heavy flavor-changing currents in nonrelativistic Quantum Chromodynamics effective theory and confirm the surprising observation of the nonconvergence behaviors of the matching coefficients for beauty-charm mesons. We demonstrate that the higher-order correction to the wave function at origin can solve the nonconvergence problem. By employing the scaling relation, we simultaneously obtain the decay constants for beauty-charm mesons up to the three-loop accuracy and the hyperfine mass splitting in agreement with the experimental data. The perturbative predictions for the leptonic decays of the beauty-charm mesons shall provide a guide in ongoing and future precision heavy flavor experiments.

hep-ph↗
arXiv2023

Three-loop matching of heavy flavor-changing (axial-)tensor currents

We present the three-loop calculations of the nonrelativistic QCD (NRQCD) current renormalization constants and corresponding anomalous dimensions, and the matching coefficients for the spatial-temporal tensor and spatial-spatial axial-tensor currents with two different heavy quark masses. We obtain the convergent decay constant ratio up to the next-to-next-to-next-to-leading order (N$^3$LO) for the $S$-wave vector meson $B_c^*$ involving the tensor and axial-tensor currents. We obtain the three-loop finite ($ε^0$) term in the ratio of the QCD heavy flavor-changing tensor current renormalization constant in the on-shell ($\mathrm{OS}$) scheme to that in the modified-minimal-subtraction ($\mathrm{\overline{MS}}$) scheme, which is helpful to obtain the three-loop matching coefficients for all heavy flavor-changing (axial-)tensor currents.

hep-ph↗
arXiv2023

Giant photon-drag-induced ultrafast photocurrent in diamond for nonlinear photonics

Diamond is emerging as an attractive third-generation wide-bandgap semiconductor for future on-chip nonlinear photonics and quantum optics due to its unique thermal, optical, and mechanical properties. However, the light-driven current under below-bandgap excitation from the second-order nonlinear optical effect in diamond is still challenging. Herein, a giant second-order nonlinear photocurrent is observed in the chemical vapor deposition (CVD) diamond by utilizing terahertz (THz) emission spectroscopy. This ultrafast photocurrent originates from the photon drag effect (PDE), during which the momentum transfer from the incident photons to the charge carriers at the rich grain boundaries of the CVD diamond after the exclusive subgap π-π* transition upon femtosecond laser excitation. Especially, the interplay between circular and linear PDE to the THz generation has been clarified and distinguished under elliptically polarized light excitation. Furthermore, the picosecond ultrafast dynamics of these charge carriers are also verified by the infrared spectroscopy. Owing to the giant photon-drag-induced ultrafast photocurrent, the CVD diamond presents the highest THz emission efficiency compared with the reported carbon allotropes, which expands the new functionality of diamond nonlinear photonics into on-chip THz devices.

physics.optics↗
arXiv2023

Performance benchmarking of an ultra-low vibration laboratory to host a commercial millikelvin scanning tunnelling microscope

Ultra-low temperature scanning tunnelling microscopy and spectroscopy (STM/STS) achieved by dilution refrigeration can provide unrivalled insight into the local electronic structure of quantum materials and atomic-scale quantum systems. Effective isolation from mechanical vibration and acoustic noise is critical in order to achieve ultimate spatial and energy resolution. Here, we report on the design and performance of an ultra-low vibration (ULV) laboratory hosting a customized but otherwise commercially available 40mK STM. The design of the vibration isolation consists of a T-shaped concrete mass block (55t), suspended by actively controlled pneumatic springs, and placed on a foundation separated from the surrounding building in a "room-within-a-room" design. Vibration levels achieved are meeting the VC-M vibration standard at >3 Hz, reached only in a limited number of laboratories worldwide. Measurement of the STM's junction noise confirms effective vibration isolation on par with custom built STMs in ULV laboratories. In this tailored low-vibration environment, the STM achieves an energy resolution of 43ueV (144 mK), promising for the investigation and control of quantum matter at atomic length scales.

cond-mat.supr-con↗
arXiv2023

Three-loop matching coefficients for heavy flavor-changing currents and the phenomenological applications

Within the framework of non-relativistic QCD (NRQCD) factorization, we compute the matching coefficients between full Quantum Chromodynamics (QCD) and NRQCD for the heavy flavor-changing vector, axial-vector, scalar and pseudo-scalar currents up to next-to-next-to-next-to-leading order (N$^3$LO). We accomplish the analytical expressions for the three-loop renormalization constants and the corresponding anomalous dimensions for all of the four NRQCD currents with two different heavy flavors. The three-loop QCD corrections to the matching coefficients turn out to be significantly larger than lower order corrections. By employing the scale relation, we obtain the N$^3$LO corrections to the wave functions at the origin for the vector $B_c^*$ meson and the pseudo-scalar $B_c$ meson from the known result for the equal-mass heavy quarkonium in potential NRQCD (pNRQCD). We find large cancellations at the third order between the matching coefficients and the wave functions at the origin, and obtain the convergent decay constants of $B^*_{c}$ and $B_{c}$ up to N$^3$LO. We present the complete perturbative QCD predictions for the decay constants, leptonic decay widths, and branching ratios of the beauty-charmed mesons.

hep-ph↗
arXiv2023

Adapting Step-size: A Unified Perspective to Analyze and Improve Gradient-based Methods for Adversarial Attacks

Learning adversarial examples can be formulated as an optimization problem of maximizing the loss function with some box-constraints. However, for solving this induced optimization problem, the state-of-the-art gradient-based methods such as FGSM, I-FGSM and MI-FGSM look different from their original methods especially in updating the direction, which makes it difficult to understand them and then leaves some theoretical issues to be addressed in viewpoint of optimization. In this paper, from the perspective of adapting step-size, we provide a unified theoretical interpretation of these gradient-based adversarial learning methods. We show that each of these algorithms is in fact a specific reformulation of their original gradient methods but using the step-size rules with only current gradient information. Motivated by such analysis, we present a broad class of adaptive gradient-based algorithms based on the regular gradient methods, in which the step-size strategy utilizing information of the accumulated gradients is integrated. Such adaptive step-size strategies directly normalize the scale of the gradients rather than use some empirical operations. The important benefit is that convergence for the iterative algorithms is guaranteed and then the whole optimization process can be stabilized. The experiments demonstrate that our AdaI-FGM consistently outperforms I-FGSM and AdaMI-FGM remains competitive with MI-FGSM for black-box attacks.

cs.LG↗
arXiv2023

Three-loop QCD matching of the flavor-changing scalar current involving the heavy charm and bottom quark

We compute the matching coefficient between the quantum chromodynamics (QCD) and the non-relativistic QCD ( NRQCD) for the flavor-changing scalar current involving the heavy charm and bottom quark, up to the three-loop order within the NRQCD factorization. For the first time, we obtain the analytical expressions for the three-loop renormalization constant $\tilde{Z}_s(x,R_f)$ and the corresponding anomalous dimension $\tildeγ_s(x,R_f)$ for the NRQCD scalar current with the two heavy bottom and charm quark. We present the precise numerical results for those relevant coefficients $(C_{FF}(x_0), \cdots, C_{FBB}(x_0))$ with an accuracy of about thirty digits. The three-loop QCD correction turns out to be significantly large. The obtained matching coefficient $C_s(μ_f,μ,m_b,m_c)$ is helpful to analyze the threshold behaviours when two different heavy quarks are close to each other and form the double heavy $B_c$ mesons.

hep-ph↗
arXiv2022

RACE: Retrieval-Augmented Commit Message Generation

Commit messages are important for software development and maintenance. Many neural network-based approaches have been proposed and shown promising results on automatic commit message generation. However, the generated commit messages could be repetitive or redundant. In this paper, we propose RACE, a new retrieval-augmented neural commit message generation method, which treats the retrieved similar commit as an exemplar and leverages it to generate an accurate commit message. As the retrieved commit message may not always accurately describe the content/intent of the current code diff, we also propose an exemplar guider, which learns the semantic similarity between the retrieved and current code diff and then guides the generation of commit message based on the similarity. We conduct extensive experiments on a large public dataset with five programming languages. Experimental results show that RACE can outperform all baselines. Furthermore, RACE can boost the performance of existing Seq2Seq models in commit message generation.

cs.SE↗
arXiv2022

Next-to-next-to-leading order matching of beauty-charmed meson $B_{c}$ and $B^*_{c}$ decay constants

We present the next-to-next-to-leading order (NNLO) QCD corrections to the decay constants for both the pseudoscalar and vector beauty-charmed mesons $B_{c}$ and $B^*_{c}$ in nonrelativistic QCD effective theory. Explicit NNLO calculation verified that the $B_c$ decay constant from pseudoscalar current is identical with the $B_c$ decay constant from axial-vector current. The NNLO result for the vector decay constant of $B^*_{c}$ meson is novel. Combined with the latest extraction of nonrelativistic QCD long-distance matrix elements of $B_c$ meson, we give the branching ratios of leptonic decays of $B_{c}$ and $B^*_{c}$ mesons. In addition, the novel anomalous dimension for the flavor-changing heavy quark vector current in nonrelativistic QCD effective theory are helpful to investigate the threshold behaviours of two different heavy quarks.

hep-ph↗