arXiv subjects

Lei Chen

Publications and source records attributed to Lei Chen.

At least 307 records · Page 17Linked to original sources

arXiv2024

Understanding Code Change with Micro-Changes

A crucial activity in software maintenance and evolution is the comprehension of the changes performed by developers, when they submit a pull request and/or perform a commit on the repository. Typically, code changes are represented in the form of code diffs, textual representations highlighting the differences between two file versions, depicting the added, removed, and changed lines. This simplistic representation must be interpreted by developers, and mentally lifted to a higher abstraction level, that more closely resembles natural language descriptions, and eases the creation of a mental model of the changes. However, the textual diff-based representation is cumbersome, and the lifting requires considerable domain knowledge and programming skills. We present an approach, based on the concept of micro-change, to overcome these difficulties, translating code diffs into a series of pre-defined change operations, which can be described in natural language. We present a catalog of micro-changes, together with an automated micro-change detector. To evaluate our approach, we performed an empirical study on a large set of open-source repositories, focusing on a subset of our micro-change catalog, namely those related to changes affecting the conditional logic. We found that our detector is capable of explaining more than 67% of the changes taking place in the systems under study.

cs.SE↗

arXiv2024

FuXi-2.0: Advancing machine learning weather forecasting model for practical applications

Machine learning (ML) models have become increasingly valuable in weather forecasting, providing forecasts that not only lower computational costs but often match or exceed the accuracy of traditional numerical weather prediction (NWP) models. Despite their potential, ML models typically suffer from limitations such as coarse temporal resolution, typically 6 hours, and a limited set of meteorological variables, limiting their practical applicability. To overcome these challenges, we introduce FuXi-2.0, an advanced ML model that delivers 1-hourly global weather forecasts and includes a comprehensive set of essential meteorological variables, thereby expanding its utility across various sectors like wind and solar energy, aviation, and marine shipping. Our study conducts comparative analyses between ML-based 1-hourly forecasts and those from the high-resolution forecast (HRES) of the European Centre for Medium-Range Weather Forecasts (ECMWF) for various practical scenarios. The results demonstrate that FuXi-2.0 consistently outperforms ECMWF HRES in forecasting key meteorological variables relevant to these sectors. In particular, FuXi-2.0 shows superior performance in wind power forecasting compared to ECMWF HRES, further validating its efficacy as a reliable tool for scenarios demanding precise weather forecasts. Additionally, FuXi-2.0 also integrates both atmospheric and oceanic components, representing a significant step forward in the development of coupled atmospheric-ocean models. Further comparative analyses reveal that FuXi-2.0 provides more accurate forecasts of tropical cyclone intensity than its predecessor, FuXi-1.0, suggesting that there are benefits of an atmosphere-ocean coupled model over atmosphere-only models.

physics.ao-ph↗

arXiv2024

Up-sampling-only and Adaptive Mesh-based GNN for Simulating Physical Systems

Traditional simulation of complex mechanical systems relies on numerical solvers of Partial Differential Equations (PDEs), e.g., using the Finite Element Method (FEM). The FEM solvers frequently suffer from intensive computation cost and high running time. Recent graph neural network (GNN)-based simulation models can improve running time meanwhile with acceptable accuracy. Unfortunately, they are hard to tailor GNNs for complex mechanical systems, including such disadvantages as ineffective representation and inefficient message propagation (MP). To tackle these issues, in this paper, with the proposed Up-sampling-only and Adaptive MP techniques, we develop a novel hierarchical Mesh Graph Network, namely UA-MGN, for efficient and effective mechanical simulation. Evaluation on two synthetic and one real datasets demonstrates the superiority of the UA-MGN. For example, on the Beam dataset, compared to the state-of-the-art MS-MGN, UA-MGN leads to 40.99% lower errors but using only 43.48% fewer network parameters and 4.49% fewer floating point operations (FLOPs).

cs.LG↗

arXiv2024

RTLRewriter: Methodologies for Large Models aided RTL Code Optimization

Register Transfer Level (RTL) code optimization is crucial for enhancing the efficiency and performance of digital circuits during early synthesis stages. Currently, optimization relies heavily on manual efforts by skilled engineers, often requiring multiple iterations based on synthesis feedback. In contrast, existing compiler-based methods fall short in addressing complex designs. This paper introduces RTLRewriter, an innovative framework that leverages large models to optimize RTL code. A circuit partition pipeline is utilized for fast synthesis and efficient rewriting. A multi-modal program analysis is proposed to incorporate vital visual diagram information as optimization cues. A specialized search engine is designed to identify useful optimization guides, algorithms, and code snippets that enhance the model ability to generate optimized RTL. Additionally, we introduce a Cost-aware Monte Carlo Tree Search (C-MCTS) algorithm for efficient rewriting, managing diverse retrieved contents and steering the rewriting results. Furthermore, a fast verification pipeline is proposed to reduce verification cost. To cater to the needs of both industry and academia, we propose two benchmarking suites: the Large Rewriter Benchmark, targeting complex scenarios with extensive circuit partitioning, optimization trade-offs, and verification challenges, and the Small Rewriter Benchmark, designed for a wider range of scenarios and patterns. Our comparative analysis with established compilers such as Yosys and E-graph demonstrates significant improvements, highlighting the benefits of integrating large models into the early stages of circuit design. We provide our benchmarks at https://github.com/yaoxufeng/RTLRewriter-Bench.

cs.AR↗

arXiv2024

Electronic properties, correlated topology and Green's function zeros

There is extensive current interest about electronic topology in correlated settings. In strongly correlated systems, contours of Green's function zeros may develop in frequency-momentum space, and their role in correlated topology has increasingly been recognized. However, whether and how the zeros contribute to electronic properties is a matter of uncertainty. Here we address the issue in an exactly solvable model for Mott insulator. We show that the Green's function zeros contribute to several physically measurable correlation functions, in a way that does not run into inconsistencies. In particular, the physical properties remain robust to chemical potential variations up to the Mott gap as it should be based on general considerations. Our work sets the stage for further understandings on the rich interplay among topology, symmetry and strong correlations.

cond-mat.str-el↗

arXiv2024

Vertex-primitive s-arc-transitive digraphs of symplectic groups

A digraph is $s$-arc-transitive if its automorphism group is transitive on directed paths with $s$ edges, that is, on $s$-arcs. Although infinite families of finite $s$-arc transitive digraphs of arbitrary valency were constructed by the third author in 1989, existence of a vertex-primitive $2$-arc-transitive digraph was not known until an infinite family was constructed by the second author with Li and Xia in 2017. This led to a conjecture by the second author and Xia in 2018 that, for a finite vertex-primitive $s$-arc-transitive digraph, $s$ is at most $2$, together with their proof that it is sufficient to prove the conjecture for digraphs with an almost simple group of automorphisms. This paper confirms the conjecture for finite symplectic groups.

math.CO↗

arXiv2024

Programmable scanning diffuse speckle contrast imaging of cerebral blood flow

Significance: Cerebral blood flow (CBF) imaging is crucial for diagnosing cerebrovascular diseases. However, existing large neuroimaging techniques with high cost, low sampling rate, and poor mobility make them unsuitable for continuous and longitudinal CBF monitoring at the bedside. Aim: This study aimed to develop a low-cost, portable, programmable scanning diffuse speckle contrast imaging (PS-DSCI) technology for fast, high-density, and depth-sensitive imaging of CBF in rodents. Approach: The PS-DSCI employed a programmable digital micromirror device (DMD) for remote line-shape laser (785 nm) scanning on tissue surface and synchronized a 2D camera for capturing boundary diffuse laser speckle contrasts. New algorithms were developed to address deformations of line-shape scanning, thus minimizing CBF reconstruction artifacts. The PS-DSCI was examined in head-simulating phantoms and adult mice. Results: The PS-DSCI enables resolving Intralipid particle flow contrasts at different tissue depths. In vivo experiments in adult mice demonstrated the capability of PS-DSCI to image global/regional CBF variations induced by 8% CO2 inhalation and transient carotid artery ligations. Conclusions: Compared to conventional point scanning, the line scanning in PS-DSCI significantly increases spatiotemporal resolution. The high sampling rate of PS-DSCI is crucial for capturing rapid CBF changes while high spatial resolution is important for visualizing brain vasculature.

q-bio.TO↗

arXiv2024

Class-aware and Augmentation-free Contrastive Learning from Label Proportion

Learning from Label Proportion (LLP) is a weakly supervised learning scenario in which training data is organized into predefined bags of instances, disclosing only the class label proportions per bag. This paradigm is essential for user modeling and personalization, where user privacy is paramount, offering insights into user preferences without revealing individual data. LLP faces a unique difficulty: the misalignment between bag-level supervision and the objective of instance-level prediction, primarily due to the inherent ambiguity in label proportion matching. Previous studies have demonstrated deep representation learning can generate auxiliary signals to promote the supervision level in the image domain. However, applying these techniques to tabular data presents significant challenges: 1) they rely heavily on label-invariant augmentation to establish multi-view, which is not feasible with the heterogeneous nature of tabular datasets, and 2) tabular datasets often lack sufficient semantics for perfect class distinction, making them prone to suboptimality caused by the inherent ambiguity of label proportion matching. To address these challenges, we propose an augmentation-free contrastive framework TabLLP-BDC that introduces class-aware supervision (explicitly aware of class differences) at the instance level. Our solution features a two-stage Bag Difference Contrastive (BDC) learning mechanism that establishes robust class-aware instance-level supervision by disassembling the nuance between bag label proportions, without relying on augmentations. Concurrently, our model presents a pioneering multi-task pretraining pipeline tailored for tabular-based LLP, capturing intrinsic tabular feature correlations in alignment with label proportion distribution. Extensive experiments demonstrate that TabLLP-BDC achieves state-of-the-art performance for LLP in the tabular domain.

cs.LG↗

arXiv2024

FuXi-ENS: A machine learning model for medium-range ensemble weather forecasting

Ensemble forecasting is crucial for improving weather predictions, especially for forecasts of extreme events. Constructing an ensemble prediction system (EPS) based on conventional NWP models is highly computationally expensive. ML models have emerged as valuable tools for deterministic weather forecasts, providing forecasts with significantly reduced computational requirements and even surpassing the forecast performance of traditional NWP models. However, challenges arise when applying ML models to ensemble forecasting. Recent ML models, such as GenCast and SEEDS model, rely on the ERA5 EDA or operational NWP ensemble members for forecast generation. Their spatial resolution is also considered too coarse for many applications. To overcome these limitations, we introduce FuXi-ENS, an advanced ML model designed to deliver 6-hourly global ensemble weather forecasts up to 15 days. This model runs at a significantly increased spatial resolution of 0.25\textdegree, incorporating 5 atmospheric variables at 13 pressure levels, along with 13 surface variables. By leveraging the inherent probabilistic nature of Variational AutoEncoder (VAE), FuXi-ENS optimizes a loss function that combines the CRPS and the KL divergence between the predicted and target distribution, facilitating the incorporation of flow-dependent perturbations in both initial conditions and forecast. This innovative approach makes FuXi-ENS an advancement over the traditional ones that use L1 loss combined with the KL loss in standard VAE models for ensemble weather forecasting. Results demonstrate that FuXi-ENS outperforms ensemble forecasts from the ECMWF, a world leading NWP model, in the CRPS of 98.1% of 360 variable and forecast lead time combinations. This achievement underscores the potential of the FuXi-ENS model to enhance ensemble weather forecasts, offering a promising direction for further development in this field.

cs.LG↗

arXiv2024

Cascading failures with group support in interdependent hypergraphs

The functionality of an entity frequently necessitates the support of a group situated in another layer of the system. To unravel the profound impact of such group support on a system's resilience against cascading failures, we devise a framework comprising a double-layer interdependent hypergraph system, wherein nodes are capable of receiving support via hyperedges. Our central hypothesis posits that the failure may transcend to another layer when all support groups of each dependent node fail, thereby initiating a potentially iterative cascade across layers. Through rigorous analytical methods, we derive the critical threshold for the initial node survival probability that marks the second-order phase transition point. A salient discovery is that as the prevalence of dependent nodes escalates, the system dynamics shift from a second-order to a first-order phase transition. Notably, irrespective of the collapse pattern, systems characterized by scale-free hyperdegree distributions within both hypergraph layers consistently demonstrate superior robustness compared to those adhering to Poisson hyperdegree distributions. In summary, our research underscores the paramount significance of group support mechanisms and intricate network topologies in determining the resilience of interconnected systems against the propagation of cascading failures. By exploring the interplay between these factors, we have gained insights into how systems can be designed or optimized to mitigate the risk of widespread disruptions, ensuring their continued functionality and stability in the face of adverse events.

physics.soc-ph↗

arXiv2024

AHMF: Adaptive Hybrid-Memory-Fusion Model for Driver Attention Prediction

Accurate driver attention prediction can serve as a critical reference for intelligent vehicles in understanding traffic scenes and making informed driving decisions. Though existing studies on driver attention prediction improved performance by incorporating advanced saliency detection techniques, they overlooked the opportunity to achieve human-inspired prediction by analyzing driving tasks from a cognitive science perspective. During driving, drivers' working memory and long-term memory play crucial roles in scene comprehension and experience retrieval, respectively. Together, they form situational awareness, facilitating drivers to quickly understand the current traffic situation and make optimal decisions based on past driving experiences. To explicitly integrate these two types of memory, this paper proposes an Adaptive Hybrid-Memory-Fusion (AHMF) driver attention prediction model to achieve more human-like predictions. Specifically, the model first encodes information about specific hazardous stimuli in the current scene to form working memories. Then, it adaptively retrieves similar situational experiences from the long-term memory for final prediction. Utilizing domain adaptation techniques, the model performs parallel training across multiple datasets, thereby enriching the accumulated driving experience within the long-term memory module. Compared to existing models, our model demonstrates significant improvements across various metrics on multiple public datasets, proving the effectiveness of integrating hybrid memories in driver attention prediction.

cs.CV↗

arXiv2024

Dissecting Multiplication in Transformers: Insights into LLMs

Transformer-based large language models have achieved remarkable performance across various natural language processing tasks. However, they often struggle with seemingly easy tasks like arithmetic despite their vast capabilities. This stark disparity raise human's concerns about their safe and ethical use, hinder their widespread adoption.In this paper, we focus on a typical arithmetic task, integer multiplication, to explore and explain the imperfection of transformers in this domain. We provide comprehensive analysis of a vanilla transformer trained to perform n-digit integer multiplication. Our observations indicate that the model decomposes multiplication task into multiple parallel subtasks, sequentially optimizing each subtask for each digit to complete the final multiplication. Based on observation and analysis, we infer the reasons of transformers deficiencies in multiplication tasks lies in their difficulty in calculating successive carryovers and caching intermediate results, and confirmed this inference through experiments. Guided by these findings, we propose improvements to enhance transformers performance on multiplication tasks. These enhancements are validated through rigorous testing and mathematical modeling, not only enhance transformer's interpretability, but also improve its performance, e.g., we achieve over 99.9% accuracy on 5-digit integer multiplication with a tiny transformer, outperform LLMs GPT-4. Our method contributes to the broader fields of model understanding and interpretability, paving the way for analyzing more complex tasks and Transformer models. This work underscores the importance of explainable AI, helping to build trust in large language models and promoting their adoption in critical applications.

cs.CL↗

arXiv2024

Chern Bands' Optimally Localized Wannier Functions and Fractional Chern Insulators

Recent development on fractional Chern insulators and proximate phases call for a real space representation of isolated Chern bands. Here we propose a new method for a general construction of optimally localized Wannier functions from such Chern bands. We do so through an optimal gauge choice of the Bloch states of a Chern band with the singularity placed at any desired position in momentum space. We apply this method to construct the optimally localized Wannier functions for kagome lattice, and use it to identify channels of interactions that are favorable to the development of fractional Chern insulators. Implications of the approach for the interplay between correlations and topology in broader contexts are discussed.

cond-mat.mes-hall↗

arXiv2024

Dynamic protected states in the non-Hermitian system

The non-Hermitian skin effect and nonreciprocal behavior are sensitive to the boundary conditions, which are unique features of non-Hermitian systems. The eigenenergies will become complex and all eigenstates are localized at the boundary, which is distinguished from the Hermitian topologies. In this work, we theoretically study the dynamic behavior of the propagation of Gaussian wavepackets inside a non-Hermitian lattice and analyze the self-acceleration process of bulk state or Gaussian wavepackets toward the system's boundary. The initial wavepackets will not only propagate toward the side where the eigenstates are localized, but also their momentum will approach to a specific value where the imaginary parts of energy dispersion are the maximum. In addition, if the wavepackets cover this specific momentum, they will eventually exhibit exponentially increasing amplitudes with time evolution, maintaining the dynamic protected condition for an extended period of time until they approach the boundary. We also take two widely used toy models as examples in one and two dimensions to verify the correspondence of the non-Hermitian skin effect and the dynamic protected state.

cond-mat.mes-hall↗

arXiv2024

Directly Estimating Mixed-State Entanglement with Bell Measurement Assistance

Entanglement plays a fundamental role in quantum physics and information processing. Here, we develop an unbiased estimator for mixed-state entanglement in the few-shot scenario and directly estimate it using random unitary evolution in a photonic system. As a supplement to traditional projective measurements, we incorporate Bell measurements on qubit-pairs, enriching the previous randomized measurement scheme, which is no-go in this task with only local unitary evolution. The scheme is scalable to n-qubits via Bell measurements on qubit-pairs. The estimator can be derived directly from a few consecutive outcomes while exhibiting greater robustness to system errors and noise compared to schemes based on shadow estimation. We find that, under a fixed measurement resource, the way with more versatile measurement settings with fewer repeats per setting is more efficient. Our protocol and demonstration advance the direct characterization of quantum states in practice.

quant-ph↗

arXiv2024

FuXi-S2S: A machine learning model that outperforms conventional global subseasonal forecast models

Skillful subseasonal forecasts are crucial for various sectors of society but pose a grand scientific challenge. Recently, machine learning based weather forecasting models outperform the most successful numerical weather predictions generated by the European Centre for Medium-Range Weather Forecasts (ECMWF), but have not yet surpassed conventional models at subseasonal timescales. This paper introduces FuXi Subseasonal-to-Seasonal (FuXi-S2S), a machine learning model that provides global daily mean forecasts up to 42 days, encompassing five upper-air atmospheric variables at 13 pressure levels and 11 surface variables. FuXi-S2S, trained on 72 years of daily statistics from ECMWF ERA5 reanalysis data, outperforms the ECMWF's state-of-the-art Subseasonal-to-Seasonal model in ensemble mean and ensemble forecasts for total precipitation and outgoing longwave radiation, notably enhancing global precipitation forecast. The improved performance of FuXi-S2S can be primarily attributed to its superior capability to capture forecast uncertainty and accurately predict the Madden-Julian Oscillation (MJO), extending the skillful MJO prediction from 30 days to 36 days. Moreover, FuXi-S2S not only captures realistic teleconnections associated with the MJO, but also emerges as a valuable tool for discovering precursor signals, offering researchers insights and potentially establishing a new paradigm in Earth system science research.

physics.ao-ph↗

arXiv2024

MindBench: A Comprehensive Benchmark for Mind Map Structure Recognition and Analysis

Multimodal Large Language Models (MLLM) have made significant progress in the field of document analysis. Despite this, existing benchmarks typically focus only on extracting text and simple layout information, neglecting the complex interactions between elements in structured documents such as mind maps and flowcharts. To address this issue, we introduce the new benchmark named MindBench, which not only includes meticulously constructed bilingual authentic or synthetic images, detailed annotations, evaluation metrics and baseline models, but also specifically designs five types of structured understanding and parsing tasks. These tasks include full parsing, partial parsing, position-related parsing, structured Visual Question Answering (VQA), and position-related VQA, covering key areas such as text recognition, spatial awareness, relationship discernment, and structured parsing. Extensive experimental results demonstrate the substantial potential and significant room for improvement in current models' ability to handle structured document information. We anticipate that the launch of MindBench will significantly advance research and application development in structured document analysis technology. MindBench is available at: https://miasanlei.github.io/MindBench.github.io/.

cs.CV↗

arXiv2024

A Small and Fast BERT for Chinese Medical Punctuation Restoration

In clinical dictation, utterances after automatic speech recognition (ASR) without explicit punctuation marks may lead to the misunderstanding of dictated reports. To give a precise and understandable clinical report with ASR, automatic punctuation restoration is required. Considering a practical scenario, we propose a fast and light pre-trained model for Chinese medical punctuation restoration based on 'pretraining and fine-tuning' paradigm. In this work, we distill pre-trained models by incorporating supervised contrastive learning and a novel auxiliary pre-training task (Punctuation Mark Prediction) to make it well-suited for punctuation restoration. Our experiments on various distilled models reveal that our model can achieve 95% performance while 10% model size relative to state-of-the-art Chinese RoBERTa.

cs.CL↗