arXiv subjects

Junwei Hu

Publications and source records attributed to Junwei Hu.

11 recordsLinked to original sources

arXiv2025

Desensitizing for Improving Corruption Robustness in Point Cloud Classification through Adversarial Training

Due to scene complexity, sensor inaccuracies, and processing imprecision, point cloud corruption is inevitable. Over-reliance on input features is the root cause of DNN vulnerabilities. It remains unclear whether this issue exists in 3D tasks involving point clouds and whether reducing dependence on these features can enhance the model's robustness to corrupted point clouds. This study attempts to answer these questions. Specifically, we quantified the sensitivity of the DNN to point cloud features using Shapley values and found that models trained using traditional methods exhibited high sensitivity values for certain features. Furthermore, under an equal pruning ratio, prioritizing the pruning of highly sensitive features causes more severe damage to model performance than random pruning. We propose `Desensitized Adversarial Training' (DesenAT), generating adversarial samples using feature desensitization and conducting training within a self-distillation framework, which aims to alleviate DNN's over-reliance on point clouds features by smoothing sensitivity. First, data points with high contribution components are eliminated, and spatial transformation is used to simulate corruption scenes, generate adversarial samples, and conduct adversarial training on the model. Next, to compensate for information loss in adversarial samples, we use the self-distillation method to transfer knowledge from clean samples to adversarial samples, and perform adversarial training in a distillation manner.Extensive experiments on ModelNet-C and PointCloud-C demonstrate show that the propose method can effectively improve the robustness of the model without reducing the performance of clean data sets. This code is publicly available at \href{https://github.com/JerkyT/DesenAT/tree/master}{https://github.com/JerkyT/DesenAT}.

cs.CV↗

arXiv2025

Joint graph entropy knowledge distillation for point cloud classification and robustness against corruptions

Classification tasks in 3D point clouds often assume that class events \replaced{are }{follow }independent and identically distributed (IID), although this assumption destroys the correlation between classes. This \replaced{study }{paper }proposes a classification strategy, \textbf{J}oint \textbf{G}raph \textbf{E}ntropy \textbf{K}nowledge \textbf{D}istillation (JGEKD), suitable for non-independent and identically distributed 3D point cloud data, \replaced{which }{the strategy } achieves knowledge transfer of class correlations through knowledge distillation by constructing a loss function based on joint graph entropy. First\deleted{ly}, we employ joint graphs to capture add{the }hidden relationships between classes\replaced{ and}{,} implement knowledge distillation to train our model by calculating the entropy of add{add }graph.\replaced{ Subsequently}{ Then}, to handle 3D point clouds \deleted{that is }invariant to spatial transformations, we construct \replaced{S}{s}iamese structures and develop two frameworks, self-knowledge distillation and teacher-knowledge distillation, to facilitate information transfer between different transformation forms of the same data. \replaced{In addition}{ Additionally}, we use the above framework to achieve knowledge transfer between point clouds and their corrupted forms, and increase the robustness against corruption of model. Extensive experiments on ScanObject, ModelNet40, ScanntV2\_cls and ModelNet-C demonstrate that the proposed strategy can achieve competitive results.

cs.CV↗

arXiv2025

RAE: A Rule-Driven Approach for Attribute Embedding in Property Graph Recommendation

Recommendation systems are crucial in modern applications to enhance the user experience and drive business conversion rates through personalization. However, insufficient utilization of attribute information within the property graph remains a significant challenge. Most existing graph convolutional network (GCN) models do not consider attribute information, and those that do often employ a simplified triple format , which fails to fully exploit the rich semantic structures of property graphs necessary for effective recommendations. To overcome these limitations, we introduce Rule-Driven Approach for Attribute Embedding (RAE), a novel methodology that enhances recommendation performance by effectively mining and utilizing semantic rules from property graphs. RAE applies a rule-mining process to extract meaningful rules that guide random walks in generating enriched attribute embeddings. These enriched embeddings are subsequently integrated into GCNs, surpassing conventional triple-based embedding techniques. We evaluate RAE on real-world datasets (e.g., Blogcatalog and Flickr) and demonstrate that RAE achieves an average improvement of 10.6% in both Recall@20 and NDCG@20 compared to state-of-the-art baselines, indicating superior relevance coverage and ranking rationality in top-20 recommendations. Additionally, RAE exhibits enhanced robustness against data sparsity and the attribute missingness problem. Our novel approach underscores the significant performance gains achieved in recommendation systems by fully leveraging attribute information within property graphs, enhancing both effectiveness and reliability.

cs.IR↗

arXiv2025

Optimizing Token Consumption in LLMs: A Nano Surge Approach for Code Reasoning Efficiency

With the increasing adoption of large language models (LLMs) in software engineering, the Chain of Thought (CoT) reasoning paradigm has become an essential approach for automated code repair. However, the explicit multi-step reasoning in CoT leads to substantial increases in token consumption, reducing inference efficiency and raising computational costs, especially for complex code repair tasks. Most prior research has focused on improving the correctness of code repair while largely overlooking the resource efficiency of the reasoning process itself. To address this challenge, this paper proposes three targeted optimization strategies: Context Awareness, Responsibility Tuning, and Cost Sensitive. Context Awareness guides the model to focus on key contextual information, Responsibility Tuning refines the structure of the reasoning process through clearer role and responsibility assignment, and Cost Sensitive incorporates resource-awareness to suppress unnecessary token generation during inference. Experiments across diverse code repair scenarios demonstrate that these methods can significantly reduce token consumption in CoT-based reasoning without compromising repair quality. This work provides novel insights and methodological guidance for enhancing the efficiency of LLM-driven code repair tasks in software engineering.

cs.SE↗

arXiv2025

FastER: On-Demand Entity Resolution in Property Graphs

Entity resolution (ER) is the problem of identifying and linking database records that refer to the same real-world entity. Traditional ER methods use batch processing, which becomes impractical with growing data volumes due to high computational costs and lack of real-time capabilities. In many applications, users need to resolve entities for only a small portion of their data, making full data processing unnecessary -- a scenario known as "ER-on-demand". This paper proposes FastER, an efficient ER-on-demand framework for property graphs. Our approach uses graph differential dependencies (GDDs) as a knowledge encoding language to design effective filtering mechanisms that leverage both structural and attribute semantics of graphs. We construct a blocking graph from filtered subgraphs to reduce the number of candidate entity pairs requiring comparison. Additionally, FastER incorporates Progressive Profile Scheduling (PPS), allowing the system to incrementally produce results throughout the resolution process. Extensive evaluations on multiple benchmark datasets demonstrate that FastER significantly outperforms state-of-the-art ER methods in computational efficiency and real-time processing for on-demand tasks while ensuring reliability. We make FastER publicly available at: https://anonymous.4open.science/r/On_Demand_Entity_Resolution-9DFB

cs.DB↗

arXiv2024

Best Practices for Distilling Large Language Models into BERT for Web Search Ranking

Recent studies have highlighted the significant potential of Large Language Models (LLMs) as zero-shot relevance rankers. These methods predominantly utilize prompt learning to assess the relevance between queries and documents by generating a ranked list of potential documents. Despite their promise, the substantial costs associated with LLMs pose a significant challenge for their direct implementation in commercial search systems. To overcome this barrier and fully exploit the capabilities of LLMs for text ranking, we explore techniques to transfer the ranking expertise of LLMs to a more compact model similar to BERT, using a ranking loss to enable the deployment of less resource-intensive models. Specifically, we enhance the training of LLMs through Continued Pre-Training, taking the query as input and the clicked title and summary as output. We then proceed with supervised fine-tuning of the LLM using a rank loss, assigning the final token as a representative of the entire sentence. Given the inherent characteristics of autoregressive language models, only the final token can encapsulate all preceding tokens. Additionally, we introduce a hybrid point-wise and margin MSE loss to transfer the ranking knowledge from LLMs to smaller models like BERT. This method creates a viable solution for environments with strict resource constraints. Both offline and online evaluations have confirmed the efficacy of our approach, and our model has been successfully integrated into a commercial web search engine as of February 2024.

cs.IR↗

arXiv2024

GIG: Graph Data Imputation With Graph Differential Dependencies

Data imputation addresses the challenge of imputing missing values in database instances, ensuring consistency with the overall semantics of the dataset. Although several heuristics which rely on statistical methods, and ad-hoc rules have been proposed. These do not generalise well and often lack data context. Consequently, they also lack explainability. The existing techniques also mostly focus on the relational data context making them unsuitable for wider application contexts such as in graph data. In this paper, we propose a graph data imputation approach called GIG which relies on graph differential dependencies (GDDs). GIG, learns the GDDs from a given knowledge graph, and uses these rules to train a transformer model which then predicts the value of missing data within the graph. By leveraging GDDs, GIG incoporates semantic knowledge into the data imputation process making it more reliable and explainable. Experimental results on seven real-world datasets highlight GIG's effectiveness compared to existing state-of-the-art approaches.

cs.AI↗

arXiv2024

A Heterogeneous Network-based Contrastive Learning Approach for Predicting Drug-Target Interaction

Drug-target interaction (DTI) prediction is crucial for drug development and repositioning. Methods using heterogeneous graph neural networks (HGNNs) for DTI prediction have become a promising approach, with attention-based models often achieving excellent performance. However, these methods typically overlook edge features when dealing with heterogeneous biomedical networks. We propose a heterogeneous network-based contrastive learning method called HNCL-DTI, which designs a heterogeneous graph attention network to predict potential/novel DTIs. Specifically, our HNCL-DTI utilizes contrastive learning to collaboratively learn node representations from the perspective of both node-based and edge-based attention within the heterogeneous structure of biomedical networks. Experimental results show that HNCL-DTI outperforms existing advanced baseline methods on benchmark datasets, demonstrating strong predictive ability and practical effectiveness. The data and source code are available at https://github.com/Zaiwen/HNCL-DTI.

q-bio.BM↗

arXiv2024

When GDD meets GNN: A Knowledge-driven Neural Connection for Effective Entity Resolution in Property Graphs

This paper studies the entity resolution (ER) problem in property graphs. ER is the task of identifying and linking different records that refer to the same real-world entity. It is commonly used in data integration, data cleansing, and other applications where it is important to have accurate and consistent data. In general, two predominant approaches exist in the literature: rule-based and learning-based methods. On the one hand, rule-based techniques are often desired due to their explainability and ability to encode domain knowledge. Learning-based methods, on the other hand, are preferred due to their effectiveness in spite of their black-box nature. In this work, we devise a hybrid ER solution, GraphER, that leverages the strengths of both systems for property graphs. In particular, we adopt graph differential dependency (GDD) for encoding the so-called record-matching rules, and employ them to guide a graph neural network (GNN) based representation learning for the task. We conduct extensive empirical evaluation of our proposal on benchmark ER datasets including 17 graph datasets and 7 relational datasets in comparison with 10 state-of-the-art (SOTA) techniques. The results show that our approach provides a significantly better solution to addressing ER in graph data, both quantitatively and qualitatively, while attaining highly competitive results on the benchmark relational datasets w.r.t. the SOTA solutions.

cs.DB↗

arXiv2023

Layer-by-layer phase transformation in Ti$_3$O$_5$ revealed by machine learning molecular dynamics simulations

Reconstructive phase transitions involving breaking and reconstruction of primary chemical bonds are ubiquitous and important for many technological applications. In contrast to displacive phase transitions, the dynamics of reconstructive phase transitions are usually slow due to the large energy barrier. Nevertheless, the reconstructive phase transformation from $\beta$- to $\lambda$-Ti$_3$O$_5$ exhibits an ultrafast and reversible behavior. Despite extensive studies, the underlying microscopic mechanism remains unclear. Here, we discover a kinetically favorable in-plane nucleated layer-by-layer transformation mechanism through metadynamics and large-scale molecular dynamics simulations. This is enabled by developing an efficient machine learning potential with near first-principles accuracy through an on-the-fly active learning method and an advanced sampling technique. Our results reveal that the $\beta$-$\lambda$ phase transformation initiates with the formation of two-dimensional nuclei in the $ab$-plane and then proceeds layer-by-layer through a multistep barrier-lowering kinetic process via intermediate metastable phases. Our work not only provides important insight into the ultrafast and reversible nature of the $\beta$-$\lambda$ transition, but also presents useful strategies and methods for tackling other complex structural phase transitions.

cond-mat.mtrl-sci↗

arXiv2023

Unlocking the Structural Mystery of Vaterite CaCO3

Calcium carbonate (CaCO3), the most abundant biogenic mineral on earth, plays a crucial role in various fields. Of the four polymorphs, calcite, aragonite, vaterite, and amorphous CaCO3, vaterite is the most enigmatic one due to an ongoing debate regarding its structure that has persisted for nearly a century. In this work, based on systematic transmission electron microscopy characterizations, elaborate crystallographic analysis and machine learning aided molecular dynamics simulations with ab initio accuracy, we reveal that vaterite can be regarded as a polytypic structure. The basic phase is a monoclinic lattice possessing pseudohexagonal symmetry. Direct imaging and atomic-scale simulations provide evidence that a single grain of vaterite can have three orientation variants. Additionally, we find that vaterite undergoes a second-order phase transition. These atomic scale insights provide a comprehensive understanding of the structure of vaterite and offer new perspectives on the biomineralization process of calcium carbonate.

cond-mat.mtrl-sci↗