arXiv subjects

Hao Wang

Publications and source records attributed to Hao Wang.

At least 829 records · Page 46Linked to original sources

arXiv2025

TD3: Tucker Decomposition Based Dataset Distillation Method for Sequential Recommendation

In the era of data-centric AI, the focus of recommender systems has shifted from model-centric innovations to data-centric approaches. The success of modern AI models is built on large-scale datasets, but this also results in significant training costs. Dataset distillation has emerged as a key solution, condensing large datasets to accelerate model training while preserving model performance. However, condensing discrete and sequentially correlated user-item interactions, particularly with extensive item sets, presents considerable challenges. This paper introduces \textbf{TD3}, a novel \textbf{T}ucker \textbf{D}ecomposition based \textbf{D}ataset \textbf{D}istillation method within a meta-learning framework, designed for sequential recommendation. TD3 distills a fully expressive \emph{synthetic sequence summary} from original data. To efficiently reduce computational complexity and extract refined latent patterns, Tucker decomposition decouples the summary into four factors: \emph{synthetic user latent factor}, \emph{temporal dynamics latent factor}, \emph{shared item latent factor}, and a \emph{relation core} that models their interconnections. Additionally, a surrogate objective in bi-level optimization is proposed to align feature spaces extracted from models trained on both original data and synthetic sequence summary beyond the naïve performance matching approach. In the \emph{inner-loop}, an augmentation technique allows the learner to closely fit the synthetic summary, ensuring an accurate update of it in the \emph{outer-loop}. To accelerate the optimization process and address long dependencies, RaT-BPTT is employed for bi-level optimization. Experiments and analyses on multiple public datasets have confirmed the superiority and cross-architecture generalizability of the proposed designs. Codes are released at https://github.com/USTC-StarTeam/TD3.

cs.IR↗

arXiv2025

FuXi-$α$: Scaling Recommendation Model with Feature Interaction Enhanced Transformer

Inspired by scaling laws and large language models, research on large-scale recommendation models has gained significant attention. Recent advancements have shown that expanding sequential recommendation models to large-scale recommendation models can be an effective strategy. Current state-of-the-art sequential recommendation models primarily use self-attention mechanisms for explicit feature interactions among items, while implicit interactions are managed through Feed-Forward Networks (FFNs). However, these models often inadequately integrate temporal and positional information, either by adding them to attention weights or by blending them with latent representations, which limits their expressive power. A recent model, HSTU, further reduces the focus on implicit feature interactions, constraining its performance. We propose a new model called FuXi-$α$ to address these issues. This model introduces an Adaptive Multi-channel Self-attention mechanism that distinctly models temporal, positional, and semantic features, along with a Multi-stage FFN to enhance implicit feature interactions. Our offline experiments demonstrate that our model outperforms existing models, with its performance continuously improving as the model size increases. Additionally, we conducted an online A/B test within the Huawei Music app, which showed a $4.76\%$ increase in the average number of songs played per user and a $5.10\%$ increase in the average listening duration per user. Our code has been released at https://github.com/USTC-StarTeam/FuXi-alpha.

cs.IR↗

arXiv2025

Model Supply Chain Poisoning: Backdooring Pre-trained Models via Embedding Indistinguishability

Pre-trained models (PTMs) are widely adopted across various downstream tasks in the machine learning supply chain. Adopting untrustworthy PTMs introduces significant security risks, where adversaries can poison the model supply chain by embedding hidden malicious behaviors (backdoors) into PTMs. However, existing backdoor attacks to PTMs can only achieve partially task-agnostic and the embedded backdoors are easily erased during the fine-tuning process. This makes it challenging for the backdoors to persist and propagate through the supply chain. In this paper, we propose a novel and severer backdoor attack, TransTroj, which enables the backdoors embedded in PTMs to efficiently transfer in the model supply chain. In particular, we first formalize this attack as an indistinguishability problem between poisoned and clean samples in the embedding space. We decompose embedding indistinguishability into pre- and post-indistinguishability, representing the similarity of the poisoned and reference embeddings before and after the attack. Then, we propose a two-stage optimization that separately optimizes triggers and victim PTMs to achieve embedding indistinguishability. We evaluate TransTroj on four PTMs and six downstream tasks. Experimental results show that our method significantly outperforms SOTA task-agnostic backdoor attacks -- achieving nearly 100% attack success rate on most downstream tasks -- and demonstrates robustness under various system settings. Our findings underscore the urgent need to secure the model supply chain against such transferable backdoor attacks. The code is available at https://github.com/haowang-cqu/TransTroj .

cs.CR↗

arXiv2025

Dr. SoW: Density Ratio of Strong-over-weak LLMs for Reducing the Cost of Human Annotation in Preference Tuning

Preference tuning relies on high-quality human preference data, which is often expensive and time-consuming to gather. In this paper, we introduce Dr.SoW (Density Ratio of Strong over Weak) a cost-effective method that eliminates the reliance for human annotation by leveraging off-the-shelf LLMs for preference data annotation. Dr.SoW uses the log-density ratio between a better-aligned and a less-aligned LLM as a reward signal. We evaluate Dr.SoW across 221 different LLM pairs and empirically find a strong correlation between the performance gap of the paired models and the quality of the reward signal. This insight provides a practical guideline for selecting LLMs for data annotation. Additionally, we introduce an end-to-end pipeline that customizes reward functions based on user query domains. Without fine-tuning, it improves accuracy on domain-specific evaluations. With a pair of Mistral-7B models, Dr.SoW achieves a RewardBench score of 82.6, outperforming the best trained reward functions from same model class and demonstrating competitive performance against SoTA models in Safety (91.0) and Reasoning (88.0) domains. Further, we preference-tune Llama-3-8B-Instruct using data annotated by Dr.SoW. Our approach pushes Llama-3-8B to achieve a 37.4 % (+15.1 %) win rate on ArenaHard and a 40.7 % (+17.8 %) win rate on length-controlled AlpacaEval 2.0.

cs.CL↗

arXiv2025

SliceOcc: Indoor 3D Semantic Occupancy Prediction with Vertical Slice Representation

3D semantic occupancy prediction is a crucial task in visual perception, as it requires the simultaneous comprehension of both scene geometry and semantics. It plays a crucial role in understanding 3D scenes and has great potential for various applications, such as robotic vision perception and autonomous driving. Many existing works utilize planar-based representations such as Bird's Eye View (BEV) and Tri-Perspective View (TPV). These representations aim to simplify the complexity of 3D scenes while preserving essential object information, thereby facilitating efficient scene representation. However, in dense indoor environments with prevalent occlusions, directly applying these planar-based methods often leads to difficulties in capturing global semantic occupancy, ultimately degrading model performance. In this paper, we present a new vertical slice representation that divides the scene along the vertical axis and projects spatial point features onto the nearest pair of parallel planes. To utilize these slice features, we propose SliceOcc, an RGB camera-based model specifically tailored for indoor 3D semantic occupancy prediction. SliceOcc utilizes pairs of slice queries and cross-attention mechanisms to extract planar features from input images. These local planar features are then fused to form a global scene representation, which is employed for indoor occupancy prediction. Experimental results on the EmbodiedScan dataset demonstrate that SliceOcc achieves a mIoU of 15.45% across 81 indoor categories, setting a new state-of-the-art performance among RGB camera-based models for indoor 3D semantic occupancy prediction. Code is available at https://github.com/NorthSummer/SliceOcc.

cs.CV↗

arXiv2025

Anomaly Detection in Cooperative Vehicle Perception Systems under Imperfect Communication

Anomaly detection is a critical requirement for ensuring safety in autonomous driving. In this work, we leverage Cooperative Perception to share information across nearby vehicles, enabling more accurate identification and consensus of anomalous behaviors in complex traffic scenarios. To account for the real-world challenge of imperfect communication, we propose a cooperative-perception-based anomaly detection framework (CPAD), which is a robust architecture that remains effective under communication interruptions, thereby facilitating reliable performance even in low-bandwidth settings. Since no multi-agent anomaly detection dataset exists for vehicle trajectories, we introduce 15,000 different scenarios with a 90,000 trajectories benchmark dataset generated through rule-based vehicle dynamics analysis. Empirical results demonstrate that our approach outperforms standard anomaly classification methods in F1-score, AUC and showcase strong robustness to agent connection interruptions.

cs.MA↗

arXiv2025

BLoB: Bayesian Low-Rank Adaptation by Backpropagation for Large Language Models

Large Language Models (LLMs) often suffer from overconfidence during inference, particularly when adapted to downstream domain-specific tasks with limited data. Previous work addresses this issue by employing approximate Bayesian estimation after the LLMs are trained, enabling them to quantify uncertainty. However, such post-training approaches' performance is severely limited by the parameters learned during training. In this paper, we go beyond post-training Bayesianization and propose Bayesian Low-Rank Adaptation by Backpropagation (BLoB), an algorithm that continuously and jointly adjusts both the mean and covariance of LLM parameters throughout the whole fine-tuning process. Our empirical results verify the effectiveness of BLoB in terms of generalization and uncertainty estimation, when evaluated on both in-distribution and out-of-distribution data.

cs.LG↗

arXiv2025

THOR: A Generic Energy Estimation Approach for On-Device Training

Battery-powered mobile devices (e.g., smartphones, AR/VR glasses, and various IoT devices) are increasingly being used for AI training due to their growing computational power and easy access to valuable, diverse, and real-time data. On-device training is highly energy-intensive, making accurate energy consumption estimation crucial for effective job scheduling and sustainable AI. However, the heterogeneity of devices and the complexity of models challenge the accuracy and generalizability of existing estimation methods. This paper proposes THOR, a generic approach for energy consumption estimation in deep neural network (DNN) training. First, we examine the layer-wise energy additivity property of DNNs and strategically partition the entire model into layers for fine-grained energy consumption profiling. Then, we fit Gaussian Process (GP) models to learn from layer-wise energy consumption measurements and estimate a DNN's overall energy consumption based on its layer-wise energy additivity property. We conduct extensive experiments with various types of models across different real-world platforms. The results demonstrate that THOR has effectively reduced the Mean Absolute Percentage Error (MAPE) by up to 30%. Moreover, THOR is applied in guiding energy-aware pruning, successfully reducing energy consumption by 50%, thereby further demonstrating its generality and potential.

cs.LG↗

arXiv2025

The silent threat of methane to ecosystems: Insights from mechanistic modelling

Over the past century, atmospheric methane levels have nearly doubled, posing a significant threat to ecosystems. Despite this, studies on its direct impact on species interactions are lacking. Although bioaccumulation theory explains the effects of contaminants in trophic levels, it is inadequate for gaseous pollutants such as methane. This study aims to bridge the gap by developing a methane-population-detritus model to investigate ecological impacts in aquatic and terrestrial ecosystems. Our findings show that low methane concentrations can enhance species growth, while moderate accumulation may induce sub-lethal effects over time. Elevated methane levels, however, lead to ecosystem collapse. Furthermore, prolonged exposure to the gas increases the sensitivity of species towards rising temperatures. Multiscale analysis reveals that rapid methane accumulation leads to long transients near the extinction states. We argue that high emission rates can push the system towards a critical threshold, where the ecosystem shifts to an alternative stable state characterized by elevated methane concentrations. This work highlights the urgent need for a better understanding of the fatal role of methane in ecosystems for developing strategies to mitigate its effects amid climate change.

q-bio.PE↗

arXiv2025

Wafer-scale Integration of Single-Crystalline MoS$_2$ for Flexible Electronics Enabled by Oxide Dry-transfer

Atomically thin, single-crystalline transition metal dichalcogenides (TMDCs) grown via chemical vapor deposition (CVD) on sapphire substrates exhibit exceptional mechanical and electrical properties, positioning them as excellent channel materials for flexible electronics. However, conventional wet-transfer processes for integrating these materials onto flexible substrates often introduce surface contamination, significantly degrading device performance. Here, we present a wafer-scale dry-transfer technique using a high-dielectric oxide as the transfer medium, enabling the integration of 4-inch single-crystalline MoS$_2$ onto flexible substrates. This method eliminates contact with polymers or solvents, thus preserving the intrinsic electronic properties of MoS$_2$. As a result, the fabricated flexible field-effect transistor (FET) arrays exhibit remarkable performance, with a mobility of 117 cm$^2$/Vs, a subthreshold swing of 68.8 mV dec$^{-1}$, and an ultra-high current on/off ratio of $10^{12}$-values comparable to those achieved on rigid substrates. Leveraging the outstanding electrical characteristics, we demonstrated MoS$_2$-based flexible inverters operating in the subthreshold regime, achieving both a high gain of 218 and ultra-low power consumption of 1.4 pW/$μ$m. Additionally, we integrated a flexible tactile sensing system driven by active-matrix MoS$_2$ FET arrays onto a robotic gripper, enabling real-time object identification. These findings demonstrate the simultaneous achievement of high electrical performance and flexibility, highlighting the immense potential of single-crystalline TMDC-based flexible electronics for real-world applications.

physics.app-ph↗

arXiv2025

Existence and asymptotic stability of a generic Lotka-Volterra system with nonlinear spatially heterogeneous cross-diffusion

This article considers a class of Lotka-Volterra systems with multiple nonlinear cross-diffusion, commonly known as prey-taxis models. The existence and stability of classic solutions for such systems with spatially homogeneous sources and taxis have been studied in one- or two-dimensional space, however, the proof is non-trivial for a more general setting with spatially heterogeneous predation functions and taxis coefficient functions in arbitrary dimensions. This study introduces a new weighted $L_ε^p$-norm and extends some classical inequalities within this normed space. Coupled energy estimates are employed to establish initial bounds, followed by applying heat kernel properties and an advanced bootstrap process to enhance solution regularity. For stability analysis, we extend LaSalle's invariance principle to a general $ L^\infty $ setting and utilize it alongside Lyapunov functions to analyze the stability of each possible constant equilibrium. All results are achieved without introducing an extra logistic growth term for predators or imposing smallness conditions on taxis coefficients.

math.AP↗

arXiv2025

Shifts of dominant personality and spatial pattern formation due to spatially heterogeneous pollution

Personality traits, such as boldness and shyness, play a significant role in shaping the survival strategies of animals. Industrial pollution has long posed serious threats to ecosystems and is typically distributed heterogeneously. However, how animals with different personalities respond to spatially heterogeneous pollution remains largely unexplored. In this study, we introduce a prey-taxis model with nonlinear cross-diffusion to examine population dynamics in such environments. The global existence of classical solutions is established by deriving initial bounds through energy estimates and improving solution regularity via heat kernel properties and a bootstrap process. Our findings reveal that behavior, population structure, and spatial distribution are heavily influenced by pollution. Bold individuals maintain a competitive advantage in pollution-free or very low-toxin environments, whereas shy individuals become dominant in regions with low to moderate toxin levels. In highly polluted areas, no populations can survive. The spatial pattern of the population is also closely tied to the distribution of toxins. Grazers tend to move along toxin gradient and exhibit periodic behavior. As toxin concentrations rise, aggregation behavior becomes increasingly pronounced across all species. Interestingly, the total population in polluted areas may initially increase when toxin levels are low to moderate, but eventually declines, leading to extinction as toxin levels continue to rise.

q-bio.PE↗

arXiv2025

A Study of Subsurface Convection Zones of Fast Rotating Massive Stars

The subsurface convective zones (CZs) of massive stars significantly influences many of their key characteristics. Previous studies have paid little attention to the impact of rotation on the subsurface convective zone (CZ), so we aim to investigate the evolution of this zone in rapidly rotating massive stars. We use the Modules for Experiments in Stellar Astrophysics (MESA) to simulate the subsurface CZs of massive stars during the main sequence phase. We establish stellar models with initial masses ranging from 5 $M_{\odot}$ to 120 $M_{\odot}$, incorporating four metallicities (Z = 0.02, 0.006, 0.002, and 0.0001) and three rotational velocities ($\mitω/ω_{\text {crit}}$ = 0, $\mitω/ω_{\text {crit}}$ = 0.50, and $\mitω/ω_{\text {crit}}$ = 0.75). We find that rapid rotation leads to an expansion of the subsurface CZ, increases convective velocities, and promotes the development of this zone. Additionally, subsurface CZs can also emerge in stars with lower metallicities. Comparing our models with observations of massive stars in the Galaxy, the Large Magellanic Cloud, and the Small Magellanic Cloud, we find that rotating models better encompass the observed samples. Rotation significantly influences the evolution of the subsurface CZ in massive stars. By comparing with the observed microturbulence on the surfaces of OB stars, we propose that the subsurface CZs may be one of the sources of microturbulence.

astro-ph.SR↗

arXiv2025

InsTex: Indoor Scenes Stylized Texture Synthesis

Generating high-quality textures for 3D scenes is crucial for applications in interior design, gaming, and augmented/virtual reality (AR/VR). Although recent advancements in 3D generative models have enhanced content creation, significant challenges remain in achieving broad generalization and maintaining style consistency across multiple viewpoints. Current methods, such as 2D diffusion models adapted for 3D texturing, suffer from lengthy processing times and visual artifacts, while approaches driven by 3D data often fail to generalize effectively. To overcome these challenges, we introduce InsTex, a two-stage architecture designed to generate high-quality, style-consistent textures for 3D indoor scenes. InsTex utilizes depth-to-image diffusion priors in a coarse-to-fine pipeline, first generating multi-view images with a pre-trained 2D diffusion model and subsequently refining the textures for consistency. Our method supports both textual and visual prompts, achieving state-of-the-art results in visual quality and quantitative metrics, and demonstrates its effectiveness across various 3D texturing applications.

cs.CV↗

arXiv2025

ImageRef-VL: Enabling Contextual Image Referencing in Vision-Language Models

Vision-Language Models (VLMs) have demonstrated remarkable capabilities in understanding multimodal inputs and have been widely integrated into Retrieval-Augmented Generation (RAG) based conversational systems. While current VLM-powered chatbots can provide textual source references in their responses, they exhibit significant limitations in referencing contextually relevant images during conversations. In this paper, we introduce Contextual Image Reference -- the ability to appropriately reference relevant images from retrieval documents based on conversation context -- and systematically investigate VLMs' capability in this aspect. We conduct the first evaluation for contextual image referencing, comprising a dedicated testing dataset and evaluation metrics. Furthermore, we propose ImageRef-VL, a method that significantly enhances open-source VLMs' image referencing capabilities through instruction fine-tuning on a large-scale, manually curated multimodal conversation dataset. Experimental results demonstrate that ImageRef-VL not only outperforms proprietary models but also achieves an 88% performance improvement over state-of-the-art open-source VLMs in contextual image referencing tasks. Our code is available at https://github.com/bytedance/ImageRef-VL.

cs.CV↗

arXiv2025

Differentially Private Synthetic Data Generation for Relational Databases

Existing differentially private (DP) synthetic data generation mechanisms typically assume a single-source table. In practice, data is often distributed across multiple tables with relationships across tables. In this paper, we introduce the first-of-its-kind algorithm that can be combined with any existing DP mechanisms to generate synthetic relational databases. Our algorithm iteratively refines the relationship between individual synthetic tables to minimize their approximation errors in terms of low-order marginal distributions while maintaining referential integrity. This algorithm eliminates the need to flatten a relational database into a master table (saving space), operates efficiently (saving time), and scales effectively to high-dimensional data. We provide both DP and theoretical utility guarantees for our algorithm. Through numerical experiments on real-world datasets, we demonstrate the effectiveness of our method in preserving fidelity to the original data.

cs.LG↗

arXiv2025

Observation of Impurity-Induced Scale-Free Localization in a Disordered Non-Hermitian Electrical Circuit

One of unique features of non-Hermitian systems is the extreme sensitive to their boundary conditions, e.g., the emergence of non-Hermitian skin effect (NHSE) under the open boundary conditions, where most of bulk states become localized at the boundaries. In the presence of impurities, the scale-free localization can appear, which is qualitatively distinct from the NHSE. Here, we experimentally design a disordered non-Hermitian electrical circuits in the presence of a single non-Hermitian impurity and the nonreciprocal hopping. We observe the anomalous scale-free accumulation of eigenstates, opposite to the bulk hopping direction. The experimental results open the door to further explore the anomalous skin effects in non-Hermitian electrical circuits.

cond-mat.dis-nn↗

arXiv2025

Knowledge-Guided Prompt Learning for Deepfake Facial Image Detection

Recent generative models demonstrate impressive performance on synthesizing photographic images, which makes humans hardly to distinguish them from pristine ones, especially on realistic-looking synthetic facial images. Previous works mostly focus on mining discriminative artifacts from vast amount of visual data. However, they usually lack the exploration of prior knowledge and rarely pay attention to the domain shift between training categories (e.g., natural and indoor objects) and testing ones (e.g., fine-grained human facial images), resulting in unsatisfactory detection performance. To address these issues, we propose a novel knowledge-guided prompt learning method for deepfake facial image detection. Specifically, we retrieve forgery-related prompts from large language models as expert knowledge to guide the optimization of learnable prompts. Besides, we elaborate test-time prompt tuning to alleviate the domain shift, achieving significant performance improvement and boosting the application in real-world scenarios. Extensive experiments on DeepFakeFaceForensics dataset show that our proposed approach notably outperforms state-of-the-art methods.

cs.CV↗