arXiv subjects

Kai Wang

Publications and source records attributed to Kai Wang.

At least 343 records · Page 19Linked to original sources

arXiv2025

Emphasizing Discriminative Features for Dataset Distillation in Complex Scenarios

Dataset distillation has demonstrated strong performance on simple datasets like CIFAR, MNIST, and TinyImageNet but struggles to achieve similar results in more complex scenarios. In this paper, we propose EDF (emphasizes the discriminative features), a dataset distillation method that enhances key discriminative regions in synthetic images using Grad-CAM activation maps. Our approach is inspired by a key observation: in simple datasets, high-activation areas typically occupy most of the image, whereas in complex scenarios, the size of these areas is much smaller. Unlike previous methods that treat all pixels equally when synthesizing images, EDF uses Grad-CAM activation maps to enhance high-activation areas. From a supervision perspective, we downplay supervision signals that have lower losses, as they contain common patterns. Additionally, to help the DD community better explore complex scenarios, we build the Complex Dataset Distillation (Comp-DD) benchmark by meticulously selecting sixteen subsets, eight easy and eight hard, from ImageNet-1K. In particular, EDF consistently outperforms SOTA results in complex scenarios, such as ImageNet-1K subsets. Hopefully, more researchers will be inspired and encouraged to improve the practicality and efficacy of DD. Our code and benchmark will be made public at https://github.com/NUS-HPC-AI-Lab/EDF.

cs.CV↗

arXiv2025

Deformations of Nijenhuis Lie algebras and Nijenhuis Lie algebroids

This paper is the second in a series dedicated to the operadic study of Nijenhuis structures, focusing on Nijenhuis Lie algebras and Nijenhuis geometry. We introduce the concept of homotopy Nijenhuis Lie algebras and establish that the differential graded (=dg) operad $\mathfrak{NjL}_{\infty}$ governing these structures serves as the minimal model of the operad $\mathfrak{NjL}$ for Nijenhuis Lie algebras. We construct an $L_\infty$-algebra that encodes the simultaneous deformations of Lie brackets and Nijenhuis operators, leading to the deformation cochain complex and an associated cohomology theory for Nijenhuis Lie algebras. Extending these ideas to geometry, we investigate the deformations of geometric Nijenhuis structures. We introduce the notion of a Nijenhuis Lie algebroid-a Lie algebroid equipped with a Nijenhuis structure, which generalizes the classical Nijenhuis structure on vector fields of manifolds. Using the framework of dg manifolds, we construct an $L_\infty$-algebra that governs the simultaneous deformations of Lie algebroid structures and Nijenhuis operators. As a computational application, we prove that a certain class of Nijenhuis operators satisfies the Poincaré Lemma, meaning its cohomology vanishes, which confirms a conjecture by Bolsinov and Konyaev.

math.DG↗

arXiv2025

Advancing CAN Network Security through RBM-Based Synthetic Attack Data Generation for Intrusion Detection Systems

The rapid development of network technologies and industrial intelligence has augmented the connectivity and intelligence within the automotive industry. Notably, in the Internet of Vehicles (IoV), the Controller Area Network (CAN), which is crucial for the communication of electronic control units but lacks inbuilt security measures, has become extremely vulnerable to severe cybersecurity threats. Meanwhile, the efficacy of Intrusion Detection Systems (IDS) is hampered by the scarcity of sufficient attack data for robust model training. To overcome this limitation, we introduce a novel methodology leveraging the Restricted Boltzmann Machine (RBM) to generate synthetic CAN attack data, thereby producing training datasets with a more balanced sample distribution. Specifically, we design a CAN Data Processing Module for transforming raw CAN data into an RBM-trainable format, and a Negative Sample Generation Module to generate data reflecting the distribution of CAN data frames denoting network intrusions. Experimental results show the generated data significantly improves IDS performance, with CANet accuracy rising from 0.6477 to 0.9725 and EfficientNet from 0.1067 to 0.1555. Code is available at https://github.com/wangkai-tech23/CANDataSynthetic.

cs.CR↗

arXiv2025

A Closer Look at Time Steps is Worthy of Triple Speed-Up for Diffusion Model Training

Training diffusion models is always a computation-intensive task. In this paper, we introduce a novel speed-up method for diffusion model training, called, which is based on a closer look at time steps. Our key findings are: i) Time steps can be empirically divided into acceleration, deceleration, and convergence areas based on the process increment. ii) These time steps are imbalanced, with many concentrated in the convergence area. iii) The concentrated steps provide limited benefits for diffusion training. To address this, we design an asymmetric sampling strategy that reduces the frequency of steps from the convergence area while increasing the sampling probability for steps from other areas. Additionally, we propose a weighting strategy to emphasize the importance of time steps with rapid-change process increments. As a plug-and-play and architecture-agnostic approach, SpeeD consistently achieves 3-times acceleration across various diffusion architectures, datasets, and tasks. Notably, due to its simple design, our approach significantly reduces the cost of diffusion model training with minimal overhead. Our research enables more researchers to train diffusion models at a lower cost.

cs.LG↗

arXiv2025

Motif Counting in Complex Networks: A Comprehensive Survey

Motif counting plays a crucial role in understanding the structural properties of networks. By computing motif frequencies, researchers can draw key insights into the structural properties of the underlying network. As networks become increasingly complex, different graph models have been proposed, giving rise to diverse motif patterns. These variations introduce unique computational challenges that require specialized algorithms tailored to specific motifs within different graph structures. This survey provides a comprehensive and structured overview of motif counting techniques across general graphs, heterogeneous graphs, and hypergraphs. We categorize existing algorithms according to their underlying computational strategies, emphasizing key similarities and distinctions. In addition to reviewing current methodologies, we examine their strengths, limitations, and computational trade-offs. Furthermore, we explore future directions in motif counting, including scalable implementations to improve efficiency in large-scale networks, algorithmic adaptations for dynamic, temporal, and attributed graphs, and deeper integration with large language models (LLMs) and graph-based retrieval-augmented generation (GraphRAG). By offering a detailed analysis of these approaches, this survey aims to support researchers and practitioners in advancing motif counting for increasingly complex network data.

cs.SI↗

arXiv2025

STATGRAPH: Effective In-vehicle Intrusion Detection via Multi-view Statistical Graph Learning

In-vehicle network (IVN) is facing complex external cyber-attacks, especially the emerging masquerade attacks with extremely high difficulty of detection while serious damaging effects. In this paper, we propose the STATGRAPH, which is an effective and fine-grained intrusion detection methodology for IVN security services via multi-view statistical graph learning on in-vehicle controller area network (CAN) messages with insight into their variations in periodicity, payload and signal combinations. Specifically, STATGRAPH generates two statistical graphs, timing correlation graph (TCG) and coupling relationship graph (CRG), in every CAN message detection window, where edge attributes in TCGs represent temporal correlation between different message IDs while edge attributes in CRGs denote the neighbour relationship and contextual similarity. Besides, a lightweight shallow layered graph convolution network is trained based on graph property of TCGs and CRGs, which learns the universal laws of various patterns more effectively and further enhance the performance of detection. To address the problem of insufficient attack types in previous intrusion detection, we select two real in-vehicle CAN datasets covering five new instances of sophisticated and stealthy masquerade attacks that are never investigated before. Experimental result shows STATGRAPH improves both detection granularity and detection performance over state-of-the-art intrusion detection methods. Code is available at https://github.com/wangkai-tech23/StatGraph.

cs.NI↗

arXiv2025

ATOM: Attention Mixer for Efficient Dataset Distillation

Recent works in dataset distillation seek to minimize training expenses by generating a condensed synthetic dataset that encapsulates the information present in a larger real dataset. These approaches ultimately aim to attain test accuracy levels akin to those achieved by models trained on the entirety of the original dataset. Previous studies in feature and distribution matching have achieved significant results without incurring the costs of bi-level optimization in the distillation process. Despite their convincing efficiency, many of these methods suffer from marginal downstream performance improvements, limited distillation of contextual information, and subpar cross-architecture generalization. To address these challenges in dataset distillation, we propose the ATtentiOn Mixer (ATOM) module to efficiently distill large datasets using a mixture of channel and spatial-wise attention in the feature matching process. Spatial-wise attention helps guide the learning process based on consistent localization of classes in their respective images, allowing for distillation from a broader receptive field. Meanwhile, channel-wise attention captures the contextual information associated with the class itself, thus making the synthetic image more informative for training. By integrating both types of attention, our ATOM module demonstrates superior performance across various computer vision datasets, including CIFAR10/100 and TinyImagenet. Notably, our method significantly improves performance in scenarios with a low number of images per class, thereby enhancing its potential. Furthermore, we maintain the improvement in cross-architectures and applications such as neural architecture search.

cs.CV↗

arXiv2025

InterLCM: Low-Quality Images as Intermediate States of Latent Consistency Models for Effective Blind Face Restoration

Diffusion priors have been used for blind face restoration (BFR) by fine-tuning diffusion models (DMs) on restoration datasets to recover low-quality images. However, the naive application of DMs presents several key limitations. (i) The diffusion prior has inferior semantic consistency (e.g., ID, structure and color.), increasing the difficulty of optimizing the BFR model; (ii) reliance on hundreds of denoising iterations, preventing the effective cooperation with perceptual losses, which is crucial for faithful restoration. Observing that the latent consistency model (LCM) learns consistency noise-to-data mappings on the ODE-trajectory and therefore shows more semantic consistency in the subject identity, structural information and color preservation, we propose InterLCM to leverage the LCM for its superior semantic consistency and efficiency to counter the above issues. Treating low-quality images as the intermediate state of LCM, InterLCM achieves a balance between fidelity and quality by starting from earlier LCM steps. LCM also allows the integration of perceptual loss during training, leading to improved restoration quality, particularly in real-world scenarios. To mitigate structural and semantic uncertainties, InterLCM incorporates a Visual Module to extract visual features and a Spatial Encoder to capture spatial details, enhancing the fidelity of restored images. Extensive experiments demonstrate that InterLCM outperforms existing approaches in both synthetic and real-world datasets while also achieving faster inference speed.

cs.CV↗

arXiv2025

ATHENA: An In-vehicle CAN Intrusion Detection Framework Based on Physical Characteristics of Vehicle Systems

With the growing interconnection between In-Vehicle Networks (IVNs) and external environments, intelligent vehicles are increasingly vulnerable to sophisticated external network attacks. This paper proposes ATHENA, the first IVN intrusion detection framework that adopts a vehicle-cloud integrated architecture to achieve better security performance for the resource-constrained vehicular environment. Specifically, in the cloud with sufficient resources, ATHENA uses the clustering method of multi-distribution mixture model combined with deep data mining technology to generate the raw Payload Rule Bank of IVN CAN messages, and then improves the rule quality with the help of exploitation on the first-principled physical knowledge of the vehicle system, after which the payload rules are periodically sent to the vehicle terminal. At the vehicle terminal, a simple LSTM component is used to generate the Time Rule Bank representing the long-term time series dependencies and the periodic characteristics of CAN messages, but not for any detection tasks as in traditional usage scenarios, where only the generated time rules are the candidates for further IVN intrusion detection tasks. Based on both the payload and time rules generated from cloud and vehicle terminal, ATHENA can achieve efficient intrusion detection capability by simple rule-base matching operations, rather than using complex black-box reasoning of resource-intensive neural network models, which is in fact only used for rule logic generation phase instead of the actual intrusion detection phase in our framework. Comparative experimental results on the ROAD dataset, which is current the most outstanding real-world in-vehicle CAN dataset covering new instances of sophisticated and stealthy masquerade attacks, demonstrate ATHENA significantly outperforms the state-of-the-art IVN intrusion detection methods in detecting complex attacks.

cs.CR↗

arXiv2025

Causal Deciphering and Inpainting in Spatio-Temporal Dynamics via Diffusion Model

Spatio-temporal (ST) prediction has garnered a De facto attention in earth sciences, such as meteorological prediction, human mobility perception. However, the scarcity of data coupled with the high expenses involved in sensor deployment results in notable data imbalances. Furthermore, models that are excessively customized and devoid of causal connections further undermine the generalizability and interpretability. To this end, we establish a causal framework for ST predictions, termed CaPaint, which targets to identify causal regions in data and endow model with causal reasoning ability in a two-stage process. Going beyond this process, we utilize the back-door adjustment to specifically address the sub-regions identified as non-causal in the upstream phase. Specifically, we employ a novel image inpainting technique. By using a fine-tuned unconditional Diffusion Probabilistic Model (DDPM) as the generative prior, we in-fill the masks defined as environmental parts, offering the possibility of reliable extrapolation for potential data distributions. CaPaint overcomes the high complexity dilemma of optimal ST causal discovery models by reducing the data generation complexity from exponential to quasi-linear levels. Extensive experiments conducted on five real-world ST benchmarks demonstrate that integrating the CaPaint concept allows models to achieve improvements ranging from 4.3% to 77.3%. Moreover, compared to traditional mainstream ST augmenters, CaPaint underscores the potential of diffusion models in ST enhancement, offering a novel paradigm for this field. Our project is available at https://anonymous.4open.science/r/12345-DFCC.

cs.CV↗

arXiv2025

Distilling Long-tailed Datasets

Dataset distillation aims to synthesize a small, information-rich dataset from a large one for efficient model training. However, existing dataset distillation methods struggle with long-tailed datasets, which are prevalent in real-world scenarios. By investigating the reasons behind this unexpected result, we identified two main causes: 1) The distillation process on imbalanced datasets develops biased gradients, leading to the synthesis of similarly imbalanced distilled datasets. 2) The experts trained on such datasets perform suboptimally on tail classes, resulting in misguided distillation supervision and poor-quality soft-label initialization. To address these issues, we first propose Distribution-agnostic Matching to avoid directly matching the biased expert trajectories. It reduces the distance between the student and the biased expert trajectories and prevents the tail class bias from being distilled to the synthetic dataset. Moreover, we improve the distillation guidance with Expert Decoupling, which jointly matches the decoupled backbone and classifier to improve the tail class performance and initialize reliable soft labels. This work pioneers the field of long-tailed dataset distillation, marking the first effective effort to distill long-tailed datasets.

cs.LG↗

arXiv2025

Ten-channel Hong-Ou-Mandel interference between independent optical combs

Dissipative Kerr soliton (DKS) frequency comb exhibits broad and narrow-linewidth frequency modes, which make it suitable for quantum communication. However, scalable quantum network based on multiple independent combs is still a challenge due to their fabrication-induced frequency mismatches. This limitation becomes critical in measurement-device-independent quantum key distribution, which requires high visibility of Hong-Ou-Mandel interference between multiple frequency channels. Here, we experimentally demonstrate two independent DKS combs with ten spectrally aligned lines without any frequency locking system. The visibility for individual comb-line pairs reaches up to $46.72 \pm 0.63\%$ via precision frequency translation, establishing a foundation for deploying DKS combs in multi-user quantum networks.

quant-ph↗

arXiv2025

MPBench: A Comprehensive Multimodal Reasoning Benchmark for Process Errors Identification

Reasoning is an essential capacity for large language models (LLMs) to address complex tasks, where the identification of process errors is vital for improving this ability. Recently, process-level reward models (PRMs) were proposed to provide step-wise rewards that facilitate reinforcement learning and data production during training and guide LLMs toward correct steps during inference, thereby improving reasoning accuracy. However, existing benchmarks of PRMs are text-based and focus on error detection, neglecting other scenarios like reasoning search. To address this gap, we introduce MPBench, a comprehensive, multi-task, multimodal benchmark designed to systematically assess the effectiveness of PRMs in diverse scenarios. MPBench employs three evaluation paradigms, each targeting a specific role of PRMs in the reasoning process: (1) Step Correctness, which assesses the correctness of each intermediate reasoning step; (2) Answer Aggregation, which aggregates multiple solutions and selects the best one; and (3) Reasoning Process Search, which guides the search for optimal reasoning steps during inference. Through these paradigms, MPBench makes comprehensive evaluations and provides insights into the development of multimodal PRMs.

cs.AI↗

arXiv2025

Make Optimization Once and for All with Fine-grained Guidance

Learning to Optimize (L2O) enhances optimization efficiency with integrated neural networks. L2O paradigms achieve great outcomes, e.g., refitting optimizer, generating unseen solutions iteratively or directly. However, conventional L2O methods require intricate design and rely on specific optimization processes, limiting scalability and generalization. Our analyses explore general framework for learning optimization, called Diff-L2O, focusing on augmenting sampled solutions from a wider view rather than local updates in real optimization process only. Meanwhile, we give the related generalization bound, showing that the sample diversity of Diff-L2O brings better performance. This bound can be simply applied to other fields, discussing diversity, mean-variance, and different tasks. Diff-L2O's strong compatibility is empirically verified with only minute-level training, comparing with other hour-levels.

cs.LG↗

arXiv2025

Optimizing for the Shortest Path in Denoising Diffusion Model

In this research, we propose a novel denoising diffusion model based on shortest-path modeling that optimizes residual propagation to enhance both denoising efficiency and quality. Drawing on Denoising Diffusion Implicit Models (DDIM) and insights from graph theory, our model, termed the Shortest Path Diffusion Model (ShortDF), treats the denoising process as a shortest-path problem aimed at minimizing reconstruction error. By optimizing the initial residuals, we improve the efficiency of the reverse diffusion process and the quality of the generated samples. Extensive experiments on multiple standard benchmarks demonstrate that ShortDF significantly reduces diffusion time (or steps) while enhancing the visual fidelity of generated samples compared to prior arts. This work, we suppose, paves the way for interactive diffusion-based applications and establishes a foundation for rapid data generation. Code is available at https://github.com/UnicomAI/ShortDF.

cs.CV↗

arXiv2025

An Improved Two-Step Attack on Lattice-Based Cryptography: A Case Study of Kyber

After three rounds of post-quantum cryptography (PQC) strict evaluations conducted by NIST, CRYSTALS-Kyber was successfully selected in July 2022 and standardized in August 2024. It becomes urgent to further evaluate Kyber's physical security for the upcoming deployment phase. In this brief, we present an improved two-step attack on Kyber to quickly recover the full secret key, s, by using much fewer power traces and less time. In the first step, we use the correlation power analysis (CPA) to obtain a portion of guess values of s with a small number of power traces. The CPA is enhanced by utilizing both Pearson and Kendall's rank correlation coefficients and modifying the leakage model to improve the accuracy. In the second step, we adopt the lattice attack to recover s based on the results of CPA. The success rate is largely built up by constructing a trial-and-error method. We deploy the reference implementations of Kyber-512, -768, and -1024 on an ARM Cortex-M4 target board and successfully recover s in approximately 9~10 minutes with at most 15 power traces, using a Xeon Gold 6342-equipped machine for the attack.

cs.CR↗

arXiv2025

HunyuanVideo: A Systematic Framework For Large Video Generative Models

Recent advancements in video generation have significantly impacted daily life for both individuals and industries. However, the leading video generation models remain closed-source, resulting in a notable performance gap between industry capabilities and those available to the public. In this report, we introduce HunyuanVideo, an innovative open-source video foundation model that demonstrates performance in video generation comparable to, or even surpassing, that of leading closed-source models. HunyuanVideo encompasses a comprehensive framework that integrates several key elements, including data curation, advanced architectural design, progressive model scaling and training, and an efficient infrastructure tailored for large-scale model training and inference. As a result, we successfully trained a video generative model with over 13 billion parameters, making it the largest among all open-source models. We conducted extensive experiments and implemented a series of targeted designs to ensure high visual quality, motion dynamics, text-video alignment, and advanced filming techniques. According to evaluations by professionals, HunyuanVideo outperforms previous state-of-the-art models, including Runway Gen-3, Luma 1.6, and three top-performing Chinese video generative models. By releasing the code for the foundation model and its applications, we aim to bridge the gap between closed-source and open-source communities. This initiative will empower individuals within the community to experiment with their ideas, fostering a more dynamic and vibrant video generation ecosystem. The code is publicly available at https://github.com/Tencent/HunyuanVideo.

cs.CV↗

arXiv2025

X-Field: A Physically Grounded Representation for 3D X-ray Reconstruction

X-ray imaging is indispensable in medical diagnostics, yet its use is tightly regulated due to potential health risks. To mitigate radiation exposure, recent research focuses on generating novel views from sparse inputs and reconstructing Computed Tomography (CT) volumes, borrowing representations from the 3D reconstruction area. However, these representations originally target visible light imaging that emphasizes reflection and scattering effects, while neglecting penetration and attenuation properties of X-ray imaging. In this paper, we introduce X-Field, the first 3D representation specifically designed for X-ray imaging, rooted in the energy absorption rates across different materials. To accurately model diverse materials within internal structures, we employ 3D ellipsoids with distinct attenuation coefficients. To estimate each material's energy absorption of X-rays, we devise an efficient path partitioning algorithm accounting for complex ellipsoid intersections. We further propose hybrid progressive initialization to refine the geometric accuracy of X-Filed and incorporate material-based optimization to enhance model fitting along material boundaries. Experiments show that X-Field achieves superior visual fidelity on both real-world human organ and synthetic object datasets, outperforming state-of-the-art methods in X-ray Novel View Synthesis and CT Reconstruction.

cs.CV↗