arXiv subjects

Kai Wang

Publications and source records attributed to Kai Wang.

At least 289 records · Page 16Linked to original sources

arXiv2025

Empowering Tabular Data Preparation with Language Models: Why and How?

Data preparation is a critical step in enhancing the usability of tabular data and thus boosts downstream data-driven tasks. Traditional methods often face challenges in capturing the intricate relationships within tables and adapting to the tasks involved. Recent advances in Language Models (LMs), especially in Large Language Models (LLMs), offer new opportunities to automate and support tabular data preparation. However, why LMs suit tabular data preparation (i.e., how their capabilities match task demands) and how to use them effectively across phases still remain to be systematically explored. In this survey, we systematically analyze the role of LMs in enhancing tabular data preparation processes, focusing on four core phases: data acquisition, integration, cleaning, and transformation. For each phase, we present an integrated analysis of how LMs can be combined with other components for different preparation tasks, highlight key advancements, and outline prospective pipelines.

cs.AI↗

arXiv2025

Unraveling the Molecular Structure of Lipid Nanoparticles through in-silico Self-Assembly for Rational Delivery Design

Lipid nanoparticles (LNPs) are a leading platform in the delivery of RNA-based therapeutics, playing a pivotal role in the clinical success of mRNA vaccines and other nucleic acid drugs. Their performance in RNA encapsulation and delivery is critically governed by the molecular structure of ionizable lipids and the overall formulation composition. However, mechanistic insight into how these factors govern LNP architecture and function remains limited, primarily owing to the challenges of capturing nanoscale assembly and organization using experimental techniques. Here, we employ coarse-grained molecular dynamics simulations to systematically investigate how ionizable lipid chemistry influences LNP self-assembly, internal organization, and surface properties. We further explore the effects of formulation ratios and pH-dependent deprotonation on both the internal structure and surface morphology of LNPs. Leveraging these insights, we demonstrate how in silico structural characteristics can inform the rational design of novel ionizable lipids and optimization of formulation ratios, supported with experimental validations. Our findings offer a molecular-level understanding of LNP assembly dynamics and architecture, thereby establishing a computational framework linking lipid chemistry and LNP formulation to the structure and performance of LNP, to advance the rational design of novel LNP delivery systems.

cond-mat.soft↗

arXiv2025

EA-ViT: Efficient Adaptation for Elastic Vision Transformer

Vision Transformers (ViTs) have emerged as a foundational model in computer vision, excelling in generalization and adaptation to downstream tasks. However, deploying ViTs to support diverse resource constraints typically requires retraining multiple, size-specific ViTs, which is both time-consuming and energy-intensive. To address this issue, we propose an efficient ViT adaptation framework that enables a single adaptation process to generate multiple models of varying sizes for deployment on platforms with various resource constraints. Our approach comprises two stages. In the first stage, we enhance a pre-trained ViT with a nested elastic architecture that enables structural flexibility across MLP expansion ratio, number of attention heads, embedding dimension, and network depth. To preserve pre-trained knowledge and ensure stable adaptation, we adopt a curriculum-based training strategy that progressively increases elasticity. In the second stage, we design a lightweight router to select submodels according to computational budgets and downstream task demands. Initialized with Pareto-optimal configurations derived via a customized NSGA-II algorithm, the router is then jointly optimized with the backbone. Extensive experiments on multiple benchmarks demonstrate the effectiveness and versatility of EA-ViT. The code is available at https://github.com/zcxcf/EA-ViT.

cs.CV↗

arXiv2025

PEBench: A Fictitious Dataset to Benchmark Machine Unlearning for Multimodal Large Language Models

Multimodal large language models (MLLMs) have achieved remarkable success in vision-language tasks, but their reliance on vast, internet-sourced data raises significant privacy and security concerns. Machine unlearning (MU) has emerged as a critical technique to address these issues, enabling the selective removal of targeted information from pre-trained models without costly retraining. However, the evaluation of MU for MLLMs remains inadequate. Existing benchmarks often lack a comprehensive scope, focusing narrowly on entities while overlooking the unlearning of broader visual concepts and the inherent semantic coupling between them. To bridge this gap, we introduce, PEBench, a novel benchmark designed to facilitate a thorough assessment of MU in MLLMs. PEBench features a fictitious dataset of personal entities and corresponding event scenes to evaluate unlearning across these distinct yet entangled concepts. We leverage this benchmark to evaluate five MU methods, revealing their unique strengths and weaknesses. Our findings show that unlearning one concept can unintentionally degrade performance on related concepts within the same image, a challenge we term cross-concept interference. Furthermore, we demonstrate the difficulty of unlearning person and event concepts simultaneously and propose an effective method to mitigate these conflicting objectives. The source code and benchmark are publicly available at https://pebench.github.io.

cs.CV↗

arXiv2025

External Large Foundation Model: How to Efficiently Serve Trillions of Parameters for Online Ads Recommendation

Ads recommendation is a prominent service of online advertising systems and has been actively studied. Recent studies indicate that scaling-up and advanced design of the recommendation model can bring significant performance improvement. However, with a larger model scale, such prior studies have a significantly increasing gap from industry as they often neglect two fundamental challenges in industrial-scale applications. First, training and inference budgets are restricted for the model to be served, exceeding which may incur latency and impair user experience. Second, large-volume data arrive in a streaming mode with data distributions dynamically shifting, as new users/ads join and existing users/ads leave the system. We propose the External Large Foundation Model (ExFM) framework to address the overlooked challenges. Specifically, we develop external distillation and a data augmentation system (DAS) to control the computational cost of training/inference while maintaining high performance. We design the teacher in a way like a foundation model (FM) that can serve multiple students as vertical models (VMs) to amortize its building cost. We propose Auxiliary Head and Student Adapter to mitigate the data distribution gap between FM and VMs caused by the streaming data issue. Comprehensive experiments on internal industrial-scale applications and public datasets demonstrate significant performance gain by ExFM.

cs.IR↗

arXiv2025

MEGA-Bench: Scaling Multimodal Evaluation to over 500 Real-World Tasks

We present MEGA-Bench, an evaluation suite that scales multimodal evaluation to over 500 real-world tasks, to address the highly heterogeneous daily use cases of end users. Our objective is to optimize for a set of high-quality data samples that cover a highly diverse and rich set of multimodal tasks, while enabling cost-effective and accurate model evaluation. In particular, we collected 505 realistic tasks encompassing over 8,000 samples from 16 expert annotators to extensively cover the multimodal task space. Instead of unifying these problems into standard multi-choice questions (like MMMU, MMBench, and MMT-Bench), we embrace a wide range of output formats like numbers, phrases, code, \LaTeX, coordinates, JSON, free-form, etc. To accommodate these formats, we developed over 40 metrics to evaluate these tasks. Unlike existing benchmarks, MEGA-Bench offers a fine-grained capability report across multiple dimensions (e.g., application, input type, output format, skill), allowing users to interact with and visualize model capabilities in depth. We evaluate a wide variety of frontier vision-language models on MEGA-Bench to understand their capabilities across these dimensions.

cs.CV↗

arXiv2025

Violation of Bell Inequality with Unentangled Photons

Violation of local realism via Bell inequality - a profound and counterintuitive manifestation of quantum theory that conflicts with the prediction of local realism - is viewed to be intimately linked with quantum entanglement. Experimental demonstrations of such a phenomenon using quantum entangled states are among the landmark experiments of modern physics and paved the way for quantum technology. Here we report the violation of the Bell inequality that cannot be described by quantum entanglement in the system but arises from quantum indistinguishability by path identity, shown by the multi-photon frustrated interference. By analyzing the measurement of four-photon frustrated interference within the standard Bell-test formalism, we find a violation of Bell inequality by more than four standard deviations. Our work establishes a connection between quantum correlation and quantum indistinguishability, providing insights into the fundamental origin of the counterintuitive characteristics observed in quantum physics.

quant-ph↗

arXiv2025

BlueLM-2.5-3B Technical Report

We present BlueLM-2.5-3B, a compact and unified dense Multimodal Large Language Model (MLLM) designed for efficient edge-device deployment, offering strong general-purpose and reasoning capabilities. To the best of our knowledge, this is the first 3B-scale MLLM to support both thinking and non-thinking modes, while also enabling explicit control over thinking token budget. BlueLM-2.5-3B is developed through diversified data curation, key data resampling, hybrid heterogeneous reinforcement learning, and a high-performance training infrastructure. Our model achieves superior multimodal capacity while preserving competitive pure-text performance with only 2.9 billion parameters. We conduct comprehensive evaluations across a broad range of multimodal and text-only benchmarks. In thinking mode, BlueLM-2.5-3B achieves comparable performance to Qwen3-4B on text-only benchmarks, and trails the larger Kimi-VL-A3B-16B by only about 5% on average across multimodal evaluations. In non-thinking mode, it outperforms Qwen2.5-VL-3B on the majority of multimodal benchmarks. Additionally, BlueLM-2.5-3B exhibits exceptional data efficiency. All of the aforementioned performance is achieved with substantially less total training data than Qwen2.5-VL-3B and Qwen3-4B. We hope our work contributes to the advancement of high-performance, on-device MLLMs and provides meaningful insights to the research community.

cs.AI↗

arXiv2025

A Hardware-Friendly Shuffling Countermeasure Against Side-Channel Attacks for Kyber

CRYSTALS-Kyber has been standardized as the only key-encapsulation mechanism (KEM) scheme by NIST to withstand attacks by large-scale quantum computers. However, the side-channel attacks (SCAs) on its implementation are still needed to be well considered for the upcoming migration. In this brief, we propose a secure and efficient hardware implementation for Kyber by incorporating a novel compact shuffling architecture. First of all, we modify the Fisher-Yates shuffle to make it more hardware-friendly. We then design an optimized shuffling architecture for the well-known open-source Kyber hardware implementation to enhance the security of all known and potential side-channel leakage points. Finally, we implement the modified Kyber design on FPGA and evaluate its security and performance. The security is verified by conducting correlation power analysis (CPA) and test vector leakage assessment (TVLA) on the hardware. Meanwhile, FPGA place-and-route results show that the proposed design reports only 8.7% degradation on the hardware efficiency compared with the original unprotected version, much better than existing hardware hiding schemes.

cs.CR↗

arXiv2025

Topology-Constrained Learning for Efficient Laparoscopic Liver Landmark Detection

Liver landmarks provide crucial anatomical guidance to the surgeon during laparoscopic liver surgery to minimize surgical risk. However, the tubular structural properties of landmarks and dynamic intraoperative deformations pose significant challenges for automatic landmark detection. In this study, we introduce TopoNet, a novel topology-constrained learning framework for laparoscopic liver landmark detection. Our framework adopts a snake-CNN dual-path encoder to simultaneously capture detailed RGB texture information and depth-informed topological structures. Meanwhile, we propose a boundary-aware topology fusion (BTF) module, which adaptively merges RGB-D features to enhance edge perception while preserving global topology. Additionally, a topological constraint loss function is embedded, which contains a center-line constraint loss and a topological persistence loss to ensure homotopy equivalence between predictions and labels. Extensive experiments on L3D and P2ILF datasets demonstrate that TopoNet achieves outstanding accuracy and computational complexity, highlighting the potential for clinical applications in laparoscopic liver surgery. Our code will be available at https://github.com/cuiruize/TopoNet.

cs.CV↗

arXiv2025

Efficient Online Reinforcement Learning for Diffusion Policy

Diffusion policies have achieved superior performance in imitation learning and offline reinforcement learning (RL) due to their rich expressiveness. However, the conventional diffusion training procedure requires samples from target distribution, which is impossible in online RL since we cannot sample from the optimal policy. Backpropagating policy gradient through the diffusion process incurs huge computational costs and instability, thus being expensive and not scalable. To enable efficient training of diffusion policies in online RL, we generalize the conventional denoising score matching by reweighting the loss function. The resulting Reweighted Score Matching (RSM) preserves the optimal solution and low computational cost of denoising score matching, while eliminating the need to sample from the target distribution and allowing learning to optimize value functions. We introduce two tractable reweighted loss functions to solve two commonly used policy optimization problems, policy mirror descent and max-entropy policy, resulting in two practical algorithms named Diffusion Policy Mirror Descent (DPMD) and Soft Diffusion Actor-Critic (SDAC). We conducted comprehensive comparisons on MuJoCo benchmarks. The empirical results show that the proposed algorithms outperform recent diffusion-policy online RLs on most tasks, and the DPMD improves more than 120% over soft actor-critic on Humanoid and Ant.

cs.LG↗

arXiv2025

Matching-Free Depth Recovery from Structured Light

We introduce a novel approach for depth estimation using images obtained from monocular structured light systems. In contrast to many existing methods that depend on image matching, our technique employs a density voxel grid to represent scene geometry. This grid is trained through self-supervised differentiable volume rendering. Our method leverages color fields derived from the projected patterns in structured light systems during the rendering process, facilitating the isolated optimization of the geometry field. This innovative approach leads to faster convergence and high-quality results. Additionally, we integrate normalized device coordinates (NDC), a distortion loss, and a distinctive surface-based color loss to enhance geometric fidelity. Experimental results demonstrate that our method outperforms current matching-based techniques in terms of geometric performance in few-shot scenarios, achieving an approximately 30% reduction in average estimated depth errors for both synthetic scenes and real-world captured scenes. Moreover, our approach allows for rapid training, being approximately three times faster than previous matching-free methods that utilize implicit representations.

cs.CV↗

arXiv2025

iLearnRobot: An Interactive Learning-Based Multi-Modal Robot with Continuous Improvement

It is crucial that robots' performance can be improved after deployment, as they are inherently likely to encounter novel scenarios never seen before. This paper presents an innovative solution: an interactive learning-based robot system powered by a Multi-modal Large Language Model(MLLM). A key feature of our system is its ability to learn from natural dialogues with non-expert users. We also propose chain of question to clarify the exact intent of the question before providing an answer and dual-modality retrieval modules to leverage these interaction events to avoid repeating same mistakes, ensuring a seamless user experience before model updates, which is in contrast to current mainstream MLLM-based robotic systems. Our system marks a novel approach in robotics by integrating interactive learning, paving the way for superior adaptability and performance in diverse environments. We demonstrate the effectiveness and improvement of our method through experiments, both quantitively and qualitatively.

cs.HC↗

arXiv2025

The physical origin of positive metallicity radial gradients in high-redshift galaxies: insights from the FIRE-2 cosmological hydrodynamic simulations

Using the FIRE-2 cosmological zoom-in simulations, we investigate the temporal evolution of gas-phase metallicity radial gradients of Milky Way-mass progenitors in the redshift range of $0.4<z<3$. We pay special attention to the occurrence of positive (i.e. inverted) metallicity gradients -- where metallicity increases with galactocentric radius. This trend, contrary to the more commonly observed negative radial gradients, has been frequently seen in recent spatially resolved grism observations. The rate of occurrence of positive gradients in FIRE-2 is about $\sim7\%$ for $0.4<z<3$ and $\sim13\%$ at higher redshifts ($1.5<z<3$), broadly consistent with observations. Moreover, we investigate the correlations among galaxy metallicity gradient, stellar mass, star formation rate (SFR), and degree of rotational support. Metallicity gradients show a strong correlation with both sSFR and the rotational-to-dispersion velocity ratio ($v_c/σ$), implying that starbursts and kinematic morphology of galaxies play significant roles in shaping these gradients. The FIRE-2 simulations indicate that galaxies with high sSFR (${\rm log(sSFR~[yr^{-1}])}\gtrsim-9.2$) and weak rotational support ($v_c/σ\lesssim 1$) are more likely -- by $\sim$15\% -- to develop positive metallicity gradients. This trend is attributed to galaxy-scale gas flows driven by stellar feedback, which effectively redistribute metals within the interstellar medium. Our results support the important role of stellar feedback in governing the chemo-structural evolution and disk formation of Milky Way-mass galaxies at the cosmic noon epoch.

astro-ph.GA↗

arXiv2025

The Dial-a-Ride Problem with Limited Pickups per Trip

The Dial-a-Ride Problem (DARP) is an optimization problem that involves determining optimal routes and schedules for several vehicles to pick up and deliver items at minimum cost. Motivated by real-world carpooling and crowdshipping scenarios, we introduce an additional constraint imposing a maximum number on the number of pickups per trip. This results in the Dial-a-Ride Problem with Limited Pickups per Trip (DARP-LPT). We apply a fragment-based method for DARP-LPT, where a fragment is a partial path. Specifically, we extend two formulations from Rist & Forbes (2021): the Fragment Flow Formulation (FFF) and the Fragment Assignment Formulation (FAF). We establish FFF's superiority over FAF, both from a theoretical as well as from a computational perspective. Furthermore, our results show that FFF and FAF significantly outperform traditional arc-based formulations in terms of solution quality and time. Additionally, compared to the two existing fragment sets, one with longer partial paths and another with shorter ones, our newly generated fragment sets perform better in terms of solution quality and time when fed into FFF.

econ.TH↗

arXiv2025

Info-Coevolution: An Efficient Framework for Data Model Coevolution

Machine learning relies heavily on data, yet the continuous growth of real-world data poses challenges for efficient dataset construction and training. A fundamental yet unsolved question is: given our current model and data, does a new data (sample/batch) need annotation/learning? Conventional approaches retain all available data, leading to non-optimal data and training efficiency. Active learning aims to reduce data redundancy by selecting a subset of samples to annotate, while it increases pipeline complexity and introduces bias. In this work, we propose Info-Coevolution, a novel framework that efficiently enables models and data to coevolve through online selective annotation with no bias. Leveraging task-specific models (and open-source models), it selectively annotates and integrates online and web data to improve datasets efficiently. For real-world datasets like ImageNet-1K, Info-Coevolution reduces annotation and training costs by 32\% without performance loss. It is able to automatically give the saving ratio without tuning the ratio. It can further reduce the annotation ratio to 50\% with semi-supervised learning. We also explore retrieval-based dataset enhancement using unlabeled open-source data. Code is available at https://github.com/NUS-HPC-AI-Lab/Info-Coevolution/.

cs.LG↗

arXiv2025

VS-Singer: Vision-Guided Stereo Singing Voice Synthesis with Consistency Schrödinger Bridge

To explore the potential advantages of utilizing spatial cues from images for generating stereo singing voices with room reverberation, we introduce VS-Singer, a vision-guided model designed to produce stereo singing voices with room reverberation from scene images. VS-Singer comprises three modules: firstly, a modal interaction network integrates spatial features into text encoding to create a linguistic representation enriched with spatial information. Secondly, the decoder employs a consistency Schrödinger bridge to facilitate one-step sample generation. Moreover, we utilize the SFE module to improve the consistency of audio-visual matching. To our knowledge, this study is the first to combine stereo singing voice synthesis with visual acoustic matching within a unified framework. Experimental results demonstrate that VS-Singer can effectively generate stereo singing voices that align with the scene perspective in a single step.

cs.SD↗

arXiv2025

Drag-and-Drop LLMs: Zero-Shot Prompt-to-Weights

Modern Parameter-Efficient Fine-Tuning (PEFT) methods such as low-rank adaptation (LoRA) reduce the cost of customizing large language models (LLMs), yet still require a separate optimization run for every downstream dataset. We introduce \textbf{Drag-and-Drop LLMs (\textit{DnD})}, a prompt-conditioned parameter generator that eliminates per-task training by mapping a handful of unlabeled task prompts directly to LoRA weight updates. A lightweight text encoder distills each prompt batch into condition embeddings, which are then transformed by a cascaded hyper-convolutional decoder into the full set of LoRA matrices. Once trained in a diverse collection of prompt-checkpoint pairs, DnD produces task-specific parameters in seconds, yielding i) up to \textbf{12,000$\times$} lower overhead than full fine-tuning, ii) average gains up to \textbf{30\%} in performance over the strongest training LoRAs on unseen common-sense reasoning, math, coding, and multimodal benchmarks, and iii) robust cross-domain generalization despite never seeing the target data or labels. Our results demonstrate that prompt-conditioned parameter generation is a viable alternative to gradient-based adaptation for rapidly specializing LLMs. Our project is available at \href{https://jerryliang24.github.io/DnD}{https://jerryliang24.github.io/DnD}.

cs.LG↗