arXiv subjects

Zeqing Wang

Publications and source records attributed to Zeqing Wang.

36 records · Page 2Linked to original sources

arXiv2025

SAMCL: Empowering SAM to Continually Learn from Dynamic Domains with Extreme Storage Efficiency

Segment Anything Model (SAM) struggles in open-world scenarios with diverse domains. In such settings, naive fine-tuning with a well-designed learning module is inadequate and often causes catastrophic forgetting issue when learning incrementally. To address this issue, we propose a novel continual learning (CL) method for SAM, termed SAMCL. Rather than relying on a fixed learning module, our method decomposes incremental knowledge into separate modules and trains a selector to choose the appropriate one during inference. However, this intuitive design introduces two key challenges: ensuring effective module learning and selection, and managing storage as tasks accumulate. To tackle these, we introduce two components: AugModule and Module Selector. AugModule reduces the storage of the popular LoRA learning module by sharing parameters across layers while maintaining accuracy. It also employs heatmaps-generated from point prompts-to further enhance domain adaptation with minimal additional cost. Module Selector leverages the observation that SAM's embeddings can effectively distinguish domains, enabling high selection accuracy by training on low-consumed embeddings instead of raw images. Experiments show that SAMCL outperforms state-of-the-art methods, achieving only 0.19% forgetting and at least 2.5% gain on unseen domains. Each AugModule requires just 0.233 MB, reducing storage by at least 24.3% over other fine-tuning approaches. The buffer storage for Module Selector is further reduced by up to 256$\times$.

cs.CV↗

arXiv2025

Vision Bridge Transformer at Scale

We introduce Vision Bridge Transformer (ViBT), a large-scale instantiation of Brownian Bridge Models designed for conditional generation. Unlike traditional diffusion models that transform noise into data, Bridge Models directly model the trajectory between inputs and outputs, creating an efficient data-to-data translation paradigm. By scaling these models to 20B and 1.3B parameters, we demonstrate their effectiveness for image and video translation tasks. To support this scale, we adopt a Transformer architecture and propose a variance-stabilized velocity-matching objective for robust training. Together, these advances highlight the power of scaling Bridge Models for instruction-based image editing and complex video translation.

cs.CV↗

arXiv2025

Universal Manipulation of Quantum Synchronization in Spin Oscillator Networks

Quantum synchronization (QS) in open many-body systems offers a promising route for controlling collective quantum dynamics, yet existing manipulation schemes often rely on dissipation engineering, which distorts limit cycles, lacks scalability, and is strongly system-dependent. Here, we propose a universal and scalable method for continuously tuning QS from maximal synchronization under isotropic interactions to complete synchronization blockade (QSB) under fully anisotropic coupling in spin oscillator networks. Our approach preserves intrinsic limit cycles and applies to both few-body and macroscopic systems. We analytically show that QS arises solely from spin flip-flop processes and their higher-order correlations, while anisotropic interactions induce non-synchronizing coherence. A geometric QS measure reveals a macroscopic QSB effect in the thermodynamic limit. The proposed mechanism is experimentally feasible using XYZ interactions and optical pumping, and provides a general framework for programmable synchronization control in complex quantum networks and dynamical phases of matter.

quant-ph↗

arXiv2025

SparseD: Sparse Attention for Diffusion Language Models

While diffusion language models (DLMs) offer a promising alternative to autoregressive models (ARs), existing open-source DLMs suffer from high inference latency. This bottleneck is mainly due to the attention's quadratic complexity with respect to context length in computing all query-key pairs. Intuitively, to reduce this complexity, a natural strategy is to restrict attention to sparse patterns that retain only the most relevant connections. Such approaches are well-established in ARs, where attention follows fixed and clearly defined sparse patterns. However, in DLMs, we observe distinct sparsity behaviors: (1) attention patterns vary across heads, (2) attention patterns in each head remain highly similar across denoising steps, and (3) early denoising steps are critical for generation. These findings render sparse attention methods designed for ARs largely incompatible with DLMs, as they fail to capture head-specific structures and risk degrading generation when applied in early denoising steps. To address these challenges, we propose SparseD, a novel sparse attention method for DLMs. Leveraging the observations, SparseD only requires pre-computing head-specific sparse patterns one time, and reuses them across all steps. This prevents recomputing sparse patterns at each denoising step. Meanwhile, SparseD uses full attention in the early steps, then switches to sparse attention later to maintain generation quality. Together, these establish SparseD as a practical and efficient solution for deploying DLMs in long-context applications. Experimental results demonstrate that SparseD achieves lossless acceleration, delivering up to $1.50\times$ speedup over FlashAttention at a 64k context length with 1,024 denoising steps.

cs.CL↗

arXiv2025

Single Particle Spectroscopies of $p$-wave and $d$-wave Interacting Bose Gases in Normal Phase

Motivated by experiments of interacting quantum gases across high partial wave resonance, we investigated the thermodynamic properties as well as single particle spectrums of Bose gases in normal phase for different interaction strengths both for $p$-wave and $d$-wave interactions. The equation of state, contact density, momentum distributions, and self-energies of single particle Green's functions are obtained in the spirit of ladder diagram approximations. Radio frequency (RF) spectrum, as an important experimental approach to detect Feshbach molecules or interaction effect, is calculated at different temperatures. A reversed temperature dependence on the BEC side and BCS side is identified for both $p$-wave and $d$-wave interactions. An estimation for the signal of RF spectra under typical experimental conditions is also provided.

cond-mat.quant-gas↗

arXiv2025

Minute-Long Videos with Dual Parallelisms

Diffusion Transformer (DiT)-based video diffusion models generate high-quality videos at scale but incur prohibitive processing latency and memory costs for long videos. To address this, we propose a novel distributed inference strategy, termed DualParal. The core idea is that, instead of generating an entire video on a single GPU, we parallelize both temporal frames and model layers across GPUs. However, a naive implementation of this division faces a key limitation: since diffusion models require synchronized noise levels across frames, this implementation leads to the serialization of original parallelisms. We leverage a block-wise denoising scheme to handle this. Namely, we process a sequence of frame blocks through the pipeline with progressively decreasing noise levels. Each GPU handles a specific block and layer subset while passing previous results to the next GPU, enabling asynchronous computation and communication. To further optimize performance, we incorporate two key enhancements. Firstly, a feature cache is implemented on each GPU to store and reuse features from the prior block as context, minimizing inter-GPU communication and redundant computation. Secondly, we employ a coordinated noise initialization strategy, ensuring globally consistent temporal dynamics by sharing initial noise patterns across GPUs without extra resource costs. Together, these enable fast, artifact-free, and infinitely long video generation. Applied to the latest diffusion transformer video generator, our method efficiently produces 1,025-frame videos with up to 6.54$\times$ lower latency and 1.48$\times$ lower memory cost on 8$\times$RTX 4090 GPUs.

cs.CV↗

arXiv2025

TimeCausality: Evaluating the Causal Ability in Time Dimension for Vision Language Models

Reasoning about temporal causality, particularly irreversible transformations of objects governed by real-world knowledge (e.g., fruit decay and human aging), is a fundamental aspect of human visual understanding. Unlike temporal perception based on simple event sequences, this form of reasoning requires a deeper comprehension of how object states change over time. Although the current powerful Vision-Language Models (VLMs) have demonstrated impressive performance on a wide range of downstream tasks, their capacity to reason about temporal causality remains underexplored. To address this gap, we introduce \textbf{TimeCausality}, a novel benchmark specifically designed to evaluate the causal reasoning ability of VLMs in the temporal dimension. Based on our TimeCausality, we find that while the current SOTA open-source VLMs have achieved performance levels comparable to closed-source models like GPT-4o on various standard visual question answering tasks, they fall significantly behind on our benchmark compared with their closed-source competitors. Furthermore, even GPT-4o exhibits a marked drop in performance on TimeCausality compared to its results on other tasks. These findings underscore the critical need to incorporate temporal causality into the evaluation and development of VLMs, and they highlight an important challenge for the open-source VLM community moving forward. Code and Data are available at \href{https://github.com/Zeqing-Wang/TimeCausality }{TimeCausality}.

cs.CV↗

arXiv2025

LightCL: Compact Continual Learning with Low Memory Footprint For Edge Device

Continual learning (CL) is a technique that enables neural networks to constantly adapt to their dynamic surroundings. Despite being overlooked for a long time, this technology can considerably address the customized needs of users in edge devices. Actually, most CL methods require huge resource consumption by the training behavior to acquire generalizability among all tasks for delaying forgetting regardless of edge scenarios. Therefore, this paper proposes a compact algorithm called LightCL, which evaluates and compresses the redundancy of already generalized components in structures of the neural network. Specifically, we consider two factors of generalizability, learning plasticity and memory stability, and design metrics of both to quantitatively assess generalizability of neural networks during CL. This evaluation shows that generalizability of different layers in a neural network exhibits a significant variation. Thus, we $\textit{Maintain Generalizability}$ by freezing generalized parts without the resource-intensive training process and $\textit{Memorize Feature Patterns}$ by stabilizing feature extracting of previous tasks to enhance generalizability for less-generalized parts with a little extra memory, which is far less than the reduction by freezing. Experiments illustrate that LightCL outperforms other state-of-the-art methods and reduces at most $\textbf{6.16$\times$}$ memory footprint. We also verify the effectiveness of LightCL on the edge device.

cs.LG↗

arXiv2025

Emergent Continuous Time Crystal in Dissipative Quantum Spin System without Driving

Time crystals are a nonequilibrium phase of matter that extend fundamental spontaneous symmetry breaking into the temporal dimension, typically requiring external driving for their realization. Here, we explore the nonequilibrium phase diagram of a two-dimensional dissipative Heisenberg spin system without external coherent or incoherent driving. Through numerical analysis of spin dynamics, we identify nonstationary steady states, some of which are limit cycles with persistent periodic oscillations, while others exhibit chaotic, aperiodic behavior. The emergence of limit cycle steady states breaks the continuous time-translation symmetry of this time-independent many-body system, classifying them as continuous time crystals. We further validate these oscillatory behaviors by testing their stability against local perturbations and assess the robustness of the emergent continuous time crystals by introducing isotropic Gaussian white noise. This work provides insights into the intricate interplay between the dissipation and spin interaction, and opens possibilities for realizing dissipation-induced, heating-immune time crystals.

quant-ph↗

arXiv2025

Tracking-Aware Deformation Field Estimation for Non-rigid 3D Reconstruction in Robotic Surgeries

Minimally invasive procedures have been advanced rapidly by the robotic laparoscopic surgery. The latter greatly assists surgeons in sophisticated and precise operations with reduced invasiveness. Nevertheless, it is still safety critical to be aware of even the least tissue deformation during instrument-tissue interactions, especially in 3D space. To address this, recent works rely on NeRF to render 2D videos from different perspectives and eliminate occlusions. However, most of the methods fail to predict the accurate 3D shapes and associated deformation estimates robustly. Differently, we propose Tracking-Aware Deformation Field (TADF), a novel framework which reconstructs the 3D mesh along with the 3D tissue deformation simultaneously. It first tracks the key points of soft tissue by a foundation vision model, providing an accurate 2D deformation field. Then, the 2D deformation field is smoothly incorporated with a neural implicit reconstruction network to obtain tissue deformation in the 3D space. Finally, we experimentally demonstrate that the proposed method provides more accurate deformation estimation compared with other 3D neural reconstruction methods in two public datasets.

cs.CV↗

arXiv2025

A Stepwise Distillation Learning Strategy for Non-differentiable Visual Programming Frameworks on Visual Reasoning Tasks

Recently, Visual Programming (VProg) has emerged as a significant framework for visual reasoning (VR) tasks due to its interpretability and cross-task generality. However, even with invoking powerful pre-trained Vision-Language models (VLMs) as visual sub-modules, the performance of VProg on specific VR tasks is markedly inferior compared to well-trained task-specific networks. Although invoking task-specific models can further enhance the performance of VProg on specific VR tasks, it greatly diminishes the cross-task generalization ability of VProg. Besides, the non-differentiable nature of VProg prevents direct fine-tuning on specific VR tasks for further performance improvement. Attempt to address these issues, we propose SDVP, a Stepwise Distillation learning strategy for non-differentiable VPorg across various VR tasks. Specifically, our SDVP stepwise distills the capabilities of existing, well-trained small task-specific models for decomposed visual sub-tasks in VProg into the much larger VLMs invoked by corresponding visual sub-modules. Besides, distilling the knowledge of little-size task-specific models into pre-trained larger VLMs rather than replacing them helps keep the cross-task abilities of VProgs. Extensive and comprehensive experimental results on different VProg frameworks demonstrate that our SDVP obtains significant performance gains on specific VR benchmarks, i.e., GQA (+2.4\%) and NLVRv2 (+6.2\%) for VisProg and GQA (+6.5\%) and NLVRv2 (+4.0\%) for ViperGPT, and also maintains a promising performance for VProg on unseen and previous VR tasks.

cs.CV↗

arXiv2025

Towards Top-Down Reasoning: An Explainable Multi-Agent Approach for Visual Question Answering

Recently, to comprehensively improve Vision Language Models (VLMs) for Visual Question Answering (VQA), several methods have been proposed to further reinforce the inference capabilities of VLMs to independently tackle VQA tasks rather than some methods that only utilize VLMs as aids to Large Language Models (LLMs). However, these methods ignore the rich common-sense knowledge inside the given VQA image sampled from the real world. Thus, they cannot fully use the powerful VLM for the given VQA question to achieve optimal performance. Attempt to overcome this limitation and inspired by the human top-down reasoning process, i.e., systematically exploring relevant issues to derive a comprehensive answer, this work introduces a novel, explainable multi-agent collaboration framework by leveraging the expansive knowledge of Large Language Models (LLMs) to enhance the capabilities of VLMs themselves. Specifically, our framework comprises three agents, i.e., Responder, Seeker, and Integrator, to collaboratively answer the given VQA question by seeking its relevant issues and generating the final answer in such a top-down reasoning process. The VLM-based Responder agent generates the answer candidates for the question and responds to other relevant issues. The Seeker agent, primarily based on LLM, identifies relevant issues related to the question to inform the Responder agent and constructs a Multi-View Knowledge Base (MVKB) for the given visual scene by leveraging the build-in world knowledge of LLM. The Integrator agent combines knowledge from the Seeker agent and the Responder agent to produce the final VQA answer. Extensive and comprehensive evaluations on diverse VQA datasets with a variety of VLMs demonstrate the superior performance and interpretability of our framework over the baseline method in the zero-shot setting without extra training cost.

cs.CV↗

arXiv2024

Is this Generated Person Existed in Real-world? Fine-grained Detecting and Calibrating Abnormal Human-body

Recent improvements in visual synthesis have significantly enhanced the depiction of generated human photos, which are pivotal due to their wide applicability and demand. Nonetheless, the existing text-to-image or text-to-video models often generate low-quality human photos that might differ considerably from real-world body structures, referred to as "abnormal human bodies". Such abnormalities, typically deemed unacceptable, pose considerable challenges in the detection and repair of them within human photos. These challenges require precise abnormality recognition capabilities, which entail pinpointing both the location and the abnormality type. Intuitively, Visual Language Models (VLMs) that have obtained remarkable performance on various visual tasks are quite suitable for this task. However, their performance on abnormality detection in human photos is quite poor. Hence, it is quite important to highlight this task for the research community. In this paper, we first introduce a simple yet challenging task, i.e., \textbf{F}ine-grained \textbf{H}uman-body \textbf{A}bnormality \textbf{D}etection \textbf{(FHAD)}, and construct two high-quality datasets for evaluation. Then, we propose a meticulous framework, named HumanCalibrator, which identifies and repairs abnormalities in human body structures while preserving the other content. Experiments indicate that our HumanCalibrator achieves high accuracy in abnormality detection and accomplishes an increase in visual comparisons while preserving the other visual content.

cs.CV↗

arXiv2024

Rabi Oscillation of High Partial Wave Interacting Atoms in Deep Optical Lattice

Motivated by the recent experiment of p-wave interacting $^{40}\mathrm{K}$ atom gases in a deep optical lattice, we investigated the Rabi oscillation for any partial wave interacting quantum gases in a deep optical lattice. We first review the solution for two particles interacting by any partial wave in a harmonic trap by using pseudopotential. We generalize the model used in recent work to any partial wave cases, repeated some of the theoretic and experiment results for p-wave interacting $^{40}\mathrm{K}$ atoms in a deep optical lattice and show the results for the d-wave case. Our results may be useful for future experiments and may stimulate further theoretic investigation.

cond-mat.quant-gas↗

arXiv2024

Dissipation induced extended-localized transition

Mobility edge (ME), representing the critical energy that distinguishes between extended and localized states, is a key concept in understanding the transition between extended (metallic) and localized (insulating) states in disordered and quasiperiodic systems. Here we explore the impact of dissipation on a quasiperiodic system featuring MEs by calculating steady-state density matrix and analyzing quench dynamics with sudden introduction of dissipation, and demonstrate that dissipation can lead the system into specific states predominantly characterized by either extended or localized states, irrespective of the initial state. Our results establish the use of dissipation as a new avenue for inducing transitions between extended and localized states, and for manipulating dynamic behaviors of particles.

cond-mat.dis-nn↗

arXiv2023

Mimic: Speaking Style Disentanglement for Speech-Driven 3D Facial Animation

Speech-driven 3D facial animation aims to synthesize vivid facial animations that accurately synchronize with speech and match the unique speaking style. However, existing works primarily focus on achieving precise lip synchronization while neglecting to model the subject-specific speaking style, often resulting in unrealistic facial animations. To the best of our knowledge, this work makes the first attempt to explore the coupled information between the speaking style and the semantic content in facial motions. Specifically, we introduce an innovative speaking style disentanglement method, which enables arbitrary-subject speaking style encoding and leads to a more realistic synthesis of speech-driven facial animations. Subsequently, we propose a novel framework called \textbf{Mimic} to learn disentangled representations of the speaking style and content from facial motions by building two latent spaces for style and content, respectively. Moreover, to facilitate disentangled representation learning, we introduce four well-designed constraints: an auxiliary style classifier, an auxiliary inverse classifier, a content contrastive loss, and a pair of latent cycle losses, which can effectively contribute to the construction of the identity-related style space and semantic-related content space. Extensive qualitative and quantitative experiments conducted on three publicly available datasets demonstrate that our approach outperforms state-of-the-art methods and is capable of capturing diverse speaking styles for speech-driven 3D facial animation. The source code and supplementary video are publicly available at: https://zeqing-wang.github.io/Mimic/

cs.CV↗

arXiv2023

Accelerating Relaxation Dynamics in Open Quantum System with Liouvillian Skin Effect

We investigate a non-Hermitian model featuring non-reciprocal gradient hoppings. Through an in-depth analysis of the Liouvillian spectrum and dynamics, we confirm the emergence of the Liouvillian skin effect resulting from the non-reciprocal nature of hoppings in this model. Furthermore, we observe that the presence of gradient hopping strength leads to an accelerated relaxation time for the system. Through numerical investigations of the Liouvillian gap, relaxation time, and steady-state localization length, we discover that the relaxation time in this model cannot be explained by the currently established relationship associated with the Liouvillian skin effect. This discrepancy highlights the need for further exploration and theoretical advancements to fully comprehend the intricate mechanisms underlying quantum relaxation processes. Motivated by these findings, we propose a theoretical approach to realize this non-Hermitian model in an atomic system with a sideband structure by employing adiabatic elimination technique. These results contribute to our deeper comprehension of quantum relaxation dynamics and provide theoretical backing for the development of techniques aimed at controlling quantum relaxation processes.

quant-ph↗

arXiv2023

Absence of Correlations in Dissipative Interacting Qubits: a No-Go Theorem

Exact solutions of model problems are elusive but potent tools for understanding many body interacting systems. We study a system of dissipative qubits with the Heisenberg interaction and obtain, for qubits under a certain condition, an exact steady state solution to the Lindblad master equation describing its dynamics. The physical content of such a solution is a remarkable no-go theorem, which states that for qubits possessing identical ratios of the damping and gain rates, no correlation can be established between them in the steady state. Two consequences of this theorem are discussed in the context of quantum synchronization of qubits. The first is a complete blockade of quantum synchronization of qubits under the aforementioned condition, an effect reminiscent of, but having a much broader scope than, that found in dissipated Kerr-anharmonic oscillators. The second, and a more important consequence is the possibility of reducing a complex all-to-all qubit network to a much simpler one-to-all network by engineering the dissipation. Such a reduction is desired because it provides an effective tool to optimize the quantum synchronization of a complex qubit network. Finally, we propose two concrete experimental schemes to implement our model and to test our predictions.

quant-ph↗