arXiv subjects

Shiwei Zhang

Publications and source records attributed to Shiwei Zhang.

At least 55 records · Page 3Linked to original sources

arXiv2025

A neutral-atom Hubbard quantum simulator in the cryogenic regime

Ultracold fermionic atoms in optical lattices offer pristine realizations of Hubbard models, which are fundamental to modern condensed matter physics. Despite significant advancements, the accessible temperatures in these optical lattice material analogs are still too high to address many open problems. Here, we demonstrate a several-fold reduction in temperature, bringing large-scale quantum simulations of the Hubbard model into an entirely new regime. This is accomplished by transforming a low entropy product state into strongly-correlated states of interest via dynamic control of the model parameters, which is extremely challenging to simulate classically. At half filling, the long-range antiferromagnetic order is close to saturated, leading to a temperature of $T/t=0.05_{-0.05}^{0.06}$ based on comparisons to numerically exact simulations. Doped away from half-filling, it is exceedingly challenging to realize systematically accurate and predictive numerical simulations. Importantly, we are able to use quantum simulation to identify a new pathway for achieving similarly low temperatures with doping. This is confirmed by comparing short-range spin correlations to state-of-the-art, but approximate, constrained-path auxiliary field quantum Monte Carlo simulations. Compared to the cuprates, the reported temperatures correspond to a reduction from far above to below room temperature, where physics such as the pseudogap and stripe phases may be expected. Our work opens the door to quantum simulations that solve open questions in material science, develop synergies with numerical methods and theoretical studies, and lead to discoveries of new physics.

cond-mat.quant-gas↗

arXiv2025

Exploiting Discriminative Codebook Prior for Autoregressive Image Generation

Advanced discrete token-based autoregressive image generation systems first tokenize images into sequences of token indices with a codebook, and then model these sequences in an autoregressive paradigm. While autoregressive generative models are trained only on index values, the prior encoded in the codebook, which contains rich token similarity information, is not exploited. Recent studies have attempted to incorporate this prior by performing naive k-means clustering on the tokens, helping to facilitate the training of generative models with a reduced codebook. However, we reveal that k-means clustering performs poorly in the codebook feature space due to inherent issues, including token space disparity and centroid distance inaccuracy. In this work, we propose the Discriminative Codebook Prior Extractor (DCPE) as an alternative to k-means clustering for more effectively mining and utilizing the token similarity information embedded in the codebook. DCPE replaces the commonly used centroid-based distance, which is found to be unsuitable and inaccurate for the token feature space, with a more reasonable instance-based distance. Using an agglomerative merging technique, it further addresses the token space disparity issue by avoiding splitting high-density regions and aggregating low-density ones. Extensive experiments demonstrate that DCPE is plug-and-play and integrates seamlessly with existing codebook prior-based paradigms. With the discriminative prior extracted, DCPE accelerates the training of autoregressive models by 42% on LlamaGen-B and improves final FID and IS performance.

cs.CV↗

arXiv2025

Single-Particle Dispersion and Density of States of the Half-Filled 2D Hubbard Model

Implementing an improved method for analytic continuation and working with imaginary-time correlation functions computed using quantum Monte Carlo simulations, we resolve the single-particle dispersion relation and the density of states (DOS) of the two-dimensional Hubbard model at half-filling. At intermediate interactions of $U/t = 4,6$, we find quadratic dispersion around the gap minimum at wave-vectors $\mathbf{k} = (\pm π/2, \pm π/2)$ (the $Σ$ points). We find saddle points at $\mathbf{k} = (\pm π,0),(0,\pm π)$ (the X points) where the dispersion is approximately quartic, leading to a sharp DOS maximum above the almost flat ledge arising from the states close to $Σ$. The fraction of quasiparticle states within the ledge is $n_{\rm ledge} \approx 0.15$. Upon doping away from half-filling, within the rigid-band approximation, these results support Fermi pockets around the $Σ$ points, with states around the X points becoming filled only at doping fractions $x \ge n_{\rm ledge}$. The high density of states away from the $Σ$ gap edge may be an important clue for a finite minimum doping level for superconductivity and other instabilities of doped Mott insulators.

cond-mat.str-el↗

arXiv2025

Spatiotemporal Order and Parametric Instabilities from First-Principles

Shaping crystal structure with light is an enduring goal of physics and materials engineering. Here we present calculations in candidate materials selected by symmetry that allow light-induced spatiotemporal parametric instabilities. We demonstrate a theoretical framework that includes a complete symmetry analysis of phonon modes that contribute to parametric instabilities across all non-centrosymmetric point groups, a detailed survey of the materials landscape and finally the computation of nonlinear couplings from first principles. We then showcase detailed results for chiral crystals, ferroelectrics, and layered van der Waals materials. Our results pave the way towards realizing designer time-crystalline order in quantum materials, detectable with time-resolved diffractive probes.

cond-mat.mtrl-sci↗

arXiv2025

FreeScale: Unleashing the Resolution of Diffusion Models via Tuning-Free Scale Fusion

Visual diffusion models achieve remarkable progress, yet they are typically trained at limited resolutions due to the lack of high-resolution data and constrained computation resources, hampering their ability to generate high-fidelity images or videos at higher resolutions. Recent efforts have explored tuning-free strategies to exhibit the untapped potential higher-resolution visual generation of pre-trained models. However, these methods are still prone to producing low-quality visual content with repetitive patterns. The key obstacle lies in the inevitable increase in high-frequency information when the model generates visual content exceeding its training resolution, leading to undesirable repetitive patterns deriving from the accumulated errors. To tackle this challenge, we propose FreeScale, a tuning-free inference paradigm to enable higher-resolution visual generation via scale fusion. Specifically, FreeScale processes information from different receptive scales and then fuses it by extracting desired frequency components. Extensive experiments validate the superiority of our paradigm in extending the capabilities of higher-resolution visual generation for both image and video models. Notably, compared with previous best-performing methods, FreeScale unlocks the 8k-resolution text-to-image generation for the first time.

cs.CV↗

arXiv2025

Implementing advanced trial wave functions in fermion quantum Monte Carlo via stochastic sampling

We introduce an efficient approach to implement correlated many-body trial wave functions in auxiliary-field quantum Monte Carlo (AFQMC). To control the sign/phase problem in AFQMC, a constraint is derived from an exact gauge condition but is typically imposed approximately through a trial wave function or trial density matrix, whose quality can affect the accuracy of the method. Furthermore, the trial wave function can also affect the efficiency through importance sampling. The most natural form of the trial wave function has been single Slater determinants or their linear combinations. More sophisticated forms, for example, with the inclusion of a Jastrow factor or other explicit correlations, have been challenging to use and their implementation is often assumed to require a quantum computer. In this work, we demonstrate that a large class of correlated wave functions, written in the general form of multi-dimensional integrals over hidden or auxiliary variables times Slater determinants, can be implemented as trial wave function by coupling the random walkers to a generalized Metropolis sampling. We discuss the fidelity of AFQMC with stochastically sampled trial wave functions, which are relevant to both quantum and classical algorithms. We illustrate the method and show that an efficient implementation can be achieved which preserves the low-polynomial computational scaling of AFQMC. We test our method in molecules under bond stretching and in transition metal diatomics. Significant improvements are seen in both accuracy and efficiency over typical trial wave functions, and the method yields total ground-state energies systematically within chemical accuracy. The method can be useful for incorporating other advanced wave functions, for example, neural quantum state wave functions optimized from machine learning techniques, or for other forms of fermion quantum Monte Carlo.

cond-mat.str-el↗

arXiv2025

Replace Anyone in Videos

The field of controllable human-centric video generation has witnessed remarkable progress, particularly with the advent of diffusion models. However, achieving precise and localized control over human motion in videos, such as replacing or inserting individuals while preserving desired motion patterns, still remains a formidable challenge. In this work, we present the ReplaceAnyone framework, which focuses on localized human replacement and insertion featuring intricate backgrounds. Specifically, we formulate this task as an image-conditioned video inpainting paradigm with pose guidance, utilizing a unified end-to-end video diffusion architecture that facilitates image-conditioned video inpainting within masked regions. To prevent shape leakage and enable granular local control, we introduce diverse mask forms involving both regular and irregular shapes. Furthermore, we implement an enriched visual guidance mechanism to enhance appearance alignment, a hybrid inpainting encoder to further preserve the detailed background information in the masked video, and a two-phase optimization methodology to simplify the training difficulty. ReplaceAnyone enables seamless replacement or insertion of characters while maintaining the desired pose motion and reference appearance within a single framework. Extensive experimental results demonstrate the effectiveness of our method in generating realistic and coherent video content. The proposed ReplaceAnyone can be seamlessly applied not only to traditional 3D-UNet base models but also to DiT-based video models such as Wan2.1. The code will be available at https://github.com/ali-vilab/UniAnimate-DiT.

cs.CV↗

arXiv2025

Wan: Open and Advanced Large-Scale Video Generative Models

This report presents Wan, a comprehensive and open suite of video foundation models designed to push the boundaries of video generation. Built upon the mainstream diffusion transformer paradigm, Wan achieves significant advancements in generative capabilities through a series of innovations, including our novel VAE, scalable pre-training strategies, large-scale data curation, and automated evaluation metrics. These contributions collectively enhance the model's performance and versatility. Specifically, Wan is characterized by four key features: Leading Performance: The 14B model of Wan, trained on a vast dataset comprising billions of images and videos, demonstrates the scaling laws of video generation with respect to both data and model size. It consistently outperforms the existing open-source models as well as state-of-the-art commercial solutions across multiple internal and external benchmarks, demonstrating a clear and significant performance superiority. Comprehensiveness: Wan offers two capable models, i.e., 1.3B and 14B parameters, for efficiency and effectiveness respectively. It also covers multiple downstream applications, including image-to-video, instruction-guided video editing, and personal video generation, encompassing up to eight tasks. Consumer-Grade Efficiency: The 1.3B model demonstrates exceptional resource efficiency, requiring only 8.19 GB VRAM, making it compatible with a wide range of consumer-grade GPUs. Openness: We open-source the entire series of Wan, including source code and all models, with the goal of fostering the growth of the video generation community. This openness seeks to significantly expand the creative possibilities of video production in the industry and provide academia with high-quality video foundation models. All the code and models are available at https://github.com/Wan-Video/Wan2.1.

cs.CV↗

arXiv2025

Taming Consistency Distillation for Accelerated Human Image Animation

Recent advancements in human image animation have been propelled by video diffusion models, yet their reliance on numerous iterative denoising steps results in high inference costs and slow speeds. An intuitive solution involves adopting consistency models, which serve as an effective acceleration paradigm through consistency distillation. However, simply employing this strategy in human image animation often leads to quality decline, including visual blurring, motion degradation, and facial distortion, particularly in dynamic regions. In this paper, we propose the DanceLCM approach complemented by several enhancements to improve visual quality and motion continuity at low-step regime: (1) segmented consistency distillation with an auxiliary light-weight head to incorporate supervision from real video latents, mitigating cumulative errors resulting from single full-trajectory generation; (2) a motion-focused loss to centre on motion regions, and explicit injection of facial fidelity features to improve face authenticity. Extensive qualitative and quantitative experiments demonstrate that DanceLCM achieves results comparable to state-of-the-art video diffusion models with a mere 2-4 inference steps, significantly reducing the inference burden without compromising video quality. The code and models will be made publicly available.

cs.CV↗

arXiv2025

UniAnimate-DiT: Human Image Animation with Large-Scale Video Diffusion Transformer

This report presents UniAnimate-DiT, an advanced project that leverages the cutting-edge and powerful capabilities of the open-source Wan2.1 model for consistent human image animation. Specifically, to preserve the robust generative capabilities of the original Wan2.1 model, we implement Low-Rank Adaptation (LoRA) technique to fine-tune a minimal set of parameters, significantly reducing training memory overhead. A lightweight pose encoder consisting of multiple stacked 3D convolutional layers is designed to encode motion information of driving poses. Furthermore, we adopt a simple concatenation operation to integrate the reference appearance into the model and incorporate the pose information of the reference image for enhanced pose alignment. Experimental results show that our approach achieves visually appearing and temporally consistent high-fidelity animations. Trained on 480p (832x480) videos, UniAnimate-DiT demonstrates strong generalization capabilities to seamlessly upscale to 720P (1280x720) during inference. The training and inference code is publicly available at https://github.com/ali-vilab/UniAnimate-DiT.

cs.CV↗

arXiv2025

Timestep Embedding Tells: It's Time to Cache for Video Diffusion Model

As a fundamental backbone for video generation, diffusion models are challenged by low inference speed due to the sequential nature of denoising. Previous methods speed up the models by caching and reusing model outputs at uniformly selected timesteps. However, such a strategy neglects the fact that differences among model outputs are not uniform across timesteps, which hinders selecting the appropriate model outputs to cache, leading to a poor balance between inference efficiency and visual quality. In this study, we introduce Timestep Embedding Aware Cache (TeaCache), a training-free caching approach that estimates and leverages the fluctuating differences among model outputs across timesteps. Rather than directly using the time-consuming model outputs, TeaCache focuses on model inputs, which have a strong correlation with the modeloutputs while incurring negligible computational cost. TeaCache first modulates the noisy inputs using the timestep embeddings to ensure their differences better approximating those of model outputs. TeaCache then introduces a rescaling strategy to refine the estimated differences and utilizes them to indicate output caching. Experiments show that TeaCache achieves up to 4.41x acceleration over Open-Sora-Plan with negligible (-0.07% Vbench score) degradation of visual quality.

cs.CV↗

arXiv2025

PersonalVideo: High ID-Fidelity Video Customization without Dynamic and Semantic Degradation

The current text-to-video (T2V) generation has made significant progress in synthesizing realistic general videos, but it is still under-explored in identity-specific human video generation with customized ID images. The key challenge lies in maintaining high ID fidelity consistently while preserving the original motion dynamic and semantic following after the identity injection. Current video identity customization methods mainly rely on reconstructing given identity images on text-to-image models, which have a divergent distribution with the T2V model. This process introduces a tuning-inference gap, leading to dynamic and semantic degradation. To tackle this problem, we propose a novel framework, dubbed $\textbf{PersonalVideo}$, that applies a mixture of reward supervision on synthesized videos instead of the simple reconstruction objective on images. Specifically, we first incorporate identity consistency reward to effectively inject the reference's identity without the tuning-inference gap. Then we propose a novel semantic consistency reward to align the semantic distribution of the generated videos with the original T2V model, which preserves its dynamic and semantic following capability during the identity injection. With the non-reconstructive reward training, we further employ simulated prompt augmentation to reduce overfitting by supervising generated results in more semantic scenarios, gaining good robustness even with only a single reference image. Extensive experiments demonstrate our method's superiority in delivering high identity faithfulness while preserving the inherent video generation qualities of the original T2V model, outshining prior methods.

cs.CV↗

arXiv2025

DreamRelation: Relation-Centric Video Customization

Relational video customization refers to the creation of personalized videos that depict user-specified relations between two subjects, a crucial task for comprehending real-world visual content. While existing methods can personalize subject appearances and motions, they still struggle with complex relational video customization, where precise relational modeling and high generalization across subject categories are essential. The primary challenge arises from the intricate spatial arrangements, layout variations, and nuanced temporal dynamics inherent in relations; consequently, current models tend to overemphasize irrelevant visual details rather than capturing meaningful interactions. To address these challenges, we propose DreamRelation, a novel approach that personalizes relations through a small set of exemplar videos, leveraging two key components: Relational Decoupling Learning and Relational Dynamics Enhancement. First, in Relational Decoupling Learning, we disentangle relations from subject appearances using relation LoRA triplet and hybrid mask training strategy, ensuring better generalization across diverse relationships. Furthermore, we determine the optimal design of relation LoRA triplet by analyzing the distinct roles of the query, key, and value features within MM-DiT's attention mechanism, making DreamRelation the first relational video generation framework with explainable components. Second, in Relational Dynamics Enhancement, we introduce space-time relational contrastive loss, which prioritizes relational dynamics while minimizing the reliance on detailed subject appearances. Extensive experiments demonstrate that DreamRelation outperforms state-of-the-art methods in relational video customization. Code and models will be made publicly available.

cs.CV↗

arXiv2025

Phaseless auxiliary-field quantum Monte Carlo method for cavity-QED matter systems

We present a generalization of the phaseless auxiliary-field quantum Monte Carlo (AFQMC) method to cavity quantum-electrodynamical (QED) matter systems. The method can be formulated in both the Coulomb and the dipole gauge. We verify its accuracy by benchmarking calculations on a set of small molecules against full configuration interaction and state-of-the-art QED coupled cluster (QED-CCSD) calculations. Our results show that (i) gauge invariance can be achieved within correlation-consistent Gaussian basis sets, (ii) the accuracy of QED-CCSD can be enhanced significantly by adding the standard perturbative triples correction without light-matter coupling, and (iii) there is a straightforward way to evaluate the differential expression for the photon occupation number that works in any gauge. The high accuracy and favorable computational scaling of our AFQMC approach will enable a broad range of applications. Besides polaritonic chemistry, the method opens a way to simulate extended QED matter systems.

physics.chem-ph↗

arXiv2025

SimpleVQA: Multimodal Factuality Evaluation for Multimodal Large Language Models

The increasing application of multi-modal large language models (MLLMs) across various sectors have spotlighted the essence of their output reliability and accuracy, particularly their ability to produce content grounded in factual information (e.g. common and domain-specific knowledge). In this work, we introduce SimpleVQA, the first comprehensive multi-modal benchmark to evaluate the factuality ability of MLLMs to answer natural language short questions. SimpleVQA is characterized by six key features: it covers multiple tasks and multiple scenarios, ensures high quality and challenging queries, maintains static and timeless reference answers, and is straightforward to evaluate. Our approach involves categorizing visual question-answering items into 9 different tasks around objective events or common knowledge and situating these within 9 topics. Rigorous quality control processes are implemented to guarantee high-quality, concise, and clear answers, facilitating evaluation with minimal variance via an LLM-as-a-judge scoring system. Using SimpleVQA, we perform a comprehensive assessment of leading 18 MLLMs and 8 text-only LLMs, delving into their image comprehension and text generation abilities by identifying and analyzing error cases.

cs.CL↗

arXiv2025

Critical gate distance for Wigner crystallization in the two-dimensional electron gas

We report on the properties of the two-dimensional electron gas in a dual-gate geometry, using quantum Monte Carlo methods to obtain aspects of the phase diagram as a function of electron density and gate distance. We identify the critical gate distance below which the Wigner crystal phase disappears. For larger gate distances, the system undergoes a re-entrant transition from crystal to liquid at sufficiently low density. We also present preliminary evidence for a fully polarized ferromagnetic liquid state at low electron density and intermediate gate distances. The quantum Monte Carlo results are compared with simpler approximate methods, which are shown to be semi-quantitatively reliable for determining key features of the phase diagram. These methods are then used to obtain the phase boundary between the Wigner crystal and liquid in the single-gate geometry.

cond-mat.str-el↗

arXiv2024

The light-matter correlation energy functional of the cavity-coupled two-dimensional electron gas via quantum Monte Carlo simulations

We perform extensive simulations of the two-dimensional cavity-coupled electron gas in a modulating potential as a minimal model for cavity quantum materials. These simulations are enabled by a newly developed quantum-electrodynamical (QED) auxiliary-field quantum Monte Carlo method. We present a procedure to greatly reduce finite-size effects in such calculations. Based on our results, we show that a modified version of weak-coupling perturbation theory is remarkably accurate for a large parameter region. We further provide a simple parameterization of the light-matter correlation energy as a functional of the cavity parameters and the electronic density. These results provide a numerical foundation for the development of the QED density functional theory, which was previously reliant on analytical approximations, to allow quantitative modeling of a wide range of systems with light-matter coupling.

cond-mat.str-el↗

arXiv2024

Animate-X: Universal Character Image Animation with Enhanced Motion Representation

Character image animation, which generates high-quality videos from a reference image and target pose sequence, has seen significant progress in recent years. However, most existing methods only apply to human figures, which usually do not generalize well on anthropomorphic characters commonly used in industries like gaming and entertainment. Our in-depth analysis suggests to attribute this limitation to their insufficient modeling of motion, which is unable to comprehend the movement pattern of the driving video, thus imposing a pose sequence rigidly onto the target character. To this end, this paper proposes Animate-X, a universal animation framework based on LDM for various character types (collectively named X), including anthropomorphic characters. To enhance motion representation, we introduce the Pose Indicator, which captures comprehensive motion pattern from the driving video through both implicit and explicit manner. The former leverages CLIP visual features of a driving video to extract its gist of motion, like the overall movement pattern and temporal relations among motions, while the latter strengthens the generalization of LDM by simulating possible inputs in advance that may arise during inference. Moreover, we introduce a new Animated Anthropomorphic Benchmark (A^2Bench) to evaluate the performance of Animate-X on universal and widely applicable animation images. Extensive experiments demonstrate the superiority and effectiveness of Animate-X compared to state-of-the-art methods.

cs.CV↗