arXiv subjects

Zhen Shen

Publications and source records attributed to Zhen Shen.

At least 19 recordsLinked to original sources

arXiv2026

RASA: Disentangled Spatial-Motional Priors for Cross-Identity Character Animation

Cross-identity character animation aims to drive a target identity from a reference image to follow the motion of a source character from a driving video. The core challenge lies in the inherent entanglement of two capabilities: cross-identity spatial mapping (aligning position, scale, and skeletal proportions) and motion control (refining joint articulation, volumetric consistency, and view coherence). We introduce Reference-Aware Structural Alignment (RASA), a framework that disentangles spatial mapping from motion control by injecting structured priors into a Diffusion Transformer (DiT). Our approach has two stages. First, a Spatial Prior Calibrator (SPC) fuses reference identity with driving pose to generate a spatially grounded initial noise latent, ensuring correct positioning, scaling, and alignment with the driving skeleton. Second, an Inherent Motional Guider (IMG) encodes shape-agnostic SMPL articulation parameters into a semantic motion vector beyond appearance-biased 2D keypoints. Injected into intermediate DiT layers, this vector complements the base pose condition for anatomically consistent articulation and view-aware volumetric refinement. We curate CIM-Bench, a high-quality benchmark with rigorous curation, for evaluation. Extensive experiments show RASA significantly outperforms state-of-the-art methods in motion fidelity and visual quality. Our work establishes a new paradigm showing disentangled spatial and motional priors are key to robust character animation. Project page: https://hidream.ai.github.io/RASA/

cs.CV↗

arXiv2026

HiDream-O1-Image: A Natively Unified Image Generative Foundation Model with Pixel-level Unified Transformer

The evolution of visual generative models has long been constrained by fragmented architectures relying on disjoint text encoders and external VAEs. In this report, we present HiDream-O1-Image, a natively unified generative foundation model via pixel-space Diffusion Transformer, that pioneers a paradigm shift from modular architectures to an end-to-end in-context visual generation engine. By mapping raw image pixels, text tokens, and task-specific conditions into a single shared token space, HiDream-O1-Image achieves a structural unification of multimodal inputs within an Unified Transformer (UiT) architecture. This native encoding paradigm eliminates the need for separate VAEs or disjoint pre-trained text encoders, allowing the model to treat diverse generation and editing tasks as a consistent in-context reasoning process. Extensive experiments show that HiDream-O1-Image excels across various generation tasks, including text-to-image generation, instruction-based editing, and subject-driven personalization. Notably, with only 8B parameters, HiDream-O1-Image (8B) achieves performance parity with or even surpasses established state-of-the-art models with significantly larger parameters (e.g., 27B Qwen-Image). Crucially, to validate the immense scalability of this paradigm, we successfully scale the architecture up to over 200B parameters. Experimental results demonstrate that this massive-scale version HiDream-O1-Image-Pro (200B+) unlocks unprecedented generative capabilities and superior performance, establishing new state-of-the-art benchmarks. Ultimately, HiDream-O1-Image highlights the immense potential of natively unified architectures and charts a highly scalable path toward next-generation multimodal AI.

cs.CV↗

arXiv2026

Grokking From Abstraction to Intelligence

Grokking in modular arithmetic has established itself as the quintessential fruit fly experiment, serving as a critical domain for investigating the mechanistic origins of model generalization. Despite its significance, existing research remains narrowly focused on specific local circuits or optimization tuning, largely overlooking the global structural evolution that fundamentally drives this phenomenon. We propose that grokking originates from a spontaneous simplification of internal model structures governed by the principle of parsimony. We integrate causal, spectral, and algorithmic complexity measures alongside Singular Learning Theory to reveal that the transition from memorization to generalization corresponds to the physical collapse of redundant manifolds and deep information compression, offering a novel perspective for understanding the mechanisms of model overfitting and generalization.

cs.AI↗

arXiv2026

Unveiling the Reasoning Process of Large Language Models

Large language models often reason beyond surface tokens, but the internal stage at which token-level information becomes abstract relational structure remains unclear. We investigate this question by analyzing how attention heads and layers transform information during autoregressive reasoning. Across mathematical and symbolic reasoning tasks, we observe a consistent layer-wise division of labor: outer layers mainly preserve and route input-related features, whereas middle layers reorganize them into more transferable rule-level representations. This interpretation is supported by representation geometry: middle-layer states occupy lower-dimensional manifolds and show stronger alignment across disjoint vocabularies that instantiate the same symbolic rules. It is further supported by causal interventions: removing middle-layer components identified by our interaction-based criterion produces substantially larger downstream changes and accuracy drops than removing components from other regions or at random. Together, these results suggest that abstract reasoning is not uniformly distributed across transformer layers, but is preferentially formed in a middle-layer computation stage that converts token-level information into reusable relational structure.

cs.AI↗

arXiv2025

Nonclassical phonon pair

Quantum-correlated photon pairs are crucial resources for modern quantum information science. Similarly, the reliable generation of nonclassical phonon pairs is vital for advancing engineerable solid-state quantum devices and hybrid quantum networks based on phonons. Here, we present a novel approach to generate quantum-correlated phonon pairs in a suspended silicon microstructure initialized in its motional ground state. By simultaneously implementing red- and blue-detuned laser pulses, equivalent high-order optomechanical nonlinearity -- specifically, an effective optomechanical four-wave mixing process -- is achieved for generating a nonclassical phonon pair, which is then read out via a subsequent red-detuned pulse. We demonstrate the nonclassical nature of the generated phonon pair through the violation of the Cauchy-Schwarz inequality. Our experimentally observed phonon pair violates the classical bound by more than 5 standard deviations and maintains a decoherence time of 132 ns. This work reveals novel quantum manipulation of phonon states enabled by equivalent high-order optomechanical nonlinearity within a pulse scheme and provides a valuable quantum resource for mechanical quantum computing.

quant-ph↗

arXiv2025

Wan-Animate: Unified Character Animation and Replacement with Holistic Replication

We introduce Wan-Animate, a unified framework for character animation and replacement. Given a character image and a reference video, Wan-Animate can animate the character by precisely replicating the expressions and movements of the character in the video to generate high-fidelity character videos. Alternatively, it can integrate the animated character into the reference video to replace the original character, replicating the scene's lighting and color tone to achieve seamless environmental integration. Wan-Animate is built upon the Wan model. To adapt it for character animation tasks, we employ a modified input paradigm to differentiate between reference conditions and regions for generation. This design unifies multiple tasks into a common symbolic representation. We use spatially-aligned skeleton signals to replicate body motion and implicit facial features extracted from source images to reenact expressions, enabling the generation of character videos with high controllability and expressiveness. Furthermore, to enhance environmental integration during character replacement, we develop an auxiliary Relighting LoRA. This module preserves the character's appearance consistency while applying the appropriate environmental lighting and color tone. Experimental results demonstrate that Wan-Animate achieves state-of-the-art performance. We are committed to open-sourcing the model weights and its source code.

cs.CV↗

arXiv2025

Wan-S2V: Audio-Driven Cinematic Video Generation

Current state-of-the-art (SOTA) methods for audio-driven character animation demonstrate promising performance for scenarios primarily involving speech and singing. However, they often fall short in more complex film and television productions, which demand sophisticated elements such as nuanced character interactions, realistic body movements, and dynamic camera work. To address this long-standing challenge of achieving film-level character animation, we propose an audio-driven model, which we refere to as Wan-S2V, built upon Wan. Our model achieves significantly enhanced expressiveness and fidelity in cinematic contexts compared to existing approaches. We conducted extensive experiments, benchmarking our method against cutting-edge models such as Hunyuan-Avatar and Omnihuman. The experimental results consistently demonstrate that our approach significantly outperforms these existing solutions. Additionally, we explore the versatility of our method through its applications in long-form video generation and precise video lip-sync editing.

cs.CV↗

arXiv2025

Photonic chip-based high-efficiency soliton microcombs via electroopitc-Kerr synergy

Temporal soliton mode-locking in coherently pumped microcavities provides a promising platform for miniaturized frequency comb systems. While significant progress has been made, achieving high conversion efficiency in such microcombs remains a critical challenge. Soliton generation through pulse pumping has emerged as an effective strategy to improve conversion efficiency. However, the on-chip integration of pulse generation with dissipative Kerr soliton (DKS) formation within the photonic chip has not yet been realized. In this work, we demonstrate a photonic chip-based soliton microcomb with high conversion efficiency, achieved by integrating on-chip pulse generation and DKS generation. The pulsed laser, fabricated on a lithium niobate-on-insulator (LNOI) platform, delivers a 35.5GHz repetition rate with broadly tunable center frequencies. By coupling these on-chip pulses to a silicon nitride microresonator, we achieve stable DKS generation with a pump-to-soliton conversion efficiency of 43.9% under steady-state conditions. This integrated architecture establishes a viable pathway toward chip-scale soliton microcombs with unprecedented efficiency, opening up new possibilities for optical communications, precision spectroscopy, and photonic sensing.

physics.optics↗

arXiv2025

Considerations on the Design of Transceivers for Ambient Internet of Things

The Ambient IoT (A-IoT) will introduce trillions of connections and enable low-cost battery-less devices. The A-IoT nodes can achieve low cost ($\sim\$ 0.1$ like RFID tag), sub-1mW average power consumption, $\leq 10$ kbps data rates, maintenance-free working for decades, cm-scale size, and support applications like supply chain and smart agriculture. The transceiver challenges in A-IoT focus on sub-mW receivers and crystal-less clock generation. The paper proposes an approximate low-IF receiver and carrier-auxiliary IF feedback LO synthesizer architecture for Type-B/C A-IoT devices, which tracks the RF carrier frequency and eliminates external crystals. The proposed receiver and LO generator are implemented using 55nm CMOS technology. After locking the LO calibration loop, the receiver sensitivity is better than -88 dBm. The proposed receiver architecture will promote zero-power devices for ubiquitous IoT connectivity, bridging digital and physical worlds.

eess.SY↗

arXiv2025

Learning-based Adaptive Safety-Critical Control With Evolving Unsafe Regions

Control barrier functions (CBFs) provide a principled framework for safety-critical control, but their construction typically requires an explicit and differentiable description of the safe or unsafe region. It becomes challenging for data-defined unsafe regions that may evolve over time. This paper proposes SafeLink, a data-driven CBF construction and adaptation method based on a cost-sensitive random vector functional link (RVFL) network. SafeLink introduces asymmetric misclassification costs to promote conservative unsafe-region representation while preserving a closed-form solution. We establish the Lipschitz continuity of the learned CBF and its derivatives, and derive sufficient conditions for conservative unsafe-region coverage and the corresponding interval-wise safety guarantees. Analytical updates are further developed for adjusting the misclassification cost and for incrementally adding or decrementally removing samples, avoiding full retraining when the unsafe region changes. Experiments on a two-link manipulator demonstrate that SafeLink rapidly adapts to evolving unsafe regions, enables collision-free target reaching, and achieves substantially lower update runtimes than baselines.

cs.RO↗

arXiv2025

Animate Anyone 2: High-Fidelity Character Image Animation with Environment Affordance

Recent character image animation methods based on diffusion models, such as Animate Anyone, have made significant progress in generating consistent and generalizable character animations. However, these approaches fail to produce reasonable associations between characters and their environments. To address this limitation, we introduce Animate Anyone 2, aiming to animate characters with environment affordance. Beyond extracting motion signals from source video, we additionally capture environmental representations as conditional inputs. The environment is formulated as the region with the exclusion of characters and our model generates characters to populate these regions while maintaining coherence with the environmental context. We propose a shape-agnostic mask strategy that more effectively characterizes the relationship between character and environment. Furthermore, to enhance the fidelity of object interactions, we leverage an object guider to extract features of interacting objects and employ spatial blending for feature injection. We also introduce a pose modulation strategy that enables the model to handle more diverse motion patterns. Experimental results demonstrate the superior performance of the proposed method.

cs.CV↗

arXiv2024

MLPHand: Real Time Multi-View 3D Hand Mesh Reconstruction via MLP Modeling

Multi-view hand mesh reconstruction is a critical task for applications in virtual reality and human-computer interaction, but it remains a formidable challenge. Although existing multi-view hand reconstruction methods achieve remarkable accuracy, they typically come with an intensive computational burden that hinders real-time inference. To this end, we propose MLPHand, a novel method designed for real-time multi-view single hand reconstruction. MLP Hand consists of two primary modules: (1) a lightweight MLP-based Skeleton2Mesh model that efficiently recovers hand meshes from hand skeletons, and (2) a multi-view geometry feature fusion prediction module that enhances the Skeleton2Mesh model with detailed geometric information from multiple views. Experiments on three widely used datasets demonstrate that MLPHand can reduce computational complexity by 90% while achieving comparable reconstruction accuracy to existing state-of-the-art baselines.

cs.CV↗

arXiv2023

Magnonic frequency comb in the magnomechanical resonator

An optical frequency comb is a spectrum of optical radiation which consists of evenly spaced and phase-coherent narrow spectral lines and is initially invented in laser for frequency metrology purposes. A direct analogue of frequency combs in the magnonic systems has not been demonstrated to date. In our experiment, we generate a new magnonic frequency comb in the resonator with giant mechanical oscillation through the magnomechanical interaction. We observe the magnonic frequency comb contains up to 20 comb lines, which are separated to the mechanical frequency of the 10.08 MHz. The thermal effect based on the strong pump power induces the cyclic oscillation of the magnon frequency shift, which leads to a periodic oscillation of the magnonic frequency comb. Moreover, we demonstrate the stabilization and control of the frequency spacing of the magnonic frequency comb via injection locking. Our work lays the groundwork of magnonic frequency combs for sensing and metrology.

physics.app-ph↗

arXiv2023

Compact Real-time Radiance Fields with Neural Codebook

Reconstructing neural radiance fields with explicit volumetric representations, demonstrated by Plenoxels, has shown remarkable advantages on training and rendering efficiency, while grid-based representations typically induce considerable overhead for storage and transmission. In this work, we present a simple and effective framework for pursuing compact radiance fields from the perspective of compression methodology. By exploiting intrinsic properties exhibiting in grid models, a non-uniform compression stem is developed to significantly reduce model complexity and a novel parameterized module, named Neural Codebook, is introduced for better encoding high-frequency details specific to per-scene models via a fast optimization. Our approach can achieve over 40 $\times$ reduction on grid model storage with competitive rendering quality. In addition, the method can achieve real-time rendering speed with 180 fps, realizing significant advantage on storage cost compared to real-time rendering methods.

cs.CV↗

arXiv2023

Atom-referenced on-chip soliton microcomb

For the applications of the frequency comb in microresonators, it is essential to obtain a fully frequency-stabilized microcomb laser source. Here, we demonstrate an atom-referenced stabilized soliton microcomb generation system based on the integrated microring resonator. The pump light around $1560.48\,\mathrm{nm}$ locked to an ultra-low-expansion (ULE) cavity, is frequency-doubled and referenced to the atomic transition of $^{87}\mathrm{Rb}$. The repetition rate of the soliton microcomb is injection-locked to an atomic-clock-stabilized radio frequency (RF) source, leading to mHz stabilization at $1$ seconds. As a result, all comb lines have been frequency-stabilized based on the atomic reference and could be determined with very high precision reaching $\sim18\,\mathrm{Hz}$ at 1 second, corresponding to the frequency stability of $9.5\times10^{-14}$. Our approach provides an integrated and fully stabilized microcomb experiment scheme with no requirement of $f-2f$ technique, which could be easily implemented and generalized to various photonic platforms, thus paving the way towards the portable and ultraprecise optical sources for high precision spectroscopy.

physics.optics↗

arXiv2023

4K-NeRF: High Fidelity Neural Radiance Fields at Ultra High Resolutions

In this paper, we present a novel and effective framework, named 4K-NeRF, to pursue high fidelity view synthesis on the challenging scenarios of ultra high resolutions, building on the methodology of neural radiance fields (NeRF). The rendering procedure of NeRF-based methods typically relies on a pixel-wise manner in which rays (or pixels) are treated independently on both training and inference phases, limiting its representational ability on describing subtle details, especially when lifting to a extremely high resolution. We address the issue by exploring ray correlation to enhance high-frequency details recovery. Particularly, we use the 3D-aware encoder to model geometric information effectively in a lower resolution space and recover fine details through the 3D-aware decoder, conditioned on ray features and depths estimated by the encoder. Joint training with patch-based sampling further facilitates our method incorporating the supervision from perception oriented regularization beyond pixel-wise loss. Benefiting from the use of geometry-aware local context, our method can significantly boost rendering quality on high-frequency details compared with modern NeRF methods, and achieve the state-of-the-art visual quality on 4K ultra-high-resolution scenarios. Code Available at \url{https://github.com/frozoul/4K-NeRF}

cs.CV↗

arXiv2022

Compressing Volumetric Radiance Fields to 1 MB

Approximating radiance fields with volumetric grids is one of promising directions for improving NeRF, represented by methods like Plenoxels and DVGO, which achieve super-fast training convergence and real-time rendering. However, these methods typically require a tremendous storage overhead, costing up to hundreds of megabytes of disk space and runtime memory for a single scene. We address this issue in this paper by introducing a simple yet effective framework, called vector quantized radiance fields (VQRF), for compressing these volume-grid-based radiance fields. We first present a robust and adaptive metric for estimating redundancy in grid models and performing voxel pruning by better exploring intermediate outputs of volumetric rendering. A trainable vector quantization is further proposed to improve the compactness of grid models. In combination with an efficient joint tuning strategy and post-processing, our method can achieve a compression ratio of 100$\times$ by reducing the overall model size to 1 MB with negligible loss on visual quality. Extensive experiments demonstrate that the proposed framework is capable of achieving unrivaled performance and well generalization across multiple methods with distinct volumetric structures, facilitating the wide use of volumetric radiance fields methods in real-world applications. Code Available at \url{https://github.com/AlgoHunt/VQRF}

cs.CV↗

arXiv2022

Streaming Radiance Fields for 3D Video Synthesis

We present an explicit-grid based method for efficiently reconstructing streaming radiance fields for novel view synthesis of real world dynamic scenes. Instead of training a single model that combines all the frames, we formulate the dynamic modeling problem with an incremental learning paradigm in which per-frame model difference is trained to complement the adaption of a base model on the current frame. By exploiting the simple yet effective tuning strategy with narrow bands, the proposed method realizes a feasible framework for handling video sequences on-the-fly with high training efficiency. The storage overhead induced by using explicit grid representations can be significantly reduced through the use of model difference based compression. We also introduce an efficient strategy to further accelerate model optimization for each frame. Experiments on challenging video sequences demonstrate that our approach is capable of achieving a training speed of 15 seconds per-frame with competitive rendering quality, which attains $1000 \times$ speedup over the state-of-the-art implicit methods. Code is available at https://github.com/AlgoHunt/StreamRF.

cs.CV↗