arXiv subjects

Qing Yan

Publications and source records attributed to Qing Yan.

At least 19 recordsLinked to original sources

arXiv2026

ActWorld: From Explorable to Interactive World Model via Action-Aware Memory

Interactive world models aim to simulate environment dynamics under real-time user actions. However, their action vocabulary is largely confined to navigation: most actions correspond to motion (e.g., walk, turn, look around), while interaction with objects in the scene (e.g., pick up plates, open doors, or trigger physical responses) is either absent, restricted to game domains, or relegated to prompt-to-full-video scenarios. The resulting worlds are visually explorable but not truly actionable. In this work, we present ActWorld, an interactive world model that extends prior navigation-centric generators to support mid-rollout object interaction within a chunk-autoregressive framework. We argue that the navigation-interaction gap stems from two bottlenecks. First, a data bottleneck: the lack of human-object interaction data with accurate, dense labels. Second, a memory bottleneck: recency-biased history compression in existing world models discards the event-transition frames that causally determine subsequent object states, leading to an action-forgetting pathology. On the data side, we construct a 100K interaction video dataset, each annotated with per-chunk captions via chain-of-thought reasoning. On the model side, we introduce a hierarchical action-aware memory design that routes history compression by interaction importance, complemented by a persistent memory bank that maintains event-update and object-identity tokens across long rollouts. Experiments show that ActWorld supports both flexible navigation and rich object interaction within a single model, substantially improving interaction fidelity over navigation-only baselines without sacrificing viewpoint control. Project page is available at https://interactwm.github.io/ActWorld.

cs.CV↗

arXiv2026

Scouting By Reward: VLM-TO-IRL-Driven Player Selection For Esports

Traditional esports scouting workflows rely heavily on manual video review and aggregate performance metrics, which often fail to capture the nuanced decision-making patterns necessary to determine if a prospect fits a specific tactical archetype. To address this, we reframe style-based player evaluation in esports as an Inverse Reinforcement Learning (IRL) problem. In this paper, we introduce a novel player selection framework that learns professional-specific reward functions from logged gameplay demonstrations, allowing organizations to rank candidates by their stylistic alignment with a target star player. Our proposed architecture utilizes a multimodal, two-branch intake: one branch encodes structured state-action trajectories derived from high-resolution in-game telemetry, while the second encodes temporally aligned tactical pseudo-commentary generated by Vision-Language Models (VLMs) from broadcast footage. These representations are fused and evaluated via a Generative Adversarial Imitation Learning (GAIL) objective, where a discriminator learns to capture the unique mechanical and tactical signatures of elite professionals. By transitioning from generic skill estimation to scouting "by reward," this framework provides a scalable, workflow-aware digital twin system that enables data-driven roster construction and targeted talent discovery across massive candidate pools.

cs.LG↗

arXiv2026

ThinkRL-Edit: Thinking in Reinforcement Learning for Reasoning-Centric Image Editing

Instruction-driven image editing with unified multimodal generative models has advanced rapidly, yet their underlying visual reasoning remains limited, leading to suboptimal performance on reasoning-centric edits. Reinforcement learning (RL) has been investigated for improving the quality of image editing, but it faces three key challenges: (1) limited reasoning exploration confined to denoising stochasticity, (2) biased reward fusion, and (3) unstable VLM-based instruction rewards. In this work, we propose ThinkRL-Edit, a reasoning-centric RL framework that decouples visual reasoning from image synthesis and expands reasoning exploration beyond denoising. To the end, we introduce Chain-of-Thought (CoT)-based reasoning sampling with planning and reflection stages prior to generation in online sampling, compelling the model to explore multiple semantic hypotheses and validate their plausibility before committing to a visual outcome. To avoid the failures of weighted aggregation, we propose an unbiased chain preference grouping strategy across multiple reward dimensions. Moreover, we replace interval-based VLM scores with a binary checklist, yielding more precise, lower-variance, and interpretable rewards for complex reasoning. Experiments show our method significantly outperforms prior work on reasoning-centric image editing, producing instruction-faithful, visually coherent, and semantically grounded edits.

cs.CV↗

arXiv2025

InstructMoLE: Instruction-Guided Mixture of Low-rank Experts for Multi-Conditional Image Generation

Parameter-Efficient Fine-Tuning of Diffusion Transformers (DiTs) for diverse, multi-conditional tasks often suffers from task interference when using monolithic adapters like LoRA. The Mixture of Low-rank Experts (MoLE) architecture offers a modular solution, but its potential is usually limited by routing policies that operate at a token level. Such local routing can conflict with the global nature of user instructions, leading to artifacts like spatial fragmentation and semantic drift in complex image generation tasks. To address these limitations, we introduce InstructMoLE, a novel framework that employs an Instruction-Guided Mixture of Low-Rank Experts. Instead of per-token routing, InstructMoLE utilizes a global routing signal, Instruction-Guided Routing (IGR), derived from the user's comprehensive instruction. This ensures that a single, coherently chosen expert council is applied uniformly across all input tokens, preserving the global semantics and structural integrity of the generation process. To complement this, we introduce an output-space orthogonality loss, which promotes expert functional diversity and mitigates representational collapse. Extensive experiments demonstrate that InstructMoLE significantly outperforms existing LoRA adapters and MoLE variants across challenging multi-conditional generation benchmarks. Our work presents a robust and generalizable framework for instruction-driven fine-tuning of generative models, enabling superior compositional control and fidelity to user intent.

cs.CV↗

arXiv2025

StoryMem: Multi-shot Long Video Storytelling with Memory

Visual storytelling requires generating multi-shot videos with cinematic quality and long-range consistency. Inspired by human memory, we propose StoryMem, a paradigm that reformulates long-form video storytelling as iterative shot synthesis conditioned on explicit visual memory, transforming pre-trained single-shot video diffusion models into multi-shot storytellers. This is achieved by a novel Memory-to-Video (M2V) design, which maintains a compact and dynamically updated memory bank of keyframes from historical generated shots. The stored memory is then injected into single-shot video diffusion models via latent concatenation and negative RoPE shifts with only LoRA fine-tuning. A semantic keyframe selection strategy, together with aesthetic preference filtering, further ensures informative and stable memory throughout generation. Moreover, the proposed framework naturally accommodates smooth shot transitions and customized story generation applications. To facilitate evaluation, we introduce ST-Bench, a diverse benchmark for multi-shot video storytelling. Extensive experiments demonstrate that StoryMem achieves superior cross-shot consistency over previous methods while preserving high aesthetic quality and prompt adherence, marking a significant step toward coherent minute-long video storytelling.

cs.CV↗

arXiv2025

GUIDES: Guidance Using Instructor-Distilled Embeddings for Pre-trained Robot Policy Enhancement

Pre-trained robot policies serve as the foundation of many validated robotic systems, which encapsulate extensive embodied knowledge. However, they often lack the semantic awareness characteristic of foundation models, and replacing them entirely is impractical in many situations due to high costs and the loss of accumulated knowledge. To address this gap, we introduce GUIDES, a lightweight framework that augments pre-trained policies with semantic guidance from foundation models without requiring architectural redesign. GUIDES employs a fine-tuned vision-language model (Instructor) to generate contextual instructions, which are encoded by an auxiliary module into guidance embeddings. These embeddings are injected into the policy's latent space, allowing the legacy model to adapt to this new semantic input through brief, targeted fine-tuning. For inference-time robustness, a large language model-based Reflector monitors the Instructor's confidence and, when confidence is low, initiates a reasoning loop that analyzes execution history, retrieves relevant examples, and augments the VLM's context to refine subsequent actions. Extensive validation in the RoboCasa simulation environment across diverse policy architectures shows consistent and substantial improvements in task success rates. Real-world deployment on a UR5 robot further demonstrates that GUIDES enhances motion precision for critical sub-tasks such as grasping. Overall, GUIDES offers a practical and resource-efficient pathway to upgrade, rather than replace, validated robot policies.

cs.RO↗

arXiv2025

The fate of disorder in twisted bilayer graphene near the magic angle

In disordered lattices, itinerant electrons typically undergo Anderson localization due to random phase interference, which suppresses their motion. By contrast, in flat-band systems where electrons are intrinsically localized owing to their vanishing group velocity, the role of disorder remains elusive. Twisted bilayer graphene (TBG) at the magic angle $\sim 1.1^\circ$ provides a representative flat-band platform to investigate this problem. Here, we perform an atomistic tight-binding quantum transport calculation on the interplay between disorder and flat-bands in TBG devices. This non-phenomenological approach provides direct evidence that moderate disorder enhances conductance, whereas stronger disorder restores localization, revealing a disorder-driven delocalization-to-localization transport behavior. The underlying physical mechanism is understood by an effective inter-moir{\'e} tunneling strength via spectral flow analysis of a disordered TBG cylinder. Moreover, by comparing magic-angle and large-angle TBG, we demonstrate qualitatively distinct disorder responses tied to the presence of flat-bands. Our quantitative results highlight the unconventional role of disorder in flat-band moir{\'e} materials and offer insights into the observation of the fractional quantum anomalous Hall effect in disordered moir{\'e} systems.

cond-mat.mes-hall↗

arXiv2025

Learning Joint ID-Textual Representation for ID-Preserving Image Synthesis

We propose a novel framework for ID-preserving generation using a multi-modal encoding strategy rather than injecting identity features via adapters into pre-trained models. Our method treats identity and text as a unified conditioning input. To achieve this, we introduce FaceCLIP, a multi-modal encoder that learns a joint embedding space for both identity and textual semantics. Given a reference face and a text prompt, FaceCLIP produces a unified representation that encodes both identity and text, which conditions a base diffusion model to generate images that are identity-consistent and text-aligned. We also present a multi-modal alignment algorithm to train FaceCLIP, using a loss that aligns its joint representation with face, text, and image embedding spaces. We then build FaceCLIP-SDXL, an ID-preserving image synthesis pipeline by integrating FaceCLIP with Stable Diffusion XL (SDXL). Compared to prior methods, FaceCLIP-SDXL enables photorealistic portrait generation with better identity preservation and textual relevance. Extensive experiments demonstrate its quantitative and qualitative superiority.

cs.CV↗

arXiv2025

Flux Already Knows -- Activating Subject-Driven Image Generation without Training

We propose a simple yet effective zero-shot framework for subject-driven image generation using a vanilla Flux model. By framing the task as grid-based image completion and simply replicating the subject image(s) in a mosaic layout, we activate strong identity-preserving capabilities without any additional data, training, or inference-time fine-tuning. This "free lunch" approach is further strengthened by a novel cascade attention design and meta prompting technique, boosting fidelity and versatility. Experimental results show that our method outperforms baselines across multiple key metrics in benchmarks and human preference studies, with trade-offs in certain aspects. Additionally, it supports diverse edits, including logo insertion, virtual try-on, and subject replacement or insertion. These results demonstrate that a pre-trained foundational text-to-image model can enable high-quality, resource-efficient subject-driven generation, opening new possibilities for lightweight customization in downstream applications.

cs.CV↗

arXiv2025

iMedImage Technical Report

Background: Chromosome karyotype analysis is crucial for diagnosing hereditary diseases, yet detecting structural abnormalities remains challenging. While AI has shown promise in medical imaging, its effectiveness varies across modalities. Leveraging advances in Foundation Models that integrate multimodal medical imaging for robust feature extraction and accurate diagnosis, we developed iMedImage, an end-to-end model for general medical image recognition, demonstrating strong performance across multiple imaging tasks, including chromosome abnormality detection. Materials and Methods: We constructed a comprehensive medical image dataset encompassing multiple modalities from common medical domains, including chromosome, cell, pathology, ultrasound, X-ray, CT, and MRI images. Based on this dataset, we developed the iMedImage model, which incorporates the following key features: (1) a unified representation method for diverse modality inputs and medical imaging tasks; (2) multi-level (case-level, image-level, patch-level) image recognition capabilities enhanced by Chain of Thought (CoT) embedding and Mixture of Experts (MoE) strategies. Results: The test set comprised data from 12 institutions across six regions in China, covering three mainstream scanning devices, and included naturally distributed, unscreened abnormal cases. On this diverse dataset, the model achieved a fully automated chromosome analysis workflow, including segmentation, karyotyping, and abnormality detection, reaching a sensitivity of 92.75% and a specificity of 91.5%. Conclusion: We propose iMedImage, an end-to-end foundation model for medical image analysis, demonstrating its superior performance across various medical imaging tasks. iMedImage provides clinicians with a precise imaging analysis tool and contributes to improving diagnostic accuracy and disease screening.

cs.CV↗

arXiv2025

InfiniteYou: Flexible Photo Recrafting While Preserving Your Identity

Achieving flexible and high-fidelity identity-preserved image generation remains formidable, particularly with advanced Diffusion Transformers (DiTs) like FLUX. We introduce InfiniteYou (InfU), one of the earliest robust frameworks leveraging DiTs for this task. InfU addresses significant issues of existing methods, such as insufficient identity similarity, poor text-image alignment, and low generation quality and aesthetics. Central to InfU is InfuseNet, a component that injects identity features into the DiT base model via residual connections, enhancing identity similarity while maintaining generation capabilities. A multi-stage training strategy, including pretraining and supervised fine-tuning (SFT) with synthetic single-person-multiple-sample (SPMS) data, further improves text-image alignment, ameliorates image quality, and alleviates face copy-pasting. Extensive experiments demonstrate that InfU achieves state-of-the-art performance, surpassing existing baselines. In addition, the plug-and-play design of InfU ensures compatibility with various existing methods, offering a valuable contribution to the broader community.

cs.CV↗

arXiv2025

Coherent detection of the oscillating acoustoelectric effect in graphene

In recent years, surface acoustic waves (SAWs) have emerged as a novel technique for generating quasiparticle transport and band modulation in condensed matter systems. SAWs interact with adjacent materials through piezoelectric and strain fields, dragging carriers in the direction of wave propagation. Most studies on the acoustoelectric effect have focused on the collective directional motion of carriers, which generates a steady electric potential difference, while the oscillating component from dynamic spatial charge modulation has remained challenging to probe. In this work, we report the coherent detection of oscillating acoustoelectric effect in graphene. This is achieved through the coherent rectification of spatial-temporal charge oscillation with electromagnetic waves emitted by interdigital transducers. We systematically investigate the frequency and gate dependence of rectified signals and quantitatively probe the carrier redistribution dynamics driven by SAWs. The observation of oscillating acoustoelectric effect provides direct access to the dynamic spatial charge modulation induced by SAWs through transport experiments.

cond-mat.mes-hall↗

arXiv2024

Rules for dissipationless topotronics

Topological systems hosting gapless boundary states have attracted huge attention as promising components for next-generation information processing, attributed to their capacity for dissipationless electronics. Nevertheless, recent theoretical and experimental inquiries have revealed the emergence of energy dissipation in precisely quantized electrical transport. Here, we present a criterion for the realization of truly no-dissipation design, characterized as $N_{in}=N_{tunl}+N_{bs}$, where $N_{in}$, $N_{tunl}$, and $N_{bs}$ represent the number of modes participating in injecting, tunneling, and backscattering processes, respectively. The key lies in matching the number of injecting, tunneling and backscattering modes, ensuring the equilibrium among all engaged modes inside the device. Among all the topological materials, we advocate for the indispensability of Chern insulators exhibiting higher Chern numbers to achieve functional devices and uphold the no-dissipation rule simultaneously. Furthermore, we design the topological current divider and collector, evading dissipation upon fulfilling the established criterion. Our work paves the path for developing the prospective topotronics.

cond-mat.mes-hall↗

arXiv2024

COAP: Memory-Efficient Training with Correlation-Aware Gradient Projection

Training large-scale neural networks in vision, and multimodal domains demands substantial memory resources, primarily due to the storage of optimizer states. While LoRA, a popular parameter-efficient method, reduces memory usage, it often suffers from suboptimal performance due to the constraints of low-rank updates. Low-rank gradient projection methods (e.g., GaLore, Flora) reduce optimizer memory by projecting gradients and moment estimates into low-rank spaces via singular value decomposition or random projection. However, they fail to account for inter-projection correlation, causing performance degradation, and their projection strategies often incur high computational costs. In this paper, we present COAP (Correlation-Aware Gradient Projection), a memory-efficient method that minimizes computational overhead while maintaining training performance. Evaluated across various vision, language, and multimodal tasks, COAP outperforms existing methods in both training speed and model performance. For LLaMA-1B, it reduces optimizer memory by 61% with only 2% additional time cost, achieving the same PPL as AdamW. With 8-bit quantization, COAP cuts optimizer memory by 81% and achieves 4x speedup over GaLore for LLaVA-v1.5-7B fine-tuning, while delivering higher accuracy.

cs.LG↗

arXiv2024

ID-Patch: Robust ID Association for Group Photo Personalization

The ability to synthesize personalized group photos and specify the positions of each identity offers immense creative potential. While such imagery can be visually appealing, it presents significant challenges for existing technologies. A persistent issue is identity (ID) leakage, where injected facial features interfere with one another, resulting in low face resemblance, incorrect positioning, and visual artifacts. Existing methods suffer from limitations such as the reliance on segmentation models, increased runtime, or a high probability of ID leakage. To address these challenges, we propose ID-Patch, a novel method that provides robust association between identities and 2D positions. Our approach generates an ID patch and ID embeddings from the same facial features: the ID patch is positioned on the conditional image for precise spatial control, while the ID embeddings integrate with text embeddings to ensure high resemblance. Experimental results demonstrate that ID-Patch surpasses baseline methods across metrics, such as face ID resemblance, ID-position association accuracy, and generation efficiency. Project Page is: https://byteaigc.github.io/ID-Patch/

cs.CV↗

arXiv2024

MoMA: Multimodal LLM Adapter for Fast Personalized Image Generation

In this paper, we present MoMA: an open-vocabulary, training-free personalized image model that boasts flexible zero-shot capabilities. As foundational text-to-image models rapidly evolve, the demand for robust image-to-image translation grows. Addressing this need, MoMA specializes in subject-driven personalized image generation. Utilizing an open-source, Multimodal Large Language Model (MLLM), we train MoMA to serve a dual role as both a feature extractor and a generator. This approach effectively synergizes reference image and text prompt information to produce valuable image features, facilitating an image diffusion model. To better leverage the generated features, we further introduce a novel self-attention shortcut method that efficiently transfers image features to an image diffusion model, improving the resemblance of the target object in generated images. Remarkably, as a tuning-free plug-and-play module, our model requires only a single reference image and outperforms existing methods in generating images with high detail fidelity, enhanced identity-preservation and prompt faithfulness. Our work is open-source, thereby providing universal access to these advancements.

cs.CV↗

arXiv2023

MagicPose: Realistic Human Poses and Facial Expressions Retargeting with Identity-aware Diffusion

In this work, we propose MagicPose, a diffusion-based model for 2D human pose and facial expression retargeting. Specifically, given a reference image, we aim to generate a person's new images by controlling the poses and facial expressions while keeping the identity unchanged. To this end, we propose a two-stage training strategy to disentangle human motions and appearance (e.g., facial expressions, skin tone and dressing), consisting of (1) the pre-training of an appearance-control block and (2) learning appearance-disentangled pose control. Our novel design enables robust appearance control over generated human images, including body, facial attributes, and even background. By leveraging the prior knowledge of image diffusion models, MagicPose generalizes well to unseen human identities and complex poses without the need for additional fine-tuning. Moreover, the proposed model is easy to use and can be considered as a plug-in module/extension to Stable Diffusion. The code is available at: https://github.com/Boese0601/MagicDance

cs.CV↗

arXiv2023

Transport theory in non-Hermitian systems

Non-Hermitian systems have garnered significant attention due to the emergence of novel topology of complex spectra and skin modes. However, investigating transport phenomena in such systems faces obstacles stemming from the non-unitary nature of time evolution. Here, we establish the continuity equation for a general non-Hermitian Hamiltonian in the Schr\"odinger picture. It attributes the universal non-conservativity to the anti-commutation relationship between particle number and non-Hermitian terms. Our work derives a comprehensive current formula for non-Hermitian systems using Green's function, applicable to both time-dependent and steady-state responses. To demonstrate the validity of our approach, we calculate the local current in models with one-dimensional and two-dimensional settings, incorporating scattering potentials. The spatial distribution of local current highlights the widespread non-Hermitian phenomena, including skin modes, non-reciprocal quantum dots, and corner states. Our findings offer valuable insights for advancing theoretical and experimental research in the transport of non-Hermitian systems.

cond-mat.mes-hall↗