arXiv subjects

Lei Jin

Publications and source records attributed to Lei Jin.

At least 37 records · Page 2Linked to original sources

arXiv2026

Distilling Latent Manifolds: Resolution Extrapolation by Variational Autoencoders

Variational Autoencoder (VAE) encoders play a critical role in modern generative models, yet their computational cost often motivates the use of knowledge distillation or quantification to obtain compact alternatives. Existing studies typically believe that the model work better on the samples closed to their training data distribution than unseen data distribution. In this work, we report a counter-intuitive phenomenon in VAE encoder distillation: a compact encoder distilled only at low resolutions exhibits poor reconstruction performance at its native resolution, but achieves dramatically improved results when evaluated at higher, unseen input resolutions. Despite never being trained beyond $256^2$ resolution, the distilled encoder generalizes effectively to $512^2$ resolution inputs, partially inheriting the teacher model's resolution preference.We further analyze latent distributions across resolutions and find that higher-resolution inputs produce latent representations more closely aligned with the teacher's manifold. Through extensive experiments on ImageNet-256, we show that simple resolution remapping-upsampling inputs before encoding and downsampling reconstructions for evaluation-leads to substantial gains across PSNR, MSE, SSIM, LPIPS, and rFID metrics. These findings suggest that VAE encoder distillation learns resolution-consistent latent manifolds rather than resolution-specific pixel mappings. This also means that the high training cost on memory, time and high-resolution datasets are not necessary conditions for distilling a VAE with high-resolution image reconstruction capabilities. On low resolution datasets, the distillation model still could learn the detailed knowledge of the teacher model in high-resolution image reconstruction.

cs.CV↗

arXiv2026

Topological entropy, mean dimension, and weakly equivalent flows

In this paper, we mainly revisit a nice theory for topological entropy of weakly equivalent flows, which was originally investigated by Ohno in 1980. We will develop a new approach, being more straightforward and elementary than the measure-theoretic one provided by Ohno, to the theory for weak equivalence of flows, and as a novelty, we study both topological entropy and mean dimension with a highly unified process in relation to such objects. In particular, for weakly equivalent flows without fixed points we recover Ohno's theorem for topological entropy relation with a substantially different method, and moreover, carry out an analogue within the framework of mean dimension; while for weakly equivalent flows with fixed points, our technique refines the procedure suggested in Ohno's construction, and strengthens Ohno's example with a view towards topological complexity of dynamical systems. Essentially, our method is topological. For this purpose, we first introduce a modification to the definition of topological entropy, by relating a spanning set inside the state space to a finite set outside, which comes to be a tiny but key difference, via a map, and further, show that it leads eventually to the same value as the topological entropy. Although to an intermediate extent, the modified quantity generally does not coincide with the one appearing in the classical definition, it has some basic combinatorial feature, which not only applies flexibly to topological entropy, but also adapts directly to the mean dimension context. Using this alternative, we are allowed to re-establish and enrich Ohno's theory.

math.DS↗

arXiv2026

Emergence of a spin Hall topological Hall effect in the non-collinear phase of the ferrimagnetic insulator terbium-iron garnet

Magnetic compensation in rare-earth iron garnets (REIGs) offers a unique setting for which competing sublattice moments can give rise to non-collinear (canted) magnetic configurations, in which the sublattice magnetizations are not aligned with each other or with the external magnetic field. We show that this compensation regime can also host non-trivial magnetic textures. To explore this behavior, we investigated (111)-oriented epitaxial Tb$_3$Fe$_5$O$_{12}$/Pt heterostructures across the compensation temperature region using combined transverse magneto-transport and polar Kerr microscopy. Notably, we observe a topological Hall-like signal in the vicinity of the compensation temperature, a feature often interpreted as evidence for skyrmions in the absence of direct imaging. Here, in contrast, complementary Kerr microscopy reveals instead a non-collinear multidomain state which collapses outside the compensation regime, correlating directly with the appearance and disappearance of the spin Hall topological Hall effect (SH-THE) signal. These observations cannot be accounted for by a simple multi-anomalous-Hall-effect model, ruling out common artifacts as the origin, but indicate the presence of a topologically non-trivial contribution to the Hall response. These results establish strained REIG films as a tunable platform for exploring topological responses arising from compensation-driven non-collinear ferrimagnetic phases.

cond-mat.mtrl-sci↗

arXiv2026

WorldArena: A Unified Benchmark for Evaluating Perception and Functional Utility of Embodied World Models

While world models have emerged as a cornerstone of embodied intelligence by enabling agents to reason about environmental dynamics through action-conditioned prediction, their evaluation remains fragmented. Current evaluation of embodied world models has largely focused on perceptual fidelity (e.g., video generation quality), overlooking the functional utility of these models in downstream decision-making tasks. In this work, we introduce WorldArena, a unified benchmark designed to systematically evaluate embodied world models across both perceptual and functional dimensions. WorldArena assesses models through three dimensions: video perception quality, measured with 16 metrics across six sub-dimensions; embodied task functionality, which evaluates world models as data engines, policy evaluators, and action planners integrating with subjective human evaluation. Furthermore, we propose EWMScore, a holistic metric integrating multi-dimensional performance into a single interpretable index. Through extensive experiments on 14 representative models, we reveal a significant perception-functionality gap, showing that high visual quality does not necessarily translate into strong embodied task capability. WorldArena benchmark with the public leaderboard is released at https://world-arena.ai, providing a framework for tracking progress toward truly functional world models in embodied AI.

cs.CV↗

arXiv2026

DemoTuner: Automatic Performance Tuning for Database Management Systems Based on Demonstration Reinforcement Learning

The performance of modern DBMSs such as MySQL and PostgreSQL heavily depends on the configuration of performance-critical knobs. Manual tuning these knobs is laborious and inefficient due to the complex and high-dimensional nature of the configuration space. Among the automated tuning methods, reinforcement learning (RL)-based methods have recently sought to improve the DBMS knobs tuning process from several different perspectives. However, they still encounter challenges with slow convergence speed during offline training. In this paper, we mainly focus on how to leverage the valuable tuning hints contained in various textual documents such as DBMS manuals and web forums to improve the offline training of RL-based methods. To this end, we propose an efficient DBMS knobs tuning framework named DemoTuner via a novel LLM-assisted demonstration reinforcement learning method. Specifically, to comprehensively and accurately mine tuning hints from documents, we design a structured chain of thought prompt to employ LLMs to conduct a condition-aware tuning hints extraction task. To effectively integrate the mined tuning hints into RL agent training, we propose a hint-aware demonstration reinforcement learning algorithm HA-DDPGfD in DemoTuner. As far as we know, DemoTuner is the first work to introduce the demonstration reinforcement learning algorithm for DBMS knobs tuning. Experimental evaluations conducted on MySQL and PostgreSQL across various workloads demonstrate that DemoTuner achieves performance gains of up to 44.01% for MySQL and 39.95% for PostgreSQL over default configurations. Compared with three representative baseline methods, DemoTuner is able to further reduce the execution time by up to 10.03%, while always consuming the least online tuning cost. Additionally, DemoTuner also exhibits superior adaptability to application scenarios with unknown workloads.

cs.LG↗

arXiv2026

Loupe: A Generalizable and Adaptive Framework for Image Forgery Detection

The proliferation of generative models has raised serious concerns about visual content forgery. Existing deepfake detection methods primarily target either image-level classification or pixel-wise localization. While some achieve high accuracy, they often suffer from limited generalization across manipulation types or rely on complex architectures. In this paper, we propose Loupe, a lightweight yet effective framework for joint deepfake detection and localization. Loupe integrates a patch-aware classifier and a segmentation module with conditional queries, allowing simultaneous global authenticity classification and fine-grained mask prediction. To enhance robustness against distribution shifts of test set, Loupe introduces a pseudo-label-guided test-time adaptation mechanism by leveraging patch-level predictions to supervise the segmentation head. Extensive experiments on the DDL dataset demonstrate that Loupe achieves state-of-the-art performance, securing the first place in the IJCAI 2025 Deepfake Detection and Localization Challenge with an overall score of 0.846. Our results validate the effectiveness of the proposed patch-level fusion and conditional query design in improving both classification accuracy and spatial localization under diverse forgery patterns. The code is available at https://github.com/Kamichanw/Loupe.

cs.CV↗

arXiv2025

ERF-BA-TFD+: A Multimodal Model for Audio-Visual Deepfake Detection

Deepfake detection is a critical task in identifying manipulated multimedia content. In real-world scenarios, deepfake content can manifest across multiple modalities, including audio and video. To address this challenge, we present ERF-BA-TFD+, a novel multimodal deepfake detection model that combines enhanced receptive field (ERF) and audio-visual fusion. Our model processes both audio and video features simultaneously, leveraging their complementary information to improve detection accuracy and robustness. The key innovation of ERF-BA-TFD+ lies in its ability to model long-range dependencies within the audio-visual input, allowing it to better capture subtle discrepancies between real and fake content. In our experiments, we evaluate ERF-BA-TFD+ on the DDL-AV dataset, which consists of both segmented and full-length video clips. Unlike previous benchmarks, which focused primarily on isolated segments, the DDL-AV dataset allows us to assess the model's performance in a more comprehensive and realistic setting. Our method achieves state-of-the-art results on this dataset, outperforming existing techniques in terms of both accuracy and processing speed. The ERF-BA-TFD+ model demonstrated its effectiveness in the "Workshop on Deepfake Detection, Localization, and Interpretability," Track 2: Audio-Visual Detection and Localization (DDL-AV), and won first place in this competition.

cs.AI↗

arXiv2025

LongScape: Advancing Long-Horizon Embodied World Models with Context-Aware MoE

Video-based world models hold significant potential for generating high-quality embodied manipulation data. However, current video generation methods struggle to achieve stable long-horizon generation: classical diffusion-based approaches often suffer from temporal inconsistency and visual drift over multiple rollouts, while autoregressive methods tend to compromise on visual detail. To solve this, we introduce LongScape, a hybrid framework that adaptively combines intra-chunk diffusion denoising with inter-chunk autoregressive causal generation. Our core innovation is an action-guided, variable-length chunking mechanism that partitions video based on the semantic context of robotic actions. This ensures each chunk represents a complete, coherent action, enabling the model to flexibly generate diverse dynamics. We further introduce a Context-aware Mixture-of-Experts (CMoE) framework that adaptively activates specialized experts for each chunk during generation, guaranteeing high visual quality and seamless chunk transitions. Extensive experimental results demonstrate that our method achieves stable and consistent long-horizon generation over extended rollouts. Our code is available at: https://github.com/tsinghua-fib-lab/Longscape.

cs.CV↗

arXiv2025

Butter: Frequency Consistency and Hierarchical Fusion for Autonomous Driving Object Detection

Hierarchical feature representations play a pivotal role in computer vision, particularly in object detection for autonomous driving. Multi-level semantic understanding is crucial for accurately identifying pedestrians, vehicles, and traffic signs in dynamic environments. However, existing architectures, such as YOLO and DETR, struggle to maintain feature consistency across different scales while balancing detection precision and computational efficiency. To address these challenges, we propose Butter, a novel object detection framework designed to enhance hierarchical feature representations for improving detection robustness. Specifically, Butter introduces two key innovations: Frequency-Adaptive Feature Consistency Enhancement (FAFCE) Component, which refines multi-scale feature consistency by leveraging adaptive frequency filtering to enhance structural and boundary precision, and Progressive Hierarchical Feature Fusion Network (PHFFNet) Module, which progressively integrates multi-level features to mitigate semantic gaps and strengthen hierarchical feature learning. Through extensive experiments on BDD100K, KITTI, and Cityscapes, Butter demonstrates superior feature representation capabilities, leading to notable improvements in detection accuracy while reducing model complexity. By focusing on hierarchical feature refinement and integration, Butter provides an advanced approach to object detection that achieves a balance between accuracy, deployability, and computational efficiency in real-time autonomous driving scenarios. Our model and implementation are publicly available at https://github.com/Aveiro-Lin/Butter, facilitating further research and validation within the autonomous driving community.

cs.CV↗

arXiv2025

RoboScape: Physics-informed Embodied World Model

World models have become indispensable tools for embodied intelligence, serving as powerful simulators capable of generating realistic robotic videos while addressing critical data scarcity challenges. However, current embodied world models exhibit limited physical awareness, particularly in modeling 3D geometry and motion dynamics, resulting in unrealistic video generation for contact-rich robotic scenarios. In this paper, we present RoboScape, a unified physics-informed world model that jointly learns RGB video generation and physics knowledge within an integrated framework. We introduce two key physics-informed joint training tasks: temporal depth prediction that enhances 3D geometric consistency in video rendering, and keypoint dynamics learning that implicitly encodes physical properties (e.g., object shape and material characteristics) while improving complex motion modeling. Extensive experiments demonstrate that RoboScape generates videos with superior visual fidelity and physical plausibility across diverse robotic scenarios. We further validate its practical utility through downstream applications including robotic policy training with generated data and policy evaluation. Our work provides new insights for building efficient physics-informed world models to advance embodied intelligence research. The code is available at: https://github.com/tsinghua-fib-lab/RoboScape.

cs.CV↗

arXiv2025

Infinite topological entropy, positive mean dimension, and factors of subshifts

We study dynamical systems with the property that all the nontrivial factors have infinite topological entropy (or, positive mean dimension). We establish an ``if and only if'' condition for this property among a typical class of dynamical systems, the subshifts of block type in the Hilbert cube. This in particular leads to a large class of concrete (and new) examples of dynamical systems having this property.

math.DS↗

arXiv2025

StickMotion: Generating 3D Human Motions by Drawing a Stickman

Text-to-motion generation, which translates textual descriptions into human motions, has been challenging in accurately capturing detailed user-imagined motions from simple text inputs. This paper introduces StickMotion, an efficient diffusion-based network designed for multi-condition scenarios, which generates desired motions based on traditional text and our proposed stickman conditions for global and local control of these motions, respectively. We address the challenges introduced by the user-friendly stickman from three perspectives: 1) Data generation. We develop an algorithm to generate hand-drawn stickmen automatically across different dataset formats. 2) Multi-condition fusion. We propose a multi-condition module that integrates into the diffusion process and obtains outputs of all possible condition combinations, reducing computational complexity and enhancing StickMotion's performance compared to conventional approaches with the self-attention module. 3) Dynamic supervision. We empower StickMotion to make minor adjustments to the stickman's position within the output sequences, generating more natural movements through our proposed dynamic supervision strategy. Through quantitative experiments and user studies, sketching stickmen saves users about 51.5% of their time generating motions consistent with their imagination. Our codes, demos, and relevant data will be released to facilitate further research and validation within the scientific community.

cs.CV↗

arXiv2025

DiffBrush:Just Painting the Art by Your Hands

The rapid development of image generation and editing algorithms in recent years has enabled ordinary user to produce realistic images. However, the current AI painting ecosystem predominantly relies on text-driven diffusion models (T2I), which pose challenges in accurately capturing user requirements. Furthermore, achieving compatibility with other modalities incurs substantial training costs. To this end, we introduce DiffBrush, which is compatible with T2I models and allows users to draw and edit images. By manipulating and adapting the internal representation of the diffusion model, DiffBrush guides the model-generated images to converge towards the user's hand-drawn sketches for user's specific needs without additional training. DiffBrush achieves control over the color, semantic, and instance of objects in images by continuously guiding the latent and instance-level attention map during the denoising process of the diffusion model. Besides, we propose a latent regeneration, which refines the randomly sampled noise in the diffusion model, obtaining a better image generation layout. Finally, users only need to roughly draw the mask of the instance (acceptable colors) on the canvas, DiffBrush can naturally generate the corresponding instance at the corresponding location.

cs.CV↗

arXiv2025

Ultra-Stable Ferrimagnetic Second-Order Topological Insulator in 2D Metal-Organic Framework

Two-dimensional (2D) magnetic second-order topological insulators (SOTIs) exhibit distinct topological phases characterized by spin-polarized zero-dimensional (0D) corner states, which have garnered significant interest. However, 2D ferrimagnetic (FiM) SOTIs, particularly those that simultaneously exhibit ultra-stable corner states, are still lacking. Here, based on first-principles calculations and theoretical analysis, we reveal such SOTI state in a 2D metal-organic framework (MOF) material, Cr(pyz)2 (pyz = pyrazine). This material exhibits FiM ground state with an easy axis aligned along [001] direction. It hosts a nontrivial real Chern number in the spin-up channel, enabled by PT symmetry, with 0D corner states observable in disk. In contrast, the spin-down channel exhibits a trivial gapped bulk state. Notably, the topological corner states in monolayer Cr(pyz)2 show high robustness, even if the symmetries are broken by introducing defects, the corner states persist. We also considered other external perturbations, including uniaxial/biaxial strain, ligand rotation, and electric fields, the corner states still remain stable. Even more, the energy positions of the corner states are also nearly unchanged. This work is the first to identify ultra-stable FiM SOTI state in the MOF system, and provide an ideal platform for future experimental investigations and applications in spintronic devices.

cond-mat.mtrl-sci↗

arXiv2024

Adapting Vision Foundation Models for Robust Cloud Segmentation in Remote Sensing Images

Cloud segmentation is a critical challenge in remote sensing image interpretation, as its accuracy directly impacts the effectiveness of subsequent data processing and analysis. Recently, vision foundation models (VFM) have demonstrated powerful generalization capabilities across various visual tasks. In this paper, we present a parameter-efficient adaptive approach, termed Cloud-Adapter, designed to enhance the accuracy and robustness of cloud segmentation. Our method leverages a VFM pretrained on general domain data, which remains frozen, eliminating the need for additional training. Cloud-Adapter incorporates a lightweight spatial perception module that initially utilizes a convolutional neural network (ConvNet) to extract dense spatial representations. These multi-scale features are then aggregated and serve as contextual inputs to an adapting module, which modulates the frozen transformer layers within the VFM. Experimental results demonstrate that the Cloud-Adapter approach, utilizing only 0.6% of the trainable parameters of the frozen backbone, achieves substantial performance gains. Cloud-Adapter consistently achieves state-of-the-art performance across various cloud segmentation datasets from multiple satellite sources, sensor series, data processing levels, land cover scenarios, and annotation granularities. We have released the code and model checkpoints at https://xavierjiezou.github.io/Cloud-Adapter/ to support further research.

cs.CV↗

arXiv2024

The Janus Interface: How Fine-Tuning in Large Language Models Amplifies the Privacy Risks

The rapid advancements of large language models (LLMs) have raised public concerns about the privacy leakage of personally identifiable information (PII) within their extensive training datasets. Recent studies have demonstrated that an adversary could extract highly sensitive privacy data from the training data of LLMs with carefully designed prompts. However, these attacks suffer from the model's tendency to hallucinate and catastrophic forgetting (CF) in the pre-training stage, rendering the veracity of divulged PIIs negligible. In our research, we propose a novel attack, Janus, which exploits the fine-tuning interface to recover forgotten PIIs from the pre-training data in LLMs. We formalize the privacy leakage problem in LLMs and explain why forgotten PIIs can be recovered through empirical analysis on open-source language models. Based upon these insights, we evaluate the performance of Janus on both open-source language models and two latest LLMs, i.e., GPT-3.5-Turbo and LLaMA-2-7b. Our experiment results show that Janus amplifies the privacy risks by over 10 times in comparison with the baseline and significantly outperforms the state-of-the-art privacy extraction attacks including prefix attacks and in-context learning (ICL). Furthermore, our analysis validates that existing fine-tuning APIs provided by OpenAI and Azure AI Studio are susceptible to our Janus attack, allowing an adversary to conduct such an attack at a low cost.

cs.CR↗

arXiv2024

Sofic mean dimension of typical actions and a comparison theorem

We refine two results in the paper entitled "Sofic mean dimension" by Hanfeng Li, improving two inequalities with two equalities, respectively, for sofic mean dimension of typical actions. On the one hand, we study sofic mean dimension of full shifts, for which, Li provided an upper bound which however is not optimal. We prove a more delicate estimate from above, which is optimal for sofic mean dimension of full shifts over arbitrary alphabets (i.e. compact metrizable spaces). Our refinement, together with the techniques (in relation to an estimate from below) in the paper entitled "Mean dimension of full shifts" by Masaki Tsukamoto, eventually allows us to get the exact value of sofic mean dimension of full shifts over any finite dimensional compact metrizable spaces. On the other hand, we investigate finite group actions. In contrast to the case that the acting group is infinite (and amenable), Li showed that if a finite group acts continuously on a finite dimensional compact metrizable space, then sofic mean dimension may be different from (strictly less than) the classical (i.e. amenable) mean dimension (an explicitly known value in this case). We strengthen this result by proving a sharp lower bound, which, combining with the upper bound, gives the exact value of sofic mean dimension for all the actions of finite groups on finite dimensional compact metrizable spaces. Furthermore, this equality leads to a satisfactory comparison theorem for those actions, deciding when sofic mean dimension would coincide with classical mean dimension. Moreover, our two results, in particular, verify for a typical class of sofic group actions that sofic mean dimension does not depend on sofic approximation sequences.

math.DS↗

arXiv2024

Sofic approximation sequences and sofic mean dimension

The main purpose of this paper is to strengthen our understanding of sofic mean dimension of two typical classes of sofic group actions. First, we study finite group actions. We prove that sofic mean dimension of any amenable group action does not depend on the choice of sofic approximation sequences. Previously, this result was known only if the acting group is an infinite amenable group. Moreover, we investigate the full shifts, for all sofic groups and all alphabets. We show that sofic mean dimension of any full shift depends purely on its alphabet. Our method is a refinement of the classical technique in relation to the estimates from above and below, respectively, for mean dimension of some typical actions. The key point of our results is that they apply to all compact metrizable spaces without any restriction (in particular, the alphabet concerned in a full shift and the space involved in a finite group action are not required to be finite-dimensional). Furthermore, we improve the quantitative knowledge of sofic mean dimension, restricted to finite-dimensional compact metrizable spaces, for those two typical classes of sofic group actions. As a direct consequence of the main ingredient of our proof, we obtain the exact value of sofic mean dimension of all the actions of finite groups on finite-dimensional compact metrizable spaces. Previously, only an upper bound for these actions was given. Besides, we also get the exact value of sofic mean dimension of full shifts when the alphabet is finite-dimensional.

math.DS↗