arXiv subjects

Zijing Zhao

Publications and source records attributed to Zijing Zhao.

14 recordsLinked to original sources

arXiv2026

SIGMark: Scalable In-Generation Watermark with Blind Extraction for Video Diffusion

Artificial Intelligence Generated Content (AIGC), particularly video generation with diffusion models, has been advanced rapidly. Invisible watermarking is a key technology for protecting AI-generated videos and tracing harmful content, and thus plays a crucial role in AI safety. Beyond post-processing watermarks which inevitably degrade video quality, recent studies have proposed distortion-free in-generation watermarking for video diffusion models. However, existing in-generation approaches are non-blind: they require maintaining all the message-key pairs and performing template-based matching during extraction, which incurs prohibitive computational costs at scale. Moreover, when applied to modern video diffusion models with causal 3D Variational Autoencoders (VAEs), their robustness against temporal disturbance becomes extremely weak. To overcome these challenges, we propose SIGMark, a Scalable In-Generation watermarking framework with blind extraction for video diffusion. To achieve blind-extraction, we propose to generate watermarked initial noise using a Global set of Frame-wise PseudoRandom Coding keys (GF-PRC), reducing the cost of storing large-scale information while preserving noise distribution and diversity for distortion-free watermarking. To enhance robustness, we further design a Segment Group-Ordering module (SGO) tailored to causal 3D VAEs, ensuring robust watermark inversion during extraction under temporal disturbance. Comprehensive experiments on modern diffusion models show that SIGMark achieves very high bit-accuracy during extraction under both temporal and spatial disturbances with minimal overhead, demonstrating its scalability and robustness. Our project is available at https://jeremyzhao1998.github.io/SIGMark-release/.

cs.CV↗

arXiv2025

Investigating Domain Gaps for Indoor 3D Object Detection

As a fundamental task for indoor scene understanding, 3D object detection has been extensively studied, and the accuracy on indoor point cloud data has been substantially improved. However, existing researches have been conducted on limited datasets, where the training and testing sets share the same distribution. In this paper, we consider the task of adapting indoor 3D object detectors from one dataset to another, presenting a comprehensive benchmark with ScanNet, SUN RGB-D and 3D Front datasets, as well as our newly proposed large-scale datasets ProcTHOR-OD and ProcFront generated by a 3D simulator. Since indoor point cloud datasets are collected and constructed in different ways, the object detectors are likely to overfit to specific factors within each dataset, such as point cloud quality, bounding box layout and instance features. We conduct experiments across datasets on different adaptation scenarios including synthetic-to-real adaptation, point cloud quality adaptation, layout adaptation and instance feature adaptation, analyzing the impact of different domain gaps on 3D object detectors. We also introduce several approaches to improve adaptation performances, providing baselines for domain adaptive indoor 3D object detection, hoping that future works may propose detectors with stronger generalization ability across domains. Our project homepage can be found in https://jeremyzhao1998.github.io/DAVoteNet-release/.

cs.CV↗

arXiv2025

AR-VRM: Imitating Human Motions for Visual Robot Manipulation with Analogical Reasoning

Visual Robot Manipulation (VRM) aims to enable a robot to follow natural language instructions based on robot states and visual observations, and therefore requires costly multi-modal data. To compensate for the deficiency of robot data, existing approaches have employed vision-language pretraining with large-scale data. However, they either utilize web data that differs from robotic tasks, or train the model in an implicit way (e.g., predicting future frames at the pixel level), thus showing limited generalization ability under insufficient robot data. In this paper, we propose to learn from large-scale human action video datasets in an explicit way (i.e., imitating human actions from hand keypoints), introducing Visual Robot Manipulation with Analogical Reasoning (AR-VRM). To acquire action knowledge explicitly from human action videos, we propose a keypoint Vision-Language Model (VLM) pretraining scheme, enabling the VLM to learn human action knowledge and directly predict human hand keypoints. During fine-tuning on robot data, to facilitate the robotic arm in imitating the action patterns of human motions, we first retrieve human action videos that perform similar manipulation tasks and have similar historical observations , and then learn the Analogical Reasoning (AR) map between human hand keypoints and robot components. Taking advantage of focusing on action keypoints instead of irrelevant visual cues, our method achieves leading performance on the CALVIN benchmark {and real-world experiments}. In few-shot scenarios, our AR-VRM outperforms previous methods by large margins , underscoring the effectiveness of explicitly imitating human actions under data scarcity.

cs.CV↗

arXiv2025

Zero Shot Domain Adaptive Semantic Segmentation by Synthetic Data Generation and Progressive Adaptation

Deep learning-based semantic segmentation models achieve impressive results yet remain limited in handling distribution shifts between training and test data. In this paper, we present SDGPA (Synthetic Data Generation and Progressive Adaptation), a novel method that tackles zero-shot domain adaptive semantic segmentation, in which no target images are available, but only a text description of the target domain's style is provided. To compensate for the lack of target domain training data, we utilize a pretrained off-the-shelf text-to-image diffusion model, which generates training images by transferring source domain images to target style. Directly editing source domain images introduces noise that harms segmentation because the layout of source images cannot be precisely maintained. To address inaccurate layouts in synthetic data, we propose a method that crops the source image, edits small patches individually, and then merges them back together, which helps improve spatial precision. Recognizing the large domain gap, SDGPA constructs an augmented intermediate domain, leveraging easier adaptation subtasks to enable more stable model adaptation to the target domain. Additionally, to mitigate the impact of noise in synthetic data, we design a progressive adaptation strategy, ensuring robust learning throughout the training process. Extensive experiments demonstrate that our method achieves state-of-the-art performance in zero-shot semantic segmentation. The code is available at https://github.com/ROUJINN/SDGPA

cs.CV↗

arXiv2025

VS-Singer: Vision-Guided Stereo Singing Voice Synthesis with Consistency Schrödinger Bridge

To explore the potential advantages of utilizing spatial cues from images for generating stereo singing voices with room reverberation, we introduce VS-Singer, a vision-guided model designed to produce stereo singing voices with room reverberation from scene images. VS-Singer comprises three modules: firstly, a modal interaction network integrates spatial features into text encoding to create a linguistic representation enriched with spatial information. Secondly, the decoder employs a consistency Schrödinger bridge to facilitate one-step sample generation. Moreover, we utilize the SFE module to improve the consistency of audio-visual matching. To our knowledge, this study is the first to combine stereo singing voice synthesis with visual acoustic matching within a unified framework. Experimental results demonstrate that VS-Singer can effectively generate stereo singing voices that align with the scene perspective in a single step.

cs.SD↗

arXiv2025

CRTrack: Low-Light Semi-Supervised Multi-object Tracking Based on Consistency Regularization

Multi-object tracking under low-light environments is prevalent in real life. Recent years have seen rapid development in the field of multi-object tracking. However, due to the lack of datasets and the high cost of annotations, multi-object tracking under low-light environments remains a persistent challenge. In this paper, we focus on multi-object tracking under low-light conditions. To address the issues of limited data and the lack of dataset, we first constructed a low-light multi-object tracking dataset (LLMOT). This dataset comprises data from MOT17 that has been enhanced for nighttime conditions as well as multiple unannotated low-light videos. Subsequently, to tackle the high annotation costs and address the issue of image quality degradation, we propose a semi-supervised multi-object tracking method based on consistency regularization named CRTrack. First, we calibrate a consistent adaptive sampling assignment to replace the static IoU-based strategy, enabling the semi-supervised tracking method to resist noisy pseudo-bounding boxes. Then, we design a adaptive semi-supervised network update method, which effectively leverages unannotated data to enhance model performance. Dataset and Code: https://github.com/ZJZhao123/CRTrack.

cs.CV↗

arXiv2025

PlanLLM: Video Procedure Planning with Refinable Large Language Models

Video procedure planning, i.e., planning a sequence of action steps given the video frames of start and goal states, is an essential ability for embodied AI. Recent works utilize Large Language Models (LLMs) to generate enriched action step description texts to guide action step decoding. Although LLMs are introduced, these methods decode the action steps into a closed-set of one-hot vectors, limiting the model's capability of generalizing to new steps or tasks. Additionally, fixed action step descriptions based on world-level commonsense may contain noise in specific instances of visual states. In this paper, we propose PlanLLM, a cross-modal joint learning framework with LLMs for video procedure planning. We propose an LLM-Enhanced Planning module which fully uses the generalization ability of LLMs to produce free-form planning output and to enhance action step decoding. We also propose Mutual Information Maximization module to connect world-level commonsense of step descriptions and sample-specific information of visual states, enabling LLMs to employ the reasoning ability to generate step sequences. With the assistance of LLMs, our method can both closed-set and open vocabulary procedure planning tasks. Our PlanLLM achieves superior performance on three benchmarks, demonstrating the effectiveness of our designs.

cs.CV↗

arXiv2024

IAIFNet: An Illumination-Aware Infrared and Visible Image Fusion Network

Infrared and visible image fusion (IVIF) is used to generate fusion images with comprehensive features of both images, which is beneficial for downstream vision tasks. However, current methods rarely consider the illumination condition in low-light environments, and the targets in the fused images are often not prominent. To address the above issues, we propose an Illumination-Aware Infrared and Visible Image Fusion Network, named as IAIFNet. In our framework, an illumination enhancement network first estimates the incident illumination maps of input images. Afterwards, with the help of proposed adaptive differential fusion module (ADFM) and salient target aware module (STAM), an image fusion network effectively integrates the salient features of the illumination-enhanced infrared and visible images into a fusion image of high visual quality. Extensive experimental results verify that our method outperforms five state-of-the-art methods of fusing infrared and visible images.

cs.CV↗

arXiv2023

SAMP: A Model Inference Toolkit of Post-Training Quantization for Text Processing via Self-Adaptive Mixed-Precision

The latest industrial inference engines, such as FasterTransformer and TurboTransformers, have verified that half-precision floating point (FP16) and 8-bit integer (INT8) quantization can greatly improve model inference speed. However, the existing INT8 quantization methods are too complicated, and improper usage will lead to model performance damage greatly. In this paper, we develop a toolkit for users to easily quantize their models for inference, in which Self-Adaptive Mixed-Precision (SAMP) is proposed to automatically control quantization rate by a mixed-precision architecture to balance model accuracy and efficiency. Experimental results show that our SAMP toolkit has a higher speedup than PyTorch and FasterTransformer while ensuring the required accuracy. In addition, SAMP is based on a modular design, decoupling the tokenizer, embedding, encoder and target layers, which allows users to handle various downstream tasks and can be seamlessly integrated into PyTorch.

cs.LG↗

arXiv2023

2D Ambipolar Vertical Transistors as Control-free Reconfigurable Logic Devices

As transistor footprint scales down to sub-10 nm regime, the process development for advancing to further technology nodes has encountered slowdowns. Achieving greater functionality within a single chip requires concurrent development at the device, circuit, and system levels. Reconfigurable transistors possess the capability to transform into both n-type and p-type transistors dynamically during operation. This transistor-level reconfigurability enables field-programmable logic circuits with fewer components compared to conventional circuits. However, the reconfigurability requires additional polarity control gates in the transistor and potentially impairs the gain from a smaller footprint. In this paper, vertical transistors with ambipolar MoTe2 channels are fabricated using the transfer-metal method. The efficient asymmetric electrostatic gating in source and drain contacts gives rise to different Schottky barriers at the two contacts. Consequently, the ambipolar conduction is reduced to unipolar conduction due to different Schottky barrier widths for electrons and holes. The current flow direction determines the preferred carrier type. Temperature-dependent measurements reveal the Schottky barrier-controlled conduction in the vertical transistors and confirm different Schottky barrier widths with and without electrostatic gating. Without the complexity overhead from polarity control gates, control-free vertical reconfigurable transistors promise higher logic density and lower cost in future integrated circuits.

physics.app-ph↗

arXiv2023

Low-Thermal-Budget Ferroelectric Field-Effect Transistors Based on CuInP2S6 and InZnO

In this paper, we demonstrate low-thermal-budget ferroelectric field-effect transistors (FeFETs) based on two-dimensional ferroelectric CuInP2S6 (CIPS) and oxide semiconductor InZnO (IZO). The CIPS/IZO FeFETs exhibit non-volatile memory windows of ~1 V, low off-state drain currents, and high carrier mobilities. The ferroelectric CIPS layer serves a dual purpose by providing electrostatic doping in IZO and acting as a passivation layer for the IZO channel. We also investigate the CIPS/IZO FeFETs as artificial synaptic devices for neural networks. The CIPS/IZO synapse demonstrates a sizeable dynamic ratio (125) and maintains stable multi-level states. Neural networks based on CIPS/IZO FeFETs achieve an accuracy rate of over 80% in recognizing MNIST handwritten digits. These ferroelectric transistors can be vertically stacked on silicon CMOS with a low thermal budget, offering broad applications in CMOS+X technologies and energy-efficient 3D neural networks.

cond-mat.mes-hall↗

arXiv2023

Physics-Based Modeling and Validation of 2D Schottky Barrier Field-Effect Transistors

In this work, we describe the charge transport in two-dimensional (2D) Schottky barrier field-effect transistors (SB-FETs) based on the carrier injection at the Schottky contacts. We first develop a numerical model for thermionic and field-emission processes of carrier injection that occur at a Schottky contact. The numerical model is then simplified to yield an analytic equation for current versus voltage ($I$-$V$) in the SB-FET. The lateral electric field at the junction, controlling the carrier injection, is obtained by accurately modeling the electrostatics and the tunneling barrier width. Unlike previous SB-FET models that are valid for near-equilibrium conditions, this model is applicable for a broad bias range as it incorporates the pertinent physics of thermionic, thermionic field-emission, and field-emission processes from a 3D metal into a 2D semiconductor. The $I$-$V$ model is validated against the measurement data of 2-, 3-, and 4-layer ambipolar MoTe$_2$ SB-FETs fabricated in our lab, as well as the published data of unipolar 2D SB-FETs using MoS$_2$. Finally, the model's physics is tested rigorously by comparing model-generated data against TCAD simulation data.

physics.app-ph↗

arXiv2023

Content Addressable Memories and Transformable Logic Circuits Based on Ferroelectric Reconfigurable Transistors for In-Memory Computing

As a promising alternative to the Von Neumann architecture, in-memory computing holds the promise of delivering high computing capacity while consuming low power. Content addressable memory (CAM) can implement pattern matching and distance measurement in memory with massive parallelism, making them highly desirable for data-intensive applications. In this paper, we propose and demonstrate a novel 1-transistor-per-bit CAM based on the ferroelectric reconfigurable transistor. By exploiting the switchable polarity of the ferroelectric reconfigurable transistor, XOR/XNOR-like matching operation in CAM can be realized in a single transistor. By eliminating the need for the complementary circuit, these non-volatile CAMs based on reconfigurable transistors can offer a significant improvement in area and energy efficiency compared to conventional CAMs. NAND- and NOR-arrays of CAMs are also demonstrated, which enable multi-bit matching in a single reading operation. In addition, the NOR array of CAM cells effectively measures the Hamming distance between the input query and stored entries. Furthermore, utilizing the switchable polarity of these ferroelectric Schottky barrier transistors, we demonstrate reconfigurable logic gates with NAND/NOR dual functions, whose input-output mapping can be transformed in real-time without changing the layout. These reconfigurable circuits will serve as important building blocks for high-density data-stream processors and reconfigurable Application-Specific Integrated Circuits (r-ASICs). The CAMs and transformable logic gates based on ferroelectric reconfigurable transistors will have broad applications in data-intensive applications from image processing to machine learning and artificial intelligence.

cond-mat.mes-hall↗

arXiv2023

Efficient current-induced spin torques and field-free magnetization switching in a room-temperature van der Waals magnet

The discovery of magnetism in van der Waals (vdW) materials has established unique building blocks for the research of emergent spintronic phenomena. In particular, owing to their intrinsically clean surface without dangling bonds, the vdW magnets hold the potential to construct a superior interface that allows for efficient electrical manipulation of magnetism. Despite several attempts in this direction, it usually requires a cryogenic condition and the assistance of external magnetic fields, which is detrimental to the real application. Here, we fabricate heterostructures based on Fe3GaTe2 flakes that possess room-temperature ferromagnetism with excellent perpendicular magnetic anisotropy. The current-driven non-reciprocal modulation of coercive fields reveals a high spin-torque efficiency in the Fe3GaTe2/Pt heterostructures, which further leads to a full magnetization switching by current. Moreover, we demonstrate the field-free magnetization switching resulting from out-of-plane polarized spin currents by asymmetric geometry design. Our work could expedite the development of efficient vdW spintronic logic, memory and neuromorphic computing devices.

cond-mat.mtrl-sci↗