arXiv subjects

Yifan Bian

Publications and source records attributed to Yifan Bian.

8 recordsLinked to original sources

arXiv2026

Real-Time Neural Video Compression with Unified Intra and Inter Coding

Neural video compression (NVC) technologies have advanced rapidly in recent years, yielding state-of-the-art schemes such as DCVC-RT that offer superior compression efficiency to H.266/VVC and real-time encoding/decoding capabilities. Nonetheless, existing NVC schemes have several limitations, including inefficiency in dealing with disocclusion and new content, interframe error propagation and accumulation, among others. To eliminate these limitations, we borrow the idea from classic video coding schemes, which allow intra coding within inter-coded frames. With the intra coding tool enabled, disocclusion and new content are properly handled, and interframe error propagation is naturally intercepted without the need for manual refresh mechanisms. We present an NVC framework with unified intra and inter coding, where every frame is processed by a single model that is trained to perform intra/inter coding adaptively. Moreover, we propose a simultaneous two-frame compression design to exploit interframe redundancy not only forwardly but also backwardly. Experimental results show that our scheme outperforms DCVC-RT by an average of 12.1% BD-rate reduction, delivers more stable bitrate and quality per frame, and retains real-time encoding/decoding performances. Code and models will be released.

cs.CV↗

arXiv2026

Recent Advances of End-to-End Video Coding Technologies for AVS Standard Development

Video coding standards are essential to enable the interoperability and widespread adoption of efficient video compression technologies. In pursuit of greater video compression efficiency, the AVS video coding working group launched the standardization exploration of end-to-end intelligent video coding, establishing the AVS End-to-End Intelligent Video Coding Exploration Model (AVS-EEM) project. A core design principle of AVS-EEM is its focus on practical deployment, featuring inherently low computational complexity and requiring strict adherence to the common test conditions of conventional video coding. This paper details the development history of AVS-EEM and provides a systematic introduction to its key technical framework, covering model architectures, training strategies, and inference optimizations. These innovations have collectively driven the project's rapid performance evolution, enabling continuous and significant gains under strict complexity constraints. Through over two years of iterative refinement and collaborative effort, the coding performance of AVS-EEM has seen substantial improvement. Experimental results demonstrate that its latest model achieves superior compression efficiency compared to the conventional AVS3 reference software, marking a significant step toward a deployable intelligent video coding standard.

eess.IV↗

arXiv2025

Neural Video Compression with Context Modulation

Efficient video coding is highly dependent on exploiting the temporal redundancy, which is usually achieved by extracting and leveraging the temporal context in the emerging conditional coding-based neural video codec (NVC). Although the latest NVC has achieved remarkable progress in improving the compression performance, the inherent temporal context propagation mechanism lacks the ability to sufficiently leverage the reference information, limiting further improvement. In this paper, we address the limitation by modulating the temporal context with the reference frame in two steps. Specifically, we first propose the flow orientation to mine the inter-correlation between the reference frame and prediction frame for generating the additional oriented temporal context. Moreover, we introduce the context compensation to leverage the oriented context to modulate the propagated temporal context generated from the propagated reference feature. Through the synergy mechanism and decoupling loss supervision, the irrelevant propagated information can be effectively eliminated to ensure better context modeling. Experimental results demonstrate that our codec achieves on average 22.7% bitrate reduction over the advanced traditional video codec H.266/VVC, and offers an average 10.1% bitrate saving over the previous state-of-the-art NVC DCVC-FM. The code is available at https://github.com/Austin4USTC/DCMVC.

eess.IV↗

arXiv2025

Augmented Deep Contexts for Spatially Embedded Video Coding

Most Neural Video Codecs (NVCs) only employ temporal references to generate temporal-only contexts and latent prior. These temporal-only NVCs fail to handle large motions or emerging objects due to limited contexts and misaligned latent prior. To relieve the limitations, we propose a Spatially Embedded Video Codec (SEVC), in which the low-resolution video is compressed for spatial references. Firstly, our SEVC leverages both spatial and temporal references to generate augmented motion vectors and hybrid spatial-temporal contexts. Secondly, to address the misalignment issue in latent prior and enrich the prior information, we introduce a spatial-guided latent prior augmented by multiple temporal latent representations. At last, we design a joint spatial-temporal optimization to learn quality-adaptive bit allocation for spatial references, further boosting rate-distortion performance. Experimental results show that our SEVC effectively alleviates the limitations in handling large motions or emerging objects, and also reduces 11.9% more bitrate than the previous state-of-the-art NVC while providing an additional low-resolution bitstream. Our code and model are available at https://github.com/EsakaK/SEVC.

eess.IV↗

arXiv2025

USTC-TD: A Test Dataset and Benchmark for Image and Video Coding in 2020s

Image/video coding has been a remarkable research area for both academia and industry for many years. Testing datasets, especially high-quality image/video datasets are desirable for the justified evaluation of coding-related research, practical applications, and standardization activities. We put forward a test dataset namely USTC-TD, which has been successfully adopted in the practical end-to-end image/video coding challenge of the IEEE International Conference on Visual Communications and Image Processing (VCIP) in 2022 and 2023. USTC-TD contains 40 images at 4K spatial resolution and 10 video sequences at 1080p spatial resolution, featuring various content due to the diverse environmental factors (e.g. scene type, texture, motion, view) and the designed imaging factors (e.g. illumination, lens, shadow). We quantitatively evaluate USTC-TD on different image/video features (spatial, temporal, color, lightness), and compare it with the previous image/video test datasets, which verifies its excellent compensation for the shortcomings of existing datasets. We also evaluate both classic standardized and recently learned image/video coding schemes on USTC-TD using objective quality metrics (PSNR, MS-SSIM, VMAF) and subjective quality metric (MOS), providing an extensive benchmark for these evaluated schemes. Based on the characteristics and specific design of the proposed test dataset, we analyze the benchmark performance and shed light on the future research and development of image/video coding. All the data are released online: https://esakak.github.io/USTC-TD.

eess.IV↗

arXiv2024

Immersive Human-Machine Teleoperation Framework for Precision Agriculture: Integrating UAV-based Digital Mapping and Virtual Reality Control

In agricultural settings, the unstructured nature of certain production environments, along with the high complexity and inherent risks of production tasks, poses significant challenges to achieving full automation and effective on-site machine control. Remote control technology, which leverages human intelligence and precise machine movements, ensures operator safety and boosts productivity. Recently, virtual reality (VR) has shown promise in remote control applications by overcoming single-view limitations and providing three-dimensional information, yet most studies have not focused on agricultural settings. Therefore, to bridge the gap, this study proposes a large-scale digital mapping and immersive human-machine teleoperation framework specifically designed for precision agriculture. In this research, a DJI unmanned aerial vehicle (UAV) was utilized for data collection, and a novel video segmentation approach based on feature points was introduced. To accommodate the variability of complex textures, this method proposes an enhanced Structure from Motion (SfM) approach. It integrates the open Multiple View Geometry (OpenMVG) framework with Local Features from Transformers (LoFTR). The enhanced SfM produces a point cloud map, which is further processed through Multi-View Stereo (MVS) to generate a complete map model. For control, a closed-loop system utilizing TCP/IP for VR control and positioning of agricultural machinery was introduced. This system offers a fully visual-based method for immersive control, allowing operators to utilize VR technology for remote operations. The experimental results demonstrate that the user-friendly remote control method also showcases its advantages over traditional video streaming-based remote operations, providing operators with a more comprehensive and immersive experience and a higher level of situational awareness.

eess.SY↗

arXiv2024

Panoptic-FlashOcc: An Efficient Baseline to Marry Semantic Occupancy with Panoptic via Instance Center

Panoptic occupancy poses a novel challenge by aiming to integrate instance occupancy and semantic occupancy within a unified framework. However, there is still a lack of efficient solutions for panoptic occupancy. In this paper, we propose Panoptic-FlashOcc, a straightforward yet robust 2D feature framework that enables realtime panoptic occupancy. Building upon the lightweight design of FlashOcc, our approach simultaneously learns semantic occupancy and class-aware instance clustering in a single network, these outputs are jointly incorporated through panoptic occupancy procession for panoptic occupancy. This approach effectively addresses the drawbacks of high memory and computation requirements associated with three-dimensional voxel-level representations. With its straightforward and efficient design that facilitates easy deployment, Panoptic-FlashOcc demonstrates remarkable achievements in panoptic occupancy prediction. On the Occ3D-nuScenes benchmark, it achieves exceptional performance, with 38.5 RayIoU and 29.1 mIoU for semantic occupancy, operating at a rapid speed of 43.9 FPS. Furthermore, it attains a notable score of 16.0 RayPQ for panoptic occupancy, accompanied by a fast inference speed of 30.2 FPS. These results surpass the performance of existing methodologies in terms of both speed and accuracy. The source code and trained models can be found at the following github repository: https://github.com/Yzichen/FlashOCC.

cs.CV↗

arXiv2023

Probing Supermassive Black Hole Binaries with Orbital Resonances of Laser-Ranged Satellite

Coalescing supermassive black hole binaries (SMBHBs) are the primary source candidates for low frequency gravitational wave (GW) detections, which could bring us deep insights into galaxy evolutions over cosmic time and violent processes of spacetime dynamics. Promising candidates had been found based on optical and X-ray observations, which claims for new and ready-to-use GW detection approaches before the operations of space-borne antennas. We show that, satellite laser ranging (SLR) missions could serve as probes of coalescing SMBHBs through the GW-induced resonant effects. Lasting and characteristic imprints caused by such resonances in the residual distances or accelerations from SLR measurements are studied, and the detection SNR is analyzed with both the current and future improved ranging precisions. Within redshift $z \sim 1$, the threshold SNR=5 requires 1-2 years of accumulated data for the current precision and months of data for improved precision, which are workable for the data processing of SLR missions. Meanwhile, joint detections with multiple SLR missions could further improve the total SNR and the confidence level. Such a detection scheme could fulfill the requirement of a tentative SMBHB probe during the preparing stage of LISA and Taiji, and it requires no further investment to any new and advanced facilities. It is also worthwhile to look back and re-process the archived data from the past decades, in where resonant signals from SMBHBs might be hidden.

gr-qc↗