SearcharxivSearch

arXiv subjects

Si Gao

Publications and source records attributed to Si Gao.

12 recordsLinked to original sources

RepNet-VSR: Reparameterizable Architecture for High-Fidelity Video Super-Resolution

As a fundamental challenge in visual computing, video super-resolution (VSR) focuses on reconstructing highdefinition video sequences from their degraded lowresolution counterparts. While deep convolutional neural networks have demonstrated state-of-the-art performance in spatial-temporal super-resolution tasks, their computationally intensive nature poses significant deployment challenges for resource-constrained edge devices, particularly in real-time mobile video processing scenarios where power efficiency and latency constraints coexist. In this work, we propose a Reparameterizable Architecture for High Fidelity Video Super Resolution method, named RepNet-VSR, for real-time 4x video super-resolution. On the REDS validation set, the proposed model achieves 27.79 dB PSNR when processing 180p to 720p frames in 103 ms per 10 frames on a MediaTek Dimensity NPU. The competition results demonstrate an excellent balance between restoration quality and deployment efficiency. The proposed method scores higher than the previous champion algorithm of MAI video super-resolution challenge.

eess.IV

Semi-supervised Video Semantic Segmentation Using Unreliable Pseudo Labels for PVUW2024

Pixel-level Scene Understanding is one of the fundamental problems in computer vision, which aims at recognizing object classes, masks and semantics of each pixel in the given image. Compared with image scene parsing, video scene parsing introduces temporal information, which can effectively improve the consistency and accuracy of prediction,because the real-world is actually video-based rather than a static state. In this paper, we adopt semi-supervised video semantic segmentation method based on unreliable pseudo labels. Then, We ensemble the teacher network model with the student network model to generate pseudo labels and retrain the student network. Our method achieves the mIoU scores of 63.71% and 67.83% on development test and final test respectively. Finally, we obtain the 1st place in the Video Scene Parsing in the Wild Challenge at CVPR 2024.

cs.CV

2nd Place Solution for PVUW Challenge 2024: Video Panoptic Segmentation

Video Panoptic Segmentation (VPS) is a challenging task that is extends from image panoptic segmentation.VPS aims to simultaneously classify, track, segment all objects in a video, including both things and stuff. Due to its wide application in many downstream tasks such as video understanding, video editing, and autonomous driving. In order to deal with the task of video panoptic segmentation in the wild, we propose a robust integrated video panoptic segmentation solution. We use DVIS++ framework as our baseline to generate the initial masks. Then,we add an additional image semantic segmentation model to further improve the performance of semantic classes.Finally, our method achieves state-of-the-art performance with a VPQ score of 56.36 and 57.12 in the development and test phases, respectively, and ultimately ranked 2nd in the VPS track of the PVUW Challenge at CVPR2024.

cs.CV

Real-Time 4K Super-Resolution of Compressed AVIF Images. AIS 2024 Challenge Survey

This paper introduces a novel benchmark as part of the AIS 2024 Real-Time Image Super-Resolution (RTSR) Challenge, which aims to upscale compressed images from 540p to 4K resolution (4x factor) in real-time on commercial GPUs. For this, we use a diverse test set containing a variety of 4K images ranging from digital art to gaming and photography. The images are compressed using the modern AVIF codec, instead of JPEG. All the proposed methods improve PSNR fidelity over Lanczos interpolation, and process images under 10ms. Out of the 160 participants, 25 teams submitted their code and models. The solutions present novel designs tailored for memory-efficiency and runtime on edge devices. This survey describes the best solutions for real-time SR of compressed high-resolution images.

cs.CV

Multi-Convergence-Angle Ptychography with Simultaneous Strong Contrast and High Resolution

Advances in bioimaging methods and hardware facilities have revolutionised the determination of numerous biological structures at atomic or near-atomic resolution. Among these developments, electron ptychography has recently attracted considerable attention because of its superior resolution, remarkable sensitivity to light elements, and high electron dose efficiency. Here, we introduce an innovative approach called multi-convergence-angle (MCA) ptychography, which can simultaneously enhance both contrast and resolution with continuous information transfer across a wide spectrum of spatial frequency. Our work provides feasibility of future applications of MCA-ptychography in providing high-quality two-dimensional images as input to three-dimensional reconstruction methods, thereby facilitating more accurate determination of biological structures.

physics.optics

4D-Explorer: A visual software for 4D-STEM data processing and image reconstruction

With the development of high-speed electron detectors, four-dimensional scanning transmission electron microscopy (4D-STEM) has emerged as a powerful tool for characterizing microstructures in material science and life science. However, the complexity of 4D-STEM data processing necessitates an intuitive graphical user interface software for researchers. In this regard, we have developed 4D-Explorer, an open-source, lightweight and extensible software for processing 4D-STEM data. It offers a visual and interactive workflow, including data preparation, calibration, image reconstruction and generating quantitative results. Furthermore, during calibration, our software includes a novel algorithm for rotational offset correction that uses a defocused 4D-STEM dataset and its axial bright field image, which has lower experimental requirements than conventional methods. We anticipate that 4D-Explorer will help researchers harness the capabilities of 4D-STEM technology.

physics.ins-det

Recyclable Semi-supervised Method Based on Multi-model Ensemble for Video Scene Parsing

Pixel-level Scene Understanding is one of the fundamental problems in computer vision, which aims at recognizing object classes, masks and semantics of each pixel in the given image. Since the real-world is actually video-based rather than a static state, learning to perform video semantic segmentation is more reasonable and practical for realistic applications. In this paper, we adopt Mask2Former as architecture and ViT-Adapter as backbone. Then, we propose a recyclable semi-supervised training method based on multi-model ensemble. Our method achieves the mIoU scores of 62.97% and 65.83% on Development test and final test respectively. Finally, we obtain the 2nd place in the Video Scene Parsing in the Wild Challenge at CVPR 2023.

eess.IV

Power Efficient Video Super-Resolution on Mobile NPUs with Deep Learning, Mobile AI & AIM 2022 challenge: Report

Video super-resolution is one of the most popular tasks on mobile devices, being widely used for an automatic improvement of low-bitrate and low-resolution video streams. While numerous solutions have been proposed for this problem, they are usually quite computationally demanding, demonstrating low FPS rates and power efficiency on mobile devices. In this Mobile AI challenge, we address this problem and propose the participants to design an end-to-end real-time video super-resolution solution for mobile NPUs optimized for low energy consumption. The participants were provided with the REDS training dataset containing video sequences for a 4X video upscaling task. The runtime and power efficiency of all models was evaluated on the powerful MediaTek Dimensity 9000 platform with a dedicated AI processing unit capable of accelerating floating-point and quantized neural networks. All proposed solutions are fully compatible with the above NPU, demonstrating an up to 500 FPS rate and 0.2 [Watt / 30 FPS] power consumption. A detailed description of all models developed in the challenge is provided in this paper.

eess.IV

NTIRE 2022 Challenge on Super-Resolution and Quality Enhancement of Compressed Video: Dataset, Methods and Results

This paper reviews the NTIRE 2022 Challenge on Super-Resolution and Quality Enhancement of Compressed Video. In this challenge, we proposed the LDV 2.0 dataset, which includes the LDV dataset (240 videos) and 95 additional videos. This challenge includes three tracks. Track 1 aims at enhancing the videos compressed by HEVC at a fixed QP. Track 2 and Track 3 target both the super-resolution and quality enhancement of HEVC compressed video. They require x2 and x4 super-resolution, respectively. The three tracks totally attract more than 600 registrations. In the test phase, 8 teams, 8 teams and 12 teams submitted the final results to Tracks 1, 2 and 3, respectively. The proposed methods and solutions gauge the state-of-the-art of super-resolution and quality enhancement of compressed video. The proposed LDV 2.0 dataset is available at https://github.com/RenYang-home/LDV_dataset. The homepage of this challenge (including open-sourced codes) is at https://github.com/RenYang-home/NTIRE22_VEnh_SR.

cs.CV

Giant Polarization and Abnormal Flexural Deformation in Bent Freestanding Perovskite Oxides

Recent realizations of ultrathin freestanding perovskite oxides offer a unique platform to probe novel properties in two-dimensional oxides. Here, we observed a giant flexoelectric response in freestanding BiFeO3 and SrTiO3 in their bent state arising from strain gradients up to 4x10e7/m, suggesting a promising approach for realizing extremely large polarizations. Additionally, a substantial reversible change in thickness was discovered in bent freestanding BiFeO3, which implies an unusual bending-expansion/shrinkage and thickness-dependence Poisson's ratios in this ferroelectric membrane that has never been seen before in crystalline materials. Our theoretical modeling reveals that this unprecedented flexural deformation within the membrane is attributable to a flexoelectricity-piezoelectricity interplay. The finding unveils intriguing nanoscale electromechanical properties and provides guidance for their practical applications in flexible nanoelectromechanical systems.

cond-mat.mtrl-sci

Repairing atomic vacancies in single-layer MoSe2 field-effect transistor and its defect dynamics

Here we repair the single-layer MoSe2 field-effect transistors by the EDTA processing, after which the devices' room-temperature carrier mobility increases from 0.1 to over 70cm2/Vs. The atomic dynamics is constructed by the combined study of the first-principle calculation, aberration-corrected transmission electron microscopy and Raman spectroscopy. Single/double Se vacancies are revealed originally, which cause some mid-gap impurity states and localize the device carriers. They are found repaired with the result of improved electronic transport. Such a picture is confirmed by a 1.5cm-1 red shift in the Raman spectra.

cond-mat.mes-hall

Probing Carrier Transport and Structure-property Relationship of Highly Ordered Organic Semiconductors at Two-dimensional Limit

One of the basic assumptions in organic field-effect transistors, the most fundamental device unit in organic electronics, is that charge transport occurs two-dimensionally in the first few molecular layers near the dielectric interface. Although the mobility of bulk organic semiconductors has increased dramatically, direct probing of intrinsic charge transport in the two-dimensional limit has not been possible due to excessive disorders and traps in ultrathin organic thin films. Here, highly ordered mono- to tetra-layer pentacene crystals are realized by van der Waals (vdW) epitaxy on hexagonal BN. We find that the charge transport is dominated by hopping in the first conductive layer, but transforms to band-like in subsequent layers. Such abrupt phase transition is attributed to strong modulation of the molecular packing by interfacial vdW interactions, as corroborated by quantitative structural characterization and density functional theory calculations. The structural modulation becomes negligible beyond the second conductive layer, leading to a mobility saturation thickness of only ~3nm. Highly ordered organic ultrathin films provide a platform for new physics and device structures (such as heterostructures and quantum wells) that are not possible in conventional bulk crystals.

cond-mat.mes-hall