arXiv subjects

Egor Chistov

Publications and source records attributed to Egor Chistov.

4 recordsLinked to original sources

arXiv2025

MEMFOF: High-Resolution Training for Memory-Efficient Multi-Frame Optical Flow Estimation

Recent advances in optical flow estimation have prioritized accuracy at the cost of growing GPU memory consumption, particularly for high-resolution (FullHD) inputs. We introduce MEMFOF, a memory-efficient multi-frame optical flow method that identifies a favorable trade-off between multi-frame estimation and GPU memory usage. Notably, MEMFOF requires only 2.09 GB of GPU memory at runtime for 1080p inputs, and 28.5 GB during training, which uniquely positions our method to be trained at native 1080p without the need for cropping or downsampling. We systematically revisit design choices from RAFT-like architectures, integrating reduced correlation volumes and high-resolution training protocols alongside multi-frame estimation, to achieve state-of-the-art performance across multiple benchmarks while substantially reducing memory overhead. Our method outperforms more resource-intensive alternatives in both accuracy and runtime efficiency, validating its robustness for flow estimation at high resolutions. At the time of submission, our method ranks first on the Spring benchmark with a 1-pixel (1px) outlier rate of 3.289, leads Sintel (clean) with an endpoint error (EPE) of 0.963, and achieves the best Fl-all error on KITTI-2015 at 2.94%. The code is available at https://github.com/msu-video-group/memfof.

cs.CV↗

arXiv2024

Can No-Reference Quality-Assessment Methods Serve as Perceptual Losses for Super-Resolution?

Perceptual losses play an important role in constructing deep-neural-network-based methods by increasing the naturalness and realism of processed images and videos. Use of perceptual losses is often limited to LPIPS, a fullreference method. Even though deep no-reference image-qualityassessment methods are excellent at predicting human judgment, little research has examined their incorporation in loss functions. This paper investigates direct optimization of several video-superresolution models using no-reference image-quality-assessment methods as perceptual losses. Our experimental results show that straightforward optimization of these methods produce artifacts, but a special training procedure can mitigate them.

eess.IV↗

arXiv2023

BASED: Benchmarking, Analysis, and Structural Estimation of Deblurring

This paper discusses the challenges of evaluating deblurring-methods quality and proposes a reduced-reference metric based on machine learning. Traditional quality-assessment metrics such as PSNR and SSIM are common for this task, but not only do they correlate poorly with subjective assessments, they also require ground-truth (GT) frames, which can be difficult to obtain in the case of deblurring. To develop and evaluate our metric, we created a new motion-blur dataset using a beam splitter. The setup captured various motion types using a static camera, as most scenes in existing datasets include blur due to camera motion. We also conducted two large subjective comparisons to aid in metric development. Our resulting metric requires no GT frames, and it correlates well with subjective human perception of blur.

cs.CV↗

arXiv2023

Color Mismatches in Stereoscopic Video: Real-World Dataset and Deep Correction Method

Stereoscopic videos can contain color mismatches between the left and right views due to minor variations in camera settings, lenses, and even object reflections captured from different positions. The presence of color mismatches can lead to viewer discomfort and headaches. This problem can be solved by transferring color between stereoscopic views, but traditional methods often lack quality, while neural-network-based methods can easily overfit on artificial data. The scarcity of stereoscopic videos with real-world color mismatches hinders the evaluation of different methods' performance. Therefore, we filmed a video dataset, which includes both distorted frames with color mismatches and ground-truth data, using a beam-splitter. Our second contribution is a deep multiscale neural network that solves the color-mismatch-correction task by leveraging stereo correspondences. The experimental results demonstrate the effectiveness of the proposed method on a conventional dataset, but there remains room for improvement on challenging real-world data.

cs.CV↗