arXiv subjects

Bochen Guan

Publications and source records attributed to Bochen Guan.

11 recordsLinked to original sources

arXiv2026

ClearText-Video: A Large-Scale Text-Centric Video Dataset Bridging Video Restoration and Scene-Text Enhancement

Multimodal Large Language Models (MLLMs) have recently made strong progress in visual--linguistic understanding. However, their performance on text-centric video reasoning remains highly sensitive to input quality. Real-world user-provided videos often contain motion blur, compression artifacts, noise, and low-resolution text, which impair reliable text reading and downstream reasoning. Whether MLLMs can robustly read and reason about real-world scene text under diverse quality conditions remains a fundamental open question. We introduce ClearText-Video (CTVid), a large-scale, scene-text-aware benchmark for studying text-centric video understanding under controlled quality variation. CTVid contains 4,639 real-world text-rich egocentric videos, 550K+ frames, 1.6M human-verified scene-text annotations, and 220K+ spatial/temporal question--answer pairs in Chinese and English. For each high-quality video, CTVid provides content-matched Degraded-Quality and Restored-Quality variants, supporting two task families: Text-Centric Video Restoration and Multi-Quality VideoQA. We evaluate 18 representative restoration methods and 16 state-of-the-art MLLMs on CTVid. The results show that visual enhancement does not guarantee textual fidelity or downstream reasoning gains: blur is more damaging than low resolution, restored videos can alter the textual evidence used by MLLMs, and OCR-only pipelines remain far below direct multimodal reasoning. CTVid exposes the gap between video restoration and text-grounded understanding, providing a rigorous foundation for restoration-aware, quality-robust text-centric video systems.

cs.CV↗

arXiv2024

PA-SAM: Prompt Adapter SAM for High-Quality Image Segmentation

The Segment Anything Model (SAM) has exhibited outstanding performance in various image segmentation tasks. Despite being trained with over a billion masks, SAM faces challenges in mask prediction quality in numerous scenarios, especially in real-world contexts. In this paper, we introduce a novel prompt-driven adapter into SAM, namely Prompt Adapter Segment Anything Model (PA-SAM), aiming to enhance the segmentation mask quality of the original SAM. By exclusively training the prompt adapter, PA-SAM extracts detailed information from images and optimizes the mask decoder feature at both sparse and dense prompt levels, improving the segmentation performance of SAM to produce high-quality masks. Experimental results demonstrate that our PA-SAM outperforms other SAM-based methods in high-quality, zero-shot, and open-set segmentation. We're making the source code and models available at https://github.com/xzz2/pa-sam.

cs.CV↗

arXiv2023

FreeControl: Training-Free Spatial Control of Any Text-to-Image Diffusion Model with Any Condition

Recent approaches such as ControlNet offer users fine-grained spatial control over text-to-image (T2I) diffusion models. However, auxiliary modules have to be trained for each type of spatial condition, model architecture, and checkpoint, putting them at odds with the diverse intents and preferences a human designer would like to convey to the AI models during the content creation process. In this work, we present FreeControl, a training-free approach for controllable T2I generation that supports multiple conditions, architectures, and checkpoints simultaneously. FreeControl designs structure guidance to facilitate the structure alignment with a guidance image, and appearance guidance to enable the appearance sharing between images generated using the same seed. Extensive qualitative and quantitative experiments demonstrate the superior performance of FreeControl across a variety of pre-trained T2I models. In particular, FreeControl facilitates convenient training-free control over many different architectures and checkpoints, allows the challenging input conditions on which most of the existing training-free methods fail, and achieves competitive synthesis quality with training-based approaches.

cs.CV↗

arXiv2023

Vision Backbone Enhancement via Multi-Stage Cross-Scale Attention

Convolutional neural networks (CNNs) and vision transformers (ViTs) have achieved remarkable success in various vision tasks. However, many architectures do not consider interactions between feature maps from different stages and scales, which may limit their performance. In this work, we propose a simple add-on attention module to overcome these limitations via multi-stage and cross-scale interactions. Specifically, the proposed Multi-Stage Cross-Scale Attention (MSCSA) module takes feature maps from different stages to enable multi-stage interactions and achieves cross-scale interactions by computing self-attention at different scales based on the multi-stage feature maps. Our experiments on several downstream tasks show that MSCSA provides a significant performance boost with modest additional FLOPs and runtime.

cs.CV↗

arXiv2023

SimHaze: game engine simulated data for real-world dehazing

Deep models have demonstrated recent success in single-image dehazing. Most prior methods consider fully supervised training and learn from paired clean and hazy images, where a hazy image is synthesized based on a clean image and its estimated depth map. This paradigm, however, can produce low-quality hazy images due to inaccurate depth estimation, resulting in poor generalization of the trained models. In this paper, we explore an alternative approach for generating paired clean-hazy images by leveraging computer graphics. Using a modern game engine, our approach renders crisp clean images and their precise depth maps, based on which high-quality hazy images can be synthesized for training dehazing models. To this end, we present SimHaze: a new synthetic haze dataset. More importantly, we show that training with SimHaze alone allows the latest dehazing models to achieve significantly better performance in comparison to previous dehazing datasets. Our dataset and code will be made publicly available.

cs.CV↗

arXiv2021

SMOF: Squeezing More Out of Filters Yields Hardware-Friendly CNN Pruning

For many years, the family of convolutional neural networks (CNNs) has been a workhorse in deep learning. Recently, many novel CNN structures have been designed to address increasingly challenging tasks. To make them work efficiently on edge devices, researchers have proposed various structured network pruning strategies to reduce their memory and computational cost. However, most of them only focus on reducing the number of filter channels per layer without considering the redundancy within individual filter channels. In this work, we explore pruning from another dimension, the kernel size. We develop a CNN pruning framework called SMOF, which Squeezes More Out of Filters by reducing both kernel size and the number of filter channels. Notably, SMOF is friendly to standard hardware devices without any customized low-level implementations, and the pruning effort by kernel size reduction does not suffer from the fixed-size width constraint in SIMD units of general-purpose processors. The pruned networks can be deployed effortlessly with significant running time reduction. We also support these claims via extensive experiments on various CNN structures and general-purpose processors for mobile devices.

cs.CV↗

arXiv2021

SpecNet: Spectral Domain Convolutional Neural Network

The memory consumption of most Convolutional Neural Network (CNN) architectures grows rapidly with increasing depth of the network, which is a major constraint for efficient network training on modern GPUs with limited memory, embedded systems, and mobile devices. Several studies show that the feature maps (as generated after the convolutional layers) are the main bottleneck in this memory problem. Often, these feature maps mimic natural photographs in the sense that their energy is concentrated in the spectral domain. Although embedding CNN architectures in the spectral domain is widely exploited to accelerate the training process, we demonstrate that it is also possible to use the spectral domain to reduce the memory footprint, a method we call Spectral Domain Convolutional Neural Network (SpecNet) that performs both the convolution and the activation operations in the spectral domain. The performance of SpecNet is evaluated on three competitive object recognition benchmark tasks (CIFAR-10, SVHN, and ImageNet), and compared with several state-of-the-art implementations. Overall, SpecNet is able to reduce memory consumption by about 60% without significant loss of performance for all tested networks.

cs.CV↗

arXiv2020

Video Logo Retrieval based on local Features

Estimation of the frequency and duration of logos in videos is important and challenging in the advertisement industry as a way of estimating the impact of ad purchases. Since logos occupy only a small area in the videos, the popular methods of image retrieval could fail. This paper develops an algorithm called Video Logo Retrieval (VLR), which is an image-to-video retrieval algorithm based on the spatial distribution of local image descriptors that measure the distance between the query image (the logo) and a collection of video images. VLR uses local features to overcome the weakness of global feature-based models such as convolutional neural networks (CNN). Meanwhile, VLR is flexible and does not require training after setting some hyper-parameters. The performance of VLR is evaluated on two challenging open benchmark tasks (SoccerNet and Standford I2V), and compared with other state-of-the-art logo retrieval or detection algorithms. Overall, VLR shows significantly higher accuracy compared with the existing methods.

eess.IV↗

arXiv2019

Entangled multi-component 4D quantum Hall states from photonic crystal defects

Recently, there has been a drive towards the realization of topological phases beyond conventional electronic materials, including phases defined in more than three dimensions. We propose a versatile and experimentally realistic approach of realizing a large variety of multi-component topological phases in 2D photonic crystals with quasi-periodically modulated defects. With a length scale introduced by a background resonator lattice, the defects are found to host various effective orbitals of $s$, $p$ and $d$-type symmetries, thus providing a monolithic platform for realizing multi-component topological states without requiring separate internal degrees of freedom in the physical setup. Notably, by coupling the defect modulations diagonally, we report the novel realization of an ``entangled'' 4D QH phase which cannot be factorized into two copies of 2D QH phases, each described by the 1st Chern number. The structure of this non-factorizability can be quantified by a classical entanglement entropy inspired by quantum information theory. In another embodiment, we present 4D p-orbital nodal lines in a nonsymmorphic photonic lattice, hosting boundary states with an exotic manifold. Our simple and versatile approach holds the promise of novel topological optoelectronic and photonic applications such as one-way optical fibers.

cond-mat.mes-hall↗

arXiv2017

Non-factorizable 4D quantum Hall state from photonic crystal defects

In the recent years, there has been a drive towards the realization of topological phases beyond conventional electronic materials, including phases defined in more than three dimensions. We propose a way to realize 2nd Chern number topological phases with photonic crystals simply made up of defect resonators embedded within a regular lattice of resonators. In particular, through a novel quasiperiodic spatial modulations in the defect radii, a defect lattice possessing topologically nontrivial Chern bands with non-abelian berry curvature living in four-dimensional synthetic space is proposed. This system cannot be factorized by a direct product of two 1st Chern number models, distinguishing itself from the Hofstadter model. Such photonic systems can be easily experimentally realized with regular photonic crystals consisting of dielectric rods in air.

cond-mat.mes-hall↗

arXiv2016

Negative differential resistance and characteristic nonlinear electromagnetic response of a Topological Insulator

Materials exhibiting negative differential resistance have important applications in technologies involving microwave generation, which range from motion sensing to radio astronomy. Despite their usefulness, there has been few physical mechanisms giving rise to materials with such properties, i.e. GaAs employed in the Gunn diode. In this work, we show that negative differential resistance also generically arise in Dirac ring systems, an example of which has been experimentally observed in the surface states of Topological Insulators. This novel realization of negative differential resistance is based on a completely different physical mechanism from that of the Gunn effect, relying on the characteristic non-monotonicity of the response curve that remains robust in the presence of nonzero temperature, chemical potential, mass gap and impurity scattering. As such, it opens up new possibilities for engineering applications, such as frequency upconversion devices which are highly sought for terahertz signal generation. Our results may be tested with thin films of Bi2Se3 Topological Insulators, and are expected to hold qualitatively even in the absence of a strictly linear Dirac dispersion, as will be the case in more generic samples of Bi2Se3 and other materials with topologically nontrivial Fermi sea regions.

cond-mat.mes-hall↗