arXiv subjects

Cong Bai

Publications and source records attributed to Cong Bai.

28 records · Page 2Linked to original sources

arXiv2024

Direction-Oriented Visual-semantic Embedding Model for Remote Sensing Image-text Retrieval

Image-text retrieval has developed rapidly in recent years. However, it is still a challenge in remote sensing due to visual-semantic imbalance, which leads to incorrect matching of non-semantic visual and textual features. To solve this problem, we propose a novel Direction-Oriented Visual-semantic Embedding Model (DOVE) to mine the relationship between vision and language. Our highlight is to conduct visual and textual representations in latent space, directing them as close as possible to a redundancy-free regional visual representation. Concretely, a Regional-Oriented Attention Module (ROAM) adaptively adjusts the distance between the final visual and textual embeddings in the latent semantic space, oriented by regional visual features. Meanwhile, a lightweight Digging Text Genome Assistant (DTGA) is designed to expand the range of tractable textual representation and enhance global word-level semantic connections using less attention operations. Ultimately, we exploit a global visual-semantic constraint to reduce single visual dependency and serve as an external constraint for the final visual and textual representations. The effectiveness and superiority of our method are verified by extensive experiments including parameter evaluation, quantitative comparison, ablation studies and visual analysis, on two benchmark datasets, RSICD and RSITMD.

cs.CV↗

arXiv2024

DERs-Aided Blackstart and Load Restoration Framework for Distribution Systems Considering Synchronization and Frequency Security Constraints

Extreme weather events have led to long-duration outages in the distribution system (DS), necessitating novel approaches to blackstart and restore the system. Existing blackstart solutions utilize blackstart units to establish multiple microgrids, sequentially energize non-blackstart units, and restore loads. However, these approaches often result in isolated microgrids. In DER-aided blackstart, the continuous operation of these microgrids is uncertain due to the finite energy capacity of commonly used blackstart units, such as battery energy storage (BES)-based grid-forming inverters (GFMIs). To address this issue, this article proposes a holistic blackstart and restoration framework that incorporates synchronization between microgrids and the entire DS with the transmission grid (TG). To support synchronization, we leveraged virtual synchronous generator-based control for GFMIs to estimate their frequency response to load pick-up events using only initial/final quasi-steady-state points. Subsequently, a synchronization switching condition was developed to model synchronizing switches, aligning them seamlessly with a linearized branch flow problem. Finally, we designed a bottom-up blackstart and restoration framework that considers the switching structure of the DS, energizing/synchronizing switches, DERs with grid-following inverters, and BES-based GFMIs with frequency security constraints. The proposed framework is validated in IEEE-123-bus system, considering cases with two and four GFMIs under various TG recovery instants.

eess.SY↗

arXiv2024

Little Strokes Fell Great Oaks: Boosting the Hierarchical Features for Multi-exposure Image Fusion

In recent years, deep learning networks have made remarkable strides in the domain of multi-exposure image fusion. Nonetheless, prevailing approaches often involve directly feeding over-exposed and under-exposed images into the network, which leads to the under-utilization of inherent information present in the source images. Additionally, unsupervised techniques predominantly employ rudimentary weighted summation for color channel processing, culminating in an overall desaturated final image tone. To partially mitigate these issues, this study proposes a gamma correction module specifically designed to fully leverage latent information embedded within source images. Furthermore, a modified transformer block, embracing with self-attention mechanisms, is introduced to optimize the fusion process. Ultimately, a novel color enhancement algorithm is presented to augment image saturation while preserving intricate details. The source code is available at https://github.com/ZhiyingDu/BHFMEF.

cs.CV↗

arXiv2023

Towards General and Fast Video Derain via Knowledge Distillation

As a common natural weather condition, rain can obscure video frames and thus affect the performance of the visual system, so video derain receives a lot of attention. In natural environments, rain has a wide variety of streak types, which increases the difficulty of the rain removal task. In this paper, we propose a Rain Review-based General video derain Network via knowledge distillation (named RRGNet) that handles different rain streak types with one pre-training weight. Specifically, we design a frame grouping-based encoder-decoder network that makes full use of the temporal information of the video. Further, we use the old task model to guide the current model in learning new rain streak types while avoiding forgetting. To consolidate the network's ability to derain, we design a rain review module to play back data from old tasks for the current model. The experimental results show that our developed general method achieves the best results in terms of running speed and derain effect.

cs.CV↗

arXiv2023

A Generalized Physical-knowledge-guided Dynamic Model for Underwater Image Enhancement

Underwater images often suffer from color distortion and low contrast resulting in various image types, due to the scattering and absorption of light by water. While it is difficult to obtain high-quality paired training samples with a generalized model. To tackle these challenges, we design a Generalized Underwater image enhancement method via a Physical-knowledge-guided Dynamic Model (short for GUPDM), consisting of three parts: Atmosphere-based Dynamic Structure (ADS), Transmission-guided Dynamic Structure (TDS), and Prior-based Multi-scale Structure (PMS). In particular, to cover complex underwater scenes, this study changes the global atmosphere light and the transmission to simulate various underwater image types (e.g., the underwater image color ranging from yellow to blue) through the formation model. We then design ADS and TDS that use dynamic convolutions to adaptively extract prior information from underwater images and generate parameters for PMS. These two modules enable the network to select appropriate parameters for various water types adaptively. Besides, the multi-scale feature extraction module in PMS uses convolution blocks with different kernel sizes and obtains weights for each feature map via channel attention block and fuses them to boost the receptive field of the network. The source code will be available at \href{https://github.com/shiningZZ/GUPDM}{https://github.com/shiningZZ/GUPDM}.

cs.CV↗

arXiv2023

Transmission and Color-guided Network for Underwater Image Enhancement

In recent years, with the continuous development of the marine industry, underwater image enhancement has attracted plenty of attention. Unfortunately, the propagation of light in water will be absorbed by water bodies and scattered by suspended particles, resulting in color deviation and low contrast. To solve these two problems, we propose an Adaptive Transmission and Dynamic Color guided network (named ATDCnet) for underwater image enhancement. In particular, to exploit the knowledge of physics, we design an Adaptive Transmission-directed Module (ATM) to better guide the network. To deal with the color deviation problem, we design a Dynamic Color-guided Module (DCM) to post-process the enhanced image color. Further, we design an Encoder-Decoder-based Compensation (EDC) structure with attention and a multi-stage feature fusion mechanism to perform color restoration and contrast enhancement simultaneously. Extensive experiments demonstrate the state-of-the-art performance of the ATDCnet on multiple benchmark datasets.

cs.CV↗

arXiv2023

Histogram-guided Video Colorization Structure with Spatial-Temporal Connection

Video colorization, aiming at obtaining colorful and plausible results from grayish frames, has aroused a lot of interest recently. Nevertheless, how to maintain temporal consistency while keeping the quality of colorized results remains challenging. To tackle the above problems, we present a Histogram-guided Video Colorization with Spatial-Temporal connection structure (named ST-HVC). To fully exploit the chroma and motion information, the joint flow and histogram module is tailored to integrate the histogram and flow features. To manage the blurred and artifact, we design a combination scheme attending to temporal detail and flow feature combination. We further recombine the histogram, flow and sharpness features via a U-shape network. Extensive comparisons are conducted with several state-of-the-art image and video-based methods, demonstrating that the developed method achieves excellent performance both quantitatively and qualitatively in two video datasets.

cs.CV↗

arXiv2022

Intra-Modal Constraint Loss For Image-Text Retrieval

Cross-modal retrieval has drawn much attention in both computer vision and natural language processing domains. With the development of convolutional and recurrent neural networks, the bottleneck of retrieval across image-text modalities is no longer the extraction of image and text features but an efficient loss function learning in embedding space. Many loss functions try to closer pairwise features from heterogeneous modalities. This paper proposes a method for learning joint embedding of images and texts using an intra-modal constraint loss function to reduce the violation of negative pairs from the same homogeneous modality. Experimental results show that our approach outperforms state-of-the-art bi-directional image-text retrieval methods on Flickr30K and Microsoft COCO datasets. Our code is publicly available: https://github.com/CanonChen/IMC.

cs.CV↗

arXiv2022

MMINR: Multi-frame-to-Multi-frame Inference with Noise Resistance for Precipitation Nowcasting with Radar

Precipitation nowcasting based on radar echo maps is essential in meteorological research. Recently, Convolutional RNNs based methods dominate this field, but they cannot be solved by parallel computation resulting in longer inference time. FCN based methods adopt a multi-frame-to-single-frame inference (MSI) strategy to avoid this problem. They feedback into the model again to predict the next time step to get multi-frame nowcasting results in the prediction phase, which will lead to the accumulation of prediction errors. In addition, precipitation noise is a crucial factor contributing to high prediction errors because of its unpredictability. To address this problem, we propose a novel Multi-frame-to-Multi-frame Inference (MMI) model with Noise Resistance (NR) named MMINR. It avoids error accumulation and resists precipitation noiseś negative effect in parallel computation. NR contains a Noise Dropout Module (NDM) and a Semantic Restore Module (SRM). NDM deliberately dropout noise simple yet efficient, and SRM supplements semantic information of features to alleviate the problem of semantic information mistakenly lost by NDM. Experimental results demonstrate that MMINR can attain competitive scores compared with other SOTAs. The ablation experiments show that the proposed NDM and SRM can solve the aforementioned problems.

cs.CV↗

arXiv2022

ISDA: Position-Aware Instance Segmentation with Deformable Attention

Most instance segmentation models are not end-to-end trainable due to either the incorporation of proposal estimation (RPN) as a pre-processing or non-maximum suppression (NMS) as a post-processing. Here we propose a novel end-to-end instance segmentation method termed ISDA. It reshapes the task into predicting a set of object masks, which are generated via traditional convolution operation with learned position-aware kernels and features of objects. Such kernels and features are learned by leveraging a deformable attention network with multi-scale representation. Thanks to the introduced set-prediction mechanism, the proposed method is NMS-free. Empirically, ISDA outperforms Mask R-CNN (the strong baseline) by 2.6 points on MS-COCO, and achieves leading performance compared with recent models. Code will be available soon.

cs.CV↗