arXiv subjects

Jiansheng Chen

Publications and source records attributed to Jiansheng Chen.

At least 55 records · Page 3Linked to original sources

arXiv2024

DomainStudio: Fine-Tuning Diffusion Models for Domain-Driven Image Generation using Limited Data

Denoising diffusion probabilistic models (DDPMs) have been proven capable of synthesizing high-quality images with remarkable diversity when trained on large amounts of data. Typical diffusion models and modern large-scale conditional generative models like text-to-image generative models are vulnerable to overfitting when fine-tuned on extremely limited data. Existing works have explored subject-driven generation using a reference set containing a few images. However, few prior works explore DDPM-based domain-driven generation, which aims to learn the common features of target domains while maintaining diversity. This paper proposes a novel DomainStudio approach to adapt DDPMs pre-trained on large-scale source datasets to target domains using limited data. It is designed to keep the diversity of subjects provided by source domains and get high-quality and diverse adapted samples in target domains. We propose to keep the relative distances between adapted samples to achieve considerable generation diversity. In addition, we further enhance the learning of high-frequency details for better generation quality. Our approach is compatible with both unconditional and conditional diffusion models. This work makes the first attempt to realize unconditional few-shot image generation with diffusion models, achieving better quality and greater diversity than current state-of-the-art GAN-based approaches. Moreover, this work also significantly relieves overfitting for conditional generation and realizes high-quality domain-driven generation, further expanding the applicable scenarios of modern large-scale text-to-image models.

cs.CV↗

arXiv2023

Few-shot Image Generation via Masked Discrimination

Few-shot image generation aims to generate images of high quality and great diversity with limited data. However, it is difficult for modern GANs to avoid overfitting when trained on only a few images. The discriminator can easily remember all the training samples and guide the generator to replicate them, leading to severe diversity degradation. Several methods have been proposed to relieve overfitting by adapting GANs pre-trained on large source domains to target domains using limited real samples. This work presents a novel approach to realize few-shot GAN adaptation via masked discrimination. Random masks are applied to features extracted by the discriminator from input images. We aim to encourage the discriminator to judge various images which share partially common features with training samples as realistic. Correspondingly, the generator is guided to generate diverse images instead of replicating training samples. In addition, we employ a cross-domain consistency loss for the discriminator to keep relative distances between generated samples in its feature space. It strengthens global image discrimination and guides adapted GANs to preserve more information learned from source domains for higher image quality. The effectiveness of our approach is demonstrated both qualitatively and quantitatively with higher quality and greater diversity on a series of few-shot image generation tasks than prior methods.

cs.CV↗

arXiv2023

Learning a Graph Neural Network with Cross Modality Interaction for Image Fusion

Infrared and visible image fusion has gradually proved to be a vital fork in the field of multi-modality imaging technologies. In recent developments, researchers not only focus on the quality of fused images but also evaluate their performance in downstream tasks. Nevertheless, the majority of methods seldom put their eyes on the mutual learning from different modalities, resulting in fused images lacking significant details and textures. To overcome this issue, we propose an interactive graph neural network (GNN)-based architecture between cross modality for fusion, called IGNet. Specifically, we first apply a multi-scale extractor to achieve shallow features, which are employed as the necessary input to build graph structures. Then, the graph interaction module can construct the extracted intermediate features of the infrared/visible branch into graph structures. Meanwhile, the graph structures of two branches interact for cross-modality and semantic learning, so that fused images can maintain the important feature expressions and enhance the performance of downstream tasks. Besides, the proposed leader nodes can improve information propagation in the same modality. Finally, we merge all graph features to get the fusion result. Extensive experiments on different datasets (TNO, MFNet and M3FD) demonstrate that our IGNet can generate visually appealing fused images while scoring averagely 2.59% mAP@.5 and 7.77% mIoU higher in detection and segmentation than the compared state-of-the-art methods. The source code of the proposed IGNet can be available at https://github.com/lok-18/IGNet.

cs.CV↗

arXiv2023

Few-shot 3D Shape Generation

Realistic and diverse 3D shape generation is helpful for a wide variety of applications such as virtual reality, gaming, and animation. Modern generative models, such as GANs and diffusion models, learn from large-scale datasets and generate new samples following similar data distributions. However, when training data is limited, deep neural generative networks overfit and tend to replicate training samples. Prior works focus on few-shot image generation to produce high-quality and diverse results using a few target images. Unfortunately, abundant 3D shape data is typically hard to obtain as well. In this work, we make the first attempt to realize few-shot 3D shape generation by adapting generative models pre-trained on large source domains to target domains using limited data. To relieve overfitting and keep considerable diversity, we propose to maintain the probability distributions of the pairwise relative distances between adapted samples at feature-level and shape-level during domain adaptation. Our approach only needs the silhouettes of few-shot target samples as training data to learn target geometry distributions and achieve generated shapes with diverse topology and textures. Moreover, we introduce several metrics to evaluate the quality and diversity of few-shot 3D shape generation. The effectiveness of our approach is demonstrated qualitatively and quantitatively under a series of few-shot 3D shape adaptation setups.

cs.CV↗

arXiv2023

Few-shot Image Generation with Diffusion Models

Denoising diffusion probabilistic models (DDPMs) have been proven capable of synthesizing high-quality images with remarkable diversity when trained on large amounts of data. However, to our knowledge, few-shot image generation tasks have yet to be studied with DDPM-based approaches. Modern approaches are mainly built on Generative Adversarial Networks (GANs) and adapt models pre-trained on large source domains to target domains using a few available samples. In this paper, we make the first attempt to study when do DDPMs overfit and suffer severe diversity degradation as training data become scarce. Then we fine-tune DDPMs pre-trained on large source domains to solve the overfitting problem when training data is limited. Although the directly fine-tuned models accelerate convergence and improve generation quality and diversity compared with training from scratch, they still fail to retain some diverse features and can only produce coarse images. Therefore, we design a DDPM pairwise adaptation (DDPM-PA) approach to optimize few-shot DDPM domain adaptation. DDPM-PA efficiently preserves information learned from source domains by keeping the relative pairwise distances between generated samples during adaptation. Besides, DDPM-PA enhances the learning of high-frequency details from source models and limited training data. DDPM-PA further improves generation quality and diversity and achieves results better than current state-of-the-art GAN-based approaches. We demonstrate the effectiveness of our approach on a series of few-shot image generation tasks qualitatively and quantitatively.

cs.CV↗

arXiv2023

MotionVideoGAN: A Novel Video Generator Based on the Motion Space Learned from Image Pairs

Video generation has achieved rapid progress benefiting from high-quality renderings provided by powerful image generators. We regard the video synthesis task as generating a sequence of images sharing the same contents but varying in motions. However, most previous video synthesis frameworks based on pre-trained image generators treat content and motion generation separately, leading to unrealistic generated videos. Therefore, we design a novel framework to build the motion space, aiming to achieve content consistency and fast convergence for video generation. We present MotionVideoGAN, a novel video generator synthesizing videos based on the motion space learned by pre-trained image pair generators. Firstly, we propose an image pair generator named MotionStyleGAN to generate image pairs sharing the same contents and producing various motions. Then we manage to acquire motion codes to edit one image in the generated image pairs and keep the other unchanged. The motion codes help us edit images within the motion space since the edited image shares the same contents with the other unchanged one in image pairs. Finally, we introduce a latent code generator to produce latent code sequences using motion codes for video generation. Our approach achieves state-of-the-art performance on the most complex video dataset ever used for unconditional video generation evaluation, UCF101.

cs.CV↗

arXiv2023

Gestalt-Guided Image Understanding for Few-Shot Learning

Due to the scarcity of available data, deep learning does not perform well on few-shot learning tasks. However, human can quickly learn the feature of a new category from very few samples. Nevertheless, previous work has rarely considered how to mimic human cognitive behavior and apply it to few-shot learning. This paper introduces Gestalt psychology to few-shot learning and proposes Gestalt-Guided Image Understanding, a plug-and-play method called GGIU. Referring to the principle of totality and the law of closure in Gestalt psychology, we design Totality-Guided Image Understanding and Closure-Guided Image Understanding to extract image features. After that, a feature estimation module is used to estimate the accurate features of images. Extensive experiments demonstrate that our method can improve the performance of existing models effectively and flexibly without retraining or fine-tuning. Our code is released on https://github.com/skingorz/GGIU.

cs.CV↗

arXiv2023

Distribution Aligned Feature Clustering for Zero-Shot Sketch-Based Image Retrieval

Zero-Shot Sketch-Based Image Retrieval (ZS-SBIR) is a challenging cross-modal retrieval task. In prior arts, the retrieval is conducted by sorting the distance between the query sketch and each image in the gallery. However, the domain gap and the zero-shot setting make neural networks hard to generalize. This paper tackles the challenges from a new perspective: utilizing gallery image features. We propose a Cluster-then-Retrieve (ClusterRetri) method that performs clustering on the gallery images and uses the cluster centroids as proxies for retrieval. Furthermore, a distribution alignment loss is proposed to align the image and sketch features with a common Gaussian distribution, reducing the domain gap. Despite its simplicity, our proposed method outperforms the state-of-the-art methods by a large margin on popular datasets, e.g., up to 31% and 39% relative improvement of mAP@all on the Sketchy and TU-Berlin datasets.

cs.CV↗

arXiv2020

Shaping Deep Feature Space towards Gaussian Mixture for Visual Classification

The softmax cross-entropy loss function has been widely used to train deep models for various tasks. In this work, we propose a Gaussian mixture (GM) loss function for deep neural networks for visual classification. Unlike the softmax cross-entropy loss, our method explicitly shapes the deep feature space towards a Gaussian Mixture distribution. With a classification margin and a likelihood regularization, the GM loss facilitates both high classification performance and accurate modeling of the feature distribution. The GM loss can be readily used to distinguish abnormal inputs, such as the adversarial examples, based on the discrepancy between feature distributions of the inputs and the training set. Furthermore, theoretical analysis shows that a symmetric feature space can be achieved by using the GM loss, which enables the models to perform robustly against adversarial attacks. The proposed model can be implemented easily and efficiently without using extra trainable parameters. Extensive evaluations demonstrate that the proposed method performs favorably not only on image classification but also on robust detection of adversarial examples generated by strong attacks under different threat models.

cs.CV↗

arXiv2020

Teacher-Critical Training Strategies for Image Captioning

Existing image captioning models are usually trained by cross-entropy (XE) loss and reinforcement learning (RL), which set ground-truth words as hard targets and force the captioning model to learn from them. However, the widely adopted training strategies suffer from misalignment in XE training and inappropriate reward assignment in RL training. To tackle these problems, we introduce a teacher model that serves as a bridge between the ground-truth caption and the caption model by generating some easier-to-learn word proposals as soft targets. The teacher model is constructed by incorporating the ground-truth image attributes into the baseline caption model. To effectively learn from the teacher model, we propose Teacher-Critical Training Strategies (TCTS) for both XE and RL training to facilitate better learning processes for the caption model. Experimental evaluations of several widely adopted caption models on the benchmark MSCOCO dataset show the proposed TCTS comprehensively enhances most evaluation metrics, especially the Bleu and Rouge-L scores, in both training stages. TCTS is able to achieve to-date the best published single model Bleu-4 and Rouge-L performances of 40.2% and 59.4% on the MSCOCO Karpathy test split. Our codes and pre-trained models will be open-sourced.

cs.CV↗

arXiv2018

Rethinking Feature Distribution for Loss Functions in Image Classification

We propose a large-margin Gaussian Mixture (L-GM) loss for deep neural networks in classification tasks. Different from the softmax cross-entropy loss, our proposal is established on the assumption that the deep features of the training set follow a Gaussian Mixture distribution. By involving a classification margin and a likelihood regularization, the L-GM loss facilitates both a high classification performance and an accurate modeling of the training feature distribution. As such, the L-GM loss is superior to the softmax loss and its major variants in the sense that besides classification, it can be readily used to distinguish abnormal inputs, such as the adversarial examples, based on their features' likelihood to the training feature distribution. Extensive experiments on various recognition benchmarks like MNIST, CIFAR, ImageNet and LFW, as well as on adversarial examples demonstrate the effectiveness of our proposal.

cs.CV↗

arXiv2017

Towards End-to-End Face Recognition through Alignment Learning

Plenty of effective methods have been proposed for face recognition during the past decade. Although these methods differ essentially in many aspects, a common practice of them is to specifically align the facial area based on the prior knowledge of human face structure before feature extraction. In most systems, the face alignment module is implemented independently. This has actually caused difficulties in the designing and training of end-to-end face recognition models. In this paper we study the possibility of alignment learning in end-to-end face recognition, in which neither prior knowledge on facial landmarks nor artificially defined geometric transformations are required. Specifically, spatial transformer layers are inserted in front of the feature extraction layers in a Convolutional Neural Network (CNN) for face recognition. Only human identity clues are used for driving the neural network to automatically learn the most suitable geometric transformation and the most appropriate facial area for the recognition task. To ensure reproducibility, our model is trained purely on the publicly available CASIA-WebFace dataset, and is tested on the Labeled Face in the Wild (LFW) dataset. We have achieved a verification accuracy of 99.08\% which is comparable to state-of-the-art single model based methods.

cs.CV↗

arXiv2010

Determination of fundamental properties of an M31 globular cluster from main-sequence photometry

M31 globular cluster B379 is the first extragalactic cluster, the age of which was determined by main-sequence photometry. In this method, the age of a cluster is obtained by fitting its CMD with stellar evolutionary models. However, different stellar evolutionary models use different parameters of stellar evolution, such as range of stellar masses, different opacities and equations of state, and different recipes, and so on. So, it is interesting to check whether different stellar evolutionary models can give consistent results for the same cluster. Brown et al. (2004a) constrained the age of B379 by comparing its CMD with isochrones of the 2006 VandenBerg models. Using SSP models of BC03 and its multi-photometry, Ma et al. (2007) independently determined the age of B379, which is in good agreement with the determination of Brown et al. (2004a). The BC03 models are calculated based on the Padova evolutionary tracks. It is necessary to check whether the age of B379 which, being determined based on the Padova evolutionary tracks, is in agreement with the determination of Brown et al. (2004a). So, in this paper, we re-determine its age using isochrones of the Padova stellar evolutionary models. In addition, the metal abundance, the distance modulus, and the reddening value for B379 are also determined in this paper. The results obtained in this paper are consistent with the previous determinations, which including the age obtained by Brown et al. (2004a). So, this paper confirms the consistence of the age scale of B379 between the Padova isochrones and the 2006 VandenBerg isochrones, i.e. the results' comparison between Brown et al. (2004a) and Ma et al. (2007) is meaningful. The results obtained in this paper are: the metallicity [M/H]=-0.325, the age $τ=11.0\pm1.5$ Gyr, the reddening value E(B-V)=0.08, and the distance modulus $(m-M)_{0}=24.44\pm0.10$.

astro-ph.CO↗

arXiv2009

Optical Variability of the Radio Source J 1128+5925 -- II. Confirmation of its Optical Quietness

The source J 1128+5925 was found recently to show strong intraday variability at radio wavelengths and its radio variability may come from interstellar scintillation. In optical, the object was quiet in our 2007 monitoring session. Here we report the results of our new optical monitoring of this source in 2008. In addition to confirm our 2007 results, that the object did not display any clear variation on timescales from hour--day to month, we provide evidence that the object does not vary on timescale of one year, and it is probably intrinsically quiet in optical domain. Its very different behaviors in optical and radio regimes can be naturally explained if its strong radio variability comes from interstellar scintillation.

astro-ph.GA↗

arXiv2007

Optical Variability of the Radio Source J 1128+5925

Very recently, J 1128+5925 was found to show strong intraday variability at radio wavelengths and may be a new source with annual modulation of the timescale of its radio variability. Therefore, its radio variability can be best explained via interstellar scintillation. Here we present the properties of its optical variability for the first time after a monitoring program in 2007 May. Our observations indicate that in this period J 1128+5925 only showed trivial optical variability on internight timescale, and did not show any clear intranight variability. This behavior is quite different from its strong radio intraday variability. Either this object was in a quiescent state in optical in this period, or it is intrinsically not so active in optical as it is in radio regimes.

astro-ph↗

arXiv2007

Metal Abundance Properties of M81 Globular Cluster System

This paper is the third in the series of papers on M81 globular clusters. In this paper, we present spatial and metal abundance properties of 95 M81 globular clusters, which comprise nearly half of all the M81 globular cluster system. These globular clusters are divided into two M81 metallicity groups by a KMM test. Our results show that, the metal-rich clusters did not demonstrate a centrally concentrated spatial distribution as ones in M31, and metal-poor clusters tend to be less spatially concentrated. In other words, the distribution of the metal-rich clusters in M81 is not very similar to that of M31. Most of the metal-rich clusters distribute at projected radii of 4-8 kpc. It is also noted that the metal-rich clusters distribute within the inner 20 kpc, and the metal-poor ones do out to radii of ~40 kpc. Like our Galaxy and M31, the metallicity distribution of globular clusters in M81 along galactocentric radius suggests that some dissipation occurred during the formation of the globular cluster system, i.e. smooth, pressure-supported collapse models of galaxies are unlikely to produce such radial distribution of metallicity presented in this paper. There is not evident correlation between globular cluster luminosity and metallicity in M81 globular clusters. The overwhelming conclusion of this paper seems to be that a more complete and thorough cluster search is needed in M81.

astro-ph↗

arXiv2006

Optical Monitoring of BL Lacertae Object S5 0716+714 with a Novel Multi-Peak Interference Filter

We at first introduce a novel photometric system, which consists of a Schmidt telescope, an objective prism, a CCD camera, and, especially, a multi-peak interference filter. The multi-peak interference filter enables light in multi passbands to pass through it simultaneously. The light in different passbands is differentially refracted by the objective prism and is focused on the CCD separately, so we have multi "images" for each object on the CCD frames. This system enables us to monitor blazars exactly simultaneously in multi wavebands on a single telescope, and to accurately trace the color change during the variation. We used this novel system to monitor the BL Lacertae object S5 0716+714 during 2006 January and February and achieved a very high temporal resolution. The object was very bright and very active during this period. Two strong flares were observed, with variation amplitudes of about 0.8 and 0.6 mags in the $V'$ band, respectively. Strong bluer-when-brighter correlations were found for both internight and intranight variations. No apparent time lag was observed between the $V'$- and $R'$-band variations, and the observed bluer-when-brighter chromatism may be mainly attributed to the larger variation amplitude at shorter wavelength. In addition to the bluer-when-brighter trend, the object also showed a bluer color when it was more active. The observed variability and its color behaviors are consistent with the shock-in-jet model.

astro-ph↗

arXiv2006

Optical Monitoring of BL Lacertae Object OJ 287: a 40-Day Period?

We present the results of our optical monitoring of the BL Lacertae object OJ 287 during the first half of 2005. The source did not show large-amplitude variations during this period and was in a relatively quiescent state. A possible period of 40 days was derived from its light curves in three BATC wavebands. A bluer-when-brighter chromatism was discovered, which is different from the extremely stable color during the outburst in 1994--96. The different color behaviors imply different variation mechanisms in the two states. We then re-visited the optical data on OJ 287 from the OJ-94 project and found as well a probable period of 40 days in its optical variability during the late-1994 outburst. The results suggest that two components contribute to the variability of OJ 287 during its outburst state. The first component is the normal {\sl blazar} variation. This component has an amplitude similar to that of the quiescent state and also may share a similar periodicity. The second component can be taken as a `low-frequency modulation' to the first component. It may be induced by the interaction of the assumed binary black holes in the center of this object. The 40-day period may be related to the helical structure of the magnetic field at the base of the jet, or to the orbital motion close to the central primary black hole.

astro-ph↗