arXiv subjects

Jianfu Zhang

Publications and source records attributed to Jianfu Zhang.

At least 55 records · Page 3Linked to original sources

arXiv2023

The diffusion and scattering of accelerating particles in compressible MHD turbulence

We numerically study the diffusion and scattering of cosmic rays (CRs) together with their acceleration processes in the framework of the modern understanding of magnetohydrodynamic (MHD) turbulence. Based on the properties of compressible MHD turbulence obtained from observations and numerical experiments, we investigate the interaction of CRs with plasma modes. We find that (1) the gyroradius of particles exponentially increases with the acceleration timescale; (2) the momentum diffusion presents the power-law relationship with the gyroradius in the strong turbulence regime, and shows a plateau in the weak turbulence regime implying a stochastic acceleration process; (3) the spatial diffusion is dominated by the parallel diffusion in the sub-Alfvénic regime, while it is dominated by the perpendicular diffusion in the super-Alfvénic one; (4) as for the interaction of CRs with plasma modes, the particle acceleration is dominated by the fast mode in the high $β$ case, while in the low $β$ case, it is dominated by the fast and slow modes; (5) in the presence of acceleration, magnetosonic modes still play a critical role in diffusion and scattering processes of CRs, which is in good agreement with the earlier theoretical predictions.

astro-ph.HE↗

arXiv2023

Virtual Accessory Try-On via Keypoint Hallucination

The virtual try-on task refers to fitting the clothes from one image onto another portrait image. In this paper, we focus on virtual accessory try-on, which fits accessory (e.g., glasses, ties) onto a face or portrait image. Unlike clothing try-on, which relies on human silhouette as guidance, accessory try-on warps the accessory into an appropriate location and shape to generate a plausible composite image. In contrast to previous try-on methods that treat foreground (i.e., accessories) and background (i.e., human faces or bodies) equally, we propose a background-oriented network to utilize the prior knowledge of human bodies and accessories. Specifically, our approach learns the human body priors and hallucinates the target locations of specified foreground keypoints in the background. Then our approach will inject foreground information with accessory priors into the background UNet. Based on the hallucinated target locations, the warping parameters are calculated to warp the foreground. Moreover, this background-oriented network can also easily incorporate auxiliary human face/body semantic segmentation supervision to further boost performance. Experiments conducted on STRAT dataset validate the effectiveness of our proposed method.

cs.CV↗

arXiv2023

Taming the Power of Diffusion Models for High-Quality Virtual Try-On with Appearance Flow

Virtual try-on is a critical image synthesis task that aims to transfer clothes from one image to another while preserving the details of both humans and clothes. While many existing methods rely on Generative Adversarial Networks (GANs) to achieve this, flaws can still occur, particularly at high resolutions. Recently, the diffusion model has emerged as a promising alternative for generating high-quality images in various applications. However, simply using clothes as a condition for guiding the diffusion model to inpaint is insufficient to maintain the details of the clothes. To overcome this challenge, we propose an exemplar-based inpainting approach that leverages a warping module to guide the diffusion model's generation effectively. The warping module performs initial processing on the clothes, which helps to preserve the local details of the clothes. We then combine the warped clothes with clothes-agnostic person image and add noise as the input of diffusion model. Additionally, the warped clothes is used as local conditions for each denoising process to ensure that the resulting output retains as much detail as possible. Our approach, namely Diffusion-based Conditional Inpainting for Virtual Try-ON (DCI-VTON), effectively utilizes the power of the diffusion model, and the incorporation of the warping module helps to produce high-quality and realistic virtual try-on results. Experimental results on VITON-HD demonstrate the effectiveness and superiority of our method.

cs.CV↗

arXiv2023

Magnetic reconnection-driven turbulence and turbulent reconnection acceleration

This paper employs an MHD-PIC method to perform numerical simulations of magnetic reconnection-driven turbulence and turbulent reconnection acceleration of particles. Focusing on the dynamics of the magnetic reconnection, the properties of self-driven turbulence, and the behavior of particle acceleration, we find that: (1) when reaching a statistically steady state of the self-driven turbulence, the magnetic energy is almost released by 50\%, while the kinetic energy of the fluid increases by no more than 15\%. (2) the properties of reconnection-driven turbulence are more complex than the traditional turbulence driven by an external force. (3) the strong magnetic field tends to enhance the turbulent reconnection efficiency to accelerate particles more efficiently, resulting in a hard spectral energy distribution. Our study provides a particular perspective on understanding turbulence properties and turbulent reconnection-accelerated particles.

astro-ph.HE↗

arXiv2022

Deep Image Harmonization by Bridging the Reality Gap

Image harmonization has been significantly advanced with large-scale harmonization dataset. However, the current way to build dataset is still labor-intensive, which adversely affects the extendability of dataset. To address this problem, we propose to construct rendered harmonization dataset with fewer human efforts to augment the existing real-world dataset. To leverage both real-world images and rendered images, we propose a cross-domain harmonization network to bridge the domain gap between two domains. Moreover, we also employ well-designed style classifiers and losses to facilitate cross-domain knowledge transfer. Extensive experiments demonstrate the potential of using rendered images for image harmonization and the effectiveness of our proposed network.

cs.CV↗

arXiv2022

DeltaGAN: Towards Diverse Few-shot Image Generation with Sample-Specific Delta

Learning to generate new images for a novel category based on only a few images, named as few-shot image generation, has attracted increasing research interest. Several state-of-the-art works have yielded impressive results, but the diversity is still limited. In this work, we propose a novel Delta Generative Adversarial Network (DeltaGAN), which consists of a reconstruction subnetwork and a generation subnetwork. The reconstruction subnetwork captures intra-category transformation, i.e., "delta", between same-category pairs. The generation subnetwork generates sample-specific "delta" for an input image, which is combined with this input image to generate a new image within the same category. Besides, an adversarial delta matching loss is designed to link the above two subnetworks together. Extensive experiments on five few-shot image datasets demonstrate the effectiveness of our proposed method.

cs.CV↗

arXiv2022

DeltaGAN: Towards Diverse Few-shot Image Generation with Sample-Specific Delta

Learning to generate new images for a novel category based on only a few images, named as few-shot image generation, has attracted increasing research interest. Several state-of-the-art works have yielded impressive results, but the diversity is still limited. In this work, we propose a novel Delta Generative Adversarial Network (DeltaGAN), which consists of a reconstruction subnetwork and a generation subnetwork. The reconstruction subnetwork captures intra-category transformation, i.e., delta, between same-category pairs. The generation subnetwork generates sample-specific delta for an input image, which is combined with this input image to generate a new image within the same category. Besides, an adversarial delta matching loss is designed to link the above two subnetworks together. Extensive experiments on six benchmark datasets demonstrate the effectiveness of our proposed method. Our code is available at https://github.com/bcmi/DeltaGAN-Few-Shot-Image-Generation.

cs.CV↗

arXiv2022

Few-shot Image Generation Using Discrete Content Representation

Few-shot image generation and few-shot image translation are two related tasks, both of which aim to generate new images for an unseen category with only a few images. In this work, we make the first attempt to adapt few-shot image translation method to few-shot image generation task. Few-shot image translation disentangles an image into style vector and content map. An unseen style vector can be combined with different seen content maps to produce different images. However, it needs to store seen images to provide content maps and the unseen style vector may be incompatible with seen content maps. To adapt it to few-shot image generation task, we learn a compact dictionary of local content vectors via quantizing continuous content maps into discrete content maps instead of storing seen images. Furthermore, we model the autoregressive distribution of discrete content map conditioned on style vector, which can alleviate the incompatibility between content map and style vector. Qualitative and quantitative results on three real datasets demonstrate that our model can produce images of higher diversity and fidelity for unseen categories than previous methods.

cs.CV↗

arXiv2022

Shadow Generation for Composite Image in Real-world Scenes

Image composition targets at inserting a foreground object into a background image. Most previous image composition methods focus on adjusting the foreground to make it compatible with background while ignoring the shadow effect of foreground on the background. In this work, we focus on generating plausible shadow for the foreground object in the composite image. First, we contribute a real-world shadow generation dataset DESOBA by generating synthetic composite images based on paired real images and deshadowed images. Then, we propose a novel shadow generation network SGRNet, which consists of a shadow mask prediction stage and a shadow filling stage. In the shadow mask prediction stage, foreground and background information are thoroughly interacted to generate foreground shadow mask. In the shadow filling stage, shadow parameters are predicted to fill the shadow area. Extensive experiments on our DESOBA dataset and real composite images demonstrate the effectiveness of our proposed method. Our dataset and code are available at https://github.com/bcmi/Object-Shadow-Generation-Dataset-DESOBA.

cs.CV↗

arXiv2022

Measurement of MHD turbulence properties by synchrotron radiation techniques

It is well-known that magnetohydrodynamic (MHD) turbulence is ubiquitous in astrophysical environments. The correct understanding of the fundamental properties of MHD turbulence is a prerequisite for revealing many key astrophysical processes. The development of observation-based measurement techniques has significantly promoted MHD turbulence theory and its implications in astrophysics. After describing the modern understanding of MHD turbulence based on theoretical analysis and direct numerical simulations, we review recent developments related to synchrotron fluctuation techniques. Specifically, we comment on the validation of synchrotron fluctuation techniques and the measurement performance of several properties of magnetic turbulence based on data cubes from MHD turbulence simulations and observations. Furthermore, we propose to strengthen the studies of the magnetization and 3D magnetic field structure's measurements of interstellar turbulence. At the same time, we also discuss the prospects of new techniques for measuring magnetic field properties and understanding astrophysical processes, using a large number of data cubes from the Low-Frequency Array (LOFAR) and the Square Kilometre Array (SKA).

astro-ph.HE↗

arXiv2021

Energetic Particle Acceleration in Compressible Magnetohydrodynamic Turbulence

Magnetohydrodynamic (MHD) turbulence is an important agent of energetic particle acceleration. Focusing on the compressible properties of magnetic turbulence, we adopt test particle method to study the particle acceleration from Alfvén, slow and fast modes in four turbulence regimes that may appear in a realistic astrophysical environment. Our studies show that (1) the second-order Fermi mechanism drives the acceleration of particles in the cascade processes of three modes by particle-turbulence interactions, regardless of whether the shock wave appears; (2) not only can the power spectra of maximum acceleration rates reveal the inertial range of compressible turbulence, but also recover the scaling and energy ratio relationship between the modes; (3) fast mode dominates the acceleration of particles, especially in the case of super-Alfvénic and supersonic turbulence, slow mode dominates the acceleration for sub-Alfvénic turbulence in the very high energy range, and the acceleration of Alfvén mode is significant at the early stage of the acceleration; (4) particle acceleration from three modes results in a power-law distribution in the certain range of evolution time. From the perspective of particle-wave mode interaction, this paper promotes the understanding for both the properties of turbulence and the behavior of particle acceleration, which will help insight into astrophysical processes involved in MHD turbulence.

astro-ph.HE↗

arXiv2021

BargainNet: Background-Guided Domain Translation for Image Harmonization

Image composition is a fundamental operation in image editing field. However, unharmonious foreground and background downgrade the quality of composite image. Image harmonization, which adjusts the foreground to improve the consistency, is an essential yet challenging task. Previous deep learning based methods mainly focus on directly learning the mapping from composite image to real image, while ignoring the crucial guidance role that background plays. In this work, with the assumption that the foreground needs to be translated to the same domain as background, we formulate image harmonization task as background-guided domain translation. Therefore, we propose an image harmonization network with a novel domain code extractor and well-tailored triplet losses, which could capture the background domain information to guide the foreground harmonization. Extensive experiments on the existing image harmonization benchmark demonstrate the effectiveness of our proposed method. Code is available at https://github.com/bcmi/BargainNet.

cs.CV↗

arXiv2020

Disentangled Information Bottleneck

The information bottleneck (IB) method is a technique for extracting information that is relevant for predicting the target random variable from the source random variable, which is typically implemented by optimizing the IB Lagrangian that balances the compression and prediction terms. However, the IB Lagrangian is hard to optimize, and multiple trials for tuning values of Lagrangian multiplier are required. Moreover, we show that the prediction performance strictly decreases as the compression gets stronger during optimizing the IB Lagrangian. In this paper, we implement the IB method from the perspective of supervised disentangling. Specifically, we introduce Disentangled Information Bottleneck (DisenIB) that is consistent on compressing source maximally without target prediction performance loss (maximum compression). Theoretical and experimental results demonstrate that our method is consistent on maximum compression, and performs well in terms of generalization, robustness to adversarial attack, out-of-distribution detection, and supervised disentangling.

cs.LG↗

arXiv2020

DoveNet: Deep Image Harmonization via Domain Verification

Image composition is an important operation in image processing, but the inconsistency between foreground and background significantly degrades the quality of composite image. Image harmonization, aiming to make the foreground compatible with the background, is a promising yet challenging task. However, the lack of high-quality publicly available dataset for image harmonization greatly hinders the development of image harmonization techniques. In this work, we contribute an image harmonization dataset iHarmony4 by generating synthesized composite images based on COCO (resp., Adobe5k, Flickr, day2night) dataset, leading to our HCOCO (resp., HAdobe5k, HFlickr, Hday2night) sub-dataset. Moreover, we propose a new deep image harmonization method DoveNet using a novel domain verification discriminator, with the insight that the foreground needs to be translated to the same domain as background. Extensive experiments on our constructed dataset demonstrate the effectiveness of our proposed method. Our dataset and code are available at https://github.com/bcmi/Image_Harmonization_Datasets.

cs.CV↗

arXiv2020

F2GAN: Fusing-and-Filling GAN for Few-shot Image Generation

In order to generate images for a given category, existing deep generative models generally rely on abundant training images. However, extensive data acquisition is expensive and fast learning ability from limited data is necessarily required in real-world applications. Also, these existing methods are not well-suited for fast adaptation to a new category. Few-shot image generation, aiming to generate images from only a few images for a new category, has attracted some research interest. In this paper, we propose a Fusing-and-Filling Generative Adversarial Network (F2GAN) to generate realistic and diverse images for a new category with only a few images. In our F2GAN, a fusion generator is designed to fuse the high-level features of conditional images with random interpolation coefficients, and then fills in attended low-level details with non-local attention module to produce a new image. Moreover, our discriminator can ensure the diversity of generated images by a mode seeking loss and an interpolation regression loss. Extensive experiments on five datasets demonstrate the effectiveness of our proposed method for few-shot image generation.

cs.CV↗

arXiv2020

Image Harmonization Dataset iHarmony4: HCOCO, HAdobe5k, HFlickr, and Hday2night

Image composition is an important operation in image processing, but the inconsistency between foreground and background significantly degrades the quality of composite image. Image harmonization, which aims to make the foreground compatible with the background, is a promising yet challenging task. However, the lack of high-quality public dataset for image harmonization, which significantly hinders the development of image harmonization techniques. Therefore, we contribute an image harmonization dataset iHarmony4 by generating synthesized composite images based on existing COCO (resp., Adobe5k, day2night) dataset, leading to our HCOCO (resp., HAdobe5k, Hday2night) sub-dataset. To enrich the diversity of our dataset, we also generate synthesized composite images based on our collected Flick images, leading to our HFlickr sub-dataset. The image harmonization dataset iHarmony4 is released at https://github.com/bcmi/Image_Harmonization_Datasets.

cs.CV↗

arXiv2020

MatchingGAN: Matching-based Few-shot Image Generation

To generate new images for a given category, most deep generative models require abundant training images from this category, which are often too expensive to acquire. To achieve the goal of generation based on only a few images, we propose matching-based Generative Adversarial Network (GAN) for few-shot generation, which includes a matching generator and a matching discriminator. Matching generator can match random vectors with a few conditional images from the same category and generate new images for this category based on the fused features. The matching discriminator extends conventional GAN discriminator by matching the feature of generated image with the fused feature of conditional images. Extensive experiments on three datasets demonstrate the effectiveness of our proposed method.

cs.CV↗

arXiv2020

Beyond without Forgetting: Multi-Task Learning for Classification with Disjoint Datasets

Multi-task Learning (MTL) for classification with disjoint datasets aims to explore MTL when one task only has one labeled dataset. In existing methods, for each task, the unlabeled datasets are not fully exploited to facilitate this task. Inspired by semi-supervised learning, we use unlabeled datasets with pseudo labels to facilitate each task. However, there are two major issues: 1) the pseudo labels are very noisy; 2) the unlabeled datasets and the labeled dataset for each task has considerable data distribution mismatch. To address these issues, we propose our MTL with Selective Augmentation (MTL-SA) method to select the training samples in unlabeled datasets with confident pseudo labels and close data distribution to the labeled dataset. Then, we use the selected training samples to add information and use the remaining training samples to preserve information. Extensive experiments on face-centric and human-centric applications demonstrate the effectiveness of our MTL-SA method.

cs.CV↗