arXiv subjects

Zhonghua Wu

Publications and source records attributed to Zhonghua Wu.

36 records · Page 2Linked to original sources

arXiv2023

DI-Net : Decomposed Implicit Garment Transfer Network for Digital Clothed 3D Human

3D virtual try-on enjoys many potential applications and hence has attracted wide attention. However, it remains a challenging task that has not been adequately solved. Existing 2D virtual try-on methods cannot be directly extended to 3D since they lack the ability to perceive the depth of each pixel. Besides, 3D virtual try-on approaches are mostly built on the fixed topological structure and with heavy computation. To deal with these problems, we propose a Decomposed Implicit garment transfer network (DI-Net), which can effortlessly reconstruct a 3D human mesh with the newly try-on result and preserve the texture from an arbitrary perspective. Specifically, DI-Net consists of two modules: 1) A complementary warping module that warps the reference image to have the same pose as the source image through dense correspondence learning and sparse flow learning; 2) A geometry-aware decomposed transfer module that decomposes the garment transfer into image layout based transfer and texture based transfer, achieving surface and texture reconstruction by constructing pixel-aligned implicit functions. Experimental results show the effectiveness and superiority of our method in the 3D virtual try-on task, which can yield more high-quality results over other existing methods.

cs.CV↗

arXiv2023

CoactSeg: Learning from Heterogeneous Data for New Multiple Sclerosis Lesion Segmentation

New lesion segmentation is essential to estimate the disease progression and therapeutic effects during multiple sclerosis (MS) clinical treatments. However, the expensive data acquisition and expert annotation restrict the feasibility of applying large-scale deep learning models. Since single-time-point samples with all-lesion labels are relatively easy to collect, exploiting them to train deep models is highly desirable to improve new lesion segmentation. Therefore, we proposed a coaction segmentation (CoactSeg) framework to exploit the heterogeneous data (i.e., new-lesion annotated two-time-point data and all-lesion annotated single-time-point data) for new MS lesion segmentation. The CoactSeg model is designed as a unified model, with the same three inputs (the baseline, follow-up, and their longitudinal brain differences) and the same three outputs (the corresponding all-lesion and new-lesion predictions), no matter which type of heterogeneous data is being used. Moreover, a simple and effective relation regularization is proposed to ensure the longitudinal relations among the three outputs to improve the model learning. Extensive experiments demonstrate that utilizing the heterogeneous data and the proposed longitudinal relation constraint can significantly improve the performance for both new-lesion and all-lesion segmentation tasks. Meanwhile, we also introduce an in-house MS-23v1 dataset, including 38 Oceania single-time-point samples with all-lesion labels. Codes and the dataset are released at https://github.com/ycwu1997/CoactSeg.

eess.IV↗

arXiv2023

LCReg: Long-Tailed Image Classification with Latent Categories based Recognition

In this work, we tackle the challenging problem of long-tailed image recognition. Previous long-tailed recognition approaches mainly focus on data augmentation or re-balancing strategies for the tail classes to give them more attention during model training. However, these methods are limited by the small number of training images for the tail classes, which results in poor feature representations. To address this issue, we propose the Latent Categories based long-tail Recognition (LCReg) method. Our hypothesis is that common latent features shared by head and tail classes can be used to improve feature representation. Specifically, we learn a set of class-agnostic latent features shared by both head and tail classes, and then use semantic data augmentation on the latent features to implicitly increase the diversity of the training sample. We conduct extensive experiments on five long-tailed image recognition datasets, and the results show that our proposed method significantly improves the baselines.

cs.CV↗

arXiv2023

Harmonizing Base and Novel Classes: A Class-Contrastive Approach for Generalized Few-Shot Segmentation

Current methods for few-shot segmentation (FSSeg) have mainly focused on improving the performance of novel classes while neglecting the performance of base classes. To overcome this limitation, the task of generalized few-shot semantic segmentation (GFSSeg) has been introduced, aiming to predict segmentation masks for both base and novel classes. However, the current prototype-based methods do not explicitly consider the relationship between base and novel classes when updating prototypes, leading to a limited performance in identifying true categories. To address this challenge, we propose a class contrastive loss and a class relationship loss to regulate prototype updates and encourage a large distance between prototypes from different classes, thus distinguishing the classes from each other while maintaining the performance of the base classes. Our proposed approach achieves new state-of-the-art performance for the generalized few-shot segmentation task on PASCAL VOC and MS COCO datasets.

cs.CV↗

arXiv2022

Long-tailed Recognition by Learning from Latent Categories

In this work, we address the challenging task of long-tailed image recognition. Previous long-tailed recognition methods commonly focus on the data augmentation or re-balancing strategy of the tail classes to give more attention to tail classes during the model training. However, due to the limited training images for tail classes, the diversity of tail class images is still restricted, which results in poor feature representations. In this work, we hypothesize that common latent features among the head and tail classes can be used to give better feature representation. Motivated by this, we introduce a Latent Categories based long-tail Recognition (LCReg) method. Specifically, we propose to learn a set of class-agnostic latent features shared among the head and tail classes. Then, we implicitly enrich the training sample diversity via applying semantic data augmentation to the latent features. Extensive experiments on five long-tailed image recognition datasets demonstrate that our proposed LCReg is able to significantly outperform previous methods and achieve state-of-the-art results.

cs.CV↗

arXiv2022

Dual Adaptive Transformations for Weakly Supervised Point Cloud Segmentation

Weakly supervised point cloud segmentation, i.e. semantically segmenting a point cloud with only a few labeled points in the whole 3D scene, is highly desirable due to the heavy burden of collecting abundant dense annotations for the model training. However, existing methods remain challenging to accurately segment 3D point clouds since limited annotated data may lead to insufficient guidance for label propagation to unlabeled data. Considering the smoothness-based methods have achieved promising progress, in this paper, we advocate applying the consistency constraint under various perturbations to effectively regularize unlabeled 3D points. Specifically, we propose a novel DAT (\textbf{D}ual \textbf{A}daptive \textbf{T}ransformations) model for weakly supervised point cloud segmentation, where the dual adaptive transformations are performed via an adversarial strategy at both point-level and region-level, aiming at enforcing the local and structural smoothness constraints on 3D point clouds. We evaluate our proposed DAT model with two popular backbones on the large-scale S3DIS and ScanNet-V2 datasets. Extensive experiments demonstrate that our model can effectively leverage the unlabeled 3D points and achieve significant performance gains on both datasets, setting new state-of-the-art performance for weakly supervised point cloud segmentation.

cs.CV↗

arXiv2022

Exploring Smoothness and Class-Separation for Semi-supervised Medical Image Segmentation

Semi-supervised segmentation remains challenging in medical imaging since the amount of annotated medical data is often scarce and there are many blurred pixels near the adhesive edges or in the low-contrast regions. To address the issues, we advocate to firstly constrain the consistency of pixels with and without strong perturbations to apply a sufficient smoothness constraint and further encourage the class-level separation to exploit the low-entropy regularization for the model training. Particularly, in this paper, we propose the SS-Net for semi-supervised medical image segmentation tasks, via exploring the pixel-level smoothness and inter-class separation at the same time. The pixel-level smoothness forces the model to generate invariant results under adversarial perturbations. Meanwhile, the inter-class separation encourages individual class features should approach their corresponding high-quality prototypes, in order to make each class distribution compact and separate different classes. We evaluated our SS-Net against five recent methods on the public LA and ACDC datasets. Extensive experimental results under two semi-supervised settings demonstrate the superiority of our proposed SS-Net model, achieving new state-of-the-art (SOTA) performance on both datasets. The code is available at https://github.com/ycwu1997/SS-Net.

eess.IV↗

arXiv2021

MV-TON: Memory-based Video Virtual Try-on network

With the development of Generative Adversarial Network, image-based virtual try-on methods have made great progress. However, limited work has explored the task of video-based virtual try-on while it is important in real-world applications. Most existing video-based virtual try-on methods usually require clothing templates and they can only generate blurred and low-resolution results. To address these challenges, we propose a Memory-based Video virtual Try-On Network (MV-TON), which seamlessly transfers desired clothes to a target person without using any clothing templates and generates high-resolution realistic videos. Specifically, MV-TON consists of two modules: 1) a try-on module that transfers the desired clothes from model images to frame images by pose alignment and region-wise replacing of pixels; 2) a memory refinement module that learns to embed the existing generated frames into the latent space as external memory for the following frame generation. Experimental results show the effectiveness of our method in the video virtual try-on task and its superiority over other existing methods.

cs.CV↗

arXiv2021

Learning Meta-class Memory for Few-Shot Semantic Segmentation

Currently, the state-of-the-art methods treat few-shot semantic segmentation task as a conditional foreground-background segmentation problem, assuming each class is independent. In this paper, we introduce the concept of meta-class, which is the meta information (e.g. certain middle-level features) shareable among all classes. To explicitly learn meta-class representations in few-shot segmentation task, we propose a novel Meta-class Memory based few-shot segmentation method (MM-Net), where we introduce a set of learnable memory embeddings to memorize the meta-class information during the base class training and transfer to novel classes during the inference stage. Moreover, for the $k$-shot scenario, we propose a novel image quality measurement module to select images from the set of support images. A high-quality class prototype could be obtained with the weighted sum of support image features based on the quality measure. Experiments on both PASCAL-$5^i$ and COCO dataset shows that our proposed method is able to achieve state-of-the-art results in both 1-shot and 5-shot settings. Particularly, our proposed MM-Net achieves 37.5\% mIoU on the COCO dataset in 1-shot setting, which is 5.1\% higher than the previous state-of-the-art.

cs.CV↗

arXiv2021

Remember What You have drawn: Semantic Image Manipulation with Memory

Image manipulation with natural language, which aims to manipulate images with the guidance of language descriptions, has been a challenging problem in the fields of computer vision and natural language processing (NLP). Currently, a number of efforts have been made for this task, but their performances are still distant away from generating realistic and text-conformed manipulated images. Therefore, in this paper, we propose a memory-based Image Manipulation Network (MIM-Net), where a set of memories learned from images is introduced to synthesize the texture information with the guidance of the textual description. We propose a two-stage network with an additional reconstruction stage to learn the latent memories efficiently. To avoid the unnecessary background changes, we propose a Target Localization Unit (TLU) to focus on the manipulation of the region mentioned by the text. Moreover, to learn a robust memory, we further propose a novel randomized memory training loss. Experiments on the four popular datasets show the better performance of our method compared to the existing ones.

cs.CV↗

arXiv2020

Exploring Bottom-up and Top-down Cues with Attentive Learning for Webly Supervised Object Detection

Fully supervised object detection has achieved great success in recent years. However, abundant bounding boxes annotations are needed for training a detector for novel classes. To reduce the human labeling effort, we propose a novel webly supervised object detection (WebSOD) method for novel classes which only requires the web images without further annotations. Our proposed method combines bottom-up and top-down cues for novel class detection. Within our approach, we introduce a bottom-up mechanism based on the well-trained fully supervised object detector (i.e. Faster RCNN) as an object region estimator for web images by recognizing the common objectiveness shared by base and novel classes. With the estimated regions on the web images, we then utilize the top-down attention cues as the guidance for region classification. Furthermore, we propose a residual feature refinement (RFR) block to tackle the domain mismatch between web domain and the target domain. We demonstrate our proposed method on PASCAL VOC dataset with three different novel/base splits. Without any target-domain novel-class images and annotations, our proposed webly supervised object detection model is able to achieve promising performance for novel classes. Moreover, we also conduct transfer learning experiments on large scale ILSVRC 2013 detection dataset and achieve state-of-the-art performance.

cs.CV↗

arXiv2019

M2E-Try On Net: Fashion from Model to Everyone

Most existing virtual try-on applications require clean clothes images. Instead, we present a novel virtual Try-On network, M2E-Try On Net, which transfers the clothes from a model image to a person image without the need of any clean product images. To obtain a realistic image of person wearing the desired model clothes, we aim to solve the following challenges: 1) non-rigid nature of clothes - we need to align poses between the model and the user; 2) richness in textures of fashion items - preserving the fine details and characteristics of the clothes is critical for photo-realistic transfer; 3) variation of identity appearances - it is required to fit the desired model clothes to the person identity seamlessly. To tackle these challenges, we introduce three key components, including the pose alignment network (PAN), the texture refinement network (TRN) and the fitting network (FTN). Since it is unlikely to gather image pairs of input person image and desired output image (i.e. person wearing the desired clothes), our framework is trained in a self-supervised manner to gradually transfer the poses and textures of the model's clothes to the desired appearance. In the experiments, we verify on the Deep Fashion dataset and MVC dataset that our method can generate photo-realistic images for the person to try-on the model clothes. Furthermore, we explore the model capability for different fashion items, including both upper and lower garments.

cs.CV↗

arXiv2018

Keypoint Based Weakly Supervised Human Parsing

Fully convolutional networks (FCN) have achieved great success in human parsing in recent years. In conventional human parsing tasks, pixel-level labeling is required for guiding the training, which usually involves enormous human labeling efforts. To ease the labeling efforts, we propose a novel weakly supervised human parsing method which only requires simple object keypoint annotations for learning. We develop an iterative learning method to generate pseudo part segmentation masks from keypoint labels. With these pseudo masks, we train an FCN network to output pixel-level human parsing predictions. Furthermore, we develop a correlation network to perform joint prediction of part and object segmentation masks and improve the segmentation performance. The experiment results show that our weakly supervised method is able to achieve very competitive human parsing results. Despite our method only uses simple keypoint annotations for learning, we are able to achieve comparable performance with fully supervised methods which use the expensive pixel-level annotations.

cs.CV↗

arXiv2015

Application of Mythen Detector In-situ XRD Study on The Thermal Expansion Behavior of Metal Indium

A Mythen detector has been equipped at the beamline 4B9A of Beijing Synchrotron Radiation Facility, which can be used for in-situ real-time measurement of X-ray diffraction (XRD) full profiles. In this paper, the thermal expansion behavior of metal indium has been studied by using the in-situ XRD technique with the Mythen detector. The indium film was heated from 30 to 160 °C with a heating rate of 2 °C/min. The in-situ XRD full-profiles were collected with a rate of one profile per 10 seconds. Rietveld refinement was used to extract the structural parameters. The results demonstrate that the thermal expansion of metal indium is nonlinear especially when the sample temperature was close to its melting point (156.5 °C). The expansion of a-axis and the contraction of c-axis of the tetragonal unit cell of metallic indium can be well described by biquadratic and cubic polynomials, respectively. The tetragonal unit cell presents a tendency to become cubic one with the increase of temperature but without detectable phase change. This study is not only beneficial to the application of metal indium, but also exhibits the capacity of in-situ time-resolved XRD experiments at the X-ray diffraction station of BSRF.

cond-mat.mtrl-sci↗

arXiv2015

Gapless quantum spin liquid ground state in the two-dimensional spin-1/2 triangular antiferromagnet YbMgGaO$_4$

Quantum spin liquid (QSL) is a novel state of matter which refuses the conventional spin freezing even at 0 K. Experimentally searching for the structurally perfect candidates is a big challenge in condensed matter physics. Here we report the successful synthesis of a new spin-1/2 triangular antiferromagnet YbMgGaO$_4$ with R$\bar{3}$m symmetry. The compound with an ideal two-dimensional and spatial isotropic magnetic triangular-lattice has no site-mixing magnetic defects and no antisymmetric Dzyaloshinsky-Moriya (DM) interactions. No spin freezing down to 60 mK (despite $Θ$$_w$ $\sim$ -4 K), the low-T power-law temperature dependence of heat capacity and nonzero susceptibility suggest that YbMgGaO$_4$ is a promising gapless ($\leq$ $|$$Θ$$_w$$|$/100) QSL candidate. The residual spin entropy, which is accurately determined with a non-magnetic reference LuMgGaO$_4$, approaches zero ($<$ 0.6 \%). This indicates that the possible QSL ground state (GS) of the frustrated spin system has been experimentally achieved at the lowest measurement temperatures.

cond-mat.str-el↗

arXiv2013

Re-entrance of Gapless Quantum Spin Liquids Observed in a Newly Synthesized Spin-1/2 Kagome Antiferromagnet $ZnCu_{3}(OH)_{6}SO_{4}$

Quantum spin liquid (QSL) is a novel state of matter with exotic excitations and was theoretically predicted to be realized most possibly in an S=1/2 kagome antiferromagnet. Experimentally searching for the candidate materials is a big challenge in condensed matter physics and only two such candidates were reported so far. Here we report the successful synthesis of a new spin-1/2 kagome antiferromagnet ZnCu3(OH)6SO4. No magnetic ordering is observed down to 50 mK, despite a moderately high Weiss temperature of θW ~ -79 K. It strongly suggests that the material is a new QSL candidate. Most interestingly, the magnetic specific heat clearly exhibits linear behaviors in two low-temperature regions. Both behaviors exactly correspond to two temperature-independent susceptibilities. These consistently reveal a novel re-entrance phenomenon of gapless QSL state at the lowest temperatures. The findings provide new insights into QSL ground and excited states and will inspire new theoretical and experimental studies.

cond-mat.str-el↗

arXiv2013

Transition-metal distribution in kagome antiferromagnet CoCu3(OH)6Cl2 revealed by resonant x-ray diffraction

The distribution of chemically similar transition-metal ions is a fundamental issue in the study of herbertsmithite-type kagome antiferromagnets. Using synchrotron radiation, we have performed resonant powder x-ray diffractions on newly synthesized CoCu3(OH)6Cl2, which provide an exact distribution of transition-metal ions in the frustrated antiferromagnet. Both magnetic susceptibility and specific heat measurements are quantitatively consistent with the occupation fractions determined by resonant x-ray diffraction. The distribution of transition-metal ions and residual magnetic entropy suggest a novel low temperature (T < 4 K) magnetism, where the interlayer triangular spins undergo a spin-glass freezing while the kagome spins still keep highly frustrated.

cond-mat.str-el↗

arXiv2012

Hierarchical structure and biomineralization in cricket tooth

Cricket is a truculent insect with stiff and sharp teeth as a fighting weapon. The structure and possible biomineralization of the cricket teeth are always interested. Synchrotron radiation X-ray fluorescence, X-ray diffraction and small angle X-ray scattering techniques were used to probe the element distribution, possible crystalline structures and size distribution of scatterers in cricket teeth. Scanning electron microscope was used to observe the nanoscaled structure. The results demonstrate that Zn is the main heavy element in cricket teeth. The surface of the cricket teeth has a crystalline compound like ZnFe2(AsO4)2(OH)2(H2O)4. While, the interior of the teeth has a crystalline compound like ZnCl2, which is from the biomineralization. The ZnCl2-like biomineral forms nanoscaled microfibrils and their axial direction points at the top of tooth cusp. The microfibrils aggregate random into intermediate filaments, forming a hierarchical structure. A sketch map of the cricket tooth cusp was proposed and a detailed discussion was given in this paper.

physics.bio-ph↗