arXiv subjects

Shan Liu

Publications and source records attributed to Shan Liu.

At least 109 records · Page 6Linked to original sources

arXiv2020

AdaDurIAN: Few-shot Adaptation for Neural Text-to-Speech with DurIAN

This paper investigates how to leverage a DurIAN-based average model to enable a new speaker to have both accurate pronunciation and fluent cross-lingual speaking with very limited monolingual data. A weakness of the recently proposed end-to-end text-to-speech (TTS) systems is that robust alignment is hard to achieve, which hinders it to scale well with very limited data. To cope with this issue, we introduce AdaDurIAN by training an improved DurIAN-based average model and leverage it to few-shot learning with the shared speaker-independent content encoder across different speakers. Several few-shot learning tasks in our experiments show AdaDurIAN can outperform the baseline end-to-end system by a large margin. Subjective evaluations also show that AdaDurIAN yields higher mean opinion score (MOS) of naturalness and more preferences of speaker similarity. In addition, we also apply AdaDurIAN to emotion transfer tasks and demonstrate its promising performance.

cs.SD↗

arXiv2020

Multi-Mapping Image-to-Image Translation with Central Biasing Normalization

Recent advances in image-to-image translation have seen a rise in approaches generating diverse images through a single network. To indicate the target domain for a one-to-many mapping, the latent code is injected into the generator network. However, we found that the injection method leads to mode collapse because of normalization strategies. Existing normalization strategies might either cause the inconsistency of feature distribution or eliminate the effect of the latent code. To solve these problems, we propose the consistency within diversity criteria for designing the multi-mapping model. Based on the criteria, we propose central biasing normalization to inject the latent code information. Experiments show that our method can improve the quality and diversity of existing image-to-image translation models, such as StarGAN, BicycleGAN, and pix2pix.

cs.CV↗

arXiv2020

RoIMix: Proposal-Fusion among Multiple Images for Underwater Object Detection

Generic object detection algorithms have proven their excellent performance in recent years. However, object detection on underwater datasets is still less explored. In contrast to generic datasets, underwater images usually have color shift and low contrast; sediment would cause blurring in underwater images. In addition, underwater creatures often appear closely to each other on images due to their living habits. To address these issues, our work investigates augmentation policies to simulate overlapping, occluded and blurred objects, and we construct a model capable of achieving better generalization. We propose an augmentation method called RoIMix, which characterizes interactions among images. Proposals extracted from different images are mixed together. Previous data augmentation methods operate on a single image while we apply RoIMix to multiple images to create enhanced samples as training data. Experiments show that our proposed method improves the performance of region-based object detectors on both Pascal VOC and URPC datasets.

cs.CV↗

arXiv2020

C3DVQA: Full-Reference Video Quality Assessment with 3D Convolutional Neural Network

Traditional video quality assessment (VQA) methods evaluate localized picture quality and video score is predicted by temporally aggregating frame scores. However, video quality exhibits different characteristics from static image quality due to the existence of temporal masking effects. In this paper, we present a novel architecture, namely C3DVQA, that uses Convolutional Neural Network with 3D kernels (C3D) for full-reference VQA task. C3DVQA combines feature learning and score pooling into one spatiotemporal feature learning process. We use 2D convolutional layers to extract spatial features and 3D convolutional layers to learn spatiotemporal features. We empirically found that 3D convolutional layers are capable to capture temporal masking effects of videos. We evaluated the proposed method on the LIVE and CSIQ datasets. The experimental results demonstrate that the proposed method achieves the state-of-the-art performance.

eess.IV↗

arXiv2020

Evidence for a new extended solid of nitrogen

A new extended solid nitrogen, referred to post-layered-polymeric nitrogen (PLP-N), was observed by further heating the layered-polymeric nitrogen (LP-N) to above 2300 K at 161 GPa. The new phase is found to be very transparent and exhibits ultra-large d-spacings ranging from 2.8 to 4.9 Å at 172 GPa, suggesting a possible large-unit-cell 2D chain-like or 0D cluster-type structure with wide bandgap. However, the observed X-ray diffraction pattern and Raman scattering data cannot match any predicted structures in the published literature. This finding further complicates the phase diagram of nitrogen and also highlights the path dependence of the high-pressure dissociative transition in nitrogen. In addition, the forming boundary between cg-N and LP-N has been determined.

cond-mat.mtrl-sci↗

arXiv2020

Linear Model based Geometry Coding for Lidar Acquired Point Clouds

In this paper, we propose a new geometry coding method for point cloud compression (PCC), where the points can be fitted and represented by straight lines. The encoding of the linear model can be expressed by two parts, including the principle component along the line direction and the offsets from the line. Compact representation and high-efficiency coding methods are presented by encoding the parameters of linear model with appropriate quantization step-sizes (QS). To maximize the coding performance, encoder optimization techniques are employed to find the optimal trade-off between coding bits and errors, involving the Lagrangian multiplier method, where the rate-distortion behavior in terms of QS and multiplier is analyzed. We implement our method on top of the MPEG G-PCC reference software, and the results have shown that the proposed method is effective in coding point clouds with explicit line structures, such as the Lidar acquired data for autonomous driving. About 20\% coding gains can be achieved on lossy geometry coding.

eess.IV↗

arXiv2019

Multi-mapping Image-to-Image Translation via Learning Disentanglement

Recent advances of image-to-image translation focus on learning the one-to-many mapping from two aspects: multi-modal translation and multi-domain translation. However, the existing methods only consider one of the two perspectives, which makes them unable to solve each other's problem. To address this issue, we propose a novel unified model, which bridges these two objectives. First, we disentangle the input images into the latent representations by an encoder-decoder architecture with a conditional adversarial training in the feature space. Then, we encourage the generator to learn multi-mappings by a random cross-domain translation. As a result, we can manipulate different parts of the latent representations to perform multi-modal and multi-domain translations simultaneously. Experiments demonstrate that our method outperforms state-of-the-art methods.

cs.CV↗

arXiv2019

BLP -- Boundary Likelihood Pinpointing Networks for Accurate Temporal Action Localization

Despite tremendous progress achieved in temporal action detection, state-of-the-art methods still suffer from the sharp performance deterioration when localizing the starting and ending temporal action boundaries. Although most methods apply boundary regression paradigm to tackle this problem, we argue that the direct regression lacks detailed enough information to yield accurate temporal boundaries. In this paper, we propose a novel Boundary Likelihood Pinpointing (BLP) network to alleviate this deficiency of boundary regression and improve the localization accuracy. Given a loosely localized search interval that contains an action instance, BLP casts the problem of localizing temporal boundaries as that of assigning probabilities on each equally divided unit of this interval. These generated probabilities provide useful information regarding the boundary location of the action inside this search interval. Based on these probabilities, we introduce a boundary pinpointing paradigm to pinpoint the accurate boundaries under a simple probabilistic framework. Compared with other C3D feature based detectors, extensive experiments demonstrate that BLP significantly improves the localization performance of recent state-of-the-art detectors, and achieves competitive detection mAP on both THUMOS' 14 and ActivityNet datasets, particularly when the evaluation tIoU is high.

cs.CV↗

arXiv2019

PointHop: An Explainable Machine Learning Method for Point Cloud Classification

An explainable machine learning method for point cloud classification, called the PointHop method, is proposed in this work. The PointHop method consists of two stages: 1) local-to-global attribute building through iterative one-hop information exchange, and 2) classification and ensembles. In the attribute building stage, we address the problem of unordered point cloud data using a space partitioning procedure and developing a robust descriptor that characterizes the relationship between a point and its one-hop neighbor in a PointHop unit. When we put multiple PointHop units in cascade, the attributes of a point will grow by taking its relationship with one-hop neighbor points into account iteratively. Furthermore, to control the rapid dimension growth of the attribute vector associated with a point, we use the Saab transform to reduce the attribute dimension in each PointHop unit. In the classification and ensemble stage, we feed the feature vector obtained from multiple PointHop units to a classifier. We explore ensemble methods to improve the classification performance furthermore. It is shown by experimental results that the PointHop method offers classification performance that is comparable with state-of-the-art methods while demanding much lower training complexity.

cs.CV↗

arXiv2019

Video-based compression for plenoptic point clouds

The plenoptic point cloud that has multiple colors from various directions, is a more complete representation than the general point cloud that usually has only one color. It is more realistic but also brings a larger volume of data that needs to be compressed efficiently. The state-of-the-art method to compress the plenoptic point cloud is an extension of the region-based adaptive hierarchical transform (RAHT). As far as we can see, in addition to RAHT, the video-based point cloud compression (V-PCC) is also an efficient point cloud compression method. However, to the best of our knowledge, no works have used a video-based solution to compress the plenoptic point cloud yet. In this paper, we first extend the V-PCC to support the plenoptic point cloud compression by generating multiple attribute videos. Then based on the observation that these videos from multiple views have very high correlations, we propose encoding them using multiview high efficiency video coding. We further propose a block-based padding method that unifies the unoccupied attribute pixels from different views to reduce their bit cost. The proposed algorithms are implemented in the V-PCC reference software. The experimental results show that the proposed algorithms can bring significant bitrate savings compared with the state-of-the-art method for plenoptic point cloud compression.

eess.IV↗

arXiv2019

High-pressure isostructural transition in nitrogen

Understanding high-pressure transitions in prototypical linear diatomic molecules, such as hydrogen, nitrogen, and oxygen, is an important objective in high-pressure physics. Recent ultrahigh-pressure study on hydrogen revealed that there exists a molecular-symmetry-breaking isostructural electronic transition in hydrogen. The pressure-induced symmetry breaking could also lead to a series of solid molecular phases in nitrogen associated with the splitting of vibron, walking a path from fluid nitrogen to molecular solids and then to polymetric nitrogen phases. However, isostructural transition in nitrogen has not been reported so far, which may provide crucial information on the crystallographic nature of the pressure-induced dissociation transitions in nitrogen. Here we present a high-pressure Raman scattering and high-pressure angle dispersive X-ray diffraction study of the previously reported lambda-N2 and reveal an isostructural transition from the lambda-N2 to a new derivative phase, lambda-N2. Under compression, lambda-N2 remains in the monoclinic crystal lattice, accompanied by a monotonic increase in anisotropy. The pressure-dependent decrease of the unit cell parameters exhibits a discontinuity at the pressure of 54 GPa, accompany with sudden broadening and turnover of vibron.

cond-mat.mtrl-sci↗

arXiv2019

StructureFlow: Image Inpainting via Structure-aware Appearance Flow

Image inpainting techniques have shown significant improvements by using deep neural networks recently. However, most of them may either fail to reconstruct reasonable structures or restore fine-grained textures. In order to solve this problem, in this paper, we propose a two-stage model which splits the inpainting task into two parts: structure reconstruction and texture generation. In the first stage, edge-preserved smooth images are employed to train a structure reconstructor which completes the missing structures of the inputs. In the second stage, based on the reconstructed structures, a texture generator using appearance flow is designed to yield image details. Experiments on multiple publicly available datasets show the superior performance of the proposed network.

cs.CV↗

arXiv2019

Generative Adversarial Network based Speaker Adaptation for High Fidelity WaveNet Vocoder

Although state-of-the-art parallel WaveNet has addressed the issue of real-time waveform generation, there remains problems. Firstly, due to the noisy input signal of the model, there is still a gap between the quality of generated and natural waveforms. Secondly, a parallel WaveNet is trained under a distillation framework, which makes it tedious to adapt a well trained model to a new speaker. To address these two problems, in this paper we propose an end-to-end adaptation method based on the generative adversarial network (GAN), which can reduce the computational cost for the training of new speaker adaptation. Our subjective experiments shows that the proposed training method can further reduce the quality gap between generated and natural waveforms.

eess.AS↗

arXiv2019

Deep AutoEncoder-based Lossy Geometry Compression for Point Clouds

Point cloud is a fundamental 3D representation which is widely used in real world applications such as autonomous driving. As a newly-developed media format which is characterized by complexity and irregularity, point cloud creates a need for compression algorithms which are more flexible than existing codecs. Recently, autoencoders(AEs) have shown their effectiveness in many visual analysis tasks as well as image compression, which inspires us to employ it in point cloud compression. In this paper, we propose a general autoencoder-based architecture for lossy geometry point cloud compression. To the best of our knowledge, it is the first autoencoder-based geometry compression codec that directly takes point clouds as input rather than voxel grids or collections of images. Compared with handcrafted codecs, this approach adapts much more quickly to previously unseen media contents and media formats, meanwhile achieving competitive performance. Our architecture consists of a pointnet-based encoder, a uniform quantizer, an entropy estimation block and a nonlinear synthesis transformation module. In lossy geometry compression of point cloud, results show that the proposed method outperforms the test model for categories 1 and 3 (TMC13) published by MPEG-3DG group on the 125th meeting, and on average a 73.15\% BD-rate gain is achieved.

cs.CV↗

arXiv2019

Graph Convolutional Label Noise Cleaner: Train a Plug-and-play Action Classifier for Anomaly Detection

Video anomaly detection under weak labels is formulated as a typical multiple-instance learning problem in previous works. In this paper, we provide a new perspective, i.e., a supervised learning task under noisy labels. In such a viewpoint, as long as cleaning away label noise, we can directly apply fully supervised action classifiers to weakly supervised anomaly detection, and take maximum advantage of these well-developed classifiers. For this purpose, we devise a graph convolutional network to correct noisy labels. Based upon feature similarity and temporal consistency, our network propagates supervisory signals from high-confidence snippets to low-confidence ones. In this manner, the network is capable of providing cleaned supervision for action classifiers. During the test phase, we only need to obtain snippet-wise predictions from the action classifier without any extra post-processing. Extensive experiments on 3 datasets at different scales with 2 types of action classifiers demonstrate the efficacy of our method. Remarkably, we obtain the frame-level AUC score of 82.12% on UCF-Crime.

cs.CV↗

arXiv2019

Occupancy-map-based rate distortion optimization for video-based point cloud compression

The state-of-the-art video-based point cloud compression scheme projects the 3D point cloud to 2D patch by patch and organizes the patches into frames to compress them using the efficient video compression scheme. Such a scheme shows a good trade-off between the number of points projected and the video continuity to utilize the video compression scheme. However, some unoccupied pixels between different patches are compressed using almost the same quality with the occupied pixels, which will lead to the waste of lots of bits since the unoccupied pixels are useless for the reconstructed point cloud. In this paper, we propose to consider only the rate instead of the rate distortion cost for the unoccupied pixels during the rate distortion optimization process. The proposed scheme can be applied to both the geometry and attribute frames. The experimental results show that the proposed algorithm can achieve an average of 11.9% and 15.4% bitrate savings for the geometry and attribute, respectively.

cs.MM↗

arXiv2017

Tailoring symmetric metallic and magnetic edge states of nanoribbon in semiconductive monolayer PtS2

Fabrication of atomic scale of metallic wire remains challenging. In present work, a nanoribbon with two parallel symmetric metallic and magnetic edges was designed from semiconductive monolayer PtS2 by employing first-principles calculations based on density functional theory. Edge energy, bonding charge density, band structure and simulated STM of possible edges states of PtS2 were systematically studied. It was found that Pt-terminated edge nanoribbons were the relatively stable metallic and magnetic edge tailored from a noble transition metal dichalcogenides PtS2. The nanoribbon with two atomic metallic wires may have promising application as nano power transmission lines, which at least two lines are needed.

cond-mat.mtrl-sci↗

arXiv2015

In vacuum diamond sensor scanner for beam halo measurements in the beam line at the KEK Accelerator Test Facility

The investigation of beam halo transverse distributions is important for the understanding of beam losses and the control of backgrounds in Future Linear Colliders (FLC). A novel in vacuum diamond sensor (DSv) scanner with four strips has been designed and developed for the investigation of the beam halo transverse distributions and also for the diagnostics of Compton recoil electrons after the interaction point (IP) of ATF2, a low energy (1.3 GeV) prototype of the final focus system for the ILC and CLIC linear collider projects. Using the DSv, a dynamic range of $\sim10^6$ has been successfully demonstrated and confirmed for the first time by simultaneous beam core ($\sim10^9$ electrons) and beam halo ($\sim10^3$ electrons) measurements at ATF2. This report presents the characterization, performance studies and tests of the diamond sensors using an $α$ source as well as using the electron beams at PHIL, a low energy ($< 10$ MeV) photo-injector at LAL, and at ATF2. First beam halo measurement results using the DSv at ATF2 with different beam intensities and vacuum levels are also presented. Such measurements not only allow one to evaluate the different sources of beam halo generation but also to define the requirements for a suitable collimation system to be installed at ATF2, as well as to optimize its performance during future operation.

physics.acc-ph↗