arXiv subjects

Rui Liu

Publications and source records attributed to Rui Liu.

At least 361 records · Page 20Linked to original sources

arXiv2021

Decoupling Speaker-Independent Emotions for Voice Conversion Via Source-Filter Networks

Emotional voice conversion (VC) aims to convert a neutral voice to an emotional (e.g. happy) one while retaining the linguistic information and speaker identity. We note that the decoupling of emotional features from other speech information (such as speaker, content, etc.) is the key to achieving remarkable performance. Some recent attempts about speech representation decoupling on the neutral speech can not work well on the emotional speech, due to the more complex acoustic properties involved in the latter. To address this problem, here we propose a novel Source-Filter-based Emotional VC model (SFEVC) to achieve proper filtering of speaker-independent emotion features from both the timbre and pitch features. Our SFEVC model consists of multi-channel encoders, emotion separate encoders, and one decoder. Note that all encoder modules adopt a designed information bottlenecks auto-encoder. Additionally, to further improve the conversion quality for various emotions, a novel two-stage training strategy based on the 2D Valence-Arousal (VA) space was proposed. Experimental results show that the proposed SFEVC along with a two-stage training strategy outperforms all baselines and achieves the state-of-the-art performance in speaker-independent emotional VC with nonparallel data.

eess.AS↗

arXiv2021

Dilations for operator-valued quantum measures

This paper concerns the dilations of Banach space operator-valued quantum measures. While the recently developed general dilation theory can lead to a projection (idempotent) valued dilation for any quantum measure over the projection lattice for a von Neumann algebra that dose not contain type $I_{2}$ direct summand, such a dilation does not necessarily guarantee the preservation of countable additivity of the quantum measure. So it remain an open question whether every countably additive $B(X)$-valued quantum measure can be dilated to a countably additive projection-valued measure.The main purpose of this paper is to prove that such a dilation can be constructed if one of the following two conditions is satisfied: (i) the underling Banach space $X = \ell_{p}$ $(1\leq p < 2$) or it has Schur property, (ii) the quantum measure has bounded $p$-variation for some $ 1\leq p < \infty $. All of these were achieved by establishing a non-commutative version of a minimal dilation theory on the so-called elementary dilation space equipping with an appropriate dilation norm. In particular, the newly introduced $p$-variation norm on the elementary dilation space allows us to prove that every operator-valued quantum measure with bounded $p$-variation has a projection-valued quantum measure dilation that preserves the boundedness of the $p$-variation.

math.FA↗

arXiv2021

FuseFormer: Fusing Fine-Grained Information in Transformers for Video Inpainting

Transformer, as a strong and flexible architecture for modelling long-range relations, has been widely explored in vision tasks. However, when used in video inpainting that requires fine-grained representation, existed method still suffers from yielding blurry edges in detail due to the hard patch splitting. Here we aim to tackle this problem by proposing FuseFormer, a Transformer model designed for video inpainting via fine-grained feature fusion based on novel Soft Split and Soft Composition operations. The soft split divides feature map into many patches with given overlapping interval. On the contrary, the soft composition operates by stitching different patches into a whole feature map where pixels in overlapping regions are summed up. These two modules are first used in tokenization before Transformer layers and de-tokenization after Transformer layers, for effective mapping between tokens and features. Therefore, sub-patch level information interaction is enabled for more effective feature propagation between neighboring patches, resulting in synthesizing vivid content for hole regions in videos. Moreover, in FuseFormer, we elaborately insert the soft composition and soft split into the feed-forward network, enabling the 1D linear layers to have the capability of modelling 2D structure. And, the sub-patch level feature fusion ability is further enhanced. In both quantitative and qualitative evaluations, our proposed FuseFormer surpasses state-of-the-art methods. We also conduct detailed analysis to examine its superiority.

cs.CV↗

arXiv2021

Confined and eruptive catastrophes of solar magnetic flux ropes caused by mass loading and unloading

It is widely accepted that coronal magnetic flux ropes are the core structures of large-scale solar eruptive activities, which inflict dramatic impacts on the solar-terrestrial system. Previous studies have demonstrated that varying magnetic properties of a coronal flux rope system could result in a catastrophe of the rope, which may trigger solar eruptive activities. Since the total mass of a flux rope also plays an important role in stabilizing the rope, we use 2.5-dimensional magnetohydrodynamic (MHD) numerical simulations in this letter to investigate how a flux rope evolves as its total mass varies. It is found that an unloading process that decreases the total mass of the rope could result in an upward (eruptive) catastrophe in the flux rope system, during which the rope jumps upward and the magnetic energy is released. This indicates that mass unloading processes could initiate the eruption of the flux rope. Moreover, when the system is not too diffusive, there is also a downward (confined) catastrophe that could be caused by mass loading processes, via which the total mass accumulates. The magnetic energy, however, is increased during the downward catastrophe, indicating that mass loading processes could cause confined activities that may contribute to the storage of energy before the onset of coronal eruptions.

astro-ph.SR↗

arXiv2021

Vector-Decomposed Disentanglement for Domain-Invariant Object Detection

To improve the generalization of detectors, for domain adaptive object detection (DAOD), recent advances mainly explore aligning feature-level distributions between the source and single-target domain, which may neglect the impact of domain-specific information existing in the aligned features. Towards DAOD, it is important to extract domain-invariant object representations. To this end, in this paper, we try to disentangle domain-invariant representations from domain-specific representations. And we propose a novel disentangled method based on vector decomposition. Firstly, an extractor is devised to separate domain-invariant representations from the input, which are used for extracting object proposals. Secondly, domain-specific representations are introduced as the differences between the input and domain-invariant representations. Through the difference operation, the gap between the domain-specific and domain-invariant representations is enlarged, which promotes domain-invariant representations to contain more domain-irrelevant information. In the experiment, we separately evaluate our method on the single- and compound-target case. For the single-target case, experimental results of four domain-shift scenes show our method obtains a significant performance gain over baseline methods. Moreover, for the compound-target case (i.e., the target is a compound of two different domains without domain labels), our method outperforms baseline methods by around 4%, which demonstrates the effectiveness of our method.

cs.CV↗

arXiv2021

Solid phase epitaxial growth of the correlated-electron transparent conducting oxide SrVO3

SrVO3 thin films with a high figure of merit for applications as transparent conductors were crystallized from amorphous layers using solid phase epitaxy (SPE). Epitaxial SrVO3 films crystallized on SrTiO3 using SPE exhibit a room temperature resistivity of 2.5 x 10-5 Ohms cm, a residual resistivity ratio of 3.8, and visible light transmission above 0.5 for a 60 nm-thick film. SrVO3 layers were deposited at room temperature using radio-frequency sputtering in an amorphous form and subsequently crystallized by heating in controlled gas environment. The lattice parameters and mosaic angular width of x-ray reflections from the crystallized films are consistent with partial relaxation of the strain resulting from the epitaxial mismatch between SrVO3 and SrTiO3. A reflection high-energy electron diffraction study of the kinetics of SPE indicates that crystallization occurs via the thermally activated propagation of the crystalline/amorphous interface, similar to SPE phenomena in other perovskite oxides. Thermodynamic calculations based on density functional theory predict the temperature and oxygen partial pressure conditions required to produce the SrVO3 phase and are consistent with the experiments. The separate control of deposition and crystallization conditions in SPE presents new possibilities for the crystallization of transparent conductors in complex geometries and over large areas.

cond-mat.mtrl-sci↗

arXiv2021

Using Query Expansion in Manifold Ranking for Query-Oriented Multi-Document Summarization

Manifold ranking has been successfully applied in query-oriented multi-document summarization. It not only makes use of the relationships among the sentences, but also the relationships between the given query and the sentences. However, the information of original query is often insufficient. So we present a query expansion method, which is combined in the manifold ranking to resolve this problem. Our method not only utilizes the information of the query term itself and the knowledge base WordNet to expand it by synonyms, but also uses the information of the document set itself to expand the query in various ways (mean expansion, variance expansion and TextRank expansion). Compared with the previous query expansion methods, our method combines multiple query expansion methods to better represent query information, and at the same time, it makes a useful attempt on manifold ranking. In addition, we use the degree of word overlap and the proximity between words to calculate the similarity between sentences. We performed experiments on the datasets of DUC 2006 and DUC2007, and the evaluation results show that the proposed query expansion method can significantly improve the system performance and make our system comparable to the state-of-the-art systems.

cs.IR↗

arXiv2021

A fundamental mechanism of solar eruption initiation

Solar eruptions are spectacular magnetic explosions in the Sun's corona, and how they are initiated remains unclear. Prevailing theories often rely on special magnetic topologies that may not generally exist in the pre-eruption source region of corona. Here, using fully three-dimensional magnetohydrodynamic simulations with high accuracy, we show that solar eruptions can be initiated in a single bipolar configuration with no additional special topology. Through photospheric shearing motion alone, an electric current sheet forms in the highly sheared core field of the magnetic arcade during its quasi-static evolution. Once magnetic reconnection sets in, the whole arcade is expelled impulsively, forming a fast-expanding twisted flux rope with a highly turbulent reconnecting region underneath. The simplicity and efficacy of this scenario argue strongly for its fundamental importance in the initiation of solar eruptions.

astro-ph.SR↗

arXiv2021

Synthesized Trust Learning from Limited Human Feedback for Human-Load-Reduced Multi-Robot Deployments

Human multi-robot system (MRS) collaboration is demonstrating potentials in wide application scenarios due to the integration of human cognitive skills and a robot team's powerful capability introduced by its multi-member structure. However, due to limited human cognitive capability, a human cannot simultaneously monitor multiple robots and identify the abnormal ones, largely limiting the efficiency of the human-MRS collaboration. There is an urgent need to proactively reduce unnecessary human engagements and further reduce human cognitive loads. Human trust in human MRS collaboration reveals human expectations on robot performance. Based on trust estimation, the work between a human and MRS will be reallocated that an MRS will self-monitor and only request human guidance in critical situations. Inspired by that, a novel Synthesized Trust Learning (STL) method was developed to model human trust in the collaboration. STL explores two aspects of human trust (trust level and trust preference), meanwhile accelerates the convergence speed by integrating active learning to reduce human workload. To validate the effectiveness of the method, tasks "searching victims in the context of city rescue" were designed in an open-world simulation environment, and a user study with 10 volunteers was conducted to generate real human trust feedback. The results showed that by maximally utilizing human feedback, the STL achieved higher accuracy in trust modeling with a few human feedback, effectively reducing human interventions needed for modeling an accurate trust, therefore reducing human cognitive load in the collaboration.

cs.RO↗

arXiv2021

Quantum resonant tunnelling enhances hydrogen bond rotation of confined water

Many studies have revealed that confined water chain flipping is closely related to the spatial size and even quantum effects of the confinement environment. Here, we show that these are not the only factors that affect the flipping process of a confined water chain. First-principles calculations and analyses confirm that quantum tunnelling effects from the water chain itself, especially resonant tunnelling, enhance the hydrogen bond rotation process. Importantly, resonant tunnelling can result in tunnelling rotation of hydrogen bonds with a probability close to 1 with only 0.597 eV provided energy. Compared to sequential tunnelling, resonant tunnelling dominants water chain flipping at temperatures up to 20 K higher. Additionally, the ratio of the resonant tunnelling probability to the thermal disturbance probability at 200 K is at least ten times larger than that of sequential tunnelling, which further illustrates the enhancement of hydrogen bond rotation brought about by resonant tunnelling.

physics.bio-ph↗

arXiv2021

Proficiency Constrained Multi-Agent Reinforcement Learning for Environment-Adaptive Multi UAV-UGV Teaming

A mixed aerial and ground robot team, which includes both unmanned ground vehicles (UGVs) and unmanned aerial vehicles (UAVs), is widely used for disaster rescue, social security, precision agriculture, and military missions. However, team capability and corresponding configuration vary since robots have different motion speeds, perceiving ranges, reaching areas, and resilient capabilities to the dynamic environment. Due to heterogeneous robots inside a team and the resilient capabilities of robots, it is challenging to perform a task with an optimal balance between reasonable task allocations and maximum utilization of robot capability. To address this challenge for effective mixed ground and aerial teaming, this paper developed a novel teaming method, proficiency aware multi-agent deep reinforcement learning (Mix-RL), to guide ground and aerial cooperation by considering the best alignments between robot capabilities, task requirements, and environment conditions. Mix-RL largely exploits robot capabilities while being aware of the adaption of robot capabilities to task requirements and environment conditions. Mix-RL's effectiveness in guiding mixed teaming was validated with the task "social security for criminal vehicle tracking".

cs.RO↗

arXiv2021

An Attention Transfer Model for Human-Assisted Failure Avoidance in Robot Manipulations

Due to real-world dynamics and hardware uncertainty, robots inevitably fail in task executions, resulting in undesired or even dangerous executions. In order to avoid failures and improve robot performance, it is critical to identify and correct abnormal robot executions at an early stage. However, due to limited reasoning capability and knowledge storage, it is challenging for robots to self-diagnose and -correct their own abnormality in both planning and executing. To improve robot self diagnosis capability, in this research a novel human-to-robot attention transfer (\textit{\textbf{H2R-AT}}) method was developed to identify robot manipulation errors by leveraging human instructions. \textit{\textbf{H2R-AT}} was developed by fusing attention mapping mechanism into a novel stacked neural networks model, transferring human verbal attention into robot visual attention. With the attention transfer, a robot understands \textit{what} and \textit{where} human concerns are to identify and correct abnormal manipulations. Two representative task scenarios: ``serve water for a human in a kitchen" and ``pick up a defective gear in a factory" were designed in a simulation framework CRAIhri with abnormal robot manipulations; and $252$ volunteers were recruited to provide about 12000 verbal reminders to learn and test \textit{\textbf{H2R-AT}}. The method effectiveness was validated by the high accuracy of $73.68\%$ in transferring attention, and the high accuracy of $66.86\%$ in avoiding grasping failures.

cs.RO↗

arXiv2021

Repairing Human Trust by Promptly Correcting Robot Mistakes with An Attention Transfer Model

In human-robot collaboration (HRC), human trust in the robot is the human expectation that a robot executes tasks with desired performance. A higher-level trust increases the willingness of a human operator to assign tasks, share plans, and reduce the interruption during robot executions, thereby facilitating human-robot integration both physically and mentally. However, due to real-world disturbances, robots inevitably make mistakes, decreasing human trust and further influencing collaboration. Trust is fragile and trust loss is triggered easily when robots show incapability of task executions, making the trust maintenance challenging. To maintain human trust, in this research, a trust repair framework is developed based on a human-to-robot attention transfer (H2R-AT) model and a user trust study. The rationale of this framework is that a prompt mistake correction restores human trust. With H2R-AT, a robot localizes human verbal concerns and makes prompt mistake corrections to avoid task failures in an early stage and to finally improve human trust. User trust study measures trust status before and after the behavior corrections to quantify the trust loss. Robot experiments were designed to cover four typical mistakes, wrong action, wrong region, wrong pose, and wrong spatial relation, validated the accuracy of H2R-AT in robot behavior corrections; a user trust study with $252$ participants was conducted, and the changes in trust levels before and after corrections were evaluated. The effectiveness of the human trust repairing was evaluated by the mistake correction accuracy and the trust improvement.

cs.RO↗

arXiv2021

Reinforcement Learning for Emotional Text-to-Speech Synthesis with Improved Emotion Discriminability

Emotional text-to-speech synthesis (ETTS) has seen much progress in recent years. However, the generated voice is often not perceptually identifiable by its intended emotion category. To address this problem, we propose a new interactive training paradigm for ETTS, denoted as i-ETTS, which seeks to directly improve the emotion discriminability by interacting with a speech emotion recognition (SER) model. Moreover, we formulate an iterative training strategy with reinforcement learning to ensure the quality of i-ETTS optimization. Experimental results demonstrate that the proposed i-ETTS outperforms the state-of-the-art baselines by rendering speech with more accurate emotion style. To our best knowledge, this is the first study of reinforcement learning in emotional text-to-speech synthesis.

cs.CL↗

arXiv2021

Investigation on the Spatiotemporal Structures of Supra-Arcade Spikes

The vertical current sheet (VCS) trailing coronal mass ejections (CMEs) is the key place where the flare energy release and the CME buildup take place through magnetic reconnection. It is often studied from the edge-on perspective for the morphological similarity with the two-dimensional ``standard'' picture, but its three dimensional structure can only be revealed when the flare arcade is observed side on. The structure and dynamics in the so-called supra-arcade region thus contain important clues to the physical processes in flares and CMEs. Here we focus on the supra-arcade spikes (SASs), interpreted as the VCS viewed side-on, to study their spatiotemporal structures. By identifying each individual spike during the decay phase of four selected flares, in which the associated CME is traversed by a near-Earth spacecraft, we found that the widths of spikes are log-normal distributed, while the Fourier power spectra of the overall supra-arcade EUV emission, including bright spikes and dark downflows as well as the diffuse background, are power-law distributed, in terms of either spatial frequency $k$ or temporal frequency $ν$, which reflects the fragmentation of the VCS. We demonstrate that coronal emission-line intensity observations dominated by Kolmogorov turbulence would exhibit a power spectrum of $E(k)\sim k^{-13/3}$ or $E(ν)\sim ν^{-7/2}$, which is consistent with our observations. By comparing the number of SASs and the turns of field lines as derived from the ICMEs, we found a consistent axial length of $\sim\,$3.5 AU for three events with a CME speed of $\sim\,$1000 km/s in the inner heliosphere, but a much longer axial length $\sim\,$8 AU) for the fourth event with an exceptionally fast CME speed of $\sim\,$1500 km/s, suggesting that this ICME is flattened and its `nose' has well passed the Earth when the spacecraft traversed its leg.

astro-ph.SR↗

arXiv2021

Magnetic Flux Ropes in the Solar Corona: Structure and Evolution toward Eruption

Magnetic flux ropes are characterized by coherently twisted magnetic field lines, which are ubiquitous in magnetized plasmas. As the core structure of various eruptive phenomena in the solar atmosphere, flux ropes hold the key to understanding the physical mechanisms of solar eruptions, which impact the heliosphere and planetary atmospheres. Strongest disturbances in the Earth's space environments are often associated with large-scale flux ropes from the Sun colliding with the Earth's magnetosphere, leading to adverse, sometimes catastrophic, space-weather effects. However, it remains elusive as to how a flux rope forms and evolves toward eruption, and how it is structured and embedded in the ambient field. The present paper addresses these important questions by reviewing current understandings of coronal flux ropes from an observer's perspective, with emphasis on their structures and nascent evolution toward solar eruptions, as achieved by combining observations of both remote sensing and in-situ detection with modeling and simulation. It highlights an initiation mechanism for coronal mass ejections (CMEs) in which plasmoids in current sheets coalesce into a `seed' flux rope whose subsequent evolution into a CME is consistent with the standard model, thereby bridging the gap between microscale and macroscale dynamics.

astro-ph.SR↗

arXiv2021

Statistical Characteristics of Driver Acceleration Behavior and Its Probability Model

Naturalistic driving data were applied to study driver acceleration behaviour, and a probability model of the driver was proposed. First, the question of whether the database is large enough is resolved using kernel density estimation and Kullback-Liebler divergence. Next, the convergence database is utilised to achieve the bivariate acceleration distribution pattern. Subsequently, two probability models are proposed to explain the pattern. Finally, the statistical characteristics of the acceleration behaviours are studied to verify the probability models. The longitudinal and lateral acceleration behaviours always approximate a similar Pareto distribution. The braking, accelerating, and steering manoeuvres become more intense at first and then less intense as the velocity increases. These behaviours characteristics reveal the mechanism of the quadrangle bivariate acceleration distribution pattern. The bivariate acceleration behaviour of the driver will never reach a circle-shaped pattern. The bivariate Pareto distribution model can be applied to describe the bivariate acceleration behaviour of the driver.

cs.RO↗

arXiv2021

DivCo: Diverse Conditional Image Synthesis via Contrastive Generative Adversarial Network

Conditional generative adversarial networks (cGANs) target at synthesizing diverse images given the input conditions and latent codes, but unfortunately, they usually suffer from the issue of mode collapse. To solve this issue, previous works mainly focused on encouraging the correlation between the latent codes and their generated images, while ignoring the relations between images generated from various latent codes. The recent MSGAN tried to encourage the diversity of the generated image but only considers "negative" relations between the image pairs. In this paper, we propose a novel DivCo framework to properly constrain both "positive" and "negative" relations between the generated images specified in the latent space. To the best of our knowledge, this is the first attempt to use contrastive learning for diverse conditional image synthesis. A novel latent-augmented contrastive loss is introduced, which encourages images generated from adjacent latent codes to be similar and those generated from distinct latent codes to be dissimilar. The proposed latent-augmented contrastive loss is well compatible with various cGAN architectures. Extensive experiments demonstrate that the proposed DivCo can produce more diverse images than state-of-the-art methods without sacrificing visual quality in multiple unpaired and paired image generation tasks.

cs.CV↗