arXiv subjects

Rui Liu

Publications and source records attributed to Rui Liu.

At least 379 records · Page 21Linked to original sources

arXiv2021

Understanding and Optimizing Packed Neural Network Training for Hyper-Parameter Tuning

As neural networks are increasingly employed in machine learning practice, how to efficiently share limited training resources among a diverse set of model training tasks becomes a crucial issue. To achieve better utilization of the shared resources, we explore the idea of jointly training multiple neural network models on a single GPU in this paper. We realize this idea by proposing a primitive, called pack. We further present a comprehensive empirical study of pack and end-to-end experiments that suggest significant improvements for hyperparameter tuning. The results suggest: (1) packing two models can bring up to 40% performance improvement over unpacked setups for a single training step and the improvement increases when packing more models; (2) the benefit of the pack primitive largely depends on a number of factors including memory capacity, chip architecture, neural network structure, and batch size; (3) there exists a trade-off between packing and unpacking when training multiple neural network models on limited resources; (4) a pack-aware Hyperband is up to 2.7x faster than the original Hyperband, with this improvement growing as memory size increases and subsequently the density of models packed.

cs.LG↗

arXiv2021

Decoupled Spatial-Temporal Transformer for Video Inpainting

Video inpainting aims to fill the given spatiotemporal holes with realistic appearance but is still a challenging task even with prosperous deep learning approaches. Recent works introduce the promising Transformer architecture into deep video inpainting and achieve better performance. However, it still suffers from synthesizing blurry texture as well as huge computational cost. Towards this end, we propose a novel Decoupled Spatial-Temporal Transformer (DSTT) for improving video inpainting with exceptional efficiency. Our proposed DSTT disentangles the task of learning spatial-temporal attention into 2 sub-tasks: one is for attending temporal object movements on different frames at same spatial locations, which is achieved by temporally-decoupled Transformer block, and the other is for attending similar background textures on same frame of all spatial positions, which is achieved by spatially-decoupled Transformer block. The interweaving stack of such two blocks makes our proposed model attend background textures and moving objects more precisely, and thus the attended plausible and temporally-coherent appearance can be propagated to fill the holes. In addition, a hierarchical encoder is adopted before the stack of Transformer blocks, for learning robust and hierarchical features that maintain multi-level local spatial structure, resulting in the more representative token vectors. Seamless combination of these two novel designs forms a better spatial-temporal attention scheme and our proposed model achieves better performance than state-of-the-art video inpainting approaches with significant boosted efficiency.

cs.CV↗

arXiv2021

Expressive TTS Training with Frame and Style Reconstruction Loss

We propose a novel training strategy for Tacotron-based text-to-speech (TTS) system to improve the expressiveness of speech. One of the key challenges in prosody modeling is the lack of reference that makes explicit modeling difficult. The proposed technique doesn't require prosody annotations from training data. It doesn't attempt to model prosody explicitly either, but rather encodes the association between input text and its prosody styles using a Tacotron-based TTS framework. Our proposed idea marks a departure from the style token paradigm where prosody is explicitly modeled by a bank of prosody embeddings. The proposed training strategy adopts a combination of two objective functions: 1) frame level reconstruction loss, that is calculated between the synthesized and target spectral features; 2) utterance level style reconstruction loss, that is calculated between the deep style features of synthesized and target speech. The proposed style reconstruction loss is formulated as a perceptual loss to ensure that utterance level speech style is taken into consideration during training. Experiments show that the proposed training strategy achieves remarkable performance and outperforms a state-of-the-art baseline in both naturalness and expressiveness. To our best knowledge, this is the first study to incorporate utterance level perceptual quality as a loss function into Tacotron training for improved expressiveness.

cs.SD↗

arXiv2021

GraphSpeech: Syntax-Aware Graph Attention Network For Neural Speech Synthesis

Attention-based end-to-end text-to-speech synthesis (TTS) is superior to conventional statistical methods in many ways. Transformer-based TTS is one of such successful implementations. While Transformer TTS models the speech frame sequence well with a self-attention mechanism, it does not associate input text with output utterances from a syntactic point of view at sentence level. We propose a novel neural TTS model, denoted as GraphSpeech, that is formulated under graph neural network framework. GraphSpeech encodes explicitly the syntactic relation of input lexical tokens in a sentence, and incorporates such information to derive syntactically motivated character embeddings for TTS attention mechanism. Experiments show that GraphSpeech consistently outperforms the Transformer TTS baseline in terms of spectrum and prosody rendering of utterances.

cs.LG↗

arXiv2021

Robot Inner Attention Modeling for Task-Adaptive Teaming of Heterogeneous Multi Robots

Attracted by team scale and function diversity, a heterogeneous multi-robot system (HMRS), where multiple robots with different functions and numbers are coordinated to perform tasks, has been widely used for complex and large-scale scenarios, including disaster search and rescue, site surveillance, and social security. However, due to the variety of the task requirements, it is challenging to accurately compose a robot team with appropriate sizes and functions to dynamically satisfy task needs while limiting the robot resource cost to a low level. To solve this problem, in this paper, a novel adaptive cooperation method, inner attention (innerATT), is developed to flexibly team heterogeneous robots to execute tasks as task types and environment change. innerATT is designed by integrating a novel attention mechanism into a multi-agent actor-critic reinforcement learning architecture. With an attention mechanism, robot capability will be analyzed to flexibly form teams to meet task requirements. Scenarios with different task variety ("Single Task", "Double Task", and "Mixed Task") were designed. The effectiveness of the innerATT was validated by its accuracy in flexible cooperation.

cs.RO↗

arXiv2021

Meta Preference Learning for Fast User Adaptation in Human-Supervisory Multi-Robot Deployments

As multi-robot systems (MRS) are widely used in various tasks such as natural disaster response and social security, people enthusiastically expect an MRS to be ubiquitous that a general user without heavy training can easily operate. However, humans have various preferences on balancing between task performance and safety, imposing different requirements onto MRS control. Failing to comply with preferences makes people feel difficult in operation and decreases human willingness of using an MRS. Therefore, to improve social acceptance as well as performance, there is an urgent need to adjust MRS behaviors according to human preferences before triggering human corrections, which increases cognitive load. In this paper, a novel Meta Preference Learning (MPL) method was developed to enable an MRS to fast adapt to user preferences. MPL based on meta learning mechanism can quickly assess human preferences from limited instructions; then, a neural network based preference model adjusts MRS behaviors for preference adaption. To validate method effectiveness, a task scenario "An MRS searches victims in an earthquake disaster site" was designed; 20 human users were involved to identify preferences as "aggressive", "medium", "reserved"; based on user guidance and domain knowledge, about 20,000 preferences were simulated to cover different operations related to "task quality", "task progress", "robot safety". The effectiveness of MPL in preference adaption was validated by the reduced duration and frequency of human interventions.

cs.RO↗

arXiv2021

Radial velocity map of solar wind transients in the field of view of STEREO/HI1 on 3 and 4 April 2010

The solar wind transients propagating out in the inner heliosphere can be observed in white-light images from Heliospheric Imager-1 (HI1), an instrument of the Sun Earth Connection Coronal and Heliospheric Investigation (SECCHI) on board the Solar Terrestrial Relations Observatory (STEREO), from two perspectives. The spatial velocity distribution inside solar wind transients is key to understanding their dynamic evolution processes. We generated a velocity map of transients in 3D space based on 2D white-light images and used it to estimate the expansion rate as well as some kinematic properties of solar wind transients. Based on the recently developed correlation-aided reconstruction (CORAR) method in our previous work, which can recognize and locate 3D solar wind transients from STEREO/HI1 image data, we further developped a new technique for deriving the spatial distribution of the radial velocities of the most pronounced features inside solar wind transients. The technique was applied to events including a coronal mass ejection (CME) and three small-scale transients, so-called blobs, observed by HI1 on 3-4 April 2010 to reconstruct their radial velocity maps. The results match the forward-modeling results, simulations, and in situ observations at $1$ AU fairly well. According to the obtained spatial distributions of height and radial velocity of the CME, we analyzed the self-similarity of the radial expansion of the CME ejecta. The dimensionless radial expansion rate of the northern and middle parts of the CME ejecta varies in the range of 0.7 - 1.0 at heliocentric distance between 25 Rs and 55Rs and the rate of the southern part in the range of 0.3 - 0.5, suggesting that the CME structure was distorted and shaped by the ambient solar wind. The technique we developed is expected to be applied to more events.

astro-ph.SR↗

arXiv2021

Trust Aware Emergency Response for A Resilient Human-Swarm Cooperative System

A human-swarm cooperative system, which mixes multiple robots and a human supervisor to form a heterogeneous team, is widely used for emergent scenarios such as criminal tracking in social security and victim assistance in a natural disaster. These emergent scenarios require a cooperative team to quickly terminate the current task and transit the system to a new task, bringing difficulty in motion planning. Moreover, due to the immediate task transitions, uncertainty from both physical systems and prior tasks is accumulated to decrease swarm performance, causing robot failures and influencing the cooperation effectiveness between the human and the robot swarm. Therefore, given the quick-transition requirements and the introduced uncertainty, it is challenging for a human-swarm system to respond to emergent tasks, compared with executing normal tasks where a gradual transition between tasks is allowed. Human trust reveals the behavior expectations of others and is used to adjust unsatisfactory behaviors for better cooperation. Inspired by human trust, in this paper, a trust-aware reflective control (Trust-R) is developed to dynamically calibrate human-swarm cooperation. Trust-R, based on a weighted mean subsequence reduced algorithm (WMSR) and human trust modeling, helps a swarm to self-reflect its performance from the perspective of human trust; then proactively correct its faulty behaviors in an early stage before a human intervenes. One typical task scenario {emergency response} was designed in the real-gravity simulation environment, and a human user study with 145 volunteers was conducted. Trust-R's effectiveness in correcting faulty behaviors in emergency response was validated by the improved swarm performance and increased trust scores.

cs.RO↗

arXiv2021

Correcting Doppler Shifts in He II 30.38 nm Line by Using the EVE and AIA data from Solar Dynamics Observatory

The Extreme-ultraviolet Variability Experiment (EVE) onboard the Solar Dynamics Observatory (SDO) detects the solar EUV spectra with high temporal cadence and spectral resolution. The wavelength shifts of emission lines provide key information of the dynamics of the Sun. However, some of EVE spectral observations are influenced by the non-uniformly distributed irradiance on the Sun, which may prevent us from correctly understanding the physical processes happened in the solar corona. Here, based on the only published on-orbit calibration data of EVE He II 30.38 nm line on 27 Jan 2011 (Chamberlin, 2016), we develop a method to correct the He II 30.38 nm line by using AIA 304 imaging data. This correction method is then applied to EVE He II 30.38 nm data from 29 Oct 2010 to 3 Mar 2011 to study the Doppler oscillations of the solar He II 30.38 nm line, in which we show that the half-month periodic Doppler oscillation is caused by non-uniformly distributed irradiance mainly due to the presence of active regions. Other EVE coronal lines also present similar Doppler oscillations, suggesting that an appropriate correction must be implemented before interpret the oscillation phenomena appearing in these lines.

astro-ph.SR↗

arXiv2021

Seen and Unseen emotional style transfer for voice conversion with a new emotional speech dataset

Emotional voice conversion aims to transform emotional prosody in speech while preserving the linguistic content and speaker identity. Prior studies show that it is possible to disentangle emotional prosody using an encoder-decoder network conditioned on discrete representation, such as one-hot emotion labels. Such networks learn to remember a fixed set of emotional styles. In this paper, we propose a novel framework based on variational auto-encoding Wasserstein generative adversarial network (VAW-GAN), which makes use of a pre-trained speech emotion recognition (SER) model to transfer emotional style during training and at run-time inference. In this way, the network is able to transfer both seen and unseen emotional style to a new utterance. We show that the proposed framework achieves remarkable performance by consistently outperforming the baseline framework. This paper also marks the release of an emotional speech dataset (ESD) for voice conversion, which has multiple speakers and languages.

cs.SD↗

arXiv2021

How flux feeding causes eruptions of solar magnetic flux ropes with the hyperbolic flux tube configuration?

Coronal magnetic flux ropes are generally considered to be the core structure of large-scale solar eruptions. Recent observations found that solar eruptions could be initiated by a sequence of "flux feeding," during which chromospheric fibrils rise upward from below, and merge with a pre-existing prominence. Further theoretical study has confirmed that the flux feeding mechanism is efficient in causing the eruption of flux ropes that are wrapped by bald patch separatrix surfaces. But it is unclear how flux feeding influences coronal flux ropes that are wrapped by hyperbolic flux tubes (HFT), and whether it is able to cause the flux-rope eruption. In this paper, we use a 2.5-dimensional magnetohydrodynamic model to simulate the flux feeding processes in HFT configurations. It is found that flux feeding injects axial magnetic flux into the flux rope, whereas the poloidal flux of the rope is reduced after flux feeding. Flux feeding is able to cause the flux rope to erupt, provided that the injected axial flux is large enough so that the critical axial flux of the rope is reached. Otherwise, the flux rope system evolves to a stable equilibrium state after flux feeding, which might be even farther away from the onset of the eruption, indicating that flux feeding could stabilize the rope system with the HFT configuration in this circumstance.

astro-ph.SR↗

arXiv2021

Erupting Magnetic Flux Rope Affects Running Penumbral Waves

It is well known that solar flares have broad impacts on the low atmosphere, but it is largely unknown how they affect sunspot waves and oscillations. It is also under debate as to whether the flare-induced photospheric changes are due to the momentum conservation with coronal mass ejections or due to magnetic reconnection. To shed light on the so-called "back reaction" of solar eruptions, we investigated how running penumbral waves (RPWs) at one foot of an erupting magnetic flux rope (MFR) responds to the rope buildup and subsequent erosion. During the rope-buildup stage, the western foot of the rope, which is completely enclosed by a hooked ribbon, expands rapidly and consequently overlaps a sunspot penumbra. This converts the original penumbral field into the rope field, which is associated with a transient increase in electric currents flowing through the ribbon-swept penumbral region. During the rope-erosion stage, the rope foot shrinks as the eastern section of the hooked ribbon slowly sweeps the same penumbral region, where the rope field is converted into flare loops. This conversion induces mixed effects on the photospheric field inclination but heats up the low atmosphere at the footpoints of these flare loops to transition-region temperatures, therefore resulting in the post-eruption RPWs with an enhanced contrast in the 1600A passband and an extended bandwidth to low frequencies at 3-5 mHz, compared with the pre-eruption RPWs that peak at 6 mHz. This observation clearly demonstrates that it is the magnetic reconnection in the corona that impacts the low atmosphere and leads to the changing behaviors of RPWs, which, in turn, offer a new window to diagnose flare reconnections.

astro-ph.SR↗

arXiv2021

A Physics-Informed Machine Learning Model for Porosity Analysis in Laser Powder Bed Fusion Additive Manufacturing

To control part quality, it is critical to analyze pore generation mechanisms, laying theoretical foundation for future porosity control. Current porosity analysis models use machine setting parameters, such as laser angle and part pose. However, these setting-based models are machine dependent, hence they often do not transfer to analysis of porosity for a different machine. To address the first problem, a physics-informed, data-driven model (PIM), which instead of directly using machine setting parameters to predict porosity levels of printed parts, it first interprets machine settings into physical effects, such as laser energy density and laser radiation pressure. Then, these physical, machine independent effects are used to predict porosity levels according to pass, flag, fail categories instead of focusing on quantitative pore size prediction. With six learning methods evaluation, PIM proved to achieve good performances with prediction error of 10$\sim$26%. Finally, pore-encouraging influence and pore-suppressing influence were analyzed for quality analysis.

cs.LG↗

arXiv2021

Pre-eruption Splitting of the Double-Decker Structure in a Solar Filament

Solar filaments often erupt partially. Although how they split remains elusive, the splitting process has the potential of revealing the filament structure and eruption mechanism. Here we investigate the pre-eruption splitting of an apparently single filament and its subsequent partial eruption on 2012 September 27. The evolution is characterized by three stages with distinct dynamics. During the quasi-static stage, the splitting proceeds gradually for about 1.5 hrs, with the upper branch rising at a few kilometers per second and displaying swirling motions about its axis. During the precursor stage that lasts for about 10 min, the upper branch rises at tens of kilometers per second, with a pair of conjugated dimming regions starting to develop at its footpoints; with the swirling motions turning chaotic, the axis of the upper branch whips southward, which drives an arc-shaped EUV front propagating in the similar direction. During the eruption stage, the upper branch erupts with the onset of a C3.7-class two-ribbon flare, while the lower branch remains stable. Judging from the well separated footpoints of the upper branch from those of the lower one, we suggest that the pre-eruption filament processes a double-decker structure composed of two distinct flux bundles, whose formation is associated with gradual magnetic flux cancellations and converging photospheric flows around the polarity inversion line.

astro-ph.SR↗

arXiv2021

Non-equilibrium Flux Rope Formation by Confined Flares Preceding a Solar Coronal Mass Ejection

We present evidence that a magnetic flux rope was formed before a coronal mass ejection (CME) and its associated long-duration flare during a pair of preceding confined eruptions and associated impulsive flares in a compound event in NOAA Active Region 12371. Extreme-ultraviolet images and the extrapolated nonlinear force-free field show that the first two, impulsive flares, SOL2015-06-21T01:42, result from the confined eruption of highly sheared low-lying flux, presumably a seed flux rope. The eruption spawns a vertical current sheet, where magnetic reconnection creates flare ribbons and loops, a nonthermal microwave source, and a sigmoidal hot channel which can only be interpreted as a magnetic flux rope. Until the subsequent long-duration flare, SOL2015-06-21T02:36, the sigmoid's elbows expand, while its center remains stationary, suggesting non-equilibrium but not yet instability. The "flare reconnection" during the confined eruptions acts like "tether-cutting reconnection" whose flux feeding of the rope leads to instability. The subsequent full eruption is seen as an accelerated rise of the entire hot channel, seamlessly evolving into the fast halo CME. Both the confined and ejective eruptions are consistent with the onset of the torus instability in the dipped decay index profile which results from the region's two-scale magnetic structure. We suggest that the formation or enhancement of a non-equilibrium but stable flux rope by confined eruptions is a generic process occurring prior to many CMEs.

astro-ph.SR↗

arXiv2020

Instrument for in situ hard x-ray nanobeam characterization during epitaxial crystallization and materials transformations

Solid-phase epitaxy (SPE) and other three-dimensional epitaxial crystallization processes pose challenging structural and chemical characterization problems. The concentration of defects, the spatial distribution of elastic strain, and the chemical state of ions each vary with nanoscale characteristic length scales and depend sensitively on the gas environment and elastic boundary conditions during growth. The lateral or three-dimensional propagation of crystalline interfaces in SPE has nanoscale or submicron characteristic distances during typical crystallization times. An in situ synchrotron hard x-ray instrument allows these features to be studied during deposition and crystallization using diffraction, resonant scattering, nanobeam and coherent diffraction imaging, and reflectivity. The instrument incorporates a compact deposition system allowing the use of short-working-distance x-ray focusing optics. Layers are deposited using radio-frequency magnetron sputtering and evaporation sources. The deposition system provides control of the gas atmosphere and sample temperature. The sample is positioned using a stable mechanical design to minimize vibration and drift and employs precise translation stages to enable nanobeam experiments. Results of in situ x-ray characterization of the amorphous thin film deposition process for a SrTiO3/BaTiO3 multilayer illustrate implementation of this instrument.

cond-mat.mtrl-sci↗

arXiv2020

A Gradual Takeover Strategy of the Active Safety System

A gradual takeover strategy is proposed, in which the dynamic driving privilege assignment in real-time and the driving privilege gradual handover are realized. Firstly, the driving privilege assignment based on the risk level is achieved. The naturalistic driving data is applied to study the driver behavior during danger. TTC (time to collision) is defined as an obvious risk measure, whereas the time before the host vehicle has to brake assuming that the target vehicle is braking is defined as the potential risk measure, i.e. the time margin (TM). A risk assessment algorithm is proposed based on the obvious risk and potential risk. Secondly, the driving privilege gradual handover is realized. The non-cooperative MPC (model predictive control) is employed to resolve the conflicts between the driver and active safety system. The naturalistic driving data are applied to verify the effectiveness of the risk assessment algorithm, and the risk assessment algorithm performs better than TTC in the ROC (receiver operating characteristic). It is identified that the Nash equilibrium of the non-cooperative MPC can be achieved by using a non-iterative method. The driving privilege gradual handover is realized by using the confidence matrixes updating. The simulation verification shows that the gradual takeover strategy can achieve the driving privilege gradual handover between the driver and active safety system.

eess.SY↗

arXiv2020

Temporal Difference Learning as Gradient Splitting

Temporal difference learning with linear function approximation is a popular method to obtain a low-dimensional approximation of the value function of a policy in a Markov Decision Process. We give a new interpretation of this method in terms of a splitting of the gradient of an appropriately chosen function. As a consequence of this interpretation, convergence proofs for gradient descent can be applied almost verbatim to temporal difference learning. Beyond giving a new, fuller explanation of why temporal difference works, our interpretation also yields improved convergence times. We consider the setting with $1/\sqrt{T}$ step-size, where previous comparable finite-time convergence time bounds for temporal difference learning had the multiplicative factor $1/(1-γ)$ in front of the bound, with $γ$ being the discount factor. We show that a minor variation on TD learning which estimates the mean of the value function separately has a convergence time where $1/(1-γ)$ only multiplies an asymptotically negligible term.

cs.LG↗