arXiv subjects

Xin Liu

Publications and source records attributed to Xin Liu.

At least 847 records · Page 47Linked to original sources

arXiv2020

Adaptive Federated Dropout: Improving Communication Efficiency and Generalization for Federated Learning

With more regulations tackling users' privacy-sensitive data protection in recent years, access to such data has become increasingly restricted and controversial. To exploit the wealth of data generated and located at distributed entities such as mobile phones, a revolutionary decentralized machine learning setting, known as Federated Learning, enables multiple clients located at different geographical locations to collaboratively learn a machine learning model while keeping all their data on-device. However, the scale and decentralization of federated learning present new challenges. Communication between the clients and the server is considered a main bottleneck in the convergence time of federated learning. In this paper, we propose and study Adaptive Federated Dropout (AFD), a novel technique to reduce the communication costs associated with federated learning. It optimizes both server-client communications and computation costs by allowing clients to train locally on a selected subset of the global model. We empirically show that this strategy, combined with existing compression methods, collectively provides up to 57x reduction in convergence time. It also outperforms the state-of-the-art solutions for communication efficiency. Furthermore, it improves model generalization by up to 1.7%.

cs.LG↗

arXiv2020

Bandit Policies for Reliable Cellular Network Handovers in Extreme Mobility

The demand for seamless Internet access under extreme user mobility, such as on high-speed trains and vehicles, has become a norm rather than an exception. However, the 4G/5G mobile network is not always reliable to meet this demand, with non-negligible failures during the handover between base stations. A fundamental challenge of reliability is to balance the exploration of more measurements for satisfactory handover, and exploitation for timely handover (before the fast-moving user leaves the serving base station's radio coverage). This paper formulates this trade-off in extreme mobility as a composition of two distinct multi-armed bandit problems. We propose Bandit and Threshold Tuning (BATT) to minimize the regret of handover failures in extreme mobility. BATT uses $ε$-binary-search to optimize the threshold of the serving cell's signal strength to initiate the handover procedure with $\mathcal{O}(\log J \log T)$ regret.It further devises opportunistic Thompson sampling, which optimizes the sequence of the target cells to measure for reliable handover with $\mathcal{O}(\log T)$ regret.Our experiment over a real LTE dataset from Chinese high-speed rails validates significant regret reduction and a 29.1% handover failure reduction.

cs.LG↗

arXiv2020

Next-to-leading-logarithm $k_T$ resummation for $B_c\to J/ψ$ decays

We derive the $k_T$ resummation for a transverse-momentum-dependent charmonium wave function, which involves the bottom quark mass $m_b$, the charm quark mass $m_c$, and the charm quark transverse momentum $k_T$, up to the next-to-leading-logarithm (NLL) accuracy under the hierarchy $m_b\gg m_c \gg k_T$. The resultant Sudakov factor is employed in the perutrbative QCD (PQCD) approach to the $B_c\to J/ψ$ transition form factor $A_0^{B_c \to J/ψ}(0)$ and the $B_c^+ \to J/ψπ^+$ decay. We compare the NLL resummation effect on these processes with the leading-logarithm one in the literature, and find a $(5-10)\%$ enhancement to the form factor $A_0^{B_c \to J/ψ}(0)$ and a $(10-20)\%$ enhancement to the decay rate ${\rm BR}(B_c^+ \to J/ψπ^+)$. The improved $k_T$ resummation formalism is applicable to the PQCD analysis of heavy meson decays to other charmonia.

hep-ph↗

arXiv2020

Advancing Non-Contact Vital Sign Measurement using Synthetic Avatars

Non-contact physiological measurement has the potential to provide low-cost, non-invasive health monitoring. However, machine vision approaches are often limited by the availability and diversity of annotated video datasets resulting in poor generalization to complex real-life conditions. To address these challenges, this work proposes the use of synthetic avatars that display facial blood flow changes and allow for systematic generation of samples under a wide variety of conditions. Our results show that training on both simulated and real video data can lead to performance gains under challenging conditions. We show state-of-the-art performance on three large benchmark datasets and improved robustness to skin type and motion.

cs.CV↗

arXiv2020

Contextual Bandits with Side-Observations

We investigate contextual bandits in the presence of side-observations across arms in order to design recommendation algorithms for users connected via social networks. Users in social networks respond to their friends' activity, and hence provide information about each other's preferences. In our model, when a learning algorithm recommends an article to a user, not only does it observe his/her response (e.g. an ad click), but also the side-observations, i.e., the response of his neighbors if they were presented with the same article. We model these observation dependencies by a graph $\mathcal{G}$ in which nodes correspond to users, and edges correspond to social links. We derive a problem/instance-dependent lower-bound on the regret of any consistent algorithm. We propose an optimization (linear programming) based data-driven learning algorithm that utilizes the structure of $\mathcal{G}$ in order to make recommendations to users and show that it is asymptotically optimal, in the sense that its regret matches the lower-bound as the number of rounds $T\to\infty$. We show that this asymptotically optimal regret is upper-bounded as $O\left(|χ(\mathcal{G})|\log T\right)$, where $|χ(\mathcal{G})|$ is the domination number of $\mathcal{G}$. In contrast, a naive application of the existing learning algorithms results in $O\left(N\log T\right)$ regret, where $N$ is the number of users.

cs.LG↗

arXiv2020

WeightAlign: Normalizing Activations by Weight Alignment

Batch normalization (BN) allows training very deep networks by normalizing activations by mini-batch sample statistics which renders BN unstable for small batch sizes. Current small-batch solutions such as Instance Norm, Layer Norm, and Group Norm use channel statistics which can be computed even for a single sample. Such methods are less stable than BN as they critically depend on the statistics of a single input sample. To address this problem, we propose a normalization of activation without sample statistics. We present WeightAlign: a method that normalizes the weights by the mean and scaled standard derivation computed within a filter, which normalizes activations without computing any sample statistics. Our proposed method is independent of batch size and stable over a wide range of batch sizes. Because weight statistics are orthogonal to sample statistics, we can directly combine WeightAlign with any method for activation normalization. We experimentally demonstrate these benefits for classification on CIFAR-10, CIFAR-100, ImageNet, for semantic segmentation on PASCAL VOC 2012 and for domain adaptation on Office-31.

cs.CV↗

arXiv2020

How machine learning can help the design and analysis of composite materials and structures?

Machine learning models are increasingly used in many engineering fields thanks to the widespread digital data, growing computing power, and advanced algorithms. Artificial neural networks (ANN) is the most popular machine learning model in recent years. Although many ANN models have been used in the design and analysis of composite materials and structures, there are still some unsolved issues that hinder the acceptance of ANN models in the practical design and analysis of composite materials and structures. Moreover, the emerging machine learning techniques are posting new opportunities and challenges in the data-based design paradigm. This paper aims to give a state-of-the-art literature review of ANN models in the nonlinear constitutive modeling, multiscale surrogate modeling, and design optimization of composite materials and structures. This review has been designed to focus on the discussion of the general frameworks and benefits of ANN models to the above problems. Moreover, challenges and opportunities in each key problem are identified and discussed. This paper is expected to open the discussion of future research scope and new directions to enable efficient, robust, and accurate data-driven design and analysis of composite materials and structures.

cond-mat.mtrl-sci↗

arXiv2020

Discovery of a Candidate Binary Supermassive Black Hole in a Periodic Quasar from Circumbinary Accretion Variability

Binary supermassive black holes (BSBHs) are expected to be a generic byproduct from hierarchical galaxy formation. The final coalescence of BSBHs is thought to be the loudest gravitational wave (GW) siren, yet no confirmed BSBH is known in the GW-dominated regime. While periodic quasars have been proposed as BSBH candidates, the physical origin of the periodicity has been largely uncertain. Here we report discovery of a periodicity (P=1607$\pm$7 days) at 99.95% significance (with a global p-value of ~$10^{-3}$ accounting for the look elsewhere effect) in the optical light curves of a redshift 1.53 quasar, SDSS J025214.67-002813.7. Combining archival Sloan Digital Sky Survey data with new, sensitive imaging from the Dark Energy Survey, the total ~20-yr time baseline spans ~4.6 cycles of the observed 4.4-yr (restframe 1.7-yr) periodicity. The light curves are best fit by a bursty model predicted by hydrodynamic simulations of circumbinary accretion disks. The periodicity is likely caused by accretion rate modulation by a milli-parsec BSBH emitting GWs, dynamically coupled to the circumbinary accretion disk. A bursty hydrodynamic variability model is statistically preferred over a smooth, sinusoidal model expected from relativistic Doppler boost, a kinematic effect proposed for PG1302-102. Furthermore, the frequency dependence of the variability amplitudes disfavors Doppler boost, lending independent support to the circumbinary accretion variability hypothesis. Given our detection rate of one BSBH candidate from circumbinary accretion variability out of 625 quasars, it suggests that future large, sensitive synoptic surveys such as the Vera C. Rubin Observatory Legacy Survey of Space and Time may be able to detect hundreds to thousands of candidate BSBHs from circumbinary accretion with direct implications for Laser Interferometer Space Antenna.

astro-ph.GA↗

arXiv2020

Intrinsic Andreev $π$-reflection and Josephson $π$-junction for centrosymmetric spin-triplet superconductors

In this work, we systematically study two phases, called Andreev $π$-phase and orbital-phase, and their influence on the Josephson effect. When the system is time-reversal invariant and centrosymmetric, these two phases only appear in the odd-parity pairings. The Andreev $π$-phase has nothing to do with the specific form of the odd-parity pairings and means an intrinsic $π$-phase between the spin-triplet Cooper pairs entering and leaving CTSCs in the Andreev reflections. The orbital-phase corresponds to the phase difference between the spin-triplet Cooper pairs with opposite spin polarization and depends on the specific form of the odd-parity gap functions. When the normal region of the Josephson junction contacts the same side of the CTSCs with some specific odd-parity parings, the competition between the two phases can lead to the Josephson $π$-junction. Note that this junction is different from that of the conventional Josephson junction (JJ) and is dubbed a U-shaped junction according to its geometry. Meanwhile, in a conventional JJ, the interplay of these two phases causes their impact on the CPR to be completely canceled out. Therefore no matter what kind of pairing symmetries the CTSC has, it will lead to Josephson 0-junction in this case. We obtain our results based on the model of the M$_{x}$Bi$_2$Se$_3$ family where M may be Cu, Sr, or Nb. Therefore, we propose to detect the pairing symmetry of M$_{x}$Bi$_2$Se$_3$ through a superconducting quantum interference device containing a U-shaped Josephson junction.

cond-mat.supr-con↗

arXiv2020

Candidate Periodically Variable Quasars from the Dark Energy Survey and the Sloan Digital Sky Survey

Periodically variable quasars have been suggested as close binary supermassive black holes. We present a systematic search for periodic light curves in 625 spectroscopically confirmed quasars with a median redshift of 1.8 in a 4.6 deg$^2$ overlapping region of the Dark Energy Survey Supernova (DES-SN) fields and the Sloan Digital Sky Survey Stripe 82 (SDSS-S82). Our sample has a unique 20-year long multi-color ($griz$) light curve enabled by combining DES-SN Y6 observations with archival SDSS-S82 data. The deep imaging allows us to search for periodic light curves in less luminous quasars (down to $r{\sim}$23.5 mag) powered by less massive black holes (with masses $\gtrsim10^{8.5}M_{\odot}$) at high redshift for the first time. We find five candidates with significant (at $>$99.74% single-frequency significance in at least two bands with a global p-value of $\sim$7$\times10^{-4}$--3$\times10^{-3}$ accounting for the look-elsewhere effect) periodicity with observed periods of $\sim$3--5 years (i.e., 1--2 years in rest frame) having $\sim$4--6 cycles spanned by the observations. If all five candidates are periodically variable quasars, this translates into a detection rate of ${\sim}0.8^{+0.5}_{-0.3}$% or ${\sim}1.1^{+0.7}_{-0.5}$ quasar per deg$^2$. Our detection rate is 4--80 times larger than those found by previous searches using shallower surveys over larger areas. This discrepancy is likely caused by differences in the quasar populations probed and the survey data qualities. We discuss implications on the future direct detection of low-frequency gravitational waves. Continued photometric monitoring will further assess the robustness and characteristics of these candidate periodic quasars to determine their physical origins.

astro-ph.HE↗

arXiv2020

Topological Kondo Device for distinguishing Quasi-Majorana and Majorana signatures

To confirm the Majorana signatures, significant effort has been devoted to distinguishing between Majorana zero modes (MZMs) and spatially separated quasi-Majorana modes (QMMs). Because both MZMs and QMMs cause a quantized zero-bias peak in the conductance measurement, their verification task is thought to be very difficult. Here, we proposed a simple device with a single nanowire, where the device could develop clear evidence of the topological Kondo effect in the topologically trivial phase with four QMMs. On the other hand, in the topological superconducting phase with MZMs, the transport signatures are significantly different. Therefore, our scheme provides a simple way to distinguish Majorana and quasi-Majorana modes.

cond-mat.mes-hall↗

arXiv2020

ENAS4D: Efficient Multi-stage CNN Architecture Search for Dynamic Inference

Dynamic inference is a feasible way to reduce the computational cost of convolutional neural network(CNN), which can dynamically adjust the computation for each input sample. One of the ways to achieve dynamic inference is to use multi-stage neural network, which contains a sub-network with prediction layer at each stage. The inference of a input sample can exit from early stage if the prediction of the stage is confident enough. However, design a multi-stage CNN architecture is a non-trivial task. In this paper, we introduce a general framework, ENAS4D, which can efficiently search for optimal multi-stage CNN architecture for dynamic inference in a well-designed search space. Firstly, we propose a method to construct the search space with multi-stage convolution. The search space include different numbers of layers, different kernel sizes and different numbers of channels for each stage and the resolution of input samples. Then, we train a once-for-all network that supports to sample diverse multi-stage CNN architecture. A specialized multi-stage network can be obtained from the once-for-all network without additional training. Finally, we devise a method to efficiently search for the optimal multi-stage network that trades the accuracy off the computational cost taking the advantage of once-for-all network. The experiments on the ImageNet classification task demonstrate that the multi-stage CNNs searched by ENAS4D consistently outperform the state-of-the-art method for dyanmic inference. In particular, the network achieves 74.4% ImageNet top-1 accuracy under 185M average MACs.

cs.CV↗

arXiv2020

Optical Variability of the Dwarf AGN NGC 4395 from the Transiting Exoplanet Survey Satellite

We present optical light curves from the Transiting Exoplanet Survey Satellite (TESS) for the archetypical dwarf active galactic nucleus (AGN) in the nearby galaxy NGC 4395 hosting a $\sim 10^5\,M_\odot$ supermassive black hole (SMBH). Significant variability is detected on timescales from weeks to hours before reaching the background noise level. The $\sim$month-long, 30 minute-cadence, high-precision TESS light curve can be well fit by a simple damped random walk (DRW) model, with the damping timescale $τ_{\rm DRW}$ constrained to be $2.3_{-0.7}^{+1.8}$~days ($1σ$). NGC 4395 lies almost exactly on the extrapolation of the $τ_{\rm DRW}-M_{\rm BH}$ relation measured for AGNs with BH masses that are more than three orders of magnitude larger. The optical variability periodogram can be well fit by a broken power law with the high-frequency slope ($-1.88\pm0.15$) and the characteristic timescale ($τ_{\rm br}\equiv 1/(2πf_{\rm br})=1.4_{-0.5}^{+1.9}\,$days) consistent with the DRW model within 1$σ$. This work demonstrates the power of TESS light curves in identifying low-mass accreting SMBHs with optical variability, and a potential global $τ_{\rm DRW}-M_{\rm BH}$ relation that can be used to estimate SMBH masses with optical variability measurements.

astro-ph.GA↗

arXiv2020

A Variational Approach to Unsupervised Sentiment Analysis

In this paper, we propose a variational approach to unsupervised sentiment analysis. Instead of using ground truth provided by domain experts, we use target-opinion word pairs as a supervision signal. For example, in a document snippet "the room is big," (room, big) is a target-opinion word pair. These word pairs can be extracted by using dependency parsers and simple rules. Our objective function is to predict an opinion word given a target word while our ultimate goal is to learn a sentiment classifier. By introducing a latent variable, i.e., the sentiment polarity, to the objective function, we can inject the sentiment classifier to the objective function via the evidence lower bound. We can learn a sentiment classifier by optimizing the lower bound. We also impose sophisticated constraints on opinion words as regularization which encourages that if two documents have similar (dissimilar) opinion words, the sentiment classifiers should produce similar (different) probability distribution. We apply our method to sentiment analysis on customer reviews and clinical narratives. The experiment results show our method can outperform unsupervised baselines in sentiment analysis task on both domains, and our method obtains comparable results to the supervised method with hundreds of labels per aspect in customer reviews domain, and obtains comparable results to supervised methods in clinical narratives domain.

cs.CL↗

arXiv2020

Searching Multi-Rate and Multi-Modal Temporal Enhanced Networks for Gesture Recognition

Gesture recognition has attracted considerable attention owing to its great potential in applications. Although the great progress has been made recently in multi-modal learning methods, existing methods still lack effective integration to fully explore synergies among spatio-temporal modalities effectively for gesture recognition. The problems are partially due to the fact that the existing manually designed network architectures have low efficiency in the joint learning of multi-modalities. In this paper, we propose the first neural architecture search (NAS)-based method for RGB-D gesture recognition. The proposed method includes two key components: 1) enhanced temporal representation via the proposed 3D Central Difference Convolution (3D-CDC) family, which is able to capture rich temporal context via aggregating temporal difference information; and 2) optimized backbones for multi-sampling-rate branches and lateral connections among varied modalities. The resultant multi-modal multi-rate network provides a new perspective to understand the relationship between RGB and depth modalities and their temporal dynamics. Comprehensive experiments are performed on three benchmark datasets (IsoGD, NvGesture, and EgoGesture), demonstrating the state-of-the-art performance in both single- and multi-modality settings.The code is available at https://github.com/ZitongYu/3DCDC-NAS

cs.CV↗

arXiv2020

Symplectic Runge-Kutta discretization of a regularized forward-backward sweep iteration for optimal control problems

Li, Chen, Tai & E. (J. Machine Learning Research, 2018) have proposed a regularization of the forward-backward sweep iteration for solving the Pontryagin maximum principle in optimal control problems. The authors prove the global convergence of the iteration in the continuous time case. In this article we show that their proof can be extended to the case of numerical discretization by symplectic Runge-Kutta pairs. We demonstrate the convergence with a simple numerical experiment.

math.NA↗

arXiv2020

2nd Place Scheme on Action Recognition Track of ECCV 2020 VIPriors Challenges: An Efficient Optical Flow Stream Guided Framework

To address the problem of training on small datasets for action recognition tasks, most prior works are either based on a large number of training samples or require pre-trained models transferred from other large datasets to tackle overfitting problems. However, it limits the research within organizations that have strong computational abilities. In this work, we try to propose a data-efficient framework that can train the model from scratch on small datasets while achieving promising results. Specifically, by introducing a 3D central difference convolution operation, we proposed a novel C3D neural network-based two-stream (Rank Pooling RGB and Optical Flow) framework for the task. The method is validated on the action recognition track of the ECCV 2020 VIPriors challenges and got the 2nd place (88.31%). It is proved that our method can achieve a promising result even without a pre-trained model on large scale datasets. The code will be released soon.

cs.CV↗

arXiv2020

Deep Low-rank Prior in Dynamic MR Imaging

The deep learning methods have achieved attractive performance in dynamic MR cine imaging. However, all of these methods are only driven by the sparse prior of MR images, while the important low-rank (LR) prior of dynamic MR cine images is not explored, which limits the further improvements on dynamic MR reconstruction. In this paper, a learned singular value thresholding (Learned-SVT) operation is proposed to explore deep low-rank prior in dynamic MR imaging for obtaining improved reconstruction results. In particular, we come up with two novel and distinct schemes to introduce the learnable low-rank prior into deep network architectures in an unrolling manner and a plug-and-play manner respectively. In the unrolling manner, we put forward a model-based unrolling sparse and low-rank network for dynamic MR imaging, dubbed SLR-Net. The SLR-Net is defined over a deep network flow graph, which is unrolled from the iterative procedures in the Iterative Shrinkage-Thresholding Algorithm (ISTA) for optimizing a sparse and low-rank based dynamic MRI model. In the plug-and-play manner, we present a plug-and-play LR network module that can be easily embedded into any other dynamic MR neural networks without changing the network paradigm. Experimental results show that both schemes can further improve the state-of-the-art CS methods, such as k-t SLR, and sparsity-driven deep learning-based methods, such as DC-CNN and CRNN, both qualitatively and quantitatively.

eess.IV↗