arXiv subjects

Qiang Liu

Publications and source records attributed to Qiang Liu.

At least 451 records · Page 25Linked to original sources

arXiv2019

Variational Inference with Tail-adaptive f-Divergence

Variational inference with α-divergences has been widely used in modern probabilistic machine learning. Compared to Kullback-Leibler (KL) divergence, a major advantage of using α-divergences (with positive α values) is their mass-covering property. However, estimating and optimizing α-divergences require to use importance sampling, which could have extremely large or infinite variances due to heavy tails of importance weights. In this paper, we propose a new class of tail-adaptive f-divergences that adaptively change the convex function f with the tail of the importance weights, in a way that theoretically guarantees finite moments, while simultaneously achieving mass-covering properties. We test our methods on Bayesian neural networks, as well as deep reinforcement learning in which our method is applied to improve a recent soft actor-critic (SAC) algorithm. Our results show that our approach yields significant advantages compared with existing methods based on classical KL and α-divergences.

cs.LG↗

arXiv2019

Improving Neural Language Modeling via Adversarial Training

Recently, substantial progress has been made in language modeling by using deep neural networks. However, in practice, large scale neural language models have been shown to be prone to overfitting. In this paper, we present a simple yet highly effective adversarial training mechanism for regularizing neural language models. The idea is to introduce adversarial noise to the output embedding layer while training the models. We show that the optimal adversarial noise yields a simple closed-form solution, thus allowing us to develop a simple and time efficient algorithm. Theoretically, we show that our adversarial mechanism effectively encourages the diversity of the embedding vectors, helping to increase the robustness of models. Empirically, we show that our method improves on the single model state-of-the-art results for language modeling on Penn Treebank (PTB) and Wikitext-2, achieving test perplexity scores of 46.01 and 38.07, respectively. When applied to machine translation, our method improves over various transformer-based translation baselines in BLEU scores on the WMT14 English-German and IWSLT14 German-English tasks.

cs.LG↗

arXiv2019

Polar Codes with Memory

Polar codes with memory (PCM) are proposed in this paper: a pair of consecutive code blocks containing a controlled number of mutual information bits. The shared mutual information bits of the succeeded block can help the failed block to recover. The underlying polar codes can employ any decoding scheme such as the successive cancellation (SC) decoding (PCM-SC), the belief propagation (BP) decoding (PCM-BP), and the successive cancellation list (SCL) decoding (PCM-SCL). The analysis shows that the packet error rate (PER) of PCM decreases to the order of PER squared while maintaining the same complexity as the underlying polar codes. Simulation results indicate that for PCM-SC, the PER is comparable to (less than 0.3 dB) the stand-alone SCL decoding with two lists for the block length $N=256$. The PER of PCM-SCL with $L$ lists can match that of the stand-alone SCL decoding with $2L$ lists. Two hardware decoders for PCM are also implemented: the in-serial (IS) decoder and the low-latency interleaved (LLI) decoder. For $N=256$, synthesis results show that in the worst case, the latency of the PCM LLI decoder is only $16.1\%$ of the adaptive SCL decoder with $L=2$, while the throughput is improved by 13 times compared to it.

cs.IT↗

arXiv2019

Learning Belief Representations for Imitation Learning in POMDPs

We consider the problem of imitation learning from expert demonstrations in partially observable Markov decision processes (POMDPs). Belief representations, which characterize the distribution over the latent states in a POMDP, have been modeled using recurrent neural networks and probabilistic latent variable models, and shown to be effective for reinforcement learning in POMDPs. In this work, we investigate the belief representation learning problem for generative adversarial imitation learning in POMDPs. Instead of training the belief module and the policy separately as suggested in prior work, we learn the belief module jointly with the policy, using a task-aware imitation loss to ensure that the representation is more aligned with the policy's objective. To improve robustness of representation, we introduce several informative belief regularization techniques, including multi-step prediction of dynamics and action-sequences. Evaluated on various partially observable continuous-control locomotion tasks, our belief-module imitation learning approach (BMIL) substantially outperforms several baselines, including the original GAIL algorithm and the task-agnostic belief learning algorithm. Extensive ablation analysis indicates the effectiveness of task-aware belief learning and belief regularization.

cs.LG↗

arXiv2019

Sequence Modeling of Temporal Credit Assignment for Episodic Reinforcement Learning

Recent advances in deep reinforcement learning algorithms have shown great potential and success for solving many challenging real-world problems, including Go game and robotic applications. Usually, these algorithms need a carefully designed reward function to guide training in each time step. However, in real world, it is non-trivial to design such a reward function, and the only signal available is usually obtained at the end of a trajectory, also known as the episodic reward or return. In this work, we introduce a new algorithm for temporal credit assignment, which learns to decompose the episodic return back to each time-step in the trajectory using deep neural networks. With this learned reward signal, the learning efficiency can be substantially improved for episodic reinforcement learning. In particular, we find that expressive language models such as the Transformer can be adopted for learning the importance and the dependency of states in the trajectory, therefore providing high-quality and interpretable learned reward signals. We have performed extensive experiments on a set of MuJoCo continuous locomotive control tasks with only episodic returns and demonstrated the effectiveness of our algorithm.

cs.LG↗

arXiv2019

PI-BA Bundle Adjustment Acceleration on Embedded FPGAs with Co-observation Optimization

Bundle adjustment (BA) is a fundamental optimization technique used in many crucial applications, including 3D scene reconstruction, robotic localization, camera calibration, autonomous driving, space exploration, street view map generation etc. Essentially, BA is a joint non-linear optimization problem, and one which can consume a significant amount of time and power, especially for large optimization problems. Previous approaches of optimizing BA performance heavily rely on parallel processing or distributed computing, which trade higher power consumption for higher performance. In this paper we propose π-BA, the first hardware-software co-designed BA engine on an embedded FPGA-SoC that exploits custom hardware for higher performance and power efficiency. Specifically, based on our key observation that not all points appear on all images in a BA problem, we designed and implemented a Co-Observation Optimization technique to accelerate BA operations with optimized usage of memory and computation resources. Experimental results confirm that π-BA outperforms the existing software implementations in terms of performance and power consumption.

eess.IV↗

arXiv2019

Pulse strategy for suppressing spreading on networks

In networked spreading models, each node can infect its neighbors and cure spontaneously. The curing is assumed to occur uniformly over time. A pulse immunization/curing strategy is more efficient and broadly applied to suppressing spreading processes. We model the epidemic process by the basic Susceptible-Infected (SI) process with a pulse curing and incorporate the underlying contact network. The mean-field epidemic threshold of the pulse SI model is shown to be $\frac{1}{λ_1}\ln\frac{1}{1-p}$, where $λ_1$ and $p$ are the largest eigenvalue of the adjacency matrix and the fraction of nodes covered by each curing, respectively. Compared to the extensively studied uniform curing process, we show that the pulse curing strategy saves about $36.8$\%, i.e. $p\approx 0.632$, of the number of curing operations invariant to the network structure. Our results may help related policy makers to estimate the cost of controlling spreading processes.

physics.soc-ph↗

arXiv2019

Acoustic-optic Q-switched cavityless weak-feedback laser based on Nd:GdVO4 bounce geometry

An AOQ (acoustic-optic Q-switched) laser with cavityless weak-feedback configuration is demonstrated based on a Nd:GdVO4 bounce geometry. The laser can operate at a repetition up to 500 kHz with the output power above 4W. The pulse-width at 100 kHz reaches 5.2 ns, which is 4.3 times the round-trip time. A theory of Q-switched cavityless weak-feedback laser is proposed for the first time, to our best knowledge. This theory is very suitable for analyzing short pulses comparable to round-trip time. By utilizing the theory, simulation is implemented for our experimental conditions. The consistence between the simulation and the experimental results proves the validity of our theory.

physics.optics↗

arXiv2019

Short pulse close to round-trip time generated by cavityless high gain Nd:GdVO4 bounce geometry

In this paper, laser pulses with pulse-widths approach to the round-trip time are generated by utilizing a cavityless high gain Nd:GdVO4 bounce geometry. By adopting an EOQ (electro-optics Q-switch), pulse-widths of 1.36 ns, 1.82 ns, and 2.39 ns are achieved at three effective cavity lengths respectively. All these pulse-widths are close to the round-trip time of corresponding effective cavity lengths. Moreover, the output power reaches watt-level and the repetition rate is kHz-level, meanwhile the M2 factor is less than 1.3. Spectrally, the laser has a continuous spectrum with 10 dB linewidth of 0.2 nm.

physics.optics↗

arXiv2019

Learning Self-Imitating Diverse Policies

The success of popular algorithms for deep reinforcement learning, such as policy-gradients and Q-learning, relies heavily on the availability of an informative reward signal at each timestep of the sequential decision-making process. When rewards are only sparsely available during an episode, or a rewarding feedback is provided only after episode termination, these algorithms perform sub-optimally due to the difficultly in credit assignment. Alternatively, trajectory-based policy optimization methods, such as cross-entropy method and evolution strategies, do not require per-timestep rewards, but have been found to suffer from high sample complexity by completing forgoing the temporal nature of the problem. Improving the efficiency of RL algorithms in real-world problems with sparse or episodic rewards is therefore a pressing need. In this work, we introduce a self-imitation learning algorithm that exploits and explores well in the sparse and episodic reward settings. We view each policy as a state-action visitation distribution and formulate policy optimization as a divergence minimization problem. We show that with Jensen-Shannon divergence, this divergence minimization problem can be reduced into a policy-gradient algorithm with shaped rewards learned from experience replays. Experimental results indicate that our algorithm works comparable to existing algorithms in environments with dense rewards, and significantly better in environments with sparse and episodic rewards. We then discuss limitations of self-imitation learning, and propose to solve them by using Stein variational policy gradient descent with the Jensen-Shannon kernel to learn multiple diverse policies. We demonstrate its effectiveness on a challenging variant of continuous-control MuJoCo locomotion tasks.

stat.ML↗

arXiv2019

Exoplanets in the Antarctic sky. II. 116 Transiting Exoplanet Candidates Found by AST3-II (CHESPA) within the Southern CVZ of TESS

We report first results from the CHinese Exoplanet Searching Program from Antarctica (CHESPA)---a wide-field high-resolution photometric survey for transiting exoplanets carried out using telescopes of the AST3 (Antarctic Survey Telescopes times 3) project. There are now three telescopes (AST3-I, AST3-II, and CSTAR-II) operating at Dome A---the highest point on the Antarctic Plateau---in a fully automatic and remote mode to exploit the superb observing conditions of the site, and its long and uninterrupted polar nights. The search for transiting exoplanets is one of the key projects for AST3. During the Austral winters of 2016 and 2017 we used the AST3-II telescope to survey a set of target fields near the southern ecliptic pole, falling within the continuous viewing zone of the TESS mission \citep{Ricker10}. The first data release of the 2016 data, including images, catalogs and lightcurves of 26578 bright stars ($7.5\le i \le15$) was presented in \citet{Zhang18}. The best precision, as measured by the RMS of the lightcurves at the optimum magnitude of the survey ($i=10$), is around 2\,mmag. We detect 222 objects with plausible transit signals from these data, 116 of which are plausible transiting exoplanet candidates according to their stellar properties as given by the TESS Input Catalog \citep{Stassun17}, Gaia DR2 \citep{Gaia18} and TESS-HERMES spectroscopy \citep{Sharma18}. With the first data release from TESS expected in late 2018, this candidate list will be a timely for improving the rejection of potential false positives.

astro-ph.EP↗

arXiv2018

Exoplanets in the Antarctic Sky. I. The First Data Release of AST3-II (CHESPA) and New Found Variables within the Southern CVZ of TESS

Located at Dome A, the highest point of the Antarctic plateau, the Chinese Kunlun station is considered to be one of the best ground-based photometric sites because of its extremely cold, dry, and stable atmosphere(Saunders et al. 2009). A target can be monitored from there for over 40 days without diurnal interruption during a polar winter. This makes Kunlun station a perfect site to search for short-period transiting exoplanets. Since 2008, an observatory has been built at Kunlun station and three telescopes are working there. Using these telescopes, the AST3 project has been carried out over the last six years with a search for transiting exoplanets as one of its key programs (CHESPA). In the austral winters of 2016 and 2017, a set of target fields in the Southern CVZ of TESS (Ricker et al. 2009) were monitored by the AST3-II telescope. In this paper, we introduce the CHESPA and present the first data release containing photometry of 26,578 bright stars (m_i < 15). The best photometric precision at the optimum magnitude for the survey is around 2 mmag. To demonstrate the data quality, we also present a catalog of 221 variables with a brightness variation greater than 5 mmag from the 2016 data. Among these variables, 179 are newly identified periodic variables not listed in the AAVSO databasea), and 67 are listed in the Candidate Target List(Stassun et al. 2017). These variables will require careful attention to avoid false-positive signals when searching for transiting exoplanets. Dozens of new transiting exoplanet candidates will be also released in a subsequent paper(Zhang et al. 2018b).

astro-ph.IM↗

arXiv2018

Network localization is unalterable by infections in bursts

To shed light on the disease localization phenomenon, we study a bursty susceptible-infected-susceptible (SIS) model and analyze the model under the mean-field approximation. In the bursty SIS model, the infected nodes infect all their neighbors periodically, and the near-threshold steady-state prevalence is non-constant and maximized by a factor equal to the largest eigenvalue $λ_1$ of the adjacency matrix of the network. We show that the maximum near-threshold prevalence of the bursty SIS process on a localized network tends to zero even if $λ_1$ diverges in the thermodynamic limit, which indicates that the burst of infection cannot turn a localized spreading into a delocalized spreading. Our result is evaluated both on synthetic and real networks.

physics.soc-ph↗

arXiv2018

Observations on a series of merging magnetic flux ropes within an interplanetary coronal mass ejection

Coronal mass ejections (CMEs) are intense solar explosive eruptions. CMEs are highly important players in solar-terrestrial relationships, and they have important consequences for major geomagnetic storms and energetic particle events. It has been unclear how CMEs evolve when they propagate in the heliosphere. Here we report an interplanetary coronal mass ejection (ICME) consisting of multiple magnetic flux ropes measured by WIND on March 25-26, 1998. These magnetic flux ropes were merging with each other. The observations indicate that internal interactions (reconnections) within multi-flux-rope CME can coalesce into large-scale ropes, which may improve our understanding of the interplanetary evolution of CMEs. In addition, we speculated that the reported rope-rope interactions may also exist between successive rope-like CMEs and are important for the space weather forecasting.

physics.space-ph↗

arXiv2018

MV-RNN: A Multi-View Recurrent Neural Network for Sequential Recommendation

Sequential recommendation is a fundamental task for network applications, and it usually suffers from the item cold start problem due to the insufficiency of user feedbacks. There are currently three kinds of popular approaches which are respectively based on matrix factorization (MF) of collaborative filtering, Markov chain (MC), and recurrent neural network (RNN). Although widely used, they have some limitations. MF based methods could not capture dynamic user's interest. The strong Markov assumption greatly limits the performance of MC based methods. RNN based methods are still in the early stage of incorporating additional information. Based on these basic models, many methods with additional information only validate incorporating one modality in a separate way. In this work, to make the sequential recommendation and deal with the item cold start problem, we propose a Multi-View Recurrent Neural Network (MV-RNN}) model. Given the latent feature, MV-RNN can alleviate the item cold start problem by incorporating visual and textual information. First, At the input of MV-RNN, three different combinations of multi-view features are studied, like concatenation, fusion by addition and fusion by reconstructing the original multi-modal data. MV-RNN applies the recurrent structure to dynamically capture the user's interest. Second, we design a separate structure and a united structure on the hidden state of MV-RNN to explore a more effective way to handle multi-view features. Experiments on two real-world datasets show that MV-RNN can effectively generate the personalized ranking list, tackle the missing modalities problem and significantly alleviate the item cold start problem.

cs.IR↗

arXiv2018

Breaking the Curse of Horizon: Infinite-Horizon Off-Policy Estimation

We consider the off-policy estimation problem of estimating the expected reward of a target policy using samples collected by a different behavior policy. Importance sampling (IS) has been a key technique to derive (nearly) unbiased estimators, but is known to suffer from an excessively high variance in long-horizon problems. In the extreme case of in infinite-horizon problems, the variance of an IS-based estimator may even be unbounded. In this paper, we propose a new off-policy estimation method that applies IS directly on the stationary state-visitation distributions to avoid the exploding variance issue faced by existing estimators.Our key contribution is a novel approach to estimating the density ratio of two stationary distributions, with trajectories sampled from only the behavior distribution. We develop a mini-max loss function for the estimation problem, and derive a closed-form solution for the case of RKHS. We support our method with both theoretical and empirical analyses.

cs.LG↗

arXiv2018

Stein Variational Gradient Descent as Moment Matching

Stein variational gradient descent (SVGD) is a non-parametric inference algorithm that evolves a set of particles to fit a given distribution of interest. We analyze the non-asymptotic properties of SVGD, showing that there exists a set of functions, which we call the Stein matching set, whose expectations are exactly estimated by any set of particles that satisfies the fixed point equation of SVGD. This set is the image of Stein operator applied on the feature maps of the positive definite kernel used in SVGD. Our results provide a theoretical framework for analyzing the properties of SVGD with different kernels, shedding insight into optimal kernel choice. In particular, we show that SVGD with linear kernels yields exact estimation of means and variances on Gaussian distributions, while random Fourier features enable probabilistic bounds for distributional approximation. Our results offer a refreshing view of the classical inference problem as fitting Stein's identity or solving the Stein equation, which may motivate more efficient algorithms.

stat.ML↗

arXiv2018

Meteorological data from KLAWS-2G for an astronomical site survey of Dome A, Antarctica

We present an analysis of meteorological data from the second generation of the Kunlun Automated Weather Station (KLAWS-2G) at Dome A, Antarctica during 2015 and 2016. We find that a strong temperature inversion exists for all the elevations up to 14 m that KLAWS-2G can reach, and lasts for more than 10 hours for 50% or more of the time when temperature inversion occurs. The average wind speeds at 4 m elevation are 4.2 m/s and 3.8 m/s during 2015 and 2016, respectively. The strong temperature inversion and moderate wind speed lead to a shallow turbulent boundary layer height at Dome A. By analyzing the temperature and wind shear profiles, we note telescopes should be elevated by at least 8 m above the ice. We also find that the duration of temperature inversions, and the wind speed, vary considerably from year to year. Therefore, long-term and continuous data are still needed for the site survey at Dome A.

astro-ph.IM↗