arXiv subjects

Zheng Li

Publications and source records attributed to Zheng Li.

At least 397 records · Page 22Linked to original sources

arXiv2020

Multimodal Topic Learning for Video Recommendation

Facilitated by deep neural networks, video recommendation systems have made significant advances. Existing video recommendation systems directly exploit features from different modalities (e.g., user personal data, user behavior data, video titles, video tags, and visual contents) to input deep neural networks, while expecting the networks to online mine user-preferred topics implicitly from these features. However, the features lacking semantic topic information limits accurate recommendation generation. In addition, feature crosses using visual content features generate high dimensionality features that heavily downgrade the online computational efficiency of networks. In this paper, we explicitly separate topic generation from recommendation generation, propose a multimodal topic learning algorithm to exploit three modalities (i.e., tags, titles, and cover images) for generating video topics offline. The topics generated by the proposed algorithm serve as semantic topic features to facilitate preference scope determination and recommendation generation. Furthermore, we use the semantic topic features instead of visual content features to effectively reduce online computational cost. Our proposed algorithm has been deployed in the Kuaibao information streaming platform. Online and offline evaluation results show that our proposed algorithm performs favorably.

cs.IR↗

arXiv2020

AP20-OLR Challenge: Three Tasks and Their Baselines

This paper introduces the fifth oriental language recognition (OLR) challenge AP20-OLR, which intends to improve the performance of language recognition systems, along with APSIPA Annual Summit and Conference (APSIPA ASC). The data profile, three tasks, the corresponding baselines, and the evaluation principles are introduced in this paper. The AP20-OLR challenge includes more languages, dialects and real-life data provided by Speechocean and the NSFC M2ASR project, and all the data is free for participants. The challenge this year still focuses on practical and challenging problems, with three tasks: (1) cross-channel LID, (2) dialect identification and (3) noisy LID. Based on Kaldi and Pytorch, recipes for i-vector and x-vector systems are also conducted as baselines for the three tasks. These recipes will be online-published, and available for participants to configure LID systems. The baseline results on the three tasks demonstrate that those tasks in this challenge are worth paying more efforts to achieve better performance.

eess.AS↗

arXiv2020

SYNC: A Copula based Framework for Generating Synthetic Data from Aggregated Sources

A synthetic dataset is a data object that is generated programmatically, and it may be valuable to creating a single dataset from multiple sources when direct collection is difficult or costly. Although it is a fundamental step for many data science tasks, an efficient and standard framework is absent. In this paper, we study a specific synthetic data generation task called downscaling, a procedure to infer high-resolution, harder-to-collect information (e.g., individual level records) from many low-resolution, easy-to-collect sources, and propose a multi-stage framework called SYNC (Synthetic Data Generation via Gaussian Copula). For given low-resolution datasets, the central idea of SYNC is to fit Gaussian copula models to each of the low-resolution datasets in order to correctly capture dependencies and marginal distributions, and then sample from the fitted models to obtain the desired high-resolution subsets. Predictive models are then used to merge sampled subsets into one, and finally, sampled datasets are scaled according to low-resolution marginal constraints. We make four key contributions in this work: 1) propose a novel framework for generating individual level data from aggregated data sources by combining state-of-the-art machine learning and statistical techniques, 2) perform simulation studies to validate SYNC's performance as a synthetic data generation algorithm, 3) demonstrate its value as a feature engineering tool, as well as an alternative to data collection in situations where gathering is difficult through two real-world datasets, 4) release an easy-to-use framework implementation for reproducibility and scalability at the production level that easily incorporates new data.

stat.AP↗

arXiv2020

COPOD: Copula-Based Outlier Detection

Outlier detection refers to the identification of rare items that are deviant from the general data distribution. Existing approaches suffer from high computational complexity, low predictive capability, and limited interpretability. As a remedy, we present a novel outlier detection algorithm called COPOD, which is inspired by copulas for modeling multivariate data distribution. COPOD first constructs an empirical copula, and then uses it to predict tail probabilities of each given data point to determine its level of "extremeness". Intuitively, we think of this as calculating an anomalous p-value. This makes COPOD both parameter-free, highly interpretable, and computationally efficient. In this work, we make three key contributions, 1) propose a novel, parameter-free outlier detection algorithm with both great performance and interpretability, 2) perform extensive experiments on 30 benchmark datasets to show that COPOD outperforms in most cases and is also one of the fastest algorithms, and 3) release an easy-to-use Python implementation for reproducibility.

stat.ML↗

arXiv2020

Research on multi-dimensional end-to-end phrase recognition algorithm based on background knowledge

At present, the deep end-to-end method based on supervised learning is used in entity recognition and dependency analysis. There are two problems in this method: firstly, background knowledge cannot be introduced; secondly, multi granularity and nested features of natural language cannot be recognized. In order to solve these problems, the annotation rules based on phrase window are proposed, and the corresponding multi-dimensional end-to-end phrase recognition algorithm is designed. This annotation rule divides sentences into seven types of nested phrases, and indicates the dependency between phrases. The algorithm can not only introduce background knowledge, recognize all kinds of nested phrases in sentences, but also recognize the dependency between phrases. The experimental results show that the annotation rule is easy to use and has no ambiguity; the matching algorithm is more consistent with the multi granularity and diversity characteristics of syntax than the traditional end-to-end algorithm. The experiment on CPWD dataset, by introducing background knowledge, the new algorithm improves the accuracy of the end-to-end method by more than one point. The corresponding method was applied to the CCL 2018 competition and won the first place in the task of Chinese humor type recognition.

cs.CL↗

arXiv2020

Research on Annotation Rules and Recognition Algorithm Based on Phrase Window

At present, most Natural Language Processing technology is based on the results of Word Segmentation for Dependency Parsing, which mainly uses an end-to-end method based on supervised learning. There are two main problems with this method: firstly, the la-beling rules are complex and the data is too difficult to label, the workload of which is large; secondly, the algorithm cannot recognize the multi-granularity and diversity of language components. In order to solve these two problems, we propose labeling rules based on phrase windows, and designed corresponding phrase recognition algorithms. The labeling rule uses phrases as the minimum unit, di-vides sentences into 7 types of nestable phrase types, and marks the grammatical dependencies between phrases. The corresponding algorithm, drawing on the idea of identifying the target area in the image field, can find the start and end positions of various phrases in the sentence, and realize the synchronous recognition of nested phrases and grammatical dependencies. The results of the experiment shows that the labeling rule is convenient and easy to use, and there is no ambiguity; the algorithm is more grammatically multi-granular and diverse than the end-to-end algorithm. Experiments on the CPWD dataset improve the accuracy of the end-to-end method by about 1 point. The corresponding method was applied to the CCL2018 competition, and the first place in the Chinese Metaphor Sentiment Analysis Task.

cs.CL↗

arXiv2020

Exploiting Visual Semantic Reasoning for Video-Text Retrieval

Video retrieval is a challenging research topic bridging the vision and language areas and has attracted broad attention in recent years. Previous works have been devoted to representing videos by directly encoding from frame-level features. In fact, videos consist of various and abundant semantic relations to which existing methods pay less attention. To address this issue, we propose a Visual Semantic Enhanced Reasoning Network (ViSERN) to exploit reasoning between frame regions. Specifically, we consider frame regions as vertices and construct a fully-connected semantic correlation graph. Then, we perform reasoning by novel random walk rule-based graph convolutional networks to generate region features involved with semantic relations. With the benefit of reasoning, semantic interactions between regions are considered, while the impact of redundancy is suppressed. Finally, the region features are aggregated to form frame-level features for further encoding to measure video-text similarity. Extensive experiments on two public benchmark datasets validate the effectiveness of our method by achieving state-of-the-art performance due to the powerful semantic reasoning.

cs.CV↗

arXiv2020

Ramsey interferometry through coherent $X^2Σ_g^+ - A^2Π_u - B^2Σ_u^+$ coupling and population transfer in N$^+_2$ air laser

The laser-like coherent emission at 391nm from N$_2$ gas irradiated by strong 800nm pump laser and weak 400nm seed laser is theoretically investigated. Recent experimental observations are well simulated, including temporal profile, optical gain and periodic modulation of the 391nm signal from N$_2^+$. Our calculation sheds light on the long standing controversy on whether population inversion is indispensable for the optical gain. We demonstrate the Ramsey interference fringes of the emission intensity at 391nm formed by additionally injecting another 800nm pump or 400nm seed, which are well explained by the coherent modulation of transition dipole moment and population between the $A^2Π_u(ν=2)$-$X^2Σ_g^+$ states as well as the $B^2Σ_u^+ (ν=0)$-$X^2Σ_g^+$ states. This study provides versatile possibilities for the coherent control of $\text{N}_2^+$ air laser.

physics.atom-ph↗

arXiv2020

Efficient asymmetric transmission of elastic waves in thin plates with lossless metasurfaces

Requiring neither active components nor complex designs, we propose and experimentally demonstrate a generic framework for undistorted asymmetric elastic-wave transmission in a thin plate just using a layer of lossless metasurface. The asymmetric transmission stems from the uneven diffraction of +1 and -1 orders on opposite sides of the metasurface, respectively. Compared with previous loss-induced strategies, the present metasurface maintains a nearly total transmission for the transportation side, but a total reflection from the opposite side, exhibiting a higher contrast ratio of transmission. Moreover, we illustrate that this strong asymmetric behavior is robust to the frequency, the incident angle and the loss effect. The present work paves new avenues to compact rectification, high resolution ultrasonography, vibration and noise control in elastodynamics and acoustics.

physics.app-ph↗

arXiv2020

Photoinduced Vibrations Drive Ultrafast Structural Distortion in Lead Halide Perovskite

Organic-inorganic perovskites have shown great promise towards their application in optoelectronics. The success of this class of material is dictated by the complex interplay between various underlying microscopic phenomena. The structural dynamics of organic cations and the inorganic sublattice after photoexcitation is hypothesized to have a direct effect on the material properties, thereby affecting the overall device performance. Here, we use two-dimensional (2D) electronic spectroscopy to reveal impulsively excited vibrational modes of methylammonium (MA) lead iodide perovskite, which drive the structural distortion after photoexcitation. The vibrational analysis of the measured data allows us to directly monitor the time evolution of the librational motion of the MA cation along with the vibrational coherences of inorganic sublattice. Wavelet analysis of the observed vibrational coherences uncovers the interplay between these two types of phonons. It reveals the coherent generation of the librational motion of the MA cation within ~300 fs, which is complemented by the coherent evolution of the skeletal motion of the inorganic sublattice. We have employed time-dependent density functional theory (TDDFT) to study the atomic motion of the MA cation and the inorganic sublattice during the process of photoexcitation. The TDDFT calculations support our experimental observations of the coherent generation of librational motions in the MA cation and highlight the importance of the anharmonic interaction between the MA cation and the inorganic sublattice. Our calculations predict the transfer of the photoinduced vibrational coherence from the MA cation to the inorganic sublattice, which drives the skeleton motion to form a polaronic state leading to long lifetimes of the charge carriers. This work may lead to novel design principles for next generation of solar cell materials.

physics.chem-ph↗

arXiv2020

Rate Splitting for Multi-Antenna Downlink: Precoder Design and Practical Implementation

Rate splitting (RS) is a potentially powerful and flexible technique for multi-antenna downlink transmission. In this paper, we address several technical challenges towards its practical implementation for beyond 5G systems. To this end, we focus on a single-cell system with a multi-antenna base station (BS) and K single-antenna receivers. We consider RS in its most general form, and joint decoding to fully exploit the potential of RS. First, we investigate the achievable rates under joint decoding and formulate the precoder design problems to maximize a general utility function, or to minimize the transmit power under pre-defined rate targets. Building upon the concave-convex procedure (CCCP), we propose precoder design algorithms for an arbitrary number of users. Our proposed algorithms approximate the intractable non-convex problems with a number of successively refined convex problems, and provably converge to stationary points of the original problems. Then, to reduce the decoding complexity, we consider the optimization of the precoder and the decoding order under successive decoding. Further, we propose a stream selection algorithm to reduce the number of precoded signals. With a reduced number of streams and successive decoding at the receivers, our proposed algorithm can even be implemented when the number of users is relatively large, whereas the complexity was previously considered as prohibitively high in the same setting. Finally, we propose a simple adaptation of our algorithms to account for the imperfection of the channel state information at the transmitter. Numerical results demonstrate that the general RS scheme provides a substantial performance gain as compared to state-of-the-art linear precoding schemes, especially with a moderately large number of users.

cs.IT↗

arXiv2020

A Dynamic Subspace Based BFGS Method for Large Scale Optimization Problem

Large-scale unconstrained optimization is a fundamental and important class of, yet not well-solved problems in numerical optimization. The main challenge in designing an algorithm is to require a few storage locations or very inexpensive computations while preserving global convergence. In this work, we propose a novel approach solving large-scale unconstrained optimization problem by combining the dynamic subspace technique and the BFGS update algorithm. It is clearly demonstrated that our approach has the same rate of convergence in the dynamic subspace as the BFGS and less memory than L-BFGS. Further, we give the convergence analysis by constructing the mapping of low-dimensional Euclidean space to the adaptive subspace. We compare our hybrid algorithm with the BFGS and L-BFGS approaches. Experimental results show that our hybrid algorithm offers several significant advantages such as parallel computing, convergence efficiency, and robustness.

math.OC↗

arXiv2019

Non-uniform Berry-Esseen Bound by Unbounded Exchangeable Pair Approach

In this paper, a new technique is introduced to obtain non-uniform Berry-Esseen bounds of normal and nonnormal approximation for unbounded exchangeable pairs. This technique does not rely on the concentration inequalities developed by Chen and Shao \cite{cls1, cls2} and can be applied to the quadratic forms, general Curie-Weiss model and an independence test. In particular, our non-uniform result about the independence test is under 6th moment condition, while the uniform bound in Chen and Shao \cite{cs2} requires 24th moment condition.

math.ST↗

arXiv2019

Multi-Agent Deep Reinforcement Learning based Spectrum Allocation for D2D Underlay Communications

Device-to-device (D2D) communication underlay cellular networks is a promising technique to improve spectrum efficiency. In this situation, D2D transmission may cause severe interference to both the cellular and other D2D links, which imposes a great technical challenge to spectrum allocation. Existing centralized schemes require global information, which causes a large signaling overhead. While existing distributed schemes requires frequent information exchange among D2D users and cannot achieve global optimization. In this paper, a distributed spectrum allocation framework based on multi-agent deep reinforcement learning is proposed, named multi-agent actor critic (MAAC). MAAC shares global historical states, actions and policies during centralized training, requires no signal interaction during execution and utilizes cooperation among users to further optimize system performance. Moreover, in order to decrease the computing complexity of the training, we further propose the neighbor-agent actor critic (NAAC) based on the neighbor users' historical information for centralized training. The simulation results show that the proposed MAAC and NAAC can effectively reduce the outage probability of cellular links, greatly improve the sum rate of D2D links and converge quickly.

cs.NI↗

arXiv2019

A Multi-Agent Deep Reinforcement Learning based Spectrum Allocation Framework for D2D Communications

Device-to-device (D2D) communication has been recognized as a promising technique to improve spectrum efficiency. However, D2D transmission as an underlay causes severe interference, which imposes a technical challenge to spectrum allocation. Existing centralized schemes require global information, which can cause serious signaling overhead. While existing distributed solution requires frequent information exchange between users and cannot achieve global optimization. In this paper, a distributed spectrum allocation framework based on multi-agent deep reinforcement learning is proposed, named Neighbor-Agent Actor Critic (NAAC). NAAC uses neighbor users' historical information for centralized training but is executed distributedly without that information, which not only has no signal interaction during execution, but also utilizes cooperation between users to further optimize system performance. The simulation results show that the proposed framework can effectively reduce the outage probability of cellular links, improve the sum rate of D2D links and have good convergence.

cs.NI↗

arXiv2019

Efficient Network Construction through Structural Plasticity

Deep Neural Networks (DNNs) on hardware is facing excessive computation cost due to the massive number of parameters. A typical training pipeline to mitigate over-parameterization is to pre-define a DNN structure first with redundant learning units (filters and neurons) under the goal of high accuracy, then to prune redundant learning units after training with the purpose of efficient inference. We argue that it is sub-optimal to introduce redundancy into training for the purpose of reducing redundancy later in inference. Moreover, the fixed network structure further results in poor adaption to dynamic tasks, such as lifelong learning. In contrast, structural plasticity plays an indispensable role in mammalian brains to achieve compact and accurate learning. Throughout the lifetime, active connections are continuously created while those no longer important are degenerated. Inspired by such observation, we propose a training scheme, namely Continuous Growth and Pruning (CGaP), where we start the training from a small network seed, then literally execute continuous growth by adding important learning units and finally prune secondary ones for efficient inference. The inference model generated from CGaP is sparse in the structure, largely decreasing the inference power and latency when deployed on hardware platforms. With popular DNN structures on representative datasets, the efficacy of CGaP is benchmarked by both algorithm simulation and architectural modeling on Field-programmable Gate Arrays (FPGA). For example, CGaP decreases the FLOPs, model size, DRAM access energy and inference latency by 63.3%, 64.0%, 11.8% and 40.2%, respectively, for ResNet-110 on CIFAR-10.

cs.NE↗

arXiv2019

Spectroscopic Signature of Chemical Bond Dissociation Revealed by Calculated Core-Electron Spectra

The advent of ultrashort soft x-ray pulse sources permits the use of established gas phase spectroscopy methods to investigate ultrafast photochemistry in isolated molecules with element and site specificity. In the present study, we simulate excited state wavepacket dynamics of a prototypical process, the ultrafast photodissociation of methyl iodide. Based on the simulation, we calculate time-dependent excited state carbon edge photoelectron and Auger electron spectra. We observe distinct signatures in both types of spectra and show their direct connection to C-I bond dissociation and charge rearrangement processes in the molecule. We demonstrate at the CH$_3$I molecule that the observed signatures allow us to map the time-dependent dynamics of ultrafast photo-induced bond breaking with unprecedented details.

physics.chem-ph↗

arXiv2019

SynC: A Unified Framework for Generating Synthetic Population with Gaussian Copula

Synthetic population generation is the process of combining multiple socioeconomic and demographic datasets from different sources and/or granularity levels, and downscaling them to an individual level. Although it is a fundamental step for many data science tasks, an efficient and standard framework is absent. In this study, we propose a multi-stage framework called SynC (Synthetic Population via Gaussian Copula) to fill the gap. SynC first removes potential outliers in the data and then fits the filtered data with a Gaussian copula model to correctly capture dependencies and marginal distributions of sampled survey data. Finally, SynC leverages predictive models to merge datasets into one and then scales them accordingly to match the marginal constraints. We make three key contributions in this work: 1) propose a novel framework for generating individual level data from aggregated data sources by combining state-of-the-art machine learning and statistical techniques, 2) demonstrate its value as a feature engineering tool, as well as an alternative to data collection in situations where gathering is difficult through two real-world datasets, 3) release an easy-to-use framework implementation for reproducibility, and 4) ensure the methodology is scalable at the production level and can easily incorporate new data.

cs.LG↗