arXiv subjects

Dawei Song

Publications and source records attributed to Dawei Song.

65 records · Page 4Linked to original sources

arXiv2018

Reconciling Irrational Human Behavior with AI based Decision Making: A Quantum Probabilistic Approach

There are many examples of human decision making which cannot be modeled by classical probabilistic and logic models, on which the current AI systems are based. Hence the need for a modeling framework which can enable intelligent systems to detect and predict cognitive biases in human decisions to facilitate better human-agent interaction. We give a few examples of irrational behavior and use a generalized probabilistic model inspired by the mathematical framework of Quantum Theory to model and explain such behavior.

cs.AI↗

arXiv2018

Quantum-inspired Complex Word Embedding

A challenging task for word embeddings is to capture the emergent meaning or polarity of a combination of individual words. For example, existing approaches in word embeddings will assign high probabilities to the words "Penguin" and "Fly" if they frequently co-occur, but it fails to capture the fact that they occur in an opposite sense - Penguins do not fly. We hypothesize that humans do not associate a single polarity or sentiment to each word. The word contributes to the overall polarity of a combination of words depending upon which other words it is combined with. This is analogous to the behavior of microscopic particles which exist in all possible states at the same time and interfere with each other to give rise to new states depending upon their relative phases. We make use of the Hilbert Space representation of such particles in Quantum Mechanics where we subscribe a relative phase to each word, which is a complex number, and investigate two such quantum inspired models to derive the meaning of a combination of words. The proposed models achieve better performances than state-of-the-art non-quantum models on the binary sentence classification task.

cs.CL↗

arXiv2018

Modeling Multidimensional User Relevance in IR using Vector Spaces

It has been shown that relevance judgment of documents is influenced by multiple factors beyond topicality. Some multidimensional user relevance models (MURM) proposed in literature have investigated the impact of different dimensions of relevance on user judgment. Our hypothesis is that a user might give more importance to certain relevance dimensions in a session which might change dynamically as the session progresses. This motivates the need to capture the weights of different relevance dimensions using feedback and build a model to rank documents for subsequent queries according to these weights. We propose a geometric model inspired by the mathematical framework of Quantum theory to capture the user's importance given to each dimension of relevance and test our hypothesis on data from a web search engine and TREC Session track.

cs.IR↗

arXiv2018

A numerical mode matching method for wave scattering in a layered medium with a stratified inhomogeneity

Numerical mode matching (NMM) methods are widely used for analyzing wave propagation and scattering in structures that are piece-wise uniform along one spatial direction. For open structures that are unbounded in transverse directions (perpendicular to the uniform direction), the NMM methods use the perfectly matched layer (PML) technique to truncate the transverse variables. When incident waves are specified in homogeneous media surrounding the main structure, the total field is not always outgoing, and the NMM methods rely on reference solutions for each uniform segment. Existing NMM methods have difficulty handing gracing incident waves and special incident waves related to the onset of total internal reflection, and are not very efficient at computing reference solutions for non-plane incident waves. In this paper, a new NMM method is developed to overcome these limitations. A Robin-type boundary condition is proposed to ensure that non-propagating and non-decaying wave field components are not reflected by truncated PMLs. Exponential convergence of the PML solutions based on the hybrid Dirichlet-Robin boundary condition is established theoretically. A fast method is developed for computing reference solutions for cylindrical incident waves. The new NMM is implemented for two-dimensional structures and polarized electromagnetic waves. Numerical experiments are carried out to validate the new NMM method and to demonstrate its performance.

math.NA↗

arXiv2017

Bipartite entanglement in AJL's algorithm for three-strand braids

Aharonov, Jones, and Landau [Algorithmica 55, 395 (2009)] have presented a polynomial quantum algorithm for approximating the Jones polynomial. We investigate the bipartite entanglement properties in AJL's algorithm for three-strand braids. We re-describe AJL's algorithm as an equivalent algorithm which involves three work qubits in some mixed state coupled to a single control qubit. Furthermore, we use the Peres entanglement criterion to study the entanglement features of the state before measurements present in the re-described algorithm for all bipartitions. We show that the state is a product state relative to the bipartition between the first work qubit and the others. And it has no entanglement between the control qubit and work ones. We also prove a sufficient and necessary condition for its entanglement between the second (third) work qubit and the others. Moreover, we discuss the relation between its bipartite entanglement and elementary crossings in the three-strand braid group. We find that braids whose trace closures are topologically identical may have different entanglement properties in AJL's algorithm.

quant-ph↗

arXiv2017

Personalised Query Suggestion for Intranet Search with Temporal User Profiling

Recent research has shown the usefulness of using collective user interaction data (e.g., query logs) to recommend query modification suggestions for Intranet search. However, most of the query suggestion approaches for Intranet search follow an "one size fits all" strategy, whereby different users who submit an identical query would get the same query suggestion list. This is problematic, as even with the same query, different users may have different topics of interest, which may change over time in response to the user's interaction with the system. We address the problem by proposing a personalised query suggestion framework for Intranet search. For each search session, we construct two temporal user profiles: a click user profile using the user's clicked documents and a query user profile using the user's submitted queries. We then use the two profiles to re-rank the non-personalised query suggestion list returned by a state-of-the-art query suggestion method for Intranet search. Experimental results on a large-scale query logs collection show that our personalised framework significantly improves the quality of suggested queries.

cs.IR↗

arXiv2016

Search Personalization with Embeddings

Recent research has shown that the performance of search personalization depends on the richness of user profiles which normally represent the user's topical interests. In this paper, we propose a new embedding approach to learning user profiles, where users are embedded on a topical interest space. We then directly utilize the user profiles for search personalization. Experiments on query logs from a major commercial web search engine demonstrate that our embedding approach improves the performance of the search engine and also achieves better search performance than other strong baselines.

cs.IR↗

arXiv2015

Further Theoretical Study of Distribution Separation Method for Information Retrieval

Recently, a Distribution Separation Method (DSM) is proposed for relevant feedback in information retrieval, which aims to approximate the true relevance distribution by separating a seed irrelevance distribution from the mixture one. While DSM achieved a promising empirical performance, theoretical analysis of DSM is still need further study and comparison with other relative retrieval model. In this article, we first generalize DSM's theoretical property, by proving that its minimum correlation assumption is equivalent to the maximum (original and symmetrized) KL-Divergence assumption. Second, we also analytically show that the EM algorithm in a well-known Mixture Model is essentially a distribution separation process and can be simplified using the linear separation algorithm in DSM. Some empirical results are also presented to support our theoretical analysis.

cs.IR↗

arXiv2015

A Confident Information First Principle for Parametric Reduction and Model Selection of Boltzmann Machines

Typical dimensionality reduction (DR) methods are often data-oriented, focusing on directly reducing the number of random variables (features) while retaining the maximal variations in the high-dimensional data. In unsupervised situations, one of the main limitations of these methods lies in their dependency on the scale of data features. This paper aims to address the problem from a new perspective and considers model-oriented dimensionality reduction in parameter spaces of binary multivariate distributions. Specifically, we propose a general parameter reduction criterion, called Confident-Information-First (CIF) principle, to maximally preserve confident parameters and rule out less confident parameters. Formally, the confidence of each parameter can be assessed by its contribution to the expected Fisher information distance within the geometric manifold over the neighbourhood of the underlying real distribution. We then revisit Boltzmann machines (BM) from a model selection perspective and theoretically show that both the fully visible BM (VBM) and the BM with hidden units can be derived from the general binary multivariate distribution using the CIF principle. This can help us uncover and formalize the essential parts of the target density that BM aims to capture and the non-essential parts that BM should discard. Guided by the theoretical analysis, we develop a sample-specific CIF for model selection of BM that is adaptive to the observed samples. The method is studied in a series of density estimation experiments and has been shown effective in terms of the estimate accuracy.

cs.LG↗

arXiv2013

Understanding Boltzmann Machine and Deep Learning via A Confident Information First Principle

Typical dimensionality reduction methods focus on directly reducing the number of random variables while retaining maximal variations in the data. In this paper, we consider the dimensionality reduction in parameter spaces of binary multivariate distributions. We propose a general Confident-Information-First (CIF) principle to maximally preserve parameters with confident estimates and rule out unreliable or noisy parameters. Formally, the confidence of a parameter can be assessed by its Fisher information, which establishes a connection with the inverse variance of any unbiased estimate for the parameter via the Cramér-Rao bound. We then revisit Boltzmann machines (BM) and theoretically show that both single-layer BM without hidden units (SBM) and restricted BM (RBM) can be solidly derived using the CIF principle. This can not only help us uncover and formalize the essential parts of the target density that SBM and RBM capture, but also suggest that the deep neural network consisting of several layers of RBM can be seen as the layer-wise application of CIF. Guided by the theoretical analysis, we develop a sample-specific CIF-based contrastive divergence (CD-CIF) algorithm for SBM and a CIF-based iterative projection procedure (IP) for RBM. Both CD-CIF and IP are studied in a series of density estimation experiments.

cs.NE↗

arXiv2010

On Tsallis Entropy Bias and Generalized Maximum Entropy Models

In density estimation task, maximum entropy model (Maxent) can effectively use reliable prior information via certain constraints, i.e., linear constraints without empirical parameters. However, reliable prior information is often insufficient, and the selection of uncertain constraints becomes necessary but poses considerable implementation complexity. Improper setting of uncertain constraints can result in overfitting or underfitting. To solve this problem, a generalization of Maxent, under Tsallis entropy framework, is proposed. The proposed method introduces a convex quadratic constraint for the correction of (expected) Tsallis entropy bias (TEB). Specifically, we demonstrate that the expected Tsallis entropy of sampling distributions is smaller than the Tsallis entropy of the underlying real distribution. This expected entropy reduction is exactly the (expected) TEB, which can be expressed by a closed-form formula and act as a consistent and unbiased correction. TEB indicates that the entropy of a specific sampling distribution should be increased accordingly. This entails a quantitative re-interpretation of the Maxent principle. By compensating TEB and meanwhile forcing the resulting distribution to be close to the sampling distribution, our generalized TEBC Maxent can be expected to alleviate the overfitting and underfitting. We also present a connection between TEB and Lidstone estimator. As a result, TEB-Lidstone estimator is developed by analytically identifying the rate of probability correction in Lidstone. Extensive empirical evaluation shows promising performance of both TEBC Maxent and TEB-Lidstone in comparison with various state-of-the-art density estimation methods.

cs.LG↗