Searcharxiv⌕ Search

arXiv subjects

Han Zhao

Publications and source records attributed to Han Zhao.

232 records · Page 13Linked to original sources
arXiv2019

Learning Markov Chain in Unordered Dataset

The assumption that data samples are independently identically distributed is the backbone of many learning algorithms. Nevertheless, datasets often exhibit rich structure in practice, and we argue that there exist some unknown order within the data instances. In this technical report, we introduce OrderNet that can be used to extract the order of data instances in an unsupervised way. By assuming that the instances are sampled from a Markov chain, our goal is to learn the transitional operator of the underlying Markov chain, as well as the order by maximizing the generation probability under all possible data permutations. Specifically, we use neural network as a compact and soft lookup table to approximate the possibly huge, but discrete transition matrix. This strategy allows us to amortize the space complexity with a single model. Furthermore, this simple and compact representation also provides a short description to the dataset and generalizes to unseen instances as well. To ensure that the learned Markov chain is ergodic, we propose a greedy batch-wise permutation scheme that allows fast training. Empirically, we show that OrderNet is able to discover an order among data instances. We also extend the proposed OrderNet to one-shot recognition task and demonstrate favorable results.

cs.LG↗
arXiv2019

Supersymmetric microring laser arrays

Coherent combination of emission power from an array of coupled semiconductor lasers operating on the same chip is of fundamental and technological importance. In general, the nonlinear competition among the array supermodes can entail incoherence and spectral broadening, leading to spatiotemporally unstable and multimode emission pattern and thus poor beam quality. Here, by harnessing notions from supersymmetric (SUSY) quantum mechanics, we report that the strategic coupling between a class III-V semiconductor microring laser array with its dissipative superpartner can be used to limit the number of supermodes available for laser actions to one. We introduce a novel approach based on second-order SUSY transformation in order to drastically simplify the superpartner array engineering. Compared to a conventional laser array, which has multimode spectrum, a SUSY laser array is observed to be capable of operating in a single (transverse) supermode. Enhancement of the peak output intensity of the SUSY laser array has been demonstrated with high-efficiency and lower lasing threshold, compared with a single laser and a conventional laser array. Our experimental findings pave the way towards broad-area and high-power light generation in a scalable and stable fashion.

physics.optics↗
arXiv2018

Frank-Wolfe Optimization for Symmetric-NMF under Simplicial Constraint

Symmetric nonnegative matrix factorization has found abundant applications in various domains by providing a symmetric low-rank decomposition of nonnegative matrices. In this paper we propose a Frank-Wolfe (FW) solver to optimize the symmetric nonnegative matrix factorization problem under a simplicial constraint, which has recently been proposed for probabilistic clustering. Compared with existing solutions, this algorithm is simple to implement, and has no hyperparameters to be tuned. Building on the recent advances of FW algorithms in nonconvex optimization, we prove an $O(1/\varepsilon^2)$ convergence rate to $\varepsilon$-approximate KKT points, via a tight bound $Θ(n^2)$ on the curvature constant, which matches the best known result in unconstrained nonconvex setting using gradient methods. Numerical results demonstrate the effectiveness of our algorithm. As a side contribution, we construct a simple nonsmooth convex problem where the FW algorithm fails to converge to the optimum. This result raises an interesting question about necessary conditions of the success of the FW algorithm on convex problems.

cs.LG↗
arXiv2018

Convolutional-Recurrent Neural Networks for Speech Enhancement

We propose an end-to-end model based on convolutional and recurrent neural networks for speech enhancement. Our model is purely data-driven and does not make any assumptions about the type or the stationarity of the noise. In contrast to existing methods that use multilayer perceptrons (MLPs), we employ both convolutional and recurrent neural network architectures. Thus, our approach allows us to exploit local structures in both the frequency and temporal domains. By incorporating prior knowledge of speech signals into the design of model structures, we build a model that is more data-efficient and achieves better generalization on both seen and unseen noise. Based on experiments with synthetic data, we demonstrate that our model outperforms existing methods, improving PESQ by up to 0.6 on seen noise and 0.64 on unseen noise.

cs.SD↗
arXiv2018

Experimental demonstration of three-dimensional broadband underwater acoustic carpet cloak

We present the design, architecture and detailed performance of a three-dimensional (3D) underwater acoustic carpet cloak (UACC). The proposed system of the 3D UACC is an octahedral pyramid which is composed of periodical steel strips. This underwater acoustic device, placed over the target to hide, is able to manipulate the scattered wavefront to mimic a reflecting plane. The effectiveness of the prototype is experimentally demonstrated in an anechoic tank. The measured acoustic pressure distributions show that the 3D UACC can work in all directions in a wide frequency range. This experimental verification of 3D device paves the way for guidelines on future practical applications.

physics.app-ph↗
arXiv2018

Active Learning of Strict Partial Orders: A Case Study on Concept Prerequisite Relations

Strict partial order is a mathematical structure commonly seen in relational data. One obstacle to extracting such type of relations at scale is the lack of large-scale labels for building effective data-driven solutions. We develop an active learning framework for mining such relations subject to a strict order. Our approach incorporates relational reasoning not only in finding new unlabeled pairs whose labels can be deduced from an existing label set, but also in devising new query strategies that consider the relational structure of labels. Our experiments on concept prerequisite relations show our proposed framework can substantially improve the classification performance with the same query budget compared to other baseline approaches.

cs.LG↗
arXiv2017

Linear Time Computation of Moments in Sum-Product Networks

Bayesian online algorithms for Sum-Product Networks (SPNs) need to update their posterior distribution after seeing one single additional instance. To do so, they must compute moments of the model parameters under this distribution. The best existing method for computing such moments scales quadratically in the size of the SPN, although it scales linearly for trees. This unfortunate scaling makes Bayesian online algorithms prohibitively expensive, except for small or tree-structured SPNs. We propose an optimal linear-time algorithm that works even when the SPN is a general directed acyclic graph (DAG), which significantly broadens the applicability of Bayesian online algorithms for SPNs. There are three key ingredients in the design and analysis of our algorithm: 1). For each edge in the graph, we construct a linear time reduction from the moment computation problem to a joint inference problem in SPNs. 2). Using the property that each SPN computes a multilinear polynomial, we give an efficient procedure for polynomial evaluation by differentiation without expanding the network that may contain exponentially many monomials. 3). We propose a dynamic programming method to further reduce the computation of the moments of all the edges in the graph from quadratic to linear. We demonstrate the usefulness of our linear time algorithm by applying it to develop a linear time assume density filter (ADF) for SPNs.

cs.LG↗
arXiv2017

Principled Hybrids of Generative and Discriminative Domain Adaptation

We propose a probabilistic framework for domain adaptation that blends both generative and discriminative modeling in a principled way. Under this framework, generative and discriminative models correspond to specific choices of the prior over parameters. This provides us a very general way to interpolate between generative and discriminative extremes through different choices of priors. By maximizing both the marginal and the conditional log-likelihoods, models derived from this framework can use both labeled instances from the source domain as well as unlabeled instances from both source and target domains. Under this framework, we show that the popular reconstruction loss of autoencoder corresponds to an upper bound of the negative marginal log-likelihoods of unlabeled instances, where marginal distributions are given by proper kernel density estimations. This provides a way to interpret the empirical success of autoencoders in domain adaptation and semi-supervised learning. We instantiate our framework using neural networks, and build a concrete model, DAuto. Empirically, we demonstrate the effectiveness of DAuto on text, image and speech datasets, showing that it outperforms related competitors when domain adaptation is possible.

cs.LG↗
arXiv2017

Multiple Source Domain Adaptation with Adversarial Training of Neural Networks

While domain adaptation has been actively researched in recent years, most theoretical results and algorithms focus on the single-source-single-target adaptation setting. Naive application of such algorithms on multiple source domain adaptation problem may lead to suboptimal solutions. As a step toward bridging the gap, we propose a new generalization bound for domain adaptation when there are multiple source domains with labeled instances and one target domain with unlabeled instances. Compared with existing bounds, the new bound does not require expert knowledge about the target distribution, nor the optimal combination rule for multisource domains. Interestingly, our theory also leads to an efficient learning strategy using adversarial neural networks: we show how to interpret it as learning feature representations that are invariant to the multiple domain shifts while still being discriminative for the learning task. To this end, we propose two models, both of which we call multisource domain adversarial networks (MDANs): the first model optimizes directly our bound, while the second model is a smoothed approximation of the first one, leading to a more data-efficient and task-adaptive model. The optimization tasks of both models are minimax saddle point problems that can be optimized by adversarial training. To demonstrate the effectiveness of MDANs, we conduct extensive experiments showing superior adaptation performance on three real-world datasets: sentiment analysis, digit classification, and vehicle counting.

cs.LG↗
arXiv2017

Exceptional Point Engineered Glass Slide for Microscopic Thermal Mapping

Thermal sensing with fine spatial resolution is important to the study of many scientific areas. While modern microscopy systems allow easy optical detection at high spatial resolution, their intrinsic functions are mainly focused on imaging but limited in detecting other physical parameters, for example, mapping thermal variations. Here, with a coating of an optical exceptional point structure on a conventional glass slide, we demonstrate a novel low-cost and efficient multifunctional microscope slide, supporting real-time monitoring and mapping of temperature distribution and heat transport in addition to conventional microscopic imaging. The square-root dependency associated with an exceptional point leads to enhanced thermal sensitivity that is critical for the precise measurement of the temperature. With a microscale resolution, real-time thermal mapping is conducted, showing dynamic temperature variation in a spatially-defined area. Our strategy of integrating low-cost and efficient optical sensing technologies on a conventional glass slide is expected to enable simultaneous detection of multiple environmental parameters, thereby leading to new opportunities for improved experimental control in many scientific research disciplines.

physics.optics↗
arXiv2017

Topological Hybrid Silicon Microlasers

Topological photonics provides a robust framework for strategically controlling light confinement and propagation dynamics. By exploiting the marriage between this notion and symmetry-constrained mode competition in an active setting, we experimentally demonstrate, for the first time, a hybrid silicon microlaser structure supporting a topologically protected zero-mode lasing. This mode is distributed over an integrated array of optical microrings, each supporting multiple modes that compete for gain. However, the mode competition is diminished in favor of a pre-defined topological state, which reflects the charge conjugation symmetry induced by the gain profile. Robust single-mode operation in the desired state prevails even with intentionally introduced perturbations. The demonstrated microlaser is hybrid-implemented on a silicon-on-insulator substrate, thereby readily suitable for integrated silicon photonics with potential applications in optical communication and computing.

physics.optics↗
arXiv2016

A Unified Approach for Learning the Parameters of Sum-Product Networks

We present a unified approach for learning the parameters of Sum-Product networks (SPNs). We prove that any complete and decomposable SPN is equivalent to a mixture of trees where each tree corresponds to a product of univariate distributions. Based on the mixture model perspective, we characterize the objective function when learning SPNs based on the maximum likelihood estimation (MLE) principle and show that the optimization problem can be formulated as a signomial program. We construct two parameter learning algorithms for SPNs by using sequential monomial approximations (SMA) and the concave-convex procedure (CCCP), respectively. The two proposed methods naturally admit multiplicative updates, hence effectively avoiding the projection operation. With the help of the unified framework, we also show that, in the case of SPNs, CCCP leads to the same algorithm as Expectation Maximization (EM) despite the fact that they are different in general.

cs.LG↗
arXiv2015

On the Relationship between Sum-Product Networks and Bayesian Networks

In this paper, we establish some theoretical connections between Sum-Product Networks (SPNs) and Bayesian Networks (BNs). We prove that every SPN can be converted into a BN in linear time and space in terms of the network size. The key insight is to use Algebraic Decision Diagrams (ADDs) to compactly represent the local conditional probability distributions at each node in the resulting BN by exploiting context-specific independence (CSI). The generated BN has a simple directed bipartite graphical structure. We show that by applying the Variable Elimination algorithm (VE) to the generated BN with ADD representations, we can recover the original SPN where the SPN can be viewed as a history record or caching of the VE inference process. To help state the proof clearly, we introduce the notion of {\em normal} SPN and present a theoretical analysis of the consistency and decomposability properties. We conclude the paper with some discussion of the implications of the proof and establish a connection between the depth of an SPN and a lower bound of the tree-width of its corresponding BN.

cs.AI↗
arXiv2015

Self-Adaptive Hierarchical Sentence Model

The ability to accurately model a sentence at varying stages (e.g., word-phrase-sentence) plays a central role in natural language processing. As an effort towards this goal we propose a self-adaptive hierarchical sentence model (AdaSent). AdaSent effectively forms a hierarchy of representations from words to phrases and then to sentences through recursive gated local composition of adjacent segments. We design a competitive mechanism (through gating networks) to allow the representations of the same sentence to be engaged in a particular learning task (e.g., classification), therefore effectively mitigating the gradient vanishing problem persistent in other recursive models. Both qualitative and quantitative analysis shows that AdaSent can automatically form and select the representations suitable for the task at hand during training, yielding superior classification performance over competitor models on 5 benchmark data sets.

cs.CL↗
arXiv2014

A Sober Look at Spectral Learning

Spectral learning recently generated lots of excitement in machine learning, largely because it is the first known method to produce consistent estimates (under suitable conditions) for several latent variable models. In contrast, maximum likelihood estimates may get trapped in local optima due to the non-convex nature of the likelihood function of latent variable models. In this paper, we do an empirical evaluation of spectral learning (SL) and expectation maximization (EM), which reveals an important gap between the theory and the practice. First, SL often leads to negative probabilities. Second, EM often yields better estimates than spectral learning and it does not seem to get stuck in local optima. We discuss how the rank of the model parameters and the amount of training data can yield negative probabilities. We also question the common belief that maximum likelihood estimators are necessarily inconsistent.

cs.LG↗