arXiv subjects

Yiwei Li

Publications and source records attributed to Yiwei Li.

141 records · Page 8Linked to original sources

arXiv2022

Diversifying Neural Dialogue Generation via Negative Distillation

Generative dialogue models suffer badly from the generic response problem, limiting their applications to a few toy scenarios. Recently, an interesting approach, namely negative training, has been proposed to alleviate this problem by reminding the model not to generate high-frequency responses during training. However, its performance is hindered by two issues, ignoring low-frequency but generic responses and bringing low-frequency but meaningless responses. In this paper, we propose a novel negative training paradigm, called negative distillation, to keep the model away from the undesirable generic responses while avoiding the above problems. First, we introduce a negative teacher model that can produce query-wise generic responses, and then the student model is required to maximize the distance with multi-level negative knowledge. Empirical results show that our method outperforms previous negative training methods significantly.

cs.CL↗

arXiv2022

Federated Stochastic Primal-dual Learning with Differential Privacy

Federated learning (FL) is a new paradigm that enables many clients to jointly train a machine learning (ML) model under the orchestration of a parameter server while keeping the local data not being exposed to any third party. However, the training of FL is an interactive process between local clients and the parameter server. Such process would cause privacy leakage since adversaries may retrieve sensitive information by analyzing the overheard messages. In this paper, we propose a new federated stochastic primal-dual algorithm with differential privacy (FedSPD-DP). Compared to the existing methods, the proposed FedSPD-DP incorporates local stochastic gradient descent (local SGD) and partial client participation (PCP) for addressing the issues of communication efficiency and straggler effects due to randomly accessed clients. Our analysis shows that the data sampling strategy and PCP can enhance the data privacy whereas the larger number of local SGD steps could increase privacy leakage, revealing a non-trivial tradeoff between algorithm communication efficiency and privacy protection. Specifically, we show that, by guaranteeing $(ε, δ)$-DP for each client per communication round, the proposed algorithm guarantees $(\mathcal{O}(qε\sqrt{p T}), δ)$-DP after $T$ communication rounds while maintaining an $\mathcal{O}(1/\sqrt{pTQ})$ convergence rate for a convex and non-smooth learning problem, where $Q$ is the number of local SGD steps, $p$ is the client sampling probability, $q=\max_{i} q_i/\sqrt{1-q_i}$ and $q_i$ is the data sampling probability of each client under PCP. Experiment results are presented to evaluate the practical performance of the proposed algorithm and comparison with state-of-the-art methods.

cs.LG↗

arXiv2021

Generating Relevant and Coherent Dialogue Responses using Self-separated Conditional Variational AutoEncoders

Conditional Variational AutoEncoder (CVAE) effectively increases the diversity and informativeness of responses in open-ended dialogue generation tasks through enriching the context vector with sampled latent variables. However, due to the inherent one-to-many and many-to-one phenomena in human dialogues, the sampled latent variables may not correctly reflect the contexts' semantics, leading to irrelevant and incoherent generated responses. To resolve this problem, we propose Self-separated Conditional Variational AutoEncoder (abbreviated as SepaCVAE) that introduces group information to regularize the latent variables, which enhances CVAE by improving the responses' relevance and coherence while maintaining their diversity and informativeness. SepaCVAE actively divides the input data into groups, and then widens the absolute difference between data pairs from distinct groups, while narrowing the relative distance between data pairs in the same group. Empirical results from automatic evaluation and detailed analysis demonstrate that SepaCVAE can significantly boost responses in well-established open-domain dialogue datasets.

cs.CL↗

arXiv2021

THINK: A Novel Conversation Model for Generating Grammatically Correct and Coherent Responses

Many existing conversation models that are based on the encoder-decoder framework have focused on ways to make the encoder more complicated to enrich the context vectors so as to increase the diversity and informativeness of generated responses. However, these approaches face two problems. First, the decoder is too simple to effectively utilize the previously generated information and tends to generate duplicated and self-contradicting responses. Second, the complex encoder tends to generate diverse but incoherent responses because the complex context vectors may deviate from the original semantics of context. In this work, we proposed a conversation model named "THINK" (Teamwork generation Hover around Impressive Noticeable Keywords) to make the decoder more complicated and avoid generating duplicated and self-contradicting responses. The model simplifies the context vectors and increases the coherence of generated responses in a reasonable way. For this model, we propose Teamwork generation framework and Semantics Extractor. Compared with other baselines, both automatic and human evaluation showed the advantages of our model.

cs.CL↗

arXiv2021

Robust Weyl Semimetal and Pressure-induced Superconductivity in Quasi-One-Dimensional (NbSe4)2I

A search for the single material system that simultaneously exhibits topological phase and intrinsic superconductivity has been largely limited, although such a system is far more favorable especially for the quantum device applications. Except artificially engineered topological superconductivity in heterostructure systems, another alternative is to have superconductivity arising from the topological materials by pressure or other clean technology. Here, based on first-principles calculations, we first show that quasi-one-dimensional compound (NbSe4)2I represents a rare example of a chiral Weyl semimetal in which the set of symmetry-related Weyl points (WPs) exhibit the same chiral charge at a certain energy. The net chiral charge (NCC) of the below Fermi level EF (or a certain energy) can be tuned by pressure. In addition, a partial disorder induced by pressure accompanied with superconductivity emerges. Although amorphization of the iodine sub-lattice under high pressure, the one-dimensional NbSe4 chains in (NbSe4)2I remain intact and provide a superconducting channel in one dimension. Our combined theoretical and experimental research provide critical insight into a new phase of the one-dimensional system, in which distinctive phase transitions and correlated topological states emerge upon compression.

cond-mat.supr-con↗

arXiv2021

A Charge-Density-Wave Topological Semimetal

Topological physics and strong electron-electron correlations in quantum materials are typically studied independently. However, there have been rapid recent developments in quantum materials in which topological phase transitions emerge when the single-particle band structure is modified by strong interactions. We here demonstrate that the room-temperature phase of (TaSe$_4$)$_2$I is a Weyl semimetal with 24 pairs of Weyl nodes. Owing to its quasi-1D structure, (TaSe$_4$)$_2$I hosts an established CDW instability just below room temperature. Using X-ray diffraction, angle-resolved photoemission spectroscopy, and first-principles calculations, we find that the CDW in (TaSe$_4$)$_2$I couples the bulk Weyl points and opens a band gap. The correlation-driven topological phase transition in (TaSe$_4$)$_2$I provides a route towards observing condensed-matter realizations of axion electrodynamics in the gapped regime, topological chiral response effects in the semimetallic phase, and represents an avenue for exploring the interplay of correlations and topology in a solid-state material.

cond-mat.mtrl-sci↗

arXiv2020

Observation of topological electronic structure in quasi-1D superconductor TaSe3

Topological superconductors (TSCs), with the capability to host Majorana bound states that can lead to non-Abelian statistics and application in quantum computation, have been one of the most intensively studied topics in condensed matter physics recently. Up to date, only a few compounds have been proposed as candidates of intrinsic TSCs, such as doped topological insulator CuxBi2Se3 and iron-based superconductor FeTe0.55Se0.45. Here, by carrying out synchrotron and laser based angle-resolved photoemission spectroscopy (ARPES), we systematically investigated the electronic structure of a quasi-1D superconductor TaSe3, and identified the nontrivial topological surface states. In addition, our scanning tunneling microscopy (STM) study revealed a clean cleaved surface with a persistent superconducting gap, proving it suitable for further investigation of potential Majorana modes. These results prove TaSe3 as a stoichiometric TSC candidate that is stable and exfoliable, therefore a great platform for the study of rich novel phenomena and application potentials.

cond-mat.mtrl-sci↗

arXiv2020

Electronic Origin for the Enhanced Thermoelectric Efficiency of Cu2Se

Thermoelectric materials (TMs) can uniquely convert waste heat into electricity, which provides a potential solution for the global energy crisis that is increasingly severe. Bulk Cu2Se, with ionic conductivity of Cu ions, exhibits a significant enhancement of its thermoelectric figure of merit zT by a factor of ~3 near its structural transition around 400 K. Here, we show a systematic study of the electronic structure of Cu2Se and its temperature evolution using high-resolution angle-resolved photoemission spectroscopy. Upon heating across the structural transition, the electronic states near the corner of the Brillouin zone gradually disappear, while the bands near the centre of Brillouin zone shift abruptly towards high binding energies and develop an energy gap. Interestingly, the observed band reconstruction well reproduces the temperature evolution of the Seebeck coefficient of Cu2Se, providing an electronic origin for the drastic enhancement of the thermoelectric performance near 400 K. The current results not only bridge among structural phase transition, electronic structures, and thermoelectric properties in a condensed matter system, but also provide valuable insights into the search and design of new generation of thermoelectric materials.

cond-mat.mtrl-sci↗

arXiv2020

Super Resolution Convolutional Neural Network for Feature Extraction in Spectroscopic Data

Two dimensional (2D) peak finding is a common practice in data analysis for physics experiments, which is typically achieved by computing the local derivatives. However, this method is inherently unstable when the local landscape is complicated, or the signal-to-noise ratio of the data is low. In this work, we propose a new method in which the peak tracking task is formalized as an inverse problem, thus can be solved with a convolutional neural network (CNN). In addition, we show that the underlying physics principle of the experiments can be used to generate the training data. By generalizing the trained neural network on real experimental data, we show that the CNN method can achieve comparable or better results than traditional derivative based methods. This approach can be further generalized in different physics experiments when the physical process is known.

physics.data-an↗

arXiv2019

Generalization of k-means Related Algorithms

This article briefly introduced Arthur and Vassilvitshii's work on \textbf{k-means++} algorithm and further generalized the center initialization process. It is found that choosing the most distant sample point from the nearest center as new center can mostly have the same effect as the center initialization process in the \textbf{k-means++} algorithm.

cs.LG↗

arXiv2018

Variational Neural Networks: Every Layer and Neuron Can Be Unique

The choice of activation function can significantly influence the performance of neural networks. The lack of guiding principles for the selection of activation function is lamentable. We try to address this issue by introducing our variational neural networks, where the activation function is represented as a linear combination of possible candidate functions, and an optimal activation is obtained via minimization of a loss function using gradient descent method. The gradient formulae for the loss function with respect to these expansion coefficients are central for the implementation of gradient descent algorithm, and here we derive these gradient formulae.

cs.LG↗

arXiv2017

Distribution of randomly diffusing particles in inhomogeneous media

Diffusion can be conceptualized, at microscopic scales, as the random hopping of particles between neighboring lattice sites. In the case of diffusion in inhomogeneous media, distinct spatial domains in the system may yield distinct particle hopping rates. Starting from the master equations (MEs) governing diffusion in inhomogeneous media we derive here, for arbitrary spatial dimensions, the deterministic lattice equations (DLEs) specifying the average particle number at each lattice site for randomly diffusing particles in inhomogeneous media. We consider the case of free diffusion with no steric constraints on the maximum particle number per lattice site as well as the case of diffusion under steric constraints imposing a maximum particle concentration. We find, for both transient and asymptotic regimes, excellent agreement between the DLEs and kinetic Monte Carlo simulations of the MEs. The DLEs provide a computationally efficient method for predicting the (average) distribution of randomly diffusing particles in inhomogeneous media, with the number of DLEs associated with a given system being independent of the number of particles in the system. From the DLEs we obtain general analytic expressions for the steady-state particle distributions for free diffusion and, in special cases, diffusion under steric constraints in inhomogeneous media. We find that, in the steady state of the system, the average fraction of particles in a given domain is independent of most system properties, such as the arrangement and shape of domains, and only depends on the number of lattice sites in each domain, the hopping rates, the number of distinct particle species, and the total number of particles of each particle species in the system. Our results provide general insights into the role of spatially inhomogeneous particle hopping rates in setting the particle distributions in inhomogeneous media.

physics.bio-ph↗

arXiv2017

SPRINT: Ultrafast protein-protein interaction prediction of the entire human interactome

Proteins perform their functions usually by interacting with other proteins. Predicting which proteins interact is a fundamental problem. Experimental methods are slow, expensive, and have a high rate of error. Many computational methods have been proposed among which sequence-based ones are very promising. However, so far no such method is able to predict effectively the entire human interactome: they require too much time or memory. We present SPRINT (Scoring PRotein INTeractions), a new sequence-based algorithm and tool for predicting protein-protein interactions. We comprehensively compare SPRINT with state-of-the-art programs on seven most reliable human PPI datasets and show that it is more accurate while running orders of magnitude faster and using very little memory. SPRINT is the only program that can predict the entire human interactome. Our goal is to transform the very challenging problem of predicting the entire human interactome into a routine task. The source code of SPRINT is freely available from github.com/lucian-ilie/SPRINT/ and the datasets and predicted PPIs from www.csd.uwo.ca/faculty/ilie/SPRINT/.

q-bio.BM↗

arXiv2017

Stochastic lattice model of synaptic membrane protein domains

Neurotransmitter receptor molecules, concentrated in synaptic membrane domains along with scaffolds and other kinds of proteins, are crucial for signal transmission across chemical synapses. In common with other membrane protein domains, synaptic domains are characterized by low protein copy numbers and protein crowding, with rapid stochastic turnover of individual molecules. We study here in detail a stochastic lattice model of the receptor-scaffold reaction-diffusion dynamics at synaptic domains that was found previously to capture, at the mean-field level, the self-assembly, stability, and characteristic size of synaptic domains observed in experiments. We show that our stochastic lattice model yields quantitative agreement with mean-field models of nonlinear diffusion in crowded membranes. Through a combination of analytic and numerical solutions of the master equation governing the reaction dynamics at synaptic domains, together with kinetic Monte Carlo simulations, we find substantial discrepancies between mean-field and stochastic models for the reaction dynamics at synaptic domains. Based on the reaction and diffusion properties of synaptic receptors and scaffolds suggested by previous experiments and mean-field calculations, we show that the stochastic reaction-diffusion dynamics of synaptic receptors and scaffolds provide a simple physical mechanism for collective fluctuations in synaptic domains, the molecular turnover observed at synaptic domains, key features of the observed single-molecule trajectories, and spatial heterogeneity in the effective rates at which receptors and scaffolds are recycled at the cell membrane. Our work sheds light on the physical mechanisms and principles linking the collective properties of membrane protein domains to the stochastic dynamics that rule their molecular~components.

q-bio.SC↗

arXiv2016

Stochastic single-molecule dynamics of synaptic membrane protein domains

Motivated by single-molecule experiments on synaptic membrane protein domains, we use a stochastic lattice model to study protein reaction and diffusion processes in crowded membranes. We find that the stochastic reaction-diffusion dynamics of synaptic proteins provide a simple physical mechanism for collective fluctuations in synaptic domains, the molecular turnover observed at synaptic domains, key features of the single-molecule trajectories observed for synaptic proteins, and spatially inhomogeneous protein lifetimes at the cell membrane. Our results suggest that central aspects of the single-molecule and collective dynamics observed for membrane protein domains can be understood in terms of stochastic reaction-diffusion processes at the cell membrane.

q-bio.SC↗