arXiv subjects

Shengyu Zhang

Publications and source records attributed to Shengyu Zhang.

At least 181 records · Page 10Linked to original sources

arXiv2022

MIC: Model-agnostic Integrated Cross-channel Recommenders

Semantically connecting users and items is a fundamental problem for the matching stage of an industrial recommender system. Recent advances in this topic are based on multi-channel retrieval to efficiently measure users' interest on items from the massive candidate pool. However, existing work are primarily built upon pre-defined retrieval channels, including User-CF (U2U), Item-CF (I2I), and Embedding-based Retrieval (U2I), thus access to the limited correlation between users and items which solely entail from partial information of latent interactions. In this paper, we propose a model-agnostic integrated cross-channel (MIC) approach for the large-scale recommendation, which maximally leverages the inherent multi-channel mutual information to enhance the matching performance. Specifically, MIC robustly models correlation within user-item, user-user, and item-item from latent interactions in a universal schema. For each channel, MIC naturally aligns pairs with semantic similarity and distinguishes them otherwise with more uniform anisotropic representation space. While state-of-the-art methods require specific architectural design, MIC intuitively considers them as a whole by enabling the complete information flow among users and items. Thus MIC can be easily plugged into other retrieval recommender systems. Extensive experiments show that our MIC helps several state-of-the-art models boost their performance on two real-world benchmarks. The satisfactory deployment of the proposed MIC on industrial online services empirically proves its scalability and flexibility.

cs.IR↗

arXiv2022

Retroformer: Pushing the Limits of Interpretable End-to-end Retrosynthesis Transformer

Retrosynthesis prediction is one of the fundamental challenges in organic synthesis. The task is to predict the reactants given a core product. With the advancement of machine learning, computer-aided synthesis planning has gained increasing interest. Numerous methods were proposed to solve this problem with different levels of dependency on additional chemical knowledge. In this paper, we propose Retroformer, a novel Transformer-based architecture for retrosynthesis prediction without relying on any cheminformatics tools for molecule editing. Via the proposed local attention head, the model can jointly encode the molecular sequence and graph, and efficiently exchange information between the local reactive region and the global reaction context. Retroformer reaches the new state-of-the-art accuracy for the end-to-end template-free retrosynthesis, and improves over many strong baselines on better molecule and reaction validity. In addition, its generative procedure is highly interpretable and controllable. Overall, Retroformer pushes the limits of the reaction reasoning ability of deep generative models.

physics.chem-ph↗

arXiv2022

SPLDExtraTrees: Robust machine learning approach for predicting kinase inhibitor resistance

Drug resistance is a major threat to the global health and a significant concern throughout the clinical treatment of diseases and drug development. The mutation in proteins that is related to drug binding is a common cause for adaptive drug resistance. Therefore, quantitative estimations of how mutations would affect the interaction between a drug and the target protein would be of vital significance for the drug development and the clinical practice. Computational methods that rely on molecular dynamics simulations, Rosetta protocols, as well as machine learning methods have been proven to be capable of predicting ligand affinity changes upon protein mutation. However, the severely limited sample size and heavy noise induced overfitting and generalization issues have impeded wide adoption of machine learning for studying drug resistance. In this paper, we propose a robust machine learning method, termed SPLDExtraTrees, which can accurately predict ligand binding affinity changes upon protein mutation and identify resistance-causing mutations. Especially, the proposed method ranks training data following a specific scheme that starts with easy-to-learn samples and gradually incorporates harder and diverse samples into the training, and then iterates between sample weight recalculations and model updates. In addition, we calculate additional physics-based structural features to provide the machine learning model with the valuable domain knowledge on proteins for this data-limited predictive tasks. The experiments substantiate the capability of the proposed method for predicting kinase inhibitor resistance under three scenarios, and achieves predictive accuracy comparable to that of molecular dynamics and Rosetta methods with much less computational costs.

q-bio.QM↗

arXiv2022

Optimizing Quantum Annealing Schedules with Monte Carlo Tree Search enhanced with neural networks

Quantum annealing is a practical approach to approximately implement the adiabatic quantum computational model under a real-world setting. The goal of an adiabatic algorithm is to prepare the ground state of a problem-encoded Hamiltonian at the end of an annealing path. This is typically achieved by driving the dynamical evolution of a quantum system slowly to enforce adiabaticity. Properly optimized annealing schedules often significantly accelerate the computational process. Inspired by the recent success of deep reinforcement learning such as DeepMind's AlphaZero, we propose a Monte Carlo Tree Search (MCTS) algorithm and its enhanced version boosted with neural networks, which we name QuantumZero (QZero), to automate the design of annealing schedules in a hybrid quantum-classical framework. Both the MCTS and QZero algorithms perform remarkably well in discovering effective annealing schedules even when the annealing time is short for the 3-SAT examples we consider in this study. Furthermore, the flexibility of neural networks allows us to apply transfer-learning techniques to boost QZero's performance. We demonstrate in benchmark studies, that MCTS and QZero perform more efficiently than other reinforcement learning algorithms in designing annealing schedules.

quant-ph↗

arXiv2021

Learning Large-scale Network Embedding from Representative Subgraph

We study the problem of large-scale network embedding, which aims to learn low-dimensional latent representations for network mining applications. Recent research in the field of network embedding has led to significant progress such as DeepWalk, LINE, NetMF, NetSMF. However, the huge size of many real-world networks makes it computationally expensive to learn network embedding from the entire network. In this work, we present a novel network embedding method called "NES", which learns network embedding from a small representative subgraph. NES leverages theories from graph sampling to efficiently construct representative subgraph with smaller size which can be used to make inferences about the full network, enabling significantly improved efficiency in embedding learning. Then, NES computes the network embedding from this representative subgraph, efficiently. Compared with well-known methods, extensive experiments on networks of various scales and types demonstrate that NES achieves comparable performance and significant efficiency superiority.

cs.SI↗

arXiv2021

Graph Self-Supervised Learning for Optoelectronic Properties of Organic Semiconductors

The search for new high-performance organic semiconducting molecules is challenging due to the vastness of the chemical space, machine learning methods, particularly deep learning models like graph neural networks (GNNs), have shown promising potential to address such challenge. However, practical applications of GNNs for chemistry are often limited by the availability of labelled data. Meanwhile, unlabelled molecular data is abundant and could potentially be utilized to alleviate the scarcity issue of labelled data. Here, we advocate the use of self-supervised learning to improve the performance of GNNs by pre-training them with unlabeled molecular data. We investigate regression problems involving ground and excited state properties, both relevant for optoelectronic properties of organic semiconductors. Additionally, we extend the self-supervised learning strategy to molecules in non-equilibrium configurations which are important for studying the effects of disorder. In all cases, we obtain considerable performance improvement over results without pre-training, in particular when labelled training data is limited, and such improvement is attributed to the capability of self-supervised learning in identifying structural similarity among unlabeled molecules.

physics.chem-ph↗

arXiv2021

Rapid and Unconditional Parametric Reset Protocol for Tunable Superconducting Qubits

Qubit initialization is a critical task in quantum computation and communication. Extensive efforts have been made to achieve this with high speed, efficiency and scalability. However, previous approaches have either been measurement-based and required fast feedback, suffered from crosstalk or required sophisticated calibration. Here, we report a fast and high-fidelity reset scheme, avoiding the issues above without any additional chip architecture. By modulating the flux through a transmon qubit, we realize a swap between the qubit and its readout resonator that suppresses the excited state population to 0.08% $\pm$ 0.08% within 34 ns (284 ns if photon depletion of the resonator is required). Furthermore, our approach (i) can achieve effective second excited state depletion, (ii) has negligible effects on neighbouring qubits, and (iii) offers a way to entangle the qubit with an itinerant single photon, useful in quantum communication applications.

quant-ph↗

arXiv2021

Shortcuts to Adiabaticity for Open Systems in Circuit Quantum Electrodynamics

Shortcuts to adiabaticity (STA) are powerful quantum control methods, allowing quick evolution into target states of otherwise slow adiabatic dynamics. Such methods have widespread applications in quantum technologies, and various STA protocols have been demonstrated in closed systems. However, realizing STA for open quantum systems has presented a greater challenge, due to complex controls required in existing proposals. Here we present the first experimental demonstration of STA for open quantum systems, using a superconducting circuit QED system consisting of two coupled bosonic oscillators and a transmon qubit. By applying a counterdiabatic driving pulse, we reduce the adiabatic evolution time of a single lossy mode from 800 ns to 100 ns. In addition, we propose and implement an optimal control protocol to achieve fast and qubit-unconditional equilibrium of multiple lossy modes. Our results pave the way for accelerating dynamics of open quantum systems and have potential applications in designing fast open-system protocols of physical and interdisciplinary interest, such as accelerating bioengineering and chemical reaction dynamics.

quant-ph↗

arXiv2021

Differentiable Quantum Architecture Search

Quantum architecture search (QAS) is the process of automating architecture engineering of quantum circuits. It has been desired to construct a powerful and general QAS platform which can significantly accelerate current efforts to identify quantum advantages of error-prone and depth-limited quantum circuits in the NISQ era. Hereby, we propose a general framework of differentiable quantum architecture search (DQAS), which enables automated designs of quantum circuits in an end-to-end differentiable fashion. We present several examples of circuit design problems to demonstrate the power of DQAS. For instance, unitary operations are decomposed into quantum gates, noisy circuits are re-designed to improve accuracy, and circuit layouts for quantum approximation optimization algorithm are automatically discovered and upgraded for combinatorial optimization problems. These results not only manifest the vast potential of DQAS being an essential tool for the NISQ application developments, but also present an interesting research topic from the theoretical perspective as it draws inspirations from the newly emerging interdisciplinary paradigms of differentiable programming, probabilistic programming, and quantum programming.

quant-ph↗

arXiv2021

Variational Quantum Simulation of Chemical Dynamics with Quantum Computers

Classical simulation of real-space quantum dynamics is challenging due to the exponential scaling of computational cost with system dimensions. Quantum computer offers the potential to simulate quantum dynamics with polynomial complexity; however, existing quantum algorithms based on the split-operator techniques require large-scale fault-tolerant quantum computers that remain elusive in the near future. Here we present variational simulations of real-space quantum dynamics suitable for implementation in Noisy Intermediate-Scale Quantum (NISQ) devices. The Hamiltonian is first encoded onto qubits using a discrete variable representation (DVR) and binary encoding scheme. We show that direct application of real-time variational quantum algorithm based on the McLachlan's principle is inefficient as the measurement cost grows exponentially with the qubit number for general potential energy and extremely small time-step size is required to achieve accurate results. Motivated by the insights that most chemical dynamics occur in the low energy subspace, we propose a subspace expansion method by projecting the total Hamiltonian, including the time-dependent driving field, onto the system low-energy eigenstate subspace using quantum computers, the exact quantum dynamics within the subspace can then be solved classically. We show that the measurement cost of the subspace approach grows polynomially with dimensionality for general potential energy. Our numerical examples demonstrate the capability of our approach, even under intense laser fields. Our work opens the possibility of simulating chemical dynamics with NISQ hardware.

quant-ph↗

arXiv2021

Stable Prediction on Graphs with Agnostic Distribution Shift

Graph is a flexible and effective tool to represent complex structures in practice and graph neural networks (GNNs) have been shown to be effective on various graph tasks with randomly separated training and testing data. In real applications, however, the distribution of training graph might be different from that of the test one (e.g., users' interactions on the user-item training graph and their actual preference on items, i.e., testing environment, are known to have inconsistencies in recommender systems). Moreover, the distribution of test data is always agnostic when GNNs are trained. Hence, we are facing the agnostic distribution shift between training and testing on graph learning, which would lead to unstable inference of traditional GNNs across different test environments. To address this problem, we propose a novel stable prediction framework for GNNs, which permits both locally and globally stable learning and prediction on graphs. In particular, since each node is partially represented by its neighbors in GNNs, we propose to capture the stable properties for each node (locally stable) by re-weighting the information propagation/aggregation processes. For global stability, we propose a stable regularizer that reduces the training losses on heterogeneous environments and thus warping the GNNs to generalize well. We conduct extensive experiments on several graph benchmarks and a noisy industrial recommendation dataset that is collected from 5 consecutive days during a product promotion festival. The results demonstrate that our method outperforms various SOTA GNNs for stable prediction on graphs with agnostic distribution shift, including shift caused by node labels and attributes.

cs.LG↗

arXiv2021

Multi-trends Enhanced Dynamic Micro-video Recommendation

The explosively generated micro-videos on content sharing platforms call for recommender systems to permit personalized micro-video discovery with ease. Recent advances in micro-video recommendation have achieved remarkable performance in mining users' current preference based on historical behaviors. However, most of them neglect the dynamic and time-evolving nature of users' preference, and the prediction on future micro-videos with historically mined preference may deteriorate the effectiveness of recommender systems. In this paper, we propose the DMR framework to explicitly model dynamic multi-trends of users' current preference and make predictions based on both the history and future potential trends. We devise the DMR framework, which comprises: 1) the implicit user network module which identifies sequence fragments from other users with similar interests and extracts the sequence fragments that are chronologically behind the identified fragments; 2) the multi-trend routing module which assigns each extracted sequence fragment into a trend group and update the corresponding trend vector; 3) the history-future trend prediction module jointly uses the history preference vectors and future trend vectors to yield the final click-through-rate. We validate the effectiveness of DMR over multiple state-of-the-art micro-video recommenders on two publicly available real-world datasets. Relatively extensive analysis further demonstrate the superiority of modeling dynamic multi-trend for micro-video recommendation.

cs.IR↗

arXiv2021

Why Do We Click: Visual Impression-aware News Recommendation

There is a soaring interest in the news recommendation research scenario due to the information overload. To accurately capture users' interests, we propose to model multi-modal features, in addition to the news titles that are widely used in existing works, for news recommendation. Besides, existing research pays little attention to the click decision-making process in designing multi-modal modeling modules. In this work, inspired by the fact that users make their click decisions mostly based on the visual impression they perceive when browsing news, we propose to capture such visual impression information with visual-semantic modeling for news recommendation. Specifically, we devise the local impression modeling module to simultaneously attend to decomposed details in the impression when understanding the semantic meaning of news title, which could explicitly get close to the process of users reading news. In addition, we inspect the impression from a global view and take structural information, such as the arrangement of different fields and spatial position of different words on the impression, into the modeling of multiple modalities. To accommodate the research of visual impression-aware news recommendation, we extend the text-dominated news recommendation dataset MIND by adding snapshot impression images and will release it to nourish the research field. Extensive comparisons with the state-of-the-art news recommenders along with the in-depth analyses demonstrate the effectiveness of the proposed method and the promising capability of modeling visual impressions for the content-based recommenders.

cs.IR↗

arXiv2021

Fast Extraction of Word Embedding from Q-contexts

The notion of word embedding plays a fundamental role in natural language processing (NLP). However, pre-training word embedding for very large-scale vocabulary is computationally challenging for most existing methods. In this work, we show that with merely a small fraction of contexts (Q-contexts)which are typical in the whole corpus (and their mutual information with words), one can construct high-quality word embedding with negligible errors. Mutual information between contexts and words can be encoded canonically as a sampling state, thus, Q-contexts can be fast constructed. Furthermore, we present an efficient and effective WEQ method, which is capable of extracting word embedding directly from these typical contexts. In practical scenarios, our algorithm runs 11$\sim$13 times faster than well-established methods. By comparing with well-known methods such as matrix factorization, word2vec, GloVeand fasttext, we demonstrate that our method achieves comparable performance on a variety of downstream NLP tasks, and in the meanwhile maintains run-time and resource advantages over all these baselines.

cs.CL↗

arXiv2021

CauseRec: Counterfactual User Sequence Synthesis for Sequential Recommendation

Learning user representations based on historical behaviors lies at the core of modern recommender systems. Recent advances in sequential recommenders have convincingly demonstrated high capability in extracting effective user representations from the given behavior sequences. Despite significant progress, we argue that solely modeling the observational behaviors sequences may end up with a brittle and unstable system due to the noisy and sparse nature of user interactions logged. In this paper, we propose to learn accurate and robust user representations, which are required to be less sensitive to (attack on) noisy behaviors and trust more on the indispensable ones, by modeling counterfactual data distribution. Specifically, given an observed behavior sequence, the proposed CauseRec framework identifies dispensable and indispensable concepts at both the fine-grained item level and the abstract interest level. CauseRec conditionally samples user concept sequences from the counterfactual data distributions by replacing dispensable and indispensable concepts within the original concept sequence. With user representations obtained from the synthesized user sequences, CauseRec performs contrastive user representation learning by contrasting the counterfactual with the observational. We conduct extensive experiments on real-world public recommendation benchmarks and justify the effectiveness of CauseRec with multi-aspects model analysis. The results demonstrate that the proposed CauseRec outperforms state-of-the-art sequential recommenders by learning accurate and robust user representations.

cs.IR↗

arXiv2021

Exploiting Different Levels of Parallelism in the Quantum Control Microarchitecture for Superconducting Qubits

As current Noisy Intermediate Scale Quantum (NISQ) devices suffer from decoherence errors, any delay in the instruction execution of quantum control microarchitecture can lead to the loss of quantum information and incorrect computation results. Hence, it is crucial for the control microarchitecture to issue quantum operations to the Quantum Processing Unit (QPU) in time. As in classical microarchitecture, parallelism in quantum programs needs to be exploited for speedup. However, three challenges emerge in the quantum scenario: 1) the quantum feedback control can introduce significant pipeline stall latency; 2) timing control is required for all quantum operations; 3) QPU requires a deterministic operation supply to prevent the accumulation of quantum errors. In this paper, we propose a novel control microarchitecture design to exploit Circuit Level Parallelism (CLP) and Quantum Operation Level Parallelism (QOLP). Firstly, we develop a Multiprocessor architecture to exploit CLP, which supports dynamic scheduling of different sub-circuits. This architecture can handle parallel feedback control and minimize the potential overhead that disrupts the timing control. Secondly, we propose a Quantum Superscalar approach that exploits QOLP by efficiently executing massive quantum instructions in parallel. Both methods issue quantum operations to QPU deterministically. In the benchmark test of a Shor syndrome measurement, a six-core implementation of our proposal achieves up to 2.59$\times$ speedup compared with a single core. For various canonical quantum computing algorithms, our superscalar approach achieves an average of 4.04$\times$ improvement over a baseline design. Finally, We perform a simultaneous randomized benchmarking (simRB) experiment on a real QPU using the proposed microarchitecture for validation.

quant-ph↗

arXiv2021

Simulation of Condensed-Phase Spectroscopy with Near-term Digital Quantum Computer

Spectroscopy is an indispensable tool in understanding the structures and dynamics of molecular systems. However computational modelling of spectroscopy is challenging due to the exponential scaling of computational complexity with system sizes unless drastic approximations are made. Quantum computer could potentially overcome these classically intractable computational tasks, but existing approaches using quantum computers to simulate spectroscopy can only handle isolated and static molecules. In this work we develop a workflow that combines multi-scale modeling and time-dependent variational quantum algorithm to compute the linear spectroscopy of systems interacting with their condensed-phase environment via the relevant time correlation function. We demonstrate the feasibility of our approach by numerically simulating the UV-Vis absorption spectra of organic semiconductors. We show that our dynamical approach captures several spectral features that are otherwise overlooked by static methods. Our method can be directly used for other linear condensed-phase spectroscopy and could potentially be extended to nonlinear multi-dimensional spectroscopy.

quant-ph↗

arXiv2021

Variational Quantum-Neural Hybrid Eigensolver

The variational quantum eigensolver (VQE) is one of the most representative quantum algorithms in the noisy intermediate-size quantum (NISQ) era, and is generally speculated to deliver one of the first quantum advantages for the ground-state simulations of some non-trivial Hamiltonians. However, short quantum coherence time and limited availability of quantum hardware resources in the NISQ hardware strongly restrain the capacity and expressiveness of VQEs. In this Letter, we introduce the variational quantum-neural hybrid eigensolver (VQNHE) in which the shallow-circuit quantum ansatz can be further enhanced by classical post-processing with neural networks. We show that VQNHE consistently and significantly outperforms VQE in simulating ground-state energies of quantum spins and molecules given the same amount of quantum resources. More importantly, we demonstrate that for arbitrary post-processing neural functions, VQNHE only incurs an polynomial overhead of processing time and represents the first scalable method to exponentially accelerate VQE with non-unitary post-processing that can be efficiently implemented in the NISQ era.

quant-ph↗