arXiv subjects

Bo Wen

Publications and source records attributed to Bo Wen.

At least 55 records · Page 3Linked to original sources

arXiv2022

Global Pointer: Novel Efficient Span-based Approach for Named Entity Recognition

Named entity recognition (NER) task aims at identifying entities from a piece of text that belong to predefined semantic types such as person, location, organization, etc. The state-of-the-art solutions for flat entities NER commonly suffer from capturing the fine-grained semantic information in underlying texts. The existing span-based approaches overcome this limitation, but the computation time is still a concern. In this work, we propose a novel span-based NER framework, namely Global Pointer (GP), that leverages the relative positions through a multiplicative attention mechanism. The ultimate goal is to enable a global view that considers the beginning and the end positions to predict the entity. To this end, we design two modules to identify the head and the tail of a given entity to enable the inconsistency between the training and inference processes. Moreover, we introduce a novel classification loss function to address the imbalance label problem. In terms of parameters, we introduce a simple but effective approximate method to reduce the training parameters. We extensively evaluate GP on various benchmark datasets. Our extensive experiments demonstrate that GP can outperform the existing solution. Moreover, the experimental results show the efficacy of the introduced loss function compared to softmax and entropy alternatives.

cs.CL↗

arXiv2022

Experimentally realized memristive memory augmented neural network

Lifelong on-device learning is a key challenge for machine intelligence, and this requires learning from few, often single, samples. Memory augmented neural network has been proposed to achieve the goal, but the memory module has to be stored in an off-chip memory due to its size. Therefore the practical use has been heavily limited. Previous works on emerging memory-based implementation have difficulties in scaling up because different modules with various structures are difficult to integrate on the same chip and the small sense margin of the content addressable memory for the memory module heavily limited the degree of mismatch calculation. In this work, we implement the entire memory augmented neural network architecture in a fully integrated memristive crossbar platform and achieve an accuracy that closely matches standard software on digital hardware for the Omniglot dataset. The successful demonstration is supported by implementing new functions in crossbars in addition to widely reported matrix multiplications. For example, the locality-sensitive hashing operation is implemented in crossbar arrays by exploiting the intrinsic stochasticity of memristor devices. Besides, the content-addressable memory module is realized in crossbars, which also supports the degree of mismatches. Simulations based on experimentally validated models show such an implementation can be efficiently scaled up for one-shot learning on the Mini-ImageNet dataset. The successful demonstration paves the way for practical on-device lifelong learning and opens possibilities for novel attention-based algorithms not possible in conventional hardware.

cs.ET↗

arXiv2022

The Role of Permanent and Induced Electrostatic Dipole Moments for Schottky Barriers in Janus MXY/Graphene Heterostructures: a First Principles Study

The Schottky barrier height ($E_{SBH}$) is a crucial factor in determining the transport properties of semiconductor materials as it directly regulates the carrier mobility in opto-electronics devices. In principle, van der Waals (vdW) Janus heterostructures offer an appealing avenue to controlling the ESBH. However, the underlying atomistic mechanisms are far from understood conclusively, which prompts for further research in the topic. To this end, here, we carry out an extensive first principles study of the electronic properties and $E_{SBH}$ of several vdW Janus MXY/Graphene (M=Mo, W; X, Y=S, Se, Te) heterostructures. The results of the simulations show that by changing the composition and geometry of the heterostructure's interface, it is possible to control its electrical contact, thence electron transport properties, from Ohmic to Schottky with nearly one order of magnitude variations in the $E_{SBH}$. Detailed analysis of the simulations enables rationalization of this highly attractive property on the basis of the interplay between the permanent dipole moment of the Janus MXY sheet and the induced one due to interfacial charge redistribution at the MXY/Gr interface. Such an interplay is shown to be highly effective in altering the electrostatic potential difference across the vdW Janus heterostructure, determining its ESBH, thence Schottky (Ohmic) contact type. These computational findings contribute guidelines to control electrical contacts in Janus heterostructures towards rational design of electrical contacts in nanoscale devices.

cond-mat.mes-hall↗

arXiv2022

PKGM: A Pre-trained Knowledge Graph Model for E-commerce Application

In recent years, knowledge graphs have been widely applied as a uniform way to organize data and have enhanced many tasks requiring knowledge. In online shopping platform Taobao, we built a billion-scale e-commerce product knowledge graph. It organizes data uniformly and provides item knowledge services for various tasks such as item recommendation. Usually, such knowledge services are provided through triple data, while this implementation includes (1) tedious data selection works on product knowledge graph and (2) task model designing works to infuse those triples knowledge. More importantly, product knowledge graph is far from complete, resulting error propagation to knowledge enhanced tasks. To avoid these problems, we propose a Pre-trained Knowledge Graph Model (PKGM) for the billion-scale product knowledge graph. On the one hand, it could provide item knowledge services in a uniform way with service vectors for embedding-based and item-knowledge-related task models without accessing triple data. On the other hand, it's service is provided based on implicitly completed product knowledge graph, overcoming the common the incomplete issue. We also propose two general ways to integrate the service vectors from PKGM into downstream task models. We test PKGM in five knowledge-related tasks, item classification, item resolution, item recommendation, scene detection and sequential recommendation. Experimental results show that PKGM introduces significant performance gains on these tasks, illustrating the useful of service vectors from PKGM.

cs.AI↗

arXiv2021

Scale-Aware Neural Architecture Search for Multivariate Time Series Forecasting

Multivariate time series (MTS) forecasting has attracted much attention in many intelligent applications. It is not a trivial task, as we need to consider both intra-variable dependencies and inter-variable dependencies. However, existing works are designed for specific scenarios, and require much domain knowledge and expert efforts, which is difficult to transfer between different scenarios. In this paper, we propose a scale-aware neural architecture search framework for MTS forecasting (SNAS4MTF). A multi-scale decomposition module transforms raw time series into multi-scale sub-series, which can preserve multi-scale temporal patterns. An adaptive graph learning module infers the different inter-variable dependencies under different time scales without any prior knowledge. For MTS forecasting, a search space is designed to capture both intra-variable dependencies and inter-variable dependencies at each time scale. The multi-scale decomposition, adaptive graph learning, and neural architecture search modules are jointly learned in an end-to-end framework. Extensive experiments on two real-world datasets demonstrate that SNAS4MTF achieves a promising performance compared with the state-of-the-art methods.

cs.LG↗

arXiv2021

Learning from Multiple Time Series: A Deep Disentangled Approach to Diversified Time Series Forecasting

Time series forecasting is a significant problem in many applications, e.g., financial predictions and business optimization. Modern datasets can have multiple correlated time series, which are often generated with global (shared) regularities and local (specific) dynamics. In this paper, we seek to tackle such forecasting problems with DeepDGL, a deep forecasting model that disentangles dynamics into global and local temporal patterns. DeepDGL employs an encoder-decoder architecture, consisting of two encoders to learn global and local temporal patterns, respectively, and a decoder to make multi-step forecasting. Specifically, to model complicated global patterns, the vector quantization (VQ) module is introduced, allowing the global feature encoder to learn a shared codebook among all time series. To model diversified and heterogenous local patterns, an adaptive parameter generation module enhanced by the contrastive multi-horizon coding (CMC) is proposed to generate the parameters of the local feature encoder for each individual time series, which maximizes the mutual information between the series-specific context variable and the long/short-term representations of the corresponding time series. Our experiments on several real-world datasets show that DeepDGL outperforms existing state-of-the-art models.

cs.LG↗

arXiv2021

Billion-scale Pre-trained E-commerce Product Knowledge Graph Model

In recent years, knowledge graphs have been widely applied to organize data in a uniform way and enhance many tasks that require knowledge, for example, online shopping which has greatly facilitated people's life. As a backbone for online shopping platforms, we built a billion-scale e-commerce product knowledge graph for various item knowledge services such as item recommendation. However, such knowledge services usually include tedious data selection and model design for knowledge infusion, which might bring inappropriate results. Thus, to avoid this problem, we propose a Pre-trained Knowledge Graph Model (PKGM) for our billion-scale e-commerce product knowledge graph, providing item knowledge services in a uniform way for embedding-based models without accessing triple data in the knowledge graph. Notably, PKGM could also complete knowledge graphs during servicing, thereby overcoming the common incompleteness issue in knowledge graphs. We test PKGM in three knowledge-related tasks including item classification, same item identification, and recommendation. Experimental results show PKGM successfully improves the performance of each task.

cs.AI↗

arXiv2019

Ferroelectric driven exciton and trion modulation in monolayer MoSe2 and WSe2

In this work, we show how domain engineered lithium niobate can be used to selectively dope monolayer MoSe2 and WSe2 and demonstrate that these ferroelectric domains can significantly enhance or inhibit photoluminescence (PL) with the most dramatic modulation occurring at the heterojunction interface between two domains. A micro-PL and Raman system is used to obtain spatially resolved images of the differently doped transition metal dichalcogenides (TMDs). The domain inverted lithium niobate causes changes in the TMDs due to electrostatic doping as a result of the remnant polarization from the substrate. Moreover, the differently doped TMDs (n-type MoSe2 and p-type WSe2) exhibit opposite PL modulation. Distinct oppositely charged domains were obtained with a 9-fold PL enhancement for the same single MoSe2 sheet when adhered to the positive (P+) and negative (P-) domains. This sharp PL modulation on the ferroelectric domain results from different free electron or hole concentrations in the materials conduction band or valence band. Moreover, excitons dissociate rapidly at the interface between the P+ and P- domains due to the built-in electric field. We are able to adjust the charge on the P+ and P- domains using temperature via the pyroelectric effect and observe rapid PL quenching over a narrow temperature range illustrating the observed PL modulation is electronic in nature. This observation creates an opportunity to harness the direct bandgap TMD 2D materials as an active optical component for the lithium niobate platform using domain engineering of the lithium niobate substrate to create optically active heterostructures that could be used for photodetectors or even electrically driven optical sources on-chip.

cond-mat.mtrl-sci↗

arXiv2017

Retention Time of Peptides in Liquid Chromatography Is Well Estimated upon Deep Transfer Learning

A fully automatic prediction for peptide retention time (RT) in liquid chromatography (LC), termed as DeepRT, was developed using deep learning approach, an ensemble of Residual Network (ResNet) and Long Short-Term Memory (LSTM). In contrast to the traditional predictor based on the hand-crafted features for peptides, DeepRT learns features from raw amino acid sequences and makes relatively accurate prediction of peptide RTs with 0.987 R2 for unmodified peptides. Furthermore, by virtue of transfer learning, DeepRT enables utilization of the peptides datasets generated from different LC conditions and of different modification status, resulting in the RT prediction of 0.992 R2 for unmodified peptides and 0.978 R2 for post-translationally modified peptides. Even though chromatographic behaviors of peptides are quite complicated, the study here demonstrated that peptide RT prediction could be largely improved by deep transfer learning. The DeepRT software is freely available at https://github.com/horsepurve/DeepRT, under Apache2 open source License.

q-bio.QM↗

arXiv2017

A proximal difference-of-convex algorithm with extrapolation

We consider a class of difference-of-convex (DC) optimization problems whose objective is level-bounded and is the sum of a smooth convex function with Lipschitz gradient, a proper closed convex function and a continuous concave function. While this kind of problems can be solved by the classical difference-of-convex algorithm (DCA) [26], the difficulty of the subproblems of this algorithm depends heavily on the choice of DC decomposition. Simpler subproblems can be obtained by using a specific DC decomposition described in [27]. This decomposition has been proposed in numerous work such as [18], and we refer to the resulting DCA as the proximal DCA. Although the subproblems are simpler, the proximal DCA is the same as the proximal gradient algorithm when the concave part of the objective is void, and hence is potentially slow in practice. In this paper, motivated by the extrapolation techniques for accelerating the proximal gradient algorithm in the convex settings, we consider a proximal difference-of-convex algorithm with extrapolation to possibly accelerate the proximal DCA. We show that any cluster point of the sequence generated by our algorithm is a stationary point of the DC optimization problem for a fairly general choice of extrapolation parameters: in particular, the parameters can be chosen as in FISTA with fixed restart [15]. In addition, by assuming the Kurdyka-Łojasiewicz property of the objective and the differentiability of the concave part, we establish global convergence of the sequence generated by our algorithm and analyze its convergence rate. Our numerical experiments on two difference-of-convex regularized least squares models show that our algorithm usually outperforms the proximal DCA and the general iterative shrinkage and thresholding algorithm proposed in [17].

math.OC↗

arXiv2017

DeepRT: deep learning for peptide retention time prediction in proteomics

Accurate predictions of peptide retention times (RT) in liquid chromatography have many applications in mass spectrometry-based proteomics. Herein, we present DeepRT, a deep learning based software for peptide retention time prediction. DeepRT automatically learns features directly from the peptide sequences using the deep convolutional Neural Network (CNN) and Recurrent Neural Network (RNN) model, which eliminates the need to use hand-crafted features or rules. After the feature learning, principal component analysis (PCA) was used for dimensionality reduction, then three conventional machine learning methods were utilized to perform modeling. Two published datasets were used to evaluate the performance of DeepRT and we demonstrate that DeepRT greatly outperforms previous state-of-the-art approaches ELUDE and GPTime.

q-bio.QM↗

arXiv2016

Unconventional Correlation between Quantum Hall Transport Quantization and Bulk State Filling in Gated Graphene Devices

We report simultaneous transport and scanning microwave impedance microscopy to examine the correlation between transport quantization and filling of the bulk Landau levels in the quantum Hall regime in gated graphene devices. Surprisingly, a comparison of these measurements reveals that quantized transport typically occurs below the complete filling of bulk Landau levels, when the bulk is still conductive. This result points to a revised understanding of transport quantization when carriers are accumulated by gating. We discuss the implications on transport study of the quantum Hall effect in graphene and related topological states in other two-dimensional electron systems.

cond-mat.mes-hall↗

arXiv2016

Linear Convergence of Proximal Gradient Algorithm with Extrapolation for a Class of Nonconvex Nonsmooth Minimization Problems

In this paper, we study the proximal gradient algorithm with extrapolation for minimizing the sum of a Lipschitz differentiable function and a proper closed convex function. Under the error bound condition used in [19] for analyzing the convergence of the proximal gradient algorithm, we show that there exists a threshold such that if the extrapolation coefficients are chosen below this threshold, then the sequence generated converges $R$-linearly to a stationary point of the problem. Moreover, the corresponding sequence of objective values is also $R$-linearly convergent. In addition, the threshold reduces to $1$ for convex problems and, as a consequence, we obtain the $R$-linear convergence of the sequence generated by FISTA with fixed restart. Finally, we present some numerical experiments to illustrate our results.

math.OC↗

arXiv2016

Electron optics with ballistic graphene junctions

Electrons transmitted across a ballistic semiconductor junction undergo refraction, analogous to light rays across an optical boundary. A pn junction theoretically provides the equivalent of a negative index medium, enabling novel electron optics such as negative refraction and perfect (Veselago) lensing. In graphene, the linear dispersion and zero-gap bandstructure admit highly transparent pn junctions by simple electrostatic gating, which cannot be achieved in conventional semiconductors. Moreover ballistic transport over micron length scales at ambient temperature has been realized, providing an ideal platform to realize a new generation of device based on electron lensing. Robust demonstration of these effects, however, has not been forthcoming. Here we employ transverse magnetic focusing to probe propagation across an electrostatically defined graphene junction. We find perfect agreement with the predicted Snells law for electrons, including observation of both positive and negative refraction. Resonant transmission across the pn junction provides a direct measurement of the angle dependent transmission coefficient, and we demonstrate good agreement with theory. Comparing experimental data with simulation reveals the crucial role played by the effective junction width, providing guidance for future device design. Our results pave the way for realizing novel electron optics based on graphene pn junctions.

cond-mat.mes-hall↗

arXiv2015

Fractional fractal quantum Hall effect in graphene superlattices

The Hofstadter energy spectrum provides a uniquely tunable system to study emergent topological order in the regime of strong interactions. Previous experiments, however, have been limited to the trivial case of low Bloch band filling where only the Landau level index plays a significant role. Here we report measurement of high mobility graphene superlattices where the complete unit cell of the Hofstadter spectrum is accessible. We observe coexistence of conventional fractional quantum Hall effect (QHE) states together with the integer QHE states associated with the fractal Hofstadter spectrum. At large magnetic field, a new series of states appear at fractional Bloch filling index. These fractional Bloch band QHE states are not anticipated by existing theoretical pictures and point towards a new type of many-body state.

cond-mat.mes-hall↗

arXiv2014

Determination of Intrinsic Magnetic Response from Local Measurements of Fringing Fields

Micron-sized Hall bars and micro-SQUIDs are now used routinely to measure the local static and dynamic magnetic response with micron-scale spatial resolution. While this provides a powerful new tool, determining the intrinsic magnetization presents new challenges, as it requires correcting for demagnetization fields that vary widely with position on a sample. In this paper we develop a method to correct for the demagnetization effect at local points of a rectangular prism shaped sample using a finite element analysis of Maxwell's equation applied to local Hall sensor measurements calibrated by bulk measurements of the magnetization. This method can be generalized to other geometric shapes to analyze data obtained with local magnetic probes.

cond-mat.mes-hall↗

arXiv2013

Quantum Fluctuations and Long-Range Order in Molecular Magnets

We review our studies of the effect of transverse fields on the susceptibility and magnetization of single crystals of the prototype single molecule magnet (SMM), Mn$_{12}$-acetate, and of a new high-symmetry variant, Mn$_{12}$-acetate-MeOH. SMM single crystals can exhibit long range ferromagnetic order associated with intermolecular dipole interactions. Transverse fields increase quantum spin fluctuation and quantum tunneling of the magnetization suppressing long range order. However, we have found that suppression of the Curie temperature by a transverse field in Mn$_{12}$-acetate is far more rapid than predicted by the Transverse-Field Ising Ferromagnetic Model (TFIFM). It appears that solvent disorder in Mn$_{12}$-acetate that results in an intrinsic distribution of small discrete tilts of the molecular magnetic easy axis from the global easy axis of the crystal ($\approx \pm 1^\circ$) gives rise to a distribution of random-fields that further suppresses long-range order. Semiquantitative agreement with the predictions of a Random-Field Ising Ferromagnet Model is found. Subsequent susceptibility studies we have conducted of the high symmetry Mn$_{12}$ variant, Mn$_{12}$-acetate-MeOH, with the same spin structure and similar lattice constants but without the same solvent disorder as Mn$_{12}$-acetate, agrees with the TFIFM. An important implication of our studies is that long-range order in these two chemically very similar SMMs are described by distinct physical models.

cond-mat.mes-hall↗

arXiv2012

Critical Behavior of a Strongly Interacting 2D Electron System

With decreasing density $n_s$ the thermopower $S$ of a low-disorder 2D electron system in silicon is found to exhibit a sharp increase by more than an order of magnitude, tending to a divergence at a finite, disorder-independent density $n_t$ consistent with the critical form $(-T/S) \propto (n_s-n_t)^x$ with $x=1.0\pm 0.1$ ($T$ is the temperature). Our results provide clear evidence for an interaction-induced transition to a new phase at low density in a strongly-interacting 2D electron system.

cond-mat.str-el↗