arXiv subjects

Junfeng Liu

Publications and source records attributed to Junfeng Liu.

49 records · Page 3Linked to original sources

arXiv2020

Elastic Bulk Synchronous Parallel Model for Distributed Deep Learning

The bulk synchronous parallel (BSP) is a celebrated synchronization model for general-purpose parallel computing that has successfully been employed for distributed training of machine learning models. A prevalent shortcoming of the BSP is that it requires workers to wait for the straggler at every iteration. To ameliorate this shortcoming of classic BSP, we propose ELASTICBSP a model that aims to relax its strict synchronization requirement. The proposed model offers more flexibility and adaptability during the training phase, without sacrificing on the accuracy of the trained model. We also propose an efficient method that materializes the model, named ZIPLINE. The algorithm is tunable and can effectively balance the trade-off between quality of convergence and iteration throughput, in order to accommodate different environments or applications. A thorough experimental evaluation demonstrates that our proposed ELASTICBSP model converges faster and to a higher accuracy than the classic BSP. It also achieves comparable (if not higher) accuracy than the other sensible synchronization models.

cs.LG↗

arXiv2019

Dynamic Graph Embedding via LSTM History Tracking

Many real world networks are very large and constantly change over time. These dynamic networks exist in various domains such as social networks, traffic networks and biological interactions. To handle large dynamic networks in downstream applications such as link prediction and anomaly detection, it is essential for such networks to be transferred into a low dimensional space. Recently, network embedding, a technique that converts a large graph into a low-dimensional representation, has become increasingly popular due to its strength in preserving the structure of a network. Efficient dynamic network embedding, however, has not yet been fully explored. In this paper, we present a dynamic network embedding method that integrates the history of nodes over time into the current state of nodes. The key contribution of our work is 1) generating dynamic network embedding by combining both dynamic and static node information 2) tracking history of neighbors of nodes using LSTM 3) significantly decreasing the time and memory by training an autoencoder LSTM model using temporal walks rather than adjacency matrices of graphs which are the common practice. We evaluate our method in multiple applications such as anomaly detection, link prediction and node classification in datasets from various domains.

cs.LG↗

arXiv2019

Dynamic Stale Synchronous Parallel Distributed Training for Deep Learning

Deep learning is a popular machine learning technique and has been applied to many real-world problems. However, training a deep neural network is very time-consuming, especially on big data. It has become difficult for a single machine to train a large model over large datasets. A popular solution is to distribute and parallelize the training process across multiple machines using the parameter server framework. In this paper, we present a distributed paradigm on the parameter server framework called Dynamic Stale Synchronous Parallel (DSSP) which improves the state-of-the-art Stale Synchronous Parallel (SSP) paradigm by dynamically determining the staleness threshold at the run time. Conventionally to run distributed training in SSP, the user needs to specify a particular staleness threshold as a hyper-parameter. However, a user does not usually know how to set the threshold and thus often finds a threshold value through trial and error, which is time-consuming. Based on workers' recent processing time, our approach DSSP adaptively adjusts the threshold per iteration at running time to reduce the waiting time of faster workers for synchronization of the globally shared parameters, and consequently increases the frequency of parameters updates (increases iteration throughput), which speedups the convergence rate. We compare DSSP with other paradigms such as Bulk Synchronous Parallel (BSP), Asynchronous Parallel (ASP), and SSP by running deep neural networks (DNN) models over GPU clusters in both homogeneous and heterogeneous environments. The results show that in a heterogeneous environment where the cluster consists of mixed models of GPUs, DSSP converges to a higher accuracy much earlier than SSP and BSP and performs similarly to ASP. In a homogeneous distributed cluster, DSSP has more stable and slightly better performance than SSP and ASP, and converges much faster than BSP.

cs.DC↗

arXiv2019

On the structure of invariant subspaces for the shift operator on Bergman spaces

It is well known that the structure of nontrival invariant subspaces for the shift operator on the Bergman space is extremely complicated and very little is known about their specific structures, and that a complete description of the structure seems unlikely. In this paper, we find that any invariant subspace $M(\neq \{0\})$ for the shift operator $M_z$ on the Bergman space $A^{^p}_α(\D)~(1\leq p< \infty,~~-1<α<\infty)$ contains a nonempty subset that lies in $A^{^p}_α(\D)\backslash H^{^p}(\D)$. To a certain extent, this result describes the specific structure of every nonzero invariant subspace for the shift operator $M_z$ on $A^{^p}_α(\D)$.

math.FA↗

arXiv2019

On Invariant Subspaces of Subdecomposable Operators

In this paper, we prove the Mohebi-Radjabalipour Conjecture under a little additional condition, and obtain a new invariant subspace theorem for subdecomposable operators. Our main results contain known results in this topic as special cases. Moreover, the condition in our theorem is really much weaker, much simpler and much more nature than that in the known results.

math.FA↗

arXiv2018

A Filter of Minhash for Image Similarity Measures

Image similarity measures play an important role in nearest neighbor search and duplicate detection for large-scale image datasets. Recently, Minwise Hashing (or Minhash) and its related hashing algorithms have achieved great performances in large-scale image retrieval systems. However, there are a large number of comparisons for image pairs in these applications, which may spend a lot of computation time and affect the performance. In order to quickly obtain the pairwise images that theirs similarities are higher than the specific threshold T (e.g., 0.5), we propose a dynamic threshold filter of Minwise Hashing for image similarity measures. It greatly reduces the calculation time by terminating the unnecessary comparisons in advance. We also find that the filter can be extended to other hashing algorithms, on when the estimator satisfies the binomial distribution, such as b-Bit Minwise Hashing, One Permutation Hashing, etc. In this pager, we use the Bag-of-Visual-Words (BoVW) model based on the Scale Invariant Feature Transform (SIFT) to represent the image features. We have proved that the filter is correct and effective through the experiment on real image datasets.

cs.MM↗

arXiv2018

Drug Selection via Joint Push and Learning to Rank

Selecting the right drugs for the right patients is a primary goal of precision medicine. In this manuscript, we consider the problem of cancer drug selection in a learning-to-rank framework. We have formulated the cancer drug selection problem as to accurately predicting 1). the ranking positions of sensitive drugs and 2). the ranking orders among sensitive drugs in cancer cell lines based on their responses to cancer drugs. We have developed a new learning-to-rank method, denoted as pLETORg , that predicts drug ranking structures in each cell line via using drug latent vectors and cell line latent vectors. The pLETORg method learns such latent vectors through explicitly enforcing that, in the drug ranking list of each cell line, the sensitive drugs are pushed above insensitive drugs, and meanwhile the ranking orders among sensitive drugs are correct. Genomics information on cell lines is leveraged in learning the latent vectors. Our experimental results on a benchmark cell line-drug response dataset demonstrate that the new pLETORg significantly outperforms the state-of-the-art method in prioritizing new sensitive drugs.

cs.LG↗

arXiv2016

A Family Group of Two-dimensional Node-Line Semimetals

Using evolutionary algorithm and first-principles calculations, we predict a family group of two-dimensional node-line semimetals MX (M=Pd, Pt; X=S, Se, Te), which has zig-zag type mono-layer structure in Pmm2 layer group. Band structure analysis reveals that node-line features are caused by band inversion and the inversion exists even in the absence of spin-orbital-coupling. Tests are carried out to confirm that the node-line loop is protected by crystal symmetry. This work extends our knowledge of node-line materials to two-dimensional cases, i.e., a group of metal-group VI compounds sharing the same lattice structure which has time reversion and crystal-mirror inversion symmetries.

cond-mat.mes-hall↗

arXiv2016

Coherent and incoherent nonparaxial self-accelerating Weber beams

We investigate the coherent and incoherent nonparaxial Weber beams, theoretically and numerically. We show that the superposition of coherent self-accelerating Weber beams with transverse displacement cannot display the nonparaxial accelerating Talbot effect. The reason is that their lobes do not accelerate in unison, which is a requirement for the appearance of the effect. While for the incoherent Weber beams, they naturally cannot display the accelerating Talbot effect but can display the nonparaxial accelerating properties, although the transverse coherence length is smaller than the beam width, based on the second-order coherence theory. Our research method directly applies to the nonparaxial Mathieu beams as well, and one will obtain similar conclusions as for the Weber beams, although this is not discussed in the paper. Our investigation identifies families of nonparaxial accelerating beams that do not exhibit the accelerating Talbot effect, and in addition broadens the understanding of coherence properties of such nonparaxial accelerating beams.

physics.optics↗

arXiv2012

The Bouleau-Yor identity for a bi-fractional Brownian motion

Let $B$ be a bi-fractional Brownian motion with indices $H\in (0,1),K\in (0,1]$, $2HK=1$ and let ${\mathscr L}(x,t)$ be its local time process. We construct a Banach space ${\mathscr H}$ of measurable functions such that the quadratic covariation $[f(B),B]$ and the integral $\int_{\mathbb R}f(x){\mathscr L}(dx,t)$ exist provided $f\in {\mathscr H}$. Moreover, the Bouleau-Yor identity $$ [f(B),B]_t=-2^{1-K}\int_{\mathbb R}f(x){\mathscr L}(dx,t),\qquad t\geq 0, $$ holds for all $f\in {\mathscr H}$.

math.PR↗

arXiv2011

The generalized quadratic covariation for fractional Brownian motion with Hurst index less than 1/2

Let $B^H$ be a fractional Brownian motion with Hurst index $0<H<1/2$. In this paper we study the {\it generalized quadratic covariation} $[f(B^H),B^H]^{(W)}$ defined by $$ [f(B^H),B^H]^{(W)}_t=\lim_{ε\downarrow 0}\frac{2H}{ε^{2H}}\int_0^t\{f(B^{H}_{s+ε})-f(B^{H}_s)\}(B^{H}_{s+ε}- B^{H}_s)s^{2H-1}ds, $$ where the limit is uniform in probability and $x\mapsto f(x)$ is a deterministic function. We construct a Banach space ${\mathscr H}$ of measurable functions such that the generalized quadratic covariation exists in $L^2$ and the Bouleau-Yor identity takes the form $$ [f(B^H),B^H]_t^{(W)}=-\int_{\mathbb {R}}f(x){\mathscr L}^{H}(dx,t) $$ provided $f\in {\mathscr H}$, where ${\mathscr L}^{H}(x,t)$ is the weighted local time of $B^H$. This allows us to write the fractional Itô formula for absolutely continuous functions with derivative belonging to ${\mathscr H}$. These are also extended to the time-dependent case.

math.PR↗

arXiv2009

Sign reversal of Hall conductivity and quantum confinement in graphene ribbons

Characterized by zigzag and armchair boundaries, the narrow ribbons display the very different characteristics in Hall conductivities. It is shown that the multi-band-crossings occur in the energy spectrum for armchair ribbons, and the number of them depends on the width of ribbons. Theoretically, it is predicated that the conductivities exhibit drastic sign reversals for narrow ribbons as the Fermi energy sweep over the band-crossings. A new classification of armchair ribbons is suggested based on the emergence of a flat band in the energy spectrum only for odd armchair ribbons. The evolution of jumped Hall conductivities to step-like plateaus and the restore of density of states at the van Hove singularity in the limitation to graphene sheet have been analyzed.

cond-mat.mes-hall↗

arXiv2008

Integration with respect to fractional local times with Hurst index $H$ greater than 1/2

Let ${\mathscr L}^H(x,t)=2H\int_0^tδ(B^H_s-x)s^{2H-1}ds$ be the weighted local time of fractional Brownian motion $B^H$ with Hurst index $1/2<H<1$. In this paper, we use Young integration to study the integral of determinate functions $\int_{\mathbb R}f(x){\mathscr L}^H(dx,t)$. As an application, we investigate the {\it weighted quadratic covariation} $[f(B^H),B^H]^{(W)}$ defined by $$ [f(B^H),B^H]^{(W)}_t:=\lim_{n\to \infty}2H\sum_{k=0}^{n-1} k^{2H-1}\{f(B^H_{t_{k+1}})-f(B^H_{t_{k}})\}(B^H_{t_{k+1}}-B^H_{t_{k}}), $$ where the limit is uniform in probability and $t_k=kt/n$. We show that it exists and $$ [f(B^H),B^H]^{(W)}_t=-\int_{\mathbb R}f(x){\mathscr L}^H(dx,t), $$ provided $f$ is of bounded $p$-variation with $1\leq p<\frac{2H}{1-H}$. Moreover, we extend this result to the time-dependent case. These allow us to write the fractional Itô formula for new classes of functions.

math.PR↗