arXiv subjects

Wenqi Jiang

Publications and source records attributed to Wenqi Jiang.

23 records · Page 2Linked to original sources

arXiv2021

Dynamic Sampling and Selective Masking for Communication-Efficient Federated Learning

Federated learning (FL) is a novel machine learning setting that enables on-device intelligence via decentralized training and federated optimization. Deep neural networks' rapid development facilitates the learning techniques for modeling complex problems and emerges into federated deep learning under the federated setting. However, the tremendous amount of model parameters burdens the communication network with a high load of transportation. This paper introduces two approaches for improving communication efficiency by dynamic sampling and top-$k$ selective masking. The former controls the fraction of selected client models dynamically, while the latter selects parameters with top-$k$ largest values of difference for federated updating. Experiments on convolutional image classification and recurrent language modeling are conducted on three public datasets to show our proposed methods' effectiveness.

cs.LG↗

arXiv2021

Geometrical and electrical modulation on the transport property of silicene constrictions

We study the electrical modulation of the transport properties of silicene constrictions with different geometrical structures by adopting the tight-binding model and non-equilibrium Green's function method. The band structure and transmission properties are discussed under the influence of the external electric field and potential energy. Especially, we investigate the effects of the position and width of the central scattering region on the conductance with increasing of Fermi energy. We find that the conductance significantly depends on the position and the width. Interestingly, the symmetrical structure of the central region can induce a resonance effect and significantly enlarge the system's conductance. Obviously, we obtain an effective method to adjust the transport property of the silicene heterojunctions. Correspondingly, we propose a novel two-channel structure with an excellent performance on the conductance compared to the one-channel structure with the same total width.

cond-mat.mes-hall↗

arXiv2021

Switch Spaces: Learning Product Spaces with Sparse Gating

Learning embedding spaces of suitable geometry is critical for representation learning. In order for learned representations to be effective and efficient, it is ideal that the geometric inductive bias aligns well with the underlying structure of the data. In this paper, we propose Switch Spaces, a data-driven approach for learning representations in product space. Specifically, product spaces (or manifolds) are spaces of mixed curvature, i.e., a combination of multiple euclidean and non-euclidean (hyperbolic, spherical) manifolds. To this end, we introduce sparse gating mechanisms that learn to choose, combine and switch spaces, allowing them to be switchable depending on the input data with specialization. Additionally, the proposed method is also efficient and has a constant computational complexity regardless of the model size. Experiments on knowledge graph completion and item recommendations show that the proposed switch space achieves new state-of-the-art performances, outperforming pure product spaces and recently proposed task-specific models.

cs.LG↗

arXiv2021

MicroRec: Efficient Recommendation Inference by Hardware and Data Structure Solutions

Deep neural networks are widely used in personalized recommendation systems. Unlike regular DNN inference workloads, recommendation inference is memory-bound due to the many random memory accesses needed to lookup the embedding tables. The inference is also heavily constrained in terms of latency because producing a recommendation for a user must be done in about tens of milliseconds. In this paper, we propose MicroRec, a high-performance inference engine for recommendation systems. MicroRec accelerates recommendation inference by (1) redesigning the data structures involved in the embeddings to reduce the number of lookups needed and (2) taking advantage of the availability of High-Bandwidth Memory (HBM) in FPGA accelerators to tackle the latency by enabling parallel lookups. We have implemented the resulting design on an FPGA board including the embedding lookup step as well as the complete inference process. Compared to the optimized CPU baseline (16 vCPU, AVX2-enabled), MicroRec achieves 13.8~14.7x speedup on embedding lookup alone and 2.5$~5.4x speedup for the entire recommendation inference in terms of throughput. As for latency, CPU-based engines needs milliseconds for inferring a recommendation while MicroRec only takes microseconds, a significant advantage in real-time recommendation systems.

cs.AR↗

arXiv2012

Common dependence on earthquake magnitudes for the trapped particles bursts approaching the earthquake

Trapped particles bursts have long been observed to be frequently occurred several hours before earthquakes, especially for strong earthquakes, from several space experiments during past decades. However, the validity of earthquake origin of particles bursts events is still unsolved. In this paper, we firstly reported the frequency distribution and time evolution of particles bursts within different time windows centered around earthquakes for various magnitudes. The results showed nearly the same systematic dependence of particle bursts frequency on earthquake magnitude and characteristic time decay behavior of average number of particles bursts for various magnitudes. These findings should strengthen the validity of earthquake origin of particles bursts and further understanding of particles bursts as possible precursor of earthquake.

physics.space-ph↗