Searcharxiv⌕ Search

arXiv subjects

Jiayu Li

Publications and source records attributed to Jiayu Li.

At least 127 records · Page 7Linked to original sources

Rank Position Forecasting in Car Racing

Forecasting is challenging since uncertainty resulted from exogenous factors exists. This work investigates the rank position forecasting problem in car racing, which predicts the rank positions at the future laps for cars. Among the many factors that bring changes to the rank positions, pit stops are critical but irregular and rare. We found existing methods, including statistical models, machine learning regression models, and state-of-the-art deep forecasting model based on encoder-decoder architecture, all have limitations in the forecasting. By elaborative analysis of pit stops events, we propose a deep model, RankNet, with the cause effects decomposition that modeling the rank position sequence and pit stop events separately. It also incorporates probabilistic forecasting to model the uncertainty inside each sub-model. Through extensive experiments, RankNet demonstrates a strong performance improvement over the baselines, e.g., MAE improves more than 10% consistently, and is also more stable when adapting to unseen new data. Details of model optimization, performance profiling are presented. It is promising to provide useful forecasting tools for the car racing analysis and shine a light on solutions to similar challenging issues in general forecasting problems.

cs.LG↗

SubGraph2Vec: Highly-Vectorized Tree-likeSubgraph Counting

Subgraph counting aims to count occurrences of a template T in a given network G(V, E). It is a powerful graph analysis tool and has found real-world applications in diverse domains. Scaling subgraph counting problems is known to be memory bounded and computationally challenging with exponential complexity. Although scalable parallel algorithms are known for several graph problems such as Triangle Counting and PageRank, this is not common for counting complex subgraphs. Here we address this challenge and study connected acyclic graphs or trees. We propose a novel vectorized subgraph counting algorithm, named Subgraph2Vec, as well as both shared memory and distributed implementations: 1) reducing algorithmic complexity by minimizing neighbor traversal; 2) achieving a highly-vectorized implementation upon linear algebra kernels to significantly improve performance and hardware utilization. 3) Subgraph2Vec improves the overall performance over the state-of-the-art work by orders of magnitude and up to 660x on a single node. 4) Subgraph2Vec in distributed mode can scale up the template size to 20 and maintain good strong scalability. 5) enabling portability to both CPU and GPU.

cs.DC↗

Half-Magnetic Topological Insulator

Topological magnets are a new family of quantum materials providing great potential to realize emergent phenomena, such as quantum anomalous Hall effect and axion-insulator state. Here we present our discovery that stoichiometric ferromagnet MnBi8Te13 with natural heterostructure MnBi2Te4-(Bi2Te3)3 is an unprecedented half-magnetic topological insulator, with the magnetization existing at the MnBi2Te4 surface but not at the opposite surface terminated by triple Bi2Te3 layers. Our angle-resolved photoemission spectroscopy measurements unveil a massive Dirac gap at the MnBi2Te4 surface, and gapless Dirac cone on the other side. Remarkably, the Dirac gap (~28 meV) at MnBi2Te4 surface decreases monotonically with increasing temperature and closes right at the Curie temperature, thereby representing the first smoking-gun spectroscopic evidence of magnetization-induced topological surface gap among all known magnetic topological materials. We further demonstrate theoretically that the half-magnetic topological insulator is desirable to realize the half-quantized surface anomalous Hall effect, which serves as a direct proof of the general concept of axion electrodynamics in condensed matter systems.

cond-mat.mtrl-sci↗

Secure Multigroup Multicast Communication Systems via Intelligent Reflecting Surface

This paper considers a secure multigroup multicast multiple-input single-output (MISO) communication system aided by an intelligent reflecting surface (IRS). Specifically, we aim to minimize the transmit power at the Alice via jointly optimizing the transmit beamformer, AN vector and phase shifts at the IRS subject to the secrecy rate constraints as well as the unit modulus constraints of IRS phase shifts. However, the optimization problem is non-convex and directly solving it is intractable. To tackle the optimization problem, we first transform it into a semidefinite relaxation (SDR) problem, and then alternately update the transmit beamformer and AN matrix as well as the phase shifts at the IRS. In order to reduce the high computational complexity, we further propose a low-complexity algorithm based on second-order cone programming (SOCP). We decouple the optimization problem into two sub-problems and optimize the transmit beamformer, AN vector and the phase shifts alternately by solving two corresponding SOCP sub-problem. Simulation results show that the proposed SDR and SOCP schemes require half or less transmit power than the scheme without IRS, which demonstrates the advantages of introducing IRS and the effectiveness of the proposed methods.

eess.SP↗

Enhanced Secrecy Rate Maximization for Directional Modulation Networks via IRS

Intelligent reflecting surface (IRS) is of low-cost and energy-efficiency and will be a promising technology for the future wireless communications like sixth generation. To address the problem of conventional directional modulation (DM) that Alice only transmits single confidential bit stream (CBS) to Bob with multiple antennas in a line-of-sight channel, IRS is proposed to create friendly multipaths for DM such that two CBSs can be transmitted from Alice to Bob. This will significantly enhance the secrecy rate (SR) of DM. To maximize the SR (Max-SR), a general non-convex optimization problem is formulated with the unit-modulus constraint of IRS phase-shift matrix (PSM), and the general alternating iterative (GAI) algorithm is proposed to jointly obtain the transmit beamforming vectors (TBVs) and PSM by alternately optimizing one and fixing another. To reduce its high complexity, a low-complexity iterative algorithm for Max-SR is proposed by placing the constraint of null-space (NS) on the TBVs, called NS projection (NSP). Here, each CBS is transmitted separately in the NSs of other CBS and AN channels. Simulation results show that the SRs of the proposed GAI and NSP can approximately double that of IRS-based DM with single CBS for massive IRS in the high signal-to-noise ratio region.

cs.IT↗

Spectroscopic realization of large surface gap in a doped magnetic topological insulator

Realization of the quantum anomalous Hall effect and axion electrodynamics in topological materials are among the paradigmatic phenomena in condensed matter physics. Recently, signatures of both phases are observed to exist in thin films of MnBi$_2$Te$_4$, a stoichiometric antiferromagnetic topological insulator. Direct evidence of the bulk topological magnetoelectric response in an axion insulator requires an energy gap at its topological surface state (TSS). However, independent spectroscopic experiments revealed that such a surface gap is absent, or much smaller than previously thought, in MnBi$_2$Te$_4$. Here, we utilize angle resolved photoemission spectroscopy and density functional theory calculations to demonstrate that a sizable TSS gap unexpectedly exists in Sb-doped MnBi$_2$Te$_4$. This gap is found to be topologically nontrivial, insensitive to the bulk antiferromagnetic-paramagnetic transition, while enlarges along with increasing Sb concentration. Our work shows that Mn(Bi$_{1-x}$Sb$_x$)$_2$Te$_4$ is a potential platform to observe the key features of the high-temperature axion insulator state, such as the topological magnetoelectric responses and half-integer quantum Hall effects.

cond-mat.mtrl-sci↗

Realization and transport investigation of a single layer-twisted bilayer graphene junction

We report on low-temperature transport study of a single layer graphene (SLG)-twisted bilayer graphene (tBLG) junction device. The SLG-tBLG junction in the device is grown by chemical vapor deposition and the device is fabricated in a Hall-bar configuration on Si/SiO$_2$ substrate. The longitudinal resistances across the SLG-tBLG junction (cross-junction resistances) on the two sides of the Hall bar and the Hall resistances of SLG and tBLG in the device are measured. In the quantum Hall regime, the measurements show that the measured cross-junction resistances exhibit a series of new quantized plateaus and the appearance of these resistance plateaus can be attributed to the presence of the well-defined edge-channel transport along the SLG-tBLG junction interface. The measurements also show that the difference between the cross-junction resistances measured on the two sides of the Hall-bar provides a sensitive measure to the edge channel transport characteristics in the two graphene layers that constitute the SLG-tBLG junction and to degeneracy lifting of the Landau levels in the tBLG layer. Temperature dependent measurements of the cross-junction resistance in the quantum Hall regime are also carried out and the influence of the transverse transport of the bulk Landau levels on the edge channel transport along the SLG-tBLG junction interface are extracted. These results enrich the understanding of the charge transport across interfaces in graphene hybrid structures and open up new opportunities for probing exotic quantum phenomena in graphene devices.

cond-mat.mes-hall↗

Distinct Topological Surface States on the Two Terminations of MnBi$_4$Te$_7$

The recent discovered intrinsic magnetic topological insulator MnBi2Te4 have been met with unusual success in hosting emergent phenomena such as the quantum anomalous Hall effect and the axion insulator states. However, the surface-bulk correspondence of the Mn-Bi-Te family, composed by the superlattice-like MnBi2Te4/(Bi2Te3)n (n = 0, 1, 2, 3 ...) layered structure, remains intriguing but elusive. Here, by using scanning tunneling microscopy (STM) and angle-resolved photoemission spectroscopy (ARPES) techniques, we unambiguously assign the two distinct surface states of MnBi4Te7 (n = 1) to the quintuple-layer (QL) Bi2Te3 termination and the septuple-layer (SL) MnBi2Te4 termination, respectively. A comparison of the experimental observations with theoretical calculations reveals the diverging topological behaviors, especially the hybridization effect between magnetic and nonmagnetic layers, on the two terminations: a gap on the QL termination originating from the topological surface states of the QL hybridizing with the bands of the beneath SL, and a gapless Dirac-cone band structure on the SL termination with time-reversal symmetry. The quasi-particle interference patterns further confirm the topological nature of the surface states for both terminations, continuing far above the Fermi energy. The QL termination carries a spin-helical Dirac state with hexagonal warping, while at the SL termination, a strongly canted helical state from the surface lies between a pair of Rashba-split states from its neighboring layer. Our work elucidates an unprecedented hybridization effect between the building blocks of the topological surface states, and also reveals the termination-dependent time-reversal symmetry breaking in a magnetic topological insulator, rendering an ideal platform to realize the half-integer quantum Hall effect and relevant quantum phenomena.

cond-mat.mtrl-sci↗

Energy-efficient Alternating Iterative Secure Structure of Maximizing Secrecy Rate for Directional Modulation Networks

In a directional modulation (DM) network, the issues of security and privacy have taken on an increasingly important role. Since the power allocation of confidential message and artificial noise will make a constructive effect on the system performance, it is important to jointly consider the relationship between the beamforming vectors and the power allocation (PA) factors. To maximize the secrecy rate (SR), an alternating iterative structure (AIS) between the beamforming and PA is proposed. With only two or three iterations, it can rapidly converge to its rate ceil. Simulation results indicate that the SR performance of proposed AIS is much better than the null-space projection (NSP) based PA strategy in the medium and large signal-to-noise ratio (SNR) regions, especially when the number of antennas at the DM transmitter is small.

cs.IT↗

Performance Analysis of Directional Modulation with Finite-quantized RF Phase Shifters in Analog Beamforming Structure

The radio frequency (RF) phase shifter with finite quantization bits in analog beamforming (AB) structure forms quantization error (QE) and causes a performance loss of received signal to interference plus noise ratio (SINR) at the receiver (called Bob). By using the law of large numbers in probability theory, the closed-form expression of SINR performance loss is derived to be inversely proportional to the square of sinc (or sin(x)/x) function. Here, a phase alignment method is applied in directional modulation transmitter with AB structure. Also, the secrecy rate (SR) expression is derived with QE. From numerical simulation results, we find that the SINR performance loss gradually decreases as the number L of quantization bits increases. This loss is less than 0.3dB when L is larger than or equal to 3. As L exceeds 5, the SINR performance loss at Bob can be approximately trivial. Similarly, SR performance loss gradually reduces as L increases. In particular, the SR performance loss is about 0.1 bits/s/Hz for L=3 at signal-to-noise ratio of 15dB.

cs.IT↗

A GraphBLAS Approach for Subgraph Counting

Subgraph counting aims to count the occurrences of a subgraph template T in a given network G. The basic problem of computing structural properties such as counting triangles and other subgraphs has found applications in diverse domains. Recent biological, social, cybersecurity and sensor network applications have motivated solving such problems on massive networks with billions of vertices. The larger subgraph problem is known to be memory bounded and computationally challenging to scale; the complexity grows both as a function of T and G. In this paper, we study the non-induced tree subgraph counting problem, propose a novel layered softwarehardware co-design approach, and implement a shared-memory multi-threaded algorithm: 1) reducing the complexity of the parallel color-coding algorithm by identifying and pruning redundant graph traversal; 2) achieving a fully-vectorized implementation upon linear algebra kernels inspired by GraphBLAS, which significantly improves cache usage and maximizes memory bandwidth utilization. Experiments show that our implementation improves the overall performance over the state-of-the-art work by orders of magnitude and up to 660x for subgraph templates with size over 12 on a dual-socket Intel(R) Xeon(R) Platinum 8160 server. We believe our approach using GraphBLAS with optimized sparse linear algebra can be applied to other massive subgraph counting problems and emerging high-memory bandwidth hardware architectures.

cs.DC↗

ADMM-NN: An Algorithm-Hardware Co-Design Framework of DNNs Using Alternating Direction Method of Multipliers

To facilitate efficient embedded and hardware implementations of deep neural networks (DNNs), two important categories of DNN model compression techniques: weight pruning and weight quantization are investigated. The former leverages the redundancy in the number of weights, whereas the latter leverages the redundancy in bit representation of weights. However, there lacks a systematic framework of joint weight pruning and quantization of DNNs, thereby limiting the available model compression ratio. Moreover, the computation reduction, energy efficiency improvement, and hardware performance overhead need to be accounted for besides simply model size reduction. To address these limitations, we present ADMM-NN, the first algorithm-hardware co-optimization framework of DNNs using Alternating Direction Method of Multipliers (ADMM), a powerful technique to deal with non-convex optimization problems with possibly combinatorial constraints. The first part of ADMM-NN is a systematic, joint framework of DNN weight pruning and quantization using ADMM. It can be understood as a smart regularization technique with regularization target dynamically updated in each ADMM iteration, thereby resulting in higher performance in model compression than prior work. The second part is hardware-aware DNN optimizations to facilitate hardware-level implementations. Without accuracy loss, we can achieve 85$\times$ and 24$\times$ pruning on LeNet-5 and AlexNet models, respectively, significantly higher than prior work. The improvement becomes more significant when focusing on computation reductions. Combining weight pruning and quantization, we achieve 1,910$\times$ and 231$\times$ reductions in overall model size on these two benchmarks, when focusing on data storage. Highly promising results are also observed on other representative DNNs such as VGGNet and ResNet-50.

cs.LG↗

A Unified Framework of DNN Weight Pruning and Weight Clustering/Quantization Using ADMM

Many model compression techniques of Deep Neural Networks (DNNs) have been investigated, including weight pruning, weight clustering and quantization, etc. Weight pruning leverages the redundancy in the number of weights in DNNs, while weight clustering/quantization leverages the redundancy in the number of bit representations of weights. They can be effectively combined in order to exploit the maximum degree of redundancy. However, there lacks a systematic investigation in literature towards this direction. In this paper, we fill this void and develop a unified, systematic framework of DNN weight pruning and clustering/quantization using Alternating Direction Method of Multipliers (ADMM), a powerful technique in optimization theory to deal with non-convex optimization problems. Both DNN weight pruning and clustering/quantization, as well as their combinations, can be solved in a unified manner. For further performance improvement in this framework, we adopt multiple techniques including iterative weight quantization and retraining, joint weight clustering training and centroid updating, weight clustering retraining, etc. The proposed framework achieves significant improvements both in individual weight pruning and clustering/quantization problems, as well as their combinations. For weight pruning alone, we achieve 167x weight reduction in LeNet-5, 24.7x in AlexNet, and 23.4x in VGGNet, without any accuracy loss. For the combination of DNN weight pruning and clustering/quantization, we achieve 1,910x and 210x storage reduction of weight data on LeNet-5 and AlexNet, respectively, without accuracy loss. Our codes and models are released at the link http://bit.ly/2D3F0np

cs.NE↗

Progressive Weight Pruning of Deep Neural Networks using ADMM

Deep neural networks (DNNs) although achieving human-level performance in many domains, have very large model size that hinders their broader applications on edge computing devices. Extensive research work have been conducted on DNN model compression or pruning. However, most of the previous work took heuristic approaches. This work proposes a progressive weight pruning approach based on ADMM (Alternating Direction Method of Multipliers), a powerful technique to deal with non-convex optimization problems with potentially combinatorial constraints. Motivated by dynamic programming, the proposed method reaches extremely high pruning rate by using partial prunings with moderate pruning rates. Therefore, it resolves the accuracy degradation and long convergence time problems when pursuing extremely high pruning ratios. It achieves up to 34 times pruning rate for ImageNet dataset and 167 times pruning rate for MNIST dataset, significantly higher than those reached by the literature work. Under the same number of epochs, the proposed method also achieves faster convergence and higher compression rates. The codes and pruned DNN models are released in the link bit.ly/2zxdlss

cs.LG↗

Some interior regularity estimates for solutions of complex Monge-Ampère equations on a ball

In this paper, we consider the Dirichlet problem of a complex Monge-Ampère equation on a ball in $\mathbb C^n$. With $\mathcal C^{1,α}$ (resp. $\mathcal C^{0,α}$) data, we prove an interior $\mathcal C^{1,α}$ (resp. $\mathcal C^{0,α}$) estimate for the solution. These estimates are generalized versions of the Bedford-Taylor interior $\mathcal C^{1,1}$ estimate.

math.DG↗

On the Universal Approximation Property and Equivalence of Stochastic Computing-based Neural Networks and Binary Neural Networks

Large-scale deep neural networks are both memory intensive and computation-intensive, thereby posing stringent requirements on the computing platforms. Hardware accelerations of deep neural networks have been extensively investigated in both industry and academia. Specific forms of binary neural networks (BNNs) and stochastic computing based neural networks (SCNNs) are particularly appealing to hardware implementations since they can be implemented almost entirely with binary operations. Despite the obvious advantages in hardware implementation, these approximate computing techniques are questioned by researchers in terms of accuracy and universal applicability. Also it is important to understand the relative pros and cons of SCNNs and BNNs in theory and in actual hardware implementations. In order to address these concerns, in this paper we prove that the "ideal" SCNNs and BNNs satisfy the universal approximation property with probability 1 (due to the stochastic behavior). The proof is conducted by first proving the property for SCNNs from the strong law of large numbers, and then using SCNNs as a "bridge" to prove for BNNs. Based on the universal approximation property, we further prove that SCNNs and BNNs exhibit the same energy complexity. In other words, they have the same asymptotic energy consumption with the growing of network size. We also provide a detailed analysis of the pros and cons of SCNNs and BNNs for hardware implementations and conclude that SCNNs are more suitable for hardware.

cs.LG↗

Structured Weight Matrices-Based Hardware Accelerators in Deep Neural Networks: FPGAs and ASICs

Both industry and academia have extensively investigated hardware accelerations. In this work, to address the increasing demands in computational capability and memory requirement, we propose structured weight matrices (SWM)-based compression techniques for both \emph{field programmable gate array} (FPGA) and \emph{application-specific integrated circuit} (ASIC) implementations. In algorithm part, SWM-based framework adopts block-circulant matrices to achieve a fine-grained tradeoff between accuracy and compression ratio. The SWM-based technique can reduce computational complexity from O($n^2$) to O($n\log n$) and storage complexity from O($n^2$) to O($n$) for each layer and both training and inference phases. For FPGA implementations on deep convolutional neural networks (DCNNs), we achieve at least 152X and 72X improvement in performance and energy efficiency, respectively using the SWM-based framework, compared with the baseline of IBM TrueNorth processor under same accuracy constraints using the data set of MNIST, SVHN, and CIFAR-10. For FPGA implementations on long short term memory (LSTM) networks, the proposed SWM-based LSTM can achieve up to 21X enhancement in performance and 33.5X gains in energy efficiency compared with the baseline accelerator. For ASIC implementations, the SWM-based ASIC design exhibits impressive advantages in terms of power, throughput, and energy efficiency. Experimental results indicate that this method is greatly suitable for applying DNNs onto both FPGAs and mobile/IoT devices.

cs.DC↗