Searcharxiv⌕ Search

arXiv subjects

Siyuan Lu

Publications and source records attributed to Siyuan Lu.

At least 55 records · Page 3Linked to original sources
arXiv2020

Learning-Based Massive Beamforming

Developing resource allocation algorithms with strong real-time and high efficiency has been an imperative topic in wireless networks. Conventional optimization-based iterative resource allocation algorithms often suffer from slow convergence, especially for massive multiple-input-multiple-output (MIMO) beamforming problems. This paper studies learning-based efficient massive beamforming methods for multi-user MIMO networks. The considered massive beamforming problem is challenging in two aspects. First, the beamforming matrix to be learned is quite high-dimensional in case with a massive number of antennas. Second, the objective is often time-varying and the solution space is not fixed due to some communication requirements. All these challenges make learning representation for massive beamforming an extremely difficult task. In this paper, by exploiting the structure of the most popular WMMSE beamforming solution, we propose convolutional massive beamforming neural networks (CMBNN) using both supervised and unsupervised learning schemes with particular design of network structure and input/output. Numerical results demonstrate the efficacy of the proposed CMBNN in terms of running time and system throughput.

eess.SP↗
arXiv2020

Hardware Accelerator for Multi-Head Attention and Position-Wise Feed-Forward in the Transformer

Designing hardware accelerators for deep neural networks (DNNs) has been much desired. Nonetheless, most of these existing accelerators are built for either convolutional neural networks (CNNs) or recurrent neural networks (RNNs). Recently, the Transformer model is replacing the RNN in the natural language processing (NLP) area. However, because of intensive matrix computations and complicated data flow being involved, the hardware design for the Transformer model has never been reported. In this paper, we propose the first hardware accelerator for two key components, i.e., the multi-head attention (MHA) ResBlock and the position-wise feed-forward network (FFN) ResBlock, which are the two most complex layers in the Transformer. Firstly, an efficient method is introduced to partition the huge matrices in the Transformer, allowing the two ResBlocks to share most of the hardware resources. Secondly, the computation flow is well designed to ensure the high hardware utilization of the systolic array, which is the biggest module in our design. Thirdly, complicated nonlinear functions are highly optimized to further reduce the hardware complexity and also the latency of the entire system. Our design is coded using hardware description language (HDL) and evaluated on a Xilinx FPGA. Compared with the implementation on GPU with the same setting, the proposed design demonstrates a speed-up of 14.6x in the MHA ResBlock, and 3.4x in the FFN ResBlock, respectively. Therefore, this work lays a good foundation for building efficient hardware accelerators for multiple Transformer networks.

eess.SP↗
arXiv2020

Statistical Modeling for Spatio-Temporal Data from Stochastic Convection-Diffusion Processes

This paper proposes a physical-statistical modeling approach for spatio-temporal data arising from a class of stochastic convection-diffusion processes. Such processes are widely found in scientific and engineering applications where fundamental physics imposes critical constraints on how data can be modeled and how models should be interpreted. The idea of spectrum decomposition is employed to approximate a physical spatio-temporal process by the linear combination of spatial basis functions and a multivariate random process of spectral coefficients. Unlike existing approaches assuming spatially- and temporally-invariant convection-diffusion, this paper considers a more general scenario with spatially-varying convection-diffusion and nonzero-mean source-sink. As a result, the temporal dynamics of spectral coefficients is coupled with each other, which can be interpreted as the non-linear energy redistribution across multiple scales from the perspective of physics. Because of the spatially-varying convection-diffusion, the space-time covariance is non-stationary in space. The theoretical results are integrated into a hierarchical dynamical spatio-temporal model. The connection is established between the proposed model and the existing models based on Integro-Difference Equations. Computational efficiency and scalability are also investigated to make the proposed approach practical. The advantages of the proposed methodology are demonstrated by numerical examples, a case study, and comprehensive comparison studies. Computer code is available on GitHub.

stat.AP↗
arXiv2020

Buried spatially-regular array of spectrally ultra-uniform single quantum dots for on-chip scalable quantum optical circuits

A long standing obstacle to realizing highly sought on-chip monolithic solid state quantum optical circuits has been the lack of a starting platform comprising buried (protected) scalable spatially ordered and spectrally uniform arrays of on-demand single photon sources (SPSs). In this paper we report the first realization of such SPS arrays based upon a class of single quantum dots (SQDs) with single photon emission purity > 99.5% and uniformity < 2nm. Such SQD synthesis approach offers rich flexibility in material combinations and thus can cover the emission wavelength regime from long- to mid- to near-infrared to the visible and ultraviolet. The buried array of SQDs naturally lend themselves to the fabrication of quantum optical circuits employing either the well-developed photonic 2D crystal platform or the use of Mie-like collective resonance of all-dielectric building block based metastructures designed for directed emission and manipulation of the emitted photons in the horizontal planar architecture inherent to on-chip optical circuits. Finite element method-based simulations of the Mie-resonance based manipulation of the emitted light are presented showing achievement of simultaneous multifunctional manipulation of photons with large spectral bandwidth of ~ 20nm that eases spectral and mode matching. Our combined experimental and simulation findings presented here open the pathway for fabrication and study of on-chip quantum optical circuits.

physics.app-ph↗
arXiv2020

Map Generation from Large Scale Incomplete and Inaccurate Data Labels

Accurately and globally mapping human infrastructure is an important and challenging task with applications in routing, regulation compliance monitoring, and natural disaster response management etc.. In this paper we present progress in developing an algorithmic pipeline and distributed compute system that automates the process of map creation using high resolution aerial images. Unlike previous studies, most of which use datasets that are available only in a few cities across the world, we utilizes publicly available imagery and map data, both of which cover the contiguous United States (CONUS). We approach the technical challenge of inaccurate and incomplete training data adopting state-of-the-art convolutional neural network architectures such as the U-Net and the CycleGAN to incrementally generate maps with increasingly more accurate and more complete labels of man-made infrastructure such as roads and houses. Since scaling the mapping task to CONUS calls for parallelization, we then adopted an asynchronous distributed stochastic parallel gradient descent training scheme to distribute the computational workload onto a cluster of GPUs with nearly linear speed-up.

cs.CV↗
arXiv2020

Event Clustering & Event Series Characterization on Expected Frequency

We present an efficient clustering algorithm applicable to one-dimensional data such as e.g. a series of timestamps. Given an expected frequency $ΔT^{-1}$, we introduce an $\mathcal{O}(N)$-efficient method of characterizing $N$ events represented by an ordered series of timestamps $t_1,t_2,\dots,t_N$. In practice, the method proves useful to e.g. identify time intervals of "missing" data or to locate "isolated events". Moreover, we define measures to quantify a series of events by varying $ΔT$ to e.g. determine the quality of an Internet of Things service.

cs.DC↗
arXiv2019

Training Deep Neural Networks Using Posit Number System

With the increasing size of Deep Neural Network (DNN) models, the high memory space requirements and computational complexity have become an obstacle for efficient DNN implementations. To ease this problem, using reduced-precision representations for DNN training and inference has attracted many interests from researchers. This paper first proposes a methodology for training DNNs with the posit arithmetic, a type- 3 universal number (Unum) format that is similar to the floating point(FP) but has reduced precision. A warm-up training strategy and layer-wise scaling factors are adopted to stabilize training and fit the dynamic range of DNN parameters. With the proposed training methodology, we demonstrate the first successful training of DNN models on ImageNet image classification task in 16 bits posit with no accuracy loss. Then, an efficient hardware architecture for the posit multiply-and-accumulate operation is also proposed, which can achieve significant improvement in energy efficiency than traditional floating-point implementations. The proposed design is helpful for future low-power DNN training accelerators.

cs.LG↗
arXiv2019

Synthesizing simulation and field data of solar irradiance

Predicting the intensity and amount of sunlight as a function of location and time is an essential component in identifying promising locations for economical solar farming. Although weather models and irradiance data are relatively abundant, these have yet, to our knowledge, been hybridized on a continental scale. Rather, much of the emphasis in the literature has been on short-term localized forecasting. This is probably because the amount of data involved in a more global analysis is prohibitive with the canonical toolkit, via the Gaussian process (GP). Here we show how GP surrogate and discrepancy models can be combined to tractably and accurately predict solar irradiance on time-aggregated and daily scales with measurements at thousands of sites across the continental United States. Our results establish short term accuracy of bias-corrected weather-based simulation of irradiance, when realizations are available in real space-time (e.g., in future days), and provide accurate surrogates for smoothing in the more common situation where reliable weather data is not available (e.g., in future years).

stat.AP↗
arXiv2019

Monge-Ampère equation with bounded periodic data

We consider the Monge-Ampère equation $\det(D^2u)=f$ in $\mathbb{R}^n$, where $f$ is a positive bounded periodic function. We prove that $u$ must be the sum of a quadratic polynomial and a periodic function. For $f\equiv 1$, this is the classic result by Jörgens, Calabi and Pogorelov. For $f\in C^α$, this was proved by Caffarelli and the first named author.

math.AP↗
arXiv2019

A Hardware-Oriented and Memory-Efficient Method for CTC Decoding

The Connectionist Temporal Classification (CTC) has achieved great success in sequence to sequence analysis tasks such as automatic speech recognition (ASR) and scene text recognition (STR). These applications can use the CTC objective function to train the recurrent neural networks (RNNs), and decode the outputs of RNNs during inference. While hardware architectures for RNNs have been studied, hardware-based CTCdecoders are desired for high-speed CTC-based inference systems. This paper, for the first time, provides a low-complexity and memory-efficient approach to build a CTC-decoder based on the beam search decoding. Firstly, we improve the beam search decoding algorithm to save the storage space. Secondly, we compress a dictionary (reduced from 26.02MB to 1.12MB) and use it as the language model. Meanwhile searching this dictionary is trivial. Finally, a fixed-point CTC-decoder for an English ASR and an STR task using the proposed method is implemented with C++ language. It is shown that the proposed method has little precision loss compared with its floating-point counterpart. Our experiments demonstrate the compression ratio of the storage required by the proposed beam search decoding algorithm are 29.49 (ASR) and 17.95 (STR).

eess.SP↗
arXiv2018

On-Chip Scalable Coupled Single Photon Emitter- All Dielectric Multifunctional Quantum Optical Circuits Working on a Single Collective Mie Resonance

We present finite element method based simulation results for on-chip controlled photon interference in optical networks based upon interconnected arrays of primitives comprising a single photon source (SPS) embedded in lithographically fabricable rectangular dielectric building block (DBB) based metastructures in which a single collective Mie resonance of the whole circuit provides the needed light manipulating functions of nanoantenna, state-preserving-guiding, beam-splitting and beam-combining. Efficient coupling of the SPSs to the co-designed collective Mie mode of DBBs is achieved. The findings provide strong incentive for the fabrication and examination of such optical circuits that underlie on-chip scalable nanophotonic quantum information processing systems.

physics.optics↗
arXiv2018

Highly pure single photon emission from spectrally uniform surface-curvature directed mesa top single quantum dot ordered array

Realizing ordered and spectrally uniform single photon source arrays integrable on-chip with light manipulating elements in scalable architecture lies at the core of building monolithic quantum optical circuits (QOCs). We demonstrate here a spatially-ordered 5 x 8 array of surface-curvature driven mesa-top GaAs(001)/InGaAs/GaAs single quantum dots (MTSQDs) that exhibit highly pure (~99% ) single photon emission as deduced from the measured g(2)(0) < 0.02 at 9.4K. Polarization-independent and polarization-resolved high resolution photoluminescence (PL) measurements show that these ordered and spectrally uniform QDs have neutral exciton emission with intrinsic linewidth ~ 10 ueV and fine structure splitting < 10 ueV, an important figure of merit for the use of QDs in QOCs. The findings point to the high potential of such MTSQD based single photon source arrays as a promising platform for on-chip scalable integration with light manipulating units (connected resonant cavity, waveguide, beam splitter, etc.) to enable constructing QOCs.

quant-ph↗
arXiv2018

Multifunctional Control of On-chip Generated Photons by a Single Collective Mode in Monolithically Integrated All-Dielectric Scalable Optical Circuits

Recently we proposed the use of a single collective Mie mode of coupled dielectric resonators arranged in metastructures designed to simultaneously enhance the emission rate, control the direction, and propagation of emitted photons from an embedded single photon source (SPS) [J. Opt. Soc. Am. B 33(12), 2414(2016); J. Appl. Phys.120, 243103(2016)]. Here we extend the analysis to include the necessary additional two functions of beam splitting and controlled combing of photons from two distinct on-chip sources for interference. Thus, the same single Mie mode of the designed system provides all the required five functions in different spatial regions of an on-chip integrated optical circuit, enabling full control over photon interference for classical and quantum optical information processing. Single photon source architectures suited for such on-chip scalable integration are discussed.

physics.optics↗
arXiv2018

On Weyl's embedding problem in Riemannian manifolds

We consider a priori estimates of Weyl's embedding problem of $(\mathbb{S}^2, g)$ in general $3$-dimensional Riemannian manifold $(N^3, \bar g)$. We establish interior $C^2$ estimate under natural geometric assumption. Together with a recent work by Li and Wang, we obtain an isometric embedding of $(\mathbb{S}^2,g)$ in Riemannian manifold. In addition, we reprove Weyl's embedding theorem in space form under the condition that $g\in C^2$ with $D^2g$ Dini continuous.

math.DG↗
arXiv2018

An Unsupervised Clustering-Based Short-Term Solar Forecasting Methodology Using Multi-Model Machine Learning Blending

Solar forecasting accuracy is affected by weather conditions, and weather awareness forecasting models are expected to improve the performance. However, it may not be available and reliable to classify different forecasting tasks by using only meteorological weather categorization. In this paper, an unsupervised clustering-based (UC-based) solar forecasting methodology is developed for short-term (1-hour-ahead) global horizontal irradiance (GHI) forecasting. This methodology consists of three parts: GHI time series unsupervised clustering, pattern recognition, and UC-based forecasting. The daily GHI time series is first clustered by an Optimized Cross-validated ClUsteRing (OCCUR) method, which determines the optimal number of clusters and best clustering results. Then, support vector machine pattern recognition (SVM-PR) is adopted to recognize the category of a certain day using the first few hours' data in the forecasting stage. GHI forecasts are generated by the most suitable models in different clusters, which are built by a two-layer Machine learning based Multi-Model (M3) forecasting framework. The developed UC-based methodology is validated by using 1-year of data with six solar features. Numerical results show that (i) UC-based models outperform non-UC (all-in-one) models with the same M3 architecture by approximately 20%; (ii) M3-based models also outperform the single-algorithm machine learning (SAML) models by approximately 20%.

cs.LG↗
arXiv2018

Variation and rigidity of quasi-local mass

Inspired by the work of Chen-Zhang \cite{Chen-Zhang}, we derive an evolution formula for the Wang-Yau quasi-local energy in reference to a static space, introduced by Chen-Wang-Wang-Yau \cite{CWWY}. If the reference static space represents a mass minimizing, static extension of the initial surface $Σ$, we observe that the derivative of the Wang-Yau quasi-local energy is equal to the derivative of the Bartnik quasi-local mass at $Σ$. Combining the evolution formula for the quasi-local energy with a localized Penrose inequality proved in \cite{Lu-Miao}, we prove a rigidity theorem for compact $3$-manifolds with nonnegative scalar curvature, with boundary. This rigidity theorem in turn gives a characterization of the equality case of the localized Penrose inequality in $3$-dimension.

math.DG↗
arXiv2017

Where computer vision can aid physics: dynamic cloud motion forecasting from satellite images

This paper describes a new algorithm for solar energy forecasting from a sequence of Cloud Optical Depth (COD) images. The algorithm is based on the following simple observation: the dynamics of clouds represented by COD images resembles the motion (transport) of a density in a fluid flow. This suggests that, to forecast the motion of COD images, it is sufficient to forecast the flow. The latter, in turn, can be accomplished by fitting a parametric model of the fluid flow to the COD images observed in the past. Namely, the learning phase of the algorithm is composed of the following steps: (i) given a sequence of COD images, the snapshots of the optical flow are estimated from two consecutive COD images; (ii) these snapshots are then assimilated into a Navier-Stokes Equation (NSE), i.e. an initial velocity field for NSE is selected so that the corresponding NSE' solution is as close as possible to the optical flow snapshots. The prediction phase consists of utilizing a linear transport equation, which describes the propagation of COD images in the fluid flow predicted by NSE, to estimate the future motion of the COD images. The algorithm has been tested on COD images provided by two geostationary operational environmental satellites from NOAA serving the west-hemisphere.

math.OC↗