arXiv subjects

Liang Yuan

Publications and source records attributed to Liang Yuan.

33 records · Page 2Linked to original sources

arXiv2024

Stencil Matrixization

Current architectures are now equipped with matrix computation units designed to enhance AI and high-performance computing applications. Within these architectures, two fundamental instruction types are matrix multiplication and vector outer product, with the latter being lighter due to its vector inputs. This characteristic not only allows for the development of flexible algorithms beyond dense linear algebra computations but also offers greater potential for implementation optimization. Stencil computations, commonly found in scientific and engineering applications, involve nested loops. This paper introduces a novel stencil algorithm leveraging vector outer products. Unlike previous approaches, this algorithm emerges from the stencil definition in scatter mode and is initially formulated using vector outer product expressions. The implementation integrates a series of optimizations to enhance memory reference patterns, execution pipeline efficiency, and data reuse. These optimizations consider various algorithmic options and data sharing among input vectors. Evaluation conducted on a simulator demonstrates that our proposed design achieves significant speedup compared to vectorized stencil algorithms.

cs.DC↗

arXiv2023

Gamify Stencil Dwarf on Cloud for Democratizing Scientific Computing

Stencil computation is one of the most important kernels in various scientific computing. Nowadays, most Stencil-driven scientific computing still relies heavily on supercomputers, suffering from expensive access, poor scalability, and duplicated optimizations. This paper proposes Tetris, the first system for high-performance Stencil on heterogeneous CPU+GPU, towards democratizing Stencil-driven scientific computing on Cloud. In Tetris, polymorphic tiling tetrominoes are first proposed to bridge different hardware architectures and various application contexts with a perfect spatial and temporal tessellation automatically. Tetris is contributed by three main components: (1) Underlying hardware characteristics are first captured to achieve a sophisticated Pattern Mapping by register-level tetrominoes; (2) An efficient Locality Enhancer is first presented for data reuse on spatial and temporal dimensions simultaneously by cache/SMEM-level tetrominoes; (3) A novel Concurrent Scheduler is first designed to exploit the full potential of on-cloud memory and computing power by memory-level tetrominoes. Tetris is orthogonal to (and complements) the optimizations or deployments for a wide variety of emerging and legacy scientific computing applications. Results of thermal diffusion simulation demonstrate that the performance is improved by 29.6x, reducing time cost from day to hour, while preserving the original accuracy.

cs.DC↗

arXiv2021

AGCM-3DLF: Accelerating Atmospheric General Circulation Model via 3D Parallelization and Leap-Format

The Atmospheric General Circulation Model (AGCM) has been an important research tool in the study of climate change for decades. As the demand for high-resolution simulation is becoming urgent, the scalability and simulation efficiency is faced with great challenges, especially for the latitude-longitude mesh-based models. In this paper, we propose a highly scalable 3D atmospheric general circulation model based on leap-format, namely AGCM-3DLF. Firstly, it utilizes a 3D decomposition method allowing for parallelism release in all three physical dimensions. Then the leap-format difference computation scheme is adopted to maintain computational stability in grid updating and avoid additional filtering at the high latitudes. A novel shifting window communication algorithm is designed for parallelization of the unified model. Furthermore, a series of optimizations are conducted to improve the effectiveness of large-scale simulations. Experiment results in different platforms demonstrate good efficiency and scalability of the model. AGCM-3DLF scales up to the entire CAS-Xiandao1 supercomputer (196,608 CPU cores), attaining the speed of 11.1 simulation-year-per-day (SYPD) at a high resolution of 25KM. In addition, simulations conducted on the Sunway TaihuLight supercomputer exhibit a 1.06 million cores scalability with 36.1% parallel efficiency.

cs.DC↗

arXiv2021

An Accurate and Efficient Large-scale Regression Method through Best Friend Clustering

As the data size in Machine Learning fields grows exponentially, it is inevitable to accelerate the computation by utilizing the ever-growing large number of available cores provided by high-performance computing hardware. However, existing parallel methods for clustering or regression often suffer from problems of low accuracy, slow convergence, and complex hyperparameter-tuning. Furthermore, the parallel efficiency is usually difficult to improve while striking a balance between preserving model properties and partitioning computing workloads on distributed systems. In this paper, we propose a novel and simple data structure capturing the most important information among data samples. It has several advantageous properties supporting a hierarchical clustering strategy that is irrelevant to the hardware parallelism, well-defined metrics for determining optimal clustering, balanced partition for maintaining the compactness property, and efficient parallelization for accelerating computation phases. Then we combine the clustering with regression techniques as a parallel library and utilize a hybrid structure of data and model parallelism to make predictions. Experiments illustrate that our library obtains remarkable performance on convergence, accuracy, and scalability.

cs.LG↗

arXiv2021

An Efficient Vectorization Scheme for Stencil Computation

Stencil computation is one of the most important kernels in various scientific and engineering applications. A variety of work has focused on vectorization and tiling techniques, aiming at exploiting the in-core data parallelism and data locality respectively. In this paper, the downsides of existing vectorization schemes are analyzed. Briefly, they either incur data alignment conflicts or hurt the data locality when integrated with tiling. Then we propose a novel transpose layout to preserve the data locality for tiling and reduce the data reorganization overhead for vectorization simultaneously. To further improve the data reuse at the register level, a time loop unroll-and-jam strategy is designed to perform multistep stencil computation along the time dimension. Experimental results on the AVX-2 and AVX-512 CPUs show that our approach obtains a competitive performance.

cs.DC↗

arXiv2021

AutoFlow: Hotspot-Aware, Dynamic Load Balancing for Distributed Stream Processing

Stream applications are widely deployed on the cloud. While modern distributed streaming systems like Flink and Spark Streaming can schedule and execute them efficiently, streaming dataflows are often dynamically changing, which may cause computation imbalance and backpressure. We introduce AutoFlow, an automatic, hotspot-aware dynamic load balance system for streaming dataflows. It incorporates a centralized scheduler which monitors the load balance in the entire dataflow dynamically and implements state migrations correspondingly. The scheduler achieves these two tasks using a simple asynchronous distributed control message mechanism and a hotspot-diminishing algorithm. The timing mechanism supports implicit barriers and a highly efficient state-migration without global barriers or pauses to operators. It also supports a time-window based load-balance measurement and feeds them to the hotspot-diminishing algorithm without user interference. We implemented AutoFlow on top of Ray, an actor-based distributed execution framework. Our evaluation based on various streaming benchmark dataset shows that AutoFlow achieves good load-balance and incurs a low latency overhead in highly data-skew workload.

eess.SY↗

arXiv2021

Reducing Redundancy in Data Organization and Arithmetic Calculation for Stencil Computations

Stencil computation is one of the most important kernels in various scientific and engineering applications. A variety of work has focused on vectorization techniques, aiming at exploiting the in-core data parallelism. Briefly, they either incur data alignment conflicts or hurt the data locality when integrated with tiling. In this paper, a novel transpose layout is devised to preserve the data locality for tiling in the data space and reduce the data reorganization overhead for vectorization simultaneously. We then propose an approach of temporal computation folding designed to further reduce the redundancy of arithmetic calculations by exploiting the register reuse, alleviating the increased register pressure, and deducing generalization with a linear regression model. Experimental results on the AVX-2 and AVX-512 CPUs show that our approach obtains a competitive performance.

cs.DC↗

arXiv2021

A Concept Knowledge-Driven Keywords Retrieval Framework for Sponsored Search

In sponsored search, retrieving synonymous keywords for exact match type is important for accurately targeted advertising. Data-driven deep learning-based method has been proposed to tackle this problem. An apparent disadvantage of this method is its poor generalization performance on entity-level long-tail instances, even though they might share similar concept-level patterns with frequent instances. With the help of a large knowledge base, we find that most commercial synonymous query-keyword pairs can be abstracted into meaningful conceptual patterns through concept tagging. Based on this fact, we propose a novel knowledge-driven conceptual retrieval framework to mitigate this problem, which consists of three parts: data conceptualization, matching via conceptual patterns and concept-augmented discrimination. Both offline and online experiments show that our method is very effective. This framework has been successfully applied to Baidu's sponsored search system, which yields a significant improvement in revenue.

cs.IR↗

arXiv2020

Temporal Vectorization for Stencils

Stencil computations represent a very common class of nested loops in scientific and engineering applications. Exploiting vector units in modern CPUs is crucial to achieving peak performance. Previous vectorization approaches often consider the data space, in particular the innermost unit-strided loop. It leads to the well-known data alignment conflict problem that vector loads are overlapped due to the data sharing between continuous stencil computations. This paper proposes a novel temporal vectorization scheme for stencils. It vectorizes the stencil computation in the iteration space and assembles points with different time coordinates in one vector. The temporal vectorization leads to a small fixed number of vector reorganizations that is irrelevant to the vector length, stencil order, and dimension. Furthermore, it is also applicable to Gauss-Seidel stencils, whose vectorization is not well-studied. The effectiveness of the temporal vectorization is demonstrated by various Jacobi and Gauss-Seidel stencils.

cs.MS↗

arXiv2020

Optimizing AD Pruning of Sponsored Search with Reinforcement Learning

Industrial sponsored search system (SSS) can be logically divided into three modules: keywords matching, ad retrieving, and ranking. During ad retrieving, the ad candidates grow exponentially. A query with high commercial value might retrieve a great deal of ad candidates such that the ranking module could not afford. Due to limited latency and computing resources, the candidates have to be pruned earlier. Suppose we set a pruning line to cut SSS into two parts: upstream and downstream. The problem we are going to address is: how to pick out the best $K$ items from $N$ candidates provided by the upstream to maximize the total system's revenue. Since the industrial downstream is very complicated and updated quickly, a crucial restriction in this problem is that the selection scheme should get adapted to the downstream. In this paper, we propose a novel model-free reinforcement learning approach to fixing this problem. Our approach considers downstream as a black-box environment, and the agent sequentially selects items and finally feeds into the downstream, where revenue would be estimated and used as a reward to improve the selection policy. To the best of our knowledge, this is first time to consider the system optimization from a downstream adaption view. It is also the first time to use reinforcement learning techniques to tackle this problem. The idea has been successfully realized in Baidu's sponsored search system, and online long time A/B test shows remarkable improvements on revenue.

cs.LG↗

arXiv2019

Deterministic generation and partial retrieval of a spin-wave excitation in an atomic ensemble

In this paper, we report a generation of a spin-wave excitation (SWE) with a near-unity () probability in a given time (~730). Such deterministic generation relies on a feedback scheme with a millisecond quantum memory. The millisecond memory is achieved by maximizing the wavelength of the spin wave and storing the SWE as the magnetic-field-insensitive transition. We then demonstrate partial retrievals of the spin wave by applying a first read pulse whose area is smaller than the value of . The remained SWE is fully retrieved by a second pulse. Anti-correlation function between the detections in the first and second readouts has been measured, which shows that the partial-retrieval operation on the SWE is in the quantum regime. The presented experiment represents an important step towards the realization of the improved DLCZ quantum repeater protocol proposed in Phys. Rev. A 77, 062301 (2008).

quant-ph↗

arXiv2019

Multiplexed spin-wave-photon entanglement source using temporal-multimode memories and feedforward-controlled readout

The sources, which generate atom-photon quantum correlations or entanglement based on quantum memory, are basic blocks for building quantum repeaters (QRs). For achieving highly entanglement-generation rates in ensemble-based QRs, spatial-, temporal- and spectral-multimode memories are needed. The previous temporal-multimode memories are based on rephrasing mechanisms in inhomogeneous-broadened media. Here, by applying a train of multi-direction write pulses into a homogeneous-broadened atomic ensemble to induce Duan-Lukin-Cirac-Zoller-like Raman processes, we prepare up to 19 pairs of modes, namely, a spin-wave mode and a photonic time bin. Spin-wave-photon (i.e., atom-photon) entanglement is probabilistically produced in the mode pairs. As a proof-in-principle demonstration, we show that the multiplexed source using all the spin-wave modes and feed-forward-control readout increases spin-wave-photon entanglement generation rate by a factor of $\sim$18.3, compared to non-multiplexed source using the individual modes. The measured Bell parameter for the multiplexed source is $2.30\pm 0.02$ combined with a memory lifetime of $30μs$.

quant-ph↗

arXiv2018

Modeling the spatio-temporal dynamics of land use change with recurrent neural networks

This study applies recurrent neural networks (RNNs), which are known for its ability to process sequential information, to model the spatio-temporal dynamics of land use change (LUC) and to forecast annual land use maps of the city of Tsukuba, Japan. We develop two categories of RNN models: 1) simple RNN, which is the basic RNN variant, 2) three RNN variants with advanced gated architecture: long short-term memory (LSTM), LSTM with peephole connection (LSTM-peephole), and gated recurrent unit (GRU) models. The four models are developed using spatio-temporal data with high temporal resolution, annual data for the periods 2000 to 2010, 2011 and 2012 to 2016 are used for training, validation and testing, respectively. The predictive performances are evaluated using classification metrics (accuracy and F1 score) and the map comparison metrics (Kappa simulation and fuzzy Kappa simulation). The results show that all RNN models achieve F1 scores higher than 0.55, and Kappa simulations higher than 0.47. Out of the four RNN models, LSTM and LSTM-peephole models significantly outperform the other two RNN models. Furthermore, LSTM-peephole model slightly outperforms the LSTM model. In addition, the results indicate that the RNN models with gated architecture, which have better ability to model longer temporal dependency, significantly outperform the simple RNN model. Moreover, the predictive performance of LSTM-peephole model gradually decreases with the decrease of temporal sequential length of the training set. These results demonstrate the benefit of taking temporal dependency into account to model the LUC process with RNNs.

stat.AP↗

arXiv2018

A Measurement Theory of Locality

Locality is a fundamental principle used extensively in program and system optimization. It can be measured in many ways. This paper formalizes the metrics of locality into a measurement theory. The new theory includes the precise definition of locality metrics based on access frequency, reuse time, reuse distance, working set, footprint, and the cache miss ratio. It gives the formal relation between these definitions and the proofs of equivalence or non-equivalence. It provides the theoretical justification for four successful locality models in operating systems, programming languages, and computer architectures which were developed empirically.

cs.PF↗

arXiv2018

Enhancement of land-use change modeling using convolutional neural networks and convolutional denoising autoencoders

The neighborhood effect is a key driving factor for the land-use change (LUC) process. This study applies convolutional neural networks (CNN) to capture neighborhood characteristics from satellite images and to enhance the performance of LUC modeling. We develop a hybrid CNN model (conv-net) to predict the LU transition probability by combining satellite images and geographical features. A spatial weight layer is designed to incorporate the distance-decay characteristics of neighborhood effect into conv-net. As an alternative model, we also develop a hybrid convolutional denoising autoencoder and multi-layer perceptron model (CDAE-net), which specifically learns latent representations from satellite images and denoises the image data. Finally, a DINAMICA-based cellular automata (CA) model simulates the LU pattern. The results show that the convolutional-based models improve the modeling performances compared with a model that accepts only the geographical features. Overall, conv-net outperforms CDAE-net in terms of LUC predictive performance. Nonetheless, CDAE-net performs better when the data are noisy.

stat.AP↗