arXiv subjects

Yanjun Zhao

Publications and source records attributed to Yanjun Zhao.

22 records · Page 2Linked to original sources

arXiv2024

Less is more: Embracing sparsity and interpolation with Esiformer for time series forecasting

Time series forecasting has played a significant role in many practical fields. But time series data generated from real-world applications always exhibits high variance and lots of noise, which makes it difficult to capture the inherent periodic patterns of the data, hurting the prediction accuracy significantly. To address this issue, we propose the Esiformer, which apply interpolation on the original data, decreasing the overall variance of the data and alleviating the influence of noise. What's more, we enhanced the vanilla transformer with a robust Sparse FFN. It can enhance the representation ability of the model effectively, and maintain the excellent robustness, avoiding the risk of overfitting compared with the vanilla implementation. Through evaluations on challenging real-world datasets, our method outperforms leading model PatchTST, reducing MSE by 6.5% and MAE by 5.8% in multivariate time series forecasting. Code is available at: https://github.com/yyg1282142265/Esiformer/tree/main.

cs.LG↗

arXiv2024

Sparse-VQ Transformer: An FFN-Free Framework with Vector Quantization for Enhanced Time Series Forecasting

Time series analysis is vital for numerous applications, and transformers have become increasingly prominent in this domain. Leading methods customize the transformer architecture from NLP and CV, utilizing a patching technique to convert continuous signals into segments. Yet, time series data are uniquely challenging due to significant distribution shifts and intrinsic noise levels. To address these two challenges,we introduce the Sparse Vector Quantized FFN-Free Transformer (Sparse-VQ). Our methodology capitalizes on a sparse vector quantization technique coupled with Reverse Instance Normalization (RevIN) to reduce noise impact and capture sufficient statistics for forecasting, serving as an alternative to the Feed-Forward layer (FFN) in the transformer architecture. Our FFN-free approach trims the parameter count, enhancing computational efficiency and reducing overfitting. Through evaluations across ten benchmark datasets, including the newly introduced CAISO dataset, Sparse-VQ surpasses leading models with a 7.84% and 4.17% decrease in MAE for univariate and multivariate time series forecasting, respectively. Moreover, it can be seamlessly integrated with existing transformer-based models to elevate their performance.

cs.LG↗

arXiv2020

Unsupervised Learning of Deep-Learned Features from Breast Cancer Images

Detecting cancer manually in whole slide images requires significant time and effort on the laborious process. Recent advances in whole slide image analysis have stimulated the growth and development of machine learning-based approaches that improve the efficiency and effectiveness in the diagnosis of cancer diseases. In this paper, we propose an unsupervised learning approach for detecting cancer in breast invasive carcinoma (BRCA) whole slide images. The proposed method is fully automated and does not require human involvement during the unsupervised learning procedure. We demonstrate the effectiveness of the proposed approach for cancer detection in BRCA and show how the machine can choose the most appropriate clusters during the unsupervised learning procedure. Moreover, we present a prototype application that enables users to select relevant groups mapping all regions related to the groups in whole slide images.

eess.IV↗

arXiv2016

An efficient and compact quantum switch for quantum circuits

The engineering of quantum devices has reached the stage where we now have small scale quantum processors containing multiple interacting qubits within them. Simple quantum circuits have been demonstrated and scaling up to larger numbers is underway. However as the number of qubits in these processors increases, it becomes challenging to implement switchable or tunable coherent coupling among them. The typical approach has been to detune each qubit from others or the quantum bus it connected to, but as the number of qubits increases this becomes problematic to achieve in practice due to frequency crowding issues. Here, we demonstrate that by applying a fast longitudinal control field to the target qubit, we can turn off its couplings to other qubits or buses (in principle on/off ratio higher than 100 dB). This has important implementations in superconducting circuits as it means we can keep the qubits at their optimal points, where the coherence properties are greatest, during coupling/decoupling processing. Our approach suggests a new way to control coupling among qubits and data buses that can be naturally scaled up to large quantum processors without the need for auxiliary circuits and yet be free of the frequency crowding problems.

quant-ph↗