arXiv subjects

Zhengyang Li

Publications and source records attributed to Zhengyang Li.

At least 37 records · Page 2Linked to original sources

arXiv2025

Intelligent road crack detection and analysis based on improved YOLOv8

As urbanization speeds up and traffic flow increases, the issue of pavement distress is becoming increasingly pronounced, posing a severe threat to road safety and service life. Traditional methods of pothole detection rely on manual inspection, which is not only inefficient but also costly. This paper proposes an intelligent road crack detection and analysis system, based on the enhanced YOLOv8 deep learning framework. A target segmentation model has been developed through the training of 4029 images, capable of efficiently and accurately recognizing and segmenting crack regions in roads. The model also analyzes the segmented regions to precisely calculate the maximum and minimum widths of cracks and their exact locations. Experimental results indicate that the incorporation of ECA and CBAM attention mechanisms substantially enhances the model's detection accuracy and efficiency, offering a novel solution for road maintenance and safety monitoring.

cs.CV↗

arXiv2025

"Frog-eyes" in Astronomy: Monitoring Binary Radial Velocity Variations Through A Pair of Narrow-Band Filters

Spectroscopic observations are a crucial step in driving major discoveries in the era of time-domain surveys. However, the pace of current spectroscopic surveys is increasingly unable to meet the demands of rapidly advancing large-scale time-domain surveys. To address this issue, we propose the ``Frog-eyes" system, which employs a pair of narrow-band filters: one positioned near a strong absorption line to capture signals from Doppler shifts, and the other placed on the adjacent continuum to monitor intrinsic variations. The combination of observations from the two filters enables the extraction of radial velocity (RV) curves from a large sample of binary stars, and is particularly efficient for single-lined binaries (SB1), using photometric techniques. Comprehensive mock simulations on SB1 demonstrate that the binary orbital parameters can be precisely measured from the extracted RV curves for binary systems where the primary star has an effective temperature greater than 6000 K. With a typical ground-based photometric precision of approximately 0.3%, the uncertainties in the derived semi-amplitude K and eccentricity e are less than 10% and 0.1, respectively, for binary systems with K $\ge$ 30 km/s. These encouraging results are further validated by real observations of the hot subdwarf-white dwarf binary system HD 265435, using a non-specialized ``Frog-eyes" system installed on the Chinese 2.16m telescope. Once this system is properly installed on large-field-of-view survey telescopes, the rate of acquiring RV curves for binaries will approach their detection rate in leading time-domain photometric surveys.

astro-ph.SR↗

arXiv2024

Large Language Model as a Catalyst: A Paradigm Shift in Base Station Siting Optimization

Traditional base station siting (BSS) methods rely heavily on drive testing and user feedback, which are laborious and require extensive expertise in communication, networking, and optimization. As large language models (LLMs) and their associated technologies advance, particularly in the realms of prompt engineering and agent engineering, network optimization will witness a revolutionary approach. This approach entails the strategic use of well-crafted prompts to infuse human experience and knowledge into these sophisticated LLMs, and the deployment of autonomous agents as a communication bridge to seamlessly connect the machine language based LLMs with human users using natural language. Furthermore, our proposed framework incorporates retrieval-augmented generation (RAG) to enhance the system's ability to acquire domain-specific knowledge and generate solutions, thereby enabling the customization and optimization of the BSS process. This integration represents the future paradigm of artificial intelligence (AI) as a service and AI for more ease. This research first develops a novel LLM-empowered BSS optimization framework, and heuristically proposes three different potential implementations: the strategies based on Prompt-optimized LLM (PoL), LLM-empowered autonomous BSS agent (LaBa), and Cooperative multiple LLM-based autonomous BSS agents (CLaBa). Through evaluation on real-world data, the experiments demonstrate that prompt-assisted LLMs and LLM-based agents can generate more efficient and reliable network deployments, noticeably enhancing the efficiency of BSS optimization and reducing trivial manual participation.

cs.AI↗

arXiv2024

Irregularity-Informed Time Series Analysis: Adaptive Modelling of Spatial and Temporal Dynamics

Irregular Time Series Data (IRTS) has shown increasing prevalence in real-world applications. We observed that IRTS can be divided into two specialized types: Natural Irregular Time Series (NIRTS) and Accidental Irregular Time Series (AIRTS). Various existing methods either ignore the impacts of irregular patterns or statically learn the irregular dynamics of NIRTS and AIRTS data and suffer from limited data availability due to the sparsity of IRTS. We proposed a novel transformer-based framework for general irregular time series data that treats IRTS from four views: Locality, Time, Spatio and Irregularity to motivate the data usage to the highest potential. Moreover, we design a sophisticated irregularity-gate mechanism to adaptively select task-relevant information from irregularity, which improves the generalization ability to various IRTS data. We implement extensive experiments to demonstrate the resistance of our work to three highly missing ratio datasets (88.4\%, 94.9\%, 60\% missing value) and investigate the significance of the irregularity information for both NIRTS and AIRTS by additional ablation study. We release our implementation in https://github.com/IcurasLW/MTSFormer-Irregular_Time_Series.git

cs.LG↗

arXiv2024

Boosting Certified Robustness for Time Series Classification with Efficient Self-Ensemble

Recently, the issue of adversarial robustness in the time series domain has garnered significant attention. However, the available defense mechanisms remain limited, with adversarial training being the predominant approach, though it does not provide theoretical guarantees. Randomized Smoothing has emerged as a standout method due to its ability to certify a provable lower bound on robustness radius under $\ell_p$-ball attacks. Recognizing its success, research in the time series domain has started focusing on these aspects. However, existing research predominantly focuses on time series forecasting, or under the non-$\ell_p$ robustness in statistic feature augmentation for time series classification~(TSC). Our review found that Randomized Smoothing performs modestly in TSC, struggling to provide effective assurances on datasets with poor robustness. Therefore, we propose a self-ensemble method to enhance the lower bound of the probability confidence of predicted labels by reducing the variance of classification margins, thereby certifying a larger radius. This approach also addresses the computational overhead issue of Deep Ensemble~(DE) while remaining competitive and, in some cases, outperforming it in terms of robustness. Both theoretical analysis and experimental results validate the effectiveness of our method, demonstrating superior performance in robustness testing compared to baseline approaches.

cs.LG↗

arXiv2024

Correlation Analysis of Adversarial Attack in Time Series Classification

This study investigates the vulnerability of time series classification models to adversarial attacks, with a focus on how these models process local versus global information under such conditions. By leveraging the Normalized Auto Correlation Function (NACF), an exploration into the inclination of neural networks is conducted. It is demonstrated that regularization techniques, particularly those employing Fast Fourier Transform (FFT) methods and targeting frequency components of perturbations, markedly enhance the effectiveness of attacks. Meanwhile, the defense strategies, like noise introduction and Gaussian filtering, are shown to significantly lower the Attack Success Rate (ASR), with approaches based on noise introducing notably effective in countering high-frequency distortions. Furthermore, models designed to prioritize global information are revealed to possess greater resistance to adversarial manipulations. These results underline the importance of designing attack and defense mechanisms, informed by frequency domain analysis, as a means to considerably reinforce the resilience of neural network models against adversarial threats.

cs.LG↗

arXiv2024

Enhancing Financial Market Predictions: Causality-Driven Feature Selection

This paper introduces the FinSen dataset that revolutionizes financial market analysis by integrating economic and financial news articles from 197 countries with stock market data. The dataset's extensive coverage spans 15 years from 2007 to 2023 with temporal information, offering a rich, global perspective with 160,000 records on financial market news. Our study leverages causally validated sentiment scores and LSTM models to enhance market forecast accuracy and reliability. Utilizing the FinSen dataset, we introduce an innovative Focal Calibration Loss, reducing Expected Calibration Error (ECE) to 3.34 percent with the DAN 3 model. This not only improves prediction accuracy but also aligns probabilistic forecasts closely with real outcomes, crucial for the financial sector where predicted probability is paramount. Our approach demonstrates the effectiveness of combining sentiment analysis with precise calibration techniques for trustworthy financial forecasting where the cost of misinterpretation can be high. Finsen Data can be found at [this github URL](https://github.com/EagleAdelaide/FinSen_Dataset.git).

cs.LG↗

arXiv2024

Quantum Dueling: an Efficient Solution for Combinatorial Optimization

In this paper, we present a new algorithm for generic combinatorial optimization, which we term quantum dueling. Traditionally, potential solutions to the given optimization problems were encoded in a ``register'' of qubits. Various techniques are used to increase the probability of finding the best solution upon measurement. Quantum dueling innovates by integrating an additional qubit register, effectively creating a ``dueling'' scenario where two sets of solutions compete. This dual-register setup allows for a dynamic amplification process: in each iteration, one register is designated as the 'opponent', against which the other register's more favorable solutions are enhanced through a controlled quantum search. This iterative process gradually steers the quantum state within both registers toward the optimal solution. With a quantitative contraction for the evolution of the state vector, classical simulation under a broad range of scenarios and hyper-parameter selection schemes shows that a quadratic speedup is achieved, which is further tested in more real-world situations. In addition, quantum dueling can be generalized to incorporate arbitrary quantum search techniques and as a quantum subroutine within a higher-level algorithm. Our work demonstrates that increasing the number of qubits allows the development of previously unthought-of algorithms, paving the way for advancement of efficient quantum algorithm design.

quant-ph↗

arXiv2023

Perception of Misalignment States for Sky Survey Telescopes with the Digital Twin and the Deep Neural Networks

Sky survey telescopes play a critical role in modern astronomy, but misalignment of their optical elements can introduce significant variations in point spread functions, leading to reduced data quality. To address this, we need a method to obtain misalignment states, aiding in the reconstruction of accurate point spread functions for data processing methods or facilitating adjustments of optical components for improved image quality. Since sky survey telescopes consist of many optical elements, they result in a vast array of potential misalignment states, some of which are intricately coupled, posing detection challenges. However, by continuously adjusting the misalignment states of optical elements, we can disentangle coupled states. Based on this principle, we propose a deep neural network to extract misalignment states from continuously varying point spread functions in different field of views. To ensure sufficient and diverse training data, we recommend employing a digital twin to obtain data for neural network training. Additionally, we introduce the state graph to store misalignment data and explore complex relationships between misalignment states and corresponding point spread functions, guiding the generation of training data from experiments. Once trained, the neural network estimates misalignment states from observation data, regardless of the impacts caused by atmospheric turbulence, noise, and limited spatial sampling rates in the detector. The method proposed in this paper could be used to provide prior information for the active optics system and the optical system alignment.

astro-ph.IM↗

arXiv2023

Data Release of the AST3-2 Automatic Survey from Dome A, Antarctica

AST3-2 is the second of the three Antarctic Survey Telescopes, aimed at wide-field time-domain optical astronomy. It is located at Dome A, Antarctica, which is by many measures the best optical astronomy site on the Earth's surface. Here we present the data from the AST3-2 automatic survey in 2016 and the photometry results. The median 5$σ$ limiting magnitude in $i$-band is 17.8 mag and the light curve precision is 4 mmag for bright stars. The data release includes photometry for over 7~million stars, from which over 3,500 variable stars were detected, with 70 of them newly discovered. We classify these new variables into different types by combining their light curve features with stellar properties from surveys such as StarHorse.

astro-ph.SR↗

arXiv2022

Bandwidth-Scalable Fully Mask-Based Deep FCRN Acoustic Echo Cancellation and Postfiltering

Although today's speech communication systems support various bandwidths from narrowband to super-wideband and beyond, state-of-the art DNN methods for acoustic echo cancellation (AEC) are lacking modularity and bandwidth scalability. Our proposed DNN model builds upon a fully convolutional recurrent network (FCRN) and introduces scalability over various bandwidths up to a fullband (FB) system (48 kHz sampling rate). This modular approach allows joint wideband (WB) pre-training of mask-based AEC and postfilter stages with dedicated losses, followed by a separate training of them on FB data. A third lightweight blind bandwidth extension stage is separately trained on FB data, flexibly allowing to extend the WB postfilter output towards higher bandwidths until reaching FB. Thereby, higher frequency noise and echo are reliably suppressed. On the ICASSP 2022 Acoustic Echo Cancellation Challenge blind test set we report a competitive performance, showing robustness even under highly delayed echo and dynamic echo path changes.

eess.AS↗

arXiv2022

Relaxed Attention for Transformer Models

The powerful modeling capabilities of all-attention-based transformer architectures often cause overfitting and - for natural language processing tasks - lead to an implicitly learned internal language model in the autoregressive transformer decoder complicating the integration of external language models. In this paper, we explore relaxed attention, a simple and easy-to-implement smoothing of the attention weights, yielding a two-fold improvement to the general transformer architecture: First, relaxed attention provides regularization when applied to the self-attention layers in the encoder. Second, we show that it naturally supports the integration of an external language model as it suppresses the implicitly learned internal language model by relaxing the cross attention in the decoder. We demonstrate the benefit of relaxed attention across several tasks with clear improvement in combination with recent benchmark approaches. Specifically, we exceed the former state-of-the-art performance of 26.90% word error rate on the largest public lip-reading LRS3 benchmark with a word error rate of 26.31%, as well as we achieve a top-performing BLEU score of 37.67 on the IWSLT14 (DE$\rightarrow$EN) machine translation task without external language models and virtually no additional model parameters. Code and models will be made publicly available.

cs.LG↗

arXiv2022

Antarctic Survey Telescope 3-3: Overview, System Performance and Preliminary Observations at Yaoan, Yunnan

The third Antarctic Survey Telescope array instrument at Dome A in Antarctica, the AST3-3 telescope, has been in commissioning from March 2021. We deployed AST3-3 at the Yaoan astronomical station in Yunnan Province for an automatic time-domain survey and follow-up observations with an optimised observation and protection system. The telescope system of AST3-3 is similar to that of AST3-1 and AST3-2, except that it is equipped with a 14K~$ \times$~10K QHY411 CMOS camera. AST3-3 has a field of view of $1.65^\circ \times 1.23^\circ$ and is currently using the $g$ band filter. During commissioning at Yaoan, AST3-3 aims to conduct an extragalactic transient survey, coupled with prompt follow-ups of opportunity targets. In this paper, we present the architecture of the AST3-3 automatic observation system. We demonstrate the data processing of observations by representatives SN 2022eyw and GRB 210420B.

astro-ph.IM↗

arXiv2022

Federated Split BERT for Heterogeneous Text Classification

Pre-trained BERT models have achieved impressive performance in many natural language processing (NLP) tasks. However, in many real-world situations, textual data are usually decentralized over many clients and unable to be uploaded to a central server due to privacy protection and regulations. Federated learning (FL) enables multiple clients collaboratively to train a global model while keeping the local data privacy. A few researches have investigated BERT in federated learning setting, but the problem of performance loss caused by heterogeneous (e.g., non-IID) data over clients remain under-explored. To address this issue, we propose a framework, FedSplitBERT, which handles heterogeneous data and decreases the communication cost by splitting the BERT encoder layers into local part and global part. The local part parameters are trained by the local client only while the global part parameters are trained by aggregating gradients of multiple clients. Due to the sheer size of BERT, we explore a quantization method to further reduce the communication cost with minimal performance loss. Our framework is ready-to-use and compatible to many existing federated learning algorithms, including FedAvg, FedProx and FedAdam. Our experiments verify the effectiveness of the proposed framework, which outperforms baseline methods by a significant margin, while FedSplitBERT with quantization can reduce the communication cost by $11.9\times$.

cs.CL↗

arXiv2022

The Quadratic Wasserstein Metric With Squaring Scaling For Seismic Velocity Inversion

The quadratic Wasserstein metric has shown its power in measuring the difference between probability densities, which benefits optimization objective function with better convexity and is insensitive to data noise. Nevertheless, it is always an important question to make the seismic signals suitable for comparison using the quadratic Wasserstein metric. The squaring scaling is worth exploring since it guarantees the convexity caused by data shift. However, as mentioned in [Commun. Inf. Syst., 2019, 19:95-145], the squaring scaling may lose uniqueness and result in more local minima to the misfit function. In our previous work [J. Comput. Phys., 2018, 373:188-209], the quadratic Wasserstein metric with squaring scaling was successfully applied to the earthquake location problem. But it only discussed the inverse problem with few degrees of freedom. In this work, we will present a more in-depth study on the combination of squaring scaling technique and the quadratic Wasserstein metric. By discarding some inapplicable data, picking seismic phases, and developing a new normalization method, we successfully invert the seismic velocity structure based on the squaring scaling technique and the quadratic Wasserstein metric. The numerical experiments suggest that this newly proposed method is an efficient approach to obtain more accurate inversion results.

math.NA↗

arXiv2021

Relaxed Attention: A Simple Method to Boost Performance of End-to-End Automatic Speech Recognition

Recently, attention-based encoder-decoder (AED) models have shown high performance for end-to-end automatic speech recognition (ASR) across several tasks. Addressing overconfidence in such models, in this paper we introduce the concept of relaxed attention, which is a simple gradual injection of a uniform distribution to the encoder-decoder attention weights during training that is easily implemented with two lines of code. We investigate the effect of relaxed attention across different AED model architectures and two prominent ASR tasks, Wall Street Journal (WSJ) and Librispeech. We found that transformers trained with relaxed attention outperform the standard baseline models consistently during decoding with external language models. On WSJ, we set a new benchmark for transformer-based end-to-end speech recognition with a word error rate of 3.65%, outperforming state of the art (4.20%) by 13.1% relative, while introducing only a single hyperparameter.

eess.AS↗

arXiv2021

Multi-Encoder Learning and Stream Fusion for Transformer-Based End-to-End Automatic Speech Recognition

Stream fusion, also known as system combination, is a common technique in automatic speech recognition for traditional hybrid hidden Markov model approaches, yet mostly unexplored for modern deep neural network end-to-end model architectures. Here, we investigate various fusion techniques for the all-attention-based encoder-decoder architecture known as the transformer, striving to achieve optimal fusion by investigating different fusion levels in an example single-microphone setting with fusion of standard magnitude and phase features. We introduce a novel multi-encoder learning method that performs a weighted combination of two encoder-decoder multi-head attention outputs only during training. Employing then only the magnitude feature encoder in inference, we are able to show consistent improvement on Wall Street Journal (WSJ) with language model and on Librispeech, without increase in runtime or parameters. Combining two such multi-encoder trained models by a simple late fusion in inference, we achieve state-of-the-art performance for transformer-based models on WSJ with a significant WER reduction of 19% relative compared to the current benchmark approach.

eess.AS↗

arXiv2021

Point Spread Function Estimation for Wide Field Small Aperture Telescopes with Deep Neural Networks and Calibration Data

The point spread function (PSF) reflects states of a telescope and plays an important role in development of data processing methods, such as PSF based astrometry, photometry and image restoration. However, for wide field small aperture telescopes (WFSATs), estimating PSF in any position of the whole field of view is hard, because aberrations induced by the optical system are quite complex and the signal to noise ratio of star images is often too low for PSF estimation. In this paper, we further develop our deep neural network (DNN) based PSF modelling method and show its applications in PSF estimation. During the telescope alignment and testing stage, our method collects system calibration data through modification of optical elements within engineering tolerances (tilting and decentering). Then we use these data to train a DNN (Tel--Net). After training, the Tel--Net can estimate PSF in any field of view from several discretely sampled star images. We use both simulated and experimental data to test performance of our method. The results show that the Tel--Net can successfully reconstruct PSFs of WFSATs of any states and in any positions of the FoV. Its results are significantly more precise than results obtained by the compared classic method - Inverse Distance Weight (IDW) interpolation. Our method provides foundations for developing of deep neural network based data processing methods for WFSATs, which require strong prior information of PSFs.

astro-ph.IM↗