arXiv subjects

Nian Shao

Publications and source records attributed to Nian Shao.

At least 19 recordsLinked to original sources

arXiv2026

Accelerating preconditioned Jacobi methods via perturbation-inspired pivoting

Perturbation theory for symmetric matrices shows that eigenvalues with small spectral gaps are more sensitive to off-diagonal perturbation, implying that different entries affect the eigenvalues unevenly. Building on this insight, we incorporate spectral gap information into the Jacobi method for symmetric eigenvalue problems and propose a new pivoting strategy, which is completely different from classical ones governed solely by the magnitude of the off-diagonal entries. When combined with a mixed-precision preconditioner that diagonalizes the matrix to low precision, numerical experiments demonstrate that the resulting strategy can significantly outperform the classical greedy approach when the original matrix has clustered eigenvalues.

math.NA↗

arXiv2026

Convergence analysis of a nonlinear eigensolver based on rational approximation of the resolvent

Given a holomorphic matrix-valued function, the poles of its sketched resolvent are generically its eigenvalues. Once a good rational approximation of the sketched resolvent is obtained, the poles of this rational approximation typically lie close to those eigenvalues, thus providing a flexible framework for solving both linear and nonlinear eigenvalue problems. However, the accuracy of the computed eigenvalues is limited and remains poorly understood. This paper analyzes the convergence of this approach and demonstrates the effectiveness of two techniques to improve accuracy: block probing and zooming in. We also establish the backward and forward stability of polefinding for a barycentric rational form via a generalized eigenproblem. Numerical experiments demonstrate the sharpness of our theoretical results.

math.NA↗

arXiv2026

Semi-Supervised Sound Event Detection with Conditional Mixup and Embedding-Level Contrastive Loss

Sound event detection (SED) is a core module for acoustic environmental analysis, yet its performance is often limited by scarce labeled data. Recent systems leverage large pretrained audio foundation models, but effective fine-tuning remains challenging because labeled data are limited while unlabeled data are abundant. A previous work, ATST-SED, addressed this problem with a pseudo-label based semi-supervised fine-tuning framework. In this work, we further improve the framework by adopting an embedding-level self-supervised contrastive loss inspired by ATST-Frame pretraining. This contrastive objective better exploits unlabeled data during fine-tuning. One challenge is that mixup serves different roles in the two objectives: pseudo-label learning uses composition mixup, while contrastive learning treats mixup as a perturbation. To resolve this mismatch, we propose conditional mixup, which combines composition mixup and perturbation mixup in one semi-supervised framework and defines the corresponding embedding-level contrastive losses. The resulting model achieves 0.645 PSDS1 and 0.822 PSDS2 on the DESED validation set, establishing a new state of the art.

eess.AS↗

arXiv2026

Linear convergence of iterative contour integral-based eigensolvers for nonlinear eigenvalue problems

Solving nonlinear eigenvalue problems is an important and challenging task in scientific computing. Contour integral-based approaches are attractive for such eigenvalue problems because they reliably target all eigenvalues in a prescribed domain. However, unlike in the linear case, many traditional methods of this type, such as Beyn's method, lack an inherent iterative refinement mechanism. Consequently, achieving high accuracy requires high-quality quadrature rules for approximating the contour integral, which often leads to prohibitive computational costs. A notable exception is the so-called NLFEAST algorithm, which combines contour integral techniques with a nonlinear Rayleigh--Ritz extraction step. In this work, we propose a general framework of iterative contour integral-based methods for nonlinear eigenvalue problems that includes NLFEAST. This allows us to prove linear convergence of NLFEAST under mild assumptions and also explains why certain nonlinear eigensolvers do not combine well with iterative methods. Numerical experiments confirm our theoretical findings; in particular that NLFEAST can achieve high accuracy even with a limited number of quadrature nodes, significantly outperforming Beyn's method on challenging problems.

math.NA↗

arXiv2026

Stabilizing the Rayleigh--Ritz procedure by randomization

Extracting approximate eigenpairs from a prescribed subspace is of fundamental importance in eigenvalue computation. While projecting the target eigenvector onto the subspace yields satisfactory accuracy, extracting an approximate eigenpair that attains a comparable convergence rate has remained a long-standing open problem. Although the standard Rayleigh--Ritz procedure is widely used for this purpose, it may suffer from deteriorated convergence of Ritz values and may even fail to produce convergent Ritz vectors. In this paper, we address this long-standing open problem by introducing a randomized Rayleigh--Ritz procedure whose output converges at a rate similar to the ideal projection. Our analysis requires only the simplicity of the target eigenvalue and extends naturally to nonlinear eigenvalue problems.

math.NA↗

arXiv2026

Lanczos with compression for symmetric eigenvalue problems

The Lanczos method with implicit restarting is one of the most popular methods for finding a few exterior eigenpairs of a large symmetric matrix $A$. Usually based on polynomial filtering, restarting is crucial to limit memory and the cost of orthogonalization. In this work, we propose a novel strategy for the same purpose, called Lanczos with compression. Unlike polynomial filtering, our approach compresses the Krylov subspace using rational approximation and, in doing so, it sacrifices the structure of the associated Krylov decomposition. Nevertheless, it remains compatible with subsequent Lanczos steps and the overall algorithm is still solely based on matrix-vector products with $A$. On the theoretical side, we show that compression introduces only a small error compared to standard (unrestarted) Lanczos and therefore has only a negligible impact on convergence. Comparable guarantees are not available for commonly used implicit restarting strategies, including the Krylov--Schur method. On the practical side, our numerical experiments demonstrate the potential of compression to outperform the Krylov--Schur method in terms of matrix-vector products.

math.NA↗

arXiv2026

Restoring similarity in randomized Krylov methods with applications to eigenvalue problems and matrix functions

The randomized Arnoldi process has been used in large-scale scientific computing because it produces a well-conditioned basis for the Krylov subspace more quickly than the standard Arnoldi process. However, the resulting Hessenberg matrix is generally not similar to the one produced by the standard Arnoldi process, which can lead to delays or spike-like irregularities in convergence. In this paper, we introduce a modification of the randomized Arnoldi process that restores similarity with the Hessenberg matrix generated by the standard Arnoldi process. This is accomplished by enforcing orthogonality between the last Arnoldi vector and the previously generated subspace, which requires solving only one additional least-squares problem. When applied to eigenvalue problems and matrix function evaluations, the modified randomized Arnoldi process produces approximations that are identical to those obtained with the standard Arnoldi process. Numerical experiments demonstrate that our approach is as fast as the randomized Arnoldi process and as robust as the standard Arnoldi process.

math.NA↗

arXiv2025

On a randomized small-block Lanczos method for large-scale null space computations

Computing the null space of a large sparse matrix $A$ is a challenging computational problem, especially if the nullity -- the dimension of the null space -- is not small. When applying a block Lanczos method to $A^\mathsf{T} A$ for this purpose, conventional wisdom suggests to use a block size $d$ that is not smaller than the nullity. In this work, we show how randomness can be utilized to allow for smaller $d$ without sacrificing convergence or reliability. Even $d = 1$, corresponding to the standard single-vector Lanczos method, becomes a safe choice. This is achieved by using a small random diagonal perturbation, which moves the zero eigenvalues of $A^\mathsf{T} A$ away from each other, and a random initial guess. We analyze the effect of the perturbation on the attainable quality of the null space and derive convergence results that establish robust convergence for $d=1$. As demonstrated by our numerical experiments, a smaller block size combined with restarting and partial reorthogonalization results in reduced memory requirements and computational effort. It also allows for the incremental computation of the null space, without requiring a priori knowledge of the nullity. Our algorithm is best suited for situations when the nullity of $A$ is moderate.

math.NA↗

arXiv2025

CleanMel: Mel-Spectrogram Enhancement for Improving Both Speech Quality and ASR

In this work, we propose CleanMel, a single-channel Mel-spectrogram denoising and dereverberation network for improving both speech quality and automatic speech recognition (ASR) performance. The proposed network takes as input the noisy and reverberant microphone recording and predicts the corresponding clean Mel-spectrogram. The enhanced Mel-spectrogram can be either transformed to the speech waveform with a neural vocoder or directly used for ASR. The proposed network is composed of interleaved cross-band and narrow-band processing in the Mel-frequency domain, for learning the full-band spectral pattern and the narrow-band properties of signals, respectively. Compared to linear-frequency domain or time-domain speech enhancement, the key advantage of Mel-spectrogram enhancement is that Mel-frequency presents speech in a more compact way and thus is easier to learn, which will benefit both speech quality and ASR. Experimental results on five English and one Chinese datasets demonstrate a significant improvement in both speech quality and ASR performance achieved by the proposed model.Code and audio examples of our model are available online.

eess.AS↗

arXiv2025

A structural bound for cluster robustness of randomized small-block Lanczos

The Lanczos method is a fast and memory-efficient algorithm for solving large-scale symmetric eigenvalue problems. However, its rapid convergence can deteriorate significantly when computing clustered eigenvalues due to a lack of cluster robustness. A promising strategy to enhance cluster robustness -- without substantially compromising convergence speed or memory efficiency -- is to use a random small-block initial, where the block size is greater than one but still much smaller than the cluster size. This leads to the Randomized Small-Block Lanczos (RSBL) method. Despite its empirical effectiveness, RSBL lacks the comprehensive theoretical understanding already available for single-vector and large-block variants. In this paper, we develop a structural bound that supports the cluster robustness of RSBL by leveraging tools from matrix polynomials. We identify an intrinsic theoretical challenge stemming from the non-commuting nature of matrix multiplication. To provide further insight, we propose a conjectured probabilistic bound for cluster robustness and validate it through empirical experiments. Finally, we discuss how insights into cluster robustness can enhance our understanding of RSBL for both eigenvalue computation and low-rank approximation.

math.NA↗

arXiv2024

A preconditioned inverse iteration with an improved convergence guarantee

Preconditioned eigenvalue solvers offer the possibility to incorporate preconditioners for the solution of large-scale eigenvalue problems, as they arise from the discretization of partial differential equations. The convergence analysis of such methods is intricate. Even for the relatively simple preconditioned inverse iteration (PINVIT), which targets the smallest eigenvalue of a symmetric positive definite matrix, the celebrated analysis by Neymeyr is highly nontrivial and only yields convergence if the starting vector is fairly close to the desired eigenvector. In this work, we prove a new non-asymptotic convergence result for a variant of PINVIT. Our proof proceeds by analyzing an equivalent Riemannian steepest descent method and leveraging convexity-like properties. We show a convergence rate that nearly matches the one of PINVIT. As a major benefit, we require a condition on the starting vector that tends to be less stringent. This improved global convergence property is demonstrated for two classes of preconditioners with theoretical bounds and a range of numerical experiments.

math.NA↗

arXiv2024

Riemannian Acceleration with Preconditioning for symmetric eigenvalue problems

The analysis of the acceleration behavior of gradient-based eigensolvers with preconditioning presents a substantial theoretical challenge. In this work, we present a novel framework for preconditioning on Riemannian manifolds and introduce a metric, the leading angle, to evaluate preconditioners for symmetric eigenvalue problems. We extend the locally optimal Riemannian accelerated gradient method for Riemannian convex optimization to develop the Riemannian Acceleration with Preconditioning (RAP) method for symmetric eigenvalue problems, thereby providing theoretical evidence to support its acceleration. Our analysis of the Schwarz preconditioner for elliptic eigenvalue problems demonstrates that RAP achieves a convergence rate of $1-Cκ^{-1/2}$, which is an improvement over the preconditioned steepest descent method's rate of $1-Cκ^{-1}$. The exponent in $κ^{-1/2}$ is sharp, and numerical experiments confirm our theoretical findings.

math.NA↗

arXiv2024

RealMAN: A Real-Recorded and Annotated Microphone Array Dataset for Dynamic Speech Enhancement and Localization

The training of deep learning-based multichannel speech enhancement and source localization systems relies heavily on the simulation of room impulse response and multichannel diffuse noise, due to the lack of large-scale real-recorded datasets. However, the acoustic mismatch between simulated and real-world data could degrade the model performance when applying in real-world scenarios. To bridge this simulation-to-real gap, this paper presents a new relatively large-scale Real-recorded and annotated Microphone Array speech&Noise (RealMAN) dataset. The proposed dataset is valuable in two aspects: 1) benchmarking speech enhancement and localization algorithms in real scenarios; 2) offering a substantial amount of real-world training data for potentially improving the performance of real-world applications. Specifically, a 32-channel array with high-fidelity microphones is used for recording. A loudspeaker is used for playing source speech signals (about 35 hours of Mandarin speech). A total of 83.7 hours of speech signals (about 48.3 hours for static speaker and 35.4 hours for moving speaker) are recorded in 32 different scenes, and 144.5 hours of background noise are recorded in 31 different scenes. Both speech and noise recording scenes cover various common indoor, outdoor, semi-outdoor and transportation environments, which enables the training of general-purpose speech enhancement and source localization networks. To obtain the task-specific annotations, speaker location is annotated with an omni-directional fisheye camera by automatically detecting the loudspeaker. The direct-path signal is set as the target clean speech for speech enhancement, which is obtained by filtering the source speech signal with an estimated direct-path propagation filter.

cs.SD↗

arXiv2024

EPIC: a provable accelerated Eigensolver based on Preconditioning and Implicit Convexity

This paper is concerned with the extraction of the smallest eigenvalue and the corresponding eigenvector of a symmetric positive definite matrix pencil. We reveal implicit convexity of the eigenvalue problem in Euclidean space. A provable accelerated eigensolver based on preconditioning and implicit convexity (EPIC) is proposed. Theoretical analysis shows the acceleration of EPIC with the rate of convergence resembling the expected rate of convergence of the well-known locally optimal preconditioned conjugate gradient (LOPCG). A complete proof of the expected rate of convergence of LOPCG is elusive so far. Numerical results confirm our theoretical findings of EPIC.

math.NA↗

arXiv2023

Fine-tune the pretrained ATST model for sound event detection

Sound event detection (SED) often suffers from the data deficiency problem. The recent baseline system in the DCASE2023 challenge task 4 leverages the large pretrained self-supervised learning (SelfSL) models to mitigate such restriction, where the pretrained models help to produce more discriminative features for SED. However, the pretrained models are regarded as a frozen feature extractor in the challenge baseline system and most of the challenge submissions, and fine-tuning of the pretrained models has been rarely studied. In this work, we study the fine-tuning method of the pretrained models for SED. We first introduce ATST-Frame, our newly proposed SelfSL model, to the SED system. ATST-Frame was especially designed for learning frame-level representations of audio signals and obtained state-of-the-art (SOTA) performances on a series of downstream tasks. We then propose a fine-tuning method for ATST-Frame using both (in-domain) unlabelled and labelled SED data. Our experiments show that, the proposed method overcomes the overfitting problem when fine-tuning the large pretrained network, and our SED system obtains new SOTA results of 0.587/0.812 PSDS1/PSDS2 scores on the DCASE challenge task 4 dataset.

eess.AS↗

arXiv2023

Self-supervised Audio Teacher-Student Transformer for Both Clip-level and Frame-level Tasks

Self-supervised learning (SSL) has emerged as a popular approach for learning audio representations. One goal of audio self-supervised pre-training is to transfer knowledge to downstream audio tasks, generally including clip-level and frame-level tasks. While frame-level tasks are important for fine-grained acoustic scene/event understanding, prior studies primarily evaluate on clip-level downstream tasks. In order to tackle both clip-level and frame-level tasks, this paper proposes Audio Teacher-Student Transformer (ATST), with a clip-level version (named ATST-Clip) and a frame-level version (named ATST-Frame), responsible for learning clip-level and frame-level representations, respectively. Both methods use a Transformer encoder and a teacher-student training scheme. We have carefully designed the view creation strategy for ATST-Clip and ATST-Frame. Specifically, ATST-Clip uses segment-wise data augmentations, and ATST-Frame integrates frame-wise data augmentations and masking. Experimental results show that our ATST-Frame model obtains state-of-the-art (SOTA) performances on most of the clip-level and frame-level downstream tasks. Especially, it outperforms other models by a large margin on the frame-level sound event detection task. In addition, the performance can be further improved by combining the two models through knowledge distillation. Our code is available online.

eess.AS↗

arXiv2023

Frame-wise streaming end-to-end speaker diarization with non-autoregressive self-attention-based attractors

This work proposes a frame-wise online/streaming end-to-end neural diarization (FS-EEND) method in a frame-in-frame-out fashion. To frame-wisely detect a flexible number of speakers and extract/update their corresponding attractors, we propose to leverage a causal speaker embedding encoder and an online non-autoregressive self-attention-based attractor decoder. A look-ahead mechanism is adopted to allow leveraging some future frames for effectively detecting new speakers in real time and adaptively updating speaker attractors. The proposed method processes the audio stream frame by frame, and has a low inference latency caused by the look-ahead frames. Experiments show that, compared with the recently proposed block-wise online methods, our method FS-EEND achieves state-of-the-art diarization results, with a low inference latency and computational cost.

eess.AS↗

arXiv2022

Convergence analysis of the Newton-Schur method for the symmetric elliptic eigenvalue problem

In this paper, we consider the Newton-Schur method in Hilbert space and obtain quadratic convergence. For the symmetric elliptic eigenvalue problem discretized by the standard finite element method and non-overlapping domain decomposition method, we use the Steklov-Poincaré operator to reduce the eigenvalue problem on the domain $Ω$ into the nonlinear eigenvalue subproblem on $Γ$, which is the union of subdomain boundaries. We prove that the convergence rate for the Newton-Schur method is $ε_{N}\leq CH^{2}(1+\ln(H/h))^{2}ε^{2}$, where the constant $C$ is independent of the fine mesh size $h$ and coarse mesh size $H$, and $ε_{N}$ and $ε$ are errors after and before one iteration step respectively. Numerical experiments confirm our theoretical analysis.

math.NA↗