arXiv subjects

Lele Liao

Publications and source records attributed to Lele Liao.

5 recordsLinked to original sources

arXiv2025

Toward a unified framework for data-efficient evaluation of large language models

Evaluating large language models (LLMs) on comprehensive benchmarks is a cornerstone of their development, yet it's often computationally and financially prohibitive. While Item Response Theory (IRT) offers a promising path toward data-efficient evaluation by disentangling model capability from item difficulty, existing IRT-based methods are hampered by significant limitations. They are typically restricted to binary correctness metrics, failing to natively handle the continuous scores used in generative tasks, and they operate on single benchmarks, ignoring valuable structural knowledge like correlations across different metrics or benchmarks. To overcome these challenges, we introduce LEGO-IRT, a unified and flexible framework for data-efficient LLM evaluation. LEGO-IRT's novel design natively supports both binary and continuous evaluation metrics. Moreover, it introduces a factorized architecture to explicitly model and leverage structural knowledge, decomposing model ability estimates into a general component and structure-specific (e.g., per-metric or per-benchmark) components. Through extensive experiments involving $70$ LLMs across $5$ benchmarks, we show that LEGO-IRT achieves stable capability estimates using just $3\%$ of the total evaluation items. We demonstrate that incorporating structural knowledge reduces estimation error by up to $10\%$ and reveal that the latent abilities estimated by our framework may align more closely with human preferences.

cs.AI↗

arXiv2022

Semi-blind source separation using convolutive transfer function for nonlinear acoustic echo cancellation

The recently proposed semi-blind source separation (SBSS) method for nonlinear acoustic echo cancellation (NAEC) outperforms adaptive NAEC in attenuating the nonlinear acoustic echo. However, the multiplicative transfer function (MTF) approximation makes it unsuitable for real-time applications especially in highly reverberant environments, and the natural gradient makes it hard to balance well between fast convergence speed and stability. In this paper, we propose two more effective SBSS methods based on auxiliary-function-based independent vector analysis (AuxIVA) and independent low-rank matrix analysis (ILRMA). The convolutive transfer function (CTF) approximation is used instead of MTF so that a long impulse response can be modeled with a short latency. The optimization schemes used in AuxIVA and ILRMA are carefully regularized according to the constrained demixing matrix of NAEC. Experimental results validate significantly better echo cancellation performance of the proposed methods.

eess.AS↗

arXiv2021

Semi-Blind Source Separation for Nonlinear Acoustic Echo Cancellation

The mismatch between the numerical and actual nonlinear models is a challenge to nonlinear acoustic echo cancellation (NAEC) when the nonlinear adaptive filter is utilized. To alleviate this problem, we combine a basis-generic expansion of the memoryless nonlinearity into semi-blind source separation (SBSS). By regarding all the basis functions of the far-end input signal as the known equivalent reference signals, an SBSS updating algorithm is derived following the constrained scaled natural gradient strategy. Unlike the commonly utilized adaptive algorithm, the proposed SBSS is based on the independence between the near-end signal and the reference signals, and is less sensitive to the mismatch of nonlinearity between the numerical and actual models. Experimental results show that the proposed method outperforms conventional methods in terms of echo return loss enhancement (ERLE) and near-end speech quality evaluated by perceptual evaluation of speech quality (PESQ) and short-time objective intelligibility (STOI).

eess.SP↗

arXiv2020

Target Speech Extraction Based on Blind Source Separation and X-vector-based Speaker Selection Trained with Data Augmentation

Extracting the desired speech from a mixture is a meaningful and challenging task. The end-to-end DNN-based methods, though attractive, face the problem of generalization. In this paper, we explore a sequential approach for target speech extraction by combining blind source separation (BSS) with the x-vector based speaker recognition (SR) module. Two promising BSS methods based on source independence assumption, independent low-rank matrix analysis (ILRMA) and multi-channel variational autoencoder (MVAE), are utilized and compared. ILRMA employs nonnegative matrix factorization (NMF) to capture spectral structures of source signals and MVAE utilizes the strong modeling power of deep neural networks (DNN). However, the investigation of MVAE has been limited to the training with very few speakers and the speech signals of test speakers are usually included. We extend the training of MVAE using clean speech signals of 500 speakers to evaluate its generalization to unseen speakers. To improve the correct extraction rate, two data augmentation strategies are implemented to train the SR module. The performance of the proposed cascaded approach is investigated with test data constructed with real room impulse responses under varied environments.

eess.AS↗

arXiv2020

Efficient Independent Vector Extraction of Dominant Target Speech

The complete decomposition performed by blind source separation is computationally demanding and superfluous when only the speech of one specific target speaker is desired. In this paper, we propose a computationally efficient blind speech extraction method based on a proper modification of the commonly utilized independent vector analysis algorithm, under the mild assumption that the average power of signal of interest outweighs interfering speech sources. Considering that the minimum distortion principle cannot be implemented since the full demixing matrix is not available, we also design a one-unit scaling operation to solve the scaling ambiguity. Simulations validate the efficacy of the proposed method in extracting the dominant speech.

cs.SD↗