arXiv subjects

Federico Landini

Publications and source records attributed to Federico Landini.

At least 19 recordsLinked to original sources

arXiv2026

PRICM3: A novel Polarization Ratio framework for three-dimensional CME reconstruction

Coronal mass ejections (CMEs) are inherently three-dimensional (3D), but traditional white-light coronagraphs provide only two-dimensional (2D) projections. While the Polarization Ratio Technique (PRT) utilizes Thomson scattering to infer proxy 3D plasma locations, its results are typically limited to 2D topographical maps, hindering geometric interpretation. To address this, we introduce PRICM3 (Polarization Ratio Integrated CME Mapper in 3D), a framework that converts PRT-derived maps into explicit 3D point-cloud reconstructions. This tool enables intuitive visualization, multi-viewpoint comparison, and direct validation against geometric models such as the Graduated Cylindrical Shell (GCS). As a proof-of-concept, we analyzed a limb CME observed on 28 October 2021 using data from Solar Orbiter/Metis, SOHO/LASCO, and STEREO-A/COR2. The independent reconstructions converged into a consistent 3D volume that aligned remarkably well with the corresponding GCS geometry. These results prove that the 3D information embedded in PRT topographical maps can be successfully recovered through spatial reconstruction. PRICM3 offers a powerful, intuitive method for interpreting polarimetric coronagraph observations, providing a vital tool for current and future solar missions such as Proba-3/ASPIICS and PUNCH.

astro-ph.SR↗

arXiv2025

Efficient and Generalizable Speaker Diarization via Structured Pruning of Self-Supervised Models

Self-supervised learning (SSL) models such as WavLM have substantially advanced speaker diarization by providing rich contextual speech representations. However, the high computational and memory costs of these models hinder deployment in real-time and resource-constrained scenarios. This work presents a systematic study on compressing SSL-based diarization models through structured pruning guided by knowledge distillation. We investigate pruning objectives that target both model parameters and computational complexity, and analyze alternative strategies, showing that a simple overall pruning approach provides the best balance between efficiency and accuracy. Our method achieves up to 80% model size reduction and 4x faster inference without performance degradation. Comprehensive experiments across eight public diarization datasets demonstrate that the pruned models consistently match or surpass the performance of their uncompressed counterparts. Furthermore, we show strong out-of-domain generalization on the CHiME-6 dataset, achieving accuracy comparable to the top systems in the CHiME-7 challenge without any domain adaptation. These results highlight that structured pruning, when guided by distillation, can yield efficient and generalizable diarization systems suitable for real-world applications.

eess.AS↗

arXiv2025

Fine-tune Before Structured Pruning: Towards Compact and Accurate Self-Supervised Models for Speaker Diarization

Self-supervised learning (SSL) models like WavLM can be effectively utilized when building speaker diarization systems but are often large and slow, limiting their use in resource constrained scenarios. Previous studies have explored compression techniques, but usually for the price of degraded performance at high pruning ratios. In this work, we propose to compress SSL models through structured pruning by introducing knowledge distillation. Different from the existing works, we emphasize the importance of fine-tuning SSL models before pruning. Experiments on far-field single-channel AMI, AISHELL-4, and AliMeeting datasets show that our method can remove redundant parameters of WavLM Base+ and WavLM Large by up to 80% without any performance degradation. After pruning, the inference speeds on a single GPU for the Base+ and Large models are 4.0 and 2.6 times faster, respectively. Our source code is publicly available.

eess.AS↗

arXiv2025

Analysis of ABC Frontend Audio Systems for the NIST-SRE24

We present a comprehensive analysis of the embedding extractors (frontends) developed by the ABC team for the audio track of NIST SRE 2024. We follow the two scenarios imposed by NIST: using only a provided set of telephone recordings for training (fixed) or adding publicly available data (open condition). Under these constraints, we develop the best possible speaker embedding extractors for the pre-dominant conversational telephone speech (CTS) domain. We explored architectures based on ResNet with different pooling mechanisms, recently introduced ReDimNet architecture, as well as a system based on the XLS-R model, which represents the family of large pre-trained self-supervised models. In open condition, we train on VoxBlink2 dataset, containing 110 thousand speakers across multiple languages. We observed a good performance and robustness of VoxBlink-trained models, and our experiments show practical recipes for developing state-of-the-art frontends for speaker recognition.

eess.AS↗

arXiv2025

Investigating Solar Wind Outflows from Open-Closed Magnetic Field Structures Using Coordinated Solar Orbiter and Hinode Observations

ESA/NASA's Solar Orbiter (SO) allows us to study the solar corona at closer distances and from different perspectives, which helps us to gain significant insights into the origins of the solar wind. In this work, we present the analysis of solar wind outflows from two locations: a narrow open-field corridor and a small, mid-latitude coronal hole. These outflows were observed off-limb by the Metis coronagraph onboard SO and on-disk by the Extreme Ultraviolet Imaging Spectrometer (EIS) onboard Hinode. Magnetic field extrapolations suggest that the upflow regions seen in EIS were the sources of the outflowing solar wind observed with Metis. We find that the plasma associated with the narrow open-field corridor has higher electron densities and lower outflow velocities compared to the coronal hole plasma in the middle corona, even though the plasma properties of the two source regions in the low corona are found to be relatively similar. The speed of solar wind from the open-field corridor also shows no correlation with the magnetic field expansion factor, unlike the coronal hole. These pronounced differences at higher altitudes may arise from the dynamic nature of the low-middle corona, in which reconnection can readily occur and may play an important role in driving solar wind variability.

astro-ph.SR↗

arXiv2024

DiCoW: Diarization-Conditioned Whisper for Target Speaker Automatic Speech Recognition

Speaker-attributed automatic speech recognition (ASR) in multi-speaker environments remains a significant challenge, particularly when systems conditioned on speaker embeddings fail to generalize to unseen speakers. In this work, we propose Diarization-Conditioned Whisper (DiCoW), a novel approach to target-speaker ASR that leverages speaker diarization outputs as conditioning information. DiCoW extends the pre-trained Whisper model by integrating diarization labels directly, eliminating reliance on speaker embeddings and reducing the need for extensive speaker-specific training data. Our method introduces frame-level diarization-dependent transformations (FDDT) and query-key biasing (QKb) techniques to refine the model's focus on target speakers while effectively handling overlapping speech. By leveraging diarization outputs as conditioning signals, DiCoW simplifies the workflow for multi-speaker ASR, improves generalization to unseen speakers and enables more reliable transcription in real-world multi-speaker recordings. Additionally, we explore the integration of a connectionist temporal classification (CTC) head to Whisper and demonstrate its ability to improve transcription efficiency through hybrid decoding. Notably, we show that our approach is not limited to Whisper; it also provides similar benefits when applied to the Branchformer model. We validate DiCoW on real-world datasets, including AMI and NOTSOFAR-1 from CHiME-8 challenge, as well as synthetic benchmarks such as Libri2Mix and LibriCSS, enabling direct comparisons with previous methods. Results demonstrate that DiCoW enhances the model's target-speaker ASR capabilities while maintaining Whisper's accuracy and robustness on single-speaker data.

eess.AS↗

arXiv2024

Joint Training of Speaker Embedding Extractor, Speech and Overlap Detection for Diarization

In spite of the popularity of end-to-end diarization systems nowadays, modular systems comprised of voice activity detection (VAD), speaker embedding extraction plus clustering, and overlapped speech detection (OSD) plus handling still attain competitive performance in many conditions. However, one of the main drawbacks of modular systems is the need to run (and train) different modules independently. In this work, we propose an approach to jointly train a model to produce speaker embeddings, VAD and OSD simultaneously and reach competitive performance at a fraction of the inference time of a standard approach. Furthermore, the joint inference leads to a simplified overall pipeline which brings us one step closer to a unified clustering-based method that can be trained end-to-end towards a diarization-specific objective.

eess.AS↗

arXiv2024

Leveraging Self-Supervised Learning for Speaker Diarization

End-to-end neural diarization has evolved considerably over the past few years, but data scarcity is still a major obstacle for further improvements. Self-supervised learning methods such as WavLM have shown promising performance on several downstream tasks, but their application on speaker diarization is somehow limited. In this work, we explore using WavLM to alleviate the problem of data scarcity for neural diarization training. We use the same pipeline as Pyannote and improve the local end-to-end neural diarization with WavLM and Conformer. Experiments on far-field AMI, AISHELL-4, and AliMeeting datasets show that our method substantially outperforms the Pyannote baseline and achieves new state-of-the-art results on AMI and AISHELL-4, respectively. In addition, by analyzing the system performance under different data quantity scenarios, we show that WavLM representations are much more robust against data scarcity than filterbank features, enabling less data hungry training strategies. Furthermore, we found that simulated data, usually used to train endto-end diarization models, does not help when using WavLM in our experiments. Additionally, we also evaluate our model on the recent CHiME8 NOTSOFAR-1 task where it achieves better performance than the Pyannote baseline. Our source code is publicly available at https://github.com/BUTSpeechFIT/DiariZen.

eess.AS↗

arXiv2024

From Modular to End-to-End Speaker Diarization

Speaker diarization is usually referred to as the task that determines ``who spoke when'' in a recording. Until a few years ago, all competitive approaches were modular. Systems based on this framework reached state-of-the-art performance in most scenarios but had major difficulties dealing with overlapped speech. More recently, the advent of end-to-end models, capable of dealing with all aspects of speaker diarization with a single model and better performing regarding overlapped speech, has brought high levels of attention. This thesis is framed during a period of co-existence of these two trends. We describe a system based on a Bayesian hidden Markov model used to cluster x-vectors (speaker embeddings obtained with a neural network), known as VBx, which has shown remarkable performance on different datasets and challenges. We comment on its advantages and limitations and evaluate results on different relevant corpora. Then, we move towards end-to-end neural diarization (EEND) methods. Due to the need for large training sets for training these models and the lack of manually annotated diarization data in sufficient quantities, the compromise solution consists in generating training data artificially. We describe an approach for generating synthetic data which resembles real conversations in terms of speaker turns and overlaps. We show how this method generating ``simulated conversations'' allows for better performance than using a previously proposed method for creating ``simulated mixtures'' when training the popular EEND with encoder-decoder attractors (EEND-EDA). We also propose a new EEND-based model, which we call DiaPer, and show that it can perform better than EEND-EDA, especially when dealing with many speakers and handling overlapped speech. Finally, we compare both VBx-based and DiaPer systems on a wide variety of corpora and comment on the advantages of each technique.

eess.AS↗

arXiv2024

Spoof Diarization: "What Spoofed When" in Partially Spoofed Audio

This paper defines Spoof Diarization as a novel task in the Partial Spoof (PS) scenario. It aims to determine what spoofed when, which includes not only locating spoof regions but also clustering them according to different spoofing methods. As a pioneering study in spoof diarization, we focus on defining the task, establishing evaluation metrics, and proposing a benchmark model, namely the Countermeasure-Condition Clustering (3C) model. Utilizing this model, we first explore how to effectively train countermeasures to support spoof diarization using three labeling schemes. We then utilize spoof localization predictions to enhance the diarization performance. This first study reveals the high complexity of the task, even in restricted scenarios where only a single speaker per audio file and an oracle number of spoofing methods are considered. Our code is available at https://github.com/nii-yamagishilab/PartialSpoof.

eess.AS↗

arXiv2024

Do End-to-End Neural Diarization Attractors Need to Encode Speaker Characteristic Information?

In this paper, we apply the variational information bottleneck approach to end-to-end neural diarization with encoder-decoder attractors (EEND-EDA). This allows us to investigate what information is essential for the model. EEND-EDA utilizes attractors, vector representations of speakers in a conversation. Our analysis shows that, attractors do not necessarily have to contain speaker characteristic information. On the other hand, giving the attractors more freedom to allow them to encode some extra (possibly speaker-specific) information leads to small but consistent diarization performance improvements. Despite architectural differences in EEND systems, the notion of attractors and frame embeddings is common to most of them and not specific to EEND-EDA. We believe that the main conclusions of this work can apply to other variants of EEND. Thus, we hope this paper will be a valuable contribution to guide the community to make more informed decisions when designing new systems.

cs.SD↗

arXiv2023

DiaPer: End-to-End Neural Diarization with Perceiver-Based Attractors

Until recently, the field of speaker diarization was dominated by cascaded systems. Due to their limitations, mainly regarding overlapped speech and cumbersome pipelines, end-to-end models have gained great popularity lately. One of the most successful models is end-to-end neural diarization with encoder-decoder based attractors (EEND-EDA). In this work, we replace the EDA module with a Perceiver-based one and show its advantages over EEND-EDA; namely obtaining better performance on the largely studied Callhome dataset, finding the quantity of speakers in a conversation more accurately, and faster inference time. Furthermore, when exhaustively compared with other methods, our model, DiaPer, reaches remarkable performance with a very lightweight design. Besides, we perform comparisons with other works and a cascaded baseline across more than ten public wide-band datasets. Together with this publication, we release the code of DiaPer as well as models trained on public and free data.

eess.AS↗

arXiv2023

Discriminative Training of VBx Diarization

Bayesian HMM clustering of x-vector sequences (VBx) has become a widely adopted diarization baseline model in publications and challenges. It uses an HMM to model speaker turns, a generatively trained probabilistic linear discriminant analysis (PLDA) for speaker distribution modeling, and Bayesian inference to estimate the assignment of x-vectors to speakers. This paper presents a new framework for updating the VBx parameters using discriminative training, which directly optimizes a predefined loss. We also propose a new loss that better correlates with the diarization error rate compared to binary cross-entropy $\unicode{x2013}$ the default choice for diarization end-to-end systems. Proof-of-concept results across three datasets (AMI, CALLHOME, and DIHARD II) demonstrate the method's capability of automatically finding hyperparameters, achieving comparable performance to those found by extensive grid search, which typically requires additional hyperparameter behavior knowledge. Moreover, we show that discriminative fine-tuning of PLDA can further improve the model's performance. We release the source code with this publication.

eess.AS↗

arXiv2023

DiaCorrect: Error Correction Back-end For Speaker Diarization

In this work, we propose an error correction framework, named DiaCorrect, to refine the output of a diarization system in a simple yet effective way. This method is inspired by error correction techniques in automatic speech recognition. Our model consists of two parallel convolutional encoders and a transform-based decoder. By exploiting the interactions between the input recording and the initial system's outputs, DiaCorrect can automatically correct the initial speaker activities to minimize the diarization errors. Experiments on 2-speaker telephony data show that the proposed DiaCorrect can effectively improve the initial model's results. Our source code is publicly available at https://github.com/BUTSpeechFIT/diacorrect.

eess.AS↗

arXiv2023

Particle monitoring capability of the Solar Orbiter Metis coronagraph through the increasing phase of solar cycle 25

Context. Galactic cosmic rays (GCRs) and solar particles with energies greater than tens of MeV penetrate spacecraft and instruments hosted aboard space missions. The Solar Orbiter Metis coronagraph is aimed at observing the solar corona in both visible (VL) and ultraviolet (UV) light. Particle tracks are observed in the Metis images of the corona. An algorithm has been implemented in the Metis processing electronics to detect the VL image pixels crossed by cosmic rays. This algorithm was initially enabled for the VL instrument only, since the process of separating the particle tracks in the UV images has proven to be very challenging. Aims. We study the impact of the overall bulk of particles of galactic and solar origin on the Metis coronagraph images. We discuss the effects of the increasing solar activity after the Solar Orbiter mission launch on the secondary particle production in the spacecraft. Methods. We compared Monte Carlo simulations of GCRs crossing or interacting in the Metis VL CMOS sensor to observations gathered in 2020 and 2022. We also evaluated the impact of solar energetic particle events of different intensities on the Metis images. Results. The study of the role of abundant and rare cosmic rays in firing pixels in the Metis VL images of the corona allows us to estimate the efficiency of the algorithm applied for cosmic-ray track removal from the images and to demonstrate that the instrument performance had remained unchanged during the first two years of the Solar Orbiter operations. The outcome of this work can be used to estimate the Solar Orbiter instrument's deep charging and the order of magnitude for energetic particles crossing the images of Metis and other instruments such as STIX and EUI.

astro-ph.SR↗

arXiv2023

Coronal Heating Rate in the Slow Solar Wind

This Letter reports the first observational estimate of the heating rate in the slowly expanding solar corona. The analysis exploits the simultaneous remote and local observations of the same coronal plasma volume with the Solar Orbiter/Metis and the Parker Solar Probe instruments, respectively, and relies on the basic solar wind magnetohydrodynamic equations. As expected, energy losses are a minor fraction of the solar wind energy flux, since most of the energy dissipation that feeds the heating and acceleration of the coronal flow occurs much closer to the Sun than the heights probed in the present study, which range from 6.3 to 13.3 solar radii. The energy deposited to the supersonic wind is then used to explain the observed slight residual wind acceleration and to maintain the plasma in a non-adiabatic state. As derived in the Wentzel-Kramers-Brillouin limit, the present energy transfer rate estimates provide a lower limit, which can be very useful in refining the turbulence-based modeling of coronal heating and subsequent solar wind acceleration.

astro-ph.SR↗

arXiv2023

Multi-Stream Extension of Variational Bayesian HMM Clustering (MS-VBx) for Combined End-to-End and Vector Clustering-based Diarization

Combining end-to-end neural speaker diarization (EEND) with vector clustering (VC), known as EEND-VC, has gained interest for leveraging the strengths of both methods. EEND-VC estimates activities and speaker embeddings for all speakers within an audio chunk and uses VC to associate these activities with speaker identities across different chunks. EEND-VC generates thus multiple streams of embeddings, one for each speaker in a chunk. We can cluster these embeddings using constrained agglomerative hierarchical clustering (cAHC), ensuring embeddings from the same chunk belong to different clusters. This paper introduces an alternative clustering approach, a multi-stream extension of the successful Bayesian HMM clustering of x-vectors (VBx), called MS-VBx. Experiments on three datasets demonstrate that MS-VBx outperforms cAHC in diarization and speaker counting performance.

eess.AS↗

arXiv2023

Multi-Speaker and Wide-Band Simulated Conversations as Training Data for End-to-End Neural Diarization

End-to-end diarization presents an attractive alternative to standard cascaded diarization systems because a single system can handle all aspects of the task at once. Many flavors of end-to-end models have been proposed but all of them require (so far non-existing) large amounts of annotated data for training. The compromise solution consists in generating synthetic data and the recently proposed simulated conversations (SC) have shown remarkable improvements over the original simulated mixtures (SM). In this work, we create SC with multiple speakers per conversation and show that they allow for substantially better performance than SM, also reducing the dependence on a fine-tuning stage. We also create SC with wide-band public audio sources and present an analysis on several evaluation sets. Together with this publication, we release the recipes for generating such data and models trained on public sets as well as the implementation to efficiently handle multiple speakers per conversation and an auxiliary voice activity detection loss.

eess.AS↗