arXiv subjects

Yuto Kondo

Publications and source records attributed to Yuto Kondo.

13 recordsLinked to original sources

arXiv2026

MeanVoiceFlow: One-step Nonparallel Voice Conversion with Mean Flows

In voice conversion (VC) applications, diffusion and flow-matching models have exhibited exceptional speech quality and speaker similarity performances. However, they are limited by slow conversion owing to their iterative inference. Consequently, we propose MeanVoiceFlow, a novel one-step nonparallel VC model based on mean flows, which can be trained from scratch without requiring pretraining or distillation. Unlike conventional flow matching that uses instantaneous velocity, mean flows employ average velocity to more accurately compute the time integral along the inference path in a single step. However, training the average velocity requires its derivative to compute the target velocity, which can cause instability. Therefore, we introduce a structural margin reconstruction loss as a zero-input constraint, which moderately regularizes the input-output behavior of the model without harmful statistical averaging. Furthermore, we propose conditional diffused-input training in which a mixture of noise and source data is used as input to the model during both training and inference. This enables the model to effectively leverage source information while maintaining consistency between training and inference. Experimental results validate the effectiveness of these techniques and demonstrate that MeanVoiceFlow achieves performance comparable to that of previous multi-step and distillation-based models, even when trained from scratch. Audio samples are available at https://www.kecl.ntt.co.jp/people/kaneko.takuhiro/projects/meanvoiceflow/.

cs.SD↗

arXiv2026

Morphological Evolution of Nickel-Fullerene Thin Film Mixtures

Hybrid systems consisting of metal-fullerene composites exhibit intriguing properties but often suffer from thermal instability. With proper control, such instability can be harnessed to enable the formation of sophisticated nanostructures with nanometric precision. These self-organization phenomena are not limited to thermal stimulation alone but can also be triggered by other external stimuli. In this work, we investigate the morphological evolution of thin films composed of evaporated C60 and sputtered nickel mixtures, focusing on how external stimuli influence both their structural and electrical properties. Thin films were prepared under controlled deposition conditions, and their surface morphology was analyzed using advanced characterization techniques. Progressive changes in film morphology were observed as a function of composition and external treatment, highlighting the interplay between metallic and molecular components. In particular, it was observed that, due to the annealing treatment, the sample undergoes strong phase separation, with the formation of structures tens of microns in diameter and an increase in electrical resistance, exhibiting insulating behavior. These findings provide insights into the mechanisms governing hybrid thin film formation and suggest potential applications in electronic, optoelectronic, and energy-related devices.

cond-mat.mtrl-sci↗

arXiv2026

Effects of pulsed and continuous light and heavy ion irradiation on the morphology and electrical properties of Ag+C60 and Au+C60 composite thin films

Metal - organic nanocomposite thin films represent a versatile class of materials whose properties can be effectively tuned through external stimuli. In this study, Ag+C60 and Au+C60 nanocomposite thin films were briefly investigated to elucidate the effects of ion irradiation on both their morphology and electrical properties. The films were synthesized by co-deposition of noble metals and fullerenes, using ion beam sputtering of metal targets combined with simultaneous thermal evaporation of C60. The as - deposited films were characterized by ion beam analysis to determine their composition and element depth distributions. Subsequently, the samples were irradiated at room temperature with either a continuous Ar ion beam or a pulsed C ion beam, both at an energy of 20 keV and a fluence of 1 x 1015 ions/cm2. Irradiation-induced morphological changes were examined by scanning electron microscopy. While the C-irradiated films retained compact and homogeneous surface morphologies, Ar irradiation induced pronounced surface restructuring, resulting in highly corrugated and porous-like surfaces. In addition to morphology, the electrical resistance of the films was measured. The results indicate that C-irradiated samples exhibit only minor changes in resistivity, whereas Ar irradiation strongly affects the electrical properties, with the most significant impact observed for the Au+C60 system. The observed changes in electrical resistance closely correlate with the irradiation-induced surface morphology. The measurement results are briefly discussed below.

cond-mat.mtrl-sci↗

arXiv2026

Hybrid films of Co - C60 preparation and changes induced by external stimuli

In this work, we report on the study on organic-metal hybrid systems, in particular Co-C60 fullerene thin films. This study mainly focused on the investigation of the morphological and structural evolution of the film surface after various external stimuli designed to provide energy to the system. For film growth, we adopted an innovative approach, combining ion-beam sputtering of a pure metal target with thermal evaporation of C60 in a co-deposition setup. The films underwent a series of treatments to induce modifications. Laser and ion irradiations were performed using a pulsed laser, a continuous Ar beam, and a pulsed C beam. In addition, thermal annealing in vacuum was performed to examine the long-term effects of temperature. The composition of deposited film was investigated using Ion Beam Analysis, the morphology and the structure, and the effects of treatments on the films were studied using SEM and TEM microscopies and Raman spectroscopy. Changes in electrical resistance were also measured to explore potential applications of these films after treatment.

cond-mat.mtrl-sci↗

arXiv2025

LatentVoiceGrad: Nonparallel Voice Conversion with Latent Diffusion/Flow-Matching Models

Previously, we introduced VoiceGrad, a nonparallel voice conversion (VC) technique enabling mel-spectrogram conversion from source to target speakers using a score-based diffusion model. The concept involves training a score network to predict the gradient of the log density of mel-spectrograms from various speakers. VC is executed by iteratively adjusting an input mel-spectrogram until resembling the target speaker's. However, challenges persist: audio quality needs improvement, and conversion is slower compared to modern VC methods designed to operate at very high speeds. To address these, we introduce latent diffusion models into VoiceGrad, proposing an improved version with reverse diffusion in the autoencoder bottleneck. Additionally, we propose using a flow matching model as an alternative to the diffusion model to further speed up the conversion process without compromising the conversion quality. Experimental results show enhanced speech quality and accelerated conversion compared to the original.

cs.SD↗

arXiv2025

FasterVoiceGrad: Faster One-step Diffusion-Based Voice Conversion with Adversarial Diffusion Conversion Distillation

A diffusion-based voice conversion (VC) model (e.g., VoiceGrad) can achieve high speech quality and speaker similarity; however, its conversion process is slow owing to iterative sampling. FastVoiceGrad overcomes this limitation by distilling VoiceGrad into a one-step diffusion model. However, it still requires a computationally intensive content encoder to disentangle the speaker's identity and content, which slows conversion. Therefore, we propose FasterVoiceGrad, a novel one-step diffusion-based VC model obtained by simultaneously distilling a diffusion model and content encoder using adversarial diffusion conversion distillation (ADCD), where distillation is performed in the conversion process while leveraging adversarial and score distillation training. Experimental evaluations of one-shot VC demonstrated that FasterVoiceGrad achieves competitive VC performance compared to FastVoiceGrad, with 6.6-6.9 and 1.8 times faster speed on a GPU and CPU, respectively.

cs.SD↗

arXiv2025

Vocoder-Projected Feature Discriminator

In text-to-speech (TTS) and voice conversion (VC), acoustic features, such as mel spectrograms, are typically used as synthesis or conversion targets owing to their compactness and ease of learning. However, because the ultimate goal is to generate high-quality waveforms, employing a vocoder to convert these features into waveforms and applying adversarial training in the time domain is reasonable. Nevertheless, upsampling the waveform introduces significant time and memory overheads. To address this issue, we propose a vocoder-projected feature discriminator (VPFD), which uses vocoder features for adversarial training. Experiments on diffusion-based VC distillation demonstrated that a pretrained and frozen vocoder feature extractor with a single upsampling step is necessary and sufficient to achieve a VC performance comparable to that of waveform discriminators while reducing the training time and memory consumption by 9.6 and 11.4 times, respectively.

cs.SD↗

arXiv2025

Learning to assess subjective impressions from speech

We tackle a new task of training neural network models that can assess subjective impressions conveyed through speech and assign scores accordingly, inspired by the work on automatic speech quality assessment (SQA). Speech impressions are often described using phrases like `cute voice.' We define such phrases as subjective voice descriptors (SVDs). Focusing on the difference in usage scenarios between the proposed task and automatic SQA, we design a framework capable of accommodating SVDs personalized to each individual, such as `my favorite voice.' In this work, we compiled a dataset containing speech labels derived from both abosolute category ratings (ACR) and comparison category ratings (CCR). As an evaluation metric for assessment performance, we introduce ppref, the accuracy of the predicted score ordering of two samples on CCR test samples. Alongside the conventional model and learning methods based on ACR data, we also investigated RankNet learning using CCR data. We experimentally find that the ppref is moderate even with very limited training data. We also discover the CCR training is superior to the ACR training. These results support the idea that assessment models based on personalized SVDs, which typically must be trained on limited data, can be effectively learned from CCR data.

cs.SD↗

arXiv2025

JIS: A Speech Corpus of Japanese Idol Speakers with Various Speaking Styles

We construct Japanese Idol Speech Corpus (JIS) to advance research in speech generation AI, including text-to-speech synthesis (TTS) and voice conversion (VC). JIS will facilitate more rigorous evaluations of speaker similarity in TTS and VC systems since all speakers in JIS belong to a highly specific category: "young female live idols" in Japan, and each speaker is identified by a stage name, enabling researchers to recruit listeners familiar with these idols for listening experiments. With its unique speaker attributes, JIS will foster compelling research, including generating voices tailored to listener preferences-an area not yet widely studied. JIS will be distributed free of charge to promote research in speech generation AI, with usage restricted to non-commercial, basic research. We describe the construction of JIS, provide an overview of Japanese live idol culture to support effective and ethical use of JIS, and offer a basic analysis to guide application of JIS.

cs.SD↗

arXiv2025

Rethinking Mean Opinion Scores in Speech Quality Assessment: Aggregation through Quantized Distribution Fitting

Speech quality assessment (SQA) aims to evaluate the quality of speech samples without relying on time-consuming listener questionnaires. Recent efforts have focused on training neural-based SQA models to predict the mean opinion score (MOS) of speech samples produced by text-to-speech or voice conversion systems. This paper targets the enhancement of MOS prediction models' performance. We propose a novel score aggregation method to address the limitations of conventional annotations for MOS, which typically involve ratings on a scale from 1 to 5. Our method is based on the hypothesis that annotators internally consider continuous scores and then choose the nearest discrete rating. By modeling this process, we approximate the generative distribution of ratings by quantizing the latent continuous distribution. We then use the peak of this latent distribution, estimated through the loss between the quantized distribution and annotated ratings, as a new representative value instead of MOS. Experimental results demonstrate that substituting MOSNet's predicted target with this proposed value improves prediction performance.

cs.SD↗

arXiv2025

Selecting N-lowest scores for training MOS prediction models

The automatic speech quality assessment (SQA) has been extensively studied to predict the speech quality without time-consuming questionnaires. Recently, neural-based SQA models have been actively developed for speech samples produced by text-to-speech or voice conversion, with a primary focus on training mean opinion score (MOS) prediction models. The quality of each speech sample may not be consistent across the entire duration, and it remains unclear which segments of the speech receive the primary focus from humans when assigning subjective evaluation for MOS calculation. We hypothesize that when humans rate speech, they tend to assign more weight to low-quality speech segments, and the variance in ratings for each sample is mainly due to accidental assignment of higher scores when overlooking the poor quality speech segments. Motivated by the hypothesis, we analyze the VCC2018 and BVCC datasets. Based on the hypothesis, we propose the more reliable representative value N_low-MOS, the mean of the $N$-lowest opinion scores. Our experiments show that LCC and SRCC improve compared to regular MOS when employing N_low-MOS to MOSNet training. This result suggests that N_low-MOS is a more intrinsic representative value of subjective speech quality and makes MOSNet a better comparator of VC models.

cs.SD↗

arXiv2024

FastVoiceGrad: One-step Diffusion-Based Voice Conversion with Adversarial Conditional Diffusion Distillation

Diffusion-based voice conversion (VC) techniques such as VoiceGrad have attracted interest because of their high VC performance in terms of speech quality and speaker similarity. However, a notable limitation is the slow inference caused by the multi-step reverse diffusion. Therefore, we propose FastVoiceGrad, a novel one-step diffusion-based VC that reduces the number of iterations from dozens to one while inheriting the high VC performance of the multi-step diffusion-based VC. We obtain the model using adversarial conditional diffusion distillation (ACDD), leveraging the ability of generative adversarial networks and diffusion models while reconsidering the initial states in sampling. Evaluations of one-shot any-to-any VC demonstrate that FastVoiceGrad achieves VC performance superior to or comparable to that of previous multi-step diffusion-based VC while enhancing the inference speed. Audio samples are available at https://www.kecl.ntt.co.jp/people/kaneko.takuhiro/projects/fastvoicegrad/.

cs.SD↗

arXiv2021

Deficient Basis Estimation of Noise Spatial Covariance Matrix for Rank-Constrained Spatial Covariance Matrix Estimation Method in Blind Speech Extraction

Rank-constrained spatial covariance matrix estimation (RCSCME) is a state-of-the-art blind speech extraction method applied to cases where one directional target speech and diffuse noise are mixed. In this paper, we proposed a new algorithmic extension of RCSCME. RCSCME complements a deficient one rank of the diffuse noise spatial covariance matrix, which cannot be estimated via preprocessing such as independent low-rank matrix analysis, and estimates the source model parameters simultaneously. In the conventional RCSCME, a direction of the deficient basis is fixed in advance and only the scale is estimated; however, the candidate of this deficient basis is not unique in general. In the proposed RCSCME model, the deficient basis itself can be accurately estimated as a vector variable by solving a vector optimization problem. Also, we derive new update rules based on the EM algorithm. We confirm that the proposed method outperforms conventional methods under several noise conditions.

cs.SD↗