arXiv subjects

Koichi Saito

Publications and source records attributed to Koichi Saito.

At least 37 records · Page 2Linked to original sources

arXiv2025

Aligning Text-to-Music Evaluation with Human Preferences

Despite significant recent advances in generative acoustic text-to-music (TTM) modeling, robust evaluation of these models lags behind, relying in particular on the popular Fréchet Audio Distance (FAD). In this work, we rigorously study the design space of reference-based divergence metrics for evaluating TTM models through (1) designing four synthetic meta-evaluations to measure sensitivity to particular musical desiderata, and (2) collecting and evaluating on MusicPrefs, the first open-source dataset of human preferences for TTM systems. We find that not only is the standard FAD setup inconsistent on both synthetic and human preference data, but that nearly all existing metrics fail to effectively capture desiderata, and are only weakly correlated with human perception. We propose a new metric, the MAUVE Audio Divergence (MAD), computed on representations from a self-supervised audio embedding model. We find that this metric effectively captures diverse musical desiderata (average rank correlation 0.84 for MAD vs. 0.49 for FAD and also correlates more strongly with MusicPrefs (0.62 vs. 0.14).

cs.SD↗

arXiv2025

SoundCTM: Unifying Score-based and Consistency Models for Full-band Text-to-Sound Generation

Sound content creation, essential for multimedia works such as video games and films, often involves extensive trial-and-error, enabling creators to semantically reflect their artistic ideas and inspirations, which evolve throughout the creation process, into the sound. Recent high-quality diffusion-based Text-to-Sound (T2S) generative models provide valuable tools for creators. However, these models often suffer from slow inference speeds, imposing an undesirable burden that hinders the trial-and-error process. While existing T2S distillation models address this limitation through 1-step generation, the sample quality of $1$-step generation remains insufficient for production use. Additionally, while multi-step sampling in those distillation models improves sample quality itself, the semantic content changes due to their lack of deterministic sampling capabilities. To address these issues, we introduce Sound Consistency Trajectory Models (SoundCTM), which allow flexible transitions between high-quality $1$-step sound generation and superior sound quality through multi-step deterministic sampling. This allows creators to efficiently conduct trial-and-error with 1-step generation to semantically align samples with their intention, and subsequently refine sample quality with preserving semantic content through deterministic multi-step sampling. To develop SoundCTM, we reframe the CTM training framework, originally proposed in computer vision, and introduce a novel feature distance using the teacher network for a distillation loss. For production-level generation, we scale up our model to 1B trainable parameters, making SoundCTM-DiT-1B the first large-scale distillation model in the sound community to achieve both promising high-quality 1-step and multi-step full-band (44.1kHz) generation.

cs.SD↗

arXiv2024

Novel features of asymmetric nuclear matter from terrestrial experiments and astrophysical observations of neutron stars

The accurate measurement of neutron skin thickness of $^{208}$Pb by the PREX Collaboration suggests a large value of the nuclear symmetry energy slope parameter, $L$, whereas the smaller $L$ is preferred to account for the small neutron-star radii from NICER observations. To resolve this discrepancy between nuclear experiments and astrophysical observations, new effective interactions have been developed using relativistic mean-field models with the isoscalar- and isovector-meson mixing. We investigate the effects of $δ$-nucleon coupling and $σ$--$δ$ mixing on the ground-state properties of finite nuclei, as well as the characteristics of isospin-asymmetric nuclear matter and neutron stars. Additionally, we explore the role of the quartic $ρ$-meson self-interaction in dense nuclear matter to mitigate the stiff equation of state for neutron stars resulting from the large $δ$-nucleon coupling. It is found that the nuclear symmetry energy undergoes a sudden softening at approximately twice the saturation density of nuclear matter, taking into account the PREX-2 result, the recent NICER observation of PSR J0437$-$4715, and the binary neutron star merger, GW170817.

nucl-th↗

arXiv2024

Effect of isoscalar and isovector scalar fields on baryon semileptonic decays in nuclear matter

The precise determination of the Cabibbo-Kobayashi-Maskawa (CKM) matrix elements is very important, because it could be a clue to new physics beyond Standard Theory. This is particular true of $V_{ud}$, because it is the main contribution to the unitary condition of the CKM matrix elements. The level of accuracy for the test of the unitarity involving the element $V_{ud}$ is now of the order of $10^{-4}$. Because the precise data for $V_{ud}$ is usually extracted from super-allowed nuclear $β$ decay, it is quite significant to investigate the breaking of SU(3) flavor symmetry on the weak vector coupling constant in nuclear matter. The purpose of this paper is to investigate how the isoscalar scalar ($σ$) and the isovector scalar ($δ$ or $a_0$) mean-fields affect the weak vector and axial-vector coupling constants for semileptonic baryon (neutron, $Λ$ or $Ξ^-$) decay in asymmetric nuclear matter. To do so, we use the quark-meson coupling (QMC) model, where nuclear matter consists of nucleons including quark degrees of freedom bound by the self-consistent exchange of scalar and vector mesons. We pay careful attention to the center of mass correction to the quark currents in matter. We then find that, for neutron $β$ decay in asymmetric nuclear matter, the defect of the vector coupling constant due to the $δ$ field can be of the order of $10^{-4}$ at the nuclear saturation density, which is the same amount as the level of the current uncertainty in the measurements. It is also interesting that, in neutron-rich matter, there exists a certain low density at which isospin symmetry is restored, that is, the $u$-$d$ quark mass difference vanishes. We conclude that the effect of the isoscalar scalar and the isovector scalar fields should be considered in baryon semileptonic decays in nuclei.

hep-ph↗

arXiv2024

DisMix: Disentangling Mixtures of Musical Instruments for Source-level Pitch and Timbre Manipulation

Existing work on pitch and timbre disentanglement has been mostly focused on single-instrument music audio, excluding the cases where multiple instruments are presented. To fill the gap, we propose DisMix, a generative framework in which the pitch and timbre representations act as modular building blocks for constructing the melody and instrument of a source, and the collection of which forms a set of per-instrument latent representations underlying the observed mixture. By manipulating the representations, our model samples mixtures with novel combinations of pitch and timbre of the constituent instruments. We can jointly learn the disentangled pitch-timbre representations and a latent diffusion transformer that reconstructs the mixture conditioned on the set of source-level representations. We evaluate the model using both a simple dataset of isolated chords and a realistic four-part chorales in the style of J.S. Bach, identify the key components for the success of disentanglement, and demonstrate the application of mixture transformation based on source-level attribute manipulation.

cs.SD↗

arXiv2024

SpecMaskGIT: Masked Generative Modeling of Audio Spectrograms for Efficient Audio Synthesis and Beyond

Recent advances in generative models that iteratively synthesize audio clips sparked great success to text-to-audio synthesis (TTA), but with the cost of slow synthesis speed and heavy computation. Although there have been attempts to accelerate the iterative procedure, high-quality TTA systems remain inefficient due to hundreds of iterations required in the inference phase and large amount of model parameters. To address the challenges, we propose SpecMaskGIT, a light-weighted, efficient yet effective TTA model based on the masked generative modeling of spectrograms. First, SpecMaskGIT synthesizes a realistic 10s audio clip by less than 16 iterations, an order-of-magnitude less than previous iterative TTA methods. As a discrete model, SpecMaskGIT outperforms larger VQ-Diffusion and auto-regressive models in the TTA benchmark, while being real-time with only 4 CPU cores or even 30x faster with a GPU. Next, built upon a latent space of Mel-spectrogram, SpecMaskGIT has a wider range of applications (e.g., the zero-shot bandwidth extension) than similar methods built on the latent wave domain. Moreover, we interpret SpecMaskGIT as a generative extension to previous discriminative audio masked Transformers, and shed light on its audio representation learning potential. We hope our work inspires the exploration of masked audio modeling toward further diverse scenarios.

cs.SD↗

arXiv2023

VRDMG: Vocal Restoration via Diffusion Posterior Sampling with Multiple Guidance

Restoring degraded music signals is essential to enhance audio quality for downstream music manipulation. Recent diffusion-based music restoration methods have demonstrated impressive performance, and among them, diffusion posterior sampling (DPS) stands out given its intrinsic properties, making it versatile across various restoration tasks. In this paper, we identify that there are potential issues which will degrade current DPS-based methods' performance and introduce the way to mitigate the issues inspired by diverse diffusion guidance techniques including the RePaint (RP) strategy and the Pseudoinverse-Guided Diffusion Models ($Π$GDM). We demonstrate our methods for the vocal declipping and bandwidth extension tasks under various levels of distortion and cutoff frequency, respectively. In both tasks, our methods outperform the current DPS-based music restoration benchmarks. We refer to \url{http://carlosholivan.github.io/demos/audio-restoration-2023.html} for examples of the restored audio samples.

eess.AS↗

arXiv2023

GibbsDDRM: A Partially Collapsed Gibbs Sampler for Solving Blind Inverse Problems with Denoising Diffusion Restoration

Pre-trained diffusion models have been successfully used as priors in a variety of linear inverse problems, where the goal is to reconstruct a signal from noisy linear measurements. However, existing approaches require knowledge of the linear operator. In this paper, we propose GibbsDDRM, an extension of Denoising Diffusion Restoration Models (DDRM) to a blind setting in which the linear measurement operator is unknown. GibbsDDRM constructs a joint distribution of the data, measurements, and linear operator by using a pre-trained diffusion model for the data prior, and it solves the problem by posterior sampling with an efficient variant of a Gibbs sampler. The proposed method is problem-agnostic, meaning that a pre-trained diffusion model can be applied to various inverse problems without fine-tuning. In experiments, it achieved high performance on both blind image deblurring and vocal dereverberation tasks, despite the use of simple generic priors for the underlying linear operators.

cs.LG↗

arXiv2023

Can the PREX-2 and CREX results be understood by relativistic mean-field models with the astrophysical constraints?

We construct new effective interactions using the relativistic mean-field models with the isoscalar- and isovector-meson mixing, $σ^{2}\bmδ^{2}$ and $ω_μω^μ\bmρ_ν\bmρ^ν$. Taking into account the particle flow data in heavy-ion collisions, the observed mass of PSR J0740$+$6620, and the tidal deformability of a neutron star from binary merger events, GW170817 and GW190814, we study the ground-state properties of finite, closed-shell nuclei, and try to explain the recent results from the PREX-2 and CREX experiments. It is found that the $σ$--$δ$ mixing is very powerful to understand the terrestrial experiments and astrophysical observations of neutron stars self-consistently. We can predict the large neutron skin thickness of $^{208}$Pb, $R_{\rm skin}^{208}=0.243$~fm, using the slope parameter of nuclear symmetry energy, $L=70$~MeV, which is consistent with the PREX-2 result. However, to explain the CREX data, it is preferable to adopt the small value of $L=20$~MeV. It is very difficult to understand the PREX-2 and CREX results simultaneously within relativistic mean-field models.

nucl-th↗

arXiv2022

Unsupervised vocal dereverberation with diffusion-based generative models

Removing reverb from reverberant music is a necessary technique to clean up audio for downstream music manipulations. Reverberation of music contains two categories, natural reverb, and artificial reverb. Artificial reverb has a wider diversity than natural reverb due to its various parameter setups and reverberation types. However, recent supervised dereverberation methods may fail because they rely on sufficiently diverse and numerous pairs of reverberant observations and retrieved data for training in order to be generalizable to unseen observations during inference. To resolve these problems, we propose an unsupervised method that can remove a general kind of artificial reverb for music without requiring pairs of data for training. The proposed method is based on diffusion models, where it initializes the unknown reverberation operator with a conventional signal processing technique and simultaneously refines the estimate with the help of diffusion models. We show through objective and perceptual evaluations that our method outperforms the current leading vocal dereverberation benchmarks.

eess.AS↗

arXiv2022

Massive neutron stars with small radii in relativistic mean-field models optimized to nuclear ground states

We present an equation of state (EoS) for neutron stars using the relativistic mean-field model with isoscalar- and isovector-meson mixing. Taking into account the results of the neutron skin thickness, $R_{\rm skin}$, of $^{208}$Pb reported by the PREX collaboration, the dimensionless tidal deformability of a canonical neutron star observed from GW170817, and a $2.6$ $M_{\odot}$ compact star implied by the secondary component of GW190814, a new effective interaction is constructed so as to reproduce the saturation condition of nuclear matter and the ground-state properties of finite, closed-shell nuclei. We find that the neutron star EoS exhibits the rapid stiffening around twice the nuclear saturation density, which is caused by the soft nuclear symmetry energy, $E_{\rm sym}$. It is also noticeable that the thick $R_{\rm skin}$ from the PREX-2 experiment can be achieved with the small slope parameter of $E_{\rm sym}$ stemming from the isoscalar-meson mixing. Thus, we speculate that the secondary component of GW190814 is the heaviest neutron star ever discovered.

nucl-th↗

arXiv2022

Asymmetric nuclear matter in relativistic mean-field models with isoscalar- and isovector-meson mixing

Using the relativistic mean-field model with nonlinear couplings between the isoscalar and isovector mesons, we study the properties of isospin-asymmetric nuclear matter. Not only the vector mixing, $ω_μω^μ\mathbfρ_ν\mathbfρ^ν$, but also the quartic interaction due to the scalar mesons, $σ^{2}\mathbfδ^{2}$, is taken into account to investigate the density dependence of nuclear symmetry energy, $E_{\rm sym}$, and the neutron-star properties. It is found that the $δ$ meson increases $E_{\rm sym}$ at high densities, whereas the $σ$-$δ$ mixing makes $E_{\rm sym}$ soft above the saturation density. Furthermore, the $δ$ meson and its mixing have a large influence on the radius and tidal deformability of a neutron star. In particular, the $σ$-$δ$ mixing reduces the neutron-star radius, and, thus, the present calculation can simultaneously reproduce the dimensionless tidal deformabilities of a canonical $1.4M_{\odot}$ neutron star observed from the binary neutron star merger, GW170817, and from the compact binary coalescence, GW190814.

nucl-th↗

arXiv2021

Training Speech Enhancement Systems with Noisy Speech Datasets

Recently, deep neural network (DNN)-based speech enhancement (SE) systems have been used with great success. During training, such systems require clean speech data - ideally, in large quantity with a variety of acoustic conditions, many different speaker characteristics and for a given sampling rate (e.g., 48kHz for fullband SE). However, obtaining such clean speech data is not straightforward - especially, if only considering publicly available datasets. At the same time, a lot of material for automatic speech recognition (ASR) with the desired acoustic/speaker/sampling rate characteristics is publicly available except being clean, i.e., it also contains background noise as this is even often desired in order to have ASR systems that are noise-robust. Hence, using such data to train SE systems is not straightforward. In this paper, we propose two improvements to train SE systems on noisy speech data. First, we propose several modifications of the loss functions, which make them robust against noisy speech targets. In particular, computing the median over the sample axis before averaging over time-frequency bins allows to use such data. Furthermore, we propose a noise augmentation scheme for mixture-invariant training (MixIT), which allows using it also in such scenarios. For our experiments, we use the Mozilla Common Voice dataset and we show that using our robust loss function improves PESQ by up to 0.19 compared to a system trained in the traditional way. Similarly, for MixIT we can see an improvement of up to 0.27 in PESQ when using our proposed noise augmentation.

eess.AS↗

arXiv2021

Sampling-Frequency-Independent Audio Source Separation Using Convolution Layer Based on Impulse Invariant Method

Audio source separation is often used as preprocessing of various applications, and one of its ultimate goals is to construct a single versatile model capable of dealing with the varieties of audio signals. Since sampling frequency, one of the audio signal varieties, is usually application specific, the preceding audio source separation model should be able to deal with audio signals of all sampling frequencies specified in the target applications. However, conventional models based on deep neural networks (DNNs) are trained only at the sampling frequency specified by the training data, and there are no guarantees that they work with unseen sampling frequencies. In this paper, we propose a convolution layer capable of handling arbitrary sampling frequencies by a single DNN. Through music source separation experiments, we show that the introduction of the proposed layer enables a conventional audio source separation model to consistently work with even unseen sampling frequencies.

cs.SD↗

arXiv2021

U(1) spin Chern-Simons theory and Arf invariants in two dimensions

The level-k U(1) Chern-Simons theory is a spin topological quantum field theory for k odd. Its dynamics is captured by the 2d CFT of a compact boson with a certain radius. Recently it was recognized that a dependence on the 2d spin structure can be given to the CFT by modifying it using the so-called Arf invariant. We demonstrate that one can reorganize the torus partition function of the modified CFT into a finite sum involving a finite number of conformal blocks. This allows us to reproduce the modular matrices of the spin theory. We use the modular matrices to calculate the partition function of the spin Chern-Simons theory on the lens space $L(a,\pm 1)$, and demonstrate the expected dependence on the 3d spin structure.

hep-th↗

arXiv2020

Constraints on Nuclear Saturation Properties from Terrestrial Experiments and Astrophysical Observations of Neutron Stars

Taking into account the terrestrial experiments and the recent astrophysical observations of neutron stars and gravitational-wave signals, we impose restrictions on the equation of state (EoS) for isospin-asymmetric nuclear matter. Using the relativistic mean-field model with SU(3) flavor symmetry, we investigate the impacts of effective nucleon mass, nuclear incompressibility, and slope parameter of nuclear symmetry energy on the nuclear and neutron-star properties. It is found that the astrophysical information of massive neutron stars and tidal deformabilities as well as the nuclear experimental data plays an important role to restrict the EoS for neutron stars. Especially, the softness of the nuclear EoS due to the existence of hyperons in the core gives stringent constraints on those physical quantities. Furthermore, it is possible to put limits on the curvature parameter of nuclear symmetry energy by means of the nuclear and astrophysical calculations.

nucl-th↗

arXiv2019

Equation of State for Neutron Stars in the Quark-Meson Coupling Model with the Cloudy Bag

Using the quark-meson coupling model with the cloudy bag, we construct the equation of state for neutron stars with hyperons in SU(3) flavor symmetry. The hyperfine interaction due to the gluon exchange and the pion-cloud effect inside a baryon is taken into account in vacuum and nuclear matter. We investigate how the quark degrees of freedom and the exchanges of gluon and pion between two quarks affect the properties of nuclear and neutron-star matter. It is found that the variation of baryon substructure in matter plays an important role in supporting massive neutron stars.

nucl-th↗

arXiv2019

Fock contributions to nuclear symmetry energy and its slope parameter based on Lorentz-covariant decomposition of nucleon self-energies

Using relativistic Hartree-Fock (RHF) approximation, we study the effect of Fock terms on the nuclear properties not only around the saturation density, $ρ_{0}$, but also at higher densities. In particular, we investigate how the momentum dependence due to the exchange contribution affects the nuclear symmetry energy and its slope parameter, using the Lorentz-covariant decomposition of nucleon self-energies in an extended version of the RHF model, in which the exchange terms are adjusted so as to reproduce the single-nucleon potential at $ρ_{0}$. We find that the Fock contribution suppresses the kinetic term of nuclear symmetry energy at the densities around and beyond $ρ_{0}$. It is noticeable that not only the isovector-vector ($ρ$) meson but also the isoscalar mesons ($σ, ω$) and pion make significant influence on the potential term of nuclear symmetry energy through the exchange diagrams. Furthermore, the exchange contribution prevents the slope parameter from increasing monotonically at high densities.

nucl-th↗