arXiv subjects

Yixiao Zhang

Publications and source records attributed to Yixiao Zhang.

At least 55 records · Page 3Linked to original sources

arXiv2023

The Song Describer Dataset: a Corpus of Audio Captions for Music-and-Language Evaluation

We introduce the Song Describer dataset (SDD), a new crowdsourced corpus of high-quality audio-caption pairs, designed for the evaluation of music-and-language models. The dataset consists of 1.1k human-written natural language descriptions of 706 music recordings, all publicly accessible and released under Creative Common licenses. To showcase the use of our dataset, we benchmark popular models on three key music-and-language tasks (music captioning, text-to-music generation and music-language retrieval). Our experiments highlight the importance of cross-dataset evaluation and offer insights into how researchers can use SDD to gain a broader understanding of model performance.

cs.SD↗

arXiv2023

AI (r)evolution -- where are we heading? Thoughts about the future of music and sound technologies in the era of deep learning

Artificial Intelligence (AI) technologies such as deep learning are evolving very quickly bringing many changes to our everyday lives. To explore the future impact and potential of AI in the field of music and sound technologies a doctoral day was held between Queen Mary University of London (QMUL, UK) and Sciences et Technologies de la Musique et du Son (STMS, France). Prompt questions about current trends in AI and music were generated by academics from QMUL and STMS. Students from the two institutions then debated these questions. This report presents a summary of the student debates on the topics of: Data, Impact, and the Environment; Responsible Innovation and Creative Practice; Creativity and Bias; and From Tools to the Singularity. The students represent the future generation of AI and music researchers. The academics represent the incumbent establishment. The student debates reported here capture visions, dreams, concerns, uncertainties, and contentious issues for the future of AI and music as the establishment is rightfully challenged by the next generation.

cs.CY↗

arXiv2023

Hypergraphs with irrational Turán density and many extremal configurations

Unlike graphs, determining Turán densities of hypergraphs is known to be notoriously hard in general. The essential reason is that for many classical families of $r$-uniform hypergraphs $\mathcal{F}$, there are perhaps many near-extremal $\mathcal{M}_t$-free configurations with very different structure. Such a phenomenon is called not stable, and Liu and Mubayi gave a first not stable example. Another perhaps reason is that little is known about the set consisting of all possible Turán densities which has cardinality of the continuum. Let $t\ge 2$ be an integer. In this paper, we construct a finite family $\mathcal{M}_t$ of 3-uniform hypergraphs such that the Turán density of $\mathcal{M}_t$ is irrational, and there are $t$ near-extremal $\mathcal{M}_t$-free configurations that are far from each other in edit-distance. This is the first not stable example that has an irrational Turán density. It also provides a new phenomenon about feasible region functions.

math.CO↗

arXiv2023

CLIP-Driven Universal Model for Organ Segmentation and Tumor Detection

An increasing number of public datasets have shown a marked impact on automated organ segmentation and tumor detection. However, due to the small size and partially labeled problem of each dataset, as well as a limited investigation of diverse types of tumors, the resulting models are often limited to segmenting specific organs/tumors and ignore the semantics of anatomical structures, nor can they be extended to novel domains. To address these issues, we propose the CLIP-Driven Universal Model, which incorporates text embedding learned from Contrastive Language-Image Pre-training (CLIP) to segmentation models. This CLIP-based label encoding captures anatomical relationships, enabling the model to learn a structured feature embedding and segment 25 organs and 6 types of tumors. The proposed model is developed from an assembly of 14 datasets, using a total of 3,410 CT scans for training and then evaluated on 6,162 external CT scans from 3 additional datasets. We rank first on the Medical Segmentation Decathlon (MSD) public leaderboard and achieve state-of-the-art results on Beyond The Cranial Vault (BTCV). Additionally, the Universal Model is computationally more efficient (6x faster) compared with dataset-specific models, generalized better to CT scans from varying sites, and shows stronger transfer learning performance on novel tasks.

eess.IV↗

arXiv2023

Exploring XAI for the Arts: Explaining Latent Space in Generative Music

Explainable AI has the potential to support more interactive and fluid co-creative AI systems which can creatively collaborate with people. To do this, creative AI models need to be amenable to debugging by offering eXplainable AI (XAI) features which are inspectable, understandable, and modifiable. However, currently there is very little XAI for the arts. In this work, we demonstrate how a latent variable model for music generation can be made more explainable; specifically we extend MeasureVAE which generates measures of music. We increase the explainability of the model by: i) using latent space regularisation to force some specific dimensions of the latent space to map to meaningful musical attributes, ii) providing a user interface feedback loop to allow people to adjust dimensions of the latent space and observe the results of these changes in real-time, iii) providing a visualisation of the musical attributes in the latent space to help people understand and predict the effect of changes to latent space dimensions. We suggest that in doing so we bridge the gap between the latent space and the generated musical outcomes in a meaningful way which makes the model and its outputs more explainable and more debuggable.

cs.AI↗

arXiv2023

Continual Learning for Abdominal Multi-Organ and Tumor Segmentation

The ability to dynamically extend a model to new data and classes is critical for multiple organ and tumor segmentation. However, due to privacy regulations, accessing previous data and annotations can be problematic in the medical domain. This poses a significant barrier to preserving the high segmentation accuracy of the old classes when learning from new classes because of the catastrophic forgetting problem. In this paper, we first empirically demonstrate that simply using high-quality pseudo labels can fairly mitigate this problem in the setting of organ segmentation. Furthermore, we put forward an innovative architecture designed specifically for continuous organ and tumor segmentation, which incurs minimal computational overhead. Our proposed design involves replacing the conventional output layer with a suite of lightweight, class-specific heads, thereby offering the flexibility to accommodate newly emerging classes. These heads enable independent predictions for newly introduced and previously learned classes, effectively minimizing the impact of new classes on old ones during the course of continual learning. We further propose incorporating Contrastive Language-Image Pretraining (CLIP) embeddings into the organ-specific heads. These embeddings encapsulate the semantic information of each class, informed by extensive image-text co-training. The proposed method is evaluated on both in-house and public abdominal CT datasets under organ and tumor segmentation tasks. Empirical results suggest that the proposed design improves the segmentation performance of a baseline neural network on newly-introduced and previously-learned classes along the learning trajectory.

eess.IV↗

arXiv2023

Cloud Behaviour on Tidally Locked Rocky Planets from Global High-resolution Modeling

Determining the behaviour of convection and clouds is one of the biggest challenges in our understanding of exoplanetary climates. Given the lack of in situ observations, one of the most preferable approaches is to use cloud-resolving or cloud-permitting models (CPM). Here we present CPM simulations in a quasi-global domain with high spatial resolution (4$\times$4 km grid) and explicit convection to study the cloud regime of 1 to 1 tidally locked rocky planets orbiting around low-mass stars. We show that the substellar region is covered by deep convective clouds and cloud albedo increases with increasing stellar flux. The CPM produces relatively less cloud liquid water concentration, smaller cloud coverage, lower cloud albedo, and deeper H2O spectral features than previous general circulation model (GCM) simulations employing empirical convection and cloud parameterizations. Furthermore, cloud streets--long bands of low-level clouds oriented nearly parallel to the direction of the mean boundary-layer winds--appear in the CPM and substantially affect energy balance and surface precipitation at a local level.

astro-ph.EP↗

arXiv2023

SQUID: Deep Feature In-Painting for Unsupervised Anomaly Detection

Radiography imaging protocols focus on particular body regions, therefore producing images of great similarity and yielding recurrent anatomical structures across patients. To exploit this structured information, we propose the use of Space-aware Memory Queues for In-painting and Detecting anomalies from radiography images (abbreviated as SQUID). We show that SQUID can taxonomize the ingrained anatomical structures into recurrent patterns; and in the inference, it can identify anomalies (unseen/modified patterns) in the image. SQUID surpasses 13 state-of-the-art methods in unsupervised anomaly detection by at least 5 points on two chest X-ray benchmark datasets measured by the Area Under the Curve (AUC). Additionally, we have created a new dataset (DigitAnatomy), which synthesizes the spatial correlation and consistent shape in chest anatomy. We hope DigitAnatomy can prompt the development, evaluation, and interpretability of anomaly detection methods.

cs.CV↗

arXiv2023

A step towards a general density Corrádi--Hajnal Theorem

For a nondegenerate $r$-graph $F$, large $n$, and $t$ in the regime $[0, c_{F} n]$, where $c_F>0$ is a constant depending only on $F$, we present a general approach for determining the maximum number of edges in an $n$-vertex $r$-graph that does not contain $t+1$ vertex-disjoint copies of $F$. In fact, our method results in a rainbow version of the above result and includes a characterization of the extremal constructions. Our approach applies to many well-studied hypergraphs (including graphs) such as the edge-critical graphs, the Fano plane, the generalized triangles, hypergraph expansions, the expanded triangles, and hypergraph books. Our results extend old results of Simonovits~\cite{SI68} and Moon~\cite{Moon68} on complete graphs and can be viewed as a step towards a general density version of the classical Corrádi--Hajnal Theorem~\cite{CH63}.

math.CO↗

arXiv2022

Convection and Clouds under Different Planetary Gravities Simulated by a Small-domain Cloud-resolving Model

In this study, we employ a cloud-resolving model (CRM) to investigate how gravity influences convection and clouds in a small-domain (96 km by 96 km) radiative-convective equilibrium (RCE). Our experiments are performed with a horizontal grid spacing of 1 km, which can resolve large (> 1 km$^2$) convective cells. We find that under a given stellar flux, sea surface temperature increases with decreasing gravity. This is because a lower-gravity planet has larger water vapor content and more clouds, resulting in a larger clear-sky greenhouse effect and a stronger cloud warming effect in the small domain. By increasing stellar flux under different gravity values, we find that the convection shifts from a quasi-steady state to an oscillatory state. In the oscillatory state, there are convection cycles with a period of several days, comprised of a short wet phase with intense surface precipitation and a dry phase with no surface precipitation. When convection shifts to the oscillatory state, water vapor content and high-level cloud fraction increase substantially, resulting in rapid warming. After the transition to the oscillatory state, the cloud net positive radiative effect decreases with increasing stellar flux, which indicates a stabilizing climate effect. In the quasi-steady state, the atmospheric absorption features of CO$_2$ are more detectable on lower-gravity planets because of their larger atmospheric heights. While in the oscillatory state, the high-level clouds mute almost all the absorption features, making the atmospheric components hard to be characterized.

astro-ph.EP↗

arXiv2022

Vis2Mus: Exploring Multimodal Representation Mapping for Controllable Music Generation

In this study, we explore the representation mapping from the domain of visual arts to the domain of music, with which we can use visual arts as an effective handle to control music generation. Unlike most studies in multimodal representation learning that are purely data-driven, we adopt an analysis-by-synthesis approach that combines deep music representation learning with user studies. Such an approach enables us to discover \textit{interpretable} representation mapping without a huge amount of paired data. In particular, we discover that visual-to-music mapping has a nice property similar to equivariant. In other words, we can use various image transformations, say, changing brightness, changing contrast, style transfer, to control the corresponding transformations in the music domain. In addition, we released the Vis2Mus system as a controllable interface for symbolic music generation.

cs.SD↗

arXiv2022

Two stability theorems for $\mathcal{K}_{\ell + 1}^{r}$-saturated hypergraphs

An $\mathcal{F}$-saturated $r$-graph is a maximal $r$-graph not containing any member of $\mathcal{F}$ as a subgraph. Let $\mathcal{K}_{\ell + 1}^{r}$ be the collection of all $r$-graphs $F$ with at most $\binom{\ell+1}{2}$ edges such that for some $\left(\ell+1\right)$-set $S$ every pair $\{u, v\} \subset S$ is covered by an edge in $F$. Our first result shows that for each $\ell \geq r \geq 2$ every $\mathcal{K}_{\ell+1}^{r}$-saturated $r$-graph on $n$ vertices with $t_{r}(n, \ell) - o(n^{r-1+1/\ell})$ edges contains a complete $\ell$-partite subgraph on $(1-o(1))n$ vertices, which extends a stability theorem for $K_{\ell+1}$-saturated graphs given by Popielarz, Sahasrabudhe and Snyder. We also show that the bound is best possible. Our second result is motivated by a celebrated theorem of Andrásfai, Erdős and Sós which states that for $\ell \geq 2$ every $K_{\ell+1}$-free graph $G$ on $n$ vertices with minimum degree $δ(G) > \frac{3\ell-4}{3\ell-1}n$ is $\ell$-partite. We give a hypergraph version of it. The \emph{minimum positive co-degree} of an $r$-graph $\mathcal{H}$, denoted by $δ_{r-1}^{+}(\mathcal{H})$, is the maximum $k$ such that if $S$ is an $(r-1)$-set contained in a edge of $\mathcal{H}$, then $S$ is contained in at least $k$ distinct edges of $\mathcal{H}$. Let $\ell\ge 3$ be an integer and $\mathcal{H}$ be a $\mathcal{K}_{\ell+1}^3$-saturated $3$-graph on $n$ vertices. We prove that if either $\ell \ge 4$ and $δ_{2}^{+}(\mathcal{H}) > \frac{3\ell-7}{3\ell-1}n$; or $\ell = 3$ and $δ_{2}^{+}(\mathcal{H}) > 2n/7$, then $\mathcal{H}$ is $\ell$-partite; and the bound is best possible. This is the first stability result on minimum positive co-degree for hypergraphs.

math.CO↗

arXiv2022

Learning Hierarchical Metrical Structure Beyond Measures

Music contains hierarchical structures beyond beats and measures. While hierarchical structure annotations are helpful for music information retrieval and computer musicology, such annotations are scarce in current digital music databases. In this paper, we explore a data-driven approach to automatically extract hierarchical metrical structures from scores. We propose a new model with a Temporal Convolutional Network-Conditional Random Field (TCN-CRF) architecture. Given a symbolic music score, our model takes in an arbitrary number of voices in a beat-quantized form, and predicts a 4-level hierarchical metrical structure from downbeat-level to section-level. We also annotate a dataset using RWC-POP MIDI files to facilitate training and evaluation. We show by experiments that the proposed method performs better than the rule-based approach under different orchestration settings. We also perform some simple musicological analysis on the model predictions. All demos, datasets and pre-trained models are publicly available on Github.

cs.SD↗

arXiv2022

Cloud Feedback on Earth's Long-term Climate Simulated by a Near-global Cloud-permitting Model

The Sun becomes brighter with time, but Earth's climate is roughly temperate for life during its long-term history; for early Earth, this is known as the Faint Young Sun Problem (FYSP). Besides the carbonate-silicate feedback, recent researches suggest that a long-term cloud feedback may partially solve the FYSP. However, the general circulation models they used cannot resolve convection and clouds explicitly. This study re-investigates the clouds using a near-global cloud-permitting model without cumulus convection parameterization. Our results confirm that a stabilizing shortwave cloud feedback does exist, and its magnitude is $\approx$6 W m$^{-2}$ or 14% of the energy required to offset a 20% fainter Sun than today, or $\approx$10 W m$^{-2}$ or 16% for a 30% fainter Sun. When insolation increases and meanwhile CO$_2$ concentration decreases, low-level clouds increase, acting to stabilize the climate by raising planetary albedo, and vice versa.

physics.ao-ph↗

arXiv2022

Interpreting Song Lyrics with an Audio-Informed Pre-trained Language Model

Lyric interpretations can help people understand songs and their lyrics quickly, and can also make it easier to manage, retrieve and discover songs efficiently from the growing mass of music archives. In this paper we propose BART-fusion, a novel model for generating lyric interpretations from lyrics and music audio that combines a large-scale pre-trained language model with an audio encoder. We employ a cross-modal attention module to incorporate the audio representation into the lyrics representation to help the pre-trained language model understand the song from an audio perspective, while preserving the language model's original generative performance. We also release the Song Interpretation Dataset, a new large-scale dataset for training and evaluating our model. Experimental results show that the additional audio information helps our model to understand words and music better, and to generate precise and fluent interpretations. An additional experiment on cross-modal music retrieval shows that interpretations generated by BART-fusion can also help people retrieve music more accurately than with the original BART.

cs.SD↗

arXiv2022

A Light-weight Interpretable Compositional Model for Nuclei Detection and Weakly-Supervised Segmentation

The field of computational pathology has witnessed great advancements since deep neural networks have been widely applied. These networks usually require large numbers of annotated data to train vast parameters. However, it takes significant effort to annotate a large histopathology dataset. We introduce a light-weight and interpretable model for nuclei detection and weakly-supervised segmentation. It only requires annotations on isolated nucleus, rather than on all nuclei in the dataset. Besides, it is a generative compositional model that first locates parts of nucleus, then learns the spatial correlation of the parts to further locate the nucleus. This process brings interpretability in its prediction. Empirical results on an in-house dataset show that in detection, the proposed method achieved comparable or better performance than its deep network counterparts, especially when the annotated data is limited. It also outperforms popular weakly-supervised segmentation methods. The proposed method could be an alternative solution for the data-hungry problem of deep learning methods.

cs.CV↗

arXiv2022

Generating non-jumps from a known one

Let $r\ge 2$ be an integer. The real number $α\in [0,1]$ is a jump for $r$ if there exists a constant $c > 0$ such that for any $ε>0$ and any integer $m \geq r$, there exists an integer $n_0(ε, m)$ satisfying any $r$-uniform graph with $n\ge n_0(ε, m)$ vertices and density at least $α+ε$ contains a subgraph with $m$ vertices and density at least $α+c$. A result of Erdős, Stone and Simonovits implies that every $α\in [0,1)$ is a jump for $r=2$. Erdős asked whether the same is true for $r\ge 3$. Frankl and Rödl gave a negative answer by showing that $1-\frac{1}{l^{r-1}}$ is not a jump for $r$ if $r\ge 3$ and $l>2r$. After that, more non-jumps are found using a method of Frankl and Rödl. In this note, we show a method to construct maps $f \colon [0,1] \to [0,1]$ that preserve non-jumps, if $α$ is a non-jump for $r$ given by the method of Frankl and Rödl, then $f(α)$ is also a non-jump for $r$. We use these maps to study hypergraph Turán densities and answer a question posed by Grosu.

math.CO↗

arXiv2022

Fast AdvProp

Adversarial Propagation (AdvProp) is an effective way to improve recognition models, leveraging adversarial examples. Nonetheless, AdvProp suffers from the extremely slow training speed, mainly because: a) extra forward and backward passes are required for generating adversarial examples; b) both original samples and their adversarial counterparts are used for training (i.e., 2$\times$ data). In this paper, we introduce Fast AdvProp, which aggressively revamps AdvProp's costly training components, rendering the method nearly as cheap as the vanilla training. Specifically, our modifications in Fast AdvProp are guided by the hypothesis that disentangled learning with adversarial examples is the key for performance improvements, while other training recipes (e.g., paired clean and adversarial training samples, multi-step adversarial attackers) could be largely simplified. Our empirical results show that, compared to the vanilla training baseline, Fast AdvProp is able to further model performance on a spectrum of visual benchmarks, without incurring extra training cost. Additionally, our ablations find Fast AdvProp scales better if larger models are used, is compatible with existing data augmentation methods (i.e., Mixup and CutMix), and can be easily adapted to other recognition tasks like object detection. The code is available here: https://github.com/meijieru/fast_advprop.

cs.CV↗