arXiv subjects

Yixiong Chen

Publications and source records attributed to Yixiong Chen.

33 records · Page 2Linked to original sources

arXiv2023

Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesis

Recent text-to-image generative models can generate high-fidelity images from text inputs, but the quality of these generated images cannot be accurately evaluated by existing evaluation metrics. To address this issue, we introduce Human Preference Dataset v2 (HPD v2), a large-scale dataset that captures human preferences on images from a wide range of sources. HPD v2 comprises 798,090 human preference choices on 433,760 pairs of images, making it the largest dataset of its kind. The text prompts and images are deliberately collected to eliminate potential bias, which is a common issue in previous datasets. By fine-tuning CLIP on HPD v2, we obtain Human Preference Score v2 (HPS v2), a scoring model that can more accurately predict human preferences on generated images. Our experiments demonstrate that HPS v2 generalizes better than previous metrics across various image distributions and is responsive to algorithmic improvements of text-to-image generative models, making it a preferable evaluation metric for these models. We also investigate the design of the evaluation prompts for text-to-image generative models, to make the evaluation stable, fair and easy-to-use. Finally, we establish a benchmark for text-to-image generative models using HPS v2, which includes a set of recent text-to-image models from the academic, community and industry. The code and dataset is available at https://github.com/tgxs002/HPSv2 .

cs.CV↗

arXiv2023

MetaLR: Meta-tuning of Learning Rates for Transfer Learning in Medical Imaging

In medical image analysis, transfer learning is a powerful method for deep neural networks (DNNs) to generalize well on limited medical data. Prior efforts have focused on developing pre-training algorithms on domains such as lung ultrasound, chest X-ray, and liver CT to bridge domain gaps. However, we find that model fine-tuning also plays a crucial role in adapting medical knowledge to target tasks. The common fine-tuning method is manually picking transferable layers (e.g., the last few layers) to update, which is labor-expensive. In this work, we propose a meta-learning-based LR tuner, named MetaLR, to make different layers automatically co-adapt to downstream tasks based on their transferabilities across domains. MetaLR learns appropriate LRs for different layers in an online manner, preventing highly transferable layers from forgetting their medical representation abilities and driving less transferable layers to adapt actively to new domains. Extensive experiments on various medical applications show that MetaLR outperforms previous state-of-the-art (SOTA) fine-tuning strategies. Codes are released.

cs.CV↗

arXiv2023

X-IQE: eXplainable Image Quality Evaluation for Text-to-Image Generation with Visual Large Language Models

This paper introduces a novel explainable image quality evaluation approach called X-IQE, which leverages visual large language models (LLMs) to evaluate text-to-image generation methods by generating textual explanations. X-IQE utilizes a hierarchical Chain of Thought (CoT) to enable MiniGPT-4 to produce self-consistent, unbiased texts that are highly correlated with human evaluation. It offers several advantages, including the ability to distinguish between real and generated images, evaluate text-image alignment, and assess image aesthetics without requiring model training or fine-tuning. X-IQE is more cost-effective and efficient compared to human evaluation, while significantly enhancing the transparency and explainability of deep image quality evaluation models. We validate the effectiveness of our method as a benchmark using images generated by prevalent diffusion models. X-IQE demonstrates similar performance to state-of-the-art (SOTA) evaluation methods on COCO Caption, while overcoming the limitations of previous evaluation models on DrawBench, particularly in handling ambiguous generation prompts and text recognition in generated images. Project website: https://github.com/Schuture/Benchmarking-Awesome-Diffusion-Models

cs.CV↗

arXiv2023

Label-Free Liver Tumor Segmentation

We demonstrate that AI models can accurately segment liver tumors without the need for manual annotation by using synthetic tumors in CT scans. Our synthetic tumors have two intriguing advantages: (I) realistic in shape and texture, which even medical professionals can confuse with real tumors; (II) effective for training AI models, which can perform liver tumor segmentation similarly to the model trained on real tumors -- this result is exciting because no existing work, using synthetic tumors only, has thus far reached a similar or even close performance to real tumors. This result also implies that manual efforts for annotating tumors voxel by voxel (which took years to create) can be significantly reduced in the future. Moreover, our synthetic tumors can automatically generate many examples of small (or even tiny) synthetic tumors and have the potential to improve the success rate of detecting small liver tumors, which is critical for detecting the early stages of cancer. In addition to enriching the training data, our synthesizing strategy also enables us to rigorously assess the AI robustness.

eess.IV↗

arXiv2022

Generating and Weighting Semantically Consistent Sample Pairs for Ultrasound Contrastive Learning

Well-annotated medical datasets enable deep neural networks (DNNs) to gain strong power in extracting lesion-related features. Building such large and well-designed medical datasets is costly due to the need for high-level expertise. Model pre-training based on ImageNet is a common practice to gain better generalization when the data amount is limited. However, it suffers from the domain gap between natural and medical images. In this work, we pre-train DNNs on ultrasound (US) domains instead of ImageNet to reduce the domain gap in medical US applications. To learn US image representations based on unlabeled US videos, we propose a novel meta-learning-based contrastive learning method, namely Meta Ultrasound Contrastive Learning (Meta-USCL). To tackle the key challenge of obtaining semantically consistent sample pairs for contrastive learning, we present a positive pair generation module along with an automatic sample weighting module based on meta-learning. Experimental results on multiple computer-aided diagnosis (CAD) problems, including pneumonia detection, breast cancer classification, and breast tumor segmentation, show that the proposed self-supervised method reaches state-of-the-art (SOTA). The codes are available at https://github.com/Schuture/Meta-USCL.

cs.CV↗

arXiv2022

Rethinking Two Consensuses of the Transferability in Deep Learning

Deep transfer learning (DTL) has formed a long-term quest toward enabling deep neural networks (DNNs) to reuse historical experiences as efficiently as humans. This ability is named knowledge transferability. A commonly used paradigm for DTL is firstly learning general knowledge (pre-training) and then reusing (fine-tuning) them for a specific target task. There are two consensuses of transferability of pre-trained DNNs: (1) a larger domain gap between pre-training and downstream data brings lower transferability; (2) the transferability gradually decreases from lower layers (near input) to higher layers (near output). However, these consensuses were basically drawn from the experiments based on natural images, which limits their scope of application. This work aims to study and complement them from a broader perspective by proposing a method to measure the transferability of pre-trained DNN parameters. Our experiments on twelve diverse image classification datasets get similar conclusions to the previous consensuses. More importantly, two new findings are presented, i.e., (1) in addition to the domain gap, a larger data amount and huge dataset diversity of downstream target task also prohibit the transferability; (2) although the lower layers learn basic image features, they are usually not the most transferable layers due to their domain sensitivity.

cs.CV↗

arXiv2022

Synthetic Tumors Make AI Segment Tumors Better

We develop a novel strategy to generate synthetic tumors. Unlike existing works, the tumors generated by our strategy have two intriguing advantages: (1) realistic in shape and texture, which even medical professionals can confuse with real tumors; (2) effective for AI model training, which can perform liver tumor segmentation similarly to a model trained on real tumors - this result is unprecedented because no existing work, using synthetic tumors only, has thus far reached a similar or even close performance to the model trained on real tumors. This result also implies that manual efforts for developing per-voxel annotation of tumors (which took years to create) can be considerably reduced for training AI models in the future. Moreover, our synthetic tumors have the potential to improve the success rate of small tumor detection by automatically generating enormous examples of small (or tiny) synthetic tumors.

eess.IV↗

arXiv2022

HiCo: Hierarchical Contrastive Learning for Ultrasound Video Model Pretraining

The self-supervised ultrasound (US) video model pretraining can use a small amount of labeled data to achieve one of the most promising results on US diagnosis. However, it does not take full advantage of multi-level knowledge for learning deep neural networks (DNNs), and thus is difficult to learn transferable feature representations. This work proposes a hierarchical contrastive learning (HiCo) method to improve the transferability for the US video model pretraining. HiCo introduces both peer-level semantic alignment and cross-level semantic alignment to facilitate the interaction between different semantic levels, which can effectively accelerate the convergence speed, leading to better generalization and adaptation of the learned model. Additionally, a softened objective function is implemented by smoothing the hard labels, which can alleviate the negative effect caused by local similarities of images between different classes. Experiments with HiCo on five datasets demonstrate its favorable results over state-of-the-art approaches. The source code of this work is publicly available at https://github.com/983632847/HiCo.

cs.CV↗

arXiv2021

QDCNN: Quantum Dilated Convolutional Neural Network

In recent years, with rapid progress in the development of quantum technologies, quantum machine learning has attracted a lot of interest. In particular, a family of hybrid quantum-classical neural networks, consisting of classical and quantum elements, has been massively explored for the purpose of improving the performance of classical neural networks. In this paper, we propose a novel hybrid quantum-classical algorithm called quantum dilated convolutional neural networks (QDCNNs). Our method extends the concept of dilated convolution, which has been widely applied in modern deep learning algorithms, to the context of hybrid neural networks. The proposed QDCNNs are able to capture larger context during the quantum convolution process while reducing the computational cost. We perform empirical experiments on MNIST and Fashion-MNIST datasets for the task of image recognition and demonstrate that QDCNN models generally enjoy better performances in terms of both accuracy and computation efficiency compared to existing quantum convolutional neural networks (QCNNs).

quant-ph↗

arXiv2021

USCL: Pretraining Deep Ultrasound Image Diagnosis Model through Video Contrastive Representation Learning

Most deep neural networks (DNNs) based ultrasound (US) medical image analysis models use pretrained backbones (e.g., ImageNet) for better model generalization. However, the domain gap between natural and medical images causes an inevitable performance bottleneck. To alleviate this problem, an US dataset named US-4 is constructed for direct pretraining on the same domain. It contains over 23,000 images from four US video sub-datasets. To learn robust features from US-4, we propose an US semi-supervised contrastive learning method, named USCL, for pretraining. In order to avoid high similarities between negative pairs as well as mine abundant visual features from limited US videos, USCL adopts a sample pair generation method to enrich the feature involved in a single step of contrastive optimization. Extensive experiments on several downstream tasks show the superiority of USCL pretraining against ImageNet pretraining and other state-of-the-art (SOTA) pretraining approaches. In particular, USCL pretrained backbone achieves fine-tuning accuracy of over 94% on POCUS dataset, which is 10% higher than 84% of the ImageNet pretrained model. The source codes of this work are available at https://github.com/983632847/USCL.

cs.CV↗

arXiv2020

When Person Re-identification Meets Changing Clothes

Person re-identification (ReID) is now an active research topic for AI-based video surveillance applications such as specific person search, but the practical issue that the target person(s) may change clothes (clothes inconsistency problem) has been overlooked for long. For the first time, this paper systematically studies this problem. We first overcome the difficulty of lack of suitable dataset, by collecting a small yet representative real dataset for testing whilst building a large realistic synthetic dataset for training and deeper studies. Facilitated by our new datasets, we are able to conduct various interesting new experiments for studying the influence of clothes inconsistency. We find that changing clothes makes ReID a much harder problem in the sense of bringing difficulties to learning effective representations and also challenges the generalization ability of previous ReID models to identify persons with unseen (new) clothes. Representative existing ReID models are adopted to show informative results on such a challenging setting, and we also provide some preliminary efforts on improving the robustness of existing models on handling the clothes inconsistency issue in the data. We believe that this study can be inspiring and helpful for encouraging more researches in this direction. The dataset is available on the project website: https://wanfb.github.io/dataset.html.

cs.CV↗

arXiv2020

Probabilistic Forecasting with Temporal Convolutional Neural Network

We present a probabilistic forecasting framework based on convolutional neural network for multiple related time series forecasting. The framework can be applied to estimate probability density under both parametric and non-parametric settings. More specifically, stacked residual blocks based on dilated causal convolutional nets are constructed to capture the temporal dependencies of the series. Combined with representation learning, our approach is able to learn complex patterns such as seasonality, holiday effects within and across series, and to leverage those patterns for more accurate forecasts, especially when historical data is sparse or unavailable. Extensive empirical studies are performed on several real-world datasets, including datasets from JD.com, China's largest online retailer. The results show that our framework outperforms other state-of-the-art methods in both accuracy and efficiency.

stat.ML↗

arXiv2016

Mixed-state form factors of $U(1)$ twist fields in the Dirac theory

Using the "Liouville space" (the space of operators) of the massive Dirac theory, we define mixed-state form factors of $U(1)$ twist fields. We consider mixed states with density matrices diagonal in the asymptotic particle basis. This includes the thermal Gibbs state as well as all generalized Gibbs ensembles of the Dirac theory. When the mixed state is specialized to a thermal Gibbs state, using a Riemann-Hilbert problem and low-temperature expansion, we obtain finite-temperature form factors of $U(1)$ twist fields. We then propose the expression for form factors of $U(1)$ twist fields in general diagonal mixed states. We verify that these form factors satisfy a system of nonlinear functional differential equations, which is derived from the trace definition of mixed-state form factors. At last, under weak analytic conditions on the eigenvalues of the density matrix, we write down the large distance form factor expansions of two-point correlation functions of these twist fields. Using the relation between the Dirac and Ising models, this provides the large-distance expansion of the R$\acute{\text{e}}$nyi entropy (for integer R$\acute{\text{e}}$nyi parameter) in the Ising model in diagonal mixed states.

cond-mat.stat-mech↗

arXiv2014

Form factors in equilibrium and non-equilibrium mixed states of the Ising model

Using the "Liouville space'' (the space of operators) of the massive Ising model of quantum field theory, there is a natural definition of form factors in any mixed state. These generalize the usual form factors, and are building blocks for mixed-state correlation functions. We study the cases of mixed states that are diagonal in the asymptotic particle basis, and obtain exact expressions for all mixed-state form factors of order and disorder fields. We use novel techniques based on deriving and solving a system of nonlinear functional differential equations. We then write down the full form factor expansion for mixed-state correlation functions of these fields. Under weak analytic conditions on the eigenvalues of the density matrix, this is an exact large-distance expansion. The form factors agree with the known finite-temperature form factors when the mixed state is specialized to a thermal Gibbs ensemble. Our results can be used to analyze correlation functions in generalized Gibbs ensembles (which occur after quantum quenches). Applying this to the density matrix for non-equilibrium steady states with energy flows, we observe that non-equilibrium form factors have branch cuts in rapidity space. We verify that this is in agreement with a non-equilibrium generalization of the KMS relations, and we conjecture that the leading large-distance behavior of order and disorder non-equilibrium correlation functions contains oscillations in the log of the distance between the fields.

cond-mat.str-el↗

arXiv2013

Thermodynamic Bethe ansatz for non-equilibrium steady states: exact energy current and fluctuations in integrable QFT

We evaluate the exact energy current and scaled cumulant generating function (related to the large-deviation function) in non-equilibrium steady states with energy flow, in any integrable model of relativistic quantum field theory (IQFT) with diagonal scattering. Our derivations are based on various recent results of D. Bernard and B. Doyon. The steady states are built by connecting homogeneously two infinite halves of the system thermalized at different temperatures $T_l$, $T_r$, and waiting for a long time. We evaluate the current $J(T_l,T_r)$ using the exact QFT density matrix describing these non-equilibrium steady states and using Al.B. Zamolodchikov's method of the thermodynamic Bethe ansatz (TBA). The scaled cumulant generating function is obtained from the extended fluctuation relations which hold in integrable models. We verify our formula in particular by showing that the conformal field theory (CFT) result is obtained in the high-temperature limit. We analyze numerically our non-equilibrium steady-state TBA equations for three models: the sinh-Gordon model, the roaming trajectories model, and the sine-Gordon model at a particular reflectionless point. Based on the numerics, we conjecture that an infinite family of non-equilibrium $c$-functions, associated to the scaled cumulants, can be defined, which we interpret physically. We study the full scaled distribution function and find that it can be described by a set of independent Poisson processes. Finally, we show that the "additivity" property of the current, which is known to hold in CFT and was proposed to hold more generally, does not hold in general IQFT, that is $J(T_l,T_r)$ is not of the form $f(T_l)-f(T_r)$.

hep-th↗