arXiv subjects

Yuting Shao

Publications and source records attributed to Yuting Shao.

3 recordsLinked to original sources

arXiv2024

Evaluating Hallucination in Text-to-Image Diffusion Models with Scene-Graph based Question-Answering Agent

Contemporary Text-to-Image (T2I) models frequently depend on qualitative human evaluations to assess the consistency between synthesized images and the text prompts. There is a demand for quantitative and automatic evaluation tools, given that human evaluation lacks reproducibility. We believe that an effective T2I evaluation metric should accomplish the following: detect instances where the generated images do not align with the textual prompts, a discrepancy we define as the `hallucination problem' in T2I tasks; record the types and frequency of hallucination issues, aiding users in understanding the causes of errors; and provide a comprehensive and intuitive scoring that close to human standard. To achieve these objectives, we propose a method based on large language models (LLMs) for conducting question-answering with an extracted scene-graph and created a dataset with human-rated scores for generated images. From the methodology perspective, we combine knowledge-enhanced question-answering tasks with image evaluation tasks, making the evaluation metrics more controllable and easier to interpret. For the contribution on the dataset side, we generated 12,000 synthesized images based on 1,000 composited prompts using three advanced T2I models. Subsequently, we conduct human scoring on all synthesized images and prompt pairs to validate the accuracy and effectiveness of our method as an evaluation metric. All generated images and the human-labeled scores will be made publicly available in the future to facilitate ongoing research on this crucial issue. Extensive experiments show that our method aligns more closely with human scoring patterns than other evaluation metrics.

cs.CV↗

arXiv2024

ODFormer: Semantic Fundus Image Segmentation Using Transformer for Optic Nerve Head Detection

Optic nerve head (ONH) detection has been a crucial area of study in ophthalmology for years. However, the significant discrepancy between fundus image datasets, each generated using a single type of fundus camera, poses challenges to the generalizability of ONH detection approaches developed based on semantic segmentation networks. Despite the numerous recent advancements in general-purpose semantic segmentation methods using convolutional neural networks (CNNs) and Transformers, there is currently a lack of benchmarks for these state-of-the-art (SoTA) networks specifically trained for ONH detection. Therefore, in this article, we make contributions from three key aspects: network design, the publication of a dataset, and the establishment of a comprehensive benchmark. Our newly developed ONH detection network, referred to as ODFormer, is based upon the Swin Transformer architecture and incorporates two novel components: a multi-scale context aggregator and a lightweight bidirectional feature recalibrator. Our published large-scale dataset, known as TongjiU-DROD, provides multi-resolution fundus images for each participant, captured using two distinct types of cameras. Our established benchmark involves three datasets: DRIONS-DB, DRISHTI-GS1, and TongjiU-DROD, created by researchers from different countries and containing fundus images captured from participants of diverse races and ages. Extensive experimental results demonstrate that our proposed ODFormer outperforms other state-of-the-art (SoTA) networks in terms of performance and generalizability. Our dataset and source code are publicly available at mias.group/ODFormer.

eess.IV↗

arXiv2015

Observation of coupling between zero- and two-dimensional semiconductor systems based on anomalous diamagnetic effects

We report the direct observation of coupling between a single self-assembled InAs quantum dot and a wetting layer, based on strong diamagnetic shifts of many-body exciton states using magneto-photoluminescence spectroscopy. An extremely large positive diamagnetic coefficient is observed when an electron in the wetting layer combines with a hole in the quantum dot; the coefficient is nearly one order of magnitude larger than that of the exciton states confined in the quantum dots. Recombination of electrons with holes in a quantum dot of the coupled system leads to an unusual negative diamagnetic effect, which is five times stronger than that in a pure quantum dot system. This effect can be attributed to the expansion of the wavefunction of remaining electrons in the wetting layer or the spread of electrons in the excited states of the quantum dot to the wetting layer after recombination. In this case, the wavefunction extent of the final states in the quantum dot plane is much larger than that of the initial states because of the absence of holes in the quantum dot to attract electrons. The properties of emitted photons that depend on the large electron wavefunction extents in the wetting layer indicate that the coupling occurs between systems of different dimensionality, which is also verified from the results obtained by applying a magnetic field in different configurations. This study paves a new way to observe hybrid states with zero- and two-dimensional structures, which could be useful for investigating the Kondo physics and implementing spin-based solid-state quantum information processing.

cond-mat.mes-hall↗