Searcharxiv⌕ Search

arXiv subjects

Weiyi Zhang

Publications and source records attributed to Weiyi Zhang.

At least 37 records · Page 2Linked to original sources
arXiv2024

UWF-RI2FA: Generating Multi-frame Ultrawide-field Fluorescein Angiography from Ultrawide-field Retinal Imaging Improves Diabetic Retinopathy Stratification

Ultrawide-field fluorescein angiography (UWF-FA) facilitates diabetic retinopathy (DR) detection by providing a clear visualization of peripheral retinal lesions. However, the intravenous dye injection with potential risks hamper its application. We aim to acquire dye-free UWF-FA images from noninvasive UWF retinal imaging (UWF-RI) using generative artificial intelligence (GenAI) and evaluate its effectiveness in DR screening. A total of 18,321 UWF-FA images of different phases were registered with corresponding UWF-RI images and fed into a generative adversarial networks (GAN)-based model for training. The quality of generated UWF-FA images was evaluated through quantitative metrics and human evaluation. The DeepDRiD dataset was used to externally assess the contribution of generated UWF-FA images to DR classification, using area under the receiver operating characteristic curve (AUROC) as outcome metrics. The generated early, mid, and late phase UWF-FA images achieved high authenticity, with multi-scale similarity scores ranging from 0.70 to 0.91 and qualitative visual scores ranging from 1.64 to 1.98 (1=real UWF-FA quality). In fifty randomly selected images, 56% to 76% of the generated images were difficult to distinguish from real images in the Turing test. Moreover, adding these generated UWF-FA images for DR classification significantly increased the AUROC from 0.869 to 0.904 compared to the baseline model using UWF-RI images (P < .001). The model successfully generates realistic multi-frame UWF-FA images for enhancing DR stratification without intravenous dye injection.

eess.IV↗
arXiv2024

EyeFound: A Multimodal Generalist Foundation Model for Ophthalmic Imaging

Artificial intelligence (AI) is vital in ophthalmology, tackling tasks like diagnosis, classification, and visual question answering (VQA). However, existing AI models in this domain often require extensive annotation and are task-specific, limiting their clinical utility. While recent developments have brought about foundation models for ophthalmology, they are limited by the need to train separate weights for each imaging modality, preventing a comprehensive representation of multi-modal features. This highlights the need for versatile foundation models capable of handling various tasks and modalities in ophthalmology. To address this gap, we present EyeFound, a multimodal foundation model for ophthalmic images. Unlike existing models, EyeFound learns generalizable representations from unlabeled multimodal retinal images, enabling efficient model adaptation across multiple applications. Trained on 2.78 million images from 227 hospitals across 11 ophthalmic modalities, EyeFound facilitates generalist representations and diverse multimodal downstream tasks, even for detecting challenging rare diseases. It outperforms previous work RETFound in diagnosing eye diseases, predicting systemic disease incidents, and zero-shot multimodal VQA. EyeFound provides a generalizable solution to improve model performance and lessen the annotation burden on experts, facilitating widespread clinical AI applications for retinal imaging.

cs.CV↗
arXiv2024

EyeGPT: Ophthalmic Assistant with Large Language Models

Artificial intelligence (AI) has gained significant attention in healthcare consultation due to its potential to improve clinical workflow and enhance medical communication. However, owing to the complex nature of medical information, large language models (LLM) trained with general world knowledge might not possess the capability to tackle medical-related tasks at an expert level. Here, we introduce EyeGPT, a specialized LLM designed specifically for ophthalmology, using three optimization strategies including role-playing, finetuning, and retrieval-augmented generation. In particular, we proposed a comprehensive evaluation framework that encompasses a diverse dataset, covering various subspecialties of ophthalmology, different users, and diverse inquiry intents. Moreover, we considered multiple evaluation metrics, including accuracy, understandability, trustworthiness, empathy, and the proportion of hallucinations. By assessing the performance of different EyeGPT variants, we identify the most effective one, which exhibits comparable levels of understandability, trustworthiness, and empathy to human ophthalmologists (all Ps>0.05). Overall, ur study provides valuable insights for future research, facilitating comprehensive comparisons and evaluations of different strategies for developing specialized LLMs in ophthalmology. The potential benefits include enhancing the patient experience in eye care and optimizing ophthalmologists' services.

cs.CL↗
arXiv2023

Towards ultra-low-cost smartphone microscopy

The outbreak of COVID-19 exposed the inadequacy of our technical tools for home health surveillance, and recent studies have shown the potential of smartphones as a universal optical microscopic imaging platform for such applications. However, most of them use laboratory-grade optomechanical components and transmitted illuminations to ensure focus tuning capability and imaging quality, which keeps the cost of the equipment high. Here we propose an ultra-low-cost solution for smartphone microscopy. To realize focus tunability, we designed a seesaw-like structure capable of converting large displacements on one side into small displacements on the other (reduced to ~9.1%), which leverages the intrinsic flexibility of 3D printing materials. We achieved a focus-tuning accuracy of ~5 micron, which is 40 times higher than the machining accuracy of the 3D-printed lens holder itself. For microscopic imaging, we use an off-the-shelf smartphone camera lens as the objective and the built-in flashlight as the illumination. To compensate for the resulting image quality degradation, we developed a learning-based image enhancement method. We use the CycleGAN architecture to establish the mapping from smartphone microscope images to benchtop microscope images without pairing. We verified the imaging performance on different biomedical samples. Except for the smartphone, we kept the full costs of the device under 4 USD. We think these efforts to lower the costs of smartphone microscopes will benefit their applications in various scenarios, such as point-of-care testing, on-site diagnosis, and home health surveillance.

eess.IV↗
arXiv2023

LP-SLAM: Language-Perceptive RGB-D SLAM system based on Large Language Model

Simultaneous localization and mapping (SLAM) is a critical technology that enables autonomous robots to be aware of their surrounding environment. With the development of deep learning, SLAM systems can achieve a higher level of perception of the environment, including the semantic and text levels. However, current works are limited in their ability to achieve a natural-language level of perception of the world. To address this limitation, we propose LP-SLAM, the first language-perceptive SLAM system that leverages large language models (LLMs). LP-SLAM has two major features: (a) it can detect text in the scene and determine whether it represents a landmark to be stored during the tracking and mapping phase, and (b) it can understand natural language input from humans and provide guidance based on the generated map. We illustrated three usages of the LLM in the system including text cluster, landmark judgment, and natural language navigation. Our proposed system represents an advancement in the field of LLMs based SLAM and opens up new possibilities for autonomous robots to interact with their environment in a more natural and intuitive way.

cs.RO↗
arXiv2023

MIXPGD: Hybrid Adversarial Training for Speech Recognition Systems

Automatic speech recognition (ASR) systems based on deep neural networks are weak against adversarial perturbations. We propose mixPGD adversarial training method to improve the robustness of the model for ASR systems. In standard adversarial training, adversarial samples are generated by leveraging supervised or unsupervised methods. We merge the capabilities of both supervised and unsupervised approaches in our method to generate new adversarial samples which aid in improving model robustness. Extensive experiments and comparison across various state-of-the-art defense methods and adversarial attacks have been performed to show that mixPGD gains 4.1% WER of better performance than previous best performing models under white-box adversarial attack setting. We tested our proposed defense method against both white-box and transfer based black-box attack settings to ensure that our defense strategy is robust against various types of attacks. Empirical results on several adversarial attacks validate the effectiveness of our proposed approach.

cs.SD↗
arXiv2022

Images and singularity subsets of pseudoholomorphic maps

We study the image and the singularity subset of a general pseudoholomorphic map. We show that the image of a proper pseudoholomorphic map is a pseudoholomorphic subvariety when the dimension of either the domain or target is four. We also prove that the singularity subset of a pseudoholomorphic map is pseudoholomorphic when the domain has dimension four.

math.DG↗
arXiv2022

Lieb lattices formed by real atoms on Ag(111) and their lattice constant dependent electronic properties

Scanning tunneling microscopy is a powerful tool to build artificial atomic structures even not exist in nature but possess exotic properties. We here constructed Lieb lattices with different lattice constants by real atoms, i.e., Fe atoms on Ag(111) and probed their electronic properties. We find a surprising long-range effective electron wavefunction overlap between Fe adatoms as it exhibits a 1/r2-dependence with the interatomic distance r instead of the theoretically predicted exponential one. Combining control experiments, tight-binding and Green's function calculations, we attribute the observed long-range overlap to be enabled by the surface state. Our findings not only enrich the understanding of the electron wavefunction overlap, but also provide a convenient platform to design and explore the artificial structures and future devices with real atoms.

cond-mat.mes-hall↗
arXiv2022

Almost complex Hodge theory

We review the recent development of Hodge theory for almost complex manifolds. This includes the determination of whether the Hodge numbers defined by $\bar\partial$-Laplacian are almost complex, almost Kähler, or birational invariants in dimension four.

math.DG↗
arXiv2022

Harmonic Forms on the Kodaira-Thurston Manifold

We introduce an effective method to solve the $\bar\partial$-harmonic forms on the Kodaira-Thurston manifold endowed with an almost complex structure and an Hermitian metric. Using the Weil-Brezin transform, we reduce the elliptic PDE system to countably many linear ODE systems. By solving a fundamental problem on linear ODE systems, the problem of finding $\bar\partial$-harmonic forms is equivalent to a generalised Gauss circle problem. We demonstrate two remarkable applications. First, the dimension of the almost complex $\bar\partial$-Hodge numbers on the Kodaira-Thurston manifold could be arbitrarily large. Second, Hodge numbers vary with different choices of Hermitian metrics. This answers a question of Kodaira and Spencer in Hirzebruch's 1954 problem list.

math.DG↗
arXiv2021

Speech Separation Using an Asynchronous Fully Recurrent Convolutional Neural Network

Recent advances in the design of neural network architectures, in particular those specialized in modeling sequences, have provided significant improvements in speech separation performance. In this work, we propose to use a bio-inspired architecture called Fully Recurrent Convolutional Neural Network (FRCNN) to solve the separation task. This model contains bottom-up, top-down and lateral connections to fuse information processed at various time-scales represented by \textit{stages}. In contrast to the traditional approach updating stages in parallel, we propose to first update the stages one by one in the bottom-up direction, then fuse information from adjacent stages simultaneously and finally fuse information from all stages to the bottom stage together. Experiments showed that this asynchronous updating scheme achieved significantly better results with much fewer parameters than the traditional synchronous updating scheme. In addition, the proposed model achieved good balance between speech separation accuracy and computational efficiency as compared to other state-of-the-art models on three benchmark datasets.

cs.SD↗
arXiv2021

Geometric structures, the Gromov order, Kodaira dimensions and simplicial volume

We introduce an axiomatic definition for the Kodaira dimension and classify Thurston geometries in dimensions $\leq 5$ in terms of this Kodaira dimension. We show that the Kodaira dimension is monotone with respect to the partial order defined by maps of non-zero degree between 5-manifolds. We study the compatibility of our definition with traditional notions of Kodaira dimension, especially the highest possible Kodaira dimension. To this end, we establish a connection between the simplicial volume and the holomorphic Kodaira dimension, which in particular implies that any smooth Kähler 3-fold with non-vanishing simplicial volume has top holomorphic Kodaira dimension.

math.GT↗
arXiv2021

Attack on practical speaker verification system using universal adversarial perturbations

In authentication scenarios, applications of practical speaker verification systems usually require a person to read a dynamic authentication text. Previous studies played an audio adversarial example as a digital signal to perform physical attacks, which would be easily rejected by audio replay detection modules. This work shows that by playing our crafted adversarial perturbation as a separate source when the adversary is speaking, the practical speaker verification system will misjudge the adversary as a target speaker. A two-step algorithm is proposed to optimize the universal adversarial perturbation to be text-independent and has little effect on the authentication text recognition. We also estimated room impulse response (RIR) in the algorithm which allowed the perturbation to be effective after being played over the air. In the physical experiment, we achieved targeted attacks with success rate of 100%, while the word error rate (WER) on speech recognition was only increased by 3.55%. And recorded audios could pass replay detection for the live person speaking.

cs.SD↗
arXiv2021

Moduli space of $J$-holomorphic subvarieties

We study the moduli space of $J$-holomorphic subvarieties in a $4$-dimensional symplectic manifold. For an arbitrary tamed almost complex structure, we show that the moduli space of a sphere class is formed by a family of linear system structures as in algebraic geometry. Among the applications, we show various uniqueness results of $J$-holomorphic subvarieties, e.g. for the fiber and exceptional classes in irrational ruled surfaces. On the other hand, non-uniqueness and other exotic phenomena of subvarieties in complex rational surfaces are explored. In particular, connected subvarieties in an exceptional class with higher genus components are constructed. The moduli space of tori is also discussed, and leads to an extension of the elliptic curve theory.

math.SG↗
arXiv2020

Almost Kähler Kodaira-Spencer problem

We show that the almost complex Hodge number $h^{0,1}$ varies with different choices of almost Kähler metrics. This answers the almost Kähler version of a question of Kodaira and Spencer.

math.DG↗
arXiv2020

Training Stronger Baselines for Learning to Optimize

Learning to optimize (L2O) has gained increasing attention since classical optimizers require laborious problem-specific design and hyperparameter tuning. However, there is a gap between the practical demand and the achievable performance of existing L2O models. Specifically, those learned optimizers are applicable to only a limited class of problems, and often exhibit instability. With many efforts devoted to designing more sophisticated L2O models, we argue for another orthogonal, under-explored theme: the training techniques for those L2O models. We show that even the simplest L2O model could have been trained much better. We first present a progressive training scheme to gradually increase the optimizer unroll length, to mitigate a well-known L2O dilemma of truncation bias (shorter unrolling) versus gradient explosion (longer unrolling). We further leverage off-policy imitation learning to guide the L2O learning, by taking reference to the behavior of analytical optimizers. Our improved training techniques are plugged into a variety of state-of-the-art L2O models, and immediately boost their performance, without making any change to their model structures. Especially, by our proposed techniques, an earliest and simplest L2O model can be trained to outperform the latest complicated L2O models on a number of tasks. Our results demonstrate a greater potential of L2O yet to be unleashed, and urge to rethink the recent progress. Our codes are publicly available at: https://github.com/VITA-Group/L2O-Training-Techniques.

cs.LG↗
arXiv2020

From Smooth to Almost Complex

This article mainly aims to overview the recent efforts on developing algebraic geometry for an arbitrary compact almost complex manifold. We review the results obtained by the guiding philosophy that a statement for smooth maps between smooth manifolds in terms of René Thom's transversality should also have its counterpart in pseudoholomorphic setting without requiring genericity of the almost complex structures. These include intersection of compact almost complex submanifolds, structure of pseudoholomorphic maps, zero locus of certain harmonic forms, and eigenvalues of Laplacian. In addition to reviewing the compact manifolds situation, we also extend these results to orbifolds and non-compact manifolds. Motivations, methodologies, applications, and further directions are discussed. The structural results on the pseudoholomorphic maps lead to a notion of birational morphism between almost complex manifolds. This motivates the study of various birational invariants, including Kodaira dimensions and plurigenera, in this setting. Some other aspects of almost complex algebraic geometry in dimension $4$ are also reviewed. These include cones of (co)homology classes and subvarieties in spherical classes.

math.DG↗