arXiv subjects

Yun Liu

Publications and source records attributed to Yun Liu.

At least 109 records · Page 6Linked to original sources

arXiv2025

Advancing Conversational Diagnostic AI with Multimodal Reasoning

Large Language Models (LLMs) have demonstrated great potential for conducting diagnostic conversations but evaluation has been largely limited to language-only interactions, deviating from the real-world requirements of remote care delivery. Instant messaging platforms permit clinicians and patients to upload and discuss multimodal medical artifacts seamlessly in medical consultation, but the ability of LLMs to reason over such data while preserving other attributes of competent diagnostic conversation remains unknown. Here we advance the conversational diagnosis and management performance of the Articulate Medical Intelligence Explorer (AMIE) through a new capability to gather and interpret multimodal data, and reason about this precisely during consultations. Leveraging Gemini 2.0 Flash, our system implements a state-aware dialogue framework, where conversation flow is dynamically controlled by intermediate model outputs reflecting patient states and evolving diagnoses. Follow-up questions are strategically directed by uncertainty in such patient states, leading to a more structured multimodal history-taking process that emulates experienced clinicians. We compared AMIE to primary care physicians (PCPs) in a randomized, blinded, OSCE-style study of chat-based consultations with patient actors. We constructed 105 evaluation scenarios using artifacts like smartphone skin photos, ECGs, and PDFs of clinical documents across diverse conditions and demographics. Our rubric assessed multimodal capabilities and other clinically meaningful axes like history-taking, diagnostic accuracy, management reasoning, communication, and empathy. Specialist evaluation showed AMIE to be superior to PCPs on 7/9 multimodal and 29/32 non-multimodal axes (including diagnostic accuracy). The results show clear progress in multimodal conversational diagnostic AI, but real-world translation needs further research.

cs.CL↗

arXiv2025

LLM-based Text Simplification and its Effect on User Comprehension and Cognitive Load

Information on the web, such as scientific publications and Wikipedia, often surpasses users' reading level. To help address this, we used a self-refinement approach to develop a LLM capability for minimally lossy text simplification. To validate our approach, we conducted a randomized study involving 4563 participants and 31 texts spanning 6 broad subject areas: PubMed (biomedical scientific articles), biology, law, finance, literature/philosophy, and aerospace/computer science. Participants were randomized to viewing original or simplified texts in a subject area, and answered multiple-choice questions (MCQs) that tested their comprehension of the text. The participants were also asked to provide qualitative feedback such as task difficulty. Our results indicate that participants who read the simplified text answered more MCQs correctly than their counterparts who read the original text (3.9% absolute increase, p<0.05). This gain was most striking with PubMed (14.6%), while more moderate gains were observed for finance (5.5%), aerospace/computer science (3.8%) domains, and legal (3.5%). Notably, the results were robust to whether participants could refer back to the text while answering MCQs. The absolute accuracy decreased by up to ~9% for both original and simplified setups where participants could not refer back to the text, but the ~4% overall improvement persisted. Finally, participants' self-reported perceived ease based on a simplified NASA Task Load Index was greater for those who read the simplified text (absolute change on a 5-point scale 0.33, p<0.05). This randomized study, involving an order of magnitude more participants than prior works, demonstrates the potential of LLMs to make complex information easier to understand. Our work aims to enable a broader audience to better learn and make use of expert knowledge available on the web, improving information accessibility.

cs.CL↗

arXiv2025

Exotic Quantum States in Spin-1 Bose-Einstein Condensate with Spin-Orbit Coupling in Concentric Annular Traps

We explore the exotic quantum states emerging in the ground state (GS) of a strongly-correlated spin-1 Bose-Einstein condensate confined in two-dimensional concentric annular traps with a spin-orbit coupling (SOC). In the antiferromagnetic case, the GS density manifests various patterns of distributions, including facial-makeup states, petal states, topological fissure states, multiple-half-ring states and property-distinguished vertical and horizonal stripe states. We notice a peculiar phenomenon of density-phase separation in the sense that the variations of density and phase tend to be independent. In ferromagnetic case, the GS exhibits a semi-circular or half-disk status of density embedded with vortices and anti-vortices. The spin distribution can self-arrange into an array of half-skyrmions and we also find a half-antiskyrmion fence separating vortex-antivortex pairs. Our study indicates that one can manipulate the emergence of exotic quantum states via the interplay of the SOC, interaction and potential geometry and the abundant state variations might also provide potential resources for quantum metrology.

cond-mat.quant-gas↗

arXiv2025

Evaluating AI systems under uncertain ground truth: a case study in dermatology

For safety, medical AI systems undergo thorough evaluations before deployment, validating their predictions against a ground truth which is assumed to be fixed and certain. However, this ground truth is often curated in the form of differential diagnoses. While a single differential diagnosis reflects the uncertainty in one expert assessment, multiple experts introduce another layer of uncertainty through disagreement. Both forms of uncertainty are ignored in standard evaluation which aggregates these differential diagnoses to a single label. In this paper, we show that ignoring uncertainty leads to overly optimistic estimates of model performance, therefore underestimating risk associated with particular diagnostic decisions. To this end, we propose a statistical aggregation approach, where we infer a distribution on probabilities of underlying medical condition candidates themselves, based on observed annotations. This formulation naturally accounts for the potential disagreements between different experts, as well as uncertainty stemming from individual differential diagnoses, capturing the entire ground truth uncertainty. Our approach boils down to generating multiple samples of medical condition probabilities, then evaluating and averaging performance metrics based on these sampled probabilities. In skin condition classification, we find that a large portion of the dataset exhibits significant ground truth uncertainty and standard evaluation severely over-estimates performance without providing uncertainty estimates. In contrast, our framework provides uncertainty estimates on common metrics of interest such as top-k accuracy and average overlap, showing that performance can change multiple percentage points. We conclude that, while assuming a crisp ground truth can be acceptable for many AI applications, a more nuanced evaluation protocol should be utilized in medical diagnosis.

cs.LG↗

arXiv2025

Exploring cosmological constraints of the weak gravitational lensing and galaxy clustering joint analysis in the CSST photometric survey

We explore the joint weak lensing and galaxy clustering analysis from the photometric survey operated by the China Space Station Telescope (CSST), and study the strength of the cosmological constraints. We employ a high-resolution JiuTian-1G simulation to construct a partial-sky light cone to $z=3$ covering 100 deg$^2$, and obtain the CSST galaxy mock samples based on an improved semi-analytical model. We perform a multi-lens-plane algorithm to generate corresponding synthetic weak lensing maps and catalogs. Then we generate the mock data based on these catalogs considering the instrumental and observational effects of the CSST, and use the Markov Chain Monte Carlo (MCMC) method to perform the constraints. The covariance matrix includes non-Gaussian contributions and super-sample covariance terms, and the systematics from intrinsic alignments, galaxy bias, photometric redshift uncertainties, shear calibration, and non-linear effects are considered in the analysis. We find that the constraint result is comparable to that from Stage III surveys, and it can be significantly improved further in the full CSST survey with 17500 deg$^2$. This indicates the CSST photometric survey is powerful for exploring the Universe.

astro-ph.CO↗

arXiv2025

Exploiting Temporal State Space Sharing for Video Semantic Segmentation

Video semantic segmentation (VSS) plays a vital role in understanding the temporal evolution of scenes. Traditional methods often segment videos frame-by-frame or in a short temporal window, leading to limited temporal context, redundant computations, and heavy memory requirements. To this end, we introduce a Temporal Video State Space Sharing (TV3S) architecture to leverage Mamba state space models for temporal feature sharing. Our model features a selective gating mechanism that efficiently propagates relevant information across video frames, eliminating the need for a memory-heavy feature pool. By processing spatial patches independently and incorporating shifted operation, TV3S supports highly parallel computation in both training and inference stages, which reduces the delay in sequential state space processing and improves the scalability for long video sequences. Moreover, TV3S incorporates information from prior frames during inference, achieving long-range temporal coherence and superior adaptability to extended sequences. Evaluations on the VSPW and Cityscapes datasets reveal that our approach outperforms current state-of-the-art methods, establishing a new standard for VSS with consistent results across long video sequences. By achieving a good balance between accuracy and efficiency, TV3S shows a significant advancement in spatiotemporal modeling, paving the way for efficient video analysis. The code is publicly available at https://github.com/Ashesham/TV3S.git.

eess.IV↗

arXiv2025

J&H: Evaluating the Robustness of Large Language Models Under Knowledge-Injection Attacks in Legal Domain

As the scale and capabilities of Large Language Models (LLMs) increase, their applications in knowledge-intensive fields such as legal domain have garnered widespread attention. However, it remains doubtful whether these LLMs make judgments based on domain knowledge for reasoning. If LLMs base their judgments solely on specific words or patterns, rather than on the underlying logic of the language, the ''LLM-as-judges'' paradigm poses substantial risks in the real-world applications. To address this question, we propose a method of legal knowledge injection attacks for robustness testing, thereby inferring whether LLMs have learned legal knowledge and reasoning logic. In this paper, we propose J&H: an evaluation framework for detecting the robustness of LLMs under knowledge injection attacks in the legal domain. The aim of the framework is to explore whether LLMs perform deductive reasoning when accomplishing legal tasks. To further this aim, we have attacked each part of the reasoning logic underlying these tasks (major premise, minor premise, and conclusion generation). We have collected mistakes that legal experts might make in judicial decisions in the real world, such as typos, legal synonyms, inaccurate external legal statutes retrieval. However, in real legal practice, legal experts tend to overlook these mistakes and make judgments based on logic. However, when faced with these errors, LLMs are likely to be misled by typographical errors and may not utilize logic in their judgments. We conducted knowledge injection attacks on existing general and domain-specific LLMs. Current LLMs are not robust against the attacks employed in our experiments. In addition we propose and compare several methods to enhance the knowledge robustness of LLMs.

cs.CL↗

arXiv2025

Absence of dehydration due to superionic transition at Earth's core-mantle boundary

The properties and stability of hydrous phases are key to unraveling the mysteries of the water cycle in Earth's interior. Under the deep lower mantle conditions, hydrous phases transition into a superionic state. However, the influence of the superionic effect on their stability and dehydration processes remains poorly understood. Using ab initio calculations and deep-learning potential molecular dynamics simulations, we discovered a doubly superionic transition in delta-AlOOH, characterized by the highly diffusive behavior of ionic hydrogen and aluminum within the oxygen sub-lattice. These highly diffusive elements contribute significant external entropy into the system, resulting in exceptional thermostability. Free energy calculations indicate that dehydration is energetically and kinetically unfavorable when water exists in a superionic state under core-mantle boundary (CMB) conditions. Consequently, water can accumulate in the deep lower mantle over Earth's history. This deep water reservoir plays a crucial role in the global deep water and hydrogen cycles.

physics.geo-ph↗

arXiv2025

Passive Heart Rate Monitoring During Smartphone Use in Everyday Life

Resting heart rate (RHR) is an important biomarker of cardiovascular health and mortality, but tracking it longitudinally generally requires a wearable device, limiting its availability. We present PHRM, a deep learning system for passive heart rate (HR) and RHR measurements during everyday smartphone use, using facial video-based photoplethysmography. Our system was developed using 225,773 videos from 495 participants and validated on 185,970 videos from 205 participants in laboratory and free-living conditions, representing the largest validation study of its kind. Compared to reference electrocardiogram, PHRM achieved a mean absolute percentage error (MAPE) < 10% for HR measurements across three skin tone groups of light, medium and dark pigmentation; MAPE for each skin tone group was non-inferior versus the others. Daily RHR measured by PHRM had a mean absolute error < 5 bpm compared to a wearable HR tracker, and was associated with known risk factors. These results highlight the potential of smartphones to enable passive and equitable heart health monitoring.

q-bio.TO↗

arXiv2025

Integrating network pharmacology, metabolomics, and gut microbiota analysis to explore the effects of Jinhong tablets on chronic superficial gastritis

Chronic superficial gastritis (CSG) severely affects quality of life and can progress to worse gastric pathologies. Traditional Chinese Medicine (TCM) effectively treats CSG, as exemplified by Jinhong Tablets (JHT) with known anti-inflammatory properties, though their mechanism remains unclear. This study integrated network pharmacology, untargeted metabolomics, and gut microbiota analyses to investigate how JHT alleviates CSG. A rat CSG model was established and evaluated via H&E staining. We identified JHT's target profiles and constructed a multi-layer biomolecular network. Differential metabolites in plasma were determined by untargeted metabolomics, and gut microbiota diversity/composition in fecal and cecal samples was assessed via 16S rRNA sequencing. JHT markedly reduced gastric inflammation. Network pharmacology highlighted metabolic pathways, particularly lipid and nitric oxide metabolism, as essential to JHT's therapeutic effect. Metabolomics identified key differential metabolites including betaine (enhancing gut microbiota), phospholipids, and citrulline (indicating severity of CSG). Pathway enrichment supported the gut microbiota's involvement. Further microbiota analysis showed that JHT increased betaine abundance, improved short-chain fatty acid production, and elevated Faecalibaculum and Bifidobacterium, thereby alleviating gastric inflammation. In conclusion, JHT alleviates CSG via diverse metabolic processes, especially lipid and energy metabolism, and influences metabolites like betaine alongside gut microbes such as Faecalibaculum and Bifidobacterium. These findings underscore JHT's therapeutic potential and deepen our understanding of TCM's role in CSG management.

q-bio.MN↗

arXiv2025

Multimodality Helps Few-shot 3D Point Cloud Semantic Segmentation

Few-shot 3D point cloud segmentation (FS-PCS) aims at generalizing models to segment novel categories with minimal annotated support samples. While existing FS-PCS methods have shown promise, they primarily focus on unimodal point cloud inputs, overlooking the potential benefits of leveraging multimodal information. In this paper, we address this gap by introducing a multimodal FS-PCS setup, utilizing textual labels and the potentially available 2D image modality. Under this easy-to-achieve setup, we present the MultiModal Few-Shot SegNet (MM-FSS), a model effectively harnessing complementary information from multiple modalities. MM-FSS employs a shared backbone with two heads to extract intermodal and unimodal visual features, and a pretrained text encoder to generate text embeddings. To fully exploit the multimodal information, we propose a Multimodal Correlation Fusion (MCF) module to generate multimodal correlations, and a Multimodal Semantic Fusion (MSF) module to refine the correlations using text-aware semantic guidance. Additionally, we propose a simple yet effective Test-time Adaptive Cross-modal Calibration (TACC) technique to mitigate training bias, further improving generalization. Experimental results on S3DIS and ScanNet datasets demonstrate significant performance improvements achieved by our method. The efficacy of our approach indicates the benefits of leveraging commonly-ignored free modalities for FS-PCS, providing valuable insights for future research. The code is available at https://github.com/ZhaochongAn/Multimodality-3D-Few-Shot

cs.CV↗

arXiv2025

Transferable and Forecastable User Targeting Foundation Model

User targeting, the process of selecting targeted users from a pool of candidates for non-expert marketers, has garnered substantial attention with the advancements in digital marketing. However, existing user targeting methods encounter two significant challenges: (i) Poor cross-domain and cross-scenario transferability and generalization, and (ii) Insufficient forecastability in real-world applications. These limitations hinder their applicability across diverse industrial scenarios. In this work, we propose FOUND, an industrial-grade, transferable, and forecastable user targeting foundation model. To enhance cross-domain transferability, our framework integrates heterogeneous multi-scenario user data, aligning them with one-sentence targeting demand inputs through contrastive pre-training. For improved forecastability, the text description of each user is derived based on anticipated future behaviors, while user representations are constructed from historical information. Experimental results demonstrate that our approach significantly outperforms existing baselines in cross-domain, real-world user targeting scenarios, showcasing the superior capabilities of FOUND. Moreover, our method has been successfully deployed on the Alipay platform and is widely utilized across various scenarios.

cs.LG↗

arXiv2025

On-demand storing time-bin qubit states with optical quantum memory

Quantum memory, serving as a crucial device for storing and releasing quantum states, holds significant importance in long-distance quantum communications. Up to date, quantum memories have been realized in many different systems. However, most of them have complex structures and high costs. Besides, it is not easy to simultaneously achieve both high storage efficiency and fidelity. In this paper, we experimentally demonstrate a low-cost optical quantum memory with high efficiency and high fidelity, by utilizing a butterfly shaped cavity consisting of one polarization beam splitter, two reflecting mirrors and one pockels cell crystal. In order to quantify the quality of the quantum memory, we carry out tomography measurements on the time-bin qubits encoded with weak coherent states after storage for N rounds. The storage efficiency per round can reach up to 95.0\%, and the overall state fidelity can exceed 99.1\%. It thus seems very promising for practical implementation in quantum communication and networks.

quant-ph↗

arXiv2025

2D watershed void clustering for probing the cosmic large-scale structure

Cosmic void has been proven to be an effective cosmological probe of the large-scale structure (LSS). However, since voids are usually identified in spectroscopic galaxy surveys, they are generally limited to low number density and redshift. We propose to utilize the clustering of two-dimensional (2D) voids identified using Voronoi tessellation and watershed algorithm without any shape assumption to explore the LSS. We generate mock galaxy and void catalogs for the next-generation Stage IV photometric surveys in $z = 0.8-2.0$ from simulations, develop the 2D void identification method, and construct the theoretical model to fit the 2D watershed void and galaxy angular power spectra. We find that our method can accurately extract the cosmological information, and the constraint accuracies of some cosmological parameters from the 2D watershed void clustering are even comparable to the galaxy angular clustering case, which can be further improved by as large as $\sim30\%$ in the void and galaxy joint constraints. This indicates that the 2D void clustering is a good complement to galaxy angular clustering measurements, especially for the forthcoming Stage IV surveys that detect high-redshift universe.

astro-ph.CO↗

arXiv2025

GPT-NAS: Evolutionary Neural Architecture Search with the Generative Pre-Trained Model

Neural Architecture Search (NAS) has emerged as one of the effective methods to design the optimal neural network architecture automatically. Although neural architectures have achieved human-level performances in several tasks, few of them are obtained from the NAS method. The main reason is the huge search space of neural architectures, making NAS algorithms inefficient. This work presents a novel architecture search algorithm, called GPT-NAS, that optimizes neural architectures by Generative Pre-Trained (GPT) model with an evolutionary algorithm (EA) as the search strategy. In GPT-NAS, we assume that a generative model pre-trained on a large-scale corpus could learn the fundamental law of building neural architectures. Therefore, GPT-NAS leverages the GPT model to propose reasonable architecture components given the basic one and then utilizes EAs to search for the optimal solution. Such an approach can largely reduce the search space by introducing prior knowledge in the search process. Extensive experimental results show that our GPT-NAS method significantly outperforms seven manually designed neural architectures and thirteen architectures provided by competing NAS methods. In addition, our experiments also indicate that the proposed algorithm improves the performance of finely tuned neural architectures by up to about 12% compared to those without GPT, further demonstrating its effectiveness in searching neural architectures.

cs.CV↗

arXiv2025

PolyPath: Adapting a Large Multimodal Model for Multi-slide Pathology Report Generation

The interpretation of histopathology cases underlies many important diagnostic and treatment decisions in medicine. Notably, this process typically requires pathologists to integrate and summarize findings across multiple slides per case. Existing vision-language capabilities in computational pathology have so far been largely limited to small regions of interest, larger regions at low magnification, or single whole-slide images (WSIs). This limits interpretation of findings that span multiple high-magnification regions across multiple WSIs. By making use of Gemini 1.5 Flash, a large multimodal model (LMM) with a 1-million token context window, we demonstrate the ability to generate bottom-line diagnoses from up to 40,000 768x768 pixel image patches from multiple WSIs at 10X magnification. This is the equivalent of up to 11 hours of video at 1 fps. Expert pathologist evaluations demonstrate that the generated report text is clinically accurate and equivalent to or preferred over the original reporting for 68% (95% CI: [60%, 76%]) of multi-slide examples with up to 5 slides. While performance decreased for examples with 6 or more slides, this study demonstrates the promise of leveraging the long-context capabilities of modern LMMs for the uniquely challenging task of medical report generation where each case can contain thousands of image patches.

cs.CV↗

arXiv2025

RGB-D Indiscernible Object Counting in Underwater Scenes

Recently, indiscernible/camouflaged scene understanding has attracted lots of research attention in the vision community. We further advance the frontier of this field by systematically studying a new challenge named indiscernible object counting (IOC), the goal of which is to count objects that are blended with respect to their surroundings. Due to a lack of appropriate IOC datasets, we present a large-scale dataset IOCfish5K which contains a total of 5,637 high-resolution images and 659,024 annotated center points. Our dataset consists of a large number of indiscernible objects (mainly fish) in underwater scenes, making the annotation process all the more challenging. IOCfish5K is superior to existing datasets with indiscernible scenes because of its larger scale, higher image resolutions, more annotations, and denser scenes. All these aspects make it the most challenging dataset for IOC so far, supporting progress in this area. Benefiting from the recent advancements of depth estimation foundation models, we construct high-quality depth maps for IOCfish5K by generating pseudo labels using the Depth Anything V2 model. The RGB-D version of IOCfish5K is named IOCfish5K-D. For benchmarking purposes on IOCfish5K, we select 14 mainstream methods for object counting and carefully evaluate them. For multimodal IOCfish5K-D, we evaluate other 4 popular multimodal counting methods. Furthermore, we propose IOCFormer, a new strong baseline that combines density and regression branches in a unified framework and can effectively tackle object counting under concealed scenes. We also propose IOCFormer-D to enable the effective usage of depth modality in helping detect and count objects hidden in their environments. Experiments show that IOCFormer and IOCFormer-D achieve state-of-the-art scores on IOCfish5K and IOCfish5K-D, respectively.

cs.CV↗

arXiv2025

Chasing Contests

This paper proposes a dynamic research contest, namely chasing contest, in which two asymmetric contestants exert costly effort to accomplish two breakthroughs. The contestants are asymmetric in that one of them is present-biased and has already achieved one breakthrough (the leader), whereas the other is time-consistent and needs to achieve two breakthroughs to win (the chaser). The principal can choose between two disclosure policies: immediately announcing the chaser's first breakthrough (public chasing contest) or announcing only the final result (hidden chasing contest). We characterize the unique x-start and y-stop equilibrium under both disclosure policies, in which the leader starts working from an instant x to the end while the chaser stops exerting effort by the instant y. In addition, the chaser will never stop earlier in the hidden chasing contest, whereas a late deadline extends the leader's effort in the public contest.

econ.TH↗