arXiv subjects

Ziyi Liu

Publications and source records attributed to Ziyi Liu.

At least 55 records · Page 3Linked to original sources

arXiv2025

Generative AI for Misalignment-Resistant Virtual Staining to Accelerate Histopathology Workflows

Accurate histopathological diagnosis often requires multiple differently stained tissue sections, a process that is time-consuming, labor-intensive, and environmentally taxing due to the use of multiple chemical stains. Recently, virtual staining has emerged as a promising alternative that is faster, tissue-conserving, and environmentally friendly. However, existing virtual staining methods face significant challenges in clinical applications, primarily due to their reliance on well-aligned paired data. Obtaining such data is inherently difficult because chemical staining processes can distort tissue structures, and a single tissue section cannot undergo multiple staining procedures without damage or loss of information. As a result, most available virtual staining datasets are either unpaired or roughly paired, making it difficult for existing methods to achieve accurate pixel-level supervision. To address this challenge, we propose a robust virtual staining framework featuring cascaded registration mechanisms to resolve spatial mismatches between generated outputs and their corresponding ground truth. Experimental results demonstrate that our method significantly outperforms state-of-the-art models across five datasets, achieving an average improvement of 3.2% on internal datasets and 10.1% on external datasets. Moreover, in datasets with substantial misalignment, our approach achieves a remarkable 23.8% improvement in peak signal-to-noise ratio compared to baseline models. The exceptional robustness of the proposed method across diverse datasets simplifies the data acquisition process for virtual staining and offers new insights for advancing its development.

cs.CV↗

arXiv2025

AI Sees Your Location, But With A Bias Toward The Wealthy World

Visual-Language Models (VLMs) have shown remarkable performance across various tasks, particularly in recognizing geographic information from images. However, VLMs still show regional biases in this task. To systematically evaluate these issues, we introduce a benchmark consisting of 1,200 images paired with detailed geographic metadata. Evaluating four VLMs, we find that while these models demonstrate the ability to recognize geographic information from images, achieving up to 53.8% accuracy in city prediction, they exhibit significant biases. Specifically, performance is substantially higher for economically developed and densely populated regions compared to less developed (-12.5%) and sparsely populated (-17.0%) areas. Moreover, regional biases of frequently over-predicting certain locations remain. For instance, they consistently predict Sydney for images taken in Australia, shown by the low entropy scores for these countries. The strong performance of VLMs also raises privacy concerns, particularly for users who share images online without the intent of being identified. Our code and dataset are publicly available at https://github.com/uscnlp-lime/FairLocator.

cs.CV↗

arXiv2025

A Unified Low-level Foundation Model for Enhancing Pathology Image Quality

Foundation models have revolutionized computational pathology by achieving remarkable success in high-level diagnostic tasks, yet the critical challenge of low-level image enhancement remains largely unaddressed. Real-world pathology images frequently suffer from degradations such as noise, blur, and low resolution due to slide preparation artifacts, staining variability, and imaging constraints, while the reliance on physical staining introduces significant costs, delays, and inconsistency. Although existing methods target individual problems like denoising or super-resolution, their task-specific designs lack the versatility to handle the diverse low-level vision challenges encountered in practice. To bridge this gap, we propose the first unified Low-level Pathology Foundation Model (LPFM), capable of enhancing image quality in restoration tasks, including super-resolution, deblurring, and denoising, as well as facilitating image translation tasks like virtual staining (H&E and special stains), all through a single adaptable architecture. Our approach introduces a contrastive pre-trained encoder that learns transferable, stain-invariant feature representations from 190 million unlabeled pathology images, enabling robust identification of degradation patterns. A unified conditional diffusion process dynamically adapts to specific tasks via textual prompts, ensuring precise control over output quality. Trained on a curated dataset of 87,810 whole slied images (WSIs) across 34 tissue types and 5 staining protocols, LPFM demonstrates statistically significant improvements (p<0.01) over state-of-the-art methods in most tasks (56/66), achieving Peak Signal-to-Noise Ratio (PSNR) gains of 10-15% for image restoration and Structural Similarity Index Measure (SSIM) improvements of 12-18% for virtual staining.

cs.CV↗

arXiv2025

Geospatial Question Answering on Historical Maps Using Spatio-Temporal Knowledge Graphs and Large Language Models

Recent advances have enabled the extraction of vectorized features from digital historical maps. To fully leverage this information, however, the extracted features must be organized in a structured and meaningful way that supports efficient access and use. One promising approach is question answering (QA), which allows users -- especially those unfamiliar with database query languages -- to retrieve knowledge in a natural and intuitive manner. In this project, we developed a GeoQA system by integrating a spatio-temporal knowledge graph (KG) constructed from historical map data with large language models (LLMs). Specifically, we have defined the ontology to guide the construction of the spatio-temporal KG and investigated workflows of two different types of GeoQA: factual and descriptive. Additional data sources, such as historical map images and internet search results, are incorporated into our framework to provide extra context for descriptive GeoQA. Evaluation results demonstrate that the system can generate answers with a high delivery rate and a high semantic accuracy. To make the framework accessible, we further developed a web application that supports interactive querying and visualization.

cs.IR↗

arXiv2025

A Versatile Pathology Co-pilot via Reasoning Enhanced Multimodal Large Language Model

Multimodal large language models (MLLMs) have emerged as powerful tools for computational pathology, offering unprecedented opportunities to integrate pathological images with language context for comprehensive diagnostic analysis. These models hold particular promise for automating complex tasks that traditionally require expert interpretation of pathologists. However, current MLLM approaches in pathology demonstrate significantly constrained reasoning capabilities, primarily due to their reliance on expensive chain-of-thought annotations. Additionally, existing methods remain limited to simplex application of visual question answering (VQA) at the region-of-interest (ROI) level, failing to address the full spectrum of diagnostic needs such as ROI classification, detection, segmentation, whole-slide-image (WSI) classification and VQA in clinical practice. In this study, we present SmartPath-R1, a versatile MLLM capable of simultaneously addressing both ROI-level and WSI-level tasks while demonstrating robust pathological reasoning capability. Our framework combines scale-dependent supervised fine-tuning and task-aware reinforcement fine-tuning, which circumvents the requirement for chain-of-thought supervision by leveraging the intrinsic knowledge within MLLM. Furthermore, SmartPath-R1 integrates multiscale and multitask analysis through a mixture-of-experts mechanism, enabling dynamic processing for diverse tasks. We curate a large-scale dataset comprising 2.3M ROI samples and 188K WSI samples for training and evaluation. Extensive experiments across 72 tasks validate the effectiveness and superiority of the proposed approach. This work represents a significant step toward developing versatile, reasoning-enhanced AI systems for precision pathology.

eess.IV↗

arXiv2025

SketchConcept: Sketching-based Concept Recomposition for Product Design using Generative AI

Conceptual product design requires designers to explore the design space of visual and functional concepts simultaneously. Sketching has long been adopted to empower concept exploration. However, current sketch-based design tools mostly emphasize visual design using emerging techniques. We present SketchConcept, a design support tool that decomposes design concepts into visual representations and functionality of concepts using sketches and textual descriptions. We propose a function-to-visual mapping workflow that maps the function descriptions generated by a Large Language Model to a component of the concept produced by image Generative Artificial Intelligence(GenAI). The function-to-visual mapping allows our system to leverage multimodal GenAI to decompose, generate, and edit the design concept to satisfy the overall function and behavior. We present multiple use cases enabled by SketchConcept to validate the workflow. Finally, we evaluated the efficacy and usability of our system with a two-session user study.

cs.HC↗

arXiv2025

Length Matters: Length-Aware Transformer for Temporal Sentence Grounding

Temporal sentence grounding (TSG) is a highly challenging task aiming to localize the temporal segment within an untrimmed video corresponding to a given natural language description. Benefiting from the design of learnable queries, the DETR-based models have achieved substantial advancements in the TSG task. However, the absence of explicit supervision often causes the learned queries to overlap in roles, leading to redundant predictions. Therefore, we propose to improve TSG by making each query fulfill its designated role, leveraging the length priors of the video-description pairs. In this paper, we introduce the Length-Aware Transformer (LATR) for TSG, which assigns different queries to handle predictions based on varying temporal lengths. Specifically, we divide all queries into three groups, responsible for segments with short, middle, and long temporal durations, respectively. During training, an additional length classification task is introduced. Predictions from queries with mismatched lengths are suppressed, guiding each query to specialize in its designated function. Extensive experiments demonstrate the effectiveness of our LATR, achieving state-of-the-art performance on three public benchmarks. Furthermore, the ablation studies validate the contribution of each component of our method and the critical role of incorporating length priors into the TSG task.

cs.CV↗

arXiv2025

A Segmentation Framework for Accurate Diagnosis of Amyloid Positivity without Structural Images

This study proposes a deep learning-based framework for automated segmentation of brain regions and classification of amyloid positivity using positron emission tomography (PET) images alone, without the need for structural MRI or CT. A 3D U-Net architecture with four layers of depth was trained and validated on a dataset of 200 F18-florbetapir amyloid-PET scans, with an 130/20/50 train/validation/test split. Segmentation performance was evaluated using Dice similarity coefficients across 30 brain regions, with scores ranging from 0.45 to 0.88, demonstrating high anatomical accuracy, particularly in subcortical structures. Quantitative fidelity of PET uptake within clinically relevant regions. Precuneus, prefrontal cortex, gyrus rectus, and lateral temporal cortex was assessed using normalized root mean square error, achieving values as low as 0.0011. Furthermore, the model achieved a classification accuracy of 0.98 for amyloid positivity based on regional uptake quantification, with an area under the ROC curve (AUC) of 0.99. These results highlight the model's potential for integration into PET only diagnostic pipelines, particularly in settings where structural imaging is not available. This approach reduces dependence on coregistration and manual delineation, enabling scalable, reliable, and reproducible analysis in clinical and research applications. Future work will focus on clinical validation and extension to diverse PET tracers including C11 PiB and other F18 labeled compounds.

eess.IV↗

arXiv2025

AI Agent Behavioral Science

Recent advances in large language models (LLMs) have enabled the development of AI agents that exhibit increasingly human-like behaviors, including planning, adaptation, and social dynamics across diverse, interactive, and open-ended scenarios. These behaviors are not solely the product of the internal architectures of the underlying models, but emerge from their integration into agentic systems operating within specific contexts, where environmental factors, social cues, and interaction feedbacks shape behavior over time. This evolution necessitates a new scientific perspective: AI Agent Behavioral Science. Rather than focusing only on internal mechanisms, this perspective emphasizes the systematic observation of behavior, design of interventions to test hypotheses, and theory-guided interpretation of how AI agents act, adapt, and interact over time. We systematize a growing body of research across individual agent, multi-agent, and human-agent interaction settings, and further demonstrate how this perspective informs responsible AI by treating fairness, safety, interpretability, accountability, and privacy as behavioral properties. By unifying recent findings and laying out future directions, we position AI Agent Behavioral Science as a necessary complement to traditional model-centric approaches, providing essential tools for understanding, evaluating, and governing the real-world behavior of increasingly autonomous AI systems.

q-bio.NC↗

arXiv2025

Discovering Pathology Rationale and Token Allocation for Efficient Multimodal Pathology Reasoning

Multimodal pathological image understanding has garnered widespread interest due to its potential to improve diagnostic accuracy and enable personalized treatment through integrated visual and textual data. However, existing methods exhibit limited reasoning capabilities, which hamper their ability to handle complex diagnostic scenarios. Additionally, the enormous size of pathological images leads to severe computational burdens, further restricting their practical deployment. To address these limitations, we introduce a novel bilateral reinforcement learning framework comprising two synergistic branches. One reinforcement branch enhances the reasoning capability by enabling the model to learn task-specific decision processes, i.e., pathology rationales, directly from labels without explicit reasoning supervision. While the other branch dynamically allocates a tailored number of tokens to different images based on both their visual content and task context, thereby optimizing computational efficiency. We apply our method to various pathological tasks such as visual question answering, cancer subtyping, and lesion detection. Extensive experiments show an average +41.7 absolute performance improvement with 70.3% lower inference costs over the base models, achieving both reasoning accuracy and computational efficiency.

cs.CV↗

arXiv2025

GesPrompt: Leveraging Co-Speech Gestures to Augment LLM-Based Interaction in Virtual Reality

Large Language Model (LLM)-based copilots have shown great potential in Extended Reality (XR) applications. However, the user faces challenges when describing the 3D environments to the copilots due to the complexity of conveying spatial-temporal information through text or speech alone. To address this, we introduce GesPrompt, a multimodal XR interface that combines co-speech gestures with speech, allowing end-users to communicate more naturally and accurately with LLM-based copilots in XR environments. By incorporating gestures, GesPrompt extracts spatial-temporal reference from co-speech gestures, reducing the need for precise textual prompts and minimizing cognitive load for end-users. Our contributions include (1) a workflow to integrate gesture and speech input in the XR environment, (2) a prototype VR system that implements the workflow, and (3) a user study demonstrating its effectiveness in improving user communication in VR environments.

cs.HC↗

arXiv2025

Bridge the Gap: From Weak to Full Supervision for Temporal Action Localization with PseudoFormer

Weakly-supervised Temporal Action Localization (WTAL) has achieved notable success but still suffers from a lack of temporal annotations, leading to a performance and framework gap compared with fully-supervised methods. While recent approaches employ pseudo labels for training, three key challenges: generating high-quality pseudo labels, making full use of different priors, and optimizing training methods with noisy labels remain unresolved. Due to these perspectives, we propose PseudoFormer, a novel two-branch framework that bridges the gap between weakly and fully-supervised Temporal Action Localization (TAL). We first introduce RickerFusion, which maps all predicted action proposals to a global shared space to generate pseudo labels with better quality. Subsequently, we leverage both snippet-level and proposal-level labels with different priors from the weak branch to train the regression-based model in the full branch. Finally, the uncertainty mask and iterative refinement mechanism are applied for training with noisy pseudo labels. PseudoFormer achieves state-of-the-art WTAL results on the two commonly used benchmarks, THUMOS14 and ActivityNet1.3. Besides, extensive ablation studies demonstrate the contribution of each component of our method.

cs.CV↗

arXiv2025

Virtual Reality in Manufacturing Education: A Scoping Review Indicating State-of-the-Art, Benefits, and Challenges Across Domains, Levels, and Entities

To address the shortage of a skilled workforce in the U.S. manufacturing industry, immersive Virtual Reality (VR)-based training solutions hold promising potential. To effectively utilize VR to meet workforce demands, it is important to understand the role of VR in manufacturing education. Therefore, we conduct a scoping review in the field. As a first step, we used a 5W1H (What, Where, Who, When, Why, How) formula as a problem-solving approach to define a comprehensive taxonomy that can consider the role of VR from all relevant possibilities. Our taxonomy categorizes VR applications across three key aspects: (1) Domains, (2) Levels, and (3) Entities. Using a systematic literature search and analysis, we reviewed 108 research articles to find the current state, benefits, challenges, and future opportunities of VR in the field. It was found that VR has been explored in a variety of areas and provides numerous benefits to learners. Despite these benefits, its adoption in manufacturing education is limited. This review discusses the identified barriers and provides actionable insights to address them. These insights can enable the widespread usage of immersive technology to nurture and develop a workforce equipped with the skills required to excel in the evolving landscape of manufacturing.

cs.HC↗

arXiv2025

Make the Most of Everything: Further Considerations on Disrupting Diffusion-based Customization

The fine-tuning technique for text-to-image diffusion models facilitates image customization but risks privacy breaches and opinion manipulation. Current research focuses on prompt- or image-level adversarial attacks for anti-customization, yet it overlooks the correlation between these two levels and the relationship between internal modules and inputs. This hinders anti-customization performance in practical threat scenarios. We propose Dual Anti-Diffusion (DADiff), a two-stage adversarial attack targeting diffusion customization, which, for the first time, integrates the adversarial prompt-level attack into the generation process of image-level adversarial examples. In stage 1, we generate prompt-level adversarial vectors to guide the subsequent image-level attack. In stage 2, besides conducting the end-to-end attack on the UNet model, we disrupt its self- and cross-attention modules, aiming to break the correlations between image pixels and align the cross-attention results computed using instance prompts and adversarial prompt vectors within the images. Furthermore, we introduce a local random timestep gradient ensemble strategy, which updates adversarial perturbations by integrating random gradients from multiple segmented timesets. Experimental results on various mainstream facial datasets demonstrate 10%-30% improvements in cross-prompt, keyword mismatch, cross-model, and cross-mechanism anti-customization with DADiff compared to existing methods.

cs.CV↗

arXiv2025

A Universal Scheme for Dynamic Partitioned Shortest Path Index: Survey, Improvement, and Experiments

Shortest Path (SP) computation is a fundamental operation in many real-life applications such as navigation on road networks, link analysis on social networks, etc. These networks tend to be massive, and graph partitioning is commonly leveraged to scale up the SP algorithms. However, the Partitioned Shortest Path (PSP) index has never been systematically investigated. Moreover, few studies have explored its index maintenance in dynamic networks. In this paper, we survey the dynamic PSP index and propose a universal scheme for its design and analysis. Specifically, we first review the SP algorithms and put forward a novel structure-based partition method classification to facilitate the selection of partition methods. Furthermore, we summarize the existing Pre-boundary PSP strategy and propose two novel strategies (No-boundary and Post-boundary) to improve its index performance. Lastly, we propose a universal scheme with three dimensions (SP algorithm, partition method, and PSP strategy) to facilitate the analysis and design of the PSP index. Based on this scheme, we put forward five new PSP indexes with a prominent query or update efficiency performance. Extensive experiments are conducted to evaluate the performance of the PSP index and the effectiveness of the proposed techniques, with valuable guidance on the PSP index design.

cs.DB↗

arXiv2025

Extracting intrinsic superconducting properties in intercalated layered superconductors using an extended 2D Tinkham model

Bulk two dimensional (2D) superconductivity has gained considerable attention due to its intricate interplay between symmetry breaking, nontrivial topology, 2D phase fluctuations, and unconventional superconductivity. However, certain intercalated layered superconductors, despite their short c-axis superconducting coherence length, have been misclassified as anisotropic three-dimensional (3D) superconductors. Here, we investigate (Li,Fe)OHFeSe superconductors with varying degrees of interlayer misalignment, revealing sample-dependent superconducting dimensionality while consistently observing Berezinskii Kosterlitz Thouless (BKT) transitions. To resolve this discrepancy, we develop an extended 2D Tinkham model that quantitatively captures the blurring effects induced by interlayer misalignment. We further demonstrate the validity of this model in both (Li,Fe)OHFeSe and cetyltrimethyl ammonium (CTA+) intercalated (CTA)0.5SnSe2 superconductors, highlighting its broad applicability. This work provides valuable insights into bulk 2D superconductivity and establishes an extended 2D Tinkham model for quantitatively extracting intrinsic superconducting properties in intercalated layered superconductors, particularly those exhibiting significant interlayer misalignments.

cond-mat.supr-con↗

arXiv2025

Improving Interactive Diagnostic Ability of a Large Language Model Agent Through Clinical Experience Learning

Recent advances in large language models (LLMs) have shown promising results in medical diagnosis, with some studies indicating superior performance compared to human physicians in specific scenarios. However, the diagnostic capabilities of LLMs are often overestimated, as their performance significantly deteriorates in interactive diagnostic settings that require active information gathering. This study investigates the underlying mechanisms behind the performance degradation phenomenon and proposes a solution. We identified that the primary deficiency of LLMs lies in the initial diagnosis phase, particularly in information-gathering efficiency and initial diagnosis formation, rather than in the subsequent differential diagnosis phase. To address this limitation, we developed a plug-and-play method enhanced (PPME) LLM agent, leveraging over 3.5 million electronic medical records from Chinese and American healthcare facilities. Our approach integrates specialized models for initial disease diagnosis and inquiry into the history of the present illness, trained through supervised and reinforcement learning techniques. The experimental results indicate that the PPME LLM achieved over 30% improvement compared to baselines. The final diagnostic accuracy of the PPME LLM in interactive diagnostic scenarios approached levels comparable to those achieved using complete clinical data. These findings suggest a promising potential for developing autonomous diagnostic systems, although further validation studies are needed.

cs.AI↗

arXiv2025

A Response to Recent Critiques of Hainmueller, Mummolo and Xu (2019) on Estimating Conditional Relationships

Simonsohn (2024a) and Simonsohn (2024b) critique Hainmueller, Mummolo and Xu (2019, HMX), arguing that failing to model nonlinear relationships between the treatment and moderator leads to biased marginal effect estimates and uncontrolled Type-I error rates. While these critiques highlight the issue of under-modeling nonlinearity in applied research, they are fundamentally flawed in several key ways. First, the causal estimand for interaction effects and the necessary identifying assumptions are not clearly defined in these critiques. Once properly stated, the critiques no longer hold. Second, the kernel estimator HMX proposes recovers the true causal effects in the scenarios presented in these recent critiques, which compared effects to the wrong benchmark, producing misleading conclusions. Third, while Generalized Additive Models (GAM) can be a useful exploratory tool (as acknowledged in HMX), they are not designed to estimate marginal effects, and better alternatives exist, particularly in the presence of additional covariates. Our response aims to clarify these misconceptions and provide updated recommendations for researchers studying interaction effects through the estimation of conditional marginal effects.

stat.ME↗