arXiv subjects

Yu Duan

Publications and source records attributed to Yu Duan.

At least 19 recordsLinked to original sources

arXiv2026

One Stone, Three Birds: Self-adaptive Optimal Transport for Multi-VLM Selection, Adaptation, and Ensembling

Vision-language models (VLMs) enable visual recognition from semantic class descriptions, which makes them attractive when target annotations are scarce or unavailable. Most deployment pipelines, however, first choose a single VLM and then adapt that model to the unlabeled target set. This single-backbone paradigm hides a critical assumption: the selected VLM is already compatible with the target domain. In realistic cross-domain deployment, several general-purpose and domain-specialized VLMs may be plausible, yet no instance-level target labels are available to identify the reliable ones. Deployment therefore requires a coupled solution for model selection, target adaptation, and prediction integration. We revisit this problem from a system-level multi-VLM perspective. Our central observation is that the three decisions above depend on the same latent object: a trustworthy sample-class structure in the target set. Different VLMs may encode different transfer biases and produce conflicting predictions, but their outputs can still provide complementary evidence for estimating this structure. We propose One Stone, Three Birds, a training-free framework based on self-adaptive optimal transport. Given a pool of frozen candidate VLMs, OSTB estimates a consensus sample-to-class transport plan without updating VLM parameters. The learned transport structure is then reused for all deployment objectives: model selection is performed by ranking the combined semantic and visual reliability induced by the consensus plan; target adaptation is obtained by fitting transport-conditioned visual classifiers; and ensembling is implemented through reliability-aware probabilistic integration. Extensive experiments on natural-image, remote-sensing, and medical-pathology benchmarks show that OSTB improves model ranking, adaptation stability, and ensemble robustness under heterogeneous candidate pools.

cs.CV↗

arXiv2026

A Self-Evolving Defect Detection Framework for Industrial Photovoltaic Systems

Reliable photovoltaic (PV) power generation requires timely detection of module defects that may reduce energy yield, accelerate degradation, and increase lifecycle operation and maintenance costs during field operation. Electroluminescence (EL) imaging has therefore been widely adopted for PV module inspection. However, automated defect detection in real operational environments remains challenging due to heterogeneous module geometries, low-resolution imaging conditions, subtle defect morphology, long-tailed defect distributions, and continual data shifts introduced by evolving inspection and labeling processes. These factors significantly limit the robustness and long-term maintainability of conventional deep-learning inspection pipelines. To address these challenges, this paper proposes SEPDD, a Self-Evolving Photovoltaic Defect Detection framework designed for evolving industrial PV inspection scenarios. SEPDD integrates automated model optimization with a continual self-evolving learning mechanism, enabling the inspection system to progressively adapt to distribution shifts and newly emerging defect patterns during long-term deployment. Experiments conducted on both a public PV defect benchmark and a private industrial EL dataset demonstrate the effectiveness of the proposed framework. Both datasets exhibit severe class imbalance and significant domain shift. SEPDD achieves a leading mAP50 of 91.4% on the public dataset and 49.5% on the private dataset. It surpasses the autonomous baseline by 14.8% and human experts by 4.7% on the public dataset, and by 4.9% and 2.5%, respectively, on the private dataset.

cs.AI↗

arXiv2026

VR ProfiLens: User Profiling Risks in Consumer Virtual Reality Apps

Virtual reality (VR) platforms and apps collect user sensor data, including motion, facial, eye, and hand data, in abstracted form. These data may expose users to unique privacy risks without their knowledge or meaningful awareness, yet the extent of these risks remains understudied. To address this gap, we propose VR ProfiLens, a framework to study user profiling based on VR sensor data and the resulting privacy risks across consumer VR apps. To systematically study this problem, we first develop a taxonomy rooted in the CCPA definition of personal information and expand it by sensor, app, and threat contexts to identify user attributes at risk. Then, we conduct a user study in which we collect VR sensor data from four sensor groups from real users interacting with 10 popular consumer VR apps, followed by a survey. We design and apply an analysis pipeline to demonstrate the feasibility of inferring user attributes using these data. Our results show that sensitive personal information can be inferred with moderately high to high risk (up to 90% F1 score) from abstracted sensor data. Through feature analysis, we further identify correlations among app groups and sensor groups in inferring user attributes. Our findings highlight risks to users, including privacy loss, tracking, targeted advertising, and safety threats. Finally, we discuss design implications and regulatory recommendations to enhance transparency and better protect users' privacy in VR.

cs.CR↗

arXiv2026

Mind2Report: Expert-Level Commercial Report Synthesis via Cognitive Deep Research Agent

Synthesizing informative commercial reports from massive and noisy web sources is critical for high-stakes business decisions. Although recent deep research agents (DRAs) achieve notable progress, their reports remain limited in quality, reliability, and coverage. These mainly stem from ambiguous intents that cause search drift, retrieved web content that rapidly saturates the context window, and single-pass synthesis that limits report comprehensiveness. In this work, we propose Mind2Report, a cognitive deep research agent that emulates commercial analysts to synthesize expert-level reports. Mind2Report first probes fine-grained commercial intent to establish a structured outline, then recursively explores web sources and distills validated evidence into research memory to preserve context efficiency. Meanwhile, the research memory and outline continuously co-evolve, refining the report structure to avoid rigid initial planning. Finally, Mind2Report iteratively synthesizes the report based on the evolving outline and accumulated evidence. Together, these designs enable reliable and context-efficient long-horizon commercial deep research. To rigorously evaluate commercial DRAs, we further construct QRC-Eval, comprising 200 real-world commercial tasks and a holistic evaluation framework covering report quality, reliability, and coverage. Extensive experiments demonstrate that Mind2Report consistently outperforms leading proprietary and open-source DRAs, while ablations verify the effectiveness of each module and further analyze the challenges they address. We expect this work to advance the development of commercial deep research agents.

cs.CL↗

arXiv2025

A Hidden Stumbling Block in Generalized Category Discovery: Distracted Attention

Generalized Category Discovery (GCD) aims to classify unlabeled data from both known and unknown categories by leveraging knowledge from labeled known categories. While existing methods have made notable progress, they often overlook a hidden stumbling block in GCD: distracted attention. Specifically, when processing unlabeled data, models tend to focus not only on key objects in the image but also on task-irrelevant background regions, leading to suboptimal feature extraction. To remove this stumbling block, we propose Attention Focusing (AF), an adaptive mechanism designed to sharpen the model's focus by pruning non-informative tokens. AF consists of two simple yet effective components: Token Importance Measurement (TIME) and Token Adaptive Pruning (TAP), working in a cascade. TIME quantifies token importance across multiple scales, while TAP prunes non-informative tokens by utilizing the multi-scale importance scores provided by TIME. AF is a lightweight, plug-and-play module that integrates seamlessly into existing GCD methods with minimal computational overhead. When incorporated into one prominent GCD method, SimGCD, AF achieves up to 15.4% performance improvement over the baseline with minimal computational overhead. The implementation code is provided in https://github.com/Afleve/AFGCD.

cs.CV↗

arXiv2025

StarDojo: Benchmarking Open-Ended Behaviors of Agentic Multimodal LLMs in Production-Living Simulations with Stardew Valley

Autonomous agents navigating human society must master both production activities and social interactions, yet existing benchmarks rarely evaluate these skills simultaneously. To bridge this gap, we introduce StarDojo, a novel benchmark based on Stardew Valley, designed to assess AI agents in open-ended production-living simulations. In StarDojo, agents are tasked to perform essential livelihood activities such as farming and crafting, while simultaneously engaging in social interactions to establish relationships within a vibrant community. StarDojo features 1,000 meticulously curated tasks across five key domains: farming, crafting, exploration, combat, and social interactions. Additionally, we provide a compact subset of 100 representative tasks for efficient model evaluation. The benchmark offers a unified, user-friendly interface that eliminates the need for keyboard and mouse control, supports all major operating systems, and enables the parallel execution of multiple environment instances, making it particularly well-suited for evaluating the most capable foundation agents, powered by multimodal large language models (MLLMs). Extensive evaluations of state-of-the-art MLLMs agents demonstrate substantial limitations, with the best-performing model, GPT-4.1, achieving only a 12.7% success rate, primarily due to challenges in visual understanding, multimodal reasoning and low-level manipulation. As a user-friendly environment and benchmark, StarDojo aims to facilitate further research towards robust, open-ended agents in complex production-living environments.

cs.AI↗

arXiv2025

POCO: Scalable Neural Forecasting through Population Conditioning

Predicting future neural activity is a core challenge in modeling brain dynamics, with applications ranging from scientific investigation to closed-loop neurotechnology. While recent models of population activity emphasize interpretability and behavioral decoding, neural forecasting-particularly across multi-session, spontaneous recordings-remains underexplored. We introduce POCO, a unified forecasting model that combines a lightweight univariate forecaster with a population-level encoder to capture both neuron-specific and brain-wide dynamics. Trained across five calcium imaging datasets spanning zebrafish, mice, and C. elegans, POCO achieves state-of-the-art accuracy at cellular resolution in spontaneous behaviors. After pre-training, POCO rapidly adapts to new recordings with minimal fine-tuning. Notably, POCO's learned unit embeddings recover biologically meaningful structure-such as brain region clustering-without any anatomical labels. Our comprehensive analysis reveals several key factors influencing performance, including context length, session diversity, and preprocessing. Together, these results position POCO as a scalable and adaptable approach for cross-session neural forecasting and offer actionable insights for future model design. By enabling accurate, generalizable forecasting models of neural dynamics across individuals and species, POCO lays the groundwork for adaptive neurotechnologies and large-scale efforts for neural foundation models. Code is available at https://github.com/yuvenduan/POCO.

q-bio.NC↗

arXiv2025

SOTA: Self-adaptive Optimal Transport for Zero-Shot Classification with Multiple Foundation Models

Foundation models have attracted widespread attention across domains due to their powerful zero-shot classification capabilities. This work is motivated by two key observations: (1) \textit{Vision-Language Models} (VLMs), such as CLIP, often over-rely on class-level textual priors and struggle to capture fine-grained visual cues, whereas \textit{Vision-only Foundation Models} (VFMs), such as DINO, provide rich and discriminative visual features but lack semantic alignment; (2) the performance of different VLMs varies considerably across datasets owing to differences in pre-training. To address these challenges, we propose \textbf{SOTA} (\textit{Self-adaptive Optimal TrAnsport}), a \textit{training-free} ensemble framework that integrates the outputs of multiple foundation models~(VFMs or VLMs) by learning a self-adaptive transport plan. Notably, \textbf{SOTA} is prior-free and automatically balances model contributions. Extensive experiments across diverse domains, including natural images, medical pathology, and remote sensing, validate the generalizability of \textbf{SOTA}. The results consistently show that it effectively leverages the complementary strengths of different foundation models and achieves substantial improvements over individual models. The implementation code is available at: https://github.com/Afleve/self-adaptive-Optimal-Transport.

cs.CV↗

arXiv2025

Efficient Multi-Instance Generation with Janus-Pro-Dirven Prompt Parsing

Recent advances in text-guided diffusion models have revolutionized conditional image generation, yet they struggle to synthesize complex scenes with multiple objects due to imprecise spatial grounding and limited scalability. We address these challenges through two key modules: 1) Janus-Pro-driven Prompt Parsing, a prompt-layout parsing module that bridges text understanding and layout generation via a compact 1B-parameter architecture, and 2) MIGLoRA, a parameter-efficient plug-in integrating Low-Rank Adaptation (LoRA) into UNet (SD1.5) and DiT (SD3) backbones. MIGLoRA is capable of preserving the base model's parameters and ensuring plug-and-play adaptability, minimizing architectural intrusion while enabling efficient fine-tuning. To support a comprehensive evaluation, we create DescripBox and DescripBox-1024, benchmarks that span diverse scenes and resolutions. The proposed method achieves state-of-the-art performance on COCO and LVIS benchmarks while maintaining parameter efficiency, demonstrating superior layout fidelity and scalability for open-world synthesis.

cs.CV↗

arXiv2024

Reentrant phase behavior in binary topological flocks with nonreciprocal alignment

We study a binary metric-free Vicsek model involving two species of self-propelled particles aligning with their Voronoi neighbors, focusing on a weakly nonreciprocal regime, where species $A$ aligns with both $A$ and $B$, but species $B$ does not align with either. Using agent-based simulations, we find that even with a small fraction of $B$ particles, the phase behavior of the system can be changed qualitatively, which becomes reentrant as a function of noise strength: traveling bands arise not only near the flocking transition, but also in the low-noise regime, separated in the phase diagram by a homogeneous polar liquid regime. We find that the ordered bands in the low-noise regime travel through an ordered background, in contrast to their metric counterparts. We develop a coarse-grained field theory, which can account for the reentrant phase behavior qualitatively, provided the higher-order angular modes are taken into consideration.

cond-mat.soft↗

arXiv2024

Phase Coexistence in Nonreciprocal Quorum-Sensing Active Matter

Motility and nonreciprocity are two primary mechanisms for self-organization in active matter. In a recent study [Phys. Rev. Lett. 131, 148301 (2023)], we explored their joint influence in a minimal model of two-species quorum-sensing active particles interacting via mutual motility regulation. Our results notably revealed a highly dynamic phase of chaotic chasing bands that is absent when either nonreciprocity or self-propulsion is missing. Here, we examine further the phase behavior of nonreciprocal quorum-sensing active particles, distinguishing between the regimes of weak and strong nonreciprocity. In the weakly nonreciprocal regime, this system exhibits multi-component motility-induced phase separation. We establish an analytical criterion for the associated phase coexistence, enabling a quantitative prediction of the phase diagram. For strong nonreciprocity, where the dynamics is chase-and-run-like, we numerically determine the phase behavior and show that it strongly depends on the scale of observation. In small systems, our numerical simulations reveal a phenomenology consistent with phenomenological models, comprising traveling phase-separated domains and spiral-like defect patterns. However, we show that these structures are generically unstable in large systems, where they are superseded by bulk phase coexistence between domains that are either homogeneous or populated by mesoscopic chasing bands. Crucially, this implies that collective motion totally vanishes at large scales, while the breakdown of our analytical criterion for this phase coexistence with multi-scale structures prevents us from predicting the corresponding phase diagram.

cond-mat.soft↗

arXiv2024

Simultaneous Localization and Recognition of Subwavelength Non-Cooperative Entities Based on SISO Time Reversal and Neural Networks

The simultaneous localization and recognition of subwavelength non-cooperative entities within complex multi-scattering environments using a simplified system continues to pose a substantial challenge. This letter addresses this challenge by synergistically integrating time reversal time-frequency phase prints (TRTFPPs) and neural networks. Initially, a time reversal (TR) single-input single-output (SISO) framework is employed to generate TRTFPPs. To enhance the models' adaptability, particularly in the presence of noise, data augmentation techniques are applied. Subsequently, neural networks are employed to comprehend the TRTFPPs. Specifically, a cascaded neural network structure is embraced, encompassing both a recognition neural network and distinct neural networks for localizing different entities. Through the devised approach, two types of subwavelength entities are successfully identified and precisely localized through numerical simulations and experimental verification in laboratory environment. The proposed methodology holds applicability across various electromagnetic systems, including but not limited to detection, imaging, human-computer interaction, and the Internet of Things (IoT).

physics.app-ph↗

arXiv2023

A Kronecker product accelerated efficient sparse Gaussian Process (E-SGP) for flow emulation

In this paper, we introduce an efficient sparse Gaussian process (E-SGP) for the surrogate modelling of fluid mechanics. This novel Bayesian machine learning algorithm allows efficient model training using databases of different structures. It is a further development of the approximated sparse GP algorithm, combining the concept of efficient GP (E-GP) and variational energy free sparse Gaussian process (VEF-SGP). The developed E-SGP approach exploits the arbitrariness of inducing points and the monotonically increasing nature of the objective function with respect to the number of inducing points in VEF-SGP. By specifying the inducing points on the orthogonal grid/input subspace and using the Kronecker product, E-SGP significantly improves computational efficiency without imposing any constraints on the covariance matrix or increasing the number of parameters that need to be optimised during training. The E-SGP algorithm developed in this paper outperforms E-GP not only in scalability but also in model quality in terms of mean standardized logarithmic loss (MSLL). The computational complexity of E-GP suffers from the cubic growth regarding the growing structured training database. However, E-SGP maintains computational efficiency whilst the resolution of the model, (i.e., the number of inducing points) remains fixed. The examples show that E-SGP produces more accurate predictions in comparison with E-GP when the model resolutions are similar in both. E-GP benefits from more training data but comes with higher computational demands, while E-SGP achieves a comparable level of accuracy but is more computationally efficient, making E-SGP a potentially preferable choice for fluid mechanic problems. Furthermore, E-SGP can produce more reasonable estimates of model uncertainty, whilst E-GP is more likely to produce over-confident predictions.

cs.LG↗

arXiv2023

BehaVR: User Identification Based on VR Sensor Data

Virtual reality (VR) platforms enable a wide range of applications, however, pose unique privacy risks. In particular, VR devices are equipped with a rich set of sensors that collect personal and sensitive information (e.g., body motion, eye gaze, hand joints, and facial expression). The data from these newly available sensors can be used to uniquely identify a user, even in the absence of explicit identifiers. In this paper, we seek to understand the extent to which a user can be identified based solely on VR sensor data, within and across real-world apps from diverse genres. We consider adversaries with capabilities that range from observing APIs available within a single app (app adversary) to observing all or selected sensor measurements across multiple apps on the VR device (device adversary). To that end, we introduce BehaVR, a framework for collecting and analyzing data from all sensor groups collected by multiple apps running on a VR device. We use BehaVR to collect data from real users that interact with 20 popular real-world apps. We use that data to build machine learning models for user identification within and across apps, with features extracted from available sensor data. We show that these models can identify users with an accuracy of up to 100%, and we reveal the most important features and sensor groups, depending on the functionality of the app and the adversary. To the best of our knowledge, BehaVR is the first to analyze user identification in VR comprehensively, i.e., considering all sensor measurements available on consumer VR devices, collected by multiple real-world, as opposed to custom-made, apps.

cs.HC↗

arXiv2023

Synthesized complex-frequency excitation for ultrasensitive molecular sensing

Detecting trace molecules remains a significant challenge. Surface-enhanced infrared absorption (SEIRA) based on plasmonic nanostructures, particularly graphene, has emerged as a promising approach to enhance sensing sensitivity. While graphene-based SEIRA offers advantages such as ultrahigh sensitivity and active tunability, intrinsic molecular damping weakens the interaction between vibrational modes and plasmons. Here, we demonstrate ultrahigh-sensitive molecular sensing based on synthesized complex-frequency waves (CFW). Our experiment shows that CFW can amplify the molecular signals (~1.2-nm-thick silk protein layer) detected by graphene-based sensor by at least an order of magnitude and can be universally applied to molecular sensing in different phases. Our approach is highly scalable and can facilitate the investigation of light-matter interactions, enabling diverse potential applications in fields such as optical spectroscopy, metasurfaces, optoelectronics, biomedicine and pharmaceutics.

physics.optics↗

arXiv2023

Dynamical pattern formation without self-attraction in quorum-sensing active matter: the interplay between nonreciprocity and motility

We study a minimal model involving two species of particles interacting via quorum-sensing rules. Combining simulations of the microscopic model and linear stability analysis of the associated coarse-grained field theory, we identify a mechanism for dynamical pattern formation that does not rely on the standard route of intra-species effective attractive interactions. Instead, our results reveal a highly dynamical phase of chasing bands induced only by the combined effects of self-propulsion and nonreciprocity in the inter-species couplings. Turning on self-attraction, we find that the system may phase separate into a macroscopic domain of such chaotic chasing bands coexisting with a dilute gas. We show that the chaotic dynamics of bands at the interfaces of this phase-separated phase results in anomalously slow coarsening.

cond-mat.stat-mech↗

arXiv2023

Hebbian and Gradient-based Plasticity Enables Robust Memory and Rapid Learning in RNNs

Rapidly learning from ongoing experiences and remembering past events with a flexible memory system are two core capacities of biological intelligence. While the underlying neural mechanisms are not fully understood, various evidence supports that synaptic plasticity plays a critical role in memory formation and fast learning. Inspired by these results, we equip Recurrent Neural Networks (RNNs) with plasticity rules to enable them to adapt their parameters according to ongoing experiences. In addition to the traditional local Hebbian plasticity, we propose a global, gradient-based plasticity rule, which allows the model to evolve towards its self-determined target. Our models show promising results on sequential and associative memory tasks, illustrating their ability to robustly form and retain memories. In the meantime, these models can cope with many challenging few-shot learning problems. Comparing different plasticity rules under the same framework shows that Hebbian plasticity is well-suited for several memory and associative learning tasks; however, it is outperformed by gradient-based plasticity on few-shot regression tasks which require the model to infer the underlying mapping. Code is available at https://github.com/yuvenduan/PlasticRNNs.

cs.NE↗

arXiv2022

Collaborative Intelligence Orchestration: Inconsistency-Based Fusion of Semi-Supervised Learning and Active Learning

While annotating decent amounts of data to satisfy sophisticated learning models can be cost-prohibitive for many real-world applications. Active learning (AL) and semi-supervised learning (SSL) are two effective, but often isolated, means to alleviate the data-hungry problem. Some recent studies explored the potential of combining AL and SSL to better probe the unlabeled data. However, almost all these contemporary SSL-AL works use a simple combination strategy, ignoring SSL and AL's inherent relation. Further, other methods suffer from high computational costs when dealing with large-scale, high-dimensional datasets. Motivated by the industry practice of labeling data, we propose an innovative Inconsistency-based virtual aDvErsarial Active Learning (IDEAL) algorithm to further investigate SSL-AL's potential superiority and achieve mutual enhancement of AL and SSL, i.e., SSL propagates label information to unlabeled samples and provides smoothed embeddings for AL, while AL excludes samples with inconsistent predictions and considerable uncertainty for SSL. We estimate unlabeled samples' inconsistency by augmentation strategies of different granularities, including fine-grained continuous perturbation exploration and coarse-grained data transformations. Extensive experiments, in both text and image domains, validate the effectiveness of the proposed algorithm, comparing it against state-of-the-art baselines. Two real-world case studies visualize the practical industrial value of applying and deploying the proposed data sampling algorithm.

cs.LG↗