arXiv subjects

Siyi Chen

Publications and source records attributed to Siyi Chen.

At least 37 records · Page 2Linked to original sources

arXiv2026

Understanding Representation Dynamics of Diffusion Models via Low-Dimensional Modeling

Diffusion models, though originally designed for generative tasks, have demonstrated impressive self-supervised representation learning capabilities. A particularly intriguing phenomenon in these models is the emergence of unimodal representation dynamics, where the quality of learned features peaks at an intermediate noise level. In this work, we conduct a comprehensive theoretical and empirical investigation of this phenomenon. Leveraging the inherent low-dimensionality structure of image data, we theoretically demonstrate that the unimodal dynamic emerges when the diffusion model successfully captures the underlying data distribution. The unimodality arises from an interplay between denoising strength and class confidence across noise scales. Empirically, we further show that, in classification tasks, the presence of unimodal dynamics reliably reflects the generalization of the diffusion model: it emerges when the model generates novel images and gradually transitions to a monotonically decreasing curve as the model begins to memorize the training data.

cs.LG↗

arXiv2026

MVT: Mask-Grounded Vision-Language Models for Taxonomy-Aligned Land-Cover Tagging

Land-cover understanding in remote sensing increasingly demands class-agnostic systems that generalize across datasets while remaining spatially precise and interpretable. We study a geometry-first discovery-and-interpretation setting under domain shift, where candidate regions are delineated class-agnostically and supervision avoids lexical class names via anonymized identifiers. Complementary to open-set recognition and open-world learning, we focus on coupling class-agnostic mask evidence with taxonomy-grounded scene interpretation, rather than unknown rejection or continual class expansion. We propose MVT, a three-stage framework that (i) extracts boundary-faithful region masks using SAM2 with domain adaptation, (ii) performs mask-grounded semantic tagging and scene description generation via dual-step LoRA fine-tuning of multimodal LLMs, and (iii) evaluates outputs with LLM-as-judge scoring calibrated by stratified expert ratings. On cross-dataset segmentation transfer (train on OpenEarthMap, evaluate on LoveDA), domain-adapted SAM2 improves mask quality; meanwhile, dual-step MLLM fine-tuning yields more accurate taxonomy-aligned tags and more informative mask-grounded scene descriptions.

cs.CV↗

arXiv2026

Quantum Data Centres: Why Entanglement Changes Everything

The Quantum Internet is key for distributed quantum computing, by interconnecting multiple quantum processors into a virtual quantum computation system. This allows to scale the number of qubits, by overcoming the inherent limitations of noisy-intermediate-scale quantum (NISQ) devices. Thus, the Quantum Internet is the foundation for large-scale, fault-tolerant quantum computation. Among the distributed architectures, Quantum Data Centres emerge as the most viable in the medium-term, since they integrate multiple quantum processors within a localized network infrastructure, by allowing modular design of quantum networking. We analyze the physical and topological constraints of Quantum Data Centres, by emphasizing the role of entanglement orchestrators in dynamically reconfiguring network topologies through local operations. We examine the major hardware challenge of quantum transduction, essential for interfacing heterogeneous quantum systems. Furthermore, we explore how interconnecting multiple Quantum Data Centres could enable large-scale quantum networks. We discuss the topological constraints of such a scaling and identify open challenges, including entanglement routing and synchronization. The carried analysis positions Quantum Data Centres as both a practical implementation platform and strategic framework for the future Quantum Internet.

quant-ph↗

arXiv2025

Perceptual Taxonomy: Evaluating and Guiding Hierarchical Scene Reasoning in Vision-Language Models

We propose Perceptual Taxonomy, a structured process of scene understanding that first recognizes objects and their spatial configurations, then infers task-relevant properties such as material, affordance, function, and physical attributes to support goal-directed reasoning. While this form of reasoning is fundamental to human cognition, current vision-language benchmarks lack comprehensive evaluation of this ability and instead focus on surface-level recognition or image-text alignment. To address this gap, we introduce Perceptual Taxonomy, a benchmark for physically grounded visual reasoning. We annotate 3173 objects with four property families covering 84 fine-grained attributes. Using these annotations, we construct a multiple-choice question benchmark with 5802 images across both synthetic and real domains. The benchmark contains 28033 template-based questions spanning four types (object description, spatial reasoning, property matching, and taxonomy reasoning), along with 50 expert-crafted questions designed to evaluate models across the full spectrum of perceptual taxonomy reasoning. Experimental results show that leading vision-language models perform well on recognition tasks but degrade by 10 to 20 percent on property-driven questions, especially those requiring multi-step reasoning over structured attributes. These findings highlight a persistent gap in structured visual understanding and the limitations of current models that rely heavily on pattern matching. We also show that providing in-context reasoning examples from simulated scenes improves performance on real-world and expert-curated questions, demonstrating the effectiveness of perceptual-taxonomy-guided prompting.

cs.CV↗

arXiv2025

Joint Computing Resource Allocation and Task Offloading in Vehicular Fog Computing Systems Under Asymmetric Information

Vehicular fog computing (VFC) has emerged as a promising paradigm, which leverages the idle computational resources of nearby fog vehicles (FVs) to complement the computing capabilities of conventional vehicular edge computing. However, utilizing VFC to meet the delay-sensitive and computation-intensive requirements of the FVs poses several challenges. First, the limited resources of road side units (RSUs) struggle to accommodate the growing and diverse demands of vehicles. This limitation is further exacerbated by the information asymmetry between the controller and FVs due to the reluctance of FVs to disclose private information and to share resources voluntarily. This information asymmetry hinders the efficient resource allocation and coordination. Second, the heterogeneity in task requirements and the varying capabilities of RSUs and FVs complicate efficient task offloading, thereby resulting in inefficient resource utilization and potential performance degradation. To address these challenges, we first present a hierarchical VFC architecture that incorporates the computing capabilities of both RSUs and FVs. Then, we formulate a delay minimization optimization problem (DMOP), which is an NP-hard mixed integer nonlinear programming problem. To solve the DMOP, we propose a joint computing resource allocation and task offloading approach (JCRATOA). Specifically, we propose a convex optimization-based method for RSU resource allocation and a contract theory-based incentive mechanism for FV resource allocation. Moreover, we present a two-sided matching method for task offloading by employing the matching game. Simulation results demonstrate that the proposed JCRATOA is able to achieve superior performances in task completion delay, task completion ratio, system throughput, and resource utilization fairness, while effectively meeting the satisfying constraints.

cs.NI↗

arXiv2025

Why and How Auxiliary Tasks Improve JEPA Representations

Joint-Embedding Predictive Architecture (JEPA) is increasingly used for visual representation learning and as a component in model-based RL, but its behavior remains poorly understood. We provide a theoretical characterization of a simple, practical JEPA variant that has an auxiliary regression head trained jointly with latent dynamics. We prove a No Unhealthy Representation Collapse theorem: in deterministic MDPs, if training drives both the latent-transition consistency loss and the auxiliary regression loss to zero, then any pair of non-equivalent observations, i.e., those that do not have the same transition dynamics or auxiliary value, must map to distinct latent representations. Thus, the auxiliary task anchors which distinctions the representation must preserve. Controlled ablations in a counting environment corroborate the theory and show that training the JEPA model jointly with the auxiliary head generates a richer representation than training them separately. Our work indicates a path to improve JEPA encoders: training them with an auxiliary function that, together with the transition dynamics, encodes the right equivalence relations.

cs.LG↗

arXiv2025

FlowDAS: A Stochastic Interpolant-based Framework for Data Assimilation

Data assimilation (DA) integrates observations with a dynamical model to estimate states of PDE-governed systems. Model-driven methods (e.g., Kalman, particle) presuppose full knowledge of the true dynamics, which is not always satisfied in practice, while purely data-driven solvers learn a deterministic mapping between observations and states and therefore miss the intrinsic stochasticity of real processes. Recently, score-based diffusion models learn a global diffusion prior and provide a good modeling of the stochastic dynamics, showing new potential for DA. However, their all-at-once generation rather than step-by-step transition limits their performance when dealing with highly complex stochastic processes and lacks physical interpretability. To tackle these drawbacks, we introduce FlowDAS, a generative DA framework that uses stochastic interpolants to directly learn state transition dynamics and achieve step-by-step transition to better model the real dynamics. We also improve the framework by combining the observation, better suiting the DA settings. Directly learning the underlying dynamics from collected data removes restrictive dynamical assumptions, and conditioning on observations at each interpolation step yields stable, measurement-consistent forecasts. Experiments on Lorenz-63, Navier-Stokes super-resolution/sparse-observation scenarios, and large-scale weather forecasting -- where dynamics are partly or wholly unknown -- show that FlowDAS surpasses model-driven methods, neural operators, and score-based baselines in accuracy and physical plausibility.

eess.SP↗

arXiv2025

Walk in Their Shoes to Navigate Your Own Path: Learning About Procrastination Through A Serious Game

Procrastination, the voluntary delay of tasks despite potential negative consequences, has prompted numerous time and task management interventions in the HCI community. While these interventions have shown promise in addressing specific behaviors, psychological theories suggest that learning about procrastination itself may help individuals develop their own coping strategies and build mental resilience. However, little research has explored how to support this learning process through HCI approaches. We present ProcrastiMate, a text adventure game where players learn about procrastination's causes and experiment with coping strategies by guiding in-game characters in managing relatable scenarios. Our field study with 27 participants revealed that ProcrastiMate facilitated learning and self-reflection while maintaining psychological distance, motivating players to integrate newly acquired knowledge in daily life. This paper contributes empirical insights on leveraging serious games to facilitate learning about procrastination and offers design implications for addressing psychological challenges through HCI approaches.

cs.HC↗

arXiv2025

CS-Eval: A Comprehensive Large Language Model Benchmark for CyberSecurity

Over the past year, there has been a notable rise in the use of large language models (LLMs) for academic research and industrial practices within the cybersecurity field. However, it remains a lack of comprehensive and publicly accessible benchmarks to evaluate the performance of LLMs on cybersecurity tasks. To address this gap, we introduce CS-Eval, a publicly accessible, comprehensive and bilingual LLM benchmark specifically designed for cybersecurity. CS-Eval synthesizes the research hotspots from academia and practical applications from industry, curating a diverse set of high-quality questions across 42 categories within cybersecurity, systematically organized into three cognitive levels: knowledge, ability, and application. Through an extensive evaluation of a wide range of LLMs using CS-Eval, we have uncovered valuable insights. For instance, while GPT-4 generally excels overall, other models may outperform it in certain specific subcategories. Additionally, by conducting evaluations over several months, we observed significant improvements in many LLMs' abilities to solve cybersecurity tasks. The benchmarks are now publicly available at https://github.com/CS-EVAL/CS-Eval.

cs.CR↗

arXiv2024

Inference of weak-form partial differential equations describing migration and proliferation mechanisms in wound healing experiments on cancer cells

Targeting signaling pathways that drive cancer cell migration or proliferation is a common therapeutic approach. A popular experimental technique, the scratch assay, measures the migration and proliferation-driven cell closure of a defect in a confluent cell monolayer. These assays do not measure dynamic effects. To improve analysis of scratch assays, we combine high-throughput scratch assays, video microscopy, and system identification to infer partial differential equation (PDE) models of cell migration and proliferation. We capture the evolution of cell density fields over time using live cell microscopy and automated image processing. We employ weak form-based system identification techniques for cell density dynamics modeled with first-order kinetics of advection-diffusion-reaction systems. We present a comparison of our methods to results obtained using traditional inference approaches on previously analyzed 1-dimensional scratch assay data. We demonstrate the application of this pipeline on high throughput 2-dimensional scratch assays and find that low levels of trametinib inhibit wound closure primarily by decreasing random cell migration by approximately 20%. Our integrated experimental and computational pipeline can be adapted for quantitatively inferring the effect of biological perturbations on cell migration and proliferation in various cell lines.

q-bio.CB↗

arXiv2024

Learning Diffusion Model from Noisy Measurement using Principled Expectation-Maximization Method

Diffusion models have demonstrated exceptional ability in modeling complex image distributions, making them versatile plug-and-play priors for solving imaging inverse problems. However, their reliance on large-scale clean datasets for training limits their applicability in scenarios where acquiring clean data is costly or impractical. Recent approaches have attempted to learn diffusion models directly from corrupted measurements, but these methods either lack theoretical convergence guarantees or are restricted to specific types of data corruption. In this paper, we propose a principled expectation-maximization (EM) framework that iteratively learns diffusion models from noisy data with arbitrary corruption types. Our framework employs a plug-and-play Monte Carlo method to accurately estimate clean images from noisy measurements, followed by training the diffusion model using the reconstructed images. This process alternates between estimation and training until convergence. We evaluate the performance of our method across various imaging tasks, including inpainting, denoising, and deblurring. Experimental results demonstrate that our approach enables the learning of high-fidelity diffusion priors from noisy data, significantly enhancing reconstruction quality in imaging inverse problems.

cs.CV↗

arXiv2024

Unfolding Videos Dynamics via Taylor Expansion

Taking inspiration from physical motion, we present a new self-supervised dynamics learning strategy for videos: Video Time-Differentiation for Instance Discrimination (ViDiDi). ViDiDi is a simple and data-efficient strategy, readily applicable to existing self-supervised video representation learning frameworks based on instance discrimination. At its core, ViDiDi observes different aspects of a video through various orders of temporal derivatives of its frame sequence. These derivatives, along with the original frames, support the Taylor series expansion of the underlying continuous dynamics at discrete times, where higher-order derivatives emphasize higher-order motion features. ViDiDi learns a single neural network that encodes a video and its temporal derivatives into consistent embeddings following a balanced alternating learning algorithm. By learning consistent representations for original frames and derivatives, the encoder is steered to emphasize motion features over static backgrounds and uncover the hidden dynamics in original frames. Hence, video representations are better separated by dynamic features. We integrate ViDiDi into existing instance discrimination frameworks (VICReg, BYOL, and SimCLR) for pretraining on UCF101 or Kinetics and test on standard benchmarks including video retrieval, action recognition, and action detection. The performances are enhanced by a significant margin without the need for large models or extensive datasets.

cs.CV↗

arXiv2024

Blind Inversion using Latent Diffusion Priors

Diffusion models have emerged as powerful tools for solving inverse problems due to their exceptional ability to model complex prior distributions. However, existing methods predominantly assume known forward operators (i.e., non-blind), limiting their applicability in practical settings where acquiring such operators is costly. Additionally, many current approaches rely on pixel-space diffusion models, leaving the potential of more powerful latent diffusion models (LDMs) underexplored. In this paper, we introduce LatentDEM, an innovative technique that addresses more challenging blind inverse problems using latent diffusion priors. At the core of our method is solving blind inverse problems within an iterative Expectation-Maximization (EM) framework: (1) the E-step recovers clean images from corrupted observations using LDM priors and a known forward model, and (2) the M-step estimates the forward operator based on the recovered images. Additionally, we propose two novel optimization techniques tailored for LDM priors and EM frameworks, yielding more accurate and efficient blind inversion results. As a general framework, LatentDEM supports both linear and non-linear inverse problems. Beyond common 2D image restoration tasks, it enables new capabilities in non-linear 3D inverse rendering problems. We validate LatentDEM's performance on representative 2D blind deblurring and 3D sparse-view reconstruction tasks, demonstrating its superior efficacy over prior arts.

cs.CV↗

arXiv2024

Robust semi-automatic vessel tracing in the human retinal image by an instance segmentation neural network

The morphology and hierarchy of the vascular systems are essential for perfusion in supporting metabolism. In human retina, one of the most energy-demanding organs, retinal circulation nourishes the entire inner retina by an intricate vasculature emerging and remerging at the optic nerve head (ONH). Thus, tracing the vascular branching from ONH through the vascular tree can illustrate vascular hierarchy and allow detailed morphological quantification, and yet remains a challenging task. Here, we presented a novel approach for a robust semi-automatic vessel tracing algorithm on human fundus images by an instance segmentation neural network (InSegNN). Distinct from semantic segmentation, InSegNN separates and labels different vascular trees individually and therefore enable tracing each tree throughout its branching. We have built-in three strategies to improve robustness and accuracy with temporal learning, spatial multi-sampling, and dynamic probability map. We achieved 83% specificity, and 50% improvement in Symmetric Best Dice (SBD) compared to literature, and outperformed baseline U-net. We have demonstrated tracing individual vessel trees from fundus images, and simultaneously retain the vessel hierarchy information. InSegNN paves a way for any subsequent morphological analysis of vascular morphology in relation to retinal diseases.

eess.IV↗

arXiv2023

Robust Scheduling in Cloud Environment Based on Heuristic Optimization Algorithm

Aiming at analyzing performance in cloud computing, some unpredictable perturbations which may lead to performance downgrade are essential factors that should not be neglected. To avoid performance downgrade in cloud computing system, it is reasonable to measure the impact of the perturbations, and further propose a robust scheduling strategy to maintain the performance of the system at an acceptable level. In this paper, we first describe the supply-demand relationship of service between cloud service providers and customers, in which the profit and waiting time are objectives they most concerned. Then, on the basis of introducing the lowest acceptable profit and longest acceptable waiting time for cloud service providers and customers respectively, we define a robustness metric method to declare that the number and speed of servers should be adequately configured in a feasible region, such that the performance of cloud computing system can stay at an acceptable level when it is subject to the perturbations. Subsequently, we discuss the robustness metric method in several cases, and propose heuristic optimization algorithm to enhance the robustness of the system as much as possible. At last, the performances of the proposed algorithm are validated by comparing with DE and PSO algorithm, the results show the superiority of the proposed algorithm.

cs.DC↗

arXiv2023

Radicalized by Thinness: Using a Model of Radicalization to Understand Pro-Anorexia Communities on Twitter

The rise in eating disorders, a condition with serious health complications, has been linked to the proliferation of idealized body images on social media platforms. However, the relationship between social media and eating disorders is more complex, with online platforms potentially enabling harmful behaviors by linking people to ``pro-ana'' communities that promote eating disorders. We conceptualize the growth of harmful pro-ana communities as a process of online radicalization. We show that a model of radicalization explains how individuals are driven to conversations about extreme behaviors, like fasting, to achieve the ``thin body'' goal, and how these conversations are validated by pro-ana communities. By facilitating social connections to like-minded others, a shared group identity and emotional support, social media platforms can trap individuals within toxic echo chambers that normalize extreme disordered eating behaviors and other forms of self-harm. Characterizing and quantifying the role of online communities in amplifying harmful conversations will support the development of strategies to mitigate their impact and promote better mental health.

cs.SI↗

arXiv2022

Understanding 3D Object Articulation in Internet Videos

We propose to investigate detecting and characterizing the 3D planar articulation of objects from ordinary videos. While seemingly easy for humans, this problem poses many challenges for computers. We propose to approach this problem by combining a top-down detection system that finds planes that can be articulated along with an optimization approach that solves for a 3D plane that can explain a sequence of observed articulations. We show that this system can be trained on a combination of videos and 3D scan datasets. When tested on a dataset of challenging Internet videos and the Charades dataset, our approach obtains strong performance. Project site: https://jasonqsy.github.io/Articulation3D

cs.CV↗

arXiv2021

Drone LAMS: A Drone-based Face Detection Dataset with Large Angles and Many Scenarios

This work presented a new drone-based face detection dataset Drone LAMS in order to solve issues of low performance of drone-based face detection in scenarios such as large angles which was a predominant working condition when a drone flies high. The proposed dataset captured images from 261 videos with over 43k annotations and 4.0k images with pitch or yaw angle in the range of -90° to 90°. Drone LAMS showed significant improvement over currently available drone-based face detection datasets in terms of detection performance, especially with large pitch and yaw angle. Detailed analysis of how key factors, such as duplication rate, annotation method, etc., impact dataset performance was also provided to facilitate further usage of a drone on face detection.

cs.CV↗