arXiv subjects

Liyun Zhu

Publications and source records attributed to Liyun Zhu.

7 recordsLinked to original sources

arXiv2026

FinMTM: A Multi-Turn Multimodal Benchmark for Financial Reasoning and Agent Evaluation

The financial domain poses substantial challenges for vision-language models (VLMs) due to specialized chart formats and knowledge-intensive reasoning requirements. However, existing financial benchmarks are largely single-turn and rely on a narrow set of question formats, limiting comprehensive evaluation in realistic application scenarios. To address this gap, we propose FinMTM, a multi-turn multimodal benchmark that expands diversity along both data and task dimensions. On the data side, we curate and annotate 11{,}133 bilingual (Chinese and English) financial QA pairs grounded in financial visuals, including candlestick charts, statistical plots, and report figures. On the task side, FinMTM covers single- and multiple-choice questions, multi-turn open-ended dialogues, and agent-based tasks. We further design task-specific evaluation protocols, including a set-overlap scoring rule for multiple-choice questions, a weighted combination of turn-level and session-level scores for multi-turn dialogues, and a composite metric that integrates planning quality with final outcomes for agent tasks. Extensive experimental evaluation of 22 VLMs reveal their limitations in fine-grained visual perception, long-context reasoning, and complex agent workflows.

cs.CV↗

arXiv2025

VAU-R1: Advancing Video Anomaly Understanding via Reinforcement Fine-Tuning

Video Anomaly Understanding (VAU) is essential for applications such as smart cities, security surveillance, and disaster alert systems, yet remains challenging due to its demand for fine-grained spatio-temporal perception and robust reasoning under ambiguity. Despite advances in anomaly detection, existing methods often lack interpretability and struggle to capture the causal and contextual aspects of abnormal events. This limitation is further compounded by the absence of comprehensive benchmarks for evaluating reasoning ability in anomaly scenarios. To address both challenges, we introduce VAU-R1, a data-efficient framework built upon Multimodal Large Language Models (MLLMs), which enhances anomaly reasoning through Reinforcement Fine-Tuning (RFT). Besides, we propose VAU-Bench, the first Chain-of-Thought benchmark tailored for video anomaly reasoning, featuring multiple-choice QA, detailed rationales, temporal annotations, and descriptive captions. Empirical results show that VAU-R1 significantly improves question answering accuracy, temporal grounding, and reasoning coherence across diverse contexts. Together, our method and benchmark establish a strong foundation for interpretable and reasoning-aware video anomaly understanding. Our code is available at https://github.com/GVCLab/VAU-R1.

cs.CV↗

arXiv2025

Enhancing Monocular 3D Scene Completion with Diffusion Model

3D scene reconstruction is essential for applications in virtual reality, robotics, and autonomous driving, enabling machines to understand and interact with complex environments. Traditional 3D Gaussian Splatting techniques rely on images captured from multiple viewpoints to achieve optimal performance, but this dependence limits their use in scenarios where only a single image is available. In this work, we introduce FlashDreamer, a novel approach for reconstructing a complete 3D scene from a single image, significantly reducing the need for multi-view inputs. Our approach leverages a pre-trained vision-language model to generate descriptive prompts for the scene, guiding a diffusion model to produce images from various perspectives, which are then fused to form a cohesive 3D reconstruction. Extensive experiments show that our method effectively and robustly expands single-image inputs into a comprehensive 3D scene, extending monocular 3D reconstruction capabilities without further training. Our code is available https://github.com/CharlieSong1999/FlashDreamer/tree/main.

cs.GR↗

arXiv2025

Learnable Expansion of Graph Operators for Multi-Modal Feature Fusion

In computer vision tasks, features often come from diverse representations, domains (e.g., indoor and outdoor), and modalities (e.g., text, images, and videos). Effectively fusing these features is essential for robust performance, especially with the availability of powerful pre-trained models like vision-language models. However, common fusion methods, such as concatenation, element-wise operations, and non-linear techniques, often fail to capture structural relationships, deep feature interactions, and suffer from inefficiency or misalignment of features across domains or modalities. In this paper, we shift from high-dimensional feature space to a lower-dimensional, interpretable graph space by constructing relationship graphs that encode feature relationships at different levels, e.g., clip, frame, patch, token, etc. To capture deeper interactions, we expand graphs through iterative graph relationship updates and introduce a learnable graph fusion operator to integrate these expanded relationships for more effective fusion. Our approach is relationship-centric, operates in a homogeneous space, and is mathematically principled, resembling element-wise relationship score aggregation via multilinear polynomials. We demonstrate the effectiveness of our graph-based fusion method on video anomaly detection, showing strong performance across multi-representational, multi-modal, and multi-domain feature fusion tasks.

cs.CV↗

arXiv2024

Advancing Video Anomaly Detection: A Concise Review and a New Dataset

Video Anomaly Detection (VAD) finds widespread applications in security surveillance, traffic monitoring, industrial monitoring, and healthcare. Despite extensive research efforts, there remains a lack of concise reviews that provide insightful guidance for researchers. Such reviews would serve as quick references to grasp current challenges, research trends, and future directions. In this paper, we present such a review, examining models and datasets from various perspectives. We emphasize the critical relationship between model and dataset, where the quality and diversity of datasets profoundly influence model performance, and dataset development adapts to the evolving needs of emerging approaches. Our review identifies practical issues, including the absence of comprehensive datasets with diverse scenarios. To address this, we introduce a new dataset, Multi-Scenario Anomaly Detection (MSAD), comprising 14 distinct scenarios captured from various camera views. Our dataset has diverse motion patterns and challenging variations, such as different lighting and weather conditions, providing a robust foundation for training superior models. We conduct an in-depth analysis of recent representative models using MSAD and highlight its potential in addressing the challenges of detecting anomalies across diverse and evolving surveillance scenarios. [Project website: https://msad-dataset.github.io/]

cs.CV↗

arXiv2020

Calculating the free energy difference by applying the Jarzynski equality to a virtual integrable system

The Jarzynski equality (JE) provides a nonequilibrium method to measure and calculate the free energy difference (FED). Note that if two systems share the same Hamiltonian at two equilibrium states, respectively, they share the same FED between these two equilibrium states as well. Therefore the calculation of the FED of a system may be facilitated by considering instead another virtual system designed to this end. Taking advantage of this flexibility and the JE, we show that by introducing an integrable virtual system, the evolution problem involved in the JE can be solved. As a consequence, FED is expressed in the form of an equilibrium equality, in contrast with the nonequilibrium JE it is based on. Numerically, this result allows FED to be computed by sampling the canonical ensemble directly and the computational cost can be significantly reduced. The effectiveness and efficiency of this scheme are illustrated with numerical studies of several representative model systems.

cond-mat.stat-mech↗

arXiv2016

Sparse aperture differential piston measurements using the pyramid wave-front sensor

In this paper we report on the laboratory experiment we settled in the Shanghai Astronomical Observatory (SHAO) to investigate the pyramid wavefront sensor (WFS) ability to measure the differential piston on a sparse aperture. The ultimate goal is to verify the ability of the pyramid WFS work in closed loop to perform the phasing of the primary mirrors of a sparse Fizeau imaging telescope. In the experiment we installed on the optical bench we performed various test checking the ability to flat the wave-front using a deformable mirror and to measure the signal of the differential piston on a two pupils setup. These steps represent the background from which we start to perform full closed loop operation on multiple apertures. These steps were also useful to characterize the achromatic double pyramids (double prisms) manufactured in the SHAO optical workshop.

astro-ph.IM↗