SearcharxivSearch

arXiv subjects

Handong Li

Publications and source records attributed to Handong Li.

16 recordsLinked to original sources

TimeThink: Reasoning with Time for Video LLMs

Video reasoning requires models to identify and verify temporally localized evidence within long video sequences. Recent Video Large Language Models (Video-LLMs) have shown promising reasoning abilities when aligned with reinforcement learning, yet existing approaches typically rely on outcome-based rewards that supervise only the final prediction. Such supervision provides limited guidance on how models should discover the relevant temporal evidence during intermediate reasoning. In this work, we propose TimeThink, a reinforcement learning framework that explicitly guides temporal evidence discovery in Video-LLMs. Our key idea is to treat temporal clue steps as the fundamental optimization primitive of video reasoning, where each reasoning step references a candidate time interval in the video. We introduce a step-wise temporal process reward that provides localized credit assignment for these clues and a joint process--outcome optimization objective that balances reasoning fidelity with task correctness. To enable scalable training, we construct TimeThink-RFT-20K, a dataset with automatically derived temporal evidence segments. Extensive experiments across video reasoning, temporal grounding, and general video understanding benchmarks show that TimeThink consistently improves both temporal localization and reasoning performance, achieving state-of-the-art results among open-source video RL models.

cs.CV

AdaSpark: Adaptive Sparsity for Efficient Long-Video Understanding

Processing long-form videos with Video Large Language Models (Video-LLMs) is computationally prohibitive. Current efficiency methods often compromise fine-grained perception through irreversible information disposal or inhibit long-range temporal modeling via rigid, predefined sparse patterns. This paper introduces AdaSpark, an adaptive sparsity framework designed to address these limitations. AdaSpark first partitions video inputs into 3D spatio-temporal cubes. It then employs two co-designed, context-aware components: (1) Adaptive Cube-Selective Attention (AdaS-Attn), which adaptively selects a subset of relevant video cubes to attend for each query token, and (2) Adaptive Token-Selective FFN (AdaS-FFN), which selectively processes only the most salient tokens within each cube. An entropy-based (Top-p) selection mechanism adaptively allocates computational resources based on input complexity. Experiments demonstrate that AdaSpark significantly reduces computational load by up to 57% FLOPs while maintaining comparable performance to dense models and preserving fine-grained, long-range dependencies, as validated on challenging hour-scale video benchmarks.

cs.CV

Thinking in Streaming Video

Real-time understanding of continuous video streams is essential for interactive assistants and multimodal agents operating in dynamic environments. However, most existing video reasoning approaches follow a batch paradigm that defers reasoning until the full video context is observed, resulting in high latency and growing computational cost that are incompatible with streaming scenarios. In this paper, we introduce ThinkStream, a framework for streaming video reasoning based on a Watch--Think--Speak paradigm that enables models to incrementally update their understanding as new video observations arrive. At each step, the model performs a short reasoning update and decides whether sufficient evidence has accumulated to produce a response. To support long-horizon streaming, we propose Reasoning-Compressed Streaming Memory (RCSM), which treats intermediate reasoning traces as compact semantic memory that replaces outdated visual tokens while preserving essential context. We further train the model using a Streaming Reinforcement Learning with Verifiable Rewards scheme that aligns incremental reasoning and response timing with the requirements of streaming interaction. Experiments on multiple streaming video benchmarks show that ThinkStream significantly outperforms existing online video models while maintaining low latency and memory usage. Code, models and data will be released at https://github.com/johncaged/ThinkStream

cs.CV

Breaking the Encoder Barrier for Seamless Video-Language Understanding

Most Video-Large Language Models (Video-LLMs) adopt an encoder-decoder framework, where a vision encoder extracts frame-wise features for processing by a language model. However, this approach incurs high computational costs, introduces resolution biases, and struggles to capture fine-grained multimodal interactions. To overcome these limitations, we propose ELVA, an encoder-free Video-LLM that directly models nuanced video-language interactions without relying on a vision encoder. ELVA employs token merging to construct a bottom-up hierarchical representation and incorporates a video guidance supervisor for direct spatiotemporal representation learning. Additionally, a hybrid-resolution mechanism strategically integrates high- and low-resolution frames as inputs to achieve an optimal balance between performance and efficiency. With only 7M publicly available video-text pairs, ELVA achieves performance on par with encoder-based Video-LLMs while reducing FLOPs by up to 95\% and inference latency by 92\%, offering a scalable and efficient solution for real-time video understanding.

cs.CV

Explore the Limits of Omni-modal Pretraining at Scale

We propose to build omni-modal intelligence, which is capable of understanding any modality and learning universal representations. In specific, we propose a scalable pretraining paradigm, named Multimodal Context (MiCo), which can scale up the numbers of modalities and amount of data, together with the model parameters, in the pretraining process. With MiCo, the pretrained models show significant emergent abilities in multimodal learning, which are evaluated on the following tasks: i) single-modality perception benchmarks of 10 different modalities, ii) 25 cross-modality understanding tasks of retrieval, question-answering, captioning, and iii) 18 multimodal large language model benchmarks. Our models establish 37 new records for state-of-the-art performance. We hope that our research could contribute to the development of omni-modal intelligence. Code and Models are at https://github.com/invictus717/MiCo

cs.CV

Unveiling Early Warning Signals of Systemic Risks in Banks: A Recurrence Network-Based Approach

Bank crisis is challenging to define but can be manifested through bank contagion. This study presents a comprehensive framework grounded in nonlinear time series analysis to identify potential early warning signals (EWS) for impending phase transitions in bank systems, with the goal of anticipating severe bank crisis. In contrast to traditional analyses of exposure networks using low-frequency data, we argue that studying the dynamic relationships among bank stocks using high-frequency data offers a more insightful perspective on changes in the banking system. We construct multiple recurrence networks (MRNs) based on multidimensional returns of listed banks' stocks in China, aiming to monitor the nonlinear dynamics of the system through the corresponding indicators and topological structures. Empirical findings indicate that key indicators of MRNs, specifically the average mutual information, provide valuable insights into periods of extreme volatility of bank system. This paper contributes to the ongoing discourse on early warning signals for bank instability, highlighting the applicability of predicting systemic risks in the context of banking networks.

q-fin.RM

COSA: Concatenated Sample Pretrained Vision-Language Foundation Model

Due to the limited scale and quality of video-text training corpus, most vision-language foundation models employ image-text datasets for pretraining and primarily focus on modeling visually semantic representations while disregarding temporal semantic representations and correlations. To address this issue, we propose COSA, a COncatenated SAmple pretrained vision-language foundation model. COSA jointly models visual contents and event-level temporal cues using only image-text corpora. We achieve this by sequentially concatenating multiple image-text pairs as inputs for pretraining. This transformation effectively converts existing image-text corpora into a pseudo long-form video-paragraph corpus, enabling richer scene transformations and explicit event-description correspondence. Extensive experiments demonstrate that COSA consistently improves performance across a broad range of downstream tasks, including long-form/short-form video-text tasks and image-text tasks such as retrieval, captioning, and question answering. Notably, COSA achieves state-of-the-art results on various competitive benchmarks. Code and model are released at https://github.com/TXH-mercury/COSA.

cs.CV

VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Dataset

Vision and text have been fully explored in contemporary video-text foundational models, while other modalities such as audio and subtitles in videos have not received sufficient attention. In this paper, we resort to establish connections between multi-modality video tracks, including Vision, Audio, and Subtitle, and Text by exploring an automatically generated large-scale omni-modality video caption dataset called VAST-27M. Specifically, we first collect 27 million open-domain video clips and separately train a vision and an audio captioner to generate vision and audio captions. Then, we employ an off-the-shelf Large Language Model (LLM) to integrate the generated captions, together with subtitles and instructional prompts into omni-modality captions. Based on the proposed VAST-27M dataset, we train an omni-modality video-text foundational model named VAST, which can perceive and process vision, audio, and subtitle modalities from video, and better support various tasks including vision-text, audio-text, and multi-modal video-text tasks (retrieval, captioning and QA). Extensive experiments have been conducted to demonstrate the effectiveness of our proposed VAST-27M corpus and VAST foundation model. VAST achieves 22 new state-of-the-art results on various cross-modality benchmarks. Code, model and dataset will be released at https://github.com/TXH-mercury/VAST.

cs.CV

Enhancing Vision-Language Pre-Training with Jointly Learned Questioner and Dense Captioner

Large pre-trained multimodal models have demonstrated significant success in a range of downstream tasks, including image captioning, image-text retrieval, visual question answering (VQA), etc. However, many of these methods rely on image-text pairs collected from the web as pre-training data and unfortunately overlook the need for fine-grained feature alignment between vision and language modalities, which requires detailed understanding of images and language expressions. While integrating VQA and dense captioning (DC) into pre-training can address this issue, acquiring image-question-answer as well as image-location-caption triplets is challenging and time-consuming. Additionally, publicly available datasets for VQA and dense captioning are typically limited in scale due to manual data collection and labeling efforts. In this paper, we propose a novel method called Joint QA and DC GEneration (JADE), which utilizes a pre-trained multimodal model and easily-crawled image-text pairs to automatically generate and filter large-scale VQA and dense captioning datasets. We apply this method to the Conceptual Caption (CC3M) dataset to generate a new dataset called CC3M-QA-DC. Experiments show that when used for pre-training in a multi-task manner, CC3M-QA-DC can improve the performance with various backbones on various downstream tasks. Furthermore, our generated CC3M-QA-DC can be combined with larger image-text datasets (e.g., CC15M) and achieve competitive results compared with models using much more data. Code and dataset are available at https://github.com/johncaged/OPT_Questioner.

cs.CV

Value-at-Risk forecasting model based on normal inverse Gaussian distribution driven by dynamic conditional score

Under the framework of dynamic conditional score, we propose a parametric forecasting model for Value-at-Risk based on the normal inverse Gaussian distribution (Hereinafter NIG-DCS-VaR), which creatively incorporates intraday information into daily VaR forecast. NIG specifies an appropriate distribution to return and the semi-additivity of the NIG parameters makes it feasible to improve the estimation of daily return in light of intraday return, and thus the VaR can be explicitly obtained by calculating the quantile of the re-estimated distribution of daily return. We conducted an empirical analysis using two main indexes of the Chinese stock market, and a variety of backtesting approaches as well as the model confidence set approach prove that the VaR forecasts of NIG-DCS model generally gain an advantage over those of realized GARCH (RGARCH) models. Especially when the risk level is relatively high, NIG-DCS-VaR beats RGARCH-VaR in terms of coverage ability and independence.

q-fin.RM

A Method for Predicting VaR by Aggregating Generalized Distributions Driven by the Dynamic Conditional Score

Constructing a more effective value at risk (VaR) prediction model has long been a goal in financial risk management. In this paper, we propose a novel parametric approach and provide a standard paradigm to demonstrate the modeling. We establish a dynamic conditional score (DCS) model based on high-frequency data and a generalized distribution (GD), namely, the GD-DCS model, to improve the forecasts of daily VaR. The model assumes that intraday returns at different moments are independent of each other and obey the same kind of GD, whose dynamic parameters are driven by DCS. By predicting the motion law of the time-varying parameters, the conditional distribution of intraday returns is determined; then, the bootstrap method is used to simulate daily returns. An empirical analysis using data from the Chinese stock market shows that Weibull-Pareto -DCS model incorporating high-frequency data is superior to traditional benchmark models, such as RGARCH, in the prediction of VaR at high risk levels, which proves that this approach contributes to the improvement of risk measurement tools.

q-fin.RM

Research on Portfolio Liquidation Strategy under Discrete Times

This paper presents an optimal strategy for portfolio liquidation under discrete time conditions. We assume that N risky assets held will be liquidated according to the same time interval and order quantity, and the basic price processes of assets are generated by an N-dimensional independent standard Brownian motion. The permanent impact generated by an asset in the portfolio during the liquidation will affect all assets, and the temporary impact generated by one asset will only affect itself. On this basis, we establish a liquidation cost model based on the VaR measurement and obtain an optimal liquidation time under discrete-time conditions. The optimal solution shows that the liquidation time is only related to the temporary impact rather than the permanent impact. In the simulation analysis, we give the relationship between volatility parameters, temporary price impact and the optimal liquidation strategy.

q-fin.TR

Tunable Semimetallic State in Compressive-strained SrIrO3 Films Revealed by Transport Behaviors

Orthorhombic SrIrO3 is a typical spin-orbit-coupling correlated metal that shows diversified physical properties under the external stimuli. Here nonlinear Hall effect and weakly temperature-dependent resistance are observed in a SrIrO3 film epitaxially grown on SrTiO3 substrate. It infers that orthorhombic SrIrO3 is a semimetal oxide. However, linear Hall effect and insensitive-temperature-dependent resistance are observed in SrIrO3 films grown on (La,Sr)(Al,Ta)O3 (LSAT) substrates, suggesting a tunable semimetallic state due to band structure change in SrIrO3 films under different compressive strain. The mechanism of this evolution is explored in detail through strain-state analysis by reciprocal space mapping and electron diffraction, carrier density and mobility calculations, as well as electronic band structure evolution under compressive strain (predicted by tight-binding approximation). It might suggest that the strain-induced band shift leads to the semimetallic tuning in the SrIrO3 film grown on from SrTiO3 to LSAT substrates. Our findings illustrate the tunability of SrIrO3 properties and pave the way to induce novel physical states in SrIrO3 such as the proposed topological insulator state in heterostructures.

cond-mat.mtrl-sci

Growth and band alignment of Bi2Se3 topological insulator on H-terminated Si(111) van der Waals surface

The van der Waals epitaxy of single crystalline Bi2Se3 film was achieved on hydrogen passivated Si(111) (H:Si) substrate by physical vapor deposition. Valence band structures of Bi2Se3/H:Si heterojunction were investigated by X-ray Photoemission Spectroscopy and Ultraviolet Photoemission Spectroscopy. The measured Schottky barrier height at the Bi2Se3-H:Si interface was 0.31 eV. The findings pave the way for economically preparing heterojunctions and multilayers of layered compound families of topological insulators.

cond-mat.mtrl-sci

Interplay between topological insulators and superconductors

Topological insulators are insulating in the bulk but possess metallic surface states protected by time-reversal symmetry. Here, we report a detailed electronic transport study in high quality Bi2Se3 topological insulator thin films contacted by superconducting (In, Al and W) electrodes. The resistance of the film shows an abrupt and significant upturn when the electrodes become superconducting. In turn, the Bi2Se3 film strongly weakens the superconductivity of the electrodes, significantly reducing both their transition temperatures and critical fields. A possible interpretation of these results is that the superconducting electrodes are accessing the surface states and the experimental results are the consequence of the interplay between the Cooper pairs of the electrodes and the spin polarized current of the surface states in Bi2Se3.

cond-mat.mtrl-sci

Current induced anisotropic magnetoresistance in topological insulator films

Topological insulators are insulating in the bulk but possess spin-momentum locked metallic surface states protected by time-reversal symmetry. The existence of these surface states has been confirmed by angle-resolved photoemission spectroscopy (ARPES) and scanning tunneling microscopy (STM). Detecting these surface states by transport measurement, which might at first appear to be the most direct avenue, was shown to be much more challenging than expected. Here, we report a detailed electronic transport study in high quality Bi2Se3 topological insulator thin films. Measurements under in-plane magnetic field, along and perpendicular to the bias current show opposite magnetoresistance. We argue that this contrasting behavior is related to the locking of the spin and current direction providing evidence for helical spin structure of the topological surface states.

cond-mat.mtrl-sci