arXiv subjects

Zhihong Jin

Publications and source records attributed to Zhihong Jin.

3 recordsLinked to original sources

arXiv2026

ReflectWorld-MM: An Entity-Oriented Multimodal Memory System for Open-Ended Video Streams

Building assistants that can continually watch the world, remember what they see, and reason over their accumulated experience is a long-standing goal, and recently multimodal agents equipped with long-term memory over video streams have attracted increasing interest. Unfortunately, existing systems either keep their memory inside the model context or in a flat feature store, and organize it around frames rather than around the persistent entities a stream is really about, which confines them to bounded videos and weakens their ability to track who and what reappears over time. In this paper, we propose ReflectWorld-MM, an entity-oriented multimodal memory system for open-ended video streams. It consists of three parts. The first is a perception front-end that turns an audiovisual stream into entity-resolved observations under a bounded short-term memory. The second is a hierarchical long-term memory, grounded in human memory theory, that couples a multi-scale episodic memory, an evolving entity-centric semantic memory, and a procedural memory. The third is a complete realization, built for real-world operation, that ingests arbitrary streams and plugs into off-the-shelf assistants. Across six long-video and lifelong-memory benchmarks, ReflectWorld-MM achieves the best accuracy on all six, outperforming strong memory agents and a frontier model.

cs.CV↗

arXiv2025

Watch Closely: Mitigating Object Hallucinations in Large Vision-Language Models with Disentangled Decoding

Large Vision-Language Models (LVLMs) bridge the gap between visual and linguistic modalities, demonstrating strong potential across a variety of domains. However, despite significant progress, LVLMs still suffer from severe hallucination issues in object recognition tasks. These models often fail to accurately identify certain objects, leading to text generation that appears fluent but does not correspond to the visual content, which can have serious consequences in real-world applications. Recently, several methods have been proposed to alleviate LVLM hallucinations, but most focus solely on reducing hallucinations in the language modality. To mitigate hallucinations in both the language and visual modalities, we introduce Hallucination Disentangled Decoding (HDD) method that requires no training. HDD enhances the original image by segmenting it and selecting images that augment the original, while also utilizing a blank image to eliminate language prior hallucinations in both the original and segmented images. This design not only reduces the model's dependence on language priors but also enhances its visual performance. (Code: https://github.com/rickeyhhh/Hallucination-Disentangled-Decoding)

cs.CV↗

arXiv2024

Universal symplectic/orthogonal functions and general branching rules

In this paper, we first introduce a family of universal symplectic functions $sp_λ(\mathbf{x}^{\pm};\mathbf{z})$ that include symplectic Schur functions $sp_λ(\mathbf{x}^{\pm})$, odd symplectic characters $sp_λ(\mathbf{x}^{\pm};z)$, universal symplectic characters $sp_λ(\mathbf{z})$ and intermediate symplectic characters as subfamilies. We then realize the universal symplectic functions by vertex operators, which naturally lead to their skew versions, and show that $sp_λ(\mathbf{x}^{\pm};\mathbf{z})$ obey the general branching rules. This also gives the Gelfand-Tsetlin representations of odd symplectic characters and a transition formula between odd symplectic characters and symplectic Schur functions. Secondly we introduce a family of universal orthogonal functions $o_λ(\mathbf{x}^{\pm};\mathbf{z})$ and their skew versions in a similar manner, and we provide their vertex operator realizations and obtain transition formulas and the branching rule. The universal orthogonal functions $o_λ(\mathbf{x}^{\pm};\mathbf{z})$ generalize orthogonal Schur functions $o_λ(\mathbf{x}^{\pm})$, odd orthogonal Schur functions $so_λ(\mathbf{x}^{\pm})$, universal orthogonal characters $o_λ(\mathbf{z})$ as well as intermediate orthogonal characters. Thirdly, we give vertex operator realizations for the $CB$-interpolating Schur functions $s^{CB}_λ(x;β)$ introduced by Bisi and Zygouras (Adv. Math., 2022) and the $DB$-interpolating Schur functions $s^{DB}_λ(x;β)$ interpolating between characters of type $D$ and $B$. As an application, we show $s^{CB}_λ(x;β)$ are equal to the orthosymplectic Schur polynomials $spo_λ(x/β)$, thus give a short proof of the generalization of the Brent-Krattenthaler-Warnaar identity obtained by Kumari (arXiv:2401.01723).

math.CO↗