arXiv subjects

Tong Jin

Publications and source records attributed to Tong Jin.

15 recordsLinked to original sources

arXiv2026

Are All Tokens Necessary for Visual Place Recognition? An Empirical Study of Token Reduction for Efficient Inference

Recent visual place recognition (VPR) methods based on vision transformers, particularly foundation models, have achieved remarkable recognition performance. However, these models process all visual tokens throughout the entire network, resulting in substantial computational overhead, which hinders their deployment in real-time and resource-constrained scenarios. A natural question thus arises: are all visual tokens necessary for VPR? To answer this question, we present the first systematic benchmark of token reduction for efficient visual place recognition. Our benchmark comprehensively evaluates representative token pruning, token merging, and hybrid pruning-merging methods across multiple state-of-the-art VPR models and diverse benchmark datasets covering urban, suburban, and natural environments. We further investigate token reduction from multiple perspectives, including recognition performance under different reduction configurations, computational complexity, inference speed, qualitative visualization, and deployment efficiency on edge devices. Through extensive experiments and in-depth analysis, our benchmark reveals multiple important characteristics of token reduction in VPR and provides several practical insights into the trade-offs between accuracy and inference efficiency. For example, token reduction can reduce computational cost by up to 29\% and improve throughput by up to 44\%, while incurring less than 1\% degradation in recognition accuracy. Overall, this work establishes a comprehensive foundation for future research on token-efficient VPR and efficient visual retrieval systems. Our codes and models will be available at https://github.com/Tong-Jin01/TokenReduction4VPR

cs.CV↗

arXiv2026

Selectivity Drives Efficiency: Dataset Pruning for Visual Place Recognition

Recent visual place recognition (VPR) studies have increasingly relied on large-scale datasets to train more robust and discriminative models. Although this trend significantly improves recognition performance, it also introduces substantial storage and training costs, especially when new architectures or training strategies need to be repeatedly developed and evaluated. Dataset pruning (DP) provides a promising way to improve data efficiency by retaining only informative training data. However, conventional DP methods mainly follow the sample-wise classification paradigm, which overlooks the relation-dependent training nature of VPR, where supervision is typically formed by image pairs rather than independent images. To address this issue, we propose a place-wise dataset pruning framework tailored for VPR. Instead of pruning individual images, our method treats each place as the basic pruning unit and introduces two complementary novel metrics, i.e., intra-place diversity (IPD) and inter-place similarity (IPS), to evaluate the training value of each place. By jointly considering these two metrics, our method ranks all places and constructs a compact yet informative coreset, thereby allowing the pruned dataset to still support the training of robust and discriminative VPR models. Extensive experiments demonstrate that our method consistently outperforms state-of-the-art DP baselines under different pruning ratios while reducing selection and training costs. Moreover, by pruning a merged dataset roughly 3.5$\times$ the size of GSV-Cities to a comparable scale, our coreset maintains highly competitive performance, achieving 94.5\% R@1 on MSLS-val and 97.0\% R@1 on Nordland with only NetVLAD. Codes will be made publicly available.

cs.CV↗

arXiv2026

A Self-Evolving Agent for Longitudinal Personal Health Management

Personal health management unfolds over repeated encounters, yet most health AI systems treat each request in isolation. We developed HealthClaw, an open-source agent architecture that updates support as a person's routines, preferences, measurements and risks change. It separates shared safety rules and medical knowledge from private longitudinal memory containing profile facts, reusable procedures and episodic traces. After each episode, induction determines what should update the profile, revise a procedure, remain episodic or be excluded. We evaluated HealthClaw with a synthetic year-long benchmark and nine 200-case biomedical tasks. Across 900 longitudinal support probes, answer accuracy increased from 0.2% with current-query prompting to 45.7% with HealthClaw, while prompt-side context exposure was 71.7% lower than with full-history prompting. In 100 privacy probes, HealthClaw produced higher privacy-aware answer quality and fewer unsafe disclosures than both baselines. Across the biomedical tasks, the mean absolute gain in the task-specific primary metric was 27.0 percentage points, and seven gains remained significant after false-discovery-rate correction. These offline benchmarks support governed, self-evolving memory for longitudinal personal health agents, although clinical effectiveness requires prospective evaluation. HealthClaw is publicly available at https://github.com/HC-Guo/HealthClaw.

cs.AI↗

arXiv2026

FSVideo: Fast Speed Video Diffusion Model in a Highly-Compressed Latent Space

We introduce FSVideo, a fast speed transformer-based image-to-video (I2V) diffusion framework. We build our framework on the following key components: 1.) a new video autoencoder with highly-compressed latent space ($64\times64\times4$ spatial-temporal downsampling ratio), achieving competitive reconstruction quality; 2.) a diffusion transformer (DIT) architecture with a new layer memory design to enhance inter-layer information flow and context reuse within DIT, and 3.) a multi-resolution generation strategy via a few-step DIT upsampler to increase video fidelity. Our final model, which contains a 14B DIT base model and a 14B DIT upsampler, achieves competitive performance against other popular open-source models, while being an order of magnitude faster. We discuss our model design as well as training strategies in this report.

cs.CV↗

arXiv2026

A modern perspective on Tutte's homotopy theorem

We begin with a review of Tutte's homotopy theory, which concerns the structure of certain graph associated to a matroid (together with some extra data). Concretely, Tutte's path theorem asserts that this graph is connected, and his homotopy theorem asserts that every cycle in the graph is a composition of ''elementary cycles'', which come in four different flavors. We present an extended version of the homotopy theorem, in which we give a more refined classification of the different types of elementary cycles. We explain in detail how the path theorem allows one to prove that the foundation of a matroid (in the sense of Baker--Lorscheid) is generated by universal cross-ratios, and how the extended homotopy theorem allows one to classify all algebraic relations between universal cross-ratios. The resulting ''fundamental presentation'' of the foundation was previously established in [Baker--Lorscheid], but the argument here is more self-contained. We then recall a few applications of the fundamental presentation to the representation theory of matroids. Finally, in the most novel but also the most speculative part of the paper, we discuss what a ''higher Tutte homotopy theorem'' might look like, and we present some preliminary computations along these lines.

math.CO↗

arXiv2025

Towards Implicit Aggregation: Robust Image Representation for Place Recognition in the Transformer Era

Visual place recognition (VPR) is typically regarded as a specific image retrieval task, whose core lies in representing images as global descriptors. Over the past decade, dominant VPR methods (e.g., NetVLAD) have followed a paradigm that first extracts the patch features/tokens of the input image using a backbone, and then aggregates these patch features into a global descriptor via an aggregator. This backbone-plus-aggregator paradigm has achieved overwhelming dominance in the CNN era and remains widely used in transformer-based models. In this paper, however, we argue that a dedicated aggregator is not necessary in the transformer era, that is, we can obtain robust global descriptors only with the backbone. Specifically, we introduce some learnable aggregation tokens, which are prepended to the patch tokens before a particular transformer block. All these tokens will be jointly processed and interact globally via the intrinsic self-attention mechanism, implicitly aggregating useful information within the patch tokens to the aggregation tokens. Finally, we only take these aggregation tokens from the last output tokens and concatenate them as the global representation. Although implicit aggregation can provide robust global descriptors in an extremely simple manner, where and how to insert additional tokens, as well as the initialization of tokens, remains an open issue worthy of further exploration. To this end, we also propose the optimal token insertion strategy and token initialization method derived from empirical studies. Experimental results show that our method outperforms state-of-the-art methods on several VPR datasets with higher efficiency and ranks 1st on the MSLS challenge leaderboard. The code is available at https://github.com/lu-feng/image.

cs.CV↗

arXiv2025

Representation-Level Counterfactual Calibration for Debiased Zero-Shot Recognition

Object-context shortcuts remain a persistent challenge in vision-language models, undermining zero-shot reliability when test-time scenes differ from familiar training co-occurrences. We recast this issue as a causal inference problem and ask: Would the prediction remain if the object appeared in a different environment? To answer this at inference time, we estimate object and background expectations within CLIP's representation space, and synthesize counterfactual embeddings by recombining object features with diverse alternative contexts sampled from external datasets, batch neighbors, or text-derived descriptions. By estimating the Total Direct Effect and simulating intervention, we further subtract background-only activation, preserving beneficial object-context interactions while mitigating hallucinated scores. Without retraining or prompt design, our method substantially improves both worst-group and average accuracy on context-sensitive benchmarks, establishing a new zero-shot state of the art. Beyond performance, our framework provides a lightweight representation-level counterfactual approach, offering a practical causal avenue for debiased and reliable multimodal reasoning.

cs.CV↗

arXiv2025

Direction-aware multi-scale gradient loss for infrared and visible image fusion

Infrared and visible image fusion aims to integrate complementary information from co-registered source images to produce a single, informative result. Most learning-based approaches train with a combination of structural similarity loss, intensity reconstruction loss, and a gradient-magnitude term. However, collapsing gradients to their magnitude removes directional information, yielding ambiguous supervision and suboptimal edge fidelity. We introduce a direction-aware, multi-scale gradient loss that supervises horizontal and vertical components separately and preserves their sign across scales. This axis-wise, sign-preserving objective provides clear directional guidance at both fine and coarse resolutions, promoting sharper, better-aligned edges and richer texture preservation without changing model architectures or training protocols. Experiments on open-source model and multiple public benchmarks demonstrate effectiveness of our approach.

cs.CV↗

arXiv2025

Vidi: Large Multimodal Models for Video Understanding and Editing

Humans naturally share information with those they are connected to, and video has become one of the dominant mediums for communication and expression on the Internet. To support the creation of high-quality large-scale video content, a modern pipeline requires a comprehensive understanding of both the raw input materials (e.g., the unedited footage captured by cameras) and the editing components (e.g., visual effects). In video editing scenarios, models must process multiple modalities (e.g., vision, audio, text) with strong background knowledge and handle flexible input lengths (e.g., hour-long raw videos), which poses significant challenges for traditional models. In this report, we introduce Vidi, a family of Large Multimodal Models (LMMs) for a wide range of video understand editing scenarios. The first release focuses on temporal retrieval, i.e., identifying the time ranges within the input videos corresponding to a given text query, which plays a critical role in intelligent editing. The model is capable of processing hour-long videos with strong temporal understanding capability, e.g., retrieve time ranges for certain queries. To support a comprehensive evaluation in real-world scenarios, we also present the VUE-TR benchmark, which introduces five key advancements. 1) Video duration: significantly longer than videos of existing temporal retrival datasets, 2) Audio support: includes audio-based queries, 3) Query format: diverse query lengths/formats, 4) Annotation quality: ground-truth time ranges are manually annotated. 5) Evaluation metric: a refined IoU metric to support evaluation over multiple time ranges. Remarkably, Vidi significantly outperforms leading proprietary models, e.g., GPT-4o and Gemini, on the temporal retrieval task, indicating its superiority in video editing scenarios.

cs.CV↗

arXiv2025

SelaVPR++: Towards Seamless Adaptation of Foundation Models for Efficient Place Recognition

Recent studies show that the visual place recognition (VPR) method using pre-trained visual foundation models can achieve promising performance. In our previous work, we propose a novel method to realize seamless adaptation of foundation models to VPR (SelaVPR). This method can produce both global and local features that focus on discriminative landmarks to recognize places for two-stage VPR by a parameter-efficient adaptation approach. Although SelaVPR has achieved competitive results, we argue that the previous adaptation is inefficient in training time and GPU memory usage, and the re-ranking paradigm is also costly in retrieval latency and storage usage. In pursuit of higher efficiency and better performance, we propose an extension of the SelaVPR, called SelaVPR++. Concretely, we first design a parameter-, time-, and memory-efficient adaptation method that uses lightweight multi-scale convolution (MultiConv) adapters to refine intermediate features from the frozen foundation backbone. This adaptation method does not back-propagate gradients through the backbone during training, and the MultiConv adapter facilitates feature interactions along the spatial axes and introduces proper local priors, thus achieving higher efficiency and better performance. Moreover, we propose an innovative re-ranking paradigm for more efficient VPR. Instead of relying on local features for re-ranking, which incurs huge overhead in latency and storage, we employ compact binary features for initial retrieval and robust floating-point (global) features for re-ranking. To obtain such binary features, we propose a similarity-constrained deep hashing method, which can be easily integrated into the VPR pipeline. Finally, we improve our training strategy and unify the training protocol of several common training datasets to merge them for better training of VPR models. Extensive experiments show that ......

cs.CV↗

arXiv2024

EDTformer: An Efficient Decoder Transformer for Visual Place Recognition

Visual place recognition (VPR) aims to determine the general geographical location of a query image by retrieving visually similar images from a large geo-tagged database. To obtain a global representation for each place image, most approaches typically focus on the aggregation of deep features extracted from a backbone through using current prominent architectures (e.g., CNNs, MLPs, pooling layer, and transformer encoder), giving little attention to the transformer decoder. However, we argue that its strong capability to capture contextual dependencies and generate accurate features holds considerable potential for the VPR task. To this end, we propose an Efficient Decoder Transformer (EDTformer) for feature aggregation, which consists of several stacked simplified decoder blocks followed by two linear layers to directly produce robust and discriminative global representations. Specifically, we do this by formulating deep features as the keys and values, as well as a set of learnable parameters as the queries. Our EDTformer can fully utilize the contextual information within deep features, then gradually decode and aggregate the effective features into the learnable queries to output the global representations. Moreover, to provide more powerful deep features for EDTformer and further facilitate the robustness, we use the foundation model DINOv2 as the backbone and propose a Low-rank Parallel Adaptation (LoPA) method to enhance its performance in VPR, which can refine the intermediate features of the backbone progressively in a memory- and parameter-efficient way. As a result, our method not only outperforms single-stage VPR methods on multiple benchmark datasets, but also outperforms two-stage VPR methods which add a re-ranking with considerable cost. Code will be available at https://github.com/Tong-Jin01/EDTformer.

cs.CV↗

arXiv2024

New building blocks for $\mathbb{F}_1$-geometry: bands and band schemes

We develop and study a generalization of commutative rings called bands, along with the corresponding geometric theory of band schemes. Bands generalize both hyperrings, in the sense of Krasner, and partial fields in the sense of Semple and Whittle. They from a ring-like counterpart to the field-like category of idylls introduced by the first and third author. The first part of the paper is dedicated to establishing fundamental properties of bands analogous to basic facts in commutative algebra. In particular, we introduce various kinds of ideals in a band and explore their properties, and we study localization, quotients, limits, and colimits. The second part of the paper studies band schemes. After giving the definition, we present some examples of band schemes, along with basic properties of band schemes and morphisms thereof, and we describe functors into some other scheme theories. In the third part, we discuss some ``visualizations'' of band schemes, which are different topological spaces that one can functorially associate to a band scheme $X$.

math.AG↗

arXiv2023

Representability of orthogonal matroids over partial fields

Let $r \leqslant n$ be nonnegative integers, and let $N = \binom{n}{r} - 1$. For a matroid $M$ of rank $r$ on the finite set $E = [n]$ and a partial field $k$ in the sense of Semple--Whittle, it is known that the following are equivalent: (a) $M$ is representable over $k$; (b) there is a point $p = (p_J) \in {\bf P}^N(k)$ with support $M$ (meaning that $\text{Supp}(p) := \{J \in \binom{E}{r} \; \vert \; p_J \ne 0\}$ of $p$ is the set of bases of $M$) satisfying the Grassmann-Plücker equations; and (c) there is a point $p = (p_J) \in {\bf P}^N(k)$ with support $M$ satisfying just the 3-term Grassmann-Plücker equations. Moreover, by a theorem of P. Nelson, almost all matroids (meaning asymptotically 100%) are not representable over any partial field. We prove analogues of these facts for Lagrangian orthogonal matroids in the sense of Gelfand-Serganova, which are equivalent to even Delta-matroids in the sense of Bouchet.

math.CO↗

arXiv2023

Orthogonal matroids over tracts

We generalize Baker-Bowler's theory of matroids over tracts to orthogonal matroids, define orthogonal matroids with coefficients in tracts in terms of Wick functions, orthogonal signatures, circuit sets, and orthogonal vector sets, and establish basic properties on functoriality, duality, and minors. Our cryptomorphic definitions of orthogonal matroids over tracts provide proofs of several representation theorems for orthogonal matroids. In particular, we give a new proof that an orthogonal matroid is regular if and only if it is representable over $\mathbb{F}_2$ and $\mathbb{F}_3$, which was originally shown by Geelen, and we prove that an orthogonal matroid is representable over the sixth-root-of-unity partial field if and only if it is representable over $\mathbb{F}_3$ and $\mathbb{F}_4$.

math.CO↗

arXiv2020

On the structure of hyperfields obtained as quotients of fields

We determine all isomorphism classes of hyperfields of a given finite order which can be obtained as quotients of finite fields of sufficiently large order. Using this result, we determine which hyperfields of order at most 4 are quotients of fields. The main ingredients in the proof are the Weil bounds from number theory and a result from Ramsey theory.

math.RA↗