Searcharxiv⌕ Search

arXiv subjects

Jihao Liu

Publications and source records attributed to Jihao Liu.

At least 73 records · Page 4Linked to original sources
arXiv2024

Enhancing Vision-Language Model with Unmasked Token Alignment

Contrastive pre-training on image-text pairs, exemplified by CLIP, becomes a standard technique for learning multi-modal visual-language representations. Although CLIP has demonstrated remarkable performance, training it from scratch on noisy web-scale datasets is computationally demanding. On the other hand, mask-then-predict pre-training approaches, like Masked Image Modeling (MIM), offer efficient self-supervised learning for single-modal representations. This paper introduces Unmasked Token Alignment (UTA), a method that leverages existing CLIP models to further enhance its vision-language representations. UTA trains a Vision Transformer (ViT) by aligning unmasked visual tokens to the corresponding image tokens from a frozen CLIP vision encoder, which automatically aligns the ViT model with the CLIP text encoder. The pre-trained ViT can be directly applied for zero-shot evaluation even without training on image-text pairs. Compared to MIM approaches, UTA does not suffer from training-finetuning inconsistency and is much more training-efficient by avoiding using the extra [MASK] tokens. Extensive experimental results demonstrate that UTA can enhance CLIP models and outperform existing MIM methods on various uni- and multi-modal benchmarks. Code and models are available at https://github.com/jihaonew/UTA.

cs.CV↗
arXiv2024

Exceptional Fano varieties with small minimal log discrepancy

We construct exceptional Fano varieties with the smallest known minimal log discrepancies in all dimensions. These varieties are well-formed hypersurfaces in weighted projective space. Their minimal log discrepancies decay doubly exponentially with dimension, and achieve the optimal value in dimension 2.

math.AG↗
arXiv2024

Complements, index theorem, and minimal log discrepancies of foliated surface singularities

We present an extension of several results on pairs and varieties to foliated surface pairs. We prove the boundedness of local complements, the local index theorem, and the uniform boundedness of minimal log discrepancies (mlds), as well as establishing the existence of uniform rational lc polytopes. Furthermore, we address two questions posed by P. Cascini and C. Spicer on foliations, providing negative responses. We also demonstrate that the Grauert-Riemenschneider type vanishing theorem generally fails for lc foliations on surfaces. In addition, we determine the set of minimal log discrepancies for foliated surface pairs with specific coefficients, which leads to the recovery of Y.-A. Chen's proof on the ascending chain condition conjecture for mlds for foliated surfaces.

math.AG↗
arXiv2024

Uniform rational polytopes of foliated threefolds and the global ACC

In this paper, we show the existence of uniform rational lc polytopes for foliations with functional boundaries in dimension $\leq 3$. As an application, we prove the global ACC for foliated threefolds with arbitrary DCC coefficients. We also provide applications on the accumulation points of lc thresholds of foliations in dimension $\leq 3$.

math.AG↗
arXiv2024

Infinitesimal structure of log canonical thresholds

We show that log canonical thresholds of fixed dimension are standardized. More precisely, we show that any sequence of log canonical thresholds in fixed dimension $d$ accumulates in a way which is i) either similar to how standard and hyperstandard sets accumulate, or ii) to log canonical thresholds in dimension $\leq d-2$. This provides an accurate description on the infinitesimal structure of the set of log canonical thresholds. We also discuss similar behaviors of minimal log discrepancies, canonical thresholds, and K-semistable thresholds.

math.AG↗
arXiv2024

DecisionNCE: Embodied Multimodal Representations via Implicit Preference Learning

Multimodal pretraining is an effective strategy for the trinity of goals of representation learning in autonomous robots: 1) extracting both local and global task progressions; 2) enforcing temporal consistency of visual representation; 3) capturing trajectory-level language grounding. Most existing methods approach these via separate objectives, which often reach sub-optimal solutions. In this paper, we propose a universal unified objective that can simultaneously extract meaningful task progression information from image sequences and seamlessly align them with language instructions. We discover that via implicit preferences, where a visual trajectory inherently aligns better with its corresponding language instruction than mismatched pairs, the popular Bradley-Terry model can transform into representation learning through proper reward reparameterizations. The resulted framework, DecisionNCE, mirrors an InfoNCE-style objective but is distinctively tailored for decision-making tasks, providing an embodied representation learning framework that elegantly extracts both local and global task progression features, with temporal consistency enforced through implicit time contrastive learning, while ensuring trajectory-level instruction grounding via multimodal joint encoding. Evaluation on both simulated and real robots demonstrates that DecisionNCE effectively facilitates diverse downstream policy learning tasks, offering a versatile solution for unified representation and reward learning. Project Page: https://2toinf.github.io/DecisionNCE/

cs.RO↗
arXiv2024

GLID: Pre-training a Generalist Encoder-Decoder Vision Model

This paper proposes a GeneraLIst encoder-Decoder (GLID) pre-training method for better handling various downstream computer vision tasks. While self-supervised pre-training approaches, e.g., Masked Autoencoder, have shown success in transfer learning, task-specific sub-architectures are still required to be appended for different downstream tasks, which cannot enjoy the benefits of large-scale pre-training. GLID overcomes this challenge by allowing the pre-trained generalist encoder-decoder to be fine-tuned on various vision tasks with minimal task-specific architecture modifications. In the GLID training scheme, pre-training pretext task and other downstream tasks are modeled as "query-to-answer" problems, including the pre-training pretext task and other downstream tasks. We pre-train a task-agnostic encoder-decoder with query-mask pairs. During fine-tuning, GLID maintains the pre-trained encoder-decoder and queries, only replacing the topmost linear transformation layer with task-specific linear heads. This minimizes the pretrain-finetune architecture inconsistency and enables the pre-trained model to better adapt to downstream tasks. GLID achieves competitive performance on various vision tasks, including object detection, image segmentation, pose estimation, and depth estimation, outperforming or matching specialist models such as Mask2Former, DETR, ViTPose, and BinsFormer.

cs.CV↗
arXiv2023

On the equivalence between the effective adjunction conjectures of Prokhorov-Shokurov and of Li

Prokhorov and Shokurov introduced the famous effective adjunction conjecture, also known as the effective base-point-freeness conjecture. This conjecture asserts that the moduli component of an lc-trivial fibration is effectively base-point-free. Li proposed a variation of this conjecture, which is known as the $Γ$-effective adjunction conjecture, and proved that a weaker version of his conjecture is implied by the original Prokhorov-Shokurov conjecture. In this paper, we establish the equivalence of Prokhorov-Shokurov's and Li's effective adjunction conjectures. The key to our proof is the formulation of a uniform rational polytope for canonical bundle formulas, which relies on recent developments in the minimal model program theory of algebraically integrable foliations by Ambro-Cascini-Shokurov-Spicer and Chen-Han-Liu-Xie.

math.AG↗
arXiv2023

On global ACC for foliated threefolds

In this paper, we prove the rational coefficient case of the global ACC for foliated threefolds. Specifically, we consider any lc foliated log Calabi-Yau triple $(X,\mathcal{F},B)$ of dimension $3$ whose coefficients belong to a set $Γ$ of rational numbers satisfying the descending chain condition, and prove that the coefficients of $B$ belong to a finite set depending only on $Γ$. To prove our main result, we introduce the concept of generalized foliated quadruples, which is a mixture of foliated triples and Birkar-Zhang's generalized pairs. With this concept, we establish a canonical bundle formula for foliations in any dimension. As for applications, we extend Shokurov's global index conjecture in the classical MMP to foliated triples and prove this conjecture for threefolds with nonzero boundaries and for surfaces. Additionally, we introduce the theory of rational polytopes for functional divisors on foliations and prove some miscellaneous results.

math.AG↗
arXiv2023

On explicit bounds of Fano threefolds

In this paper, we study the explicit geometry of threefolds, in particular, Fano varieties. We find an explicitly computable positive integer $N$, such that all but a bounded family of Fano threefolds have $N$-complements. This result has many applications on finding explicit bounds of algebraic invariants for threefolds. We provide explicit lower bounds for the first gap of the $\mathbb R$-complementary thresholds for threefolds, the first gap of the global lc thresholds, the smallest minimal log discrepancy of exceptional threefolds, and the volume of log threefolds with reduced boundary and ample log canonical divisor. We also provide an explicit upper bound of the anti-canonical volume of exceptional threefolds. While the bounds in this paper may not and are not expected to be optimal, they are the first explicit bounds of these invariants in dimension three.

math.AG↗
arXiv2023

GeoMIM: Towards Better 3D Knowledge Transfer via Masked Image Modeling for Multi-view 3D Understanding

Multi-view camera-based 3D detection is a challenging problem in computer vision. Recent works leverage a pretrained LiDAR detection model to transfer knowledge to a camera-based student network. However, we argue that there is a major domain gap between the LiDAR BEV features and the camera-based BEV features, as they have different characteristics and are derived from different sources. In this paper, we propose Geometry Enhanced Masked Image Modeling (GeoMIM) to transfer the knowledge of the LiDAR model in a pretrain-finetune paradigm for improving the multi-view camera-based 3D detection. GeoMIM is a multi-camera vision transformer with Cross-View Attention (CVA) blocks that uses LiDAR BEV features encoded by the pretrained BEV model as learning targets. During pretraining, GeoMIM's decoder has a semantic branch completing dense perspective-view features and the other geometry branch reconstructing dense perspective-view depth maps. The depth branch is designed to be camera-aware by inputting the camera's parameters for better transfer capability. Extensive results demonstrate that GeoMIM outperforms existing methods on nuScenes benchmark, achieving state-of-the-art performance for camera-based 3D object detection and 3D segmentation. Code and pretrained models are available at https://github.com/Sense-X/GeoMIM.

cs.CV↗
arXiv2023

Semi-ampleness of NQC generalized log canonical pairs

We establish a Kollár-type gluing theory for NQC generalized log canonical pairs and use it to prove semi-ampleness results of NQC generalized pairs. As consequences, we prove the existence of flips for any NQC generalized log canonical pair, and show that NQC generalized log canonical singularities are Du Bois.

math.AG↗
arXiv2023

Optimal bounds on surfaces

We prove that the first gap of $\mathbb R$-complementary thresholds of surfaces is $\frac{1}{13}$. More precisely, the largest $\mathbb R$-complementary threshold for surfaces that is strictly less than $1$ is $\frac{12}{13}$. This result has many applications in explicit birational geometry of surfaces and threefolds and allows us to find several other optimal bounds on surfaces. We show that the first gap of global log canonical threshold for surfaces is $\frac{1}{13}$, answering a question of V. Alexeev and W. Liu. We show that the minimal volume of log surfaces with reduced boundary and ample log canonical divisor is $\frac{1}{462}$, answering a question of J. Kollár. We show that the smallest minimal log discrepancy (mld) of exceptional surfaces is $\frac{1}{13}$. As a special case, we show that the smallest mld of klt Calabi-Yau surfaces is $\frac{1}{13}$, reproving a recent result of L. Esser, B. Totaro, and C. Wang. After a more detailed classification, we classify all exceptional del Pezzo surfaces that are not $\frac{1}{11}$-lt, and show that the smallest mld of exceptional del Pezzo surfaces is $\frac{3}{35}$. We also get better upper bounds of $n$-complements and Tian's $α$-invariants for surfaces. Finally, as an analogue of our main theorem in high dimensions, we propose a question associating the gaps of $\mathbb R$-complementary thresholds with the gaps of mld's and study some special cases of this question.

math.AG↗
arXiv2023

Remark on complements on surfaces

We give an explicit characterization on the singularities of exceptional pairs in any dimension. In particular, we show that any exceptional Fano surface is $\frac{1}{42}$-lc. As corollaries, we show that any $\mathbb R$-complementary surface $X$ has an $n$-complement for some integer $n\leq 192\cdot 84^{128\cdot 42^5}\approx 10^{10^{10.5}}$, and Tian's alpha invariant for any surface is $\leq 3\sqrt{2}\cdot 84^{64\cdot 42^5}\approx 10^{10^{10.2}}$. Although the latter two values are expected to be far from being optimal, they are the first explicit upper bounds of these two algebraic invariants for surfaces.

math.AG↗
arXiv2023

Relative Nakayama-Zariski decomposition and minimal models of generalized pairs

We prove some basic properties of the relative Nakayama-Zariski decomposition. We apply them to the study of lc generalized pairs. We prove the existence of log minimal models or Mori fiber spaces for (relative) lc generalized pairs polarized by an ample divisor. This extends a result of Hashizume-Hu to generalized pairs. We also show that, for any lc generalized pair $(X,B+A,{\bf{M}})/Z$ such that $K_X+B+A+{\bf{M}}_X\sim_{\mathbb R,Z}0$ and $B\geq 0,A\geq 0$, $(X,B,{\bf{M}})/Z$ has either a log minimal model or a Mori fiber space. This is an analogue of a result of Birkar/Hacon-Xu and Hashizume in the category of generalized pairs, and is later shown to be crucial to the proof of the existence of lc generalized flips in full generality.

math.AG↗
arXiv2023

Vanishing theorems for generalized pairs

We establish the Kodaira vanishing theorem and the Kawamata-Viehweg vanishing theorem for lc generalized pairs. As a consequence, we provide a new proof of the base-point-freeness theorem for lc generalized pairs. This new approach allows us to prove the contraction theorem for lc generalized pairs without using Kollár's gluing theory.

math.AG↗
arXiv2023

TokenMix: Rethinking Image Mixing for Data Augmentation in Vision Transformers

CutMix is a popular augmentation technique commonly used for training modern convolutional and transformer vision networks. It was originally designed to encourage Convolution Neural Networks (CNNs) to focus more on an image's global context instead of local information, which greatly improves the performance of CNNs. However, we found it to have limited benefits for transformer-based architectures that naturally have a global receptive field. In this paper, we propose a novel data augmentation technique TokenMix to improve the performance of vision transformers. TokenMix mixes two images at token level via partitioning the mixing region into multiple separated parts. Besides, we show that the mixed learning target in CutMix, a linear combination of a pair of the ground truth labels, might be inaccurate and sometimes counter-intuitive. To obtain a more suitable target, we propose to assign the target score according to the content-based neural activation maps of the two images from a pre-trained teacher model, which does not need to have high performance. With plenty of experiments on various vision transformer architectures, we show that our proposed TokenMix helps vision transformers focus on the foreground area to infer the classes and enhances their robustness to occlusion, with consistent performance gains. Notably, we improve DeiT-T/S/B with +1% ImageNet top-1 accuracy. Besides, TokenMix enjoys longer training, which achieves 81.2% top-1 accuracy on ImageNet with DeiT-S trained for 400 epochs. Code is available at https://github.com/Sense-X/TokenMix.

cs.CV↗