arXiv subjects
Publications and source records attributed to Jian Ding.
We consider the noisy majority vote process on infinite regular trees with degree $d\geq 3$, and we prove the non-ergodicity, i.e., there exist multiple equilibrium measures. Our work extends a result of Bramson and Gray (2021) for $d\geq 5$.
We consider the long-range random field Ising model in dimension $d = 1, 2$, whereas the long-range interaction is of the form $J_{xy} = |x-y|^{-α}$ with $1< α< 3/2$ for $d=1$ and with $2 < α\leq 3$ for $d = 2$. Our main results establish phase transitions in these regimes. In one dimension, we employ a Peierls argument with some novel modification, suitable for dealing with the randomness coming from the external field; in two dimensions, our proof follows that of Affonso, Bissacot, and Maia (2023) with some adaptations, but new ideas are required in the critical case of $α=3$.
The robustness of object detection models is a major concern when applied to real-world scenarios. The performance of most models tends to degrade when confronted with images affected by corruptions, since they are usually trained and evaluated on clean datasets. While numerous studies have explored the robustness of object detection models on natural images, there is a paucity of research focused on models applied to aerial images, which feature complex backgrounds, substantial variations in scales, and orientations of objects. This paper addresses the challenge of assessing the robustness of object detection models on aerial images, with a specific emphasis on scenarios where images are affected by clouds. In this study, we introduce two novel benchmarks based on DOTA-v1.0. The first benchmark encompasses 19 prevalent corruptions, while the second focuses on the cloud-corrupted condition-a phenomenon uncommon in natural images yet frequent in aerial photography. We systematically evaluate the robustness of mainstream object detection models and perform necessary ablation experiments. Through our investigations, we find that rotation-invariant modeling and enhanced backbone architectures can improve the robustness of models. Furthermore, increasing the capacity of Transformer-based backbones can strengthen their robustness. The benchmarks we propose and our comprehensive experimental analyses can facilitate research on robust object detection on aerial images. The codes and datasets are available at: https://github.com/hehaodong530/DOTA-C.
Detecting oriented tiny objects, which are limited in appearance information yet prevalent in real-world applications, remains an intricate and under-explored problem. To address this, we systemically introduce a new dataset, benchmark, and a dynamic coarse-to-fine learning scheme in this study. Our proposed dataset, AI-TOD-R, features the smallest object sizes among all oriented object detection datasets. Based on AI-TOD-R, we present a benchmark spanning a broad range of detection paradigms, including both fully-supervised and label-efficient approaches. Through investigation, we identify a learning bias presents across various learning pipelines: confident objects become increasingly confident, while vulnerable oriented tiny objects are further marginalized, hindering their detection performance. To mitigate this issue, we propose a Dynamic Coarse-to-Fine Learning (DCFL) scheme to achieve unbiased learning. DCFL dynamically updates prior positions to better align with the limited areas of oriented tiny objects, and it assigns samples in a way that balances both quantity and quality across different object shapes, thus mitigating biases in prior settings and sample selection. Extensive experiments across eight challenging object detection datasets demonstrate that DCFL achieves state-of-the-art accuracy, high efficiency, and remarkable versatility. The dataset, benchmark, and code are available at https://chasel-tsui.github.io/AI-TOD-R/.
We introduce a new benchmark designed to advance the development of general-purpose, large-scale vision-language models for remote sensing images. Although several vision-language datasets in remote sensing have been proposed to pursue this goal, existing datasets are typically tailored to single tasks, lack detailed object information, or suffer from inadequate quality control. Exploring these improvement opportunities, we present a Versatile vision-language Benchmark for Remote Sensing image understanding, termed VRSBench. This benchmark comprises 29,614 images, with 29,614 human-verified detailed captions, 52,472 object references, and 123,221 question-answer pairs. It facilitates the training and evaluation of vision-language models across a broad spectrum of remote sensing image understanding tasks. We further evaluated state-of-the-art models on this benchmark for three vision-language tasks: image captioning, visual grounding, and visual question answering. Our work aims to significantly contribute to the development of advanced vision-language models in the field of remote sensing. The data and code can be accessed at https://github.com/lx709/VRSBench.
We study random walks in random environments generated by the two-dimensional Gaussian free field. More specifically, we consider a rescaled lattice with a small mesh size and view it as a random network where each edge is equipped with an electric resistance given by a regularization for the exponentiation of the Gaussian free field. We prove the tightness of random walks on such random networks at high temperature as the mesh size tends to 0. Our proof is based on a careful analysis of the (random) effective resistances as well as their connections to random walks.
Most current LLM-based models for video understanding can process videos within minutes. However, they struggle with lengthy videos due to challenges such as "noise and redundancy", as well as "memory and computation" constraints. In this paper, we present Goldfish, a methodology tailored for comprehending videos of arbitrary lengths. We also introduce the TVQA-long benchmark, specifically designed to evaluate models' capabilities in understanding long videos with questions in both vision and text content. Goldfish approaches these challenges with an efficient retrieval mechanism that initially gathers the top-k video clips relevant to the instruction before proceeding to provide the desired response. This design of the retrieval mechanism enables the Goldfish to efficiently process arbitrarily long video sequences, facilitating its application in contexts such as movies or television series. To facilitate the retrieval process, we developed MiniGPT4-Video that generates detailed descriptions for the video clips. In addressing the scarcity of benchmarks for long video evaluation, we adapted the TVQA short video benchmark for extended content analysis by aggregating questions from entire episodes, thereby shifting the evaluation from partial to full episode comprehension. We attained a 41.78% accuracy rate on the TVQA-long benchmark, surpassing previous methods by 14.94%. Our MiniGPT4-Video also shows exceptional performance in short video comprehension, exceeding existing state-of-the-art methods by 3.23%, 2.03%, 16.5% and 23.59% on the MSVD, MSRVTT, TGIF, and TVQA short video benchmarks, respectively. These results indicate that our models have significant improvements in both long and short-video understanding. Our models and code have been made publicly available at https://vision-cair.github.io/Goldfish_website/
For the critical level-set of the Gaussian free field on the metric graph of $\mathbb Z^d$, we consider the one-arm probability $θ_d(N)$, i.e., the probability that the boundary of a box of side length $2N$ is connected to the center. We prove that $θ_d(N)$ is $O(N^{-\frac{d}{2}+1})$ for $3\le d\le 5$, and is $N^{-2+o(1)}$ for $d=6$. Our upper bounds match the lower bounds in a previous work by Ding and Wirth up to a constant factor for $3\le d\le 5$, and match the exponent therein for $d=6$. Combined with our previous result that $θ_d(N) \asymp N^{-2}$ for $d>6$, this seems to present the first percolation model whose one-arm probabilities are essentially completely understood in all dimensions. In particular, these results fully confirm Werner's conjectures (2021) on the one-arm exponents: \begin{equation*} \text{(1) for}\ 3\le d d_c,\ θ_d(N)=N^{-2+o(1)}. \end{equation*} Prior to our work, Drewitz, Prévost and Rodriguez obtained upper bounds for $d\in \{3, 4\}$, which are very sharp although lose some diverging factors. In the same work, they conjectured that $θ_{d_c}(N) = N^{-2+o(1)}$, which is now established. In addition, in a recent concurrent work, Drewitz, Prévost and Rodriguez independently obtained the up-to-constant upper bound for $d=3$.
Instance segmentation of point clouds is a crucial task in 3D field with numerous applications that involve localizing and segmenting objects in a scene. However, achieving satisfactory results requires a large number of manual annotations, which is a time-consuming and expensive process. To alleviate dependency on annotations, we propose a novel framework, FreePoint, for underexplored unsupervised class-agnostic instance segmentation on point clouds. In detail, we represent the point features by combining coordinates, colors, and self-supervised deep features. Based on the point features, we perform a bottom-up multicut algorithm to segment point clouds into coarse instance masks as pseudo labels, which are used to train a point cloud instance segmentation model. We propose an id-as-feature strategy at this stage to alleviate the randomness of the multicut algorithm and improve the pseudo labels' quality. During training, we propose a weakly-supervised two-step training strategy and corresponding losses to overcome the inaccuracy of coarse masks. FreePoint has achieved breakthroughs in unsupervised class-agnostic instance segmentation on point clouds and outperformed previous traditional methods by over 18.2% and a competitive concurrent work UnScene3D by 5.5% in AP. Additionally, when used as a pretext task and fine-tuned on S3DIS, FreePoint performs significantly better than existing self-supervised pre-training methods with limited annotations and surpasses CSC by 6.0% in AP with 10% annotation masks.
In this work, we study the critical long-range percolation on $\mathbb{Z}$, where an edge connects $i$ and $j$ independently with probability $1-\exp\{-β|i-j|^{-2}\}$ for some fixed $β>0$. Viewing this as a random electric network where each edge has a unit conductance, we show that with high probability the effective resistances from the origin 0 to $[-N, N]^c$ and from the interval $[-N,N]$ to $[-2N,2N]^c$ (conditioned on no edge joining $[-N,N]$ and $[-2N,2N]^c$) both have a polynomial lower bound in $N$. Our bound holds for all $β>0$ and thus rules out a potential phase transition (around $β= 1$) which seemed to be a reasonable possibility.
For a continuous map $T: X\rightarrow X$ on a compact metric space $(X,d)$, we say that a function $f: X \rightarrow \mathbb{R}$ has the property $\mathscr{P}_T$ if its time averages along forward orbits of $T$ are maximized at a periodic orbit. In this paper, we prove that for the one-side full shift of two symbols, the property $\mathscr{P}_T$ is prevalent (in the sense of Hunt--Sauer--Yorke) in spaces of Lipschitz functions with respect to metrics with mildly fast decaying rate on the diameters of cylinder sets. This result is a strengthening of \cite[Theorem~A]{BZ16}, confirms the prediction mentioned in the ICM proceeding contribution of J. Bochi (\cite[Seciton 1]{Boc18}) suggested by experimental evidence, and is another step towards the Hunt--Ott conjectures in the area of ergodic optimization.
This paper introduces MiniGPT4-Video, a multimodal Large Language Model (LLM) designed specifically for video understanding. The model is capable of processing both temporal visual and textual data, making it adept at understanding the complexities of videos. Building upon the success of MiniGPT-v2, which excelled in translating visual features into the LLM space for single images and achieved impressive results on various image-text benchmarks, this paper extends the model's capabilities to process a sequence of frames, enabling it to comprehend videos. MiniGPT4-video does not only consider visual content but also incorporates textual conversations, allowing the model to effectively answer queries involving both visual and text components. The proposed model outperforms existing state-of-the-art methods, registering gains of 4.22%, 1.13%, 20.82%, and 13.1% on the MSVD, MSRVTT, TGIF, and TVQA benchmarks respectively. Our models and code have been made publicly available here https://vision-cair.github.io/MiniGPT4-video/
We propose an efficient algorithm for matching two correlated Erdős--Rényi graphs with $n$ vertices whose edges are correlated through a latent vertex correspondence. When the edge density $q= n^{- α+o(1)}$ for a constant $α\in [0,1)$, we show that our algorithm has polynomial running time and succeeds to recover the latent matching as long as the edge correlation is non-vanishing. This is closely related to our previous work on a polynomial-time algorithm that matches two Gaussian Wigner matrices with non-vanishing correlation, and provides the first polynomial-time random graph matching algorithm (regardless of the regime of $q$) when the edge correlation is below the square root of the Otter's constant (which is $\approx 0.338$).
We study the random field Ising model in a two-dimensional box with side length $N$ where the external field is given by independent normal variables with mean $0$ and variance $ε^2$. Our primary result is the following phase transition at $T = T_c$: for $ε\ll N^{-7/8}$ the boundary influence (i.e., the difference between the spin averages at the center of the box with the plus and the minus boundary conditions) decays as $N^{-1/8}$ and thus the disorder essentially has no effect on the boundary influence; for $ε\gg N^{-7/8}$, the boundary influence decays as $N^{-\frac{1}{8}}e^{-Θ(ε^{8/7}\, N)}$ (i.e., the disorder contributes a factor of $e^{-Θ(ε^{8/7}\, N)}$ to the decay rate). For a natural notion of the correlation length, i.e., the minimal size of the box where the boundary influence shrinks by a factor of $2$ from that with no external field, we also prove the following: as $ε\downarrow 0$ the correlation length transits from $Θ(ε^{-8/7})$ at $T_c$ to $e^{Θ(ε^{-4/3}\,\,)}$ for $T < T_c$.
Never having seen an object and heard its sound simultaneously, can the model still accurately localize its visual position from the input audio? In this work, we concentrate on the Audio-Visual Localization and Segmentation tasks but under the demanding zero-shot and few-shot scenarios. To achieve this goal, different from existing approaches that mostly employ the encoder-fusion-decoder paradigm to decode localization information from the fused audio-visual feature, we introduce the encoder-prompt-decoder paradigm, aiming to better fit the data scarcity and varying data distribution dilemmas with the help of abundant knowledge from pre-trained models. Specifically, we first propose to construct Semantic-aware Audio Prompt (SAP) to help the visual foundation model focus on sounding objects, meanwhile, the semantic gap between the visual and audio modalities is also encouraged to shrink. Then, we develop a Correlation Adapter (ColA) to keep minimal training efforts as well as maintain adequate knowledge of the visual foundation model. By equipping with these means, extensive experiments demonstrate that this new paradigm outperforms other fusion-based methods in both the unseen class and cross-dataset settings. We hope that our work can further promote the generalization study of Audio-Visual Localization and Segmentation in practical application scenarios.
In this work, we present Uni3DL, a unified model for 3D and Language understanding. Distinct from existing unified vision-language models in 3D which are limited in task variety and predominantly dependent on projected multi-view images, Uni3DL operates directly on point clouds. This approach significantly expands the range of supported tasks in 3D, encompassing both vision and vision-language tasks in 3D. At the core of Uni3DL, a query transformer is designed to learn task-agnostic semantic and mask outputs by attending to 3D visual features, and a task router is employed to selectively generate task-specific outputs required for diverse tasks. With a unified architecture, our Uni3DL model enjoys seamless task decomposition and substantial parameter sharing across tasks. Uni3DL has been rigorously evaluated across diverse 3D vision-language understanding tasks, including semantic segmentation, object detection, instance segmentation, visual grounding, 3D captioning, and text-3D cross-modal retrieval. It demonstrates performance on par with or surpassing state-of-the-art (SOTA) task-specific models. We hope our benchmark and Uni3DL model will serve as a solid step to ease future research in unified models in the realm of 3D and language understanding. Project page: https://uni3dl.github.io.
Given two Erdős-Rényi graphs with $n$ vertices whose edges are correlated through a latent vertex correspondence, we study complexity lower bounds for the associated correlation detection problem for the class of low-degree polynomial algorithms. We provide evidence that any degree-$O(ρ^{-1})$ polynomial algorithm fails for detection, where $ρ$ is the edge correlation. Furthermore, in the sparse regime where the edge density $q=n^{-1+o(1)}$, we provide evidence that any degree-$d$ polynomial algorithm fails for detection, as long as $\log d=o\big( \frac{\log n}{\log nq} \wedge \sqrt{\log n} \big)$ and the correlation $ρ<\sqrtα$ where $α\approx 0.338$ is the Otter's constant. Our result suggests that several state-of-the-art algorithms on correlation detection and exact matching recovery may be essentially the best possible.
Let $h$ be the planar Gaussian free field and let $D_h$ be a supercritical Liouville quantum gravity (LQG) metric associated with $h$. Such metrics arise as subsequential scaling limits of supercritical Liouville first passage percolation (Ding-Gwynne, 2020) and correspond to values of the matter central charge $\mathbf{c}_{\mathrm M} \in (1,25)$. We show that a.s. the boundary of each complementary connected component of a $D_h$-metric ball is a Jordan curve and is compact and finite-dimensional with respect to $D_h$. This is in contrast to the \emph{whole} boundary of the $D_h$-metric ball, which is non-compact and infinite-dimensional with respect to $D_h$ (Pfeffer, 2021). Using our regularity results for boundaries of complementary connected components of $D_h$-metric balls, we extend the confluence of geodesics results of Gwynne-Miller (2019) to the case of supercritical Liouville quantum gravity. These results show that two $D_h$-geodesics with the same starting point and different target points coincide for a non-trivial initial time interval.