arXiv subjects

Yanting Zhang

Publications and source records attributed to Yanting Zhang.

At least 37 records · Page 2Linked to original sources

arXiv2025

Vehicular Road Crack Detection with Deep Learning: A New Online Benchmark for Comprehensive Evaluation of Existing Algorithms

In the emerging field of urban digital twins (UDTs), advancing intelligent road inspection (IRI) vehicles with automatic road crack detection systems is essential for maintaining civil infrastructure. Over the past decade, deep learning-based road crack detection methods have been developed to detect cracks more efficiently, accurately, and objectively, with the goal of replacing manual visual inspection. Nonetheless, there is a lack of systematic reviews on state-of-the-art (SoTA) deep learning techniques, especially data-fusion and label-efficient algorithms for this task. This paper thoroughly reviews the SoTA deep learning-based algorithms, including (1) supervised, (2) unsupervised, (3) semi-supervised, and (4) weakly-supervised methods developed for road crack detection. Also, we create a dataset called UDTIRI-Crack, comprising $2,500$ high-quality images from seven public annotated sources, as the first extensive online benchmark in this field. Comprehensive experiments are conducted to compare the detection performance, computational efficiency, and generalizability of public SoTA deep learning-based algorithms for road crack detection. In addition, the feasibility of foundation models and large language models (LLMs) for road crack detection is explored. Afterwards, the existing challenges and future development trends of deep learning-based road crack detection algorithms are discussed. We believe this review can serve as practical guidance for developing intelligent road detection vehicles with the next-generation road condition assessment systems. The released benchmark UDTIRI-Crack is available at https://udtiri.com/submission/.

cs.CV↗

arXiv2025

ViPOcc: Leveraging Visual Priors from Vision Foundation Models for Single-View 3D Occupancy Prediction

Inferring the 3D structure of a scene from a single image is an ill-posed and challenging problem in the field of vision-centric autonomous driving. Existing methods usually employ neural radiance fields to produce voxelized 3D occupancy, lacking instance-level semantic reasoning and temporal photometric consistency. In this paper, we propose ViPOcc, which leverages the visual priors from vision foundation models (VFMs) for fine-grained 3D occupancy prediction. Unlike previous works that solely employ volume rendering for RGB and depth image reconstruction, we introduce a metric depth estimation branch, in which an inverse depth alignment module is proposed to bridge the domain gap in depth distribution between VFM predictions and the ground truth. The recovered metric depth is then utilized in temporal photometric alignment and spatial geometric alignment to ensure accurate and consistent 3D occupancy prediction. Additionally, we also propose a semantic-guided non-overlapping Gaussian mixture sampler for efficient, instance-aware ray sampling, which addresses the redundant and imbalanced sampling issue that still exists in previous state-of-the-art methods. Extensive experiments demonstrate the superior performance of ViPOcc in both 3D occupancy prediction and depth estimation tasks on the KITTI-360 and KITTI Raw datasets. Our code is available at: \url{https://mias.group/ViPOcc}.

cs.CV↗

arXiv2024

STEVE Series: Step-by-Step Construction of Agent Systems in Minecraft

Building an embodied agent system with a large language model (LLM) as its core is a promising direction. Due to the significant costs and uncontrollable factors associated with deploying and training such agents in the real world, we have decided to begin our exploration within the Minecraft environment. Our STEVE Series agents can complete basic tasks in a virtual environment and more challenging tasks such as navigation and even creative tasks, with an efficiency far exceeding previous state-of-the-art methods by a factor of $2.5\times$ to $7.3\times$. We begin our exploration with a vanilla large language model, augmenting it with a vision encoder and an action codebase trained on our collected high-quality dataset STEVE-21K. Subsequently, we enhanced it with a Critic and memory to transform it into a complex system. Finally, we constructed a hierarchical multi-agent system. Our recent work explored how to prune the agent system through knowledge distillation. In the future, we will explore more potential applications of STEVE agents in the real world.

cs.CV↗

arXiv2024

Towards Effective Multi-Moving-Camera Tracking: A New Dataset and Lightweight Link Model

Ensuring driving safety for autonomous vehicles has become increasingly crucial, highlighting the need for systematic tracking of on-road pedestrians. Most vehicles are equipped with visual sensors, however, the large-scale visual data has not been well studied yet. Multi-target multi-camera (MTMC) tracking systems are composed of two modules: single-camera tracking (SCT) and inter-camera tracking (ICT). To reliably coordinate between them, MTMC tracking has been a very complicated task, while tracking across multiple moving cameras makes it even more challenging. In this paper, we focus on multi-target multi-moving-camera (MTMMC) tracking, which is attracting increasing attention from the research community. Observing there are few datasets for MTMMC tracking, we collect a new dataset, called Multi-Moving-Camera Track (MMCT), which contains sequences under various driving scenarios. To address the common problems of identity switch easily faced by most existing SCT trackers, especially for moving cameras due to ego-motion between the camera and targets, a lightweight appearance-free global link model, called Linker, is proposed to mitigate the identity switch by associating two disjoint tracklets of the same target into a complete trajectory within the same camera. Incorporated with Linker, existing SCT trackers generally obtain a significant improvement. Moreover, to alleviate the impact of the image style variations caused by different cameras, a color transfer module is effectively incorporated to extract cross-camera consistent appearance features for pedestrian association across moving cameras for ICT, resulting in a much improved MTMMC tracking system, which can constitute a step further towards coordinated mining of multiple moving cameras. The project page is available at https://dhu-mmct.github.io/.

cs.CV↗

arXiv2024

Every subcubic graph is packing $(1,1,2,2,3)$-colorable

For a sequence $S=(s_1, \ldots, s_k)$ of non-decreasing integers, a packing $S$-coloring of a graph $G$ is a partition of its vertex set $V(G)$ into $V_1, \ldots, V_k$ such that for every pair of distinct vertices $u,v \in V_i$, where $1 \le i \le k$, the distance between $u$ and $v$ is at least $s_i+1$. The packing chromatic number, $χ_p(G)$, of a graph $G$ is the smallest integer $k$ such that $G$ has a packing $(1,2, \ldots, k)$-coloring. Gastineau and Togni asked an open question ``Is it true that the $1$-subdivision ($D(G)$) of any subcubic graph $G$ has packing chromatic number at most $5$?'' and later Brešar, Klavžar, Rall, and Wash conjectured that it is true. In this paper, we prove that every subcubic graph has a packing $(1,1,2,2,3)$-coloring and it is sharp due to the existence of subcubic graphs that are not packing $(1,1,2,2)$-colorable. As a corollary of our result, $χ_p(D(G)) \le 6$ for every subcubic graph $G$, improving a previous bound ($8$) due to Balogh, Kostochka, and Liu in 2019, and we are now just one step away from fully solving the conjecture.

math.CO↗

arXiv2024

Maximizing the spectral radius of graphs of given size with forbidden a subgraph

Let $H_7$ denote the $7$-vertex \textit{fan graph} consisting of a $6$-vertex path plus a vertex adjacent to each vertex of the path. Let $K_3 \vee \frac{m-3}{3}K_1$ be the graph obtained by joining each vertex of a triangle $K_3$ to $\frac{m-3}{3}$ isolated vertices. In this paper, we show that if $G$ is an $H_{7}$-free graph with size $m\geq 33$, then the spectral radius $ ρ(G)\leq 1+\sqrt{m-2},$ equality holds if and only if $G\cong K_3 \vee \frac{m-3}{3}K_1$ (possibly, with some isolated vertices).

math.CO↗

arXiv2024

Do We Really Need a Complex Agent System? Distill Embodied Agent into a Single Model

With the power of large language models (LLMs), open-ended embodied agents can flexibly understand human instructions, generate interpretable guidance strategies, and output executable actions. Nowadays, Multi-modal Language Models~(MLMs) integrate multi-modal signals into LLMs, further bringing richer perception to entity agents and allowing embodied agents to perceive world-understanding tasks more delicately. However, existing works: 1) operate independently by agents, each containing multiple LLMs, from perception to action, resulting in gaps between complex tasks and execution; 2) train MLMs on static data, struggling with dynamics in open-ended scenarios; 3) input prior knowledge directly as prompts, suppressing application flexibility. We propose STEVE-2, a hierarchical knowledge distillation framework for open-ended embodied tasks, characterized by 1) a hierarchical system for multi-granular task division, 2) a mirrored distillation method for parallel simulation data, and 3) an extra expert model for bringing additional knowledge into parallel simulation. After distillation, embodied agents can complete complex, open-ended tasks without additional expert guidance, utilizing the performance and knowledge of a versatile MLM. Extensive evaluations on navigation and creation tasks highlight the superior performance of STEVE-2 in open-ended tasks, with $1.4 \times$ - $7.3 \times$ in performance.

cs.AI↗

arXiv2024

Hierarchical Auto-Organizing System for Open-Ended Multi-Agent Navigation

Due to the dynamic and unpredictable open-world setting, navigating complex environments in Minecraft poses significant challenges for multi-agent systems. Agents must interact with the environment and coordinate their actions with other agents to achieve common objectives. However, traditional approaches often struggle to efficiently manage inter-agent communication and task distribution, crucial for effective multi-agent navigation. Furthermore, processing and integrating multi-modal information (such as visual, textual, and auditory data) is essential for agents to comprehend their goals and navigate the environment successfully and fully. To address this issue, we design the HAS framework to auto-organize groups of LLM-based agents to complete navigation tasks. In our approach, we devise a hierarchical auto-organizing navigation system, which is characterized by 1) a hierarchical system for multi-agent organization, ensuring centralized planning and decentralized execution; 2) an auto-organizing and intra-communication mechanism, enabling dynamic group adjustment under subtasks; 3) a multi-modal information platform, facilitating multi-modal perception to perform the three navigation tasks with one system. To assess organizational behavior, we design a series of navigation tasks in the Minecraft environment, which includes searching and exploring. We aim to develop embodied organizations that push the boundaries of embodied AI, moving it towards a more human-like organizational structure.

cs.CV↗

arXiv2024

MovieChat: From Dense Token to Sparse Memory for Long Video Understanding

Recently, integrating video foundation models and large language models to build a video understanding system can overcome the limitations of specific pre-defined vision tasks. Yet, existing systems can only handle videos with very few frames. For long videos, the computation complexity, memory cost, and long-term temporal connection impose additional challenges. Taking advantage of the Atkinson-Shiffrin memory model, with tokens in Transformers being employed as the carriers of memory in combination with our specially designed memory mechanism, we propose the MovieChat to overcome these challenges. MovieChat achieves state-of-the-art performance in long video understanding, along with the released MovieChat-1K benchmark with 1K long video and 14K manual annotations for validation of the effectiveness of our method.

cs.CV↗

arXiv2024

Generalized Correspondence Matching via Flexible Hierarchical Refinement and Patch Descriptor Distillation

Correspondence matching plays a crucial role in numerous robotics applications. In comparison to conventional hand-crafted methods and recent data-driven approaches, there is significant interest in plug-and-play algorithms that make full use of pre-trained backbone networks for multi-scale feature extraction and leverage hierarchical refinement strategies to generate matched correspondences. The primary focus of this paper is to address the limitations of deep feature matching (DFM), a state-of-the-art (SoTA) plug-and-play correspondence matching approach. First, we eliminate the pre-defined threshold employed in the hierarchical refinement process of DFM by leveraging a more flexible nearest neighbor search strategy, thereby preventing the exclusion of repetitive yet valid matches during the early stages. Our second technical contribution is the integration of a patch descriptor, which extends the applicability of DFM to accommodate a wide range of backbone networks pre-trained across diverse computer vision tasks, including image classification, semantic segmentation, and stereo matching. Taking into account the practical applicability of our method in real-world robotics applications, we also propose a novel patch descriptor distillation strategy to further reduce the computational complexity of correspondence matching. Extensive experiments conducted on three public datasets demonstrate the superior performance of our proposed method. Specifically, it achieves an overall performance in terms of mean matching accuracy of 0.68, 0.92, and 0.95 with respect to the tolerances of 1, 3, and 5 pixels, respectively, on the HPatches dataset, outperforming all other SoTA algorithms. Our source code, demo video, and supplement are publicly available at mias.group/GCM.

cs.CV↗

arXiv2024

On friendship and cyclic parking functions

In parking problems, a given number of cars enter a one-way street sequentially, and try to park according to a specified preferred spot in the street. Various models are possible depending on the chosen rule for collisions, when two cars have the same preferred spot. In classical parking functions, if a car's preferred spot is already occupied by a previous car, it drives forward and looks for the first unoccupied spot to park. In this work, we introduce a variant of classical parking functions, called "friendship parking functions", which imposes additional restrictions on where cars can park. Namely, a car can only end up parking next to cars which are its friends (friendship will correspond to adjacency in an underlying graph). We characterise and enumerate such friendship parking functions according to their outcome permutation, which describes the final configuration when all cars have parked. We apply this to the case where the underlying friendship graph is the cycle graph. Finally, we consider a subset of classical parking functions, called "cyclic parking functions", where cars end up in an increasing cyclic order. We enumerate these cyclic parking functions and exhibit a bijection to permutation components.

math.CO↗

arXiv2023

UniAP: Towards Universal Animal Perception in Vision via Few-shot Learning

Animal visual perception is an important technique for automatically monitoring animal health, understanding animal behaviors, and assisting animal-related research. However, it is challenging to design a deep learning-based perception model that can freely adapt to different animals across various perception tasks, due to the varying poses of a large diversity of animals, lacking data on rare species, and the semantic inconsistency of different tasks. We introduce UniAP, a novel Universal Animal Perception model that leverages few-shot learning to enable cross-species perception among various visual tasks. Our proposed model takes support images and labels as prompt guidance for a query image. Images and labels are processed through a Transformer-based encoder and a lightweight label encoder, respectively. Then a matching module is designed for aggregating information between prompt guidance and the query image, followed by a multi-head label decoder to generate outputs for various tasks. By capitalizing on the shared visual characteristics among different animals and tasks, UniAP enables the transfer of knowledge from well-studied species to those with limited labeled data or even unseen species. We demonstrate the effectiveness of UniAP through comprehensive experiments in pose estimation, segmentation, and classification tasks on diverse animal species, showcasing its ability to generalize and adapt to new classes with minimal labeled examples.

cs.CV↗

arXiv2023

The $α$-index of graphs without intersecting triangles/quadrangles as a minor

The $A_α$-matrix of a graph $G$ is the convex linear combination of the adjacency matrix $A(G)$ and the diagonal matrix of vertex degrees $D(G)$, i.e., $A_α(G) = αD(G) + (1 - α)A(G)$, where $0\leqα\leq1$. The $α$-index of $G$ is the largest eigenvalue of $A_α(G)$. Particularly, the matrix $A_0(G)$ (resp. $2A_{\frac{1}{2}}(G)$) is exactly the adjacency matrix (resp. signless Laplacian matrix) of $G$. He, Li and Feng [arXiv:2301.06008 (2023)] determined the extremal graphs with maximum adjacency spectral radius among all graphs of sufficiently large order without intersecting triangles and quadrangles as a minor, respectively. Motivated by the above results of He, Li and Feng, in this paper we characterize the extremal graphs with maximum $α$-index among all graphs of sufficiently large order without intersecting triangles and quadrangles as a minor for any $0<α<1$, respectively. As by-products, we determine the extremal graphs with maximum signless Laplacian radius among all graphs of sufficiently large order without intersecting triangles and quadrangles as a minor, respectively.

math.CO↗

arXiv2023

DiffFashion: Reference-based Fashion Design with Structure-aware Transfer by Diffusion Models

Image-based fashion design with AI techniques has attracted increasing attention in recent years. We focus on a new fashion design task, where we aim to transfer a reference appearance image onto a clothing image while preserving the structure of the clothing image. It is a challenging task since there are no reference images available for the newly designed output fashion images. Although diffusion-based image translation or neural style transfer (NST) has enabled flexible style transfer, it is often difficult to maintain the original structure of the image realistically during the reverse diffusion, especially when the referenced appearance image greatly differs from the common clothing appearance. To tackle this issue, we present a novel diffusion model-based unsupervised structure-aware transfer method to semantically generate new clothes from a given clothing image and a reference appearance image. In specific, we decouple the foreground clothing with automatically generated semantic masks by conditioned labels. And the mask is further used as guidance in the denoising process to preserve the structure information. Moreover, we use the pre-trained vision Transformer (ViT) for both appearance and structure guidance. Our experimental results show that the proposed method outperforms state-of-the-art baseline models, generating more realistic images in the fashion design task. Code and demo can be found at https://github.com/Rem105-210/DiffFashion.

cs.CV↗

arXiv2022

A generalization on spectral extrema of $K_{s,t}$-minor free graphs

The spectral extrema problems on forbidding minors have aroused wide attention. Very recently, Zhai and Lin [J. Combin. Theory Ser. B 157 (2022) 184--215] determined the extremal graph with maximum adjacency spectral radius among all $K_{s,t}$-minor free graphs of sufficiently large order. The matrix $A_α(G)$ is a generalization of the adjacency matrix $A(G)$, which is defined by Nikiforov \cite{Nikiforov2} as $$A_α(G) = αD(G) + (1 - α)A(G),$$ where $0\leqα\leq1$. Given a graph $F$, the $A_α$-spectral extrema problem is to determine the maximum spectral radius of $A_α(G)$ or characterize the extremal graph among all graphs with no subgraph isomorphic to $F$. For $α=0$, the matrix $A_α(G)$ is exactly the adjacency matrix $A(G)$. Motivated by the nice work of Zhai and Lin, in this paper we determine the extremal graph with maximum $A_α$-spectral radius among all $K_{s,t}$-minor free graphs of sufficiently large order, where $0<α<1$ and $2\leq s\leq t$. As by-products, we completely solve the Conjecture posed by Chen and Zhang in [Linear Multilinear Algebra 69 (10) (2021) 1922--1934].

math.CO↗

arXiv2022

Maxima of the $Q$-index: Graphs with no $K_{1,t}$-minor

A graph is said to be \textit{$H$-minor free} if it does not contain $H$ as a minor. In this paper, we characteristic the unique extremal graph with maximal $Q$-index among all $n$-vertex $K_{1,t}$-minor free graphs ($t\ge3$).

math.CO↗

arXiv2022

High-fidelity 3D Model Compression based on Key Spheres

In recent years, neural signed distance function (SDF) has become one of the most effective representation methods for 3D models. By learning continuous SDFs in 3D space, neural networks can predict the distance from a given query space point to its closest object surface,whose positive and negative signs denote inside and outside of the object, respectively. Training a specific network for each 3D model, which individually embeds its shape, can realize compressed representation of objects by storing fewer network (and possibly latent) parameters. Consequently, reconstruction through network inference and surface recovery can be achieved. In this paper, we propose an SDF prediction network using explicit key spheres as input. Key spheres are extracted from the internal space of objects, whose centers either have relatively larger SDF values (sphere radii), or are located at essential positions. By inputting the spatial information of multiple spheres which imply different local shapes, the proposed method can significantly improve the reconstruction accuracy with a negligible storage cost. Compared to previous works, our method achieves the high-fidelity and high-compression 3D object coding and reconstruction. Experiments conducted on three datasets verify the superior performance of our method.

cs.CV↗

arXiv2021

SN-Graph: a Minimalist 3D Object Representation for Classification

Using deep learning techniques to process 3D objects has achieved many successes. However, few methods focus on the representation of 3D objects, which could be more effective for specific tasks than traditional representations, such as point clouds, voxels, and multi-view images. In this paper, we propose a Sphere Node Graph (SN-Graph) to represent 3D objects. Specifically, we extract a certain number of internal spheres (as nodes) from the signed distance field (SDF), and then establish connections (as edges) among the sphere nodes to construct a graph, which is seamlessly suitable for 3D analysis using graph neural network (GNN). Experiments conducted on the ModelNet40 dataset show that when there are fewer nodes in the graph or the tested objects are rotated arbitrarily, the classification accuracy of SN-Graph is significantly higher than the state-of-the-art methods.

cs.CV↗