SearcharxivSearch

arXiv subjects

Wenyi Chen

Publications and source records attributed to Wenyi Chen.

8 recordsLinked to original sources

Hallucination in Multimodal Foundation Models: A Survey on Causes, Corrections, and Evaluations

Multimodal Foundation Models represent a significant leap in artificial intelligence. Among them, Large Vision-Language Models (LVLMs) serve as the typical representative of these foundation models, which integrate visual modality directly into Large Language Models (LLMs). They have demonstrated strong capabilities in information processing and generation. However, the existence of hallucinations has limited the potential and practical effectiveness of LVLM in various fields. Although lots of work has been devoted to hallucination mitigation and correction, there are few reviews to summarize them. To address this gap, this survey provides a systematic review of the hallucination landscape in LVLMs. We categorize the causes related to model architecture and data quality, and construct a comprehensive taxonomy of existing mitigation strategies. Furthermore, we critically assess current hallucination evaluation benchmarks from both discriminative and generative perspectives, highlighting the limitations of existing metrics. This survey concludes by discussing open challenges and future research directions to advance the reliability and trustworthiness of LVLMs.

cs.AI

Unsupervised Foggy Scene Understanding via Self Spatial-Temporal Label Diffusion

Understanding foggy image sequence in the driving scenes is critical for autonomous driving, but it remains a challenging task due to the difficulty in collecting and annotating real-world images of adverse weather. Recently, the self-training strategy has been considered a powerful solution for unsupervised domain adaptation, which iteratively adapts the model from the source domain to the target domain by generating target pseudo labels and re-training the model. However, the selection of confident pseudo labels inevitably suffers from the conflict between sparsity and accuracy, both of which will lead to suboptimal models. To tackle this problem, we exploit the characteristics of the foggy image sequence of driving scenes to densify the confident pseudo labels. Specifically, based on the two discoveries of local spatial similarity and adjacent temporal correspondence of the sequential image data, we propose a novel Target-Domain driven pseudo label Diffusion (TDo-Dif) scheme. It employs superpixels and optical flows to identify the spatial similarity and temporal correspondence, respectively and then diffuses the confident but sparse pseudo labels within a superpixel or a temporal corresponding pair linked by the flow. Moreover, to ensure the feature similarity of the diffused pixels, we introduce local spatial similarity loss and temporal contrastive loss in the model re-training stage. Experimental results show that our TDo-Dif scheme helps the adaptive model achieve 51.92% and 53.84% mean intersection-over-union (mIoU) on two publicly available natural foggy datasets (Foggy Zurich and Foggy Driving), which exceeds the state-of-the-art unsupervised domain adaptive semantic segmentation methods. Models and data can be found at https://github.com/velor2012/TDo-Dif.

cs.CV

A gradient descent akin method for inequality constrained optimization

We propose a first-order method for solving inequality constrained optimization problems. The method is derived from our previous work [12], a modified search direction method (MSDM) that applies the singular-value decomposition of normalized gradients. In this work, we simplify its computational framework to a "gradient descent akin" method, i.e., the search direction is computed using a linear combination of the negative and normalized objective and constraint gradient. The main focus of this work is to provide a mathematical aspect to the method. We analyze the global behavior and convergence of the method using a dynamical systems approach. We then prove that the resulting trajectories find local solutions by asymptotically converging to the central path(s) for the logarithmic barrier interior-point method under the so-called relative convex condition. Numerical examples are reported, which include both common test examples and applications in shape optimization.

math.OC

Optimal function spaces for the weak continuity of the distributional $k$-Hessian

In this paper we introduce the notion of distributional $k$-Hessian associated with Besov type functions in Euclidean $n$-space, $k=2,\ldots,n$. Particularly, inspired by recent work of Baer and Jerison on distributional Hessian determinant, we show that the distributional $k$-Hessian is weak continuous on the Besov space $B(2-\frac{2}{k},k)$, and the result is optimal in the framework of the space $B(s,p)$, i.e., the distributional $k$-Hessian is well defined in $B(s,p)$ if and only if $B(s,p)\subset B_{loc}(2-\frac{2}{k},k)$.

math.AP

Characterizations of the BMO and Lipschitz spaces via commutators on weak Lebesgue and Morrey spaces

We prove that the weak Morrey space $WM^{p}_{q}$ is contained in the Morrey space $M^{p}_{q_{1}}$ for $1\leq q_{1}< q\leq p<\infty$. As applications, we show that if the commutator $[b,T]$ is bounded from $L^p$ to $L^{p,\infty}$ for some $p\in (1,\infty)$, then $b\in \mathrm{BMO}$, where $T$ is a Calderón-Zygmund operator. Also, for $1<p\leq q<\infty$, $b\in \mathrm{BMO}$ if and only if $[b,T]$ is bounded from $M^{p}_{q}$ to $WM_{q}^{p}$. For $b$ belonging to Lipschitz class, we obtain similar results.

math.FA

Characterization of CMO via compactness of the commutators of bilinear fractional integral operators

Let $I_α$ be the bilinear fractional integral operator, $B_α$ be a more singular family of bilinear fractional integral operators and $\vec{b}=(b,b)$. Bényi et al. in \cite{B1} showed that if $b\in {\rm CMO}$, the {\rm BMO}-closure of $C^{\infty}_{c}(\mathbb{R}^n)$, the commutator $[b,B_α]_{i}(i=1,2)$ is a separately compact operator. In this paper, it is proved that $b\in {\rm CMO}$ is necessary for $[b,B_α]_{i}(i=1,2)$ is a compact operator. Also, the authors characterize the compactness of the {\bf iterated} commutator $[Π\vec{b},I_α]$ of bilinear fractional integral operator. More precisely, the commutator $[Π\vec{b},I_α]$ is a compact operator if and only if $b\in {\rm CMO}$.

math.FA

Maps with prescribed tension fields

We consider maps into Riemannian manifolds of non-positive curvature and start developing a systematic PDE theory. We control the Sobolev $H^{2,2}$-norm of such a map in terms of its energy, the $L^2$-norm of its tension field and a topological term depending on the homotopy class. We also solve a Dirchlet problem without an underlying variational structure, as an extension of the topic of harmonic maps with potentials.

math.DG