arXiv subjects

Zhenhao Chen

Publications and source records attributed to Zhenhao Chen.

22 records · Page 2Linked to original sources

arXiv2023

BenchLMM: Benchmarking Cross-style Visual Capability of Large Multimodal Models

Large Multimodal Models (LMMs) such as GPT-4V and LLaVA have shown remarkable capabilities in visual reasoning with common image styles. However, their robustness against diverse style shifts, crucial for practical applications, remains largely unexplored. In this paper, we propose a new benchmark, BenchLMM, to assess the robustness of LMMs against three different styles: artistic image style, imaging sensor style, and application style, where each style has five sub-styles. Utilizing BenchLMM, we comprehensively evaluate state-of-the-art LMMs and reveal: 1) LMMs generally suffer performance degradation when working with other styles; 2) An LMM performs better than another model in common style does not guarantee its superior performance in other styles; 3) LMMs' reasoning capability can be enhanced by prompting LMMs to predict the style first, based on which we propose a versatile and training-free method for improving LMMs; 4) An intelligent LMM is expected to interpret the causes of its errors when facing stylistic variations. We hope that our benchmark and analysis can shed new light on developing more intelligent and versatile LMMs.

cs.CV↗

arXiv2023

Unsupervised Sampling Promoting for Stochastic Human Trajectory Prediction

The indeterminate nature of human motion requires trajectory prediction systems to use a probabilistic model to formulate the multi-modality phenomenon and infer a finite set of future trajectories. However, the inference processes of most existing methods rely on Monte Carlo random sampling, which is insufficient to cover the realistic paths with finite samples, due to the long tail effect of the predicted distribution. To promote the sampling process of stochastic prediction, we propose a novel method, called BOsampler, to adaptively mine potential paths with Bayesian optimization in an unsupervised manner, as a sequential design strategy in which new prediction is dependent on the previously drawn samples. Specifically, we model the trajectory sampling as a Gaussian process and construct an acquisition function to measure the potential sampling value. This acquisition function applies the original distribution as prior and encourages exploring paths in the long-tail region. This sampling method can be integrated with existing stochastic predictive models without retraining. Experimental results on various baseline methods demonstrate the effectiveness of our method.

cs.CV↗

arXiv2020

Multimodal Learning for Hateful Memes Detection

Memes are used for spreading ideas through social networks. Although most memes are created for humor, some memes become hateful under the combination of pictures and text. Automatically detecting the hateful memes can help reduce their harmful social impact. Unlike the conventional multimodal tasks, where the visual and textual information is semantically aligned, the challenge of hateful memes detection lies in its unique multimodal information. The image and text in memes are weakly aligned or even irrelevant, which requires the model to understand the content and perform reasoning over multiple modalities. In this paper, we focus on multimodal hateful memes detection and propose a novel method that incorporates the image captioning process into the memes detection process. We conduct extensive experiments on multimodal meme datasets and illustrated the effectiveness of our approach. Our model achieves promising results on the Hateful Memes Detection Challenge.

cs.CV↗

arXiv2017

Optimal coupling patterns in interconnected communication networks

Traffic dynamics on single or isolated complex networks has been extensively studied in the past decade. Recently, several coupled network models have been developed to describe the interactions between real-world networked systems. In a system of interconnected networks, the coupling links refer to the physical links between networks and provide paths for traffic transmission. Thus, the coupling pattern, i.e., the way coupling links are added, has a profound influence on the overall traffic performance. In this paper, we employ a simulated annealing (SA) algorithm to find a near-optimal configuration of the coupling links, which effectively improves the overall traffic capacity compared with random, assortative and disassortative couplings. Furthermore, we investigate the optimal configuration of coupling links given by the SA algorithm and develop a faster method to select the coupling links.

physics.soc-ph↗