Yuansheng Zhong | Searcharxiv

arXiv2024

Effectiveness Assessment of Recent Large Vision-Language Models

The advent of large vision-language models (LVLMs) represents a remarkable advance in the quest for artificial general intelligence. However, the model's effectiveness in both specialized and general tasks warrants further investigation. This paper endeavors to evaluate the competency of popular LVLMs in specialized and general tasks, respectively, aiming to offer a comprehensive understanding of these novel models. To gauge their effectiveness in specialized tasks, we employ six challenging tasks in three different application scenarios: natural, healthcare, and industrial. These six tasks include salient/camouflaged/transparent object detection, as well as polyp detection, skin lesion detection, and industrial anomaly detection. We examine the performance of three recent open-source LVLMs, including MiniGPT-v2, LLaVA-1.5, and Shikra, on both visual recognition and localization in these tasks. Moreover, we conduct empirical investigations utilizing the aforementioned LVLMs together with GPT-4V, assessing their multi-modal understanding capabilities in general tasks including object counting, absurd question answering, affordance reasoning, attribute recognition, and spatial relation reasoning. Our investigations reveal that these LVLMs demonstrate limited proficiency not only in specialized tasks but also in general tasks. We delve deep into this inadequacy and uncover several potential factors, including limited cognition in specialized tasks, object hallucination, text-to-image interference, and decreased robustness in complex problems. We hope that this study can provide useful insights for the future development of LVLMs, helping researchers improve LVLMs for both general and specialized applications.

cs.CV↗

arXiv2017

Inferring users' preferences through leveraging their social relationships

Recommender systems, inferring users' preferences from their historical activities and personal profiles, have been an enormous success in the last several years. Most of the existing works are based on the similarities of users, objects or both that derived from their purchases records in the online shopping platforms. Such approaches, however, are facing bottlenecks when the known information is limited. The extreme case is how to recommend products to new users, namely the so-called cold-start problem. The rise of the online social networks gives us a chance to break the glass ceiling. Birds of a feather flock together. Close friends may have similar hidden pattern of selecting products and the advices from friends are more trustworthy. In this paper, we integrate the individual's social relationships into recommender systems and propose a new method, called Social Mass Diffusion (SMD), based on a mass diffusion process in the combined network of users' social network and user-item bipartite network. The results show that the SMD algorithm can achieve higher recommendation accuracy than the Mass Diffusion (MD) purely on the bipartite network. Especially, the improvement is striking for small degree users. Moreover, SMD provides a good solution to the cold-start problem. The recommendation accuracy for new users significantly higher than that of the conventional popularity-based algorithm. These results may shed some light on the new designs of better personalized recommender systems and information services.

cs.SI↗