arXiv subjects

Seongbeom Park

Publications and source records attributed to Seongbeom Park.

5 recordsLinked to original sources

arXiv2025

Watermarking for Factuality: Guiding Vision-Language Models Toward Truth via Tri-layer Contrastive Decoding

Large Vision-Language Models (LVLMs) have recently shown promising results on various multimodal tasks, even achieving human-comparable performance in certain cases. Nevertheless, LVLMs remain prone to hallucinations -- they often rely heavily on a single modality or memorize training data without properly grounding their outputs. To address this, we propose a training-free, tri-layer contrastive decoding with watermarking, which proceeds in three steps: (1) select a mature layer and an amateur layer among the decoding layers, (2) identify a pivot layer using a watermark-related question to assess whether the layer is visually well-grounded, and (3) apply tri-layer contrastive decoding to generate the final output. Experiments on public benchmarks such as POPE, MME and AMBER demonstrate that our method achieves state-of-the-art performance in reducing hallucinations in LVLMs and generates more visually grounded responses.

cs.CV↗

arXiv2025

Reducing Profile-Based Matching to the Maximum Weight Matching Problem

The profile-based matching problem is the problem of finding a matching that optimizes profile from an instance $(G, r, \langle u_1, \dots, u_r \rangle)$, where $G$ is a bipartite graph $(A \cup B, E)$, $r$ is the number of utility functions, and $u_i: E \to \{ 0, 1, \dots, U_i \}$ is utility functions for $1 \le i \le r$. A matching is optimal if the matching maximizes the sum of the 1st utility, subject to this, maximizes the sum of the 2nd utility, and so on. The profile-based matching can express rank-maximal matching \cite{irving2006rank}, fair matching \cite{huang2016fair}, and weight-maximal matching \cite{huang2012weight}. These problems can be reduced to maximum weight matching problems, but the reduction is known to be inefficient due to the huge weights. This paper presents the condition for a weight function to find an optimal matching by reducing profile-based matching to the maximum weight matching problem. It is shown that a weight function which represents utilities as a mixed-radix numeric system with base-$(2U_i+1)$ can be used, so the complexity of the problem is $O(m\sqrt{n}(\log{n} + \sum_{i=1}^{r}\log{U_i}))$ for $n = |V|$, $m = |E|$. In addition, it is demonstrated that the weight lower bound for rank-maximal/fair/weight-maximal matching, better computational complexity for fair/weight-maximal matching, and an algorithm to verify a maximum weight matching can be reduced to rank-maximal matching. Finally, the effectiveness of the profile-based algorithm is evaluated with real data for school choice lottery.

cs.DM↗

arXiv2024

Stable Marriage with One-Sided Preference

Many countries around the world, including Korea, use the school choice lottery system. However, this method has a problem in that many students are assigned to less-preferred schools based on the lottery results. In addition, the task of finding a good assignment with ties often has a time complexity of NP, making it a very difficult problem to improve the quality of the assignment. In this paper, we prove that the problem of finding a stable matching that maximizes the student-oriented preference utility in a two-sided market with one-sided preference can be solved in polynomial time, and we verify through experiments that the quality of assignment is improved. The main contributions of this paper are as follows. We found that stable student-oriented allocation in a two-sided market with one-sided preferences is the same as stable allocation in a two-sided market with symmetric preferences. In addition, we defined a method to quantify the quality of allocation from a preference utilitarian perspective. Based on the above two, it was proven that the problem of finding a stable match that maximizes the preference utility in a two-sided market with homogeneous preferences can be reduced to an allocation problem. In this paper, through an experiment, we quantitatively verified that optimal student assignment assigns more students to schools of higher preference, even in situations where many students are assigned to schools of low preference using the existing assignment method.

cs.DM↗

arXiv2023

Ensuring Visual Commonsense Morality for Text-to-Image Generation

Text-to-image generation methods produce high-resolution and high-quality images, but these methods should not produce immoral images that may contain inappropriate content from the perspective of commonsense morality. In this paper, we aim to automatically judge the immorality of synthesized images and manipulate these images into morally acceptable alternatives. To this end, we build a model that has three main primitives: (1) recognition of the visual commonsense immorality in a given image, (2) localization or highlighting of immoral visual (and textual) attributes that contribute to the immorality of the image, and (3) manipulation of an immoral image to create a morally-qualifying alternative. We conduct experiments and human studies using the state-of-the-art Stable Diffusion text-to-image generation model, demonstrating the effectiveness of our ethical image manipulation approach.

cs.CV↗

arXiv2022

Zero-shot Visual Commonsense Immorality Prediction

Artificial intelligence is currently powering diverse real-world applications. These applications have shown promising performance, but raise complicated ethical issues, i.e. how to embed ethics to make AI applications behave morally. One way toward moral AI systems is by imitating human prosocial behavior and encouraging some form of good behavior in systems. However, learning such normative ethics (especially from images) is challenging mainly due to a lack of data and labeling complexity. Here, we propose a model that predicts visual commonsense immorality in a zero-shot manner. We train our model with an ETHICS dataset (a pair of text and morality annotation) via a CLIP-based image-text joint embedding. In a testing phase, the immorality of an unseen image is predicted. We evaluate our model with existing moral/immoral image datasets and show fair prediction performance consistent with human intuitions. Further, we create a visual commonsense immorality benchmark with more general and extensive immoral visual contents. Codes and dataset are available at https://github.com/ku-vai/Zero-shot-Visual-Commonsense-Immorality-Prediction. Note that this paper might contain images and descriptions that are offensive in nature.

cs.CV↗