arXiv subjects

Metin Sezgin

Publications and source records attributed to Metin Sezgin.

7 recordsLinked to original sources

arXiv2026

Hybrid Intelligence in Cartoon Captioning: Evaluating AI as a Creative Writing Partner

Crafting cartoon captions requires an understanding of humor, context, and the relationship between image and text. Traditionally, illustrators and writers collaborate to strengthen visual storytelling and comedic timing. With advances in natural language generation, Large Language Models (LLMs) can assist in this process. This study examines AI's role in caption generation by testing GPT-4o via the ChatGPT interface on IEEE Computer magazine cartoons. By removing captions and prompting AI to generate replacements, we assess its ability to produce jokes that match the depicted situation and narrative intent. Our findings show that while AI-generated captions are often humorous and contextually relevant, they sometimes diverge from the cartoon's intended meaning, for example, by missing irony, cultural references, or contextual constraints. However, AI can also produce alternatives that broaden creative exploration and occasionally improve upon the original humor. We argue that current AI systems are best used as an assistant rather than a replacement for human creativity. By integrating AI-generated suggestions, cartoonists can explore diverse humor styles, streamline ideation, and refine final captions while retaining creative control. This study highlights AI's potential as a practical tool for caption ideation within a hybrid human-AI workflow.

cs.HC↗

arXiv2025

Comparing Human and AI Performance in Visual Storytelling through Creation of Comic Strips: A Case Study

This article presents a case study comparing the capabilities of humans and artificial intelligence (AI) for visual storytelling. We developed detailed instructions to recreate a three-panel Nancy cartoon strip by Ernie Bushmiller and provided them to both humans and AI systems. The human participants were 20-something students with basic artistic training but no experience or knowledge of this comic strip. The AI systems used were popular commercial models trained to draw and paint like artists, though their training sets may not necessarily include Bushmiller's work. Results showed that AI systems excel at mimicking professional art but struggle to create coherent visual stories. In contrast, humans proved highly adept at transforming instructions into meaningful visual narratives.

cs.HC↗

arXiv2024

On The Power of Subtle Expressive Cues in the Perception of Human Affects

In this study, we introduce a sketch-based method for testing how subtle expressive cues influence the perception of affect in illustrations of human figures. We specifically study the impact of human posture and gaze direction, implicitly specified through nose orientation, on perceived emotions and mood. Through a series of user studies using sketchy illustrations of a running figure, where a professional illustrator manipulated gaze direction through adjustments on the nose orientation, we found that this simple change resulted in a diverse range of perceived affects, spanning from fear to concern and wonder. These findings shed light on the importance of fine details in defining context for context-aware system designs and underscore the importance of recognizing and expressing affect. Understanding minor expressive cues is crucial to developing emotionally intelligent systems capable of expressing affect.

cs.HC↗

arXiv2022

An Overview of Affective Speech Synthesis and Conversion in the Deep Learning Era

Speech is the fundamental mode of human communication, and its synthesis has long been a core priority in human-computer interaction research. In recent years, machines have managed to master the art of generating speech that is understandable by humans. But the linguistic content of an utterance encompasses only a part of its meaning. Affect, or expressivity, has the capacity to turn speech into a medium capable of conveying intimate thoughts, feelings, and emotions -- aspects that are essential for engaging and naturalistic interpersonal communication. While the goal of imparting expressivity to synthesised utterances has so far remained elusive, following recent advances in text-to-speech synthesis, a paradigm shift is well under way in the fields of affective speech synthesis and conversion as well. Deep learning, as the technology which underlies most of the recent advances in artificial intelligence, is spearheading these efforts. In the present overview, we outline ongoing trends and summarise state-of-the-art approaches in an attempt to provide a comprehensive overview of this exciting field.

cs.SD↗

arXiv2020

AffectON: Incorporating Affect Into Dialog Generation

Due to its expressivity, natural language is paramount for explicit and implicit affective state communication among humans. The same linguistic inquiry (e.g., How are you?) might induce responses with different affects depending on the affective state of the conversational partner(s) and the context of the conversation. Yet, most dialog systems do not consider affect as constitutive aspect of response generation. In this paper, we introduce AffectON, an approach for generating affective responses during inference. For generating language in a targeted affect, our approach leverages a probabilistic language model and an affective space. AffectON is language model agnostic, since it can work with probabilities generated by any language model (e.g., sequence-to-sequence models, neural language models, n-grams). Hence, it can be employed for both affective dialog and affective language generation. We experimented with affective dialog generation and evaluated the generated text objectively and subjectively. For the subjective part of the evaluation, we designed a custom user interface for rating and provided recommendations for the design of such interfaces. The results, both subjective and objective demonstrate that our approach is successful in pulling the generated language toward the targeted affect, with little sacrifice in syntactic coherence.

cs.CL↗

arXiv2019

Stroke-based sketched symbol reconstruction and segmentation

Hand-drawn objects usually consist of multiple semantically meaningful parts. For example, a stick figure consists of a head, a torso, and pairs of legs and arms. Efficient and accurate identification of these subparts promises to significantly improve algorithms for stylization, deformation, morphing and animation of 2D drawings. In this paper, we propose a neural network model that segments symbols into stroke-level components. Our segmentation framework has two main elements: a fixed feature extractor and a Multilayer Perceptron (MLP) network that identifies a component based on the feature. As the feature extractor we utilize an encoder of a stroke-rnn, which is our newly proposed generative Variational Auto-Encoder (VAE) model that reconstructs symbols on a stroke by stroke basis. Experiments show that a single encoder could be reused for segmenting multiple categories of sketched symbols with negligible effects on segmentation accuracies. Our segmentation scores surpass existing methodologies on an available small state of the art dataset. Moreover, extensive evaluations on our newly annotated big dataset demonstrate that our framework obtains significantly better accuracies as compared to baseline models. We release the dataset to the community.

cs.GR↗

arXiv2018

Proceedings of eNTERFACE 2015 Workshop on Intelligent Interfaces

The 11th Summer Workshop on Multimodal Interfaces eNTERFACE 2015 was hosted by the Numediart Institute of Creative Technologies of the University of Mons from August 10th to September 2015. During the four weeks, students and researchers from all over the world came together in the Numediart Institute of the University of Mons to work on eight selected projects structured around intelligent interfaces. Eight projects were selected and their reports are shown here.

cs.HC↗