arXiv subjects

Damien Rudaz

Publications and source records attributed to Damien Rudaz.

4 recordsLinked to original sources

arXiv2026

(Computer) Vision in Action: Comparing Remote Sighted Assistance and a Multimodal Voice Agent in Inspection Sequences

Does human-AI assistance unfold in the same way as human-human assistance? This research explores what can be learned from the expertise of blind individuals and sighted volunteers to inform the design of multimodal voice agents and address the enduring challenge of proactivity. Drawing on granular analysis of two representative fragments from a larger corpus, we contrast the practices co-produced by an experienced human remote sighted assistant and a blind participant-as they collaborate to find a stain on a blanket over the phone-with those achieved when the same participant worked with a multimodal voice agent on the same task, a few moments earlier. This comparison enables us to specify precisely which fundamental proactive practices the agent did not enact in situ. We conclude that, so long as multimodal voice agents cannot produce environmentally occasioned vision-based actions, they will lack a key resource relied upon by human remote sighted assistants.

cs.HC↗

arXiv2025

Everything Counts: The Managed Omnirelevance of Speech in Human-Voice Agent Interaction

To this day, turn-taking models determining voice agents' conduct have been examined primarily from a technical point of view, while the ways in which they emerge as interactional constraints or resources for human conversationalists in situ remain underexplored. Drawing on a detailed analysis of corpora of naturalistic data, we document how humans' conduct was produced in reference to the ever-present risk that, each time they spoke, their talk might trigger a new uncalled-for contribution from the artificial agent. We examine this phenomenon in interactions involving rule-based robots from a 'pre-LLM era' as well as the most recent voice agents. This 'omnirelevance of human speech' (i.e., the possibility that a conversational agent may erroneously respond to any speech it detects) emerged as a constitutive feature of these human-agent encounters. We describe some of the practices through which humans managed these artificial agents' turn-taking conduct. Given recent improvements in voice capture technology, we ask whether this 'omnirelevance of human speech' weighs even more heavily on human practices today than in the past.

cs.HC↗

arXiv2025

Public speech recognition transcripts as a configuring parameter

Displaying a written transcript of what a human said (i.e. producing an "automatic speech recognition transcript") is a common feature for smartphone vocal assistants: the utterance produced by a human speaker (e.g. a question) is displayed on the screen while it is being verbally responded to by the vocal assistant. Although very rarely, this feature also exists on some "social" robots which transcribe human interactants' speech on a screen or a tablet. We argue that this informational configuration is pragmatically consequential on the interaction, both for human participants and for the embodied conversational agent. Based on a corpus of co-present interactions with a humanoid robot, we attempt to show that this transcript is a contextual feature which can heavily impact the actions ascribed by humans to the robot: that is, the way in which humans respond to the robot's behavior as constituting a specific type of action (rather than another) and as constituting an adequate response to their own previous turn.

cs.HC↗

arXiv2025

"Playing the robot's advocate": Bystanders' descriptions of a robot's conduct in public settings

Relying on a large corpus of natural interactions between visitors and a robot in a museum setting, we study a recurrent practice through which humans "worked" to maintain the robot as a competent participant: the description by bystanders, in a way that was made accessible to the main speaker, of the social action that the robot was taken to be accomplishing. Doing so, bystanders maintained the robot's (sometimes incongruous) behaviour as relevant to the activity at hand and preserved the robot itself as a competent participant. Relying on these data, we argue that ex ante definitions of a robot as "social" (i.e. before any interaction occurred) run the risk of naturalizing as self-evident the observable result from micro-sociological processes: namely, the interactional work of co-present humans through which the robot's conduct is reconfigured as contextually relevant.

cs.HC↗