Searcharxiv⌕ Search

arXiv subjects

Aaron S. Cohen

Publications and source records attributed to Aaron S. Cohen.

3 recordsLinked to original sources

Do People Trust AI Assistants Even When They Are Consistently Wrong?

People are increasingly using conversational AI systems to make decisions despite concerns around their reliability. How well do people calibrate to AI systems' accuracy in multi-turn conversations? We measure how people's reliance on LLMs' advice changes across repeated interactions, varying whether the AI assistant is helpful (100% accurate), unhelpful (0% accurate), or mixed (50% accurate). Across 4 experiments, we find people learn to trust helpful models more than unhelpful ones (67% vs 21% of the time), but only when they receive external feedback on the models' accuracy. Without feedback, people continue to be persuaded 48% of the time by a consistently unhelpful model (vs 38% for helpful). Rather than gaining trust in helpful models, participant reliance on LLM advisors tended to be high in early trials and only fell with evidence of inaccuracy. Participants trusted LLM advisors more than they trusted human advisors (83% vs 65%, Experiment 4), and LLMs benefited more from having a helpful track record than humans did. In contrast with previous work, we find limited evidence of an advantage for arguing for the truth (versus a falsehood). Truth advantages emerged only when participants were already very confident in their position, and did not emerge at all for static hints. Collectively, the results suggest that people may be swayed by LLM advice, even when it is consistently false, unless they have access to timely ground-truth feedback. Unfortunately, in many domains such feedback may not be available, leading to overreliance on AI.

cs.HC↗

Discovering Lexical Gaps Using Embeddings from Multilingual LLMs

Lexical gaps are words that do not exist in certain languages. They pose challenges for building multilingual lexical resources, for machine translation, and for cross-lingual transfer. Existing lexical gap detection relies on human judgments or fixed conceptual taxonomies. We propose a data-driven framework for identifying cross-lingual lexical gaps. We extracted contextualized embeddings from Korean-English bilingual LLMs for Korean-to-English and English-to-Korean translation pairs. Combinations of LLMs, embedding types, dimensionality, and orthogonal transformations across 100 train-test splits yielded 4000 distinct embedding spaces in each source language. In each space, we computed the semantic similarity between each source word and its nearest neighbor in the target language, and compared their distribution for gap words versus non-gap words. In 94% (Korean-to-English) and 97% (English-to-Korean) of embedding spaces, gap words showed weaker cross-lingual semantic alignment than non-gap words. Logistic classifiers trained on unaligned embedding spaces can reliably separate gap words from non-gap words, achieving AUCs of 0.81 (Korean-to-English) and 0.76 (English-to-Korean) and retrieving 18/19 Korean and 26/27 English gap words. This approach provides a language-agnostic and taxonomy-free method for scalable lexical gap identification.

cs.CL↗

Near-infrared K-band imaging of a sample of Ultra-Steep-Spectrum radio sources selected at 74 MHz

In this paper we present near-infrared K-band imaging of a sample of ultra-steep spectrum (USS) radio sources selected at 74 MHz. The dual selection criteria of low frequency and USS means that we should be sensitive to the highest redshift (z > 5) radio galaxies. We have obtained $K-$band magnitudes for all of the objects in our sample of 26 and discuss the properties of each. There is a pronounced bias in this sample towards fainter magnitudes and thus higher redshifts when compared to complete unfiltered samples such as 7CRS (Willott et al. 2002), implying that the steep-spectrum technique is still viable at 74 MHz. However, there are more bright (K < 17) sources in the 74 MHz sample than in a similar sample selected at 151 MHz, namely 6C*. This is principally due to the additional selection criterion of a small angular size for the radio sources in 6C*, four of the six sources in the 74 MHz USS sample with K < 17 have angular sizes > 15 arcsec (the angular size cutoff of 6C*). We find that the distribution of K-band magnitudes from a sample selected at 74 MHz is statistically indistinguishable from the 6C* sample, when similar angular size filtering is applied to the 74 MHz sample.

astro-ph↗