Searcharxiv⌕ Search

arXiv subjects

Francesco Di Cursi

Publications and source records attributed to Francesco Di Cursi.

4 recordsLinked to original sources

Large Language Models as Digital Twins: a Distribution-Aware Questionnaire-Based Evaluation

This paper investigates whether Large Language Models can answer a personality questionnaire as a specific individual would. Given a user's social-media posts, the model completes the BFI-2-60 Big Five questionnaire as that user. The predicted responses are converted into trait scores and compared with the user's actual scores. We evaluate GPT-4, Gemma, Llama 3.1, and Mistral on the MyPersonality dataset. To determine whether predictions reflect information from the user's writing, we compare them with answers produced without user text and with values derived from general population tendencies. We measure prediction error across all users and separately for individuals with low, intermediate, and high trait scores. We also test whether an LLM can express an explicit personality profile when given only the target Big Five scores, without user text, and examine how predictions change as more text is provided. The results show that user text does not consistently improve performance. Models perform better near the middle of the trait distribution, while at the lower and upper ends they often remain close to central values or perform well at one end but poorly at the other. When given an explicit profile, GPT-4 most clearly translates trait information into consistent questionnaire responses, particularly for Extraversion and Neuroticism. Increasing context does not systematically reduce error: open-source models fluctuate more, whereas GPT-4 remains comparatively stable. Overall, the evaluated models do not consistently reproduce the questionnaire behavior of specific individuals.

cs.AI↗

Layered Ego Networks in Email Communication: From Enron to the Jmail Archive

Email archives offer a rare view of social relationships through repeated communication, but it remains unclear how well classical ego network layering applies to digital interaction data. This paper compares two public email archives with sharply contrasting structures: Enron, a workplace corpus involving around 150 users, and Jmail, a single-ego archive centered on an exceptionally active focal actor whose communication volume is more than twenty times higher than the average Enron user. We ask, in each case, whether Dunbar-like layered organization is recoverable from email communication frequency and how it should be interpreted. For Jmail, we show that extreme communication intensity causes standard layering methods (whether clustering-based or threshold-based) to break down. Jmail is not a broad communication environment with many occasional contacts, but a selective pool of high-interest alters operating on a much higher frequency scale than ordinary email. Once the Dunbar frequency ladder is anchored to the empirical support-clique boundary, a clearer layered structure emerges. Reciprocity analysis confirms that the recovered layers reflect genuine bidirectional relationships rather than artifacts of the focal actor's outgoing activity. Enron serves as a workplace benchmark that grounds the comparison: its ego networks partially reproduce Dunbar-like organization, with stable inner circles and an outermost recovered layer corresponding to Dunbar's affinity group ($\sim50$), confirming that layered structure is recoverable from ordinary organizational email. Overall, the findings show that Dunbar-like organization can be meaningfully studied in email archives, but that selective high-frequency archives require frequency normalization before the layered structure becomes interpretable.

cs.SI↗

Mind Reading or Misreading? LLMs on the Big Five Personality Test

We evaluate large language models (LLMs) for automatic personality prediction from text under the binary Five Factor Model (BIG5). Five models -- including GPT-4 and lightweight open-source alternatives -- are tested across three heterogeneous datasets (Essays, MyPersonality, Pandora) and two prompting strategies (minimal vs. enriched with linguistic and psychological cues). Enriched prompts reduce invalid outputs and improve class balance, but also introduce a systematic bias toward predicting trait presence. Performance varies substantially: Openness and Agreeableness are relatively easier to detect, while Extraversion and Neuroticism remain challenging. Although open-source models sometimes approach GPT-4 and prior benchmarks, no configuration yields consistently reliable predictions in zero-shot binary settings. Moreover, aggregate metrics such as accuracy and macro-F1 mask significant asymmetries, with per-class recall offering clearer diagnostic value. These findings show that current out-of-the-box LLMs are not yet suitable for APPT, and that careful coordination of prompt design, trait framing, and evaluation metrics is essential for interpretable results.

cs.CL↗

A Herd of Young Mastodonts: the User-Centered Footprints of Newcomers After Twitter Acquisition

The tremendous success of major Online Social Networks (OSNs) platforms has raised increasing concerns about negative phenomena, such as mass control, fake news, and echo chambers. In addition, the increasingly strict control over users' data by platform owners questions their trustworthiness as open interaction tools. These trends and, notably, the recent drastic change in X (formerly Twitter) policies and data accessibility through public APIs, have fuelled significant migration of users towards Fediverse platforms (primarily Mastodon). In this work, we provide an initial analysis of the microscopic properties of Mastodon users' social structures. Specifically, according to the Ego network model, we analyse interaction patterns between a large set of users (egos) and the other users they interact with (alters) to characterise the properties of those users' ego networks. As was observed previously in other OSNs, we found a quite regular structure compatible with the reference Dunbar's Ego Network model. Quite interestingly, our results show clear signs of ego network formation during the initial diffusion of a social networking tool, coherent with the recent surge of Mastodon activity. Therefore, our analysis motivates the use of Mastodon as an open "big data microscope" to characterise human social behaviour, making it a prime candidate to replace those OSN platforms that, unfortunately, cannot be used anymore for this purpose.

cs.SI↗