arXiv subjects

Ilia Karpov

Publications and source records attributed to Ilia Karpov.

5 recordsLinked to original sources

arXiv2026

MafiaScope: Non-Invasive, Time-Resolved Belief Probing for LLM Agents in Social Deduction Games

An LLM agent's public behaviour reveals little about its social reasoning: an agent that votes correctly may be guessing, and an agent that lies well leaves no trace of what it actually believes. We present MafiaScope, an open testbed that turns the social deduction game Mafia into a measurement instrument for machine Theory of Mind. It distinguishes whether an agent lost because it misread the game or because it failed to act on a correct assessment, a distinction that is invisible from outcomes and dialogue transcripts alone. After every public utterance, each agent privately answers structured probe questions whose responses never re-enter the game and are scored against the ground truth known to the engine. An interactive visualizer replays games from the perspective of an individual agent's beliefs, displays timeline-aligned accuracy and calibration, and supports counterfactual replay from any recorded step. In a case study across two model families comprising tens of thousands of parsed probe responses, we find that stated confidence is poorly calibrated, agents overestimate how often they are suspected by a factor of 1.5, and single-vote counterfactual replays rarely change game outcomes: outcome flips occur primarily when the agent had already formed a correct belief state, whereas decisions made under an incorrect model of the world remain largely unchanged under resampling. The engine, visualizer, recorded games, and counterfactual replay corpus are released under an open-source licence. Code: https://github.com/karpovilia/mafiascope. Live demo: https://karpovilia.github.io/mafiascope/. Screencast: https://vimeo.com/1208920221.

cs.CL↗

arXiv2024

Regional inflation analysis using social network data

Inflation is one of the most important macroeconomic indicators that have a great impact on the population of any country and region. Inflation is influenced by range of factors, one of which is inflation expectations. Many central banks take this factor into consideration while implementing monetary policy within the inflation targeting regime. Nowadays, a lot of people are active users of the Internet, especially social networks. There is a hypothesis that people search, read, and discuss mainly only those issues that are of particular interest to them. It is logical to assume that the dynamics of prices may also be in the focus of user discussions. So, such discussions could be regarded as an alternative source of more rapid information about inflation expectations. This study is based on unstructured data from Vkontakte social network to analyze upward and downward inflationary trends (on the example of the Omsk region). The sample of more than 8.5 million posts was collected between January 2010 and May 2022. The authors used BERT neural networks to solve the problem. These models demonstrated better results than the benchmarks (e.g., logistic regression, decision tree classifier, etc.). It makes possible to define pro-inflationary and disinflationary types of keywords in different contexts and get their visualization with SHAP method. This analysis provides additional operational information about inflationary processes at the regional level The proposed approach can be scaled for other regions. At the same time the limitation of the work is the time and power costs for the initial training of similar models for all regions of Russia.

q-fin.ST↗

arXiv2021

SocialBERT -- Transformers for Online SocialNetwork Language Modelling

The ubiquity of the contemporary language understanding tasks gives relevance to the development of generalized, yet highly efficient models that utilize all knowledge, provided by the data source. In this work, we present SocialBERT - the first model that uses knowledge about the author's position in the network during text analysis. We investigate possible models for learning social network information and successfully inject it into the baseline BERT model. The evaluation shows that embedding this information maintains a good generalization, with an increase in the quality of the probabilistic model for the given author up to 7.5%. The proposed model has been trained on the majority of groups for the chosen social network, and still able to work with previously unknown groups. The obtained model, as well as the code of our experiments, is available for download and use in applied tasks.

cs.CL↗

arXiv2021

Social Network Analysis of the Professional Community Interaction -- Movie Industry Case

With the rise of the competition in the movie production market, because of new players such as Netflix, Hulu, HBO Max, and Amazon Prime, whose primary goal is producing a large amount of exclusive content in order to gain a competitive advantage, it is extremely important to minimize the number of unsuccessful titles. This paper focuses on new approaches to predict film success, based on the movie industry community structure, and highlights the role of the casting director in movie success. Based on publicly available data we create an "actor"-"casting director"-"talent agent"-"director" communication graph and show that usage of additional knowledge leads to better movie rating prediction.

cs.SI↗

arXiv2020

Detecting Automatically Managed Accounts in Online Social Networks: Graph Embedding Approach

The widespread of Online Social Networks and the opportunity to commercialize popular accounts have attracted a large number of automated programs, known as artificial accounts. This paper focuses on the classification of human and fake accounts on the social network, by employing several graph neural networks, to efficiently encode attributes and network graph features of the account. Our work uses both network structure and attributes to distinguish human and artificial accounts and compares attributed and traditional graph embeddings. Separating complex, human-like artificial accounts into a standalone task demonstrates significant limitations of profile-based algorithms for bot detection and shows the efficiency of network structure-based methods for detecting sophisticated bot accounts. Experiments show that our approach can achieve competitive performance compared with existing state-of-the-art bot detection systems with only network-driven features. The source code of this paper is available at: http://github.com/karpovilia/botdetection.

cs.SI↗