Searcharxiv⌕ Search

arXiv subjects

Ivan P. Yamshchikov

Publications and source records attributed to Ivan P. Yamshchikov.

53 records · Page 3Linked to original sources
arXiv2022

Moving Other Way: Exploring Word Mover Distance Extensions

The word mover's distance (WMD) is a popular semantic similarity metric for two texts. This position paper studies several possible extensions of WMD. We experiment with the frequency of words in the corpus as a weighting factor and the geometry of the word vector space. We validate possible extensions of WMD on six document classification datasets. Some proposed extensions show better results in terms of the k-nearest neighbor classification error than WMD.

cs.CL↗
arXiv2021

StoryDB: Broad Multi-language Narrative Dataset

This paper presents StoryDB - a broad multi-language dataset of narratives. StoryDB is a corpus of texts that includes stories in 42 different languages. Every language includes 500+ stories. Some of the languages include more than 20 000 stories. Every story is indexed across languages and labeled with tags such as a genre or a topic. The corpus shows rich topical and language variation and can serve as a resource for the study of the role of narrative in natural language processing across various languages including low resource ones. We also demonstrate how the dataset could be used to benchmark three modern multilanguage models, namely, mDistillBERT, mBERT, and XLM-RoBERTa.

cs.CL↗
arXiv2021

Rethinking Crowd Sourcing for Semantic Similarity

Estimation of semantic similarity is crucial for a variety of natural language processing (NLP) tasks. In the absence of a general theory of semantic information, many papers rely on human annotators as the source of ground truth for semantic similarity estimation. This paper investigates the ambiguities inherent in crowd-sourced semantic labeling. It shows that annotators that treat semantic similarity as a binary category (two sentences are either similar or not similar and there is no middle ground) play the most important role in the labeling. The paper offers heuristics to filter out unreliable annotators and stimulates further discussions on human perception of semantic similarity.

cs.CL↗
arXiv2021

Intuitive Contrasting Map for Antonym Embeddings

This paper shows that, modern word embeddings contain information that distinguishes synonyms and antonyms despite small cosine similarities between corresponding vectors. This information is encoded in the geometry of the embeddings and could be extracted with a straight-forward and intuitive manifold learning procedure or a contrasting map. Such a map is trained on a small labeled subset of the data and can produce new embeddings that explicitly highlight specific semantic attributes of the word. The new embeddings produced by the map are shown to improve the performance on downstream tasks.

cs.CL↗
arXiv2021

DYPLODOC: Dynamic Plots for Document Classification

Narrative generation and analysis are still on the fringe of modern natural language processing yet are crucial in a variety of applications. This paper proposes a feature extraction method for plot dynamics. We present a dataset that consists of the plot descriptions for thirteen thousand TV shows alongside meta-information on their genres and dynamic plots extracted from them. We validate the proposed tool for plot dynamics extraction and discuss possible applications of this method to the tasks of narrative analysis and generation.

cs.CL↗
arXiv2021

Artificial Neural Networks Jamming on the Beat

This paper addresses the issue of long-scale correlations that is characteristic for symbolic music and is a challenge for modern generative algorithms. It suggests a very simple workaround for this challenge, namely, generation of a drum pattern that could be further used as a foundation for melody generation. The paper presents a large dataset of drum patterns alongside with corresponding melodies. It explores two possible methods for drum pattern generation. Exploring a latent space of drum patterns one could generate new drum patterns with a given music style. Finally, the paper demonstrates that a simple artificial neural network could be trained to generate melodies corresponding with these drum patters used as inputs. Resulting system could be used for end-to-end generation of symbolic music with song-like structure and higher long-scale correlations between the notes.

eess.AS↗
arXiv2020

Style-transfer and Paraphrase: Looking for a Sensible Semantic Similarity Metric

The rapid development of such natural language processing tasks as style transfer, paraphrase, and machine translation often calls for the use of semantic similarity metrics. In recent years a lot of methods to measure the semantic similarity of two short texts were developed. This paper provides a comprehensive analysis for more than a dozen of such methods. Using a new dataset of fourteen thousand sentence pairs human-labeled according to their semantic similarity, we demonstrate that none of the metrics widely used in the literature is close enough to human judgment in these tasks. A number of recently proposed metrics provide comparable results, yet Word Mover Distance is shown to be the most reasonable solution to measure semantic similarity in reformulated texts at the moment.

cs.CL↗
arXiv2020

Paranoid Transformer: Reading Narrative of Madness as Computational Approach to Creativity

This papers revisits the receptive theory in context of computational creativity. It presents a case study of a Paranoid Transformer - a fully autonomous text generation engine with raw output that could be read as the narrative of a mad digital persona without any additional human post-filtering. We describe technical details of the generative system, provide examples of output and discuss the impact of receptive theory, chance discovery and simulation of fringe mental state on the understanding of computational creativity.

cs.CL↗
arXiv2019

Decomposing Textual Information For Style Transfer

This paper focuses on latent representations that could effectively decompose different aspects of textual information. Using a framework of style transfer for texts, we propose several empirical methods to assess information decomposition quality. We validate these methods with several state-of-the-art textual style transfer methods. Higher quality of information decomposition corresponds to higher performance in terms of bilingual evaluation understudy (BLEU) between output and human-written reformulations.

cs.CL↗
arXiv2019

Style Transfer for Texts: Retrain, Report Errors, Compare with Rewrites

This paper shows that standard assessment methodology for style transfer has several significant problems. First, the standard metrics for style accuracy and semantics preservation vary significantly on different re-runs. Therefore one has to report error margins for the obtained results. Second, starting with certain values of bilingual evaluation understudy (BLEU) between input and output and accuracy of the sentiment transfer the optimization of these two standard metrics diverge from the intuitive goal of the style transfer task. Finally, due to the nature of the task itself, there is a specific dependence between these two metrics that could be easily manipulated. Under these circumstances, we suggest taking BLEU between input and human-written reformulations into consideration for benchmarks. We also propose three new architectures that outperform state of the art in terms of this metric.

cs.CL↗
arXiv2018

Music generation with variational recurrent autoencoder supported by history

A new architecture of an artificial neural network that helps to generate longer melodic patterns is introduced alongside with methods for post-generation filtering. The proposed approach called variational autoencoder supported by history is based on a recurrent highway gated network combined with a variational autoencoder. Combination of this architecture with filtering heuristics allows generating pseudo-live acoustically pleasing and melodically diverse music.

cs.SD↗
arXiv2018

Guess who? Multilingual approach for the automated generation of author-stylized poetry

This paper addresses the problem of stylized text generation in a multilingual setup. A version of a language model based on a long short-term memory (LSTM) artificial neural network with extended phonetic and semantic embeddings is used for stylized poetry generation. The quality of the resulting poems generated by the network is estimated through bilingual evaluation understudy (BLEU), a survey and a new cross-entropy based metric that is suggested for the problems of such type. The experiments show that the proposed model consistently outperforms random sample and vanilla-LSTM baselines, humans also tend to associate machine generated texts with the target author.

cs.CL↗
arXiv2018

What is wrong with style transfer for texts?

A number of recent machine learning papers work with an automated style transfer for texts and, counter to intuition, demonstrate that there is no consensus formulation of this NLP task. Different researchers propose different algorithms, datasets and target metrics to address it. This short opinion paper aims to discuss possible formalization of this NLP task in anticipation of a further growing interest to it.

cs.CL↗
arXiv2018

Elephants, Donkeys, and Colonel Blotto

This paper employs a novel method for the empirical analysis of political discourse and develops a model that demonstrates dynamics comparable with the empirical data. Applying a set of binary text classifiers based on convolutional neural networks, we label statements in the political programs of the Democratic and the Republican Party in the United States. Extending the framework of the Colonel Blotto game by a stochastic activation structure, we show that, under a simple learning rule, the simulated game exhibits dynamics that resemble the empirical data.

physics.soc-ph↗
arXiv2015

Optimization problem for a portfolio with an illiquid asset: Lie group analysis

Management of a portfolio that includes an illiquid asset is an important problem of modern mathematical finance. One of the ways to model illiquidity among others is to build an optimization problem and assume that one of the assets in a portfolio can not be sold until a certain finite, infinite or random moment of time. This approach arises a certain amount of models that are actively studied at the moment. Working in the Merton's optimal consumption framework with continuous time we consider an optimization problem for a portfolio with an illiquid, a risky and a risk-free asset. Our goal in this paper is to carry out a complete Lie group analysis of PDEs describing value function and investment and consumption strategies for a portfolio with an illiquid asset that is sold in an exogenous random moment of time with a prescribed liquidation time distribution. The problem of such type leads to three dimensional nonlinear Hamilton-Jacobi-Bellman (HJB) equations. Such equations are not only tedious for analytical methods but are also quite challenging form a numeric point of view. To reduce the three-dimensional problem to a two-dimensional one or even to an ODE one usually uses some substitutions, yet the methods used to find such substitutions are rarely discussed by the authors. We find the admitted Lie algebra for a broad class of liquidation time distributions in cases of HARA and log utility functions and formulate corresponding theorems for all these cases. We use found Lie algebras to obtain reductions of the studied equations. Several of similar substitutions were used in other papers before whereas others are new to our knowledge. This method gives us the possibility to provide a complete set of non-equivalent substitutions and reduced equations.

q-fin.MF↗
arXiv2014

Portfolio optimization in the case of an asset with a given liquidation time distribution

Management of the portfolios containing low liquidity assets is a tedious problem. The buyer proposes the price that can differ greatly from the paper value estimated by the seller, the seller, on the other hand, can not liquidate his portfolio instantly and waits for a more favorable offer. To minimize losses in this case we need to develop new methods. One of the steps moving the theory towards practical needs is to take into account the time lag of the liquidation of an illiquid asset. This task became especially significant for the practitioners in the time of the global financial crises. Working in the Merton's optimal consumption framework with continuous time we consider an optimization problem for a portfolio with an illiquid, a risky and a risk-free asset. While a standard Black-Scholes market describes the liquid part of the investment the illiquid asset is sold at a random moment with prescribed liquidation time distribution. In the moment of liquidation it generates additional liquid wealth dependent on illiquid assets paper value. The investor has the logarithmic utility function as a limit case of a HARA-type utility. Different distributions of the liquidation time of the illiquid asset are under consideration - a classical exponential distribution and Weibull distribution that is more practically relevant. Under certain conditions we show the existence of the viscosity solution in both cases. Applying numerical methods we compare classical Merton's strategies and the optimal consumption-allocation strategies for portfolios with different liquidation-time distributions of an illiquid asset.

q-fin.PM↗