arXiv subjects

Przemysław Rola

Publications and source records attributed to Przemysław Rola.

7 recordsLinked to original sources

arXiv2026

An equality condition for the Dobrushin bound on attention rollout and how often it holds in trained transformers

The Dobrushin coefficient of each attention-rollout factor satisfies $κ(\frac12(I+A))\le\frac12(1+κ(A))$, and multiplying these inequalities over layers bounds the coefficient of the whole rollout. We characterise exactly when the layerwise bound is tight: equality holds if and only if some token pair attaining $κ(A)$ is mutually self-dominant - each of the two attends to itself at least as strongly as the other attends to it. The condition is far from automatic: uniformly random stochastic matrices satisfy it only 24-30% of the time. When tested on the head-averaged attention of each individual input and restricted to content tokens - image patches, words or tabular features, excluding cls, register and separator tokens - the condition holds for essentially every input at every layer of DINOv2 (three model sizes), RoBERTa and DistilBERT. In the supervised models DeiT-B and ViT-B/16 it holds for 91% and 64% of input-layer pairs respectively, with all failures occurring late in depth. In FT-Transformer trained on two standard tabular benchmarks it holds for only 11-44% of input-layer pairs. The special tokens account for almost all failures in DINOv2 and the language models: when they are included, the condition holds for only 82-97% of input-layer pairs.

cs.LG↗

arXiv2026

EntroPath: Maximum Entropy Path Ensemble Embedding for Manifold Learning

We introduce EntroPath, a manifold learning method that recovers geodesic geometry from data graphs through ensembles of diffusion paths. Many existing graph-based embeddings rely either on locally normalised random walks or on shortest-path distances. The former can concentrate diffusion in densely sampled regions, while the latter are sensitive to spurious shortcut edges in the graph. EntroPath instead builds its dissimilarities from the maximum entropy random walk (MERW), which aggregates the full ensemble of k-step paths between points rather than relying on any single trajectory. We show that the resulting free-energy dissimilarity converges to squared geodesic distance in the short-time limit, via Varadhan's heat-kernel formula. The diffusion depth k interpolates smoothly between local neighbourhood structure and global manifold geometry, and the symmetrised kernel admits an exact Gram factorisation connecting EntroPath to kernel methods. We further provide scalable extensions via landmark projection and diffusion-potential pseudotime. Across synthetic manifolds and single-cell benchmarks, EntroPath consistently matches or outperforms diffusion- and shortest-path-based methods, while remaining competitive with neighbourhood-preserving embeddings (UMAP, t-SNE) on local-structure metrics. Its gains are most pronounced on manifolds with non-uniform sampling density and well-separated branching trajectories, where path-ensemble diffusion more faithfully preserves the underlying geodesic geometry.

cs.LG↗

arXiv2025

Boltzmann Price: Toward Understanding the Fair Price in High-Frequency Markets

In this paper, we introduce a parametrized family of prices derived from the Maximum Entropy Principle. The price is obtained from the distribution that minimizes bias, given the bid and ask volume imbalance at the top of the order book. Under specific parameter choices, it closely approximates the mid-price or the weighted mid-price. Using probabilities of bid and ask states, we propose a model of price dynamics in which both drift and volatility are driven by volume imbalance. Compared to standard models like Bachelier or Geometric Brownian Motion with constant volatility, our model can generate higher kurtosis and heavy-tailed distributions. Additionally, the drift term naturally emerges as a consequence of the order book imbalance. We validate the model through simulation and demonstrate its fit to historical equity data. The model provides a theoretical framework, integrating price, volume imbalance, and spread.

q-fin.TR↗

arXiv2023

Contrastive Hierarchical Clustering

Deep clustering has been dominated by flat models, which split a dataset into a predefined number of groups. Although recent methods achieve an extremely high similarity with the ground truth on popular benchmarks, the information contained in the flat partition is limited. In this paper, we introduce CoHiClust, a Contrastive Hierarchical Clustering model based on deep neural networks, which can be applied to typical image data. By employing a self-supervised learning approach, CoHiClust distills the base network into a binary tree without access to any labeled data. The hierarchical clustering structure can be used to analyze the relationship between clusters, as well as to measure the similarity between data points. Experiments demonstrate that CoHiClust generates a reasonable structure of clusters, which is consistent with our intuition and image semantics. Moreover, it obtains superior clustering accuracy on most of the image datasets compared to the state-of-the-art flat clustering models.

cs.LG↗

arXiv2022

Prediction of motor insurance claims occurrence as an imbalanced machine learning problem

The insurance industry, with its large datasets, is a natural place to use big data solutions. However it must be stressed, that significant number of applications for machine learning in insurance industry, like fraud detection or claim prediction, deals with the problem of machine learning on an imbalanced data set. This is due to the fact that frauds or claims are rare events when compared with the entire population of drivers. The problem of imbalanced learning is often hard to overcome. Therefore, the main goal of this work is to present and apply various methods of dealing with an imbalanced dataset in the context of claim occurrence prediction in car insurance. In addition, the above techniques are used to compare the results of machine learning algorithms in the context of claim occurrence prediction in car insurance. Our study covers the following techniques: logistic-regression, decision tree, random forest, xgBoost, feed-forward network. The problem is the classification one.

q-fin.ST↗

arXiv2017

ICA based on the data asymmetry

Independent Component Analysis (ICA) - one of the basic tools in data analysis - aims to find a coordinate system in which the components of the data are independent. Most of existing methods are based on the minimization of the function of fourth-order moment (kurtosis). Skewness (third-order moment) has received much less attention. In this paper we present a competitive approach to ICA based on the Split Gaussian distribution, which is well adapted to asymmetric data. Consequently, we obtain a method which works better than the classical approaches, especially in the case when the underlying density is not symmetric, which is a typical situation in the color distribution in images.

math.ST↗

arXiv2014

Arbitrage in markets with bid-ask spreads

In this paper a finite discrete time market with an arbitrary state space and bid-ask spreads is considered. The notion of an equivalent bid-ask martingale measure (EBAMM) is introduced and the fundamental theorem of asset pricing is proved using (EBAMM) as an equivalent condition for no-arbitrage. The Cox-Ross-Rubinstein model with bid-ask spreads is presented as an application of our results.

q-fin.PR↗