arXiv subjects

Jacek Karwowski

Publications and source records attributed to Jacek Karwowski.

14 recordsLinked to original sources

arXiv2026

Incoherence in Goal-Conditioned Autoregressive Models

We investigate mathematically the notion of incoherence: a structural issue with reinforcement learning policies derived by naive goal-conditioning of autoregressive models. We focus on the process of re-training models on their own actions, that is, fine-tuning offline-learned policies with online RL. We prove that it decreases incoherence and leads to an improvement in return, and we aim to characterize the resulting trajectory of policies. By re-framing standard notions of control-as-inference and soft Q learning, we establish a three-way correspondence with two other ways of understanding the iterative re-training process: as folding the posterior into the reward and, in the deterministic case, as decreasing the temperature parameter; the correspondence has computational content via the training-inference trade-off. Through soft-conditioning generative models, we discuss the link between incoherence and the effective horizon.

cs.LG↗

arXiv2026

Likelihood Hacking in Probabilistic Program Synthesis

When language models are trained by reinforcement learning (RL) to write probabilistic programs, they can artificially inflate their marginal-likelihood reward by producing programs whose data distribution fails to normalise instead of fitting the data better. We call this failure likelihood hacking (LH). We formalise LH in a core probabilistic programming language (PPL) and give sufficient syntactic conditions for its prevention, proving that a safe language fragment $\mathcal{L}_{\text{safe}}$ satisfying these conditions cannot produce likelihood-hacking programs. Empirically, we show that GRPO-trained models generating PyMC code discover LH exploits within the first few training steps, driving violation rates well above the untrained-model baseline. We implement $\mathcal{L}_{\text{safe}}$'s conditions as $\texttt{SafeStan}$, a LH-resistant modification of Stan, and show empirically that it suppresses LH under optimisation pressure. These results show that language-level safety constraints are both theoretically grounded and effective in practice for automated Bayesian model discovery.

cs.LG↗

arXiv2026

Geometric structures and deviations on James' symmetric positive-definite matrix bicone domain

Symmetric positive-definite (SPD) matrix datasets play a central role across numerous scientific disciplines, including signal processing, statistics, finance, computer vision, information theory, and machine learning among others. The set of SPD matrices forms a cone which can be viewed as a global coordinate chart of the underlying SPD manifold. Rich differential-geometric structures may be defined on the SPD cone manifold. Among the most widely used geometric frameworks on this manifold are the affine-invariant Riemannian structure and the dual information-geometric log-determinant barrier structure, each associated with dissimilarity measures (distance and divergence, respectively). In this work, we introduce two new structures, a Finslerian structure and a dual information-geometric structure, both derived from James' bicone reparameterization of the SPD domain. Those structures ensure that geodesics correspond to straight lines in appropriate coordinate systems. The closed bicone domain includes the spectraplex (the set of positive semi-definite diagonal matrices with unit trace) as an affine subspace, and the Hilbert VPM distance is proven to generalize the Hilbert simplex distance which found many applications in machine learning. Finally, we discuss several applications of these Finsler/dual Hessian structures and provide various inequalities between the new and traditional dissimilarities.

stat.ML↗

arXiv2025

Hilbert geometry of the symmetric positive-definite bicone: Application to the geometry of the extended Gaussian family

The extended Gaussian family is the closure of the Gaussian family obtained by completing the Gaussian family with the counterpart elements induced by degenerate covariance or degenerate precision matrices, or a mix of both degeneracies. The parameter space of the extended Gaussian family forms a symmetric positive semi-definite matrix bicone, i.e. two partial symmetric positive semi-definite matrix cones joined at their bases. In this paper, we study the Hilbert geometry of such an open bounded convex symmetric positive-definite bicone. We report the closed-form formula for the corresponding Hilbert metric distance and study exhaustively its invariance properties. We also touch upon potential applications of this geometry for dealing with extended Gaussian distributions.

cs.CG↗

arXiv2024

Characterizing stable regions in the residual stream of LLMs

We identify stable regions in the residual stream of Transformers, where the model's output remains insensitive to small activation changes, but exhibits high sensitivity at region boundaries. These regions emerge during training and become more defined as training progresses or model size increases. The regions appear to be much larger than previously studied polytopes. Our analysis suggests that these stable regions align with semantic distinctions, where similar prompts cluster within regions, and activations from the same region lead to similar next token predictions. This work provides a promising research direction for understanding the complexity of neural networks, shedding light on training dynamics, and advancing interpretability.

cs.LG↗

arXiv2024

Erfonium: A Hooke Atom with Soft Interaction Potential

Properties of erfonium, a Hooke atom with the Coulomb interaction potential $1/r$ replaced by a non-singular $\text{erf}(μr)/r$ potential are investigated. The structure of the Hooke atom potential and properties of its energy spectrum, relative to the ones of the spherical harmonic oscillator and of harmonium, are analyzed. It is shown, that at a certain value of $μ$ the system changes its behavior from a harmonium-like regime to a harmonic-oscillator-like regime.

physics.chem-ph↗

arXiv2024

Limitations of Agents Simulated by Predictive Models

There is increasing focus on adapting predictive models into agent-like systems, most notably AI assistants based on language models. We outline two structural reasons for why these models can fail when turned into agents. First, we discuss auto-suggestive delusions. Prior work has shown theoretically that models fail to imitate agents that generated the training data if the agents relied on hidden observations: the hidden observations act as confounding variables, and the models treat actions they generate as evidence for nonexistent observations. Second, we introduce and formally study a related, novel limitation: predictor-policy incoherence. When a model generates a sequence of actions, the model's implicit prediction of the policy that generated those actions can serve as a confounding variable. The result is that models choose actions as if they expect future actions to be suboptimal, causing them to be overly conservative. We show that both of those failures are fixed by including a feedback loop from the environment, that is, re-training the models on their own actions. We give simple demonstrations of both limitations using Decision Transformers and confirm that empirical results agree with our conceptual and formal analysis. Our treatment provides a unifying view of those failure modes, and informs the question of why fine-tuning offline learned policies with online learning makes them more effective.

cs.AI↗

arXiv2023

Probabilistic programming interfaces for random graphs: Markov categories, graphons, and nominal sets

We study semantic models of probabilistic programming languages over graphs, and establish a connection to graphons from graph theory and combinatorics. We show that every well-behaved equational theory for our graph probabilistic programming language corresponds to a graphon, and conversely, every graphon arises in this way. We provide three constructions for showing that every graphon arises from an equational theory. The first is an abstract construction, using Markov categories and monoidal indeterminates. The second and third are more concrete. The second is in terms of traditional measure theoretic probability, which covers 'black-and-white' graphons. The third is in terms of probability monads on the nominal sets of Gabbay and Pitts. Specifically, we use a variation of nominal sets induced by the theory of graphs, which covers Erdős-Rényi graphons. In this way, we build new models of graph probabilistic programming from graphons.

cs.PL↗

arXiv2023

Second-order adiabatic connection: The theory and application to two electrons in a parabolic confinement

The adiabatic connection formalism, usually based on the first-order perturbation theory, has been generalized to an arbitrary order. The generalization stems from the observation that the formalism can be derived from a properly arranged Taylor expansion. The second-order theory is developed in detail and applied to the description of two electrons in a parabolic confinement (harmonium). A significant improvement relative to the first-order theory has been obtained.

physics.chem-ph↗

arXiv2023

Goodhart's Law in Reinforcement Learning

Implementing a reward function that perfectly captures a complex task in the real world is impractical. As a result, it is often appropriate to think of the reward function as a proxy for the true objective rather than as its definition. We study this phenomenon through the lens of Goodhart's law, which predicts that increasing optimisation of an imperfect proxy beyond some critical point decreases performance on the true objective. First, we propose a way to quantify the magnitude of this effect and show empirically that optimising an imperfect proxy reward often leads to the behaviour predicted by Goodhart's law for a wide range of environments and reward functions. We then provide a geometric explanation for why Goodhart's law occurs in Markov decision processes. We use these theoretical insights to propose an optimal early stopping method that provably avoids the aforementioned pitfall and derive theoretical regret bounds for this method. Moreover, we derive a training method that maximises worst-case reward, for the setting where there is uncertainty about the true reward function. Finally, we evaluate our early stopping method experimentally. Our results support a foundation for a theoretically-principled study of reinforcement learning under reward misspecification.

cs.LG↗

arXiv2023

Correcting models with long-range electron interaction using generalized cusp conditions

Sources of energy errors resulting from the replacement of the physical Coulomb interaction by its long-range $\mathrm{erfc}(μr)/r$ approximation are explored. It is demonstrated that the results can be dramatically improved and the range of $μ$ giving energies within chemical accuracy limits significantly extended, if the generalized cusp conditions are used to represent the wave function at small $r$. The numerical results for two-electron harmonium are presented and discussed.

physics.chem-ph↗

arXiv2022

Two-particle coalescence conditions revisited

The notion of the n-th order local energy, generated by the n-th power of the Hamiltonian, has been introduced. The n-th order two-particle coalescence conditions have been derived from the requirements that the n-th order local energy at the coalescence point is non-singular and equal to the n-th power of the Hamiltonian eigenvalue. The first condition leads to energy-independent constraints. The second one is state-specific. The analysis has been done using a radial, one-dimensional, model Hamiltonian. The model is valid in the asymptotic region of r ~ 0. The coalescence conditions set the relations between the expansion coefficients of the radial wave function into a power series with respect to r.

physics.chem-ph↗

arXiv2021

Symmetries in Reversible Programming: From Symmetric Rig Groupoids to Reversible Programming Languages

The $\mathitΠ$ family of reversible programming languages for boolean circuits is presented as a syntax of combinators witnessing type isomorphisms of algebraic datatypes. In this paper, we give a denotational semantics for this language, using the language of weak groupoids à la Homotopy Type Theory, and show how to derive an equational theory for it, presented by 2-combinators witnessing equivalences of reversible circuits. We establish a correspondence between the syntactic groupoid of the language and a formally presented univalent subuniverse of finite types. The correspondence relates 1-combinators to 1-paths, and 2-combinators to 2-paths in the universe, which is shown to be sound and complete for both levels, establishing full abstraction and adequacy. We extend the already established Curry-Howard correspondence for $\mathitΠ$ to a Curry-Howard-Lambek correspondence between Reversible Logic, Reversible Programming Languages, and Symmetric Rig Groupoids, by showing that the syntax of $\mathitΠ$ is presented by the free symmetric rig groupoid, given by finite sets and permutations. Our proof uses techniques from the theory of group presentations and rewriting systems to solve the word problem for symmetric groups. Using the formalisation of our results, we show how to perform normalisation-by-evaluation, verification, and synthesis of reversible logic gates, motivated by examples from quantum computing.

cs.PL↗

arXiv2016

Smooth models for the Coulomb potential

Smooth model potentials with parameters selected to reproduce the spectrum of one-electron atoms are used to approximate the singular Coulomb potential. Even when the potentials do not mimic the Coulomb singularity, much of the spectrum is reproduced within the chemical accuracy. For the Hydrogen atom, the smooth approximations to the Coulomb potential are more accurate for higher angular momentum states. The transferability of the model potentials from an attractive interaction (Hydrogen atom) to a repulsive one (Harmonium and the uniform electron gas) is discussed.

physics.chem-ph↗