arXiv subjects

Alexander Mozeika

Publications and source records attributed to Alexander Mozeika.

At least 19 recordsLinked to original sources

arXiv2023

Replica analysis of overfitting in regression models for time to event data: the impact of censoring

We use statistical mechanics techniques, viz. the replica method, to model the effect of censoring on overfitting in Cox's proportional hazards model, the dominant regression method for time-to-event data. In the overfitting regime, Maximum Likelihood parameter estimators are known to be biased already for small values of the ratio of the number of covariates over the number of samples. The inclusion of censoring was avoided in previous overfitting analyses for mathematical convenience, but is vital to make any theory applicable to real-world medical data, where censoring is ubiquitous. Upon constructing efficient algorithms for solving the new (and more complex) RS equations and comparing the solutions with numerical simulation data, we find excellent agreement, even for large censoring rates. We then address the practical problem of using the theory to correct the biased ML estimators {without} knowledge of the data-generating distribution. This is achieved via a novel numerical algorithm that self-consistently approximates all relevant parameters of the data generating distribution while simultaneously solving the RS equations. We investigate numerically the statistics of the corrected estimators, and show that the proposed new algorithm indeed succeeds in removing the bias of the ML estimators, for both the association parameters and for the cumulative hazard.

stat.ME↗

arXiv2023

Carnot: A highly Scalable and Responsive BFT Consensus protocol

We present Carnot, a leader-based Byzantine Fault Tolerant (BFT) consensus protocol that is responsive and operates under the partially synchronous model. Responsive BFT consensus protocols exhibit wire-speed operation and deliver instantaneous finality, thereby addressing a fundamental need in distributed systems. A key challenge in scaling these protocols has been the computational complexity associated with authenticator verification. We demonstrate that Carnot effectively addresses this bottleneck by adeptly streamlining the verification and aggregation of $O(log(N))$ authenticators per node. This notable advancement marks a substantial improvement over the prevailing $O(N)$ state-of-the-art approaches. Leveraging this inherent property, Carnot demonstrates its capacity to seamlessly scale to networks comprising tens to hundreds of thousands of nodes. We envision Carnot as a critical stride towards bridging the gap between classical BFT consensus mechanisms and blockchain technology.

cs.DC↗

arXiv2023

How to generate a fault-resilient network at a lower cost

Blockchains facilitate decentralization, security, identity, and data management in cyber-physical systems. However, consensus protocols used in blockchains are prone to high message and computational complexity costs and are not suitable to be used in IoT. One way to reduce message complexity is to randomly assign network nodes into committees or shards. Keeping committee sizes small is then desirable in order to achieve lower message complexity, but this comes with a penalty of reduced reliability as there is a higher probability that a large number of faulty nodes will end up in a committee. In this work, we study the problem of estimating a probability of a failure in randomly sharded networks. We provide new results and improve existing bounds on the failure probability. Thus, our framework also paves the way to reduce committee sizes without reducing reliability.

cs.DC↗

arXiv2022

Dynamics of sparse Boolean networks with multi-node and self-interactions

We analyse the equilibrium behaviour and non-equilibrium dynamics of sparse Boolean networks with self-interactions that evolve according to synchronous Glauber dynamics. Equilibrium analysis is achieved via a novel application of the cavity method to the temperature-dependent pseudo-Hamiltonian that characterises the equilibrium state of systems with parallel dynamics. Similarly, the non-equilibrium dynamics can be analysed by using the dynamical version of the cavity method. It is well known, however, that when self-interactions are present, direct application of the dynamical cavity method is cumbersome, due to the presence of strong memory effects, which prevent explicit analysis of the dynamics beyond a few time steps. To overcome this difficulty, we show that it is possible to map a system of $N$ variables to an equivalent bipartite system of $2N$ variables, for which the dynamical cavity method can be used under the usual one time approximation scheme. This substantial technical advancement allows for the study of transient and long-time behaviour of systems with self-interactions. Finally, we study the dynamics of systems with multi-node interactions, recently used to model gene regulatory networks, by mapping this to a bipartite system of Boolean variables with 2-body interactions. We show that when interactions have a degree of bidirectionality such systems are able to support a multiplicity of diverse attractors, an important requirement for a gene-regulatory network to sustain multi-cellular life.

cond-mat.dis-nn↗

arXiv2021

Vaccination with partial transmission and social distancing on contact networks

We study the impact of vaccination on the risk of epidemics spreading through structured networks using the cavity method of statistical physics. We relax the assumption that vaccination prevents all transmission of a disease used in previous studies, such that vaccinated nodes have a small probability of transmission. To do so we extend the cavity method to study networks where nodes have heterogeneous transmissibility. We find that vaccination with partial transmission still provides herd immunity and show how the herd immunity threshold depends upon the assortativity between nodes of different transmissibility. In addition, we study the impact of social distancing via bond percolation and show that percolation targeting links between nodes of high transmissibility can reduce the risk of an epidemic greater than targeting links between nodes of high degree. Finally, we extend recent methods to compute the distributional equations of risk in populations with heterogeneous transmissibility and show how targeted social distancing measures may reduce overall risk greater than untargeted vaccination campaigns, by comparing the effect of random and targeted strategies of node and link deletion on the risk distribution.

physics.soc-ph↗

arXiv2021

Exact results on high-dimensional linear regression via statistical physics

It is clear that conventional statistical inference protocols need to be revised to deal correctly with the high-dimensional data that are now common. Most recent studies aimed at achieving this revision rely on powerful approximation techniques, that call for rigorous results against which they can be tested. In this context, the simplest case of high-dimensional linear regression has acquired significant new relevance and attention. In this paper we use the statistical physics perspective on inference to derive a number of new exact results for linear regression in the high-dimensional regime.

math.ST↗

arXiv2020

Space of Functions Computed by Deep-Layered Machines

We study the space of functions computed by random-layered machines, including deep neural networks and Boolean circuits. Investigating the distribution of Boolean functions computed on the recurrent and layer-dependent architectures, we find that it is the same in both models. Depending on the initial conditions and computing elements used, we characterize the space of functions computed at the large depth limit and show that the macroscopic entropy of Boolean functions is either monotonically increasing or decreasing with the growing depth.

cs.LG↗

arXiv2019

Replica analysis of Bayesian data clustering

We use statistical mechanics to study model-based Bayesian data clustering. In this approach, each partition of the data into clusters is regarded as a microscopic system state, the negative data log-likelihood gives the energy of each state, and the data set realisation acts as disorder. Optimal clustering corresponds to the ground state of the system, and is hence obtained from the free energy via a low `temperature' limit. We assume that for large sample sizes the free energy density is self-averaging, and we use the replica method to compute the asymptotic free energy density. The main order parameter in the resulting (replica symmetric) theory, the distribution of the data over the clusters, satisfies a self-consistent equation which can be solved by a population dynamics algorithm. From this order parameter one computes the average free energy, and all relevant macroscopic characteristics of the problem. The theory describes numerical experiments perfectly, and gives a significant improvement over the mean-field theory that was used to study this model in past.

cond-mat.dis-nn↗

arXiv2019

Roles of repertoire diversity in robustness of humoral immune response

The adaptive immune system relies on diversity of its repertoire of receptors to protect the organism from a great variety of pathogens. Since the initial repertoire is the result of random gene rearrangement, binding of receptors is not limited to pathogen-associated antigens but also includes self antigens. There is a fine balance between having a diverse repertoire, protecting from many different pathogens, and yet reducing its self-reactivity as far as possible to avoid damage to self. In the ageing immune system this balance is altered, manifesting in reduced specificity of response to pathogens or vaccination on a background of higher self-reactivity. To answer the question whether age-related changes of repertoire in the diversity and self/non-self affinity balance of antibodies could explain the reduced efficacy of the humoral response in older people, we construct a minimal mathematical model of the humoral immune response. The principle of least damage allows us, for a given repertoire of antibodies, to resolve a tension between the necessity to neutralise target antigens as quickly as possible and the requirement to limit the damage to self antigens leading to an optimal dynamics of immune response. The model predicts slowing down of immune response for repertoires with reduced diversity and increased self-reactivity.

q-bio.CB↗

arXiv2018

Mean-field theory of Bayesian clustering

We show that model-based Bayesian clustering, the probabilistically most systematic approach to the partitioning of data, can be mapped into a statistical physics problem for a gas of particles, and as a result becomes amenable to a detailed quantitative analysis. A central role in the resulting statistical physics framework is played by an entropy function. We demonstrate that there is a relevant parameter regime where mean-field analysis of this function is exact, and that, under natural assumptions, the lowest entropy state of the hypothetical gas corresponds to the optimal clustering of data. The byproduct of our analysis is a simple but effective clustering algorithm, which infers both the most plausible number of clusters in the data and the corresponding partitions. Describing Bayesian clustering in statistical mechanical terms is found to be natural and surprisingly effective.

cond-mat.dis-nn↗

arXiv2016

Statistical mechanics of clonal expansion in lymphocyte networks modelled with slow and fast variables

We study the Langevin dynamics of the adaptive immune system, modelled by a lymphocyte network in which the B cells are interacting with the T cells and antigen. We assume that B clones and T clones are evolving in different thermal noise environments and on different timescales. We derive stationary distributions and use statistical mechanics to study clonal expansion of B clones in this model when the B and T clone sizes are assumed to be the slow and fast variables respectively and vice versa. We derive distributions of B clone sizes and use general properties of ferromagnetic systems to predict characteristics of these distributions, such as the average B cell concentration, in some regimes where T cells can be modelled as binary variables. This analysis is independent of network topologies and its results are qualitatively consistent with experimental observations. In order to obtain full distributions we assume that the network topologies are random and locally equivalent to trees. The latter allows us to employ the Bethe-Peierls approach and to develop a theoretical framework which can be used to predict the distributions of B clone sizes. As an example we use this theory to compute distributions for the models of immune system defined on random regular networks.

cond-mat.dis-nn↗

arXiv2016

The role of idiotypic interactions in the adaptive immune system: a belief-propagation approach

In this work we use belief-propagation techniques to study the equilibrium behaviour of a minimal model for the immune system comprising interacting T and B clones. We investigate the effect of the so-called idiotypic interactions among complementary B clones on the system's activation. Our result shows that B-B interactions increase the system's resilience to noise, making clonal activation more stable, while increasing the cross-talk between different clones. We derive analytically the noise level at which a B clone gets activated, in the absence of cross-talk, and find that this increases with the strength of idiotypic interactions and with the number of T cells signalling the B clone. We also derive, analytically and numerically, via population dynamics, the critical line where clonal cross-talk arises. Our approach allows us to derive the B clone size distribution, which can be experimentally measured and gives important information about the adaptive immune system response to antigens and vaccination.

cond-mat.dis-nn↗

arXiv2015

Spin systems on hypercubic Bethe lattices: A Bethe-Peierls approach

We study spin systems on Bethe lattices constructed from d-dimensional hypercubes. Although these lattices are not tree-like, and therefore closer to real cubic lattices than Bethe lattices or regular random graphs, one can still use the Bethe-Peierls method to derive exact equations for the magnetization and other thermodynamic quantities. We compute phase diagrams for ferromagnetic Ising models on hypercubic Bethe lattices with dimension d=2, 3, and 4. Our results are in good agreement with the results of the same models on d-dimensional cubic lattices, for low and high temperatures, and offer an improvement over the conventional Bethe lattice with connectivity k=2d.

cond-mat.dis-nn↗

arXiv2014

On reliable computation by noisy random Boolean formulas

We study noisy computation in randomly generated k-ary Boolean formulas. We establish bounds on the noise level above which the results of computation by random formulas are not reliable. This bound is saturated by formulas constructed from a single majority-like gates. We show that these gates can be used to compute any Boolean function reliably below the noise bound.

cond-mat.dis-nn↗

arXiv2014

Consistent inference of a general model using the pseudo-likelihood method

Recently maximum pseudo-likelihood (MPL) inference method has been successfully applied to statistical physics models with intractable likelihoods. We use information theory to derive a relation between the pseudo-likelihood and likelihood functions. We use this relation to show consistency of the pseudo-likelihood method for a general model.

cond-mat.dis-nn↗

arXiv2013

Transfer matrix analysis of one-dimensional majority cellular automata with thermal noise

Thermal noise in a cellular automaton refers to a random perturbation to its function which eventually leads this automaton to an equilibrium state controlled by a temperature parameter. We study the 1-dimensional majority-3 cellular automaton under this model of noise. Without noise, each cell in this automaton decides its next state by majority voting among itself and its left and right neighbour cells. Transfer matrix analysis shows that the automaton always reaches a state in which every cell is in one of its two states with probability 1/2 and thus cannot remember even one bit of information. Numerical experiments, however, support the possibility of reliable computation for a long but finite time.

cond-mat.stat-mech↗

arXiv2013

Projected generalized free energies for non-equilibrium states

We develop a systematic procedure to approximate generalized free energy in out of equilibrium stochastic systems. The procedure only requires knowledge of the averages of macroscopic observables and uses quasi-equilibrium distribution to this task. As an application we consider model systems in the regime of diverging relaxation times. We find that geometry of the approximate generalized free energy changes at the onset of this phenomena.

cond-mat.dis-nn↗

arXiv2012

Islands of equilibrium in a dynamical world

Many natural, technological and social systems are inherently not in equilibrium. We show, by detailed analysis of exemplar models, the emergence of equilibrium-like behavior in localized or nonlocalized domains within non-equilibrium systems as conjectured in some real systems. Equilibrium domains are shown to emerge either abruptly or gradually depending on the system parameters and disappear, becoming indistinguishable from the remainder of the system for other parameter values. The models studied, defined on densely and sparsely connected networks, provide a useful representation of many real systems.

cond-mat.dis-nn↗