arXiv subjects

Alexander N. Gorban

Publications and source records attributed to Alexander N. Gorban.

At least 55 records · Page 3Linked to original sources

arXiv2019

Mobility cost and degenerated diffusion in kinesis models

A new critical effect is predicted in population dispersal. It is based on the fact that a trade-off between the advantages of mobility and the cost of mobility breaks with a significant deterioration in living conditions. The recently developed model of purposeful kinesis (Gorban \& Çabukoǧlu, Ecological Complexity 33, 2018) is based on the "let well enough alone" idea: mobility decreases for high reproduction coefficient and, therefore, animals stay longer in good conditions and leave quicker bad conditions. Mobility has a cost, which should be measured in the changes of the reproduction coefficient. Introduction of the cost of mobility into the reproduction coefficient leads to an equation for mobility. It can be solved in a closed form using Lambert $W$-function. Surprisingly, the "let well enough alone" models with the simple linear cost of mobility have an intrinsic phase transition: when conditions worsen then the mobility increases up to some critical value of the reproduction coefficient. For worse conditions, there is no solution for mobility. We interpret this critical effect as the complete loss of mobility that is degeneration of diffusion. Qualitatively, this means that mobility increases with worsening of conditions up to some limit, and after that, mobility is nullified.

q-bio.PE↗

arXiv2019

Fast Construction of Correcting Ensembles for Legacy Artificial Intelligence Systems: Algorithms and a Case Study

This paper presents a technology for simple and computationally efficient improvements of a generic Artificial Intelligence (AI) system, including Multilayer and Deep Learning neural networks. The improvements are, in essence, small network ensembles constructed on top of the existing AI architectures. Theoretical foundations of the technology are based on Stochastic Separation Theorems and the ideas of the concentration of measure. We show that, subject to mild technical assumptions on statistical properties of internal signals in the original AI system, the technology enables instantaneous and computationally efficient removal of spurious and systematic errors with probability close to one on the datasets which are exponentially large in dimension. The method is illustrated with numerical examples and a case study of ten digits recognition from American Sign Language.

cs.AI↗

arXiv2019

One-Trial Correction of Legacy AI Systems and Stochastic Separation Theorems

We consider the problem of efficient "on the fly" tuning of existing, or {\it legacy}, Artificial Intelligence (AI) systems. The legacy AI systems are allowed to be of arbitrary class, albeit the data they are using for computing interim or final decision responses should posses an underlying structure of a high-dimensional topological real vector space. The tuning method that we propose enables dealing with errors without the need to re-train the system. Instead of re-training a simple cascade of perceptron nodes is added to the legacy system. The added cascade modulates the AI legacy system's decisions. If applied repeatedly, the process results in a network of modulating rules "dressing up" and improving performance of existing AI systems. Mathematical rationale behind the method is based on the fundamental property of measure concentration in high dimensional spaces. The method is illustrated with an example of fine-tuning a deep convolutional network that has been pre-trained to detect pedestrians in images.

stat.ML↗

arXiv2019

Multivariate Gaussian and Student$-t$ Process Regression for Multi-output Prediction

Gaussian process model for vector-valued function has been shown to be useful for multi-output prediction. The existing method for this model is to re-formulate the matrix-variate Gaussian distribution as a multivariate normal distribution. Although it is effective in many cases, re-formulation is not always workable and is difficult to apply to other distributions because not all matrix-variate distributions can be transformed to respective multivariate distributions, such as the case for matrix-variate Student$-t$ distribution. In this paper, we propose a unified framework which is used not only to introduce a novel multivariate Student$-t$ process regression model (MV-TPR) for multi-output prediction, but also to reformulate the multivariate Gaussian process regression (MV-GPR) that overcomes some limitations of the existing methods. Both MV-GPR and MV-TPR have closed-form expressions for the marginal likelihoods and predictive distributions under this unified framework and thus can adopt the same optimization approaches as used in the conventional GPR. The usefulness of the proposed methods is illustrated through several simulated and real data examples. In particular, we verify empirically that MV-TPR has superiority for the datasets considered, including air quality prediction and bike rent prediction. At last, the proposed methods are shown to produce profitable investment strategies in the stock markets.

stat.ML↗

arXiv2018

Robust And Scalable Learning Of Complex Dataset Topologies Via Elpigraph

Large datasets represented by multidimensional data point clouds often possess non-trivial distributions with branching trajectories and excluded regions, with the recent single-cell transcriptomic studies of developing embryo being notable examples. Reducing the complexity and producing compact and interpretable representations of such data remains a challenging task. Most of the existing computational methods are based on exploring the local data point neighbourhood relations, a step that can perform poorly in the case of multidimensional and noisy data. Here we present ElPiGraph, a scalable and robust method for approximation of datasets with complex structures which does not require computing the complete data distance matrix or the data point neighbourhood graph. This method is able to withstand high levels of noise and is capable of approximating complex topologies via principal graph ensembles that can be combined into a consensus principal graph. ElPiGraph deals efficiently with large and complex datasets in various fields from biology, where it can be used to infer gene dynamics from single-cell RNA-Seq, to astronomy, where it can be used to explore complex structures in the distribution of galaxies.

cs.LG↗

arXiv2018

Augmented Artificial Intelligence: a Conceptual Framework

All artificial Intelligence (AI) systems make errors. These errors are unexpected, and differ often from the typical human mistakes ("non-human" errors). The AI errors should be corrected without damage of existing skills and, hopefully, avoiding direct human expertise. This paper presents an initial summary report of project taking new and systematic approach to improving the intellectual effectiveness of the individual AI by communities of AIs. We combine some ideas of learning in heterogeneous multiagent systems with new and original mathematical approaches for non-iterative corrections of errors of legacy AI systems. The mathematical foundations of AI non-destructive correction are presented and a series of new stochastic separation theorems is proven. These theorems provide a new instrument for the development, analysis, and assessment of machine learning methods and algorithms in high dimension. They demonstrate that in high dimensions and even for exponentially large samples, linear classifiers in their classical Fisher's form are powerful enough to separate errors from correct responses with high probability and to provide efficient solution to the non-destructive corrector problem. In particular, we prove some hypotheses formulated in our paper `Stochastic Separation Theorems' (Neural Networks, 94, 255--259, 2017), and answer one general problem published by Donoho and Tanner in 2009.

cs.AI↗

arXiv2018

Hilbert's Sixth Problem: the endless road to rigour

Introduction to the special issue of Phil. Trans. R. Soc. A 376, 2018, `Hilbert's Sixth Problem'. The essence of the Sixth Problem is discussed and the content of this issue is introduced. In 1900, David Hilbert presented 23 problems for the advancement of mathematical science. Hilbert's Sixth Problem proposed the expansion of the axiomatic method outside of mathematics, in physics and beyond. Its title was shocking: "Mathematical Treatment of the Axioms of Physics." Axioms of physics did not exist and were not expected. During further explanation, Hilbert specified this problem with special focus on probability and "the limiting processes, ... which lead from the atomistic view to the laws of motion of continua". The programmatic call was formulated "to treat, by means of axioms, those physical sciences in which already today mathematics plays an important part." This issue presents a modern slice of the work on the Sixth Problem, from quantum probability to fluid dynamics and machine learning, and from review of solid mathematical and physical results to opinion pieces with new ambitious ideas. Some expectations were broken: The continuum limit of atomistic kinetics may differ from the classical fluid dynamics. The "curse of dimensionality" in machine learning turns into the "blessing of dimensionality" that is closely related to statistical physics. Quantum probability facilitates the modelling of geological uncertainty and hydrocarbon reservoirs. And many other findings are presented.

physics.hist-ph↗

arXiv2018

High-dimensional brain. A tool for encoding and rapid learning of memories by single neurons

Codifying memories is one of the fundamental problems of modern Neuroscience. The functional mechanisms behind this phenomenon remain largely unknown. Experimental evidence suggests that some of the memory functions are performed by stratified brain structures such as, e.g., the hippocampus. In this particular case, single neurons in the CA1 region receive a highly multidimensional input from the CA3 area, which is a hub for information processing. We thus assess the implication of the abundance of neuronal signalling routes converging onto single cells on the information processing. We show that single neurons can selectively detect and learn arbitrary information items, given that they operate in high dimensions. The argument is based on Stochastic Separation Theorems and the concentration of measure phenomena. We demonstrate that a simple enough functional neuronal model is capable of explaining: i) the extreme selectivity of single neurons to the information content, ii) simultaneous separation of several uncorrelated stimuli or informational items from a large set, and iii) dynamic learning of new items by associating them with already "known" ones. These results constitute a basis for organization of complex memories in ensembles of single neurons. Moreover, they show that no a priori assumptions on the structural organization of neuronal ensembles are necessary for explaining basic concepts of static and dynamic memories.

q-bio.NC↗

arXiv2017

Knowledge Transfer Between Artificial Intelligence Systems

We consider the fundamental question: how a legacy "student" Artificial Intelligent (AI) system could learn from a legacy "teacher" AI system or a human expert without complete re-training and, most importantly, without requiring significant computational resources. Here "learning" is understood as an ability of one system to mimic responses of the other and vice-versa. We call such learning an Artificial Intelligence knowledge transfer. We show that if internal variables of the "student" Artificial Intelligent system have the structure of an $n$-dimensional topological vector space and $n$ is sufficiently high then, with probability close to one, the required knowledge transfer can be implemented by simple cascades of linear functionals. In particular, for $n$ sufficiently large, with probability close to one, the "student" system can successfully and non-iteratively learn $k\ll n$ new examples from the "teacher" (or correct the same number of mistakes) at the cost of two additional inner products. The concept is illustrated with an example of knowledge transfer from a pre-trained convolutional neural network to a simple linear classifier with HOG features.

cs.AI↗

arXiv2017

Pseudo-Outcrop Visualization of Borehole Images and Core Scans

A pseudo-outcrop visualization is demonstrated for borehole and full-diameter rock core images to augment the ubiquitous unwrapped cylinder view and thereby to assist non-specialist interpreters. The pseudo-outcrop visualization is equivalent to a nonlinear projection of the image from borehole to earth frame of reference that creates a solid volume sliced longitudinally to reveal two or more faces in which the orientations of geological features indicate what is observed in the subsurface. A proxy for grain size is used to modulate the external dimensions of the plot to mimic profiles seen in real outcrops. The volume is created from a mixture of geological boundary elements and texture, the latter being the residue after the sum of boundary elements is subtracted from the original data. In the case of measurements from wireline microresistivity tools, whose circumferential coverage is substantially less than 100%, the missing circumferential data is first inpainted using multiscale directional transforms, which decompose the image into its elemental building structures, before reconstructing the full image. The pseudo-outcrop view enables direct observation of the angular relationships between features and aids visual comparison between borehole and core images, especially for the interested non-specialist.

physics.geo-ph↗

arXiv2015

Approximation with Random Bases: Pro et Contra

In this work we discuss the problem of selecting suitable approximators from families of parameterized elementary functions that are known to be dense in a Hilbert space of functions. We consider and analyze published procedures, both randomized and deterministic, for selecting elements from these families that have been shown to ensure the rate of convergence in $L_2$ norm of order $O(1/N)$, where $N$ is the number of elements. We show that both randomized and deterministic procedures are successful if additional information about the families of functions to be approximated is provided. In the absence of such additional information one may observe exponential growth of the number of terms needed to approximate the function and/or extreme sensitivity of the outcome of the approximation to parameters. Implications of our analysis for applications of neural networks in modeling and control are illustrated with examples.

math.NA↗

arXiv2015

Leaders do not look back, or do they?

We study the effect of adding to a directed chain of interconnected systems a directed feedback from the last element in the chain to the first. The problem is closely related to the fundamental question of how a change in network topology may influence the behavior of coupled systems. We begin the analysis by investigating a simple linear system. The matrix that specifies the system dynamics is the transpose of the network Laplacian matrix, which codes the connectivity of the network. Our analysis shows that for any nonzero complex eigenvalue $λ$ of this matrix, the following inequality holds: $\frac{|\Im λ|}{|\Re λ|} \leq \cot\fracπ{n}$. This bound is sharp, as it becomes an equality for an eigenvalue of a simple directed cycle with uniform interaction weights. The latter has the slowest decay of oscillations among all other network configurations with the same number of states. The result is generalized to directed rings and chains of identical nonlinear oscillators. For directed rings, a lower bound $σ_c$ for the connection strengths that guarantees asymptotic synchronization is found to follow a similar pattern: $σ_c=\frac{1}{1-\cos\left( 2π/n\right)} $. Numerical analysis revealed that, depending on the network size $n$, multiple dynamic regimes co-exist in the state space of the system. In addition to the fully synchronous state a rotating wave solution occurs. The effect is observed in networks exceeding a certain critical size. The emergence of a rotating wave highlights the importance of long chains and loops in networks of oscillators: the larger the size of chains and loops, the more sensitive the network dynamics becomes to removal or addition of a single connection.

math.DS↗

arXiv2015

Long and short range multi-locus QTL interactions in a complex trait of yeast

We analyse interactions of Quantitative Trait Loci (QTL) in heat selected yeast by comparing them to an unselected pool of random individuals. Here we re-examine data on individual F12 progeny selected for heat tolerance, which have been genotyped at 25 locations identified by sequencing a selected pool [Parts, L., Cubillos, F. A., Warringer, J., Jain, K., Salinas, F., Bumpstead, S. J., Molin, M., Zia, A., Simpson, J. T., Quail, M. A., Moses, A., Louis, E. J., Durbin, R., and Liti, G. (2011). Genome research, 21(7), 1131-1138]. 960 individuals were genotyped at these locations and multi-locus genotype frequencies were compared to 172 sequenced individuals from the original unselected pool (a control group). Various non-random associations were found across the genome, both within chromosomes and between chromosomes. Some of the non-random associations are likely due to retention of linkage disequilibrium in the F12 population, however many, including the inter-chromosomal interactions, must be due to genetic interactions in heat tolerance. One region of particular interest involves 3 linked loci on chromosome IV where the central variant responsible for heat tolerance is antagonistic, coming from the heat sensitive parent and the flanking ones are from the more heat tolerant parent. The 3-locus haplotypes in the selected individuals represent a highly biased sample of the population haplotypes with rare double recombinants in high frequency. These were missed in the original analysis and would never be seen without the multigenerational approach. We show that a statistical analysis of entropy and information gain in genotypes of a selected population can reveal further interactions than previously seen. Importantly this must be done in comparison to the unselected population's genotypes to account for inherent biases in the original population.

q-bio.GN↗

arXiv2014

General H-theorem and entropies that violate the second law

$H$-theorem states that the entropy production is nonnegative and, therefore, the entropy of a closed system should monotonically change in time. In information processing, the entropy production is positive for random transformation of signals (the information processing lemma). Originally, the $H$-theorem and the information processing lemma were proved for the classical Boltzmann-Gibbs-Shannon entropy and for the correspondent divergence (the relative entropy). Many new entropies and divergences have been proposed during last decades and for all of them the $H$-theorem is needed. This note proposes a simple and general criterion to check whether the $H$-theorem is valid for a convex divergence $H$ and demonstrates that some of the popular divergences obey no $H$-theorem. We consider systems with $n$ states $A_i$ that obey first order kinetics (master equation). A convex function $H$ is a Lyapunov function for all master equations with given equilibrium if and only if its conditional minima properly describe the equilibria of pair transitions $A_i \rightleftharpoons A_j$. This theorem does not depend on the principle of detailed balance and is valid for general Markov kinetics. Elementary analysis of pair equilibria demonstrates that the popular Bregman divergences like Euclidean distance or Itakura-Saito distance in the space of distribution cannot be the universal Lyapunov functions for the first-order kinetics and can increase in Markov processes. Therefore, they violate the second law and the information processing lemma. In particular, for these measures of information (divergences) random manipulation with data may add information to data. The main results are extended to nonlinear generalized mass action law kinetic equations. In Appendix, a new family of the universal Lyapunov functions for the generalized mass action law kinetics is described.

cond-mat.stat-mech↗

arXiv2014

ViDaExpert: user-friendly tool for nonlinear visualization and analysis of multidimensional vectorial data

ViDaExpert is a tool for visualization and analysis of multidimensional vectorial data. ViDaExpert is able to work with data tables of "object-feature" type that might contain numerical feature values as well as textual labels for rows (objects) and columns (features). ViDaExpert implements several statistical methods such as standard and weighted Principal Component Analysis (PCA) and the method of elastic maps (non-linear version of PCA), Linear Discriminant Analysis (LDA), multilinear regression, K-Means clustering, a variant of decision tree construction algorithm. Equipped with several user-friendly dialogs for configuring data point representations (size, shape, color) and fast 3D viewer, ViDaExpert is a handy tool allowing to construct an interactive 3D-scene representing a table of data in multidimensional space and perform its quick and insightfull statistical analysis, from basic to advanced methods.

cs.MS↗

arXiv2012

Hydrodynamics from Grad's equations: What can we learn from exact solutions?

A detailed treatment of the classical Chapman-Enskog derivation of hydrodynamics is given in the framework of Grad's moment equations. Grad's systems are considered as the minimal kinetic models where the Chapman-Enskog method can be studied exactly, thereby providing the basis to compare various approximations in extending the hydrodynamic description beyond the Navier-Stokes approximation. Various techniques, such as the method of partial summation, Pade approximants, and invariance principle are compared both in linear and nonlinear situations.

cond-mat.stat-mech↗

arXiv2012

Reduction of dynamical biochemical reaction networks in computational biology

Biochemical networks are used in computational biology, to model the static and dynamical details of systems involved in cell signaling, metabolism, and regulation of gene expression. Parametric and structural uncertainty, as well as combinatorial explosion are strong obstacles against analyzing the dynamics of large models of this type. Multi-scaleness is another property of these networks, that can be used to get past some of these obstacles. Networks with many well separated time scales, can be reduced to simpler networks, in a way that depends only on the orders of magnitude and not on the exact values of the kinetic parameters. The main idea used for such robust simplifications of networks is the concept of dominance among model elements, allowing hierarchical organization of these elements according to their effects on the network dynamics. This concept finds a natural formulation in tropical geometry. We revisit, in the light of these new ideas, the main approaches to model reduction of reaction networks, such as quasi-steady state and quasi-equilibrium approximations, and provide practical recipes for model reduction of linear and nonlinear networks. We also discuss the application of model reduction to backward pruning machine learning techniques.

q-bio.MN↗

arXiv2010

Time Step Expansions and the Invariant Manifold Approach to Lattice Boltzmann Models

The classical method for deriving the macroscopic dynamics of a lattice Boltzmann system is to use a combination of different approximations and expansions. Usually a Chapman-Enskog analysis is performed, either on the continuous Boltzmann system, or its discrete velocity counterpart. Separately a discrete time approximation is introduced to the discrete velocity Boltzmann system, to achieve a practically useful approximation to the continuous system, for use in computation. Thereafter, with some additional arguments, the dynamics of the Chapman-Enskog expansion are linked to the discrete time system to produce the dynamics of the completely discrete scheme. In this paper we put forward a different route to the macroscopic dynamics. We begin with the system discrete in both velocity space and time. We hypothesize that the alternating steps of advection and relaxation, common to all lattice Boltzmann schemes, give rise to a slow invariant manifold. We perform a time step expansion of the discrete time dynamics using the invariance of the manifold. Finally we calculate the dynamics arising from this system. By choosing the fully discrete scheme as a starting point we avoid mixing approximations and arrive at a general form of the microscopic dynamics up to the second order in the time step. We calculate the macroscopic dynamics of two commonly used lattice schemes up to the first order, and hence find the precise form of the deviation from the Navier-Stokes equations in the dissipative term, arising from the discretization of velocity space. Finally we perform a short wave perturbation on the dynamics of these example systems, to find the necessary conditions for their stability.

cond-mat.stat-mech↗