arXiv subjects

A. N. Gorban

Publications and source records attributed to A. N. Gorban.

At least 37 records · Page 2Linked to original sources

arXiv2016

Robust principal graphs for data approximation

Revealing hidden geometry and topology in noisy data sets is a challenging task. Elastic principal graph is a computationally efficient and flexible data approximator based on embedding a graph into the data space and minimizing the energy functional penalizing the deviation of graph nodes both from data points and from pluri-harmonic configuration (generalization of linearity). The structure of principal graph is learned from data by application of a topological grammar which in the simplest case leads to the construction of principal curves or trees. In order to more efficiently cope with noise and outliers, here we suggest using a trimmed data approximation term to increase the robustness of the method. The modification of the method that we suggest does not affect either computational efficiency or general convergence properties of the original elastic graph method. The trimmed elastic energy functional remains a Lyapunov function for the optimization algorithm. On several examples of complex data distributions we demonstrate how the robust principal graphs learn the global data structure and show the advantage of using the trimmed data approximation term for the construction of principal graphs and other popular data approximators.

cs.DS↗

arXiv2016

Piece-wise quadratic approximations of arbitrary error functions for fast and robust machine learning

Most of machine learning approaches have stemmed from the application of minimizing the mean squared distance principle, based on the computationally efficient quadratic optimization methods. However, when faced with high-dimensional and noisy data, the quadratic error functionals demonstrated many weaknesses including high sensitivity to contaminating factors and dimensionality curse. Therefore, a lot of recent applications in machine learning exploited properties of non-quadratic error functionals based on $L_1$ norm or even sub-linear potentials corresponding to quasinorms $L_p$ ($0<p<1$). The back side of these approaches is increase in computational cost for optimization. Till so far, no approaches have been suggested to deal with {\it arbitrary} error functionals, in a flexible and computationally efficient framework. In this paper, we develop a theory and basic universal data approximation algorithms ($k$-means, principal components, principal manifolds and graphs, regularized and sparse regression), based on piece-wise quadratic error potentials of subquadratic growth (PQSQ potentials). We develop a new and universal framework to minimize {\it arbitrary sub-quadratic error potentials} using an algorithm with guaranteed fast convergence to the local or global error minimum. The theory of PQSQ potentials is based on the notion of the cone of minorant functions, and represents a natural approximation formalism based on the application of min-plus algebra. The approach can be applied in most of existing machine learning methods, including methods of data approximation and regularized and sparse regression, leading to the improvement in the computational cost/accuracy trade-off. We demonstrate that on synthetic and real-life datasets PQSQ-based machine learning methods achieve orders of magnitude faster computational performance than the corresponding state-of-the-art methods.

cs.LG↗

arXiv2016

Fast Sampling of Evolving Systems with Periodic Trajectories

We propose a novel method for fast and scalable evaluation of periodic solutions of systems of ordinary differential equations for a given set of parameter values and initial conditions. The equations governing the system dynamics are supposed to be of a special class, albeit admitting nonlinear parametrization and state nonlinearities. The method enables to represent a given periodic solution as sums of computable integrals and functions that are explicitly dependent on parameters of interest and initial conditions. This allows invoking parallel computational streams in order to increase speed of calculations. Performance and practical implications of the method are illustrated with examples including classical predator-prey system and models of neuronal cells.

math.DS↗

arXiv2016

Evolution of adaptation mechanisms: adaptation energy, stress, and oscillating death

In 1938, H. Selye proposed the notion of adaptation energy and published "Experimental evidence supporting the conception of adaptation energy". Adaptation of an animal to different factors appears as the spending of one resource. Adaptation energy is a hypothetical extensive quantity spent for adaptation. This term causes much debate when one takes it literally, as a physical quantity, i.e. a sort of energy. The controversial points of view impede the systematic use of the notion of adaptation energy despite experimental evidence. Nevertheless, the response to many harmful factors often has general non-specific form and we suggest that the mechanisms of physiological adaptation admit a very general and nonspecific description. We aim to demonstrate that Selye's adaptation energy is the cornerstone of the top-down approach to modelling of non-specific adaptation processes. We analyse Selye's axioms of adaptation energy together with Goldstone's modifications and propose a series of models for interpretation of these axioms. {\em Adaptation energy is considered as an internal coordinate on the `dominant path' in the model of adaptation}. The phenomena of `oscillating death' and `oscillating remission' are predicted on the base of the dynamical models of adaptation. Natural selection plays a key role in the evolution of mechanisms of physiological adaptation. We use the fitness optimization approach to study of the distribution of resources for neutralization of harmful factors, during adaptation to a multifactor environment, and analyse the optimal strategies for different systems of factors.

q-bio.PE↗

arXiv2015

Generalized Mass Action Law and Thermodynamics of Nonlinear Markov Processes

The nonlinear Markov processes are the measure-valued dynamical systems which preserve positivity. They can be represented as the law of large numbers limits of general Markov models of interacting particles. In physics, the kinetic equations allow Lyapunov functionals (entropy, free energy, etc.). This may be considered as a sort of inheritance of the Lyapunov functionals from the microscopic master equations. We study nonlinear Markov processes that inherit thermodynamic properties from the microscopic linear Markov processes. We develop the thermodynamics of nonlinear Markov processes and analyze the asymptotic assumption, which are sufficient for this inheritance.

physics.chem-ph↗

arXiv2014

Further Results on Lyapunov-Like Conditions of Forward Invariance and Boundedness for a Class of Unstable Systems

We provide several characterizations of convergence to unstable equilibria in nonlinear systems. Our current contribution is three-fold. First we present simple algebraic conditions for establishing local convergence of non-trivial solutions of nonlinear systems to unstable equilibria. The conditions are based on the earlier work (A.N. Gorban, I.Yu. Tyukin, E. Steur, and H. Nijmeijer, SIAM Journal on Control and Optimization, Vol. 51, No. 3, 2013) and can be viewed as an extension of the Lyapunov's first method in that they apply to systems in which the corresponding Jacobian has one zero eigenvalue. Second, we show that for a relevant subclass of systems, persistency of excitation of a function of time in the right-hand side of the equations governing dynamics of the system ensure existence of an attractor basin such that solutions passing through this basin in forward time converge to the origin exponentially. Finally we demonstrate that conditions developed in (A.N. Gorban, I.Yu. Tyukin, E. Steur, and H. Nijmeijer, SIAM Journal on Control and Optimization, Vol. 51, No. 3, 2013) may be remarkably tight.

math.DS↗

arXiv2014

Detailed balance in micro- and macrokinetics and micro-distinguishability of macro-processes

We develop a general framework for the discussion of detailed balance and analyse its microscopic background. We find that there should be two additions to the well-known $T$- or $PT$-invariance of the microscopic laws of motion: 1. Equilibrium should not spontaneously break the relevant $T$- or $PT$-symmetry. 2. The macroscopic processes should be microscopically distinguishable to guarantee persistence of detailed balance in the model reduction from micro- to macrokinetics. We briefly discuss examples of the violation of these rules and the corresponding violation of detailed balance.

cond-mat.stat-mech↗

arXiv2014

Nonlinear Quality of Life Index

We present details of the analysis of the nonlinear quality of life index for 171 countries. This index is based on four indicators: GDP per capita by Purchasing Power Parities, Life expectancy at birth, Infant mortality rate, and Tuberculosis incidence. We analyze the structure of the data in order to find the optimal and independent on expert's opinion way to map several numerical indicators from a multidimensional space onto the one-dimensional space of the quality of life. In the 4D space we found a principal curve that goes "through the middle" of the dataset and project the data points on this curve. The order along this principal curve gives us the ranking of countries. Projection onto the principal curve provides a solution to the classical problem of unsupervised ranking of objects. It allows us to find the independent on expert's opinion way to project several numerical indicators from a multidimensional space onto the one-dimensional space of the index values. This projection is, in some sense, optimal and preserves as much information as possible. For computation we used ViDaExpert, a tool for visualization and analysis of multidimensional vectorial data (arXiv:1406.5550).

cs.NE↗

arXiv2014

Computational diagnosis and risk evaluation for canine lymphoma

The canine lymphoma blood test detects the levels of two biomarkers, the acute phase proteins (C-Reactive Protein and Haptoglobin). This test can be used for diagnostics, for screening, and for remission monitoring as well. We analyze clinical data, test various machine learning methods and select the best approach to these problems. Three family of methods, decision trees, kNN (including advanced and adaptive kNN) and probability density evaluation with radial basis functions, are used for classification and risk estimation. Several pre-processing approaches were implemented and compared. The best of them are used to create the diagnostic system. For the differential diagnosis the best solution gives the sensitivity and specificity of 83.5% and 77%, respectively (using three input features, CRP, Haptoglobin and standard clinical symptom). For the screening task, the decision tree method provides the best result, with sensitivity and specificity of 81.4% and >99%, respectively (using the same input features). If the clinical symptoms (Lymphadenopathy) are considered as unknown then a decision tree with CRP and Hapt only provides sensitivity 69% and specificity 83.5%. The lymphoma risk evaluation problem is formulated and solved. The best models are selected as the system for computational lymphoma diagnosis and evaluation the risk of lymphoma as well. These methods are implemented into a special web-accessed software and are applied to problem of monitoring dogs with lymphoma after treatment. It detects recurrence of lymphoma up to two months prior to the appearance of clinical signs. The risk map visualisation provides a friendly tool for explanatory data analysis.

q-bio.QM↗

arXiv2014

The Michaelis-Menten-Stueckelberg Theorem

We study chemical reactions with complex mechanisms under two assumptions: (i) intermediates are present in small amounts (this is the quasi-steady-state hypothesis or QSS) and (ii) they are in equilibrium relations with substrates (this is the quasiequilibrium hypothesis or QE). Under these assumptions, we prove the generalized mass action law together with the basic relations between kinetic factors, which are sufficient for the positivity of the entropy production but hold even without microreversibility, when the detailed balance is not applicable. Even though QE and QSS produce useful approximations by themselves, only the combination of these assumptions can render the possibility beyond the "rarefied gas" limit or the "molecular chaos" hypotheses. We do not use any a priori form of the kinetic law for the chemical reactions and describe their equilibria by thermodynamic relations. The transformations of the intermediate compounds can be described by the Markov kinetics because of their low density ({\em low density of elementary events}). This combination of assumptions was introduced by Michaelis and Menten in 1913. In 1952, Stueckelberg used the same assumptions for the gas kinetics and produced the remarkable semi-detailed balance relations between collision rates in the Boltzmann equation that are weaker than the detailed balance conditions but are still sufficient for the Boltzmann $H$-theorem to be valid. Our results are obtained within the Michaelis-Menten-Stueckelbeg conceptual framework.

physics.chem-ph↗

arXiv2013

Entropy: The Markov Ordering Approach

The focus of this article is on entropy and Markov processes. We study the properties of functionals which are invariant with respect to monotonic transformations and analyze two invariant "additivity" properties: (i) existence of a monotonic transformation which makes the functional additive with respect to the joining of independent systems and (ii) existence of a monotonic transformation which makes the functional additive with respect to the partitioning of the space of states. All Lyapunov functionals for Markov chains which have properties (i) and (ii) are derived. We describe the most general ordering of the distribution space, with respect to which all continuous-time Markov processes are monotonic (the {\em Markov order}). The solution differs significantly from the ordering given by the inequality of entropy growth. For inference, this approach results in a convex compact set of conditionally "most random" distributions.

physics.data-an↗

arXiv2013

Maxallent: Maximizers of all Entropies and Uncertainty of Uncertainty

The entropy maximum approach (Maxent) was developed as a minimization of the subjective uncertainty measured by the Boltzmann--Gibbs--Shannon entropy. Many new entropies have been invented in the second half of the 20th century. Now there exists a rich choice of entropies for fitting needs. This diversity of entropies gave rise to a Maxent "anarchism". Maxent approach is now the conditional maximization of an appropriate entropy for the evaluation of the probability distribution when our information is partial and incomplete. The rich choice of non-classical entropies causes a new problem: which entropy is better for a given class of applications? We understand entropy as a measure of uncertainty which increases in Markov processes. In this work, we describe the most general ordering of the distribution space, with respect to which all continuous-time Markov processes are monotonic (the Markov order). For inference, this approach results in a set of conditionally "most random" distributions. Each distribution from this set is a maximizer of its own entropy. This "uncertainty of uncertainty" is unavoidable in analysis of non-equilibrium systems. Surprisingly, the constructive description of this set of maximizers is possible. Two decomposition theorems for Markov processes provide a tool for this description.

physics.data-an↗

arXiv2013

Hilbert's 6th Problem: Exact and Approximate Hydrodynamic Manifolds for Kinetic Equations

The problem of the derivation of hydrodynamics from the Boltzmann equation and related dissipative systems is formulated as the problem of slow invariant manifold in the space of distributions. We review a few instances where such hydrodynamic manifolds were found analytically both as the result of summation of the Chapman--Enskog asymptotic expansion and by the direct solution of the invariance equation. These model cases, comprising Grad's moment systems, both linear and nonlinear, are studied in depth in order to gain understanding of what can be expected for the Boltzmann equation. Particularly, the dispersive dominance and saturation of dissipation rate of the exact hydrodynamics in the short-wave limit and the viscosity modification at high divergence of the flow velocity are indicated as severe obstacles to the resolution of Hilbert's 6th Problem. Furthermore, we review the derivation of the approximate hydrodynamic manifold for the Boltzmann equation using Newton's iteration and avoiding smallness parameters, and compare this to the exact solutions. Additionally, we discuss the problem of projection of the Boltzmann equation onto the approximate hydrodynamic invariant manifold using entropy concepts. Finally, a set of hypotheses is put forward where we describe open questions and set a horizon for what can be derived exactly or proven about the hydrodynamic manifolds for the Boltzmann equation in the future.

math-ph↗

arXiv2013

Explicit Reduced-Order Integral Formulations of State and Parameter Estimation Problems for a Class of Nonlinear Systems

We propose a technique for reformulation of state and parameter estimation problems as that of matching explicitly computable definite integrals with known kernels to data. The technique applies for a class of systems of nonlinear ordinary differential equations and is aimed to exploit parallel computational streams in order to increase speed of calculations. The idea is based on the classical adaptive observers design. It has been shown that in case the data is periodic it may be possible to reduce dimensionality of the inference problem to that of the dimension of the vector of parameters entering the right-hand side of the model nonlinearly. Performance and practical implications of the method are illustrated on a benchmark model governing dynamics of voltage in generated in barnacle giant muscle.

math.OC↗

arXiv2013

Multiscale principal component analysis

Principal component analysis (PCA) is an important tool in exploring data. The conventional approach to PCA leads to a solution which favours the structures with large variances. This is sensitive to outliers and could obfuscate interesting underlying structures. One of the equivalent definitions of PCA is that it seeks the subspaces that maximize the sum of squared pairwise distances between data projections. This definition opens up more flexibility in the analysis of principal components which is useful in enhancing PCA. In this paper we introduce scales into PCA by maximizing only the sum of pairwise distances between projections for pairs of datapoints with distances within a chosen interval of values [l,u]. The resulting principal component decompositions in Multiscale PCA depend on point (l,u) on the plane and for each point we define projectors onto principal components. Cluster analysis of these projectors reveals the structures in the data at various scales. Each structure is described by the eigenvectors at the medoid point of the cluster which represent the structure. We also use the distortion of projections as a criterion for choosing an appropriate scale especially for data with outliers. This method was tested on both artificial distribution of data and real data. For data with multiscale structures, the method was able to reveal the different structures of the data and also to reduce the effect of outliers in the principal component analysis.

stat.ME↗

arXiv2013

Is it possible to predict long-term success with k-NN? Case Study of four market indices (FTSE100, DAX, HANGSENG, NASDAQ)

This case study tests the possibility of prediction for "success" (or "winner") components of four stock & shares market indices in a time period of three years from 02-Jul-2009 to 29-Jun-2012.We compare their performance ain two time frames: initial frame three months at the beginning (02/06/2009-30/09/2009) and the final three month frame (02/04/2012-29/06/2012). To label the components, average price ratio between two time frames in descending order is computed. The average price ratio is defined as the ratio between the mean prices of the beginning and final time period. The "winner" components are referred to the top one third of total components in the same order as average price ratio it means the mean price of final time period is relatively higher than the beginning time period. The "loser" components are referred to the last one third of total components in the same order as they have higher mean prices of beginning time period. We analyse, is there any information about the winner-looser separation in the initial fragments of the daily closing prices log-returns time series. The Leave-One-Out Cross-Validation with k-NN algorithm is applied on the daily log-return of components using a distance and proximity in the experiment. By looking at the error analysis, it shows that for HANGSENG and DAX index, there are clear signs of possibility to evaluate the probability of long-term success. The correlation distance matrix histograms and 2-D/3-D elastic maps generated from ViDaExpert show that the winner components are closer to each other and winner/loser components are separable on elastic maps for HANGSENG and DAX index while for the negative possibility indices, there is no sign of separation.

q-fin.ST↗

arXiv2013

Geometrical complexity of data approximators

There are many methods developed to approximate a cloud of vectors embedded in high-dimensional space by simpler objects: starting from principal points and linear manifolds to self-organizing maps, neural gas, elastic maps, various types of principal curves and principal trees, and so on. For each type of approximators the measure of the approximator complexity was developed too. These measures are necessary to find the balance between accuracy and complexity and to define the optimal approximations of a given type. We propose a measure of complexity (geometrical complexity) which is applicable to approximators of several types and which allows comparing data approximations of different types.

stat.ML↗

arXiv2013

Grasping Complexity

The century of complexity has come. The face of science has changed. Surprisingly, when we start asking about the essence of these changes and then critically analyse the answers, the result are mostly discouraging. Most of the answers are related to the properties that have been in the focus of scientific research already for more than a century (like non-linearity). This paper is Preface to the special issue "Grasping Complexity" of the journal "Computers and Mathematics with Applications". We analyse the change of era in science, its reasons and main changes in scientific activity and give a brief review of the papers in the issue.

cs.GL↗