arXiv subjects

Andrei A. Klishin

Publications and source records attributed to Andrei A. Klishin.

15 recordsLinked to original sources

arXiv2026

Origins and mitigation of double descent in reduced order modeling

Latent low-dimensional structure in datasets of natural and engineered systems enables their sparse sensing, or full-state reconstruction from historical data and very few carefully chosen localized measurements. Depending on the reconstruction algorithm, sensor locations, and measurement noise, the reconstruction risk curves demonstrate a diversity of patterns including a dramatic peak in error known as double descent in Machine Learning literature. Here we explore those scenarios under a unified Data-Noise Averaging theory. Qualitatively, we formulate sufficient criteria for double descent to emerge through a catastrophic amplification of a pathological signal in reconstruction. Quantitatively, we predict the detailed risk curves at a fraction of computational cost, trace reconstruction instability to individual sensors and their combinations, and provide regularization mechanisms to mitigate the instability. We demonstrate results for both static reconstruction of Sea Surface Temperature patterns and time integration of a reduced order model of a PDE.

stat.ML↗

arXiv2025

From STLS to Projection-based Dictionary Selection in Sparse Regression for System Identification

In this work, we revisit dictionary-based sparse regression, in particular, Sequential Threshold Least Squares (STLS), and propose a score-guided library selection to provide practical guidance for data-driven modeling, with emphasis on SINDy-type algorithms. STLS is an algorithm to solve the $\ell_0$ sparse least-squares problem, which relies on splitting to efficiently solve the least-squares portion while handling the sparse term via proximal methods. It produces coefficient vectors whose components depend on both the projected reconstruction errors, here referred to as the scores, and the mutual coherence of dictionary terms. The first contribution of this work is a theoretical analysis of the score and dictionary-selection strategy. This could be understood in both the original and weak SINDy regime. Second, numerical experiments on ordinary and partial differential equations highlight the effectiveness of score-based screening, improving both accuracy and interpretability in dynamical system identification. These results suggest that integrating score-guided methods to refine the dictionary more accurately may help SINDy users in some cases to enhance their robustness for data-driven discovery of governing equations.

stat.ML↗

arXiv2025

PySensors 2.0: A Python Package for Sparse Sensor Placement

PySensors is a Python package for selecting and placing a sparse set of sensors for reconstruction and classification tasks. In this major update to PySensors, we introduce spatially constrained sensor placement capabilities, allowing users to enforce constraints such as maximum or exact sensor counts in specific regions, incorporate predetermined sensor locations, and maintain minimum distances between sensors. We extend functionality to support custom basis inputs, enabling integration of any data-driven or spectral basis. We also propose a thermodynamic approach that goes beyond a single "optimal" sensor configuration and maps the complete landscape of sensor interactions induced by the training data. This comprehensive view facilitates integration with external selection criteria and enables assessment of sensor replacement impacts. The new optimization technique also accounts for over- and under-sampling of sensors, utilizing a regularized least squares approach for robust reconstruction. Additionally, we incorporate noise-induced uncertainty quantification of the estimation error and provide visual uncertainty heat maps to guide deployment decisions. To highlight these additions, we provide a brief description of the mathematical algorithms and theory underlying these new capabilities. We demonstrate the usage of new features with illustrative code examples and include practical advice for implementation across various application domains. Finally, we outline a roadmap of potential extensions to further enhance the package's functionality and applicability to emerging sensing challenges.

cs.RO↗

arXiv2025

Data-Induced Interactions of Sparse Sensors Using Statistical Physics

Large-dimensional empirical data in science and engineering frequently have a low-rank structure and can be represented as a combination of just a few eigenmodes. Because of this structure, we can use just a few spatially localized sensor measurements to reconstruct the full state of a complex system. The quality of this reconstruction, especially in the presence of sensor noise, depends significantly on the spatial configuration of the sensors. Multiple algorithms based on gappy interpolation and QR factorization have been proposed to optimize sensor placement. Here, instead of an algorithm that outputs a single "optimal" sensor configuration, we take a statistical mechanics view to compute the full landscape of sensor interactions induced by the training data. The two key advances of this paper are the recasting of the sensor placement landscape in an Ising model form and a regularized reconstruction that significantly decreases reconstruction error for few sensors. In addition, we provide first uncertainty quantification of the sparse sensing reconstruction and open questions about the shape of reconstruction risk curve. Mapping out these data-induced sensor interactions allows combining them with external selection criteria and anticipating sensor replacement impacts.

cond-mat.stat-mech↗

arXiv2025

Statistical Mechanics of Dynamical System Identification

Recovering dynamical equations from observed noisy data is the central challenge of system identification. We develop a statistical mechanics approach to analyze sparse equation discovery algorithms, which typically balance data fit and parsimony via hyperparameter tuning. In this framework, statistical mechanics offers tools to analyze the interplay between complexity and fitness similarly to that of entropy and energy in physical systems. To establish this analogy, we define the hyperparameter optimization procedure as a two-level Bayesian inference problem that separates variable selection from coefficient inference and enables the computation of the posterior parameter distribution in closed form. Our approach provides uncertainty quantification, crucial in the low-data limit that is frequently encountered in real-world applications. A key advantage of employing statistical mechanical concepts, such as free energy and the partition function, is to connect the large data limit to thermodynamic limit and characterize the sparsity- and noise-induced phase transitions that delineate correct from incorrect identification. We thus provide a method for closed-loop inference, estimating the noise in a given model and checking if the model is tolerant to that noise amount. This perspective of sparse equation discovery is versatile and can be adapted to various other equation discovery algorithms.

cond-mat.stat-mech↗

arXiv2023

Human Learning of Hierarchical Graphs

Humans are constantly exposed to sequences of events in the environment. Those sequences frequently evince statistical regularities, such as the probabilities with which one event transitions to another. Collectively, inter-event transition probabilities can be modeled as a graph or network. Many real-world networks are organized hierarchically and understanding how humans learn these networks is an ongoing aim of current investigations. While much is known about how humans learn basic transition graph topology, whether and to what degree humans can learn hierarchical structures in such graphs remains unknown. We investigate how humans learn hierarchical graphs of the Sierpiński family using computer simulations and behavioral laboratory experiments. We probe the mental estimates of transition probabilities via the surprisal effect: a phenomenon in which humans react more slowly to less expected transitions, such as those between communities or modules in the network. Using mean-field predictions and numerical simulations, we show that surprisal effects are stronger for finer-level than coarser-level hierarchical transitions. Surprisal effects at coarser levels of the hierarchy are difficult to detect for limited learning times or in small samples. Using a serial response experiment with human participants (n=$100$), we replicate our predictions by detecting a surprisal effect at the finer-level of the hierarchy but not at the coarser-level of the hierarchy. To further explain our findings, we evaluate the presence of a trade-off in learning, whereby humans who learned the finer-level of the hierarchy better tended to learn the coarser-level worse, and vice versa. Our study elucidates the processes by which humans learn hierarchical sequential events. Our work charts a road map for future investigation of the neural underpinnings and behavioral manifestations of graph learning.

q-bio.NC↗

arXiv2023

No Free Lunch for Avoiding Clustering Vulnerabilities in Distributed Systems

Emergent design failures are ubiquitous in complex systems, and often arise when system elements cluster. Approaches to systematically reduce clustering could improve a design's resilience, but reducing clustering is difficult if it is driven by collective interactions among design elements. Here, we use techniques from statistical physics to identify mechanisms by which spatial clusters of design elements emerge in complex systems modelled by heterogeneous networks. We find that, in addition to naive, attraction-driven clustering, heterogeneous networks can exhibit emergent, repulsion-driven clustering. We draw quantitative connections between our results on a model system in naval engineering to entropy-driven phenomena in nanoscale self-assembly, and give a general argument that the clustering phenomena we observe should arise in many distributed systems. We identify circumstances under which generic design problems will exhibit trade-offs between clustering and uncertainty in design objectives, and we present a framework to identify and quantify trade-offs to manage clustering vulnerabilities.

physics.soc-ph↗

arXiv2022

Learning Dynamic Graphs, Too Slow

The structure of knowledge is commonly described as a network of key concepts and semantic relations between them. A learner of a particular domain can discover this network by navigating the nodes and edges presented by instructional material, such as a textbook, workbook, or other text. While over a long temporal period such exploration processes are certain to discover the whole connected network, little is known about how the learning is affected by the dual pressures of finite study time and human mental errors. Here we model the learning of linear algebra textbooks with finite length random walks over the corresponding semantic networks. We show that if a learner does not keep up with the pace of material presentation, the learning can be an order of magnitude worse than it is in the asymptotic limit. Further, we find that this loss is compounded by three types of mental errors: forgetting, shuffling, and reinforcement. Broadly, our study informs the design of teaching materials from both structural and temporal perspectives.

cond-mat.stat-mech↗

arXiv2022

Exposure theory for learning complex networks with random walks

Random walks are a common model for exploration and discovery of complex networks. While numerous algorithms have been proposed to map out an unknown network, a complementary question arises: in a known network, which nodes and edges are most likely to be discovered by a random walker in finite time? Here we introduce exposure theory, a statistical mechanics framework that predicts the learning of nodes and edges across several types of networks, including weighted and temporal, and show that edge learning follows a universal trajectory. While the learning of individual nodes and edges is noisy, exposure theory produces a highly accurate prediction of aggregate exploration statistics.

cond-mat.stat-mech↗

arXiv2021

Citation inequity and gendered citation practices in contemporary physics

The historical and contemporary under-attribution of women's contributions to scientific scholarship is well-known and well-studied, with effects that are felt today in myriad ways by women scientists. One measure of this under-attribution is the so-called citation gap between men and women: the under-citation of papers authored by women relative to expected rates coupled with a corresponding over-citation of papers authored by men relative to expected rates. We explore the citation gap in contemporary physics, analyzing over one million articles published over the last 25 years in 35 physics journals that span a wide range of subfields. Using a model that predicts papers' expected citation rates according to a set of characteristics separate from author gender, we find a global bias wherein papers authored by women are significantly under-cited, and papers authored by men are significantly over-cited. Moreover, we find that citation behavior varies along several dimensions, such that imbalances differ according to who is citing, where they are citing, and how they are citing. Specifically, citation imbalance in favor of man-authored papers is highest for papers authored by men, papers published in general physics journals, and papers likely to be less familiar to citing authors. Our results suggest that, although deciding which papers to cite is an individual choice, the cumulative effects of these choices needlessly harm a subset of scholars. We discuss several strategies for the mitigation of these effects, including conscious behavioral changes at the individual, journal, and community levels.

physics.soc-ph↗

arXiv2021

Optimizing the Human Learnability of Abstract Network Representations

Precisely how humans process relational patterns of information in knowledge, language, music, and society is not well understood. Prior work in the field of statistical learning has demonstrated that humans process such information by building internal models of the underlying network structure. However, these mental maps are often inaccurate due to limitations in human information processing. The existence of such limitations raises clear questions: Given a target network that one wishes for a human to learn, what network should one present to the human? Should one simply present the target network as-is, or should one emphasize certain parts of the network to proactively mitigate expected errors in learning? To answer these questions, we study the optimization of network learnability. Evaluating an array of synthetic and real-world networks, we find that learnability is enhanced by reinforcing connections within modules or clusters. In contrast, when networks contain significant core-periphery structure we find that learnability is best optimized by reinforcing peripheral edges between low-degree nodes. Overall, our findings suggest that the accuracy of human network learning can be systematically enhanced by targeted emphasis and de-emphasis of prescribed sectors of information.

physics.soc-ph↗

arXiv2021

Topological Design of Heterogeneous Self-Assembly

Controlling the topology of structures self-assembled from a set of heterogeneous building blocks is highly desirable for many applications, but is poorly understood theoretically. Here we show that the thermodynamic theory of self-assembly involves an inevitable divergence in chemical potential. The divergence and its detailed structure are controlled by the spectrum of the transfer matrix, which summarizes all of self-assembly design degrees of freedom. By analyzing the transfer matrix, we map out the phase boundary between the designable structures and the unstructured aggregates, driven by the level of cross-talk.

cond-mat.soft↗

arXiv2021

Avoidance, Adjacency, and Association in Distributed Systems Design

Patterns of avoidance, adjacency, and association in complex systems design emerge from the system's underlying logical architecture (functional relationships among components) and physical architecture (component physical properties and spatial location). Understanding the physical--logical architecture interplay that gives rise to patterns of arrangement requires a quantitative approach that bridges both descriptions. Here, we show that statistical physics reveals patterns of avoidance, adjacency, and association across sets of complex, distributed system design solutions. Using an example arrangement problem and tensor network methods, we identify several phenomena in complex systems design, including placement symmetry breaking, propagating correlation, and emergent localization. Our approach generalizes straightforwardly to a broad range of complex systems design settings where it can provide a platform for investigating basic design phenomena.

physics.soc-ph↗

arXiv2020

Robust Design from Systems Physics

A crucial challenge in engineering modern, integrated systems is to produce robust designs. Ensuring robust design is difficult because subsystem couplings produce unpredictable response to changes in whole system specifications. Here, we show that the response of design elements to whole-system specification changes can be characterized, as materials are, using strong/weak and brittle/ductile dichotomies. We find these dichotomies emerge from a mesoscale treatment of early stage design problems that we cast in terms of stress-strain relationships. Compared with other state-of-the-art techniques, we propose a two-factor robustness metric that is computable for sets of design solutions. We illustrate the use of this approach with examples of arrangement problems drawn from naval engineering, however our approach is immediately applicable to a broad range of problems in integrated systems design.

physics.soc-ph↗

arXiv2018

Statistical Physics of Design

A key challenge in complex design problems that permeate science and engineering is the need to balance design objectives for specific design elements or subsystems with global system objectives. Global objectives give rise to competing design pressures, whose effects can be difficult to trace in subsystem design. Here, using examples from arrangement problems, we show that the systems-level application of statistical physics principles, which we term "systems physics", provides a detailed characterization of subsystem design in terms of the concepts of stress and strain from materials physics. We analyze instances of routing problems in naval architectures, and show that systems physics provides a direct means of classifying architecture types, and quantifying trade-offs between subsystem- and overall performance. Our approach generalizes straightforwardly to design problems in a wide range of other disciplines that require concrete understanding of how the pressure to meet overall design objectives drives the outcomes for component subsystems.

physics.soc-ph↗