arXiv subjects

Choy Heng Lai

Publications and source records attributed to Choy Heng Lai.

17 recordsLinked to original sources

arXiv2026

Self-Organization to the Edge of Ergodicity Breaking in a Complex Adaptive System

Self-organized criticality is widely invoked for collective behavior, yet its role in objective-driven, heterogeneous adaptive systems is unclear. We introduce {\tt EvoSK}: agents learn on a Sherrington--Kirkpatrick landscape while the least fit are replaced. It self-organizes to the edge of ergodicity breaking, with scale-free avalanches ($τ\approx -1.5$) and rewards beating any tuned non-evolutionary regime. Its cascade's branching ratio is the spectral radius of the learning dynamics' Jacobian, making critical branching and ergodicity breaking one marginal-stability condition fixing the exponent. The attraction to criticality follows from the selection--mutation balance: subcritical cascades decay too fast to dislodge frozen agents, supercritical cascades shield them from selection; only the critical power-law tail supplies the polynomial rate the balance requires.

nlin.AO↗

arXiv2026

Multiple Descents in Deep Learning as a Sequence of Order-Chaos Transitions in LSTM Networks

We observe a novel `multiple-descent' phenomenon during the learning process of a recurrent neural network called long-short-term memory (LSTM) networks during its training on real-world task, in which the performance goes through long cycles of up and down trends multiple times after the model is overtrained. By carrying out asymptotic stability analysis of the models, we found that the cycles in performance -- indicated by loss function in test data -- are closely associated with the phase transition process between order and chaos of the model, and the local optimal training step are consistently at the critical transition point between the two phases. More importantly, the most optimal point of the model usually occurs at the first transition from order to chaos, where the `width' of the `edge of chaos' is often the widest, allowing the best exploration of weight configurations for learning.

cs.LG↗

arXiv2021

Edge of chaos as a guiding principle for modern neural network training

The success of deep neural networks in real-world problems has prompted many attempts to explain their training dynamics and generalization performance, but more guiding principles for the training of neural networks are still needed. Motivated by the edge of chaos principle behind the optimal performance of neural networks, we study the role of various hyperparameters in modern neural network training algorithms in terms of the order-chaos phase diagram. In particular, we study a fully analytical feedforward neural network trained on the widely adopted Fashion-MNIST dataset, and study the dynamics associated with the hyperparameters in back-propagation during the training process. We find that for the basic algorithm of stochastic gradient descent with momentum, in the range around the commonly used hyperparameter values, clear scaling relations are present with respect to the training time during the ordered phase in the phase diagram, and the model's optimal generalization power at the edge of chaos is similar across different training parameter combinations. In the chaotic phase, the same scaling no longer exists. The scaling allows us to choose the training parameters to achieve faster training without sacrificing performance. In addition, we find that the commonly used model regularization method - weight decay - effectively pushes the model towards the ordered phase to achieve better performance. Leveraging on this fact and the scaling relations in the other hyperparameters, we derived a principled guideline for hyperparameter determination, such that the model can achieve optimal performance by saturating it at the edge of chaos. Demonstrated on this simple neural network model and training algorithm, our work improves the understanding of neural network training dynamics, and can potentially be extended to guiding principles of more complex model architectures and algorithms.

cs.LG↗

arXiv2020

Optimal Machine Intelligence at the Edge of Chaos

It has long been suggested that the biological brain operates at some critical point between two different phases, possibly order and chaos. Despite many indirect empirical evidence from the brain and analytical indication on simple neural networks, the foundation of this hypothesis on generic non-linear systems remains unclear. Here we develop a general theory that reveals the exact edge of chaos is the boundary between the chaotic phase and the (pseudo)periodic phase arising from Neimark-Sacker bifurcation. This edge is analytically determined by the asymptotic Jacobian norm values of the non-linear operator and influenced by the dimensionality of the system. The optimality at the edge of chaos is associated with the highest information transfer between input and output at this point similar to that of the logistic map. As empirical validations, our experiments on the various deep learning models in computer vision demonstrate the optimality of the models near the edge of chaos, and we observe that the state-of-art training algorithms push the models towards such edge as they become more accurate. We further establishes the theoretical understanding of deep learning model generalization through asymptotic stability.

cs.LG↗

arXiv2016

Critical Transitions in Public Opinion: A Case Study of American Presidential Election

At the tipping point, it is known that small incident can trigger dramatic societal shift. Getting early-warning signals for such changes are valuable to avoid detrimental outcomes such as riots or collapses of nations. However, it is notoriously hard to capture the processes of such transitions in the real-world. Here, we demonstrate the occurrence of a major shift in public opinion in the form of political support. Instead of simple swapping of ruling parties, we study the regime shift of a party popularity based on its attractiveness by examining the American presidential elections during 1980-2012. A single irreversible transition is detected in 1991. Once a transition happens, recovery to the original level of attractiveness does not bring popularity of the political party back. Remarkably, this transition is corroborated by tell-tale early-warning signature of critical slowing down. Our approach is applicable to shifts in public attitude within any social system.

physics.soc-ph↗

arXiv2013

Spectral Anlysis on Explosive Percolation

We study the spectral properties of the process of explosive percolation. In particular, we explore how the maximum eigenvalue of the adjacency matrix of a network which governs the spreading efficiency evolves as the density of connection increases. Interestingly, for networks with connectivity that grow in an explosive way, information spreading and mass transport are found to be carried out inefficiently. In the conventional explosive percolation models that we studied, the sudden emergences of large-scale connectivity are found to come with relatively lowered efficiency of spreading. Nevertheless, the spreading efficiency of the explosive model can be increased by introducing heterogeneous structures into the networks.

physics.soc-ph↗

arXiv2012

Epidemic spreading induced by diversity of agents' mobility

In this paper, we study into the impact of the preference of an individual for public transport on the spread of infectious disease, through a quantity known as the public mobility. Our theoretical and numerical results based on a constructed model reveal that if the average public mobility of the agents is fixed, an increase in the diversity of the agents' public mobility reduces the epidemic threshold, beyond which an enhancement in the rate of infection is observed. Our findings provide an approach to improve the resistance of a society against infectious disease, while preserving the utilization rate of the public transportation system.

physics.soc-ph↗

arXiv2011

Network Extreme Eigenvalue - from Multimodal to Scale-free Network

The extreme eigenvalues of adjacency matrices are important indicators on the influences of topological structures to collective dynamical behavior of complex networks. Recent findings on the ensemble averageability of the extreme eigenvalue further authenticate its sensibility in the study of network dynamics. Here we determine the ensemble average of the extreme eigenvalue and characterize the deviation across the ensemble through the discrete form of random scale-free network. Remarkably, the analytical approximation derived from the discrete form shows significant improvement over the previous results. This has also led us to the same conclusion as [Phys. Rev. Lett. 98, 248701 (2007)] that deviation in the reduced extreme eigenvalues vanishes as the network size grows.

physics.soc-ph↗

arXiv2011

Competition between Intra-community and Inter-community Synchronization

In this paper the effects of external links on the synchronization performance of community networks, especially on the competition between individual community and the whole network, are studied in detail. The study is organized from two aspects: the number or portion of external links and the connecting strategy of external links between different communities. It is found that increasing the number of external links will enhance the global synchronizability but degrade the ynchronization performance of individual community before some critical point. After that the individual community will synchronize better and better as part of the whole network because the community structure is not so prominent. Among various connection strategies, connecting nodes belonging to different communities randomly rather than connecting nodes with larger degrees is the most efficient way to enhance global synchronization of the network. However, a preferential connection scheme linking most of the hubs from the communities will allow rather efficient global synchronization while maintaining strong dynamical clustering of the communities. Interestingly, the observations are found to be relevant in a realistic network of cat cortex. The synchronization state is just at the critical point, which shows a reasonable combination of segregated function in individual communities and coordination among them. Our work sheds light on principles underlying the emergence of modular architectures in real network systems and provides guidance for the manipulation of synchronization in community networks.

physics.soc-ph↗

arXiv2009

Protecting infrastructure networks from cost-based attacks

It has been known that heterogeneous networks are vulnerable to the intentional removal of a small fraction of highly connected or loaded nodes, which implies that, to protect a network effectively, a few important nodes should be allocated with more defense resources than the others. However, if too many resources are allocated to the few important nodes, the numerous less-important nodes will be less protected, which, when attacked all together, still capable of causing a devastating damage. A natural question therefore is how to efficiently distribute the limited defense resources among the network nodes such that the network damage is minimized whatever attack strategy the attacker may take. In this paper, taking into account the factor of attack cost, we will revisit the problem of network security and search for efficient network defense against the cost-based attacks. The study shows that, for a general complex network, there will exist an optimal distribution of the defense resources, with which the network is well protected from cost-based attacks. Furthermore, it is found that the configuration of the optimal defense is dependent on the network parameters. Specifically, network that has a larger size, sparser connection and more heterogeneous structure will be more benefited from the defense optimization.

cs.NI↗

arXiv2008

Transition to amplitude death in scale-free networks

Transition to amplitude death in scale-free networks of nonlinear oscillators is investigated. As the coupling strength increases, the network will undergo three stages in approaching to the state of complete amplitude death. The first stage is featured by a \emph{"stair-like"} distribution of the node amplitude, and the transition is accomplished by a \emph{hierarchical death} of the amplitude stairs. The second and third stages are characterized by, respectively, a continuing elimination of the synchronous clusters and a fast death of the non-synchronized nodes.

nlin.AO↗

arXiv2008

Effect of the shape of periodic forces and second periodic forces on horseshoe chaos in Duffing oscillator

The effect of the shape of six different periodic forces and second periodic forces on the onset of horseshoe chaos are studied both analytically and numerically in a Duffing oscillator. The external periodic forces considered are sine wave, square wave, symmetric saw-tooth wave, asymmetric saw-tooth wave, rectified sine wave, and modulus of sine wave. An analytical threshold condition for the onset of horseshoe chaos is obtained in the Duffing oscillator driven by various periodic forces using the Melnikov method. Melnikov threshold curve is drawn in a parameter space. For all the forces except modulus of sine wave, the onset of cross-well asymptotic chaos is observed just above the Melnikov threshold curve for onset of horseshoe chaos. For the modulus of sine wave long time transient motion followed by a periodic attractor is realized. The possibility of controlling of horseshoe and asymptotic chaos in the Duffing oscillator by an addition of second periodic force is then analyzed. Parametric regimes where suppression of horseshoe chaos occurs are predicted. Analytical prediction is demonstrated through direct numerical simulations. Starting from asymptotic chaos we show the recovery of periodic motion for a range of values of amplitude and phase of the second periodic force. Interestingly, suppression of chaos is found in the parametric regimes where the Melnikov function does not change sign.

nlin.CD↗

arXiv2007

Enhancing synchronization in complex networks of coupled phase oscillators

By a model of coupled phase oscillators, we show analytically how synchronization in {\em non-identical} complex networks can be enhanced by introducing a proper gradient into the couplings. It is found that, by pointing the gradient from the large-degree to the small-degree nodes on each link, increasing the gradient strength will bring forward the {\em onset} of network synchronization monotonically, and, under the same gradient strength, heterogeneous networks are more synchronizable than homogeneous networks. These findings are verified by extensive simulations.

nlin.CD↗

arXiv2007

Optimization of synchronization in gradient clustered networks

We consider complex clustered networks with a gradient structure, where sizes of the clusters are distributed unevenly. Such networks describe more closely actual networks in biophysical systems and in technological applications than previous models. Theoretical analysis predicts that the network synchronizability can be optimized by the strength of the gradient field but only when the gradient field points from large to small clusters. A remarkable finding is that, if the gradient field is sufficiently strong, synchronizability of the network is mainly determined by the properties of the subnetworks in the two largest clusters. These results are verified by numerical eigenvalue analysis and by direct simulation of synchronization dynamics on coupled-oscillator networks.

nlin.CD↗

arXiv2007

The multiple effects of gradient coupling on network synchronization

Recent studies have shown that synchronizability of complex networks can be significantly improved by asymmetric couplings, and increase of coupling gradient is always in favor of network synchronization. Here we argue and demonstrate that, for typical complex networks, there usually exists an optimal coupling gradient under which the maximum network synchronizability is achieved. After this optimal value, increase of coupling gradient could deteriorate synchronization. We attribute the suppression of network synchronization at large gradient to the phenomenon of network breaking, and find that, in comparing with sparsely connected homogeneous networks, densely connected heterogeneous networks have the superiority of adopting large gradient. The findings are supported by indirect simulations of eigenvalue analysis and direct simulations of coupled nonidentical oscillator networks.

nlin.AO↗

arXiv2007

Nonstationary pattern in unsynchronizable complex networks

Pattern formation and evolution in unsynchronizable complex networks are investigated. Due to the asymmetric topology, the synchronous patterns formed in complex networks are irregular and nonstationary. For coupling strength immediately out of the synchronizable region, the typical phenomenon is the on-off intermittency of the system dynamics. The patterns appeared in this process are signatured by the coexistence of a giant cluster, which comprises most of the nodes, and a few number of small clusters. The pattern evolution is characterized by the giant cluster irregularly absorbs or emits the small clusters. As the coupling strength leaves away from the synchronization bifurcation point, the giant cluster is gradually dissolved into a number of small clusters, and the system dynamics is characterized by the integration and separation of the small clusters. Dynamical mechanisms and statistical properties of the nonstationary pattern evolution are analyzed and conducted, and some scalings are newly revealed. Remarkably, it is found that the few active nodes, which escape from the giant cluster with a high frequency, are independent of the coupling strength while are sensitive to the bifurcation types. We hope our findings about nonstationary pattern could give additional understandings to the dynamics of complex systems and have implications to some real problems where systems maintain their normal functions only in the unsynchronizable state.

nlin.CD↗

arXiv2007

Synchronization in Gradient Networks

The contradiction between the fact that many empirical networks possess power-law degree distribution and the finding that network of heterogeneous degree distribution is difficult to synchronize has been a paradox in the study of network synchronization. Surprisingly, we find that this paradox can be well fixed when proper gradients are introduced to the network links, i.e. heterogeneity is in favor of synchronization in gradient networks. We analyze the statistical properties of gradient networks and explore their dependence to the other network parameters. Based on these understandings, we further propose a new scheme for network synchronization distinguished by using less network information while reaching stronger synchronizability, as supported by analytical estimates of eigenvalues and directed simulations of coupled chaotic oscillators. Our findings suggest that, with gradient, scale-free network is a natural choice for synchronization.

nlin.CD↗