arXiv subjects

K. Y. Michael Wong

Publications and source records attributed to K. Y. Michael Wong.

At least 19 recordsLinked to original sources

arXiv2026

Optimality and annealing path planning of dynamical analog solvers

Recently proposed analog solvers based on dynamical systems, such as Ising machines, are promising platforms for large-scale combinatorial optimization. Yet, given the heuristic nature of the field, there is very limited insight on optimality guarantees of the solvers, as well as how parameter schedules shape dynamics and outcomes. Here, we develop a dynamical mean-field framework to analyze Ising-machine dynamics for finding the ground state energy of the Sherrington-Kirkpatrick(SK) model of spin glasses and identify mechanisms that enable rapid convergence to provenly near-optimal energies. For a fixed target energy density Ec, we show that solutions are typically reached within O(1) matrix vector multiplications, indicating constant time complexity. We further delineate theoretical limitations arising from different parameter-scheduling trajectories and demonstrate a pronounced benefit of temperature-only annealing for the Coherent Ising Machine. Building on these insights, we propose a general framework for designing optimized parameter schedules, thereby improving the practical effectiveness of Ising machines for complex optimization tasks. The superior performance of the dynamical solvers is illustrated by the attainment of the ground state energy of the SK model.

cond-mat.dis-nn↗

arXiv2025

Phase analysis of Ising machines and their implications on optimization

Ising machines, which are dynamical systems designed to operate in a parallel and iterative manner, have emerged as a new paradigm for solving combinatorial optimization problems. Despite computational advantages, the quality of solutions depends heavily on the form of dynamics and tuning of parameters, which are in general set heuristically due to the lack of systematic insights. Here, we focus on optimal Ising machine design by analyzing phase diagrams of spin distributions in the Sherrington-Kirkpatrick model. We find that that the ground state can be achieved in the phase where the spin distribution becomes binary, and optimal solutions are produced where the binary phase and gapless phase coexist. Our analysis shows that such coexistence phase region can be expanded by carefully placing a digitization operation, giving rise to a family of superior Ising machines, as illustrated by the proposed algorithm digCIM.

cond-mat.stat-mech↗

arXiv2023

Bimodular continuous attractor neural networks with static and moving stimuli

We investigated the dynamical behaviors of bimodular continuous attractor neural networks, each processing a modality of sensory input and interacting with each other. We found that when bumps coexist in both modules, the position of each bump is shifted towards the other input when the intermodular couplings are excitatory and is shifted away when inhibitory. When one intermodular coupling is excitatory while another is moderately inhibitory, temporally modulated population spikes can be generated. On further increase of the inhibitory coupling, momentary spikes will emerge. In the regime of bump coexistence, bump heights are primarily strengthened by excitatory intermodular couplings, but there is a lesser weakening effect due to a bump being displaced from the direct input. When bimodular networks serve as decoders of multisensory integration, we extend the Bayesian framework to show that excitatory and inhibitory couplings encode attractive and repulsive priors, respectively. At low disparity, the bump positions decode the posterior means in the Bayesian framework, whereas at high disparity, multiple steady states exist. In the regime of multiple steady states, the less stable state can be accessed if the input causing the more stable state arrives after a sufficiently long delay. When one input is moving, the bump in the corresponding module is pinned when the moving stimulus is weak, unpinned at intermediate stimulus strength, and tracks the input at strong stimulus strength, and the stimulus strengths for these transitions increase with the velocity of the moving stimulus. These results are important to understanding multisensory integration of static and dynamic stimuli.

q-bio.NC↗

arXiv2021

Associative memory model with arbitrary Hebbian length

Conversion of temporal to spatial correlations in the cortex is one of the most intriguing functions in the brain. The learning at synapses triggering the correlation conversion can take place in a wide integration window, whose influence on the correlation conversion remains elusive. Here, we propose a generalized associative memory model with arbitrary Hebbian length. The model can be analytically solved, and predicts that a small Hebbian length can already significantly enhance the correlation conversion, i.e., the stimulus-induced attractor can be highly correlated with a significant number of patterns in the stored sequence, thereby facilitating state transitions in the neural representation space. Moreover, an anti-Hebbian component is able to reshape the energy landscape of memories, akin to the function of sleep. Our work thus establishes the fundamental connection between associative memory, Hebbian length, and correlation conversion in the brain.

cond-mat.dis-nn↗

arXiv2020

Bandwidth Allocation and Resource Adjustment for Stability Enhancement in Complex Networks

We introduce the discrete Green's function to elucidate how resource fluctuations determine flow fluctuations in a network optimizing a global cost function. To enhance the robustness of the network against fluctuations, we develop the schemes of optimal bandwidth allocation in links and optimal resource adjustment in nodes. With the total bandwidth of the network fixed, the approach of optimal bandwidth allocation is to increase the bandwidth in links such that the number of overloaded links or the amount of excess flows in networks under fluctuations can be minimized. Similarly, the approach of optimal resource adjustment is to minimize the number of overloaded links in networks under fluctuations with the total resource change in the network fixed. Compared with the conventional approach of proportionate bandwidth assignment or resource reduction, it is found that the optimized bandwidth allocation or resource adjustment can highly enhance the stability of the networks against fluctuations. The changes of loads and currents prescribed by the optimal bandwidth allocation and resource adjustment schemes are correlated with each other, except for some nodes that exhibit relay effects.

physics.soc-ph↗

arXiv2019

Minimal model of permutation symmetry in unsupervised learning

Permutation of any two hidden units yields invariant properties in typical deep generative neural networks. This permutation symmetry plays an important role in understanding the computation performance of a broad class of neural networks with two or more hidden units. However, a theoretical study of the permutation symmetry is still lacking. Here, we propose a minimal model with only two hidden units in a restricted Boltzmann machine, which aims to address how the permutation symmetry affects the critical learning data size at which the concept-formation (or spontaneous symmetry breaking in physics language) starts, and moreover semi-rigorously prove a conjecture that the critical data size is independent of the number of hidden units once this number is finite. Remarkably, we find that the embedded correlation between two receptive fields of hidden units reduces the critical data size. In particular, the weakly-correlated receptive fields have the benefit of significantly reducing the minimal data size that triggers the transition, given less noisy data. Inspired by the theory, we also propose an efficient fully-distributed algorithm to infer the receptive fields of hidden units. Furthermore, our minimal model reveals that the permutation symmetry can also be spontaneously broken following the spontaneous symmetry breaking. Overall, our results demonstrate that the unsupervised learning is a progressive combination of spontaneous symmetry breaking and permutation symmetry breaking which are both spontaneous processes driven by data streams (observations). All these effects can be analytically probed based on the minimal model, providing theoretical insights towards understanding unsupervised learning in a more general context.

cond-mat.dis-nn↗

arXiv2018

Segment-wise Description of the Dynamics of Traffic Congestion

We compare the point-wise and segment-wise descriptions of the traffic system. Using real data from the Taiwan highway system with a tremendous volume of segment-wise data, we find that the segment-wise description is much more informative of the evolution of the system during congestion. Congestion is characterized by a loopy trajectory in the fundamental diagram. By considering the area enclosed by the loop, we find that there are two types of congestion dynamics -- moderate flow and serious congestion. They are different in terms of whether the area enclosed vanishes. Data extracted from the time delays of individual vehicles show that the area enclosed is a measure of the economic loss due to congestion. The use of the loss area in helping to understand various road characteristics is also explored.

physics.soc-ph↗

arXiv2018

How Market Structure Drives Commodity Prices

We introduce an agent-based model, in which agents set their prices to maximize profit. At steady state the market self-organizes into three groups: excess producers, consumers and balanced agents, with prices determined by their own resource level and a couple of macroscopic parameters that emerge naturally from the analysis, akin to mean-field parameters in statistical mechanics. When resources are scarce prices rise sharply below a turning point that marks the disappearance of excess producers. To compare the model with real empirical data, we study the relations between commodity prices and stock-to-use ratios of a range of commodities such as agricultural products and metals. By introducing an elasticity parameter to mitigate noise and long-term changes in commodities data, we confirm the trend of rising prices, provide evidence for turning points, and indicate yield points for less essential commodities.

q-fin.GN↗

arXiv2017

Optimizing Synchronization Stability of the Kuramoto Model in Complex Networks and Power Grids

Maintaining the stability of synchronization state is crucial for the functioning of many natural and artificial systems. In this study, we develop methods to optimize the synchronization stability of the Kuramoto model by minimizing the dominant Lyapunov exponent. With the help of the recently proposed cut-set space approximation of the steady states, we greatly simplify the objective function, and further derive its gradient and Hessian with respect to natural frequencies, which leads to an efficient algorithm with the quasi-Newton's method. The optimized systems are demonstrated to achieve better synchronization stability for the Kuramoto model with or without inertia in certain regimes. Hence our method is applicable in improving the stability of power grids. It is also viable to adjust the coupling strength of each link to improve the stability of the system. Various operational constraints can also be easily integrated into our scope by employing the interior point method in convex optimization. The properties of the optimized networks are also discussed.

math.OC↗

arXiv2016

Evolving Powergrids in Self-Organized Criticality: An analogy with Sandpile and Earthquakes

The stability of powergrid is crucial since its disruption affects systems ranging from street lightings to hospital life-support systems. Nevertheless, large blackouts are inevitable if powergrids are in the state of self-organized criticality (SOC). In this paper, we introduce a simple model of evolving powergrid and establish its connection with the sandpile model, i.e. a prototype of SOC, and earthquakes, i.e. a system considered to be in SOC. Various aspects are examined, including the power-law distribution of blackout magnitudes, their inter-event waiting time, the predictability of large blackouts, as well as the spatial-temporal rescaling of blackout data. We verified our observations on simulated networks as well as the IEEE 118-bus system, and show that both simulated and empirical blackout waiting times can be rescaled in space and time similarly to those observed between earthquakes. Finally, we suggested proactive maintenance strategies to drive the powergrids away from SOC to suppress large blackouts.

physics.soc-ph↗

arXiv2015

Fluctuation-response Relation Unifies Dynamical Behaviors in Neural Fields

Anticipation is a strategy used by neural fields to compensate for transmission and processing delays during the tracking of dynamical information, and can be achieved by slow, localized, inhibitory feedback mechanisms such as short-term synaptic depression, spike-frequency adaptation, or inhibitory feedback from other layers. Based on the translational symmetry of the mobile network states, we derive generic fluctuation-response relations, providing unified predictions that link their tracking behaviors in the presence of external stimuli to the intrinsic dynamics of the neural fields in their absence.

cond-mat.dis-nn↗

arXiv2015

A Rich Spectrum of Neural Field Dynamics in the Presence of Short-Term Synaptic Depression

In continuous attractor neural networks (CANNs), spatially continuous information such as orientation, head direction, and spatial location is represented by Gaussian-like tuning curves that can be displaced continuously in the space of the preferred stimuli of the neurons. We investigate how short-term synaptic depression (STD) can reshape the intrinsic dynamics of the CANN model and its responses to a single static input. In particular, CANNs with STD can support various complex firing patterns and chaotic behaviors. These chaotic behaviors have the potential to encode various stimuli in the neuronal system.

q-bio.NC↗

arXiv2015

Dynamics of Neural Networks with Continuous Attractors

We investigate the dynamics of continuous attractor neural networks (CANNs). Due to the translational invariance of their neuronal interactions, CANNs can hold a continuous family of stationary states. We systematically explore how their neutral stability facilitates the tracking performance of a CANN, which is believed to have wide applications in brain functions. We develop a perturbative approach that utilizes the dominant movement of the network stationary states in the state space. We quantify the distortions of the bump shape during tracking, and study their effects on the tracking performance. Results are obtained on the maximum speed for a moving stimulus to be trackable, and the reaction time to catch up an abrupt change in stimulus.

cond-mat.dis-nn↗

arXiv2014

Coverage versus Supply Cost in Facility Location: Physics of Frustrated Spin Systems

A comprehensive coverage is crucial for communication, supply and transportation networks, yet it is limited by the requirement of extensive infrastructure and heavy energy consumption. Here we draw an analogy between spins in antiferromagnet and outlets in supply networks, and apply techniques from the studies of disordered systems to elucidate the effects of balancing the coverage and supply costs on the network behavior. A readily applicable, coverage optimization algorithm is derived. Simulation results show that magnetized and antiferromagnetic domains emerge and coexist to balance the need for coverage and energy saving. The scaling of parameters with system size agrees with the continuum approximation in two dimensions and the tree approximation in random graphs. Due to frustration caused by the competition between coverage and supply cost, a transition between easy and hard computation regimes is observed. We further suggest a local expansion approach to greatly simplify the message updates which shed light on simplifications in other problems.

physics.soc-ph↗

arXiv2014

The network source location problem: ground state energy, entropy and effects of freezing

Ground state entropy of the network source location problem is evaluated at both the replica symmetric level and one-step replica symmetry breaking level using the entropic cavity method. The regime that is a focus of this study, is closely related to the vertex cover problem with randomly quenched covered nodes. The resulting entropic message passing inspired decimation and reinforcement algorithms are used to identify the optimal location of sources in single instances of transportation networks. The conventional belief propagation without taking the entropic effect into account is also compared. We find that in the glassy phase the entropic message passing inspired decimation yields a lower ground state energy compared to the belief propagation without taking the entropic effect. Using the extremal optimization algorithm, we study the ground state energy and the fraction of frozen hubs, and extend the algorithm to collect statistics of the entropy. The theoretical results are compared with the extremal optimization results.

cond-mat.dis-nn↗

arXiv2013

From the Physics of Interacting Polymers to Optimizing Routes on the London Underground

Optimizing paths on networks is crucial for many applications, from subway traffic to Internet communication. As global path optimization that takes account of all path-choices simultaneously is computationally hard, most existing routing algorithms optimize paths individually, thus providing sub-optimal solutions. We employ the physics of interacting polymers and disordered systems to analyze macroscopic properties of generic path-optimization problems and derive a simple, principled, generic and distributed routing algorithm capable of considering simultaneously all individual path choices. We demonstrate the efficacy of the new algorithm by applying it to: (i) random graphs resembling Internet overlay networks; (ii) travel on the London underground network based on Oyster-card data; and (iii) the global airport network. Analytically derived macroscopic properties give rise to insightful new routing phenomena, including phase transitions and scaling laws, which facilitate better understanding of the appropriate operational regimes and their limitations that are difficult to obtain otherwise.

physics.soc-ph↗

arXiv2013

Entropy landscape of solutions in the binary perceptron problem

The statistical picture of the solution space for a binary perceptron is studied. The binary perceptron learns a random classification of input random patterns by a set of binary synaptic weights. The learning of this network is difficult especially when the pattern (constraint) density is close to the capacity, which is supposed to be intimately related to the structure of the solution space. The geometrical organization is elucidated by the entropy landscape from a reference configuration and of solution-pairs separated by a given Hamming distance in the solution space. We evaluate the entropy at the annealed level as well as replica symmetric level and the mean field result is confirmed by the numerical simulations on single instances using the proposed message passing algorithms. From the first landscape (a random configuration as a reference), we see clearly how the solution space shrinks as more constraints are added. From the second landscape of solution-pairs, we deduce the coexistence of clustering and freezing in the solution space.

cond-mat.dis-nn↗

arXiv2013

Resolution enhancement in neural networks with dynamical synapses

Conventionally, information is represented by spike rates in the neural system. Here, we consider the ability of temporally modulated activities in neuronal networks to carry information extra to spike rates. These temporal modulations, commonly known as population spikes, are due to the presence of synaptic depression in a neuronal network model. We discuss its relevance to an experiment on transparent motions in macaque monkeys by Treue et al. in 2000. They found that if the moving directions of objects are too close, the firing rate profile will be very similar to that with one direction. As the difference in the moving directions of objects is large enough, the neuronal system would respond in such a way that the network enhances the resolution in the moving directions of the objects. In this paper, we propose that this behavior can be reproduced by neural networks with dynamical synapses when there are multiple external inputs. We will demonstrate how resolution enhancement can be achieved, and discuss the conditions under which temporally modulated activities are able to enhance information processing performances in general.

q-bio.NC↗