Searcharxiv⌕ Search

arXiv subjects

Menachem Stern

Publications and source records attributed to Menachem Stern.

At least 19 recordsLinked to original sources
arXiv2026

Contrastive learning in tunable dynamical systems

We generalize the theory of supervised contrastive learning, previously applied to physical systems at equilibrium or steady state, to systems following any dynamics described by coupled ordinary differential equations. We show that if physical dynamics break time reversal symmetry, gradient descent on a cost function embodying the desired behavior cannot be achieved with a scalable process, even in principle. We therefore introduce Probably Approximately Right (PAR) learning processes, composed of a local contrastive learning rule and a scalable supervision protocol. We show that approximate, local supervision with forward propagation of the error signal can be used to successfully train several tunable models of physical dynamics inspired by examples in biological and machine learning.

cond-mat.dis-nn↗
arXiv2026

Learning Associations in Reconfigurable Particle Packings via Local Cyclic Driving

We investigate associative-memory behavior in a reconfigurable particle packing programmed by purely local cyclic driving. The system is a two-dimensional bidisperse Lennard--Jones particle assembly with periodic boundaries evolved under athermal quasistatic relaxation. During training, a fixed set of input particles is driven cyclically while output particles are selected on-the-fly by a region-driving rule and driven according to a prescribed flow pattern; during retrieval, only the inputs are driven. Associative-memory performance is quantified by the cosine similarity between realized and target output displacement directions. Unlike physical learning systems with fixed architecture, learning here arises through emergent weight updates: localized rearrangements modify the contact network and reshape the effective mechanical couplings between inputs and outputs. Across task difficulty we identify three regimes. In an easy setting, the intrinsic mechanical response already produces coherent motion in the right-hand region under input-only driving, yielding high performance without training. In a hard setting, the desired mapping conflicts with the dominant collective drift, resulting in low baseline performance and only modest training gains; introducing intermittent relaxation cycles reduces train--retrieval mismatch and improves performance. In an intermediate quadrupolar task, repositioning the input--output geometry stabilizes the desired response and converts initially stochastic trajectories into reproducible learned motions. Together these results identify minimal physical ingredients for association-based functionality in athermally driven particulate media and motivate an association learning phase diagram for reconfigurable matter.

cond-mat.dis-nn↗
arXiv2025

Harnessing intuitive local evolution rules for physical learning

Machine Learning, however popular and accessible, is computationally intensive and highly power-consuming, prompting interest in alternative physical implementations of learning tasks. We introduce a training scheme for physical systems that minimize power dissipation in which only boundary parameters (i.e. inputs and outputs) are externally controlled. Using this scheme, these Boundary-Enabled Adaptive State Tuning Systems (BEASTS) learn by exploiting local physical rules. Our scheme, BEASTAL (BEAST-Adaline), is the closest analog of the Adaline algorithm for such systems. We demonstrate this autonomous learning in silico for regression and classification tasks. Our approach advances previous physical learning schemes by using intuitive, local evolution rules without requiring large-scale memory or complex internal architectures. BEASTAL can perform any linear task, achieving best performance when the local evolution rule is non-linear.

cs.LG↗
arXiv2024

Microscopic imprints of learned solutions in adaptive resistor networks

In physical networks trained using supervised learning, physical parameters are adjusted to produce desired responses to inputs. An example is electrical contrastive local learning networks of nodes connected by edges that are resistors that adjust their conductances during training. When an edge conductance changes, it upsets the current balance of every node. In response, physics adjusts the node voltages to minimize the dissipated power. Learning in these systems is therefore a coupled double-optimization process, in which the network descends both a cost landscape in the high-dimensional space of edge conductances, and a physical landscape -- the power -- in the high-dimensional space of node voltages. Because of this coupling, the physical landscape of a trained network contains information about the learned task. Here we demonstrate that all the physical information relevant to the trained input-output relation can be captured by a susceptibility, an experimentally measurable quantity. We supplement our theoretical results with simulations to show that the susceptibility is positively correlated with functional importance and that we can extract physical insight into how the system performs the task from the conductances of highly susceptible edges.

cond-mat.dis-nn↗
arXiv2024

Physical networks become what they learn

Physical networks can develop diverse responses, or functions, by design, evolution or learning. We focus on electrical networks of nodes connected by resistive edges. Such networks can learn by adapting edge conductances to lower a cost function that penalizes deviations from a desired response. The network must also satisfy Kirchhoff's law, balancing currents at nodes, or, equivalently, minimizing total power dissipation by adjusting node voltages. The adaptation is thus a double optimization process, in which a cost function is minimized with respect to conductances, while dissipated power is minimized with respect to node voltages. Here we study how this physical adaptation couples the cost landscape, the landscape of the cost function in the high-dimensional space of edge conductances, to the physical landscape, the dissipated power in the high-dimensional space of node voltages. We show how adaptation links the physical and cost Hessian matrices, suggesting that the physical response of networks to perturbations holds significant information about the functions to which they are adapted.

cond-mat.dis-nn↗
arXiv2023

Applying statistical learning theory to deep learning

Although statistical learning theory provides a robust framework to understand supervised learning, many theoretical aspects of deep learning remain unclear, in particular how different architectures may lead to inductive bias when trained using gradient based methods. The goal of these lectures is to provide an overview of some of the main questions that arise when attempting to understand deep learning from a learning theory perspective. After a brief reminder on statistical learning theory and stochastic optimization, we discuss implicit bias in the context of benign overfitting. We then move to a general description of the mirror descent algorithm, showing how we may go back and forth between a parameter space and the corresponding function space for a given learning problem, as well as how the geometry of the learning problem may be represented by a metric tensor. Building on this framework, we provide a detailed study of the implicit bias of gradient descent on linear diagonal networks for various regression tasks, showing how the loss function, scale of parameters at initialization and depth of the network may lead to various forms of implicit bias, in particular transitioning between kernel or feature learning.

cs.LG↗
arXiv2023

Machine Learning Without a Processor: Emergent Learning in a Nonlinear Electronic Metamaterial

Standard deep learning algorithms require differentiating large nonlinear networks, a process that is slow and power-hungry. Electronic learning metamaterials offer potentially fast, efficient, and fault-tolerant hardware for analog machine learning, but existing implementations are linear, severely limiting their capabilities. These systems differ significantly from artificial neural networks as well as the brain, so the feasibility and utility of incorporating nonlinear elements have not been explored. Here we introduce a nonlinear learning metamaterial -- an analog electronic network made of self-adjusting nonlinear resistive elements based on transistors. We demonstrate that the system learns tasks unachievable in linear systems, including XOR and nonlinear regression, without a computer. We find our nonlinear learning metamaterial reduces modes of training error in order (mean, slope, curvature), similar to spectral bias in artificial neural networks. The circuitry is robust to damage, retrainable in seconds, and performs learned tasks in microseconds while dissipating only picojoules of energy across each transistor. This suggests enormous potential for fast, low-power computing in edge systems like sensors, robotic controllers, and medical devices, as well as manufacturability at scale for performing and studying emergent learning.

cond-mat.soft↗
arXiv2023

Experimental Demonstration of Coupled Learning in Elastic Networks

Coupled learning is a contrastive scheme for tuning the properties of individual elements within a network in order to achieve desired functionality of the system. It takes advantage of physics both to learn using local rules and to "compute" the output response to input data, thus enabling the system to perform decentralized computation without the need for a processor or external memory. We demonstrate a proof-of-concept mechanical network that can learn simple tasks such as self-symmetrizing via iterative tuning of individual spring rest lengths. These mechanical networks could feasibly be scaled and automated to solve increasingly complex tasks, hinting at a new class of smart metamaterials.

cond-mat.soft↗
arXiv2023

Physical learning of power-efficient solutions

As the size and ubiquity of artificial intelligence and computational machine learning (ML) models grow, their energy consumption for training and use is rapidly becoming economically and environmentally unsustainable. Neuromorphic computing, or the implementation of ML in hardware, has the potential to reduce this cost. In particular, recent laboratory prototypes of self-learning electronic circuits, examples of ``physical learning machines," open the door to analog hardware that directly employs physics to learn desired functions from examples. In this work, we show that this hardware platform allows for even further reduction of energy consumption by using good initial conditions as well as a new learning algorithm. Using analytical calculations, simulation and experiment, we show that a trade-off emerges when learning dynamics attempt to minimize both the error and the power consumption of the solution--greater power reductions can be achieved at the cost of decreasing solution accuracy. Finally, we demonstrate a practical procedure to weigh the relative importance of error and power minimization, improving power efficiency given a specific tolerance to error.

cond-mat.dis-nn↗
arXiv2023

The Physical Effects of Learning

Interacting many-body physical systems ranging from neural networks in the brain to folding proteins to self-modifying electrical circuits can learn to perform diverse tasks. This learning, both in nature and in engineered systems, can occur through evolutionary selection or through dynamical rules that drive active learning from experience. Here, we show that \added{learning in linear physical networks with weak input signals} leaves architectural imprints on the Hessian of a physical system. Compared to a generic organization of the system components, (a) the effective physical dimension of the response to inputs decreases, (b) the response of physical degrees of freedom to random perturbations (or system ``susceptibility'') increases, and (c) the low-eigenvalue eigenvectors of the Hessian align with the task. Overall, these effects embody the typical scenario for learning processes in physical systems in the weak input regime, suggesting ways of discovering whether a physical network may have been trained.

cond-mat.dis-nn↗
arXiv2022

Learning to self-fold at a bifurcation

Disordered mechanical systems can deform along a network of pathways that branch and recombine at special configurations called bifurcation points. Multiple pathways are accessible from these bifurcation points; consequently, computer-aided design algorithms have been sought to achieve a specific structure of pathways at bifurcations by rationally designing the geometry and material properties of these systems. Here, we explore an alternative physical training framework in which the topology of folding pathways in a disordered sheet is changed in a desired manner due to changes in crease stiffnesses induced by prior folding. We study the quality and robustness of such training for different `learning rules', that is, different quantitative ways in which local strain changes the local folding stiffness. We experimentally demonstrate these ideas using sheets with epoxy-filled creases whose stiffnesses change due to folding before the epoxy sets. Our work shows how specific forms of plasticity in materials enable them to learn non-linear behaviors through their prior deformation history in a robust manner.

cond-mat.soft↗
arXiv2022

Learning without neurons in physical systems

Learning is traditionally studied in biological or computational systems. The power of learning frameworks in solving hard inverse-problems provides an appealing case for the development of `physical learning' in which physical systems adopt desirable properties on their own without computational design. It was recently realized that large classes of physical systems can physically learn through local learning rules, autonomously adapting their parameters in response to observed examples of use. We review recent work in the emerging field of physical learning, describing theoretical and experimental advances in areas ranging from molecular self-assembly to flow networks and mechanical materials. Physical learning machines provide multiple practical advantages over computer designed ones, in particular by not requiring an accurate model of the system, and their ability to autonomously adapt to changing needs over time. As theoretical constructs, physical learning machines afford a novel perspective on how physical constraints modify abstract learning theory.

cond-mat.dis-nn↗
arXiv2022

Desynchronous Learning in a Physics-Driven Learning Network

In a neuron network, synapses update individually using local information, allowing for entirely decentralized learning. In contrast, elements in an artificial neural network (ANN) are typically updated simultaneously using a central processor. Here we investigate the feasibility and effect of desynchronous learning in a recently introduced decentralized, physics-driven learning network. We show that desynchronizing the learning process does not degrade performance for a variety of tasks in an idealized simulation. In experiment, desynchronization actually improves performance by allowing the system to better explore the discretized state space of solutions. We draw an analogy between desynchronization and mini-batching in stochastic gradient descent, and show that they have similar effects on the learning process. Desynchronizing the learning process establishes physics-driven learning networks as truly fully distributed learning machines, promoting better performance and scalability in deployment.

cond-mat.soft↗
arXiv2021

Physical learning beyond the quasistatic limit

Physical networks, such as biological neural networks, can learn desired functions without a central processor, using local learning rules in space and time to learn in a fully distributed manner. Learning approaches such as equilibrium propagation, directed aging, and coupled learning similarly exploit local rules to accomplish learning in physical networks such as mechanical, flow, or electrical networks. In contrast to certain natural neural networks, however, such approaches have so far been restricted to the quasistatic limit, where they learn on time scales slow compared to their physical relaxation. This quasistatic constraint slows down learning, limiting the use of these methods as machine learning algorithms, and potentially restricting physical networks that could be used as learning platforms. Here we explore learning in an electrical resistor network that implements coupled learning, both in the lab and on the computer, at rates that range from slow to far above the quasistatic limit. We find that up to a critical threshold in the ratio of the learning rate to the physical rate of relaxation, learning speeds up without much change of behavior or error. Beyond the critical threshold, the error exhibits oscillatory dynamics but the networks still learn successfully.

cond-mat.dis-nn↗
arXiv2021

Demonstration of Decentralized, Physics-Driven Learning

In typical artificial neural networks, neurons adjust according to global calculations of a central processor, but in the brain neurons and synapses self-adjust based on local information. Contrastive learning algorithms have recently been proposed to train physical systems, such as fluidic, mechanical, or electrical networks, to perform machine learning tasks from local evolution rules. However, to date such systems have only been implemented in silico due to the engineering challenge of creating elements that autonomously evolve based on their own response to two sets of global boundary conditions. Here we introduce and implement a physics-driven contrastive learning scheme for a network of variable resistors, using circuitry to locally compare the response of two identical networks subjected to the two different sets of boundary conditions. Using this innovation, our system effectively trains itself, optimizing its resistance values without use of a central processor or external information storage. Once the system is trained for a specified allostery, regression, or classification task, the task is subsequently performed rapidly and automatically by the physical imperative to minimize power dissipation in response to the given voltage inputs. We demonstrate that, unlike typical computers, such learning systems are robust to extreme damage (and thus manufacturing defects) due to their decentralized learning. Our twin-network approach is therefore readily scalable to extremely large or nonlinear networks where its distributed nature will be an enormous advantage; a laboratory network of only 500 edges will already outpace its in silico counterpart.

cond-mat.dis-nn↗
arXiv2020

Supervised learning in physical networks: From machine learning to learning machines

Materials and machines are often designed with particular goals in mind, so that they exhibit desired responses to given forces or constraints. Here we explore an alternative approach, namely physical coupled learning. In this paradigm, the system is not initially designed to accomplish a task, but physically adapts to applied forces to develop the ability to perform the task. Crucially, we require coupled learning to be facilitated by physically plausible learning rules, meaning that learning requires only local responses and no explicit information about the desired functionality. We show that such local learning rules can be derived for any physical network, whether in equilibrium or in steady state, with specific focus on two particular systems, namely disordered flow networks and elastic networks. By applying and adapting advances of statistical learning theory to the physical world, we demonstrate the plausibility of new classes of smart metamaterials capable of adapting to users' needs in-situ.

cond-mat.soft↗
arXiv2019

Supervised learning in a mechanical system

Mechanical metamaterials are usually designed to show desired responses to prescribed forces. In some applications, the desired force-response relationship might be hard to specify exactly, although examples of forces and corresponding desired responses are easily available. Here we propose a framework for supervised learning in a thin creased sheet that learns the desired force-response behavior from training examples of spatial force patterns and can then respond correctly to previously unseen test forces. During training, we fold the sheet using different training forces and assume a learning rule that changes stiffness of creases in response to their folding strain. We find that this learning process reshapes non-linearities inherent in folding a sheet so as to show the correct response for previously unseen test forces. We study the relationship between training error, test error and sheet size which plays the role of model complexity. Our framework shows how the complex energy landscape of disordered mechanical materials can be reshaped using an iterative local learning rule.

cond-mat.soft↗
arXiv2019

Learned multi-stability in mechanical networks

We contrast the distinct frameworks of materials design and physical learning in creating elastic networks with desired stable states. In design, the desired states are specified in advance and material parameters can be optimized on a computer with this knowledge. In learning, the material physically experiences the desired stable states in sequence, changing the material so as to stabilize each additional state. We show that while designed states are stable in networks of linear Hookean springs, sequential learning requires specific non-linear elasticity. We find that such non-linearity stabilizes states in which strain is zero in some springs and large in others, thus playing the role of Bayesian priors used in sparse statistical regression. Our model shows how specific material properties allow continuous learning of new functions through deployment of the material itself.

cond-mat.soft↗