arXiv subjects

Raffaele Marino

Publications and source records attributed to Raffaele Marino.

At least 37 records · Page 2Linked to original sources

arXiv2024

Fast Analysis of the OpenAI O1-Preview Model in Solving Random K-SAT Problem: Does the LLM Solve the Problem Itself or Call an External SAT Solver?

In this manuscript, I present an analysis on the performance of OpenAI O1-preview model in solving random K-SAT instances for K$\in {2,3,4}$ as a function of $α=M/N$ where $M$ is the number of clauses and $N$ is the number of variables of the satisfiable problem. I show that the model can call an external SAT solver to solve the instances, rather than solving them directly. Despite using external solvers, the model reports incorrect assignments as output. Moreover, I propose and present an analysis to quantify whether the OpenAI O1-preview model demonstrates a spark of intelligence or merely makes random guesses when outputting an assignment for a Boolean satisfiability problem.

cs.CL↗

arXiv2024

Evidence of dual energy transfer driven by magnetic reconnection at sub-ion scales

The properties of energy transfer in the kinetic range of plasma turbulence have fundamental implications on the turbulent heating of space and astrophysical plasmas. It was recently suggested that magnetic reconnection may be responsible for driving the sub-ion scale cascade, and that this process would be characterized by a direct energy transfer towards even smaller scales (until dissipation), and a simultaneous inverse transfer of energy towards larger scales, until the ion break. Here we employ the space-filter technique on high-resolution 2D3V hybrid-Vlasov simulations of continuously driven turbulence providing for the first time quantitative evidence that magnetic reconnection is indeed able to trigger a dual energy transfer originating at sub-ion scales.

physics.plasm-ph↗

arXiv2024

Stochastic Gradient Descent-like relaxation is equivalent to Metropolis dynamics in discrete optimization and inference problems

Is Stochastic Gradient Descent (SGD) substantially different from Metropolis Monte Carlo dynamics? This is a fundamental question at the time of understanding the most used training algorithm in the field of Machine Learning, but it received no answer until now. Here we show that in discrete optimization and inference problems, the dynamics of an SGD-like algorithm resemble very closely that of Metropolis Monte Carlo with a properly chosen temperature, which depends on the mini-batch size. This quantitative matching holds both at equilibrium and in the out-of-equilibrium regime, despite the two algorithms having fundamental differences (e.g.\ SGD does not satisfy detailed balance). Such equivalence allows us to use results about performances and limits of Monte Carlo algorithms to optimize the mini-batch size in the SGD-like algorithm and make it efficient at recovering the signal in hard inference problems.

cond-mat.dis-nn↗

arXiv2024

Stable Attractors for Neural networks classification via Ordinary Differential Equations (SA-nODE)

A novel approach for supervised classification is presented which sits at the intersection of machine learning and dynamical systems theory. At variance with other methodologies that employ ordinary differential equations for classification purposes, the untrained model is a priori constructed to accommodate for a set of pre-assigned stationary stable attractors. Classifying amounts to steer the dynamics towards one of the planted attractors, depending on the specificity of the processed item supplied as an input. Asymptotically the system will hence converge on a specific point of the explored multi-dimensional space, flagging the category of the object to be eventually classified. Working in this context, the inherent ability to perform classification, as acquired ex post by the trained model, is ultimately reflected in the shaped basin of attractions associated to each of the target stable attractors. The performance of the proposed method is here challenged against simple toy models crafted for the purpose, as well as by resorting to well established reference standards. Although this method does not reach the performance of state-of-the-art deep learning algorithms, it illustrates that continuous dynamical systems with closed analytical interaction terms can serve as high-performance classifiers.

cond-mat.dis-nn↗

arXiv2024

Engineered Ordinary Differential Equations as Classification Algorithm (EODECA): thorough characterization and testing

EODECA (Engineered Ordinary Differential Equations as Classification Algorithm) is a novel approach at the intersection of machine learning and dynamical systems theory, presenting a unique framework for classification tasks [1]. This method stands out with its dynamical system structure, utilizing ordinary differential equations (ODEs) to efficiently handle complex classification challenges. The paper delves into EODECA's dynamical properties, emphasizing its resilience against random perturbations and robust performance across various classification scenarios. Notably, EODECA's design incorporates the ability to embed stable attractors in the phase space, enhancing reliability and allowing for reversible dynamics. In this paper, we carry out a comprehensive analysis by expanding on the work [1], and employing a Euler discretization scheme. In particular, we evaluate EODECA's performance across five distinct classification problems, examining its adaptability and efficiency. Significantly, we demonstrate EODECA's effectiveness on the MNIST and Fashion MNIST datasets, achieving impressive accuracies of $98.06\%$ and $88.21\%$, respectively. These results are comparable to those of a multi-layer perceptron (MLP), underscoring EODECA's potential in complex data processing tasks. We further explore the model's learning journey, assessing its evolution in both pre and post training environments and highlighting its ability to navigate towards stable attractors. The study also investigates the invertibility of EODECA, shedding light on its decision-making processes and internal workings. This paper presents a significant step towards a more transparent and robust machine learning paradigm, bridging the gap between machine learning algorithms and dynamical systems methodologies.

cs.LG↗

arXiv2024

Daydreaming Hopfield Networks and their surprising effectiveness on correlated data

To improve the storage capacity of the Hopfield model, we develop a version of the dreaming algorithm that perpetually reinforces the patterns to be stored (as in the Hebb rule), and erases the spurious memories (as in dreaming algorithms). For this reason, we called it Daydreaming. Daydreaming is not destructive and it converges asymptotically to stationary retrieval maps. When trained on random uncorrelated examples, the model shows optimal performance in terms of the size of the basins of attraction of stored examples and the quality of reconstruction. We also train the Daydreaming algorithm on correlated data obtained via the random-features model and argue that it spontaneously exploits the correlations thus increasing even further the storage capacity and the size of the basins of attraction. Moreover, the Daydreaming algorithm is also able to stabilize the features hidden in the data. Finally, we test Daydreaming on the MNIST dataset and show that it still works surprisingly well, producing attractors that are close to unseen examples and class prototypes.

cond-mat.dis-nn↗

arXiv2024

Comparative study of the kinetic properties of proton and alpha beams in the Alfvénic wind observed by SWA-PAS onboard Solar Orbiter

The problems of heating and acceleration of solar wind particles are of significant and enduring interest in astrophysics. The interactions between waves and particles are crucial in determining the distributions of proton and alpha particles, resulting in non-Maxwellian characteristics including temperature anisotropies and particle beams. These processes can be better understood as long as the beam can be separated from the core for the two major components of the solar wind. We utilized an alternative numerical approach that leverages the clustering technique employed in Machine Learning to differentiate the primary populations within the velocity distribution, rather than employing the conventional bi-Maxwellian fitting method. Separation of the core and beam revealed new features for protons and alphas. We estimated that the total temperature of the two beams was slightly higher than that of their respective cores, and the temperature anisotropy for the cores and beams was larger than 1. We concluded that the temperature ratio between alphas and protons largely over 4 is due to the presence of a massive alpha beam, which is approximately 50\% of the alpha core. We provided evidence that the alpha core and beam populations are sensitive to Alfvénic fluctuations and the surfing effect found in the literature can be recovered only when considering the core and beam as a single population. Several similarities between proton and alpha beams would suggest a common and local generation mechanism not shared with the alpha core, which may not have necessarily been accelerated and heated locally.

astro-ph.SR↗

arXiv2024

Nonlocal Contributions to the Turbulent Cascade in Magnetohydrodynamic Turbulence

We present evidence for nonlocal contributions to the turbulent energy cascade in magnetohydrodynamic (MHD) plasmas. Therefore, we revisit a well-known result derived directly from the MHD equations, i.e., the Politano and Pouquet (P&P) law for the transfer of kinetic and magnetic energy in scale. We propose adding a term that accounts for nonlocal transfer and represents the influence of fluctuations from large scales due to the Alfvén effect. Supported by direct numerical simulations of homogeneous and isotropic MHD turbulence, we verify that in some plasma configurations, neglecting the additional nonlocal term might consistently overestimate energy dissipation rates and, thus, the contributions of turbulent energy dissipation potentially affecting solar wind heating; a central puzzle in space plasma physics that motivates the present work.

physics.plasm-ph↗

arXiv2024

The Garden of Forking Paths: Observing Dynamic Parameters Distribution in Large Language Models

A substantial gap persists in understanding the reasons behind the exceptional performance of the Transformer architecture in NLP. A particularly unexplored area involves the mechanistic description of how the distribution of parameters evolves over time during training. In this work we suggest that looking at the time evolution of the statistic distribution of model parameters, and specifically at bifurcation effects, can help understanding the model quality, potentially reducing training costs and evaluation efforts and empirically showing the reasons behind the effectiveness of weights sparsification.

cs.CL↗

arXiv2024

Lagrangian irreversibility and energy exchanges in rotating-stratified turbulent flows

Turbulence in stratified and rotating turbulent flows is characterized by an interplay between waves and eddies, resulting in continuous exchanges between potential and kinetic energy. Here, we study how these processes affect the turbulent energy cascade from large to small scales, which manifests itself by an irreversible evolution of the relative kinetic energy between two tracer particles. We find that when $r_0$, the separation between particles, is below a characteristic length $\ell_t$, potential energy is on average transferred to kinetic energy, reducing time irreversibility, and conversely when $r_0 > \ell_t$. Our study reveals that the scale $\ell_t$ coincides with the buoyancy length scale $L_B$ over a broad range of configurations until a transitional wave-dominated regime is reached.

physics.flu-dyn↗

arXiv2024

Low-dimensional representation of intermittent geophysical turbulence with High-Order Statistics-informed Neural Networks (H-SiNN)

We present a novel machine learning approach to reduce the dimensionality of state variables in stratified turbulent flows governed by the Navier-Stokes equations in the Boussinesq approximation. The aim of the new method is to perform an accurate reconstruction of the temperature and the three-dimensional velocity of geophysical turbulent flows developing non-homogeneities, starting from a low-dimensional representation in latent space, yet conserving important information about non-Gaussian structures captured by high-order moments of distributions. To achieve this goal we modify the standard Convolutional Autoencoder (CAE) by implementing a customized loss function that enforces the accuracy of the reconstructed high order statistical moments. We present results for compression coefficients up to 16 demonstrating how the proposed method is more efficient than a standard CAE in performing dimensionality reduction of simulations of stratified geophysical flows characterized by intermittent phenomena, as observed in the atmosphere and the oceans.

physics.flu-dyn↗

arXiv2024

Phase transitions in the mini-batch size for sparse and dense two-layer neural networks

The use of mini-batches of data in training artificial neural networks is nowadays very common. Despite its broad usage, theories explaining quantitatively how large or small the optimal mini-batch size should be are missing. This work presents a systematic attempt at understanding the role of the mini-batch size in training two-layer neural networks. Working in the teacher-student scenario, with a sparse teacher, and focusing on tasks of different complexity, we quantify the effects of changing the mini-batch size $m$. We find that often the generalization performances of the student strongly depend on $m$ and may undergo sharp phase transitions at a critical value $m_c$, such that for $m m_c$ the student learns perfectly or generalizes very well the teacher. Phase transitions are induced by collective phenomena firstly discovered in statistical mechanics and later observed in many fields of science. Observing a phase transition by varying the mini-batch size across different architectures raises several questions about the role of this hyperparameter in the neural network learning process.

cond-mat.dis-nn↗

arXiv2023

Complex Recurrent Spectral Network

This paper presents a novel approach to advancing artificial intelligence (AI) through the development of the Complex Recurrent Spectral Network ($\mathbb{C}$-RSN), an innovative variant of the Recurrent Spectral Network (RSN) model. The $\mathbb{C}$-RSN is designed to address a critical limitation in existing neural network models: their inability to emulate the complex processes of biological neural networks dynamically and accurately. By integrating key concepts from dynamical systems theory and leveraging principles from statistical mechanics, the $\mathbb{C}$-RSN model introduces localized non-linearity, complex fixed eigenvalues, and a distinct separation of memory and input processing functionalities. These features collectively enable the $\mathbb{C}$-RSN evolving towards a dynamic, oscillating final state that more closely mirrors biological cognition. Central to this work is the exploration of how the $\mathbb{C}$-RSN manages to capture the rhythmic, oscillatory dynamics intrinsic to biological systems, thanks to its complex eigenvalue structure and the innovative segregation of its linear and non-linear components. The model's ability to classify data through a time-dependent function, and the localization of information processing, is demonstrated with an empirical evaluation using the MNIST dataset. Remarkably, distinct items supplied as a sequential input yield patterns in time which bear the indirect imprint of the insertion order (and of the time of separation between contiguous insertions).

cs.LG↗

arXiv2023

Where do hard problems really exist?

This chapter delves into the realm of computational complexity, exploring the world of challenging combinatorial problems and their ties with statistical physics. Our exploration starts by delving deep into the foundations of combinatorial challenges, emphasizing their nature. We will traverse the class P, which comprises problems solvable in polynomial time using deterministic algorithms, contrasting it with the class NP, where finding efficient solutions remains an enigmatic endeavor, understanding the intricacies of algorithmic transitions and thresholds demarcating the boundary between tractable and intractable problems. We will discuss the implications of the P versus NP problem, representing one of the profoundest unsolved enigmas of computer science and mathematics, bearing a tantalizing reward for its resolution. Drawing parallels between combinatorics and statistical physics, we will uncover intriguing interconnections that shed light on the nature of challenging problems. Statistical physics unveils profound analogies with complexities witnessed in combinatorial landscapes. Throughout this chapter, we will discuss the interplay between computational complexity theory and statistical physics. By unveiling the mysteries surrounding challenging problems, we aim to deepen understanding of the very essence of computation and its boundaries. Through this interdisciplinary approach, we aspire to illuminate the intricate tapestry of complexity underpinning the mathematical and physical facets of hard problems.

cond-mat.dis-nn↗

arXiv2023

Efficient kinetic Lattice Boltzmann simulation of three-dimensional Hall-MHD Turbulence

Simulating plasmas in the Hall-MagnetoHydroDynamics (Hall-MHD) regime represents a valuable {approach for the investigation of} complex non-linear dynamics developing in astrophysical {frameworks} and {fusion machines}. Taking into account the Hall electric field is {computationally very challenging as} it involves {the integration of} an additional term, proportional to $\bNabla \times ((\bNabla\times\mathbf{B})\times \mathbf{B})$ in the Faraday's induction {law}. {The latter feeds back on} the magnetic field $\mathbf{B}$ at small scales (between the ion and electron inertial scales), {requiring} very high resolution{s} in both space and time {in order to properly describe its dynamics.} The computational {advantage provided by the} kinetic Lattice Boltzmann (LB) approach is {exploited here to develop a new} code, the \textbf{\textsc{F}}ast \textbf{\textsc{L}}attice-Boltzmann \textbf{\textsc{A}}lgorithm for \textbf{\textsc{M}}hd \textbf{\textsc{E}}xperiments (\textsc{flame}). The \textsc{flame} code integrates the plasma dynamics in lattice units coupling two kinetic schemes, one for the fluid protons (including the Lorentz force), the other to solve the induction equation describing the evolution of the magnetic field. Here, the newly developed algorithm is tested against an analytical wave-solution of the dissipative Hall-MHD equations, pointing out its stability and second-order convergence, over a wide range of the control parameters. Spectral properties of the simulated plasma are finally compared with those obtained from numerical solutions from the well-established pseudo-spectral code \textsc{ghost}. Furthermore, the LB simulations we present, varying the Hall parameter, highlightthe transition from the MHD to the Hall-MHD regime, in excellent agreement with the magnetic field spectra measured in the solar wind.

physics.plasm-ph↗

arXiv2023

Ergodic observables in non-ergodic systems: the example of the harmonic chain

In the framework of statistical mechanics the properties of macroscopic systems are deduced starting from the laws of their microscopic dynamics. One of the key assumptions in this procedure is the ergodic property, namely the equivalence between time averages and ensemble averages. This property can be proved only for a limited number of systems; however, as proved by Khinchin [1], weak forms of it hold even in systems that are not ergodic at the microscopic scale, provided that extensive observables are considered. Here we show in a pedagogical way the validity of the ergodic hypothesis, at a practical level, in the paradigmatic case of a chain of harmonic oscillators. By using analytical results and numerical computations, we provide evidence that this non-chaotic integrable system shows ergodic behavior in the limit of many degrees of freedom. In particular, the Maxwell-Boltzmann distribution turns out to fairly describe the statistics of the single particle velocity. A study of the typical time-scales for relaxation is also provided.

cond-mat.stat-mech↗

arXiv2023

Does Turbulence along the Coronal Current Sheet Drive Ion Cyclotron Waves?

Evidence for the presence of ion cyclotron waves, driven by turbulence, at the boundaries of the current sheet is reported in this paper. By exploiting the full potential of the joint observations performed by Parker Solar Probe and the Metis coronagraph on board Solar Orbiter, local measurements of the solar wind can be linked with the large-scale structures of the solar corona. The results suggest that the dynamics of the current sheet layers generates turbulence, which in turn creates a sufficiently strong temperature anisotropy to make the solar-wind plasma unstable to anisotropy-driven instabilities such as the Alfvén ion-cyclotron, mirror-mode, and firehose instabilities. The study of the polarization state of high-frequency magnetic fluctuations reveals that ion cyclotron waves are indeed present along the current sheet, thus linking the magnetic topology of the remotely imaged coronal source regions with the wave bursts observed in situ. The present results may allow improvement of state-of-the-art models based on the ion cyclotron mechanism, providing new insights into the processes involved in coronal heating.

astro-ph.SR↗

arXiv2023

Helios 2 observations of solar wind turbulence decay in the inner heliosphere

The linear scaling of the mixed third-order moment of the magnetohydrodynamic fluctuations is used to estimate the energy transfer rate of the turbulent cascade in the expanding solar wind. In 1976 the Helios 2 spacecraft measured three samples of fast solar wind originating from the same coronal hole, at different distance from the sun. Along with the adjacent slow solar wind streams, these represent a unique database for studying the radial evolution of turbulence in samples of undisturbed solar wind. A set of direct numerical simulations of the MHD equations performed with the Lattice-Boltzmann code FLAME is also used for interpretation. We show that the turbulence energy transfer rate decays approximately as a power law of the distance, and that both the amplitude and decay law correspond to the observed radial temperature profile in the fast wind case. Results from magnetohydrodynamic numerical simulations of decaying magnetohydrodynamic turbulence show a similar trend for the total dissipation, suggesting an interpretation of the observed dynamics in terms of decaying turbulence, and that multi-spacecraft studies of the solar wind radial evolution may help clarifying the nature of the evolution of the turbulent fluctuations in the ecliptic solar wind.

physics.space-ph↗