arXiv subjects

Jorge Amaya

Publications and source records attributed to Jorge Amaya.

13 recordsLinked to original sources

arXiv2025

Energy Efficiency trends in HPC: what high-energy and astrophysicists need to know

The growing energy demands of HPC systems have made energy efficiency a critical concern for system developers and operators. However, HPC users are generally less aware of how these energy concerns influence the design, deployment, and operation of supercomputers even though they experience the consequences. This paper examines the implications of HPC's energy consumption, providing an overview of current trends aimed at improving energy efficiency. We describe how hardware innovations such as energy-efficient processors, novel system architectures, power management techniques, and advanced scheduling policies do have a direct impact on how applications need to be programmed and executed on HPC systems. For application developers, understanding how these new systems work and how to analyse and report the performances of their own software is critical in the dialog with HPC system designers and administrators. The paper aims to raise awareness about energy efficiency among users, particularly in the high energy physics and astrophysics domains, offering practical advice on how to analyse and optimise applications to reduce their energy consumption without compromising on performance.

cs.DC↗

arXiv2025

Bypassing the static input size of neural networks in flare forecasting by using spatial pyramid pooling

The spatial extension of active regions (ARs) of the Sun can vary from one case to the next. This is a problem when studying solar flares with Convolutional Neural Networks (CNNs) as they generally use input images of a fixed size. Different processes can be performed to retrieve a database with homogeneous-sized data, such as resizing. Unfortunately, key features can be lost or distorted during these processes. This can lead to a deterioration of the ability of CNNs to classify flares of different soft X-ray classes, especially those from ARs with complex structures. Our work aims to implement and test a CNN architecture that retains the full features of the original resolution of the input images. We compare the performance of two CNN architectures for solar flare prediction: the first is a traditional CNN with resized input whereas the other implements a spatial pyramid pooling (SPP) layer without any input resizing. Both are trained on the Spaceweather HMI Active Region Patch line-of-sight magnetogram database. We also study two cases of binary classification. In the first case, our model distinguishes ARs producing flares in less than 24h of class greater or equal to C1.0 from ARs producing flares in more than 24h or never; in the second case, it distinguishes ARs producing flares in less than 24h of class greater or equal to M1.0 from the other ARs. Our models implementing an SPP layer outperform the traditional CNN models when predicting flares greater or equal to C1.0 within 24h. However, their performances degrade sharply along the other models studied in this paper, when trained to classify images greater or equal to M1.0 flares. The degradation in SPP models when classifying only images greater or equal to M1.0 flares as positive may be attributed to its success in identifying features that appear in ARs a few hours before the flare, independently of their soft X-ray class.

astro-ph.SR↗

arXiv2024

DRAM Errors and Cosmic Rays: Space Invaders or Science Fiction?

It is widely accepted that cosmic rays are a plausible cause of DRAM errors in high-performance computing (HPC) systems, and various studies suggest that they could explain some aspects of the observed DRAM error behavior. However, this phenomenon is insufficiently studied in production environments. We analyze the correlations between cosmic rays and DRAM errors on two HPC clusters: a production supercomputer with server-class DDR3-1600 and a prototype with LPDDR3-1600 and no hardware error correction. Our error logs cover 2000 billion MB-hours for the MareNostrum 3 supercomputer and 135 million MB-hours for the Mont-Blanc prototype. Our analysis combines quantitative analysis, formal statistical methods and machine learning. We detect no indications that cosmic rays have any influence on the DRAM errors. To understand whether the findings are specific to systems under study, located at 100 meters above the sea level, the analysis should be repeated on other HPC clusters, especially the ones located on higher altitudes. Also, analysis can (and should) be applied to revisit and extend numerous previous studies which use cosmic rays as a hypothetical explanation for some aspects of the observed DRAM error behaviors.

cs.DC↗

arXiv2022

Identification of high order closure terms from fully kinetic simulations using machine learning

Simulations of large-scale plasma systems are typically based on a fluid approximation approach. These models construct a moment-based system of equations that approximate the particle-based physics as a fluid, but as a result lack the small-scale physical processes available to fully kinetic models. Traditionally, empirical closure relations are used to close the moment-based system of equations, which typically approximate the pressure tensor or heat flux. The more accurate the closure relation, the stronger the simulation approaches kinetic-based results. In this paper, new closure terms are constructed using machine learning techniques. Two different machine learning models, a multi-layer perceptron and a gradient boosting regressor, synthesize a local closure relation for the pressure tensor and heat flux vector from fully kinetic simulations of a 2D magnetic reconnection problem. The models are compared to an existing closure relation for the pressure tensor, and the applicability of the models is discussed. The initial results show that the models can capture the diagonal components of the pressure tensor accurately, and show promising results for the heat flux, opening the way for new experiments in multi-scale modeling. We find that the sampling of the points used to train both models play a capital role in their accuracy.

physics.plasm-ph↗

arXiv2021

Unsupervised classification of simulated magnetospheric regions

In magnetospheric missions, burst mode data sampling should be triggered in the presence of processes of scientific or operational interest. We present an unsupervised classification method for magnetospheric regions, that could constitute the first-step of a multi-step method for the automatic identification of magnetospheric processes of interest. Our method is based on Self Organizing Maps (SOMs), and we test it preliminarily on data points from global magnetospheric simulations obtained with the OpenGGCM-CTIM-RCM code. The dimensionality of the data is reduced with Principal Component Analysis before classification. The classification relies exclusively on local plasma properties at the selected data points, without information on their neighborhood or on their temporal evolution. We classify the SOM nodes into an automatically selected number of classes, and we obtain clusters that map to well defined magnetospheric regions. We validate our classification results by plotting the classified data in the simulated space and by comparing with K-means classification. For the sake of result interpretability, we examine the SOM feature maps (magnetospheric variables are called features in the context of classification), and we use them to unlock information on the clusters. We repeat the classification experiments using different sets of features, we quantitatively compare different classification results, and we obtain insights on which magnetospheric variables make more effective features for unsupervised classification.

physics.space-ph↗

arXiv2020

Automatic classification of plasma regions in near-Earth space with supervised machine learning: application to Magnetospheric Multi Scale 2016-2019 observations

The proper classification of plasma regions in near-Earth space is crucial to perform unambiguous statistical studies of fundamental plasma processes such as shocks, magnetic reconnection, waves and turbulence, jets and their combinations. The majority of available studies have been performed by using human-driven methods, such as visual data selection or the application of predefined thresholds to different observable plasma quantities. While human-driven methods have allowed performing many statistical studies, these methods are often time-consuming and can introduce important biases. On the other hand, the recent availability of large, high-quality spacecraft databases, together with major advances in machine-learning algorithms, can now allow meaningful applications of machine learning to in-situ plasma data. In this study, we apply the fully convolutional neural network (FCN) deep machine-leaning algorithm to the recent Magnetospheric Multi Scale (MMS) mission data in order to classify ten key plasma regions in near-Earth space for the period 2016-2019. For this purpose, we use available intervals of time series for each such plasma region, which were labeled by using human-driven selective downlink applied to MMS burst data. We discuss several quantitative parameters to assess the accuracy of both methods. Our results indicate that the FCN method is reliable to accurately classify labeled time series data since it takes into account the dynamical features of the plasma data in each region. We also present good accuracy of the FCN method when applied to unlabeled MMS data. Finally, we show how this method used on MMS data can be extended to data from the Cluster mission, indicating that such method can be successfully applied to any in situ spacecraft plasma database.

physics.plasm-ph↗

arXiv2020

Visualizing and Interpreting Unsupervised Solar Wind Classifications

One of the goals of machine learning is to eliminate tedious and arduous repetitive work. The manual and semi-automatic classification of millions of hours of solar wind data from multiple missions can be replaced by automatic algorithms that can discover, in mountains of multi-dimensional data, the real differences in the solar wind properties. In this paper we present how unsupervised clustering techniques can be used to segregate different types of solar wind. We propose the use of advanced data reduction methods to pre-process the data, and we introduce the use of Self-Organizing Maps to visualize and interpret 14 years of ACE data. Finally, we show how these techniques can potentially be used to uncover hidden information, and how they compare with previous manual and automatic categorizations.

physics.space-ph↗

arXiv2020

Dynamic Time Warping as a New Evaluation for Dst Forecast with Machine Learning

Models based on neural networks and machine learning are seeing a rise in popularity in space physics. In particular, the forecasting of geomagnetic indices with neural network models is becoming a popular field of study. These models are evaluated with metrics such as the root-mean-square error (RMSE) and Pearson correlation coefficient. However, these classical metrics sometimes fail to capture crucial behavior. To show where the classical metrics are lacking, we trained a neural network, using a long short-term memory network, to make a forecast of the disturbance storm time index at origin time $t$ with a forecasting horizon of 1 up to 6 hours, trained on OMNIWeb data. Inspection of the model's results with the correlation coefficient and RMSE indicated a performance comparable to the latest publications. However, visual inspection showed that the predictions made by the neural network were behaving similarly to the persistence model. In this work, a new method is proposed to measure whether two time series are shifted in time with respect to each other, such as the persistence model output versus the observation. The new measure, based on Dynamical Time Warping, is capable of identifying results made by the persistence model and shows promising results in confirming the visual observations of the neural network's output. Finally, different methodologies for training the neural network are explored in order to remove the persistence behavior from the results.

cs.LG↗

arXiv2020

Tokamak disruption prediction using different machine learning techniques

Disruption prediction and mitigation is of key importance in the development of sustainable tokamakreactors. Machine learning has become a key tool in this endeavour. In this paper multiple machinelearning models will be tested and compared. A particular focus has been placed on their portability.This describes how easily the models can be used with data from new devices. The methods used inthis paper are support vector machine, 2-tiered support vector machine, random forest, gradient boostedtrees and long-short term memory. The results show that the support vector machine performanceis marginally better among the standard models, while the gradient boosted trees performed the worst.The portable variant of each model had lower performance. Random forest obtained the highest portableperformance. Results also suggest that disruptions can be detected as early as 600ms before the event.An analysis of the computational cost showed all models run in less than 1ms, allowing sufficient timefor disruption mitigation.

physics.plasm-ph↗

arXiv2019

Characterizing magnetic reconnection regions using Gaussian mixture models on particle velocity distributions

We present a method based on unsupervised machine learning to identify regions of interest using particle velocity distributions as a signature pattern. An automatic density estimation technique is applied to particle distributions provided by PIC simulations to study magnetic reconnection. The key components of the method involve: i) a Gaussian mixture model determining the presence of a given number of subpopulations within an overall population, and ii) a model selection technique with Bayesian Information Criterion to estimate the appropriate number of subpopulations. Thus, this method identifies automatically the presence of complex distributions, such as beams or other non-Maxwellian features, and can be used as a detection algorithm able to identify reconnection regions. The approach is demonstrated for specific double Harris sheet simulations but it can in principle be applied to any other type of simulation and observational data on the particle distribution function.

physics.plasm-ph↗

arXiv2019

The DEEP-ER project: I/O and resiliency extensions for the Cluster-Booster architecture

The recently completed research project DEEP-ER has developed a variety of hardware and software technologies to improve the I/O capabilities of next generation high-performance computers, and to enable applications recovering from the larger hardware failure rates expected on these machines. The heterogeneous Cluster-Booster architecture --first introduced in the predecessor DEEP project-- has been extended by a multi-level memory hierarchy employing non-volatile and network-attached memory devices. Based on this hardware infrastructure, an I/O and resiliency software stack has been implemented combining and extending well established libraries and software tools, and sticking to standard user-interfaces. Real-world scientific codes have tested the projects' developments and demonstrated the improvements achieved without compromising the portability of the applications.

cs.DC↗

arXiv2019

Application performance on a Cluster-Booster system

The DEEP projects have developed a variety of hardware and software technologies aiming at improving the efficiency and usability of next generation high-performance computers. They evolve around an innovative concept for heterogeneous systems: the Cluster-Booster architecture. In it, a general purpose cluster is tightly coupled to a many-core system (the Booster). This modular way of integrating heterogeneous components enables applications to freely choose the kind of computing resources on which it runs most efficiently. Codes might even be partitioned to map specific requirements of code-parts onto the best suited hardware. This paper presents for the first time measurements done by a real world scientific application demonstrating the performance gain achieved with this kind of code-partition approach.

cs.DC↗

arXiv2014

The PAC2MAN mission: a new tool to understand and predict solar energetic events

An accurate forecast of flare and CME initiation requires precise measurements of the magnetic energy build up and release in the active regions of the solar atmosphere. We designed a new space weather mission that performs such measurements using new optical instruments based on the Hanle and Zeeman effects. The mission consists of two satellites, one orbiting the L1 Lagrangian point (Spacecraft Earth, SCE) and the second in heliocentric orbit at 1AU trailing the Earth by 80$^\circ$ (Spacecraft 80, SC80). Optical instruments measure the vector magnetic field in multiple layers of the solar atmosphere. The orbits of the spacecraft allow for a continuous imaging of nearly 73\% of the total solar surface. In-situ plasma instruments detect solar wind conditions at 1AU and ahead of our planet. Earth directed CMEs can be tracked using the stereoscopic view of the spacecraft and the strategic placement of the SC80 satellite. Forecasting of geoeffective space weather events is possible thanks to an accurate surveillance of the magnetic energy build up in the Sun, an optical tracking through the interplanetary space, and in-situ measurements of the near-Earth environment.

astro-ph.IM↗