arXiv subjects

Pavlos Protopapas

Publications and source records attributed to Pavlos Protopapas.

At least 37 records · Page 2Linked to original sources

arXiv2025

ASTROCO: Self-Supervised Conformer-Style Transformers for Light-Curve Embeddings

We present AstroCo, a Conformer-style encoder for irregular stellar light curves. By combining attention with depthwise convolutions and gating, AstroCo captures both global dependencies and local features. On MACHO R-band, AstroCo outperforms Astromer v1 and v2, yielding 70 percent and 61 percent lower error respectively and a relative macro-F1 gain of about 7 percent, while producing embeddings that transfer effectively to few-shot classification. These results highlight AstroCo's potential as a strong and label-efficient foundation for time-domain astronomy.

astro-ph.IM↗

arXiv2025

Efficient PINNs via Multi-Head Unimodular Regularization of the Solutions Space

Non-linear differential equations are a fundamental tool to describe different phenomena in nature. However, we still lack a well-established method to tackle stiff differential equations. Here we present a machine learning framework to facilitate the solution of nonlinear multiscale differential equations and, especially, inverse problems using Physics-Informed Neural Networks (PINNs). This framework is based on what is called \textit{multi-head} (MH) training, which involves training the network to learn a general space of all solutions for a given set of equations with certain variability, rather than learning a specific solution of the system. This setup is used with a second novel technique that we call Unimodular Regularization (UR) of the latent space of solutions. We show that the multi-head approach, combined with Unimodular Regularization, significantly improves the efficiency of PINNs by facilitating the transfer learning process thereby enabling the finding of solutions for nonlinear, coupled, and multiscale differential equations.

cs.LG↗

arXiv2025

Evolution of linear matter perturbations with error-bounded bundle physics-informed neural networks

We consider the evolution of linear matter perturbations in the context of the standard cosmological model ($Λ$CDM) and a phenomenological modified gravity model. We use the physics-informed neural network (PINN) bundle method, which allows to integrate differential systems as an alternative to the traditional numerical method. We apply the PINN bundle method to the equation that describes the matter perturbation evolution, to compare its outcomes with recent data on structure growth, $fσ_8$. Unlike our previous works, we can calculate a bound on the error of this observable without using the numerical solution of the equation. For this, we use a method developed previously by ourselves to calculate an exact bound on the PINN-based solution using only the outcomes of the network and its residual. On the other hand, the use of an updated data set allows us to obtain more stringent constraints on the plane $Ω_m-σ_8$ than previous works.

astro-ph.CO↗

arXiv2025

Astro-MoE: Mixture of Experts for Multiband Astronomical Time Series

Multiband astronomical time series exhibit heterogeneous variability patterns, sampling cadences, and signal characteristics across bands. Standard transformers apply shared parameters to all bands, potentially limiting their ability to model this rich structure. In this work, we introduce Astro-MoE, a foundational transformer architecture that enables dynamic processing via a Mixture of Experts module. We validate our model on both simulated (ELAsTiCC-1) and real-world datasets (Pan-STARRS1).

astro-ph.IM↗

arXiv2025

Image-Based Multi-Survey Classification of Light Curves with a Pre-Trained Vision Transformer

We explore the use of Swin Transformer V2, a pre-trained vision Transformer, for photometric classification in a multi-survey setting by leveraging light curves from the Zwicky Transient Facility (ZTF) and the Asteroid Terrestrial-impact Last Alert System (ATLAS). We evaluate different strategies for integrating data from these surveys and find that a multi-survey architecture which processes them jointly achieves the best performance. These results highlight the importance of modeling survey-specific characteristics and cross-survey interactions, and provide guidance for building scalable classifiers for future time-domain astronomy.

astro-ph.IM↗

arXiv2025

Improved Uncertainty Quantification in Physics-Informed Neural Networks Using Error Bounds and Solution Bundles

Physics-Informed Neural Networks (PINNs) have been widely used to obtain solutions to various physical phenomena modeled as Differential Equations. As PINNs are not naturally equipped with mechanisms for Uncertainty Quantification, some work has been done to quantify the different uncertainties that arise when dealing with PINNs. In this paper, we use a two-step procedure to train Bayesian Neural Networks that provide uncertainties over the solutions to differential equation systems provided by PINNs. We use available error bounds over PINNs to formulate a heteroscedastic variance that improves the uncertainty estimation. Furthermore, we solve forward problems and utilize the obtained uncertainties when doing parameter estimation in inverse problems in cosmology.

cs.LG↗

arXiv2025

Applying Vision Transformers on Spectral Analysis of Astronomical Objects

We apply pre-trained Vision Transformers (ViTs), originally developed for image recognition, to the analysis of astronomical spectral data. By converting traditional one-dimensional spectra into two-dimensional image representations, we enable ViTs to capture both local and global spectral features through spatial self-attention. We fine-tune a ViT pretrained on ImageNet using millions of spectra from the SDSS and LAMOST surveys, represented as spectral plots. Our model is evaluated on key tasks including stellar object classification and redshift ($z$) estimation, where it demonstrates strong performance and scalability. We achieve classification accuracy higher than Support Vector Machines and Random Forests, and attain $R^2$ values comparable to AstroCLIP's spectrum encoder, even when generalizing across diverse object types. These results demonstrate the effectiveness of using pretrained vision models for spectroscopic data analysis. To our knowledge, this is the first application of ViTs to large-scale, which also leverages real spectroscopic data and does not rely on synthetic inputs.

astro-ph.IM↗

arXiv2025

Parameter Inference of Black Hole Images using Deep Learning in Visibility Space

Using very long baseline interferometry, the Event Horizon Telescope (EHT) collaboration has resolved the shadows of two supermassive black holes. Model comparison is traditionally performed in image space, where imaging algorithms introduce uncertainties in the recovered structure. Here, we develop a deep learning framework to perform parameter inference in visibility space, directly using the data measured by the interferometer without introducing potential errors and biases from image reconstruction. First, we train and validate our framework on synthetic data derived from general relativistic magnetohydrodynamics (GRMHD) simulations that vary in magnetic field state, spin, and $R_\mathrm{high}$. Applying these models to the real data obtained during the 2017 EHT campaign, and only considering total intensity, we do not derive meaningful constraints on either of these parameters. At present, our method is limited both by theoretical uncertainties in the GRMHD simulations and variation between snapshots of the same underlying physical model. However, we demonstrate that spin and $R_\mathrm{high}$ could be recovered using this framework through continuous monitoring of our sources, which mitigates variations due to turbulence. In future work, we anticipate that including spectral or polarimetric information will greatly improve the performance of this framework.

astro-ph.GA↗

arXiv2025

Recent Advances of NeuroDiffEq -- An Open-Source Library for Physics-Informed Neural Networks

Solving differential equations is a critical challenge across a host of domains. While many software packages efficiently solve these equations using classical numerical approaches, there has been less effort in developing a library for researchers interested in solving such systems using neural networks. With PyTorch as its backend, NeuroDiffEq is a software library that exploits neural networks to solve differential equations. In this paper, we highlight the latest features of the NeuroDiffEq library since its debut. We show that NeuroDiffEq can solve complex boundary value problems in arbitrary dimensions, tackle boundary conditions at infinity, and maintain flexibility for dynamic injection at runtime.

cs.LG↗

arXiv2025

Stiff Transfer Learning for Physics-Informed Neural Networks

Stiff differential equations are prevalent in various scientific domains, posing significant challenges due to the disparate time scales of their components. As computational power grows, physics-informed neural networks (PINNs) have led to significant improvements in modeling physical processes described by differential equations. Despite their promising outcomes, vanilla PINNs face limitations when dealing with stiff systems, known as failure modes. In response, we propose a novel approach, stiff transfer learning for physics-informed neural networks (STL-PINNs), to effectively tackle stiff ordinary differential equations (ODEs) and partial differential equations (PDEs). Our methodology involves training a Multi-Head-PINN in a low-stiff regime, and obtaining the final solution in a high stiff regime by transfer learning. This addresses the failure modes related to stiffness in PINNs while maintaining computational efficiency by computing "one-shot" solutions. The proposed approach demonstrates superior accuracy and speed compared to PINNs-based methods, as well as comparable computational efficiency with implicit numerical methods in solving stiff-parameterized linear and polynomial nonlinear ODEs and PDEs under stiff conditions. Furthermore, we demonstrate the scalability of such an approach and the superior speed it offers for simulations involving initial conditions and forcing function reparametrization.

cs.LG↗

arXiv2024

Uncertainty estimation for time series classification: Exploring predictive uncertainty in transformer-based models for variable stars

Classifying variable stars is key for understanding stellar evolution and galactic dynamics. With the demands of large astronomical surveys, machine learning models, especially attention-based neural networks, have become the state-of-the-art. While achieving high accuracy is crucial, enhancing model interpretability and uncertainty estimation is equally important to ensure that insights are both reliable and comprehensible. We aim to enhance transformer-based models for classifying astronomical light curves by incorporating uncertainty estimation techniques to detect misclassified instances. We tested our methods on labeled datasets from MACHO, OGLE-III, and ATLAS, introducing a framework that significantly improves the reliability of automated classification for the next-generation surveys. We used Astromer, a transformer-based encoder designed for capturing representations of single-band light curves. We enhanced its capabilities by applying three methods for quantifying uncertainty: Monte Carlo Dropout (MC Dropout), Hierarchical Stochastic Attention (HSA), and a novel hybrid method combining both approaches, which we have named Hierarchical Attention with Monte Carlo Dropout (HA-MC Dropout). We compared these methods against a baseline of deep ensembles (DEs). To estimate uncertainty estimation scores for the misclassification task, we selected Sampled Maximum Probability (SMP), Probability Variance (PV), and Bayesian Active Learning by Disagreement (BALD) as uncertainty estimates. In predictive performance tests, HA-MC Dropout outperforms the baseline, achieving macro F1-scores of 79.8+-0.5 on OGLE, 84+-1.3 on ATLAS, and 76.6+-1.8 on MACHO. When comparing the PV score values, the quality of uncertainty estimation by HA-MC Dropout surpasses that of all other methods, with improvements of 2.5+-2.3 for MACHO, 3.3+-2.1 for ATLAS and 8.5+-1.6 for OGLE-III.

astro-ph.IM↗

arXiv2024

Exact and approximate error bounds for physics-informed neural networks

The use of neural networks to solve differential equations, as an alternative to traditional numerical solvers, has increased recently. However, error bounds for the obtained solutions have only been developed for certain equations. In this work, we report important progress in calculating error bounds of physics-informed neural networks (PINNs) solutions of nonlinear first-order ODEs. We give a general expression that describes the error of the solution that the PINN-based method provides for a nonlinear first-order ODE. In addition, we propose a technique to calculate an approximate bound for the general case and an exact bound for a particular case. The error bounds are computed using only the residual information and the equation structure. We apply the proposed methods to particular cases and show that they can successfully provide error bounds without relying on the numerical solution.

cs.LG↗

arXiv2024

Transformer-Based Astronomical Time Series Model with Uncertainty Estimation for Detecting Misclassified Instances

In this work, we present a framework for estimating and evaluating uncertainty in deep-attention-based classifiers for light curves for variable stars. We implemented three techniques, Deep Ensembles (DEs), Monte Carlo Dropout (MCD) and Hierarchical Stochastic Attention (HSA) and evaluated models trained on three astronomical surveys. Our results demonstrate that MCD and HSA offers a competitive and computationally less expensive alternative to DE, allowing the training of transformers with the ability to estimate uncertainties for large-scale light curve datasets. We conclude that the quality of the uncertainty estimation is evaluated using the ROC AUC metric.

astro-ph.IM↗

arXiv2024

Faster Bayesian inference with neural network bundles and new results for $f(R)$ models

In the last few years, there has been significant progress in the development of machine learning methods tailored to astrophysics and cosmology. We have recently applied one of these, namely, the neural network bundle method, to the cosmological scenario. Moreover, we showed that in some cases the computational times of the Bayesian inference process can be reduced. In this paper, we present an improvement to the neural network bundle method that results in a significant reduction of the computational times of the statistical analysis. The novel aspect consists of the use of the neural network bundle method to calculate the luminosity distance of type Ia supernovae, which is usually computed through an integral with numerical methods. In this work, we have applied this improvement to the Hu-Sawicki and Starobinsky $f(R)$ models. We also performed a statistical analysis with data from type Ia supernovae of the Pantheon+ compilation and cosmic chronometers. Another original aspect of this work is the different treatment we provide for the absolute magnitude of type Ia supernovae during the inference process, which results in different estimates of the distortion parameter than the ones obtained in the literature. We show that the statistical analyses carried out with our new method require lower computational times than the ones performed with both the numerical and the neural network method from our previous work. This reduction in time is more significant in the case of a difficult computational problem such as the ones addressed in this work.

astro-ph.CO↗

arXiv2024

Gravitational Duals from Equations of State

Holography relates gravitational theories in five dimensions to four-dimensional quantum field theories in flat space. Under this map, the equation of state of the field theory is encoded in the black hole solutions of the gravitational theory. Solving the five-dimensional Einstein's equations to determine the equation of state is an algorithmic, direct problem. Determining the gravitational theory that gives rise to a prescribed equation of state is a much more challenging, inverse problem. We present a novel approach to solve this problem based on physics-informed neural networks. The resulting algorithm is not only data-driven but also informed by the physics of the Einstein's equations. We successfully apply it to theories with crossovers, first- and second-order phase transitions.

hep-th↗

arXiv2023

Generating Images of the M87* Black Hole Using GANs

In this paper, we introduce a novel data augmentation methodology based on Conditional Progressive Generative Adversarial Networks (CPGAN) to generate diverse black hole (BH) images, accounting for variations in spin and electron temperature prescriptions. These generated images are valuable resources for training deep learning algorithms to accurately estimate black hole parameters from observational data. Our model can generate BH images for any spin value within the range of [-1, 1], given an electron temperature distribution. To validate the effectiveness of our approach, we employ a convolutional neural network to predict the BH spin using both the GRMHD images and the images generated by our proposed model. Our results demonstrate a significant performance improvement when training is conducted with the augmented dataset while testing is performed using GRMHD simulated data, as indicated by the high R2 score. Consequently, we propose that GANs can be employed as cost effective models for black hole image generation and reliably augment training datasets for other parameterization algorithms.

astro-ph.GA↗

arXiv2023

One-Shot Transfer Learning for Nonlinear ODEs

We introduce a generalizable approach that combines perturbation method and one-shot transfer learning to solve nonlinear ODEs with a single polynomial term, using Physics-Informed Neural Networks (PINNs). Our method transforms non-linear ODEs into linear ODE systems, trains a PINN across varied conditions, and offers a closed-form solution for new instances within the same non-linear ODE class. We demonstrate the effectiveness of this approach on the Duffing equation and suggest its applicability to similarly structured PDEs and ODE systems.

cs.LG↗

arXiv2023

Con$^{2}$DA: Simplifying Semi-supervised Domain Adaptation by Learning Consistent and Contrastive Feature Representations

In this work, we present Con$^{2}$DA, a simple framework that extends recent advances in semi-supervised learning to the semi-supervised domain adaptation (SSDA) problem. Our framework generates pairs of associated samples by performing stochastic data transformations to a given input. Associated data pairs are mapped to a feature representation space using a feature extractor. We use different loss functions to enforce consistency between the feature representations of associated data pairs of samples. We show that these learned representations are useful to deal with differences in data distributions in the domain adaptation problem. We performed experiments to study the main components of our model and we show that (i) learning of the consistent and contrastive feature representations is crucial to extract good discriminative features across different domains, and ii) our model benefits from the use of strong augmentation policies. With these findings, our method achieves state-of-the-art performances in three benchmark datasets for SSDA.

cs.LG↗