arXiv subjects

Pavlos Protopapas

Publications and source records attributed to Pavlos Protopapas.

At least 19 recordsLinked to original sources

arXiv2026

Variational Boosting for Physics-Informed Neural Networks

Physics-Informed Neural Networks (PINNs) solve differential equations by minimizing the residual of a nonlinear operator over a neural parameterization of the solution. However, monolithic PINNs often suffer from ill-conditioning, spectral bias, and optimization instability. We introduce a variational boosting framework in which solutions are constructed additively in function space. Each stage trains a weak learner whose converged correction satisfies a local orthogonality condition, equivalent to a projected functional gradient descent step onto the tangent space of the network's function manifold. Because each correction network is deliberately small, the restricted minimization admits full Newton or conjugate gradient updates, which are typically infeasible in large PINNs. The resulting method separates global nonlinear refinement into a sequence of well-conditioned subproblems while preserving the full variational structure of the operator. This framework provides a geometric interpretation of multi-stage PINNs as projected functional gradient descent and enables stable second-order optimization for nonlinear differential equations.

cs.LG↗

arXiv2026

AGNFormer I: Reconstruction of AGN spectra using a probabilistic transformer model

We explore how an uncertainty-aware transformer-based architecture can leverage information embedded across the entire observed optical spectra of AGN, focusing on the algorithm's ability to predict unseen or masked parts of luminous AGN spectra. This provides a direct probe of the learnable correlations between AGN continua and broad lines. We introduce AGNFormer, a transformer model trained to predict the mean expected flux and variance in masked spectral regions (major broad lines to ${\pm}10^{4}$kms$^{-1}$; missing halves), inputting rest-frame spectral fluxes and uncertainties across the entire redshift range of the SDSS DR16 Quasar Catalogue. We evaluate the performance of the model on both full (no S/N limit) and high-quality (S/N > 10) spectral samples using the negative-log likelihood, and via comparisons with existing C IV and ly-a reconstruction algorithms. The model successfully reconstructs unseen AGN broad lines to better than 10-16% (4-8%) of the flux for the full (S/N > 10) test sets, up to an error floor of $\approx$2-6% of the flux at S/N $\approx$ 40, while predictions for larger unseen halves grow to 12-25% (5-15%) of the flux the further away they are from the cut-off wavelength of the seen input spectrum. Predictions faithfully reproduce the broad AGN spectral diversity across the entire optical and UV QSO main sequence parameter spaces, including both Gaussian and Lorentzian profile regimes, Feii complexes, and narrow emission lines. Performance is similar or better compared to previous spectral reconstruction algorithms. The high precision of the broad-line region reconstruction demonstrates that the method successfully aggregates information across the spectrum and highlights how the AGN continuum and weaker lines/complexes have the potential to assist astronomers in the extraction of the entire wealth of information embedded in AGN spectra.

astro-ph.GA↗

arXiv2026

Physics-Informed Neural Embeddings of PDE Solution Families

We introduce a physics-informed framework for learning finite-dimensional embeddings of solution families of partial differential equations. The method uses a multihead Physics-Informed Neural Network in which a shared body learns a latent manifold representing the solution space, while linear heads reconstruct individual solutions associated with different initial conditions. A head-orthogonalization penalty removes degeneracies in the latent representation and stabilizes the principal-component spectrum across training realizations. Because the initial condition is built into the network output by construction, these principal components measure the additional variability the network learns on top of the initial profile, not the full solution itself. We apply the method to the one-dimensional viscous Burgers equation, with the heat and wave equations as robustness checks. For a latent dimension $n_b=20$, the learned manifolds exhibit pronounced effective dimensional reduction: for Burgers dynamics, only $2$-$4$ principal components capture about $95\%$ of the latent-space variance, while $4$-$7$ capture about $99\%$, depending on the initial-condition family; the same qualitative compression holds for the heat and wave equations. We also split the wavenumber axis into bands (``Fourier shells'') and measure how much each band contributes to every principal component. The resulting frequency profile is invariant under the change-of-basis freedom that the orthogonalization penalty leaves in the latent space, and is therefore reproducible across independent training runs. More broadly, this establishes the learned spectral profiles and principal components as robust observables of solution-manifold geometry.

cs.LG↗

arXiv2026

Gravitational Duals from Equations of State II: Large Hierarchies and False Vacua

We investigate the reconstruction of holographic duals for strongly coupled quantum field theories in regimes characterized by large hierarchies and the presence of false vacua. Within the gauge/gravity duality, these features translate into non-trivial thermodynamic behaviour and exotic renormalization group flows, including skipping flows between non-adjacent fixed points. Building on previous work based on Physics-Informed Neural Networks (PINNs), we extend the holographic inverse problem of reconstructing the bulk scalar potential from boundary thermodynamic data into this new regime. This setting presents a variety of conceptual and numerical challenges, such as near-degenerate states, large hierarchies of energy scales, and regions of the potential that are not directly probed by the input data. We develop a set of methodological advances that overcome these obstacles, thereby improving the established PINNs-based methodology and extending it to new physical regimes of interest that were previously out of reach. Applying the developed framework, we demonstrate accurate reconstruction of scalar potentials deep into the false vacuum regime, achieving robust agreement with the physical features of the underlying thermodynamics despite significant numerical stiffness. Our results extend the bridge between holography and machine learning, and suggest that data-driven approaches can provide new insights into the structure of strongly coupled systems.

hep-th↗

arXiv2026

Identifying Observational Signatures of Flux Eruption Events in Supermassive Black Hole Accretion Flows with Machine Learning

Simulated black hole accretion flows with strong magnetic fields often exhibit "flux eruption events" (FEEs), transient and localized expulsions of matter near the event horizon due to magnetic reconnection. It may now be possible to image them with the Event Horizon Telescope (EHT), a global network of millimeter-wave observatories that images black holes. Here we use machine learning as an interpretable inference tool to identify observational signatures of FEEs that could be accessible to the EHT. First, we train a convolutional neural network to learn task-relevant representations of FEEs in uncorrupted simulated images. After using this network to label a larger set of images, we then train interpretable models (random forest and logistic regression) to determine observational signatures. We find that during a FEE, images in the millimeter tend toward more diffuse emission, higher linear polarization, and lower total fluxes, but these signatures are weak for most FEEs compared to the usual time variability of these features. Moreover, the Q-U loop rotation rate decreases during FEEs, contrary to a picture in which FEEs could jointly cause both millimeter Q-U loops and flares. Our random forest trained on observable summary statistics achieves ~80% class-weighted accuracy, suggesting that the CNN learns FEE structure not fully mapped onto these traditional summary statistics. Our results imply that image size and polarization fraction can be used to flag candidate FEEs, but high-resolution, high-dynamic range images will still be important to confirm FEEs and test accretion flows for this phenomenon.

astro-ph.HE↗

arXiv2026

The Chandra-Gaia Catalog of Counterparts: Resolving ambiguous Gaia matches to X-ray sources in the Chandra Source Catalog using Machine Learning

We present a framework to cross-match sources from the Chandra Source Catalog (CSC v2.1) with optical sources from Gaia Data Release 3. Unlike purely spatial approaches, we use source properties such as magnitudes, colors, and distances to identify true counterparts, detect chance coincidences, and resolve ambiguities when multiple plausible candidates exist. We define a training set of high-confidence matches using NWAY, a Bayesian cross-matching framework that accounts for positional errors and source densities. We train a gradient-boosted classifier (LightGBM) on a variety of features from both catalogs. Of the ~$254$k unique X-ray sources, we find counterparts for ~$113$k sources, of which plausible multiple counterparts are found for ~$7$k. We find no counterparts for ~$20$k sources for which separation-based cross-matching does find a match, and attribute half of these to chance coincidences. We validate the pipeline on the Chandra Orion Ultradeep Project (COUP), where the machine-learning matches reproduce 95% of NWAY cross-matches without using any positional information. We release a catalog of the ~$113$k Chandra-Gaia counterparts, together with ~$7$k alternative matches and ~$20$k ambiguous NWAY associations, supporting future population studies of sources detectable by both Chandra and Gaia. We discuss limitations and provide a generalization of the framework that is applicable in other cross-matching scenarios.

astro-ph.IM↗

arXiv2026

Gradient Scaling Effects in Adaptive Spectral PINNs for Stiff Nonlinear ODEs

Physics-Informed Neural Networks (PINNs) often struggle to train reliably on stiff and oscillatory dynamical systems due to poor optimization conditioning. While prior work has emphasized representational remedies such as spectral parameterizations, the optimization implications of initial-condition (IC) embeddings in adaptive spectral PINNs have not been well characterized. In this work, we show that the choice of IC gating function induces explicit time-dependent gradient scaling, which interacts with spectral representations during training. Using a nonlinear stiff spring-pendulum ODE as a controlled benchmark, we compare exponential and linear IC gates in combination with fixed and adaptive Fourier spectral trunks. We observe stiffness-dependent changes in relative dominance for adaptive PINNs: at moderate stiffness ($k=20$), exponential gating often yields lower error but exhibits heterogeneous behavior across random seeds, whereas at higher stiffness ($k=60$), linear gating becomes preferable, with additional reversals observed at larger $k$. These trends hold for both relative $L^2$ error and maximum pointwise error and are confirmed by paired Wilcoxon signed-rank tests with Holm correction. Overall, our results demonstrate that IC embeddings are not a neutral design choice in PINNs: the induced gradient scaling materially shapes optimization conditioning in stiff regimes, with distinct sensitivity patterns in baseline and adaptive spectral models.

cs.LG↗

arXiv2026

Chebyshev-Augmented One-Shot Transfer Learning for PINNs on Nonlinear Differential Equations

Physics-Informed Neural Networks (PINNs) offer a flexible paradigm for solving differential equations by embedding governing laws into the training objective. A persistent limitation is instance specificity: standard PINNs typically require retraining for each new forcing term, boundary/initial condition, or parameter setting. One-shot transfer learning (OTL) addresses this bottleneck for linear operators by freezing a pretrained latent representation and computing optimal output weights in closed form, but for nonlinear problems closed-form adaptation is generally unavailable because the loss is nonconvex in the output layer. In this paper we substantially broaden the class of nonlinearities amenable to one-shot PINN transfer by combining OTL with Chebyshev polynomial surrogates. We approximate general smooth weakly nonlinear terms by truncated Chebyshev expansions over a prescribed solution range, yielding a polynomial nonlinearity that can be handled by a perturbative decomposition into linear subproblems. A multi-head PINN learns a reusable latent space associated with the dominant linear operator; at test time, solutions to new instances are obtained via a sequence of closed-form linear solves in the output layer, without retraining the network body. We provide a unified derivation of the framework for ODEs and PDEs and demonstrate accuracy and fast online adaptation on nonlinear benchmarks, including non-polynomial and singular ODE nonlinearities as well as a reaction-diffusion PDE with saturating kinetics, demonstrating the method's utility in many-query regimes.

cs.LG↗

arXiv2026

Learning embeddings of non-linear PDEs: the Burgers' equation

Embeddings provide low-dimensional representations that organize complex function spaces and support generalization. They provide a geometric representation that supports efficient retrieval, comparison, and generalization. In this work we generalize the concept to Physics Informed Neural Networks. We present a method to construct solution embedding spaces of nonlinear partial differential equations using a multi-head setup, and extract non-degenerate information from them using principal component analysis (PCA). We test this method by applying it to viscous Burgers' equation, which is solved simultaneously for a family of initial conditions and values of the viscosity. A shared network body learns a latent embedding of the solution space, while linear heads map this embedding to individual realizations. By enforcing orthogonality constraints on the heads, we obtain a principal-component decomposition of the latent space that is robust to training degeneracies and admits a direct physical interpretation. The obtained components for Burgers' equation exhibit rapid saturation, indicating that a small number of latent modes captures the dominant features of the dynamics.

math.AP↗

arXiv2026

PTL-PINNs: Perturbation-Guided Transfer Learning with Physics- Informed Neural Networks for Nonlinear Systems

Accurately and efficiently solving nonlinear differential equations is crucial for modeling dynamic behavior across science and engineering. Physics-Informed Neural Networks (PINNs) have emerged as a powerful solution that embeds physical laws in training by enforcing equation residuals. However, these struggle to model nonlinear dynamics, suffering from limited generalization across problems and long training times. To address these limitations, we propose a perturbation-guided transfer learning framework for PINNs (PTL-PINN), which integrates perturbation theory with transfer learning to efficiently solve nonlinear equations. Unlike gradient-based transfer learning, PTL-PINNs solve an approximate linear perturbative system using closed-form expressions, enabling rapid generalization with the time complexity of matrix-vector multiplication. We show that PTL-PINNs achieve accuracy comparable to various Runge-Kutta methods, with computational speeds up to one order of magnitude faster. To benchmark performance, we solve a broad set of problems, including nonlinear oscillators across various damping regimes, the equilibrium-centered Lotka-Volterra system, the KPP-Fisher and the Wave equation. Since perturbation theory sets the accuracy bound of PTL-PINNs, we systematically evaluate its practical applicability. This work connects long-standing perturbation methods with PINNs, demonstrating how perturbation theory can guide foundational models to solve nonlinear systems with speeds comparable to those of classical solvers.

cs.LG↗

arXiv2026

Improving Router Security using BERT

Previous work on home router security has shown that using system calls to train a transformer-based language model built on a BERT-style encoder using contrastive learning is effective in detecting several types of malware, but the performance remains limited at low false positive rates. In this work, we demonstrate that using a high-fidelity eBPF-based system call sensor, together with contrastive augmented learning (which introduces controlled mutations of negative samples), improves detection performance at a low false positive rate. In addition, we introduce a network packet abstraction language that enables the creation of a pipeline similar to network packet data, and we show that network behavior provides complementary detection signals-yielding improved performance for network-focused malware at low false positive rates. Lastly, we implement these methods in an online router anomaly detection framework to validate the approach in an Internet of Things (IoT) deployment environment.

cs.CR↗

arXiv2025

One-Shot Transfer Learning for Nonlinear PDEs with Perturbative PINNs

We propose a framework for solving nonlinear partial differential equations (PDEs) by combining perturbation theory with one-shot transfer learning in Physics-Informed Neural Networks (PINNs). Nonlinear PDEs with polynomial terms are decomposed into a sequence of linear subproblems, which are efficiently solved using a Multi-Head PINN. Once the latent representation of the linear operator is learned, solutions to new PDE instances with varying perturbations, forcing terms, or boundary/initial conditions can be obtained in closed form without retraining. We validate the method on KPP-Fisher and wave equations, achieving errors on the order of 1e-3 while adapting to new problem instances in under 0.2 seconds; comparable accuracy to classical solvers but with faster transfer. Sensitivity analyses show predictable error growth with epsilon and polynomial degree, clarifying the method's effective regime. Our contributions are: (i) extending one-shot transfer learning from nonlinear ODEs to PDEs, (ii) deriving a closed-form solution for adapting to new PDE instances, and (iii) demonstrating accuracy and efficiency on canonical nonlinear PDEs. We conclude by outlining extensions to derivative-dependent nonlinearities and higher-dimensional PDEs.

math.NA↗

arXiv2025

Leveraging pre-trained vision Transformers for multi-band photometric light curve classification

This study investigates the potential of a pre-trained vision Transformer (VT) model, specifically the Swin Transformer V2 (SwinV2), to classify photometric light curves without the need for feature extraction or multi-band preprocessing. The goal is to assess whether this image-based approach can accurately differentiate astronomical phenomena and serve as a viable option for working with multi-band photometric light curves. We transformed each multi-band light curve into an image. These images serve as input to the SwinV2 model, which is pre-trained on ImageNet-21K. The datasets employed include the public Catalog of Variable Stars from the Massive Compact Halo Object (MACHO) survey, using both one and two bands, and the first round of the recent Extended LSST Astronomical Time-Series Classification Challenge (ELAsTiCC), which includes six bands. The performance of the model was evaluated on six classes for the MACHO dataset and 20 distinct classes of variable stars and transient events for the ELAsTiCC dataset. The fine-tuned SwinV2 achieved better performance than models specifically designed for light curves, such as Astromer and the Astronomical Transformer for Time Series and Tabular Data (ATAT). When trained on the full MACHO dataset, it attained a macro F1-score of 80.2 and outperformed Astromer in single-band experiments. Incorporating a second band further improved performance, increasing the F1-score to 84.1. In the ELAsTiCC dataset, SwinV2 achieved a macro F1-score of 65.5, slightly surpassing ATAT by 1.3.

astro-ph.IM↗

arXiv2025

The Denario project: Deep knowledge AI agents for scientific discovery

We present Denario, an AI multi-agent system designed to serve as a scientific research assistant. Denario can perform many different tasks, such as generating ideas, checking the literature, developing research plans, writing and executing code, making plots, and drafting and reviewing a scientific paper. The system has a modular architecture, allowing it to handle specific tasks, such as generating an idea, or carrying out end-to-end scientific analysis using Cmbagent as a deep-research backend. In this work, we describe in detail Denario and its modules, and illustrate its capabilities by presenting multiple AI-generated papers generated by it in many different scientific disciplines such as astrophysics, biology, biophysics, biomedical informatics, chemistry, material science, mathematical physics, medicine, neuroscience and planetary science. Denario also excels at combining ideas from different disciplines, and we illustrate this by showing a paper that applies methods from quantum physics and machine learning to astrophysical data. We report the evaluations performed on these papers by domain experts, who provided both numerical scores and review-like feedback. We then highlight the strengths, weaknesses, and limitations of the current system. Finally, we discuss the ethical implications of AI-driven research and reflect on how such technology relates to the philosophy of science. We publicly release the code at https://github.com/AstroPilot-AI/Denario. A Denario demo can also be run directly on the web at https://huggingface.co/spaces/astropilot-ai/Denario, and the full app will be deployed on the cloud.

cs.AI↗

arXiv2025

ASTROCO: Self-Supervised Conformer-Style Transformers for Light-Curve Embeddings

We present AstroCo, a Conformer-style encoder for irregular stellar light curves. By combining attention with depthwise convolutions and gating, AstroCo captures both global dependencies and local features. On MACHO R-band, AstroCo outperforms Astromer v1 and v2, yielding 70 percent and 61 percent lower error respectively and a relative macro-F1 gain of about 7 percent, while producing embeddings that transfer effectively to few-shot classification. These results highlight AstroCo's potential as a strong and label-efficient foundation for time-domain astronomy.

astro-ph.IM↗

arXiv2025

Efficient PINNs via Multi-Head Unimodular Regularization of the Solutions Space

Non-linear differential equations are a fundamental tool to describe different phenomena in nature. However, we still lack a well-established method to tackle stiff differential equations. Here we present a machine learning framework to facilitate the solution of nonlinear multiscale differential equations and, especially, inverse problems using Physics-Informed Neural Networks (PINNs). This framework is based on what is called \textit{multi-head} (MH) training, which involves training the network to learn a general space of all solutions for a given set of equations with certain variability, rather than learning a specific solution of the system. This setup is used with a second novel technique that we call Unimodular Regularization (UR) of the latent space of solutions. We show that the multi-head approach, combined with Unimodular Regularization, significantly improves the efficiency of PINNs by facilitating the transfer learning process thereby enabling the finding of solutions for nonlinear, coupled, and multiscale differential equations.

cs.LG↗

arXiv2025

Evolution of linear matter perturbations with error-bounded bundle physics-informed neural networks

We consider the evolution of linear matter perturbations in the context of the standard cosmological model ($Λ$CDM) and a phenomenological modified gravity model. We use the physics-informed neural network (PINN) bundle method, which allows to integrate differential systems as an alternative to the traditional numerical method. We apply the PINN bundle method to the equation that describes the matter perturbation evolution, to compare its outcomes with recent data on structure growth, $fσ_8$. Unlike our previous works, we can calculate a bound on the error of this observable without using the numerical solution of the equation. For this, we use a method developed previously by ourselves to calculate an exact bound on the PINN-based solution using only the outcomes of the network and its residual. On the other hand, the use of an updated data set allows us to obtain more stringent constraints on the plane $Ω_m-σ_8$ than previous works.

astro-ph.CO↗

arXiv2025

ABC-SN: Attention Based Classifier for Supernova Spectra

While significant advances have been made in photometric classification ahead of the millions of transient events and hundreds of supernovae (SNe) each night that the Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST) will discover, classifying SNe spectroscopically remains the best way to determine most subtypes of SNe. Traditional spectrum classification tools use template matching techniques (eg., SNID, SuperFit) and require significant human supervision. Two deep learning spectral classifiers, DASH and SNIascore, define the state of the art, but SNIascore is a binary classifier devoted to maximizing the purity of the SN Ia-norm sample, while DASH is no longer maintained and the original work suffers from contamination of multi-epoch spectra in the training and test sets. We have explored several neural network architectures in order to create a new automated method for classifying SN subtypes, settling on an attention-based model we call ABC-SN. We benchmark our results against an updated version of DASH, thus providing the community with an up-to-date general-purpose SN classifier. Our dataset is comprised of ten different SN subtypes, including subtypes of SN Ia, core collapse, and interacting SNe. We find that ABC-SN outperforms DASH for nearly all classes, including an improvement of $26\%$ in SN Ia completeness ($\sim88\%$) and $2.4\%$ in SN Ia purity ($\sim95\%$) when unthresholded (improvements for each class can further be obtained by tuned thresholds), and we discuss the limitations of current SN datasets for benchmarking performance.

astro-ph.IM↗