arXiv subjects

Andreas D. Demou

Publications and source records attributed to Andreas D. Demou.

6 recordsLinked to original sources

arXiv2026

Air Quality Downscaling with Station-Guided Pseudo-Supervision

Super-resolving coarse atmospheric fields to local PM$_{2.5}$ variations is uniquely challenged by a mismatch in spatial support: while pixels represent regional averages, ground-truth observations are discrete, unaligned samples of a continuous spatial signal. To bridge this gap, we present a station-guided framework for high-resolution PM$_{2.5}$ downscaling over Europe. Taking coarse CAMS atmospheric composition fields alongside heterogeneous side information (i.e., human activity, land cover, elevation, satellite aerosol observations, and wind fields) our framework jointly super-resolves ($\times 40$, $\approx$ 1 km) and bias-corrects CAMS rasters, without relying on temporal sequence modelling. To address the challenge of densely supervising our multi-scale transformer network with sparse in-situ data, we introduce a time-agnostic propagation strategy that utilises spatial Gaussian blending of interpolated OpenAQ observations. Extensive qualitative and station-level evaluations across Europe demonstrate that our model recovers fine-grained spatial structures and effectively mitigates localised CAMS biases.

cs.LG↗

arXiv2026

PolySAE: Modeling Feature Interactions in Sparse Autoencoders via Polynomial Decoding

Sparse autoencoders (SAEs) interpret neural network representations by decomposing activations into sparse combinations of dictionary atoms. However, SAEs assume features combine additively through linear reconstruction, an assumption that cannot capture compositional structure: linear models cannot distinguish whether ''Starbucks'' arises from the composition of ''star'' and ''coffee'' features or merely their co-occurrence. This forces SAEs to allocate monolithic features for compound concepts rather than decomposing them into interpretable constituents. We introduce PolySAE, which extends the SAE decoder with higher-order terms to model feature interactions while preserving the linear encoder essential for interpretability. Through low-rank tensor factorization on a shared projection subspace, PolySAE captures pairwise and triple feature interactions with small parameter overhead (3% on GPT2). Across four language models and three SAE variants, PolySAE achieves an average improvement of $\sim$8% in probing F1 while maintaining comparable reconstruction error, and produces 2--10$\times$ larger Wasserstein distances between class-conditional feature distributions. Critically, learned interaction weights exhibit negligible correlation with co-occurrence frequency ($r = 0.06$ vs $r = 0.82$ for SAE feature covariance), suggesting that polynomial terms capture compositional structure largely independent of surface statistics. Finally, the learned interaction directions causally steer model outputs toward the corresponding compositional semantics.

cs.LG↗

arXiv2026

fmxcoders: Factorized Masked Crosscoders for Cross-Layer Feature Discovery

Many features in pretrained Transformers span multiple layers: they emerge through stages of inference, persist in the residual stream, or are built jointly by parallel MLPs. Crosscoders (namely, sparse dictionaries trained jointly across layers) aim to recover these cross-layer features in a single shared latent space. We show that standard crosscoders largely fail at this purpose. Although their decoder weight norms spread evenly across layers, a functional coherence metric we introduce reveals that each latent's activation is effectively driven by only one or two layers on average. While functionally coherent latents act as human-interpretable concept detectors (e.g., US states and cities), the layer-localized latents that crosscoders predominantly learn collapse onto surface-level patterns such as digit detectors. We trace this failure to two structural limitations: unconstrained cross-layer parameterization and unregularized cross-layer dependence. We address both by introducing fmxcoders, which (i) replace the encoder and decoder with low-rank tensor factorizations that draw every latent's per-layer weights from a shared cross-layer basis, and (ii) apply stochastic layer masking, a denoising regularizer along the layer axis that penalizes latents whose contribution collapses when a single layer is masked. Across GPT2-Small, Pythia-410M, Pythia-1.4B, and Gemma2-2B, fmxcoders lift mean probing F1 by 10-30 points, surpassing per-layer SAE baselines that standard crosscoders fail to reach, reduce reconstruction MSE by 25-50%, and roughly double mean functional coherence. An LLM-as-a-judge evaluation further shows that fmxcoders recover 3-13$\times$ more semantically coherent latents than standard crosscoders across all four base LLMs.

cs.LG↗

arXiv2022

Evaporating Rayleigh-Bénard convection: prediction of interface temperature and global heat transfer modulation

We propose an analytical model to estimate the interface temperature $Θ_Γ$ and the Nusselt number $Nu$ for an evaporating two-layer Rayleigh-Bénard configuration in statistically stationary conditions. The model is based on three assumptions: (i) the Oberbeck-Boussinesq approximation can be applied to the liquid phase, while the gas thermophysical properties are generic functions of thermodynamic pressure, local temperature, and vapour composition, (ii) the Grossmann-Lohse theory for thermal convection can be applied to the liquid and gas layers separately, (iii) the vapour content in the gas can be taken as the mean value at the gas-liquid interface. We validate this setting using direct numerical simulations (DNS) in a parameter space composed of the Rayleigh number ($10^6\leq Ra\leq 10^8$) and the temperature differential ($0.05\leq\varepsilon\leq 0.20$), which modulates the variation of state variables in the gas layer. To better disentangle the variable property effects on $Θ_Γ$ and $Nu$, simulations are performed in two conditions. First, we consider the case of uniform gas properties except for the gas density and gas-liquid diffusion coefficient. Second, we include the variation of specific heat capacity, dynamic viscosity, and thermal conductivity using realistic equations of state. Irrespective of the employed setting, the proposed model agrees very well with the numerical simulations over the entire range of $Ra-\varepsilon$ investigated.

physics.flu-dyn↗

arXiv2022

FluTAS: A GPU-accelerated finite difference code for multiphase flows

We present the Fluid Transport Accelerated Solver, FluTAS, a scalable GPU code for multiphase flows with thermal effects. The code solves the incompressible Navier-Stokes equation for two-fluid systems, with a direct FFT-based Poisson solver for the pressure equation. The interface between the two fluids is represented with the Volume of Fluid (VoF) method, which is mass conserving and well suited for complex flows thanks to its capacity of handling topological changes. The energy equation is explicitly solved and coupled with the momentum equation through the Boussinesq approximation. The code is conceived in a modular fashion so that different numerical methods can be used independently, the existing routines can be modified, and new ones can be included in a straightforward and sustainable manner. FluTAS is written in modern Fortran and parallelized using hybrid MPI/OpenMP in the CPU-only version and accelerated with OpenACC directives in the GPU implementation. We present different benchmarks to validate the code, and two large-scale simulations of fundamental interest in turbulent multiphase flows: isothermal emulsions in HIT and two-layer Rayleigh-Bénard convection. FluTAS is distributed through a MIT license and arises from a collaborative effort of several scientists, aiming to become a flexible tool to study complex multiphase flows.

physics.flu-dyn↗

arXiv2021

A pressure-based diffuse interface method for low-Mach multiphase flows with mass transfer

This study presents a novel pressure-based methodology for the efficient numerical solution of a four-equation two-phase diffuse interface model. The proposed methodology has the potential to simulate low-Mach flows with mass transfer. In contrast to the classical conservative four-equation model formulation, the adopted set of equations features volume fraction, temperature, velocity and pressure as the primary variables. The model includes the effects of viscosity, surface tension, thermal conductivity and gravity, and has the ability to incorporate complex equations of state. Additionally, a Gibbs free energy relaxation procedure is used to model mass transfer. A key characteristic of the proposed methodology is the use of high performance and scalable solvers for the solution of the Helmholtz equation for the pressure, which drastically reduces the computational cost compared to analogous density-based approaches. We demonstrate the capabilities of the methodology to simulate flows with large density and viscosity ratios through extended verification against a range of different test cases. Finally, the potential of the methodology to tackle challenging phase change flows is demonstrated with the simulation of three-dimensional nucleate boiling.

physics.flu-dyn↗