arXiv subjects

Lander Ver Hoef

Publications and source records attributed to Lander Ver Hoef.

5 recordsLinked to original sources

arXiv2025

Persistent Homology for Labeled Datasets: Gromov-Hausdorff Stability and Generalized Landscapes

Techniques from metric geometry have become fundamental tools in modern mathematical data science, providing principled methods for comparing datasets modeled as finite metric spaces. Two of the central tools in this area are the Gromov-Hausdorff distance and persistent homology, both of which yield isometry-invariant notions of distance between datasets. However, these frameworks do not account for categorical labels, which are intrinsic to many real-world datasets, such as labeled images, pre-clustered data, and semantically segmented shapes. In this paper, we introduce a general framework for labeled metric spaces and develop new notions of Gromov-Hausdorff distance and persistent homology which are adapted to this setting. Our main result shows that our persistent homology construction is stable with respect to our novel notion of Gromov-Hausdorff distance, extending a classic result in topological data analysis. To facilitate computation, we also introduce a labeled version of persistence landscapes and show that the landscape map is Lipschitz.

math.AT↗

arXiv2025

Using machine learning to downscale coarse-resolution environmental variables for understanding the spatial frequency of convective storms

Global climate models (GCMs), typically run at ~100-km resolution, capture large-scale environmental conditions but cannot resolve convection and cloud processes at kilometer scales. Convection-permitting models offer higher-resolution simulations that explicitly simulate convection but are computationally expensive and impractical for large ensemble runs. This study explores machine learning (ML) as a bridge between these approaches. We train simple, pixel-based neural networks to predict convective storm frequency from environmental variables produced by a regional convection-permitting model. The ML models achieve promising results, with structural similarity index measure (SSIM) values exceeding 0.8, capturing the diurnal cycle and orographic convection without explicit temporal or spatial coordinates as input. Model performance declines when fewer input features are used or specific regions are excluded, underscoring the role of diverse physical mechanisms in convective activity. These findings highlight ML potential as a computationally efficient tool for representing convection and as a means of scientific discovery, offering insights into convective processes. Unlike convolutional neural networks, which depend on spatial structure and grid size, the pixel-based model treats each grid point independently, enabling value-to-value prediction without spatial context. This design enhances adaptability to resolution changes and supports generalization to unseen environmental regimes, making it particularly suited for linking environmental conditions to convective features and for application across diverse model grids or climate scenarios.

physics.ao-ph↗

arXiv2025

Knowledge-Guided Machine Learning: Illustrating the use of Explainable Boosting Machines to Identify Overshooting Tops in Satellite Imagery

Machine learning (ML) algorithms have emerged in many meteorological applications. However, these algorithms struggle to extrapolate beyond the data they were trained on, i.e., they may adopt faulty strategies that lead to catastrophic failures. These failures are difficult to predict due to the opaque nature of ML algorithms. In high-stakes applications, such as severe weather forecasting, is is crucial to avoid such failures. One approach to address this issue is to develop more interpretable ML algorithms. The primary goal of this work is to illustrate the use of a specific interpretable ML algorithm that has not yet found much use in meteorology, Explainable Boosting Machines (EBMs). We demonstrate that EBMs are particularly suitable to implement human-guided strategies in an ML algorithm. As guiding example, we show how to develop an EBM to detect overshooting tops (OTs) in satellite imagery. EBMs require input features to be scalar. We use techniques from Knowledge-Guided Machine Learning to first extract scalar features from meteorological imagery. For the application of identifying OTs this includes extracting cloud texture from satellite imagery using Gray-Level Co-occurrence Matrices. Once trained, the EBM was examined and minimally altered to more closely match strategies used by domain scientists to identify OTs. The result of our efforts is a fully interpretable ML algorithm developed in a human-machine collaboration that uses human-guided strategies. While the final model does not reach the accuracy of more complex approaches, it performs reasonably well and we hope paves the way for building more interpretable ML algorithms for this and other meteorological applications.

cs.CV↗

arXiv2025

How to use score-based diffusion in earth system science: A satellite nowcasting example

Machine learning (ML) is used for many earth science applications; however, traditional ML methods trained with squared errors often create blurry forecasts. Diffusion models are an emerging generative ML technique with the ability to produce sharper, more realistic images by learning the underlying data distribution. Diffusion models are becoming more prevalent, yet adapting them for earth science applications can be challenging because most articles focus on theoretical aspects of the approach, rather than making the method widely accessible. This work illustrates score-based diffusion models with a well-known problem in atmospheric science: cloud nowcasting (zero-to-three-hour forecast). After discussing the background and intuition of score-based diffusion models using examples from geostationary satellite infrared imagery, we experiment with three types of diffusion models: a standard score-based diffusion model (Diff); a residual correction diffusion model (CorrDiff); and a latent diffusion model (LDM). Our results show that the diffusion models not only advect existing clouds, but also generate and decay clouds, including convective initiation. A case study qualitatively shows the preservation of high-resolution features longer into the forecast than a conventional U-Net. The best of the three diffusion models tested was the CorrDiff approach, outperforming all other diffusion models, the conventional U-Net, and persistence. The diffusion models also enable out-of-the-box ensemble generation with skillful calibration. By explaining and exploring diffusion models for a common problem and ending with lessons learned from adapting diffusion models for our task, this work provides a starting point for the community to utilize diffusion models for a variety of earth science applications.

cs.LG↗

arXiv2022

A Primer on Topological Data Analysis to Support Image Analysis Tasks in Environmental Science

Topological data analysis (TDA) is a tool from data science and mathematics that is beginning to make waves in environmental science. In this work, we seek to provide an intuitive and understandable introduction to a tool from TDA that is particularly useful for the analysis of imagery, namely persistent homology. We briefly discuss the theoretical background but focus primarily on understanding the output of this tool and discussing what information it can glean. To this end, we frame our discussion around a guiding example of classifying satellite images from the Sugar, Fish, Flower, and Gravel Dataset produced for the study of mesocale organization of clouds by Rasp et. al. in 2020 (arXiv:1906:01906). We demonstrate how persistent homology and its vectorization, persistence landscapes, can be used in a workflow with a simple machine learning algorithm to obtain good results, and explore in detail how we can explain this behavior in terms of image-level features. One of the core strengths of persistent homology is how interpretable it can be, so throughout this paper we discuss not just the patterns we find, but why those results are to be expected given what we know about the theory of persistent homology. Our goal is that a reader of this paper will leave with a better understanding of TDA and persistent homology, be able to identify problems and datasets of their own for which persistent homology could be helpful, and gain an understanding of results they obtain from applying the included GitHub example code.

cs.LG↗