Searcharxiv⌕ Search

arXiv subjects

Michael Schmitt

Publications and source records attributed to Michael Schmitt.

At least 37 records · Page 2Linked to original sources
arXiv2022

Broadband Four-Wave Mixing Enhanced by Plasmonic Surface Lattice Resonance and Localized Surface Plasmon Resonance in an Azimuthally Chirped Grating

Plasmonic enhancement of nonlinear light-matter interaction can be achieved via dedicated optimization of resonant plasmonic modes that are spectrally matched to the different wavelengths involved in the particular nonlinear optical process. In this work, we investigate the generation and enhancement of broadband four-wave mixing (FWM) in a plasmonic azimuthally chirped grating (ACG). The azimuthally varying grating periodicity in an ACG offers a well-defined channel to mediate the near field and the far field over a broad range of wavelengths. However, the particular mechanism responsible for field enhancement in such a platform depends on the interplay between the effects manifested by both the groove geometry and the grating's periodicity. This work delineates the collective contribution of groove geometry-dependent localized surface plasmon resonance (LSPR) and periodicity-dependent plasmonic surface lattice resonance (PSLR) over a broad range of wavelengths to bring into effect the enhancement of broadband FWM in an ACG.

physics.optics↗
arXiv2022

EOD: The IEEE GRSS Earth Observation Database

In the era of deep learning, annotated datasets have become a crucial asset to the remote sensing community. In the last decade, a plethora of different datasets was published, each designed for a specific data type and with a specific task or application in mind. In the jungle of remote sensing datasets, it can be hard to keep track of what is available already. With this paper, we introduce EOD - the IEEE GRSS Earth Observation Database (EOD) - an interactive online platform for cataloguing different types of datasets leveraging remote sensing imagery.

cs.CV↗
arXiv2022

Nonlinear optical signal generation mediated by a plasmonic azimuthally chirped grating

The deployment of plasmonic nanostructures to enhance nonlinear signal generation requires effective far-to-near field coupling and phase matching for frequency conversion. While the latter can be easily achieved at plasmonic hotspots, the former is an antenna problem that requires dedicated structural design and optimization. Plasmonic gratings are a simple but effective platform for nonlinear signal generation since they provide a well-defined momentum for photon-plasmon coupling and local hotspots for frequency conversion. In this work, a plasmonic azimuthally chirped grating (ACG), which provides spatially resolved broadband momentum for photon-plasmon coupling, was exploited to investigate the plasmonic enhancement effect in two nonlinear optical processes, namely two-photon photoluminescence (TPPL) and second-harmonic generation (SHG). The spatial distributions of the nonlinear signals were determined experimentally by hyperspectral mapping with ultrashort pulsed excitation. The experimental spatial distributions of nonlinear signals agree very well with the analytical prediction based solely on photon-plasmon coupling with the momentum of the ACG, revealing the antenna function of the grating in plasmonic nonlinear signal generation. This work highlights the importance of the antenna effect of the gratings for nonlinear signal generation and provides insight into the enhancement mechanism of plasmonic gratings in addition to local hotspot engineering.

physics.optics↗
arXiv2022

SEN12MS-CR-TS: A Remote Sensing Data Set for Multi-modal Multi-temporal Cloud Removal

About half of all optical observations collected via spaceborne satellites are affected by haze or clouds. Consequently, cloud coverage affects the remote sensing practitioner's capabilities of a continuous and seamless monitoring of our planet. This work addresses the challenge of optical satellite image reconstruction and cloud removal by proposing a novel multi-modal and multi-temporal data set called SEN12MS-CR-TS. We propose two models highlighting the benefits and use cases of SEN12MS-CR-TS: First, a multi-modal multi-temporal 3D-Convolution Neural Network that predicts a cloud-free image from a sequence of cloudy optical and radar images. Second, a sequence-to-sequence translation model that predicts a cloud-free time series from a cloud-covered time series. Both approaches are evaluated experimentally, with their respective models trained and tested on SEN12MS-CR-TS. The conducted experiments highlight the contribution of our data set to the remote sensing community as well as the benefits of multi-modal and multi-temporal information to reconstruct noisy information. Our data set is available at https://patrickTUM.github.io/cloud_removal

cs.CV↗
arXiv2021

Deep-Learning-Based Single-Image Height Reconstruction from Very-High-Resolution SAR Intensity Data

Originally developed in fields such as robotics and autonomous driving with image-based navigation in mind, deep learning-based single-image depth estimation (SIDE) has found great interest in the wider image analysis community. Remote sensing is no exception, as the possibility to estimate height maps from single aerial or satellite imagery bears great potential in the context of topographic reconstruction. A few pioneering investigations have demonstrated the general feasibility of single image height prediction from optical remote sensing images and motivate further studies in that direction. With this paper, we present the first-ever demonstration of deep learning-based single image height prediction for the other important sensor modality in remote sensing: synthetic aperture radar (SAR) data. Besides the adaptation of a convolutional neural network (CNN) architecture for SAR intensity images, we present a workflow for the generation of training data, and extensive experimental results for different SAR imaging modes and test sites. Since we put a particular emphasis on transferability, we are able to confirm that deep learning-based single-image height estimation is not only possible, but also transfers quite well to unseen data, even if acquired by different imaging modes and imaging parameters.

cs.CV↗
arXiv2021

Synthesizing Optical and SAR Imagery From Land Cover Maps and Auxiliary Raster Data

We synthesize both optical RGB and synthetic aperture radar (SAR) remote sensing images from land cover maps and auxiliary raster data using generative adversarial networks (GANs). In remote sensing, many types of data, such as digital elevation models (DEMs) or precipitation maps, are often not reflected in land cover maps but still influence image content or structure. Including such data in the synthesis process increases the quality of the generated images and exerts more control on their characteristics. Spatially adaptive normalization layers fuse both inputs and are applied to a full-blown generator architecture consisting of encoder and decoder to take full advantage of the information content in the auxiliary raster data. Our method successfully synthesizes medium (10 m) and high (1 m) resolution images when trained with the corresponding data set. We show the advantage of data fusion of land cover maps and auxiliary information using mean intersection over unions (mIoUs), pixel accuracy, and Fréchet inception distances (FIDs) using pretrained U-Net segmentation models. Handpicked images exemplify how fusing information avoids ambiguities in the synthesized images. By slightly editing the input, our method can be used to synthesize realistic changes, i.e., raising the water levels. The source code is available at https://github.com/gbaier/rs_img_synth and we published the newly created high-resolution dataset at https://ieee-dataport.org/open-access/geonrw.

cs.CV↗
arXiv2021

Remote Sensing Image Classification with the SEN12MS Dataset

Image classification is one of the main drivers of the rapid developments in deep learning with convolutional neural networks for computer vision. So is the analogous task of scene classification in remote sensing. However, in contrast to the computer vision community that has long been using well-established, large-scale standard datasets to train and benchmark high-capacity models, the remote sensing community still largely relies on relatively small and often application-dependend datasets, thus lacking comparability. With this letter, we present a classification-oriented conversion of the SEN12MS dataset. Using that, we provide results for several baseline models based on two standard CNN architectures and different input data configurations. Our results support the benchmarking of remote sensing image classification and provide insights to the benefit of multi-spectral data and multi-sensor data fusion over conventional RGB imagery.

cs.CV↗
arXiv2021

Spatially Resolving the Enhancement Effect in Surface-Enhanced Coherent Anti-Stokes Raman Scattering by Plasmonic Doppler Gratings

In this work, we introduce the platform of plasmonic Doppler grating (PDG) to experimentally investigate the enhancement effect of plasmonic gratings in the input and output beams of nonlinear surface-enhanced coherent anti-Stokes Raman scattering (SECARS). PDGs are designable azimuthally chirped gratings that provide broadband and spatially dispersed plasmonic enhancement. Therefore, they offer the opportunity to observe and compare the overall enhancement from different combinations of enhancement in individual input and output beams. We first confirm PDG's capability of spatially separating the input and output enhancement in linear surface-enhanced fluorescence and Raman scattering. We then investigate spatially resolved enhancement in nonlinear SECARS, where coherent interaction of the pump, Stokes, and anti-Stokes beams is enhanced by the plasmonic gratings. By mapping the SECARS signal and analyzing the azimuthal angle-dependent intensity, we characterize the enhancement at individual frequencies. Together with theoretical analysis, we show that while simultaneous enhancement in the input and output beams is important for SECARS, the enhancement in the pump and anti-Stokes beams plays a more critical role in the overall enhancement than that in the Stokes beam. This work provides an insight into the enhancement mechanism of plasmon-enhanced spectroscopy, which is important for the design and optimization of plasmonic gratings. The PDG platform may also be applied to study enhancement mechanisms in other nonlinear light-matter interactions or the impact of plasmonic gratings on the fluorescence lifetime.

physics.optics↗
arXiv2020

Multi-task Learning for Human Settlement Extent Regression and Local Climate Zone Classification

Human Settlement Extent (HSE) and Local Climate Zone (LCZ) maps are both essential sources, e.g., for sustainable urban development and Urban Heat Island (UHI) studies. Remote sensing (RS)- and deep learning (DL)-based classification approaches play a significant role by providing the potential for global mapping. However, most of the efforts only focus on one of the two schemes, usually on a specific scale. This leads to unnecessary redundancies, since the learned features could be leveraged for both of these related tasks. In this letter, the concept of multi-task learning (MTL) is introduced to HSE regression and LCZ classification for the first time. We propose a MTL framework and develop an end-to-end Convolutional Neural Network (CNN), which consists of a backbone network for shared feature learning, attention modules for task-specific feature learning, and a weighting strategy for balancing the two tasks. We additionally propose to exploit HSE predictions as a prior for LCZ classification to enhance the accuracy. The MTL approach was extensively tested with Sentinel-2 data of 13 cities across the world. The results demonstrate that the framework is able to provide a competitive solution for both tasks.

cs.CV↗
arXiv2020

Mapping horizontal and vertical urban densification in Denmark with Landsat time-series from 1985 to 2018: a semantic segmentation solution

Landsat imagery is an unparalleled freely available data source that allows reconstructing horizontal and vertical urban form. This paper addresses the challenge of using Landsat data, particularly its 30m spatial resolution, for monitoring three-dimensional urban densification. We compare temporal and spatial transferability of an adapted DeepLab model with a simple fully convolutional network (FCN) and a texture-based random forest (RF) model to map urban density in the two morphological dimensions: horizontal (compact, open, sparse) and vertical (high rise, low rise). We test whether a model trained on the 2014 data can be applied to 2006 and 1995 for Denmark, and examine whether we could use the model trained on the Danish data to accurately map other European cities. Our results show that an implementation of deep networks and the inclusion of multi-scale contextual information greatly improve the classification and the model's ability to generalize across space and time. DeepLab provides more accurate horizontal and vertical classifications than FCN when sufficient training data is available. By using DeepLab, the F1 score can be increased by 4 and 10 percentage points for detecting vertical urban growth compared to FCN and RF for Denmark. For mapping the other European cities with training data from Denmark, DeepLab also shows an advantage of 6 percentage points over RF for both the dimensions. The resulting maps across the years 1985 to 2018 reveal different patterns of urban growth between Copenhagen and Aarhus, the two largest cities in Denmark, illustrating that those cities have used various planning policies in addressing population growth and housing supply challenges. In summary, we propose a transferable deep learning approach for automated, long-term mapping of urban form from Landsat images.

eess.IV↗
arXiv2020

Multi-Sensor Data Fusion for Cloud Removal in Global and All-Season Sentinel-2 Imagery

This work has been accepted by IEEE TGRS for publication. The majority of optical observations acquired via spaceborne earth imagery are affected by clouds. While there is numerous prior work on reconstructing cloud-covered information, previous studies are oftentimes confined to narrowly-defined regions of interest, raising the question of whether an approach can generalize to a diverse set of observations acquired at variable cloud coverage or in different regions and seasons. We target the challenge of generalization by curating a large novel data set for training new cloud removal approaches and evaluate on two recently proposed performance metrics of image quality and diversity. Our data set is the first publically available to contain a global sample of co-registered radar and optical observations, cloudy as well as cloud-free. Based on the observation that cloud coverage varies widely between clear skies and absolute coverage, we propose a novel model that can deal with either extremes and evaluate its performance on our proposed data set. Finally, we demonstrate the superiority of training models on real over synthetic data, underlining the need for a carefully curated data set of real observations. To facilitate future research, our data set is made available online

eess.IV↗
arXiv2020

Modeling Magnetic Fields with Helical Solutions to Laplace's Equation

The series solution to Laplace's equation in a helical coordinate system is derived and refined using symmetry and chirality arguments. These functions and their more commonplace counterparts are used to model solenoidal magnetic fields via linear, multidimensional curve-fitting. A judicious choice of functional forms, a small number of free parameters and sparse input data can lead to highly accurate, fine-grained modeling of solenoidal magnetic fields, including helical features arising from the winding of the solenoid, with overall field accuracy at better than one part per million.

physics.ins-det↗
arXiv2020

Multi-level Feature Fusion-based CNN for Local Climate Zone Classification from Sentinel-2 Images: Benchmark Results on the So2Sat LCZ42 Dataset

As a unique classification scheme for urban forms and functions, the local climate zone (LCZ) system provides essential general information for any studies related to urban environments, especially on a large scale. Remote sensing data-based classification approaches are the key to large-scale mapping and monitoring of LCZs. The potential of deep learning-based approaches is not yet fully explored, even though advanced convolutional neural networks (CNNs) continue to push the frontiers for various computer vision tasks. One reason is that published studies are based on different datasets, usually at a regional scale, which makes it impossible to fairly and consistently compare the potential of different CNNs for real-world scenarios. This study is based on the big So2Sat LCZ42 benchmark dataset dedicated to LCZ classification. Using this dataset, we studied a range of CNNs of varying sizes. In addition, we proposed a CNN to classify LCZs from Sentinel-2 images, Sen2LCZ-Net. Using this base network, we propose fusing multi-level features using the extended Sen2LCZ-Net-MF. With this proposed simple network architecture and the highly competitive benchmark dataset, we obtain results that are better than those obtained by the state-of-the-art CNNs, while requiring less computation with fewer layers and parameters. Large-scale LCZ classification examples of completely unseen areas are presented, demonstrating the potential of our proposed Sen2LCZ-Net-MF as well as the So2Sat LCZ42 dataset. We also intensively investigated the influence of network depth and width and the effectiveness of the design choices made for Sen2LCZ-Net-MF. Our work will provide important baselines for future CNN-based algorithm developments for both LCZ classification and other urban land cover land use classification.

eess.IV↗
arXiv2020

Weakly Supervised Semantic Segmentation of Satellite Images for Land Cover Mapping -- Challenges and Opportunities

Fully automatic large-scale land cover mapping belongs to the core challenges addressed by the remote sensing community. Usually, the basis of this task is formed by (supervised) machine learning models. However, in spite of recent growth in the availability of satellite observations, accurate training data remains comparably scarce. On the other hand, numerous global land cover products exist and can be accessed often free-of-charge. Unfortunately, these maps are typically of a much lower resolution than modern day satellite imagery. Besides, they always come with a significant amount of noise, as they cannot be considered ground truth, but are products of previous (semi-)automatic prediction tasks. Therefore, this paper seeks to make a case for the application of weakly supervised learning strategies to get the most out of available data sources and achieve progress in high-resolution large-scale land cover mapping. Challenges and opportunities are discussed based on the SEN12MS dataset, for which also some baseline results are shown. These baselines indicate that there is still a lot of potential for dedicated approaches designed to deal with remote sensing-specific forms of weak supervision.

cs.CV↗
arXiv2019

So2Sat LCZ42: A Benchmark Dataset for Global Local Climate Zones Classification

Access to labeled reference data is one of the grand challenges in supervised machine learning endeavors. This is especially true for an automated analysis of remote sensing images on a global scale, which enables us to address global challenges such as urbanization and climate change using state-of-the-art machine learning techniques. To meet these pressing needs, especially in urban research, we provide open access to a valuable benchmark dataset named "So2Sat LCZ42," which consists of local climate zone (LCZ) labels of about half a million Sentinel-1 and Sentinel-2 image patches in 42 urban agglomerations (plus 10 additional smaller areas) across the globe. This dataset was labeled by 15 domain experts following a carefully designed labeling work flow and evaluation process over a period of six months. As rarely done in other labeled remote sensing dataset, we conducted rigorous quality assessment by domain experts. The dataset achieved an overall confidence of 85%. We believe this LCZ dataset is a first step towards an unbiased globallydistributed dataset for urban growth monitoring using machine learning methods, because LCZ provide a rather objective measure other than many other semantic land use and land cover classifications. It provides measures of the morphology, compactness, and height of urban areas, which are less dependent on human and culture. This dataset can be accessed from http://doi.org/10.14459/2018mp1483140.

cs.CV↗
arXiv2019

SEN12MS -- A Curated Dataset of Georeferenced Multi-Spectral Sentinel-1/2 Imagery for Deep Learning and Data Fusion

The availability of curated large-scale training data is a crucial factor for the development of well-generalizing deep learning methods for the extraction of geoinformation from multi-sensor remote sensing imagery. While quite some datasets have already been published by the community, most of them suffer from rather strong limitations, e.g. regarding spatial coverage, diversity or simply number of available samples. Exploiting the freely available data acquired by the Sentinel satellites of the Copernicus program implemented by the European Space Agency, as well as the cloud computing facilities of Google Earth Engine, we provide a dataset consisting of 180,662 triplets of dual-pol synthetic aperture radar (SAR) image patches, multi-spectral Sentinel-2 image patches, and MODIS land cover maps. With all patches being fully georeferenced at a 10 m ground sampling distance and covering all inhabited continents during all meteorological seasons, we expect the dataset to support the community in developing sophisticated deep learning-based approaches for common tasks such as scene classification or semantic segmentation for land cover mapping.

cs.CV↗