Searcharxiv⌕ Search

arXiv subjects

Sebastian Schmitt

Publications and source records attributed to Sebastian Schmitt.

At least 55 records · Page 3Linked to original sources

PREPRINT: Comparison of deep learning and hand crafted features for mining simulation data

Computational Fluid Dynamics (CFD) simulations are a very important tool for many industrial applications, such as aerodynamic optimization of engineering designs like cars shapes, airplanes parts etc. The output of such simulations, in particular the calculated flow fields, are usually very complex and hard to interpret for realistic three-dimensional real-world applications, especially if time-dependent simulations are investigated. Automated data analysis methods are warranted but a non-trivial obstacle is given by the very large dimensionality of the data. A flow field typically consists of six measurement values for each point of the computational grid in 3D space and time (velocity vector values, turbulent kinetic energy, pressure and viscosity). In this paper we address the task of extracting meaningful results in an automated manner from such high dimensional data sets. We propose deep learning methods which are capable of processing such data and which can be trained to solve relevant tasks on simulation data, i.e. predicting drag and lift forces applied on an airfoil. We also propose an adaptation of the classical hand crafted features known from computer vision to address the same problem and compare a large variety of descriptors and detectors. Finally, we compile a large dataset of 2D simulations of the flow field around airfoils which contains 16000 flow fields with which we tested and compared approaches. Our results show that the deep learning-based methods, as well as hand crafted feature based approaches, are well-capable to accurately describe the content of the CFD simulation output on the proposed dataset.

cs.CV↗

Preprint: Norm Loss: An efficient yet effective regularization method for deep neural networks

Convolutional neural network training can suffer from diverse issues like exploding or vanishing gradients, scaling-based weight space symmetry and covariant-shift. In order to address these issues, researchers develop weight regularization methods and activation normalization methods. In this work we propose a weight soft-regularization method based on the Oblique manifold. The proposed method uses a loss function which pushes each weight vector to have a norm close to one, i.e. the weight matrix is smoothly steered toward the so-called Oblique manifold. We evaluate our method on the very popular CIFAR-10, CIFAR-100 and ImageNet 2012 datasets using two state-of-the-art architectures, namely the ResNet and wide-ResNet. Our method introduces negligible computational overhead and the results show that it is competitive to the state-of-the-art and in some cases superior to it. Additionally, the results are less sensitive to hyperparameter settings such as batch size and regularization factor.

cs.CV↗

Real-World Anomaly Detection by using Digital Twin Systems and Weakly-Supervised Learning

The continuously growing amount of monitored data in the Industry 4.0 context requires strong and reliable anomaly detection techniques. The advancement of Digital Twin technologies allows for realistic simulations of complex machinery, therefore, it is ideally suited to generate synthetic datasets for the use in anomaly detection approaches when compared to actual measurement data. In this paper, we present novel weakly-supervised approaches to anomaly detection for industrial settings. The approaches make use of a Digital Twin to generate a training dataset which simulates the normal operation of the machinery, along with a small set of labeled anomalous measurement from the real machinery. In particular, we introduce a clustering-based approach, called Cluster Centers (CC), and a neural architecture based on the Siamese Autoencoders (SAE), which are tailored for weakly-supervised settings with very few labeled data samples. The performance of the proposed methods is compared against various state-of-the-art anomaly detection algorithms on an application to a real-world dataset from a facility monitoring system, by using a multitude of performance measures. Also, the influence of hyper-parameters related to feature extraction and network architecture is investigated. We find that the proposed SAE based solutions outperform state-of-the-art anomaly detection approaches very robustly for many different hyper-parameter settings on all performance measures.

cs.LG↗

hxtorch: PyTorch for BrainScaleS-2 -- Perceptrons on Analog Neuromorphic Hardware

We present software facilitating the usage of the BrainScaleS-2 analog neuromorphic hardware system as an inference accelerator for artificial neural networks. The accelerator hardware is transparently integrated into the PyTorch machine learning framework using its extension interface. In particular, we provide accelerator support for vector-matrix multiplications and convolutions; corresponding software-based autograd functionality is provided for hardware-in-the-loop training. Automatic partitioning of neural networks onto one or multiple accelerator chips is supported. We analyze implementation runtime overhead during training as well as inference, provide measurements for existing setups and evaluate the results in terms of the accelerator hardware design limitations. As an application of the introduced framework, we present a model that classifies activities of daily living with smartphone sensor data.

cs.NE↗

Accelerated physical emulation of Bayesian inference in spiking neural networks

The massively parallel nature of biological information processing plays an important role for its superiority to human-engineered computing devices. In particular, it may hold the key to overcoming the von Neumann bottleneck that limits contemporary computer architectures. Physical-model neuromorphic devices seek to replicate not only this inherent parallelism, but also aspects of its microscopic dynamics in analog circuits emulating neurons and synapses. However, these machines require network models that are not only adept at solving particular tasks, but that can also cope with the inherent imperfections of analog substrates. We present a spiking network model that performs Bayesian inference through sampling on the BrainScaleS neuromorphic platform, where we use it for generative and discriminative computations on visual data. By illustrating its functionality on this platform, we implicitly demonstrate its robustness to various substrate-specific distortive effects, as well as its accelerated capability for computation. These results showcase the advantages of brain-inspired physical computation and provide important building blocks for large-scale neuromorphic applications.

cs.NE↗

Exploring the fitness landscape of a realistic turbofan rotor blade optimization

Aerodynamic shape optimization has established itself as a valuable tool in the engineering design process to achieve highly efficient results. A central aspect for such approaches is the mapping from the design parameters which encode the geometry of the shape to be improved to the quality criteria which describe its performance. The choices to be made in the setup of the optimization process strongly influence this mapping and thus are expected to have a profound influence on the achievable result. In this work we explore the influence of such choices on the effects on the shape optimization of a turbofan rotor blade as it can be realized within an aircraft engine design process. The blade quality is assessed by realistic three dimensional computational fluid dynamics (CFD) simulations. We investigate the outcomes of several optimization runs which differ in various configuration options, such as optimization algorithm, initialization, number of degrees of freedom for the parametrization. For all such variations, we generally find that the achievable improvement of the blade quality is comparable for most settings and thus rather insensitive to the details of the setup. On the other hand, even supposedly minor changes in the settings, such as using a different random seed for the initialization of the optimizer algorithm, lead to very different shapes. Optimized shapes which show comparable performance usually differ quite strongly in their geometries over the complete blade. Our analyses indicate that the fitness landscape for such a realistic turbofan rotor blade optimization is highly multi-modal with many local optima, where very different shapes show similar performance.

cs.CE↗

Limits and trade-offs of topological network robustness

We investigate the trade-off between the robustness against random and targeted removal of nodes from a network. To this end we utilize the stochastic block model to study ensembles of infinitely large networks with arbitrary large-scale structures. We present results from numerical two-objective optimization simulations for networks with various fixed mean degree and number of blocks. The results provide strong evidence that three different blocks are sufficient to realize the best trade-off between the two measures of robustness, i.e.\ to obtain the complete front of Pareto-optimal networks. For all values of the mean degree, a characteristic three block structure emerges over large parts of the Pareto-optimal front. This structure can be often characterized as a core-periphery structure, composed of a group of core nodes with high degree connected among themselves and to a periphery of low-degree nodes, in addition to a third group of nodes which is disconnected from the periphery, and weakly connected to the core. Only at both extremes of the Pareto-optimal front, corresponding to maximal robustness against random and targeted node removal, a two-block core-periphery structure or a one-block fully random network are found, respectively.

cs.NI↗

Pattern representation and recognition with accelerated analog neuromorphic systems

Despite being originally inspired by the central nervous system, artificial neural networks have diverged from their biological archetypes as they have been remodeled to fit particular tasks. In this paper, we review several possibilites to reverse map these architectures to biologically more realistic spiking networks with the aim of emulating them on fast, low-power neuromorphic hardware. Since many of these devices employ analog components, which cannot be perfectly controlled, finding ways to compensate for the resulting effects represents a key challenge. Here, we discuss three different strategies to address this problem: the addition of auxiliary network components for stabilizing activity, the utilization of inherently robust architectures and a training method for hardware-emulated networks that functions without perfect knowledge of the system's dynamics and parameters. For all three scenarios, we corroborate our theoretical considerations with experimental results on accelerated analog neuromorphic platforms.

q-bio.NC↗

Neuromorphic Hardware In The Loop: Training a Deep Spiking Network on the BrainScaleS Wafer-Scale System

Emulating spiking neural networks on analog neuromorphic hardware offers several advantages over simulating them on conventional computers, particularly in terms of speed and energy consumption. However, this usually comes at the cost of reduced control over the dynamics of the emulated networks. In this paper, we demonstrate how iterative training of a hardware-emulated network can compensate for anomalies induced by the analog substrate. We first convert a deep neural network trained in software to a spiking network on the BrainScaleS wafer-scale neuromorphic system, thereby enabling an acceleration factor of 10 000 compared to the biological time domain. This mapping is followed by the in-the-loop training, where in each training step, the network activity is first recorded in hardware and then used to compute the parameter updates in software via backpropagation. An essential finding is that the parameter updates do not have to be precise, but only need to approximately follow the correct gradient, which simplifies the computation of updates. Using this approach, after only several tens of iterations, the spiking network shows an accuracy close to the ideal software-emulated prototype. The presented techniques show that deep spiking networks emulated on analog neuromorphic devices can attain good computational performance despite the inherent variations of the analog substrate.

cs.NE↗

Spectral properties of the two-impurity Anderson model with varying distance and various interactions

We present a novel approximation for two interacting magnetic impurities immersed into a noninteracting metallic host. The scheme is based on direct perturbation theory in the hybridization between the impurity and band electrons. This two-impurity enhanced noncrossing approximation (TIE) fully incorporates the indirect interaction between the impurities which are mediated by the conduction electrons as well as additional arbitrary direct interaction matrix elements. We qualify the approximation by investigating the uncoupled case and conclude the TIE to be equally accurate as its single impurity counterpart. The physical properties of the two-impurity Anderson model are first investigated in some limiting cases. In a regime, where each of the uncoupled two impurities would exhibit a pronounced Kondo effect, we ignore the indirect coupling via the conduction band and only incorporate direct interactions. For a ferromagnetic direct exchange coupling, the system displays a behavior similar to a spin-one Kondo effect, while an antiferromagnetic coupling competes with the Kondo effect and produces a pseudogap in the many-body Kondo resonance of the single-particle spectral function. Interestingly, a direct one-particle hopping also produces a pseudogap but additionally pronounced side-peaks emerge. This gap is characteristically different from the case with antiferromagnetic coupling, since it emerges as a consequence of distinct Kondo effects for the bonding and anti-bonding orbital, i.e.\ it reflects a splitting of even and odd parity states.For the general case of only indirect coupling the results show signatures of all the previously discussed limiting cases as function of the impurity-impurity distance. We are led to the conclusion that the well known Doniach-scenario captures essential aspects this model, but the details especially at small distances are more complicated.

cond-mat.str-el↗

Extension of dynamical mean-field theory by inclusion of nonlocal two-site correlations with variable distance

We present a novel approximation scheme for the treatment of strongly correlated electrons in arbitrary crystal lattices. The approach extends the well-known dynamical mean field theory to include nonlocal two-site correlations of arbitrary spatial extent. We extract the nonlocal correlation functions from two-impurity Anderson models where the impurity-impurity distance defines the spatial extent of the correlations included. Translational invariance is fully respected by our approach since correlation functions of any two-impurity cluster are periodically embedded to $k$-space via a Fourier transform. As a first application, we study the two-dimensional Hubbard model on a simple-cubic lattice. We demonstrate how pseudogap formation in the many-body resonance at the Fermi level results from the inclusion of nonlocal correlations.

cond-mat.str-el↗

Itinerant and local-moment magnetism in strongly correlated electron systems

Detailed analysis of the magnetic properties of the Hubbard model within dynamical mean-field theory (DMFT) is presented. Using a RPA-like decoupling of two-particle propagators we derive a universal form for susceptibilities, which captures essential aspects of localized and itinerant pictures. This expression is shown to be quantitatively valid whenever long-range coherence of particle-hole excitations can be neglected, as is the case in large parts of the phase diagram where antiferromag- netism is dominant. The applicability of an interpretation in terms of the two archetypical pictures of magnetism is investigated for the Hubbard model on a body-centered cubic lattice with additional next-nearest neighbor hopping t'. For large values of the Coulomb interaction, local-moment mag- netism is found to be dominant, while for weakly interacting band electrons itinerant quasiparticle magnetism prevails. In the intermediate regime and for finite t' an re-entrant behavior is discovered, where antiferromagnetism only exists in a finite temperature interval.

cond-mat.str-el↗

Kinks in the electronic dispersion of the Hubbard model away from half filling

We study kinks in the electronic dispersion of a generic strongly correlated system by dynamic mean-field theory (DMFT). The focus is on doped systems away from particle-hole symmetry where valence fluctuations matter potentially. Three different algorithms are compared to asses their strengths and weaknesses, as well as to clearly distinguish physical features from algorithmic artifacts. Our findings extend a view previously established for half-filled systems where kinks reflect the coupling of the fermionic quasiparticles to emergent collective modes, which are identified here as spin fluctuations. Kinks are observed when strong spin fluctuations are present and, additionally, a separation of energy scales for spin and charge excitations exists. Both criteria are met by strongly correlated systems close to a Mott-insulator transition. The energies of the kinks and their doping dependence fit well to the kinks in the cuprates, which is surprising in view of the spatial correlations neglected by DMFT.

cond-mat.str-el↗

Nonequilibrium Zeeman-splitting in quantum transport through nanoscale junctions

We calculate the nonequilibrium differential conductance $G(V)$ through a quantum dot as function of bias voltage $V$ and applied magnetic field $H$. We use a Keldysh conserving approximation for weakly correlated and the scattering states numerical renormalization group for the intermediate and strongly correlated regime out of equilibrium. In the weakly correlated regime, the Zeeman splitting observable in $G(V)$ strongly depends on the asymmetry of the coupling to the two leads, as well as on particle-hole asymmetry of the quantum dot. In contrast, in the strongly correlated regime, where Kondo-correlations dominate, the position $Δ_K$ of the Zeeman-split zero-bias anomaly is independent of such asymmetries and always found to be of the order of the Zeeman energy $Δ_0$. We find a crossover from the purely spin-fluctuation driven Kondo regime at small magnetic fields with $Δ_K<Δ_0$ to a regime at large fields where the contribution of charge fluctuations induces larger splittings with $Δ_K>Δ_0$ as it was observed in recent experiments.

cond-mat.str-el↗

Comment on "Scaling feature of magnetic field induced Kondo-peak splittings"

In a recent work Zhang and coworkers (PRB 82, 075111 (2010)) studied the Zeeman splitting of the Kondo resonance for the single impurity Anderson model in a finite magnetic field $B$ with the numerical renormalization group (NRG) method. There, it was found that with increasing magnetic field $B$ the position of the Kondo resonance in the total spectral function \textit{does not} approach its position in the spin resolved spectral function. Additionally, the position of the Kondo maximum exceeded the Zeeman energy for $B/ T_K\gtrsim 5-10$, where $T_K$ is the low energy Kondo scale of the model ($g=2$, $μ_B=k_B=\hbar=1$). In this comment we argue that both these findings are produced by an improper choice of NRG parameter values. However, we reproduce the crossover in the splitting from Kondo-like behavior to a non-universal splitting larger than the Zeeman energy, but this crossover occurs at much larger fields of the order of the charge scale.

cond-mat.str-el↗

Non-Fermi liquid signatures in the Hubbard Model due to van Hove singularities

When a van-Hove singularity is located in the vicinity of the Fermi level, the electronic scattering rate acquires a non-analytic contribution. This invalidates basic assumptions of Fermi liquid theory and within perturbative treatments leads to a non-Fermi liquid self-energy and transport properties.Such anomalies are shown to also occur in the strongly correlated metallic state. We consider the Hubbard model on a two-dimensional square lattice with nearest and next-nearest neighbor hopping within the single-site dynamical mean-field theory. At temperatures on the order of the low-energy scale $T_0$ an unusual maximum emerges in the imaginary part of the self-energy which is renormalized towards the Fermi level for finite doping. At zero temperature this double-well structure is suppressed, but an anomalous energy dependence of the self-energy remains. For the frustrated Hubbard model on the square lattice with next-nearest neighbor hopping, the presence of the van Hove singularity changes the asymptotic low temperature behavior of the resistivity from a Fermi liquid to non-Fermi liquid dependency as function of doping. The results of this work are discussed regarding their relevance for high-temperature cuprate superconductors.

cond-mat.str-el↗

Comparison between scattering-states numerical renormalization group and the Kadanoff-Baym-Keldysh approach to quantum transport: Crossover from weak to strong correlations

The quantum transport through nanoscale junctions is governed by the charging energy $U$ of the device. We employ the recently developed scattering-states numerical renormalization group approach to open quantum systems to study nonequilibrium Green functions and current-voltage characteristics of such junctions for small and intermediate values of $U$. The reliability of the approach is established by the excellent agreement with diagrammatic Kadanoff-Baym-Keldysh results at small values of the $U$. We demonstrate the limits of the diagrammatic approaches at intermediate Coulomb repulsion. These approaches predict two different low-energy scale for magnetic and charge fluctuations in zero bias while the numerical renormalization group approach correctly yields only one single, universal scale. At large voltages and intermediate values of the Coulomb repulsion the self-consistent second Born as well as the GW approximation reproduce the SNRG spectral functions quite well for a symmetric junctions, while for the asymmetric model the voltage-dependent redistribution of spectral weight differs significantly. The second-order perturbation theory does not capture the correct single-particle dynamics at large bias and violates current conservation for asymmetric junctions.

cond-mat.str-el↗

Dynamic susceptibilities of the single impurity Anderson model within an enhanced non-crossing approximation

The single impurity Anderson model (SIAM) is studied within an enhanced non-crossing approximation (ENCA). This method is extended to the calculation of susceptibilities and thoroughly tested, also in order to prepare applications as a building block for the calculation of susceptibilities and phase transitions in correlated lattice systems. A wide range of model parameters, such as impurity occupancy, temperature, local Coulomb repulsion and hybridization strength, are studied. Results for the spin and charge susceptibilities are presented. By comparing the static quantities to exact Bethe ansatz results, it is shown that the description of the magnetic excitations of the impurity within the ENCA is excellent, even in situations with large valence fluctuations or vanishing Coulomb repulsion. The description of the charge susceptibility is quite accurate in situations where the singly occupied ionic configuration is the unperturbed ground state; however, it seems to overestimate charge fluctuations in the asymmetric model at too low temperatures. The dynamic spin excitation spectra is dominated by the Kondo-screening of the impurity spin through the conduction band, i.e. the formation of the local Kondo-singlet. A finite local Coulomb interaction U leads to a drastic reduction of the charge response via processes involving the doubly occupied impurity state. In the asymmetric model, the charge susceptibility is enhanced for excitation energies smaller than the Kondo scale T_K due to the influence of valence fluctuations.

cond-mat.str-el↗