arXiv subjects

Markus Eisenbach

Publications and source records attributed to Markus Eisenbach.

At least 37 records · Page 2Linked to original sources

arXiv2023

A Little Bit Attention Is All You Need for Person Re-Identification

Person re-identification plays a key role in applications where a mobile robot needs to track its users over a long period of time, even if they are partially unobserved for some time, in order to follow them or be available on demand. In this context, deep-learning based real-time feature extraction on a mobile robot is often performed on special-purpose devices whose computational resources are shared for multiple tasks. Therefore, the inference speed has to be taken into account. In contrast, person re-identification is often improved by architectural changes that come at the cost of significantly slowing down inference. Attention blocks are one such example. We will show that some well-performing attention blocks used in the state of the art are subject to inference costs that are far too high to justify their use for mobile robotic applications. As a consequence, we propose an attention block that only slightly affects the inference speed while keeping up with much deeper networks or more complex attention blocks in terms of re-identification accuracy. We perform extensive neural architecture search to derive rules at which locations this attention block should be integrated into the architecture in order to achieve the best trade-off between speed and accuracy. Finally, we confirm that the best performing configuration on a re-identification benchmark also performs well on an indoor robotic dataset.

cs.RO↗

arXiv2022

Order Parameter Engineering for Random Systems

The chemical short-range order (CSRO) in the crystalline materials influences the properties and its effect is particularly important in the context of the multicomponent materials. We propose a scheme for CSRO parameter or $\mathrm{Δ-}$parameter in terms of number of like and unlike bonds in the multicomponent systems. The OPERA or $\bf{O}$rder $\bf{P}$arameter $\bf{E}$ngineering for $\bf{RA}$ndom Systems scheme for both semi-canonical as well as canonical ensemble is proposed. The proposed framework provides a high-throughput scheme for exploration of the CSRO in terms of the $\mathrm{Δ-}$parameter. We demonstrate the applicability of the $\mathrm{Δ-}$parameter for describing the CSRO in multicomponent alloys and oxides (FCC-CoCrNi, BCC-MoNbTaW and (CoCuMgNiZn)O). We show that the $\mathrm{Δ-}$parameter not only inherits the merit of Warren-Cowley order parameter, but it also addresses its limitations. The $\mathrm{Δ-}$parameter being a scalar quantity can represent the chemical order and OPERA framework can deal with both semi-canonical and canonical ensemble

cond-mat.mtrl-sci↗

arXiv2022

Few-Shot Object Detection: A Comprehensive Survey

Humans are able to learn to recognize new objects even from a few examples. In contrast, training deep-learning-based object detectors requires huge amounts of annotated data. To avoid the need to acquire and annotate these huge amounts of data, few-shot object detection aims to learn from few object instances of new categories in the target domain. In this survey, we provide an overview of the state of the art in few-shot object detection. We categorize approaches according to their training scheme and architectural layout. For each type of approaches, we describe the general realization as well as concepts to improve the performance on novel categories. Whenever appropriate, we give short takeaways regarding these concepts in order to highlight the best ideas. Eventually, we introduce commonly used datasets and their evaluation protocols and analyze reported benchmark results. As a result, we emphasize common challenges in evaluation and identify the most promising current trends in this emerging field of few-shot object detection.

cs.CV↗

arXiv2022

Machine Learning for First Principles Calculations of Material Properties for Ferromagnetic Materials

The investigation of finite temperature properties using Monte-Carlo (MC) methods requires a large number of evaluations of the system's Hamiltonian to sample the phase space needed to obtain physical observables as function of temperature. DFT calculations can provide accurate evaluations of the energies, but they are too computationally expensive for routine simulations. To circumvent this problem, machine-learning (ML) based surrogate models have been developed and implemented on high-performance computing (HPC) architectures. In this paper, we describe two ML methods (linear mixing model and HydraGNN) as surrogates for first principles density functional theory (DFT) calculations with classical MC simulations. These two surrogate models are used to learn the dependence of target physical properties from complex compositions and interactions of their constituents. We present the predictive performance of these two surrogate models with respect to their complexity while avoiding the danger of overfitting the model. An important aspect of our approach is the periodic retraining with newly generated first principles data based on the progressive exploration of the system's phase space by the MC simulation. The numerical results show that HydraGNN model attains superior predictive performance compared to the linear mixing model for magnetic alloy materials.

cond-mat.mtrl-sci↗

arXiv2022

Ab initio Approaches to High Entropy Alloys: A Comparison of CPA, SQS, and Supercell Methods

We present a comparative study of different modeling approaches to the electronic properties of the $\textrm{Hf}_{0.05}\textrm{Nb}_{0.05}\textrm{Ta}_{0.8}\textrm{Ti}_{0.05}\textrm{Zr}_{0.05}$ high entropy alloy. Common to our modeling is the methodology to compute the one-particle Green's function in the framework of density functional theory. We demonstrate that the special quasi-random structures modeling and the supercell, i.e. the locally self-consistent multiple-scatering methods provide very similar results for the ground state properties such as the spectral function (density of states) and the equilibrium lattice parameter. To reconcile the multiple-scattering single-site coherent potential approximation with the real space supercell methods, we included the effect of screening of the net charges of the alloy components. Based on the analysis of the total energy and spectral functions computed within the density functional theory, we found no signature for the long-range or local magnetic moments formation in the $\textrm{Hf}_{0.05}\textrm{Nb}_{0.05}\textrm{Ta}_{0.8}\textrm{Ti}_{0.05}\textrm{Zr}_{0.05}$ high entropy alloy, instead we find possible superconductivity below $\sim 9$K.

cond-mat.mtrl-sci↗

arXiv2021

A scalable constructive algorithm for the optimization of neural network architectures

We propose a new scalable method to optimize the architecture of an artificial neural network. The proposed algorithm, called Greedy Search for Neural Network Architecture, aims to determine a neural network with minimal number of layers that is at least as performant as neural networks of the same structure identified by other hyperparameter search algorithms in terms of accuracy and computational cost. Numerical results performed on benchmark datasets show that, for these datasets, our method outperforms state-of-the-art hyperparameter optimization algorithms in terms of attainable predictive performance by the selected neural network architecture, and time-to-solution for the hyperparameter optimization to complete.

cs.LG↗

arXiv2020

Integrating Deep Learning in Domain Sciences at Exascale

This paper presents some of the current challenges in designing deep learning artificial intelligence (AI) and integrating it with traditional high-performance computing (HPC) simulations. We evaluate existing packages for their ability to run deep learning models and applications on large-scale HPC systems efficiently, identify challenges, and propose new asynchronous parallelization and optimization techniques for current large-scale heterogeneous systems and upcoming exascale systems. These developments, along with existing HPC AI software capabilities, have been integrated into MagmaDNN, an open-source HPC deep learning framework. Many deep learning frameworks are targeted at data scientists and fall short in providing quality integration into existing HPC workflows. This paper discusses the necessities of an HPC deep learning framework and how those needs can be provided (e.g., as in MagmaDNN) through a deep integration with existing HPC libraries, such as MAGMA and its modular memory management, MPI, CuBLAS, CuDNN, MKL, and HIP. Advancements are also illustrated through the use of algorithmic enhancements in reduced- and mixed-precision, as well as asynchronous optimization methods. Finally, we present illustrations and potential solutions for enhancing traditional compute- and data-intensive applications at ORNL and UTK with AI. The approaches and future challenges are illustrated in materials science, imaging, and climate applications.

cs.LG↗

arXiv2020

Machine Learning Assisted Insight to Spin Ice Dy$_2$Ti$_2$O$_7$

Complex behavior poses challenges in extracting models from experiment. An example is spin liquid formation in frustrated magnets like Dy$_2$Ti$_2$O$_7$. Understanding has been hindered by issues including disorder, glass formation, and interpretation of scattering data. Here, we use a novel automated capability to extract model Hamiltonians from data, and to identify different magnetic regimes. This involves training an autoencoder to learn a compressed representation of three-dimensional diffuse scattering, over a wide range of spin Hamiltonians. The autoencoder finds optimal matches according to scattering and heat capacity data and provides confidence intervals. Validation tests indicate that our optimal Hamiltonian accurately predicts temperature and field dependence of both magnetic structure and magnetization, as well as glass formation and irreversibility in Dy$_2$Ti$_2$O$_7$. The autoencoder can also categorize different magnetic behaviors and eliminate background noise and artifacts in raw data. Our methodology is readily applicable to other materials and types of scattering problems.

cond-mat.str-el↗

arXiv2020

Fast and stable deep-learning predictions of material properties for solid solution alloys

We present a novel deep learning (DL) approach to produce highly accurate predictions of macroscopic physical properties of solid solution binary alloys and magnetic systems. The major idea is to make use of the correlations between different physical properties in alloy systems to improve the prediction accuracy of neural network (NN) models. We use multitasking NN models to simultaneously predict the total energy, charge density and magnetic moment. These physical properties mutually serve as constraints during the training of the multitasking NN, resulting in more reliable DL models because multiple physics properties are correctly learned by a single model. Two binary alloys, copper-gold (CuAu) and iron-platinum (FePt), were studied. Our results show that once the multitasking NN's are trained, they can estimate the material properties for a specific configuration hundreds of times faster than first-principles density functional theory calculations while retaining comparable accuracy. We used a simple measure based on the root-mean-squared errors (RMSE) to quantify the quality of the NN models, and found that the inclusion of charge density and magnetic moment as physical constraints leads to more stable models that exhibit improved accuracy and reduced uncertainty for the energy predictions.

physics.comp-ph↗

arXiv2020

Monte Carlo simulation of order-disorder transition in refractory high entropy alloys: a data-driven approach

High entropy alloys (HEAs) are a series of novel materials that demonstrate many exceptional mechanical properties. To understand the origin of these attractive properties, it is important to investigate the thermodynamics and elucidate the evolution of various chemical phases. In this work, we introduce a data-driven approach to construct the effective Hamiltonian and study the thermodynamics of HEAs through canonical Monte Carlo simulation. The main characteristic of our method is to use pairwise interactions between atoms as features and systematically improve the representativeness of the dataset using samples from Monte Carlo simulation. We find this method produces highly robust and accurate effective Hamiltonians that give less than 0.1 mRy test error for all the three refractory HEAs: MoNbTaW, MoNbTaVW, and MoNbTaTiW. Using replica exchange to speed up the MC simulation, we calculated the specific heats and short-range order parameters in a wide range of temperatures. For all the studied materials, we find there are two major order-disorder transitions occurring respectively at $T_1$ and $T_2$, where $T_1$ is near room temperature but $T_2$ is much higher. We further demonstrate that the transition at $T_1$ is caused by W and Nb while the one at $T_2$ is caused by the other elements. By comparing with experiments, {\color{black} the results provide insight into the role of chemical ordering in the strength and ductility of HEAs.

cond-mat.mtrl-sci↗

arXiv2020

Tuning Fermi Levels in Intrinsic Antiferromagnetic Topological Insulators MnBi2Te4 and MnBi4Te7 by Defect Engineering and Chemical Doping

MnBi2Te4 and MnBi4Te7 are intrinsic antiferromagnetic topological insulators, offering a promising materials platform for realizing exotic topological quantum states. However, high densities of intrinsic defects in these materials not only cause bulk metallic conductivity, preventing the measurement of quantum transport in surface states, but may also affect magnetism and topological properties. In this paper, we show by density functional theory calculations that the strain induced by the internal heterostructure promotes the formation of large-size-mismatched antisite defect BiMn in MnBi2Te4; such strain is further enhanced in MnBi4Te7, giving rise to even higher BiMn density. The abundance of intrinsic BiMn donors results in degenerate n-type conductivity under the Te-poor growth condition. Our calculations suggest that growths in a Te-rich condition can lower the Fermi level, which is supported by our transport measurements. We further show that the internal strain can also enable efficient doping by large-size-mismatched substitutional NaMn acceptors, which can compensate BiMn donors and lower the Fermi level. Na doping may pin the Fermi level inside the bulk band gap even at the Te-poor limit in MnBi2Te4. Furthermore, facile defect formation in MnSb2Te4 and its implication in Sb doping in MnBi2Te4 as well as the defect segregation in MnBi4Te7 are discussed. The defect engineering and doping strategies proposed in this paper will stimulate further studies for improving synthesis and for manipulating magnetic and topological properties in MnBi2Te4, MnBi4Te7, and related compounds.

cond-mat.mtrl-sci↗

arXiv2020

Predicting the phase stability of multi-component high entropy compounds

A generic method to estimate the relative feasibility of formation of high entropy compounds in a single phase, directly from first principles, is developed. As a first step, the relative formation abilities of 56 multi-component, AO, oxides were evaluated. These were constructed from 5 cation combinations chosen from A={Ca, Co, Cu, Fe, Mg, Mn, Ni, Zn}. Candidates for multi-component oxides are predicted from descriptors related to the enthalpy and configurational entropy obtained from the mixing enthalpies of two component oxides. The utility of this approach is evaluated by comparing the predicted combinations with the experimentally realized entropy stabilized oxide, (MgCoCuNiZn)O. In the second step, Monte Carlo simulations are utilized to investigate the phase composition and local ionic segregation as a function of temperature. This approach allows for the evaluation of potential secondary phases, thereby making realistic predictions of novel multi-component compounds that can be synthesized.

cond-mat.mtrl-sci↗

arXiv2019

Machine Learning the Effective Hamiltonian in High Entropy Alloys

The development of machine learning sheds new light on the problem of statistical thermodynamics in multicomponent alloys. However, a data-driven approach to construct the effective Hamiltonian requires sufficiently large data sets, which is expensive to calculate with conventional density functional theory (DFT). To solve this problem, we propose to use the atomic local energy as the target variable, and harness the power of the linear-scaling DFT to accelerate the data generating process. Using the large amounts of DFT data sets, various complex models are devised and applied to learn the effective Hamiltonians of a range of refractory high entropy alloys (HEAs). The testing $R^2$ scores of the effective pair interaction model are higher than 0.99, demonstrating that the pair interactions within the 6-th coordination shell provide an excellent description of the atomic local energies for all the four HEAs. This model is further improved by including nonlinear and multi-site interactions. In particular, the deep neural networks (DNNs) built directly in the local configuration space (therefore no hand-crafted features) are employed to model the effective Hamiltonian. The results demonstrate that neural networks are promising for the modeling of effective Hamiltonian due to its excellent representation power.

cond-mat.mtrl-sci↗

arXiv2019

Electron spin mediated distortion in metallic systems

The deviation of positions of atoms from their ideal lattice sites in crystalline solid state systems causes distortion and can lead to variation in structural [1] and functional properties [2]. Distortion in molecular systems has been traditionally understood in the term of Jahn-Teller distortion [3], while for the one-dimensional chain of metals, the Peierls distortion mechanism has been proposed [4]. In a real three dimensional metallic systems, a fundamental description of the distortion is missing, which we need to design alloys with attractive structural properties. The present investigation presents the evolution of distortion in metallic systems in terms of magnetovolume effects arising due to magnetic ground-state of the system. Particularly the significant distortion in the case of Cr due to presence of other transition metals is seen. Cr alloyed with transition metals as a candidate system to study the correlation between charge disproportion, spin-flip, magnetovolume effects and Fermi surface nesting. This study provides a novel explanation of strengthening effect of Cr observed in alloys due to its unique magnetic properties.

cond-mat.mtrl-sci↗

arXiv2019

Robust data-driven approach for predicting the configurational energy of high entropy alloys

High entropy alloys (HEAs) have been increasingly attractive as promising next-generation materials due to their various excellent properties. It's necessary to essentially characterize the degree of chemical ordering and identify order-disorder transitions through efficient simulation and modeling of thermodynamics. In this study, a robust data-driven framework based on Bayesian approaches is proposed and demonstrated on the accurate and efficient prediction of configurational energy of high entropy alloys. The proposed effective pair interaction (EPI) model with ensemble sampling is used to map the configuration and its corresponding energy. Given limited data calculated by first-principles calculations, Bayesian regularized regression not only offers an accurate and stable prediction but also effectively quantifies the uncertainties associated with EPI parameters. Compared with the arbitrary determination of model complexity, we further conduct a physical feature selection to identify the truncation of coordination shells in EPI model using Bayesian information criterion. The results achieve efficient and robust performance in predicting the configurational energy, particularly given small data. The developed methodology is applied to study a series of refractory HEAs, i.e. NbMoTaW, NbMoTaWV and NbMoTaWTi where it is demonstrated how dataset size affects the confidence we can place in statistical estimates of configurational energy when data are sparse.

cond-mat.mtrl-sci↗

arXiv2019

Chemical complexity in high entropy alloys: A pair-interaction perspective

The recently proposed pair-interaction model is applied to study a series of refractory high entropy alloys. The results demonstrate the simplicity, robustness, and high accuracy of this model in predicting the configuration energies of NbMoTaW, NbMoTaWV and NbMoTaWTi. The element-element pair interaction parameters obtained from the linear regression of first-principle data provide a new perspective to understand the strengthening mechanism in HEAs, as revealed by comparing the effects of adding vanadium and titanium. Using the pair-interaction model, an expression for the intrinsic energy fluctuation is derived, which provides guidance on both theoretical modeling and first principles calculation.

cond-mat.mtrl-sci↗

arXiv2019

Machine learning modeling of high entropy alloy: the role of short-range order

The development of machine learning sheds new light on the traditionally complicated problem of thermodynamics in multicomponent alloys. Successful application of such a method, however, strongly depends on the quality of the data and model. Here we propose a scheme to improve the representativeness of the data by utilizing the short-range order (SRO) parameters to survey the configuration space. Using the improved data, a pair interaction model is trained for the NbMoTaW high entropy alloy using linear regression. Benefiting from the physics incorporated into the model, the learned effective Hamiltonian demonstrates excellent predictability over the whole configuration space. By including pair interactions within the 6th nearest-neighbor shell, this model achieves an $R^2$ testing score of 0.997 and root mean square error of 0.43 meV. We further perform a detailed analysis on the effects of training data, testing data, and model parameters. The results reveal the vital importance of representative data and physical model. On the other hand, we also examined the performance neural networks, which is found to demonstrate a strong tendency to overfit the data.

cond-mat.mtrl-sci↗

arXiv2019

Locally self-consistent embedding approach for disordered electronic systems

We present a new embedding scheme for the locally self-consistent method to study disordered electron systems. We test this method in a tight-binding basis and apply it to the single band Anderson model. The local interaction zone is used to efficiently compute the local Green's function of a supercell embeded into a local typical medium. We find a quick convergence as the size of the local interaction zone which reduces the computational costs as expected. This method captures the Anderson localization transition and accurately predicts the critical disorder strength. The present work opens the path towards the development of a typical medium embedding scheme for the $O(N)$ multiple scattering methods.

cond-mat.dis-nn↗