arXiv subjects

Ulrich Rüde

Publications and source records attributed to Ulrich Rüde.

At least 55 records · Page 3Linked to original sources

arXiv2018

Dynamic Load Balancing Techniques for Particulate Flow Simulations

Parallel multiphysics simulations often suffer from load imbalances originating from the applied coupling of algorithms with spatially and temporally varying workloads. It is thus desirable to minimize these imbalances to reduce the time to solution and to better utilize the available hardware resources. Taking particulate flows as an illustrating example application, we present and evaluate load balancing techniques that tackle this challenging task. This involves a load estimation step in which the currently generated workload is predicted. We describe in detail how such a workload estimator can be developed. In a second step, load distribution strategies like space-filling curves or graph partitioning are applied to dynamically distribute the load among the available processes. To compare and analyze their performance, we employ these techniques to a benchmark scenario and observe a reduction of the load imbalances by almost a factor of four. This results in a decrease of the overall runtime by 14% for space-filling curves.

cs.DC↗

arXiv2018

An improved lattice Boltzmann D3Q19 method based on an alternative equilibrium discretization

Lattice Boltzmann simulations of three-dimensional, isothermal hydrodynamics often use either the D3Q19 or the D3Q27 velocity sets. While both models correctly approximate Navier-Stokes in the continuum limit, the D3Q19 model is computationally less expensive but has some known deficiencies regarding Galilean invariance, especially for high Reynolds number flows. In this work we present a novel methodology to construct lattice Boltzmann equilibria for hydrodynamics directly from the continuous Maxwellian equilibrium. While our new approach reproduces the well known LBM equilibrium for D2Q9 and D3Q27 lattice models, it yields a different equilibrium formulation for the D3Q19 stencil. This newly proposed formulation is shown to be more accurate than the widely used second order equilibrium, while having the same computation costs. We present a steady state Chapman-Enskog analysis of the standard and the improved D3Q19 model and conduct numerical experiments that demonstrate the superior accuracy of our newly developed D3Q19 equilibrium.

physics.comp-ph↗

arXiv2018

A local parallel communication algorithm for polydisperse rigid body dynamics

The simulation of large ensembles of particles is usually parallelized by partitioning the domain spatially and using message passing to communicate between the processes handling neighboring subdomains. The particles are represented as individual geometric objects and are associated to the subdomains. Handling collisions and migrating particles between subdomains, as required for proper parallel execution, requires a complex communication protocol. Typically, the parallelization is restricted to handling only particles that are smaller than a subdomain. In many applications, however, particle sizes may vary drastically with some of them being larger than a subdomain. In this article we propose a new communication and synchronization algorithm that can handle the parallelization without size restrictions on the particles. Despite the additional complexity and extended functionality, the new algorithm introduces only minimal overhead. We demonstrate the scalability of the previous and the new communication algorithms up to almost two million parallel processes and for handling ten billion (1e10) geometrically resolved particles on a state-of-the-art petascale supercomputer. Different scenarios are presented to analyze the performance of the new algorithm and to demonstrate its capability to simulate polydisperse scenarios, where large individual particles can extend across several subdomains.

cs.DC↗

arXiv2018

A Scalable and Modular Software Architecture for Finite Elements on Hierarchical Hybrid Grids

In this article, a new generic higher-order finite-element framework for massively parallel simulations is presented. The modular software architecture is carefully designed to exploit the resources of modern and future supercomputers. Combining an unstructured topology with structured grid refinement facilitates high geometric adaptability and matrix-free multigrid implementations with excellent performance. Different abstraction levels and fully distributed data structures additionally ensure high flexibility, extensibility, and scalability. The software concepts support sophisticated load balancing and flexibly combining finite element spaces. Example scenarios with coupled systems of PDEs show the applicability of the concepts to performing geophysical simulations.

cs.MS↗

arXiv2018

A Scalable Multiphysics Algorithm for Massively Parallel Direct Numerical Simulations of Electrophoresis

In this article we introduce a novel coupled algorithm for massively parallel direct numerical simulations of electrophoresis in microfluidic flows. This multiphysics algorithm employs an Eulerian description of fluid and ions, combined with a Lagrangian representation of moving charged particles. The fixed grid facilitates efficient solvers and the employed lattice Boltzmann method can efficiently handle complex geometries. Validation experiments with more than $70\,000$ time steps are presented, together with scaling experiments with over ${4\cdot10^{6}}$ particles and ${1.96\cdot10^{11}}$ grid cells for both hydrodynamics and electric potential. We achieve excellent performance and scaling on up to $65\,536$ cores of a current supercomputer.

cs.CE↗

arXiv2018

Adaptive control in rollforward recovery for extreme scale multigrid

With the increasing number of compute components, failures in future exa-scale computer systems are expected to become more frequent. This motivates the study of novel resilience techniques. Here, we extend a recently proposed algorithm-based recovery method for multigrid iterations by introducing an adaptive control. After a fault, the healthy part of the system continues the iterative solution process, while the solution in the faulty domain is re-constructed by an asynchronous on-line recovery. The computations in both the faulty and healthy subdomains must be coordinated in a sensitive way, in particular, both under and over-solving must be avoided. Both of these waste computational resources and will therefore increase the overall time-to-solution. To control the local recovery and guarantee an optimal re-coupling, we introduce a stopping criterion based on a mathematical error estimator. It involves hierarchical weighted sums of residuals within the context of uniformly refined meshes and is well-suited in the context of parallel high-performance computing. The re-coupling process is steered by local contributions of the error estimator. We propose and compare two criteria which differ in their weights. Failure scenarios when solving up to $6.9\cdot10^{11}$ unknowns on more than 245\,766 parallel processes will be reported on a state-of-the-art peta-scale supercomputer demonstrating the robustness of the method.

cs.MS↗

arXiv2018

Extreme-Scale Block-Structured Adaptive Mesh Refinement

In this article, we present a novel approach for block-structured adaptive mesh refinement (AMR) that is suitable for extreme-scale parallelism. All data structures are designed such that the size of the meta data in each distributed processor memory remains bounded independent of the processor number. In all stages of the AMR process, we use only distributed algorithms. No central resources such as a master process or replicated data are employed, so that an unlimited scalability can be achieved. For the dynamic load balancing in particular, we propose to exploit the hierarchical nature of the block-structured domain partitioning by creating a lightweight, temporary copy of the core data structure. This copy acts as a local and fully distributed proxy data structure. It does not contain simulation data, but only provides topological information about the domain partitioning into blocks. Ultimately, this approach enables an inexpensive, local, diffusion-based dynamic load balancing scheme. We demonstrate the excellent performance and the full scalability of our new AMR implementation for two architecturally different petascale supercomputers. Benchmarks on an IBM Blue Gene/Q system with a mesh containing 3.7 trillion unknowns distributed to 458,752 processes confirm the applicability for future extreme-scale parallel machines. The algorithms proposed in this article operate on blocks that result from the domain partitioning. This concept and its realization support the storage of arbitrary data. In consequence, the software framework can be used for different simulation methods, including mesh based and meshless methods. In this article, we demonstrate fluid simulations based on the lattice Boltzmann method.

cs.DC↗

arXiv2018

A Scalable and Extensible Checkpointing Scheme for Massively Parallel Simulations

Realistic simulations in engineering or in the materials sciences can consume enormous computing resources and thus require the use of massively parallel supercomputers. The probability of a failure increases both with the runtime and with the number of system components. For future exascale systems it is therefore considered critical that strategies are developed to make software resilient against failures. In this article, we present a scalable, distributed, diskless, and resilient checkpointing scheme that can create and recover snapshots of a partitioned simulation domain. We demonstrate the efficiency and scalability of the checkpoint strategy for simulations with up to $40$ billion computational cells executing on more than $400$ billion floating point values. A checkpoint creation is shown to require only a few seconds and the new checkpointing scheme scales almost perfectly up to more than $260\,000$ ($2^{18}$) processes. To recover from a diskless checkpoint during runtime, we realize the recovery algorithms using ULFM MPI. The checkpointing mechanism is fully integrated in a state-of-the-art high-performance multi-physics simulation framework. We demonstrate the efficiency and robustness of the method with a realistic phase-field simulation originating in the material sciences and with a lattice Boltzmann method implementation.

cs.DC↗

arXiv2018

Research and Education in Computational Science and Engineering

Over the past two decades the field of computational science and engineering (CSE) has penetrated both basic and applied research in academia, industry, and laboratories to advance discovery, optimize systems, support decision-makers, and educate the scientific and engineering workforce. Informed by centuries of theory and experiment, CSE performs computational experiments to answer questions that neither theory nor experiment alone is equipped to answer. CSE provides scientists and engineers of all persuasions with algorithmic inventions and software systems that transcend disciplines and scales. Carried on a wave of digital technology, CSE brings the power of parallelism to bear on troves of data. Mathematics-based advanced computing has become a prevalent means of discovery and innovation in essentially all areas of science, engineering, technology, and society; and the CSE community is at the core of this transformation. However, a combination of disruptive developments---including the architectural complexity of extreme-scale computing, the data revolution that engulfs the planet, and the specialization required to follow the applications to new frontiers---is redefining the scope and reach of the CSE endeavor. This report describes the rapid expansion of CSE and the challenges to sustaining its bold advances. The report also presents strategies and directions for CSE research and education for the next decade.

cs.CE↗

arXiv2017

Direct simulation of liquid-gas-solid flow with a free surface lattice Boltzmann method

Direct numerical simulation of liquid-gas-solid flows is uncommon due to the considerable computational cost. As the grid spacing is determined by the smallest involved length scale, large grid sizes become necessary -- in particular if the bubble-particle aspect ratio is on the order of 10 or larger. Hence, it arises the question of both feasibility and reasonability. In this paper, we present a fully parallel, scalable method for direct numerical simulation of bubble-particle interaction at a size ratio of 1-2 orders of magnitude that makes simulations feasible on currently available super-computing resources. With the presented approach, simulations of bubbles in suspension columns consisting of more than $100\,000$ fully resolved particles become possible. Furthermore, we demonstrate the significance of particle-resolved simulations by comparison to previous unresolved solutions. The results indicate that fully-resolved direct numerical simulation is indeed necessary to predict the flow structure of bubble-particle interaction problems correctly.

physics.comp-ph↗

arXiv2017

A Coupled Lattice Boltzmann Method and Discrete Element Method for Discrete Particle Simulations of Particulate Flows

Discrete particle simulations are widely used to study large-scale particulate flows in complex geometries where particle-particle and particle-fluid interactions require an adequate representation but the computational cost has to be kept low. In this work, we present a novel coupling approach for such simulations. A lattice Boltzmann formulation of the generalized Navier-Stokes equations is used to describe the fluid motion. This promises efficient simulations suitable for high performance computing and, since volume displacement effects by the solid phase are considered, our approach is also applicable to non-dilute particulate systems. The discrete element method is combined with an explicit evaluation of interparticle lubrication forces to simulate the motion of individual submerged particles. Drag, pressure and added mass forces determine the momentum transfer by fluid-particle interactions. A stable coupling algorithm is presented and discussed in detail. We demonstrate the validity of our approach for dilute as well as dense systems by predicting the settling velocity of spheres over a broad range of solid volume fractions in good agreement with semi-empirical correlations. Additionally, the accuracy of particle-wall interactions in a viscous fluid is thoroughly tested and established. Our approach can thus be readily used for various particulate systems and can be extended straightforward to e.g. non-spherical particles.

cs.CE↗

arXiv2017

The Maximum Dissipation Principle in Rigid-Body Dynamics with Purely Inelastic Impacts

Formulating a consistent theory for rigid-body dynamics with impacts is an intricate problem. Twenty years ago Stewart published the first consistent theory with purely inelastic impacts and an impulsive friction model analogous to Coulomb friction. In this paper we demonstrate that the consistent impact model can exhibit multiple solutions with a varying degree of dissipation even in the single-contact case. Replacing the impulsive friction model based on Coulomb friction by a model based on the maximum dissipation principle resolves the non-uniqueness in the single-contact impact problem. The paper constructs the alternative impact model and presents integral equations describing rigid-body dynamics with a non-impulsive and non-compliant contact model and an associated purely inelastic impact model maximizing dissipation. An analytic solution is derived for the single-contact impact problem. The models are then embedded into a time-stepping scheme. The macroscopic behaviour is compared to Coulomb friction in a large-scale granular flow problem.

cs.CE↗

arXiv2017

A comparative study of fluid-particle coupling methods for fully resolved lattice Boltzmann simulations

The direct numerical simulation of particulate systems offers a unique approach to study the dynamics of fluid-solid suspensions by fully resolving the submerged particles and without introducing empirical models. For the lattice Boltzmann method, different variants exist to incorporate the fluid-particle interaction into the simulation. This paper provides a detailed and systematic comparison of two different methods, namely the momentum exchange method and the partially saturated cells method by Noble and Torczynski. Three subvariants of each method are used in the benchmark scenario of a single heavy sphere settling in ambient fluid to study their characteristics and accuracy for particle Reynolds numbers from 185 up to 365. The sphere must be resolved with at least 24 computational cells per diameter to achieve velocity errors below 5%. The momentum exchange method is found to be more accurate in predicting the streamwise velocity component whereas the partially saturated cells method is more accurate in the spanwise components. The study reveals that the resolution should be chosen with respect to the coupling dynamics, and not only based on the flow properties, to avoid large errors in the fluid-particle interaction.

cs.CE↗

arXiv2016

On the analysis of block smoothers for saddle point problems

In this article, we discuss several classes of Uzawa smoothers for the application in multigrid methods in the context of saddle point problems. Beside commonly used variants, such as the inexact and block factorization version, we also introduce a new symmetric method, belonging to the class of Uzawa smoothers. For these variants we unify the analysis of the smoothing properties, which is an important part in the multigrid convergence theory. These methods are applied to the Stokes problem for which all smoothers are implemented as pointwise relaxation methods. Several numerical examples illustrate the theoretical results.

math.NA↗

arXiv2016

Microswimming with inertia

Microswimmers, especially in theoretical treatments, are generally taken to be completely inertia-free, since inertial effects on their motion are typically small and assuming their absence simplifies the problem considerably. Yet in nature there is no discrete break between swimmers for which inertia is negligibly small and for which it is detectable. Here we study a microswimming model for which the effect of inertia is calculated explicitly in the regime of transition between the Stokesian and the non-Stokesian flow limits, which we term the intermediate regime. The model in the inertialess limit is the bead-spring swimmer. We first show that in the intermediate regime a mechanical microswimmer exhibits damped inertial coasting like an underdamped harmonic oscillator. We then calculate analytically the swimmer's velocity by including a mass-acceleration term in the equations of motion which are otherwise based on the Stokes flow. We show that this hybrid treatment combining aspects of underdamped and overdamped dynamics provides an accurate description of the motion in the intermediate regime, as verified here by comparison to simulations using the lattice Boltzmann method, and is a significant improvement over the results from the inertialess theory when either the mass of the swimmer or the forces driving its motion is/are large enough.

cond-mat.soft↗

arXiv2016

A two-scale approach for efficient on-the-fly operator assembly in massively parallel high performance multigrid codes

Matrix-free finite element implementations of massively parallel geometric multigrid save memory and are often significantly faster than implementations using classical sparse matrix techniques. They are especially well suited for hierarchical hybrid grids on polyhedral domains. In the case of constant coefficients all fine grid node stencils in the interior of a coarse macro element are equal. However, for non-polyhedral domains the situation changes. Then even for the Laplace operator, the non-linear element mapping leads to fine grid stencils that can vary from grid point to grid point. This observation motivates a new two-scale approach that exploits a piecewise polynomial approximation of the fine grid operator with respect to the coarse mesh size. The low-cost evaluation of these surrogate polynomials results in an efficient stencil assembly on-the-fly for non-polyhedral domains that can be significantly more efficient than matrix-free techniques that are based on an element-wise assembly. The performance analysis and additional hardware-aware code optimizations are based on the Execution-Cache-Memory model. Several aspects such as two-scale a priori error bounds and double discretization techniques are presented. Weak and strong scaling results illustrate the benefits of the new technique when used within large scale PDE solvers.

math.NA↗

arXiv2016

Highly sparse surface couplings for subdomain-wise isoviscous Stokes finite element discretizations

The Stokes system with constant viscosity can be cast into different formulations by exploiting the incompressibility constraint. For instance the strain in the weak formulation can be replaced by the gradient to decouple the velocity components in the different coordinate directions. Thus the discretization of the simplified problem leads to fewer nonzero entries in the stiffness matrix. This is of particular interest in large scale simulations where a reduced memory bandwidth requirement can help to significantly accelerate the computations. In the case of a piecewise constant viscosity, as it typically arises in multi-phase flows, or when the boundary conditions involve traction, the situation is more complex, and one has to treat the cross derivatives in the original Stokes system with care. A naive application of the standard vectorial Laplacian results in a physically incorrect solution, while formulations based on the strain increase the computational effort everywhere, even when the inconsistencies arise only from an incorrect treatment in a small fraction of the computational domain. Here we propose a new approach that is consistent with the strain-based formulation and preserves the decoupling advantages of the gradient-based formulation in isoviscous subdomains. The modification is equivalent to locally changing the discretization stencils, hence the more expensive discretization is restricted to a lower dimensional interface, making the additional computational cost asymptotically negligible. We demonstrate the consistency and convergence properties of the method and show that in a massively parallel setup, the multigrid solution of the resulting discrete systems is faster than for the classical strain-based formulation. Moreover, we give an application example which is inspired by geophysical research.

math.NA↗

arXiv2016

Free Surface Lattice Boltzmann with Enhanced Bubble Model

This paper presents an enhancement to the free surface lattice Boltzmann method (FSLBM) for the simulation of bubbly flows including rupture and breakup of bubbles. The FSLBM uses a volume of fluid approach to reduce the problem of a liquid-gas two-phase flow to a single-phase free surface simulation. In bubbly flows compression effects leading to an increase or decrease of pressure in the suspended bubbles cannot be neglected. Therefore, the free surface simulation is augmented by a bubble model that supplies the missing information by tracking the topological changes of the free surface in the flow. The new model presented here is capable of handling the effects of bubble breakup and coalesce without causing a significant computational overhead. Thus, the enhanced bubble model extends the applicability of the FSLBM to a new range of practically relevant problems, like bubble formation and development in chemical reactors or foaming processes.

physics.comp-ph↗