arXiv subjects

David Sondak

Publications and source records attributed to David Sondak.

17 recordsLinked to original sources

arXiv2025

Recent Advances of NeuroDiffEq -- An Open-Source Library for Physics-Informed Neural Networks

Solving differential equations is a critical challenge across a host of domains. While many software packages efficiently solve these equations using classical numerical approaches, there has been less effort in developing a library for researchers interested in solving such systems using neural networks. With PyTorch as its backend, NeuroDiffEq is a software library that exploits neural networks to solve differential equations. In this paper, we highlight the latest features of the NeuroDiffEq library since its debut. We show that NeuroDiffEq can solve complex boundary value problems in arbitrary dimensions, tackle boundary conditions at infinity, and maintain flexibility for dynamic injection at runtime.

cs.LG↗

arXiv2024

Extending a Physics-Informed Machine Learning Network for Superresolution Studies of Rayleigh-Bénard Convection

Advancing our understanding of astrophysical turbulence is bottlenecked by the limited resolution of numerical simulations that may not fully sample scales in the inertial range. Machine learning (ML) techniques have demonstrated promise in up-scaling resolution in both image analysis and numerical simulations (i.e., superresolution). Here we employ and further develop a physics-constrained convolutional neural network (CNN) ML model called "MeshFreeFlowNet'' (MFFN) for superresolution studies of turbulent systems. The model is trained both on the simulation images as well as the evaluated PDEs, making it sensitive to the underlying physics of a particular fluid system. We develop a framework for 2D turbulent Rayleigh-Bénard convection (RBC) generated with the \textsc{Dedalus} code by modifying the MFFN architecture to include the full set of simulation PDEs and the boundary conditions. Our training set includes fully developed turbulence sampling Rayleigh numbers ($Ra$) of $Ra=10^6-10^{10}$. We evaluate the success of the learned simulations by comparing the power spectra of the direct \textsc{Dedalus} simulation to the predicted model output, and compare both ground truth and predicted power spectral inertial range scalings to theoretical predictions. We find that the updated network performs well at all $Ra$ studied here in recovering large-scale information, including the inertial range slopes. The superresolution prediction is overly dissipative at smaller scales than that of the inertial range in all cases, but the smaller-scales are better recovered in more turbulent, than laminar, regimes. This is likely because more turbulent systems have a rich variety of structures at many length scales compared to laminar flows.

physics.flu-dyn↗

arXiv2022

DEQGAN: Learning the Loss Function for PINNs with Generative Adversarial Networks

Solutions to differential equations are of significant scientific and engineering relevance. Physics-Informed Neural Networks (PINNs) have emerged as a promising method for solving differential equations, but they lack a theoretical justification for the use of any particular loss function. This work presents Differential Equation GAN (DEQGAN), a novel method for solving differential equations using generative adversarial networks to "learn the loss function" for optimizing the neural network. Presenting results on a suite of twelve ordinary and partial differential equations, including the nonlinear Burgers', Allen-Cahn, Hamilton, and modified Einstein's gravity equations, we show that DEQGAN can obtain multiple orders of magnitude lower mean squared errors than PINNs that use $L_2$, $L_1$, and Huber loss functions. We also show that DEQGAN achieves solution accuracies that are competitive with popular numerical methods. Finally, we present two methods to improve the robustness of DEQGAN to different hyperparameter settings.

cs.LG↗

arXiv2022

Hamiltonian neural networks for solving equations of motion

There has been a wave of interest in applying machine learning to study dynamical systems. We present a Hamiltonian neural network that solves the differential equations that govern dynamical systems. This is an equation-driven machine learning method where the optimization process of the network depends solely on the predicted functions without using any ground truth data. The model learns solutions that satisfy, up to an arbitrarily small error, Hamilton's equations and, therefore, conserve the Hamiltonian invariants. The choice of an appropriate activation function drastically improves the predictability of the network. Moreover, an error analysis is derived and states that the numerical errors depend on the overall network performance. The Hamiltonian network is then employed to solve the equations for the nonlinear oscillator and the chaotic Henon-Heiles dynamical system. In both systems, a symplectic Euler integrator requires two orders more evaluation points than the Hamiltonian network in order to achieve the same order of the numerical error in the predicted phase space trajectories.

physics.comp-ph↗

arXiv2022

Multi-Task Learning based Convolutional Models with Curriculum Learning for the Anisotropic Reynolds Stress Tensor in Turbulent Duct Flow

The Reynolds-averaged Navier-Stokes (RANS) equations require accurate modeling of the anisotropic Reynolds stress tensor. Traditional closure models, while sophisticated, often only apply to restricted flow configurations. Researchers have started using machine learning approaches to tackle this problem by developing more general closure models informed by data. In this work we build upon recent convolutional neural network architectures used for turbulence modeling and propose a multi-task learning-based fully convolutional neural network that is able to accurately predict the normalized anisotropic Reynolds stress tensor for turbulent duct flows. Furthermore, we also explore the application of curriculum learning to data-driven turbulence modeling.

physics.flu-dyn↗

arXiv2021

Port-Hamiltonian Neural Networks for Learning Explicit Time-Dependent Dynamical Systems

Accurately learning the temporal behavior of dynamical systems requires models with well-chosen learning biases. Recent innovations embed the Hamiltonian and Lagrangian formalisms into neural networks and demonstrate a significant improvement over other approaches in predicting trajectories of physical systems. These methods generally tackle autonomous systems that depend implicitly on time or systems for which a control signal is known apriori. Despite this success, many real world dynamical systems are non-autonomous, driven by time-dependent forces and experience energy dissipation. In this study, we address the challenge of learning from such non-autonomous systems by embedding the port-Hamiltonian formalism into neural networks, a versatile framework that can capture energy dissipation and time-dependent control forces. We show that the proposed \emph{port-Hamiltonian neural network} can efficiently learn the dynamics of nonlinear physical systems of practical interest and accurately recover the underlying stationary Hamiltonian, time-dependent force, and dissipative coefficient. A promising outcome of our network is its ability to learn and predict chaotic systems such as the Duffing equation, for which the trajectories are typically hard to learn.

cs.LG↗

arXiv2021

Convolutional Neural Network Models and Interpretability for the Anisotropic Reynolds Stress Tensor in Turbulent One-dimensional Flows

The Reynolds-averaged Navier-Stokes (RANS) equations are widely used in turbulence applications. They require accurately modeling the anisotropic Reynolds stress tensor, for which traditional Reynolds stress closure models only yield reliable results in some flow configurations. In the last few years, there has been a surge of work aiming at using data-driven approaches to tackle this problem. The majority of previous work has focused on the development of fully-connected networks for modeling the anisotropic Reynolds stress tensor. In this paper, we expand upon recent work for turbulent channel flow and develop new convolutional neural network (CNN) models that are able to accurately predict the normalized anisotropic Reynolds stress tensor. We apply the new CNN model to a number of one-dimensional turbulent flows. Additionally, we present interpretability techniques that help drive the model design and provide guidance on the model behavior in relation to the underlying physics.

physics.flu-dyn↗

arXiv2020

Learning a Reduced Basis of Dynamical Systems using an Autoencoder

Machine learning models have emerged as powerful tools in physics and engineering. Although flexible, a fundamental challenge remains on how to connect new machine learning models with known physics. In this work, we present an autoencoder with latent space penalization, which discovers finite dimensional manifolds underlying the partial differential equations of physics. We test this method on the Kuramoto-Sivashinsky (K-S), Korteweg-de Vries (KdV), and damped KdV equations. We show that the resulting optimal latent space of the K-S equation is consistent with the dimension of the inertial manifold. The results for the KdV equation imply that there is no reduced latent space, which is consistent with the truly infinite dimensional dynamics of the KdV equation. In the case of the damped KdV equation, we find that the number of active dimensions decreases with increasing damping coefficient. We then uncover a nonlinear basis representing the manifold of the latent space for the K-S equation.

physics.comp-ph↗

arXiv2020

Unsupervised Learning of Solutions to Differential Equations with Generative Adversarial Networks

Solutions to differential equations are of significant scientific and engineering relevance. Recently, there has been a growing interest in solving differential equations with neural networks. This work develops a novel method for solving differential equations with unsupervised neural networks that applies Generative Adversarial Networks (GANs) to \emph{learn the loss function} for optimizing the neural network. We present empirical results showing that our method, which we call Differential Equation GAN (DEQGAN), can obtain multiple orders of magnitude lower mean squared errors than an alternative unsupervised neural network method based on (squared) $L_2$, $L_1$, and Huber loss functions. Moreover, we show that DEQGAN achieves solution accuracy that is competitive with traditional numerical methods. Finally, we analyze the stability of our approach and find it to be sensitive to the selection of hyperparameters, which we provide in the appendix. Code available at https://github.com/dylanrandle/denn. Please address any electronic correspondence to dylanrandle@alumni.harvard.edu.

cs.LG↗

arXiv2020

Coherent Solutions and Transition to Turbulence in Two-Dimensional Rayleigh-Bénard Convection

For two-dimensional Rayleigh-Bénard convection, classes of unstable, steady solutions were previously computed using numerical continuation (Waleffe, 2015; Sondak, 2015). The `primary' steady solution bifurcates from the conduction state at $Ra \approx 1708$, and has a characteristic aspect ratio (length/height) of approximately $2$. The primary solution corresponds to one pair of counterclockwise-clockwise convection rolls with a temperature updraft in between and an adjacent downdraft on the sides. By adjusting the horizontal length of the domain, (Waleffe, 2015; Sondak, 2015) also found steady, maximal heat transport solutions, with characteristic aspect ratio less than $2$ and decreasing with increasing $Ra$. Compared to the primary solutions, optimal heat transport solutions have modifications to boundary layer thickness, the horizontal length scale of the plume, and the structure of the downdrafts. The current study establishes a direct link between these (unstable) steady solutions and transition to turbulence for $Pr = 7$ and $Pr = 100$. For transitional values of $Ra$, the primary and optimal heat transport solutions both appear prominently in appropriately-sized sub-fields of the time-evolving temperature fields. For $Ra$ beyond transitional, our data analysis shows persistence of the primary solution for $Pr = 7$, while the optimal heat transport solutions are more easily detectable for $Pr = 100$. In both cases $Pr = 7$ and $Pr = 100$, the relative prevalence of primary and optimal solutions is consistent with the $Nu$ vs. $Ra$ scalings for the numerical data and the steady solutions.

physics.flu-dyn↗

arXiv2020

Solving Differential Equations Using Neural Network Solution Bundles

The time evolution of dynamical systems is frequently described by ordinary differential equations (ODEs), which must be solved for given initial conditions. Most standard approaches numerically integrate ODEs producing a single solution whose values are computed at discrete times. When many varied solutions with different initial conditions to the ODE are required, the computational cost can become significant. We propose that a neural network be used as a solution bundle, a collection of solutions to an ODE for various initial states and system parameters. The neural network solution bundle is trained with an unsupervised loss that does not require any prior knowledge of the sought solutions, and the resulting object is differentiable in initial conditions and system parameters. The solution bundle exhibits fast, parallelizable evaluation of the system state, facilitating the use of Bayesian inference for parameter estimation in real dynamical systems.

cs.LG↗

arXiv2020

High Rayleigh number variational multiscale large eddy simulations of Rayleigh-Bénard Convection

The variational multiscale (VMS) formulation is used to develop residual-based VMS large eddy simulation (LES) models for Rayleigh-Bénard convection. The resulting model is a mixed model that incorporates the VMS model and an eddy viscosity model. The Wall-Adapting Local Eddy-viscosity (WALE) model is used as the eddy viscosity model in this work. The new LES models were implemented in the finite element code Drekar. Simulations are performed using continuous, piecewise linear finite elements. The simulations ranged from $Ra = 10^6$ to $Ra = 10^{14}$ and were conducted at $Pr = 1$ and $Pr = 7$. Two domains were considered: a two-dimensional domain of aspect ratio 2 with a fluid confined between two parallel plates and a three-dimensional cylinder of aspect ratio $1/4$. The Nusselt number from the VMS results is compared against three dimensional direct numerical simulations and experiments. In all cases, the VMS results are in good agreement with existing literature.

physics.comp-ph↗

arXiv2019

Neural Network Models for the Anisotropic Reynolds Stress Tensor in Turbulent Channel Flow

Reynolds-averaged Navier-Stokes (RANS) equations are presently one of the most popular models for simulating turbulence. Performing RANS simulation requires additional modeling for the anisotropic Reynolds stress tensor, but traditional Reynolds stress closure models lead to only partially reliable predictions. Recently, data-driven turbulence models for the Reynolds anisotropy tensor involving novel machine learning techniques have garnered considerable attention and have been rapidly developed. Focusing on modeling the Reynolds stress closure for the specific case of turbulent channel flow, this paper proposes three modifications to a standard neural network to account for the no-slip boundary condition of the anisotropy tensor, the Reynolds number dependence, and spatial non-locality. The modified models are shown to provide increased predicative accuracy compared to the standard neural network when they are trained and tested on channel flow at different Reynolds numbers. The best performance is yielded by the model combining the boundary condition enforcement and Reynolds number injection. This model also outperforms the Tensor Basis Neural Network (Ling et al., 2016) on the turbulent channel flow dataset.

physics.flu-dyn↗

arXiv2018

Deep learning for turbulent channel flow

Turbulence modeling is a classical approach to address the multiscale nature of fluid turbulence. Instead of resolving all scales of motion, which is currently mathematically and numerically intractable, reduced models that capture the large-scale behavior are derived. One of the most popular reduced models is the Reynolds averaged Navier-Stokes (RANS) equations. The goal is to solve the RANS equations for the mean velocity and pressure field. However, the RANS equations contain a term called the Reynolds stress tensor, which is not known in terms of the mean velocity field. Many RANS turbulence models have been proposed to model the Reynolds stress tensor in terms of the mean velocity field, but are usually not suitably general for all flow fields of interest. Data-driven turbulence models have recently garnered considerable attention and have been rapidly developed. In a seminal work, Ling et al (2016) developed the tensor basis neural network (TBNN), which was used to learn a general Galilean invariant model for the Reynolds stress tensor. The TBNN was applied to a variety of flow fields with encouraging results. In the present study, the TBNN is applied to the turbulent channel flow. Its performance is compared with classical turbulence models as well as a neural network model that does not preserve Galilean invariance. A sensitivity study on the TBNN reveals that the network attempts to adjust to the dataset, but is limited by the mathematical form that guarantees Galilean invariance.

physics.flu-dyn↗

arXiv2016

Can phoretic particles swim in two dimensions?

Artificial phoretic particles swim using self-generated gradients in chemical species (self-diffusiophoresis) or charges and currents (self-electrophoresis). These particles can be used to study the physics of collective motion in active matter and might have promising applications in bioengineering. In the case of self-diffusiophoresis, the classical physical model relies on a steady solution of the diffusion equation, from which chemical gradients, phoretic flows and ultimately the swimming velocity, may be derived. Motivated by disk-shaped particles in thin films and under confinement, we examine the extension to two dimensions. Because the two-dimensional diffusion equation lacks a steady state with the correct boundary conditions, Laplace transforms must be used to study the long-time behavior of the problem and determine the swimming velocity. For fixed chemical fluxes on the particle surface, we find that the swimming velocity ultimately always decays logarithmically in time. In the case of finite Peclet numbers, we solve the full advection-diffusion equation numerically and show that this decay can be avoided by the particle moving to regions of unconsumed reactant. Finite advection thus regularizes the two-dimensional phoretic problem.

cond-mat.soft↗

arXiv2015

Optimal Heat Transport in Rayleigh-Bénard Convection

Steady flows that optimize heat transport are obtained for two-dimensional Rayleigh-Bénard convection with no-slip horizontal walls for a variety of Prandtl numbers $Pr$ and Rayleigh number up to $Ra\sim 10^9$. Power law scalings of $Nu\sim Ra^γ$ are observed with $γ\approx 0.31$, where the Nusselt number $Nu$ is a non-dimensional measure of the vertical heat transport. Any dependence of the scaling exponent on $Pr$ is found to be extremely weak. On the other hand, the presence of two local maxima of $Nu$ with different horizontal wavenumbers at the same $Ra$ leads to the emergence of two different flow structures as candidates for optimizing the heat transport. For $Pr \lesssim 7$, optimal transport is achieved at the smaller maximal wavenumber. In these fluids, the optimal structure is a plume of warm rising fluid which spawns left/right horizontal arms near the top of the channel, leading to downdrafts adjacent to the central updraft. For $Pr > 7$ at high-enough Ra, the optimal structure is a single updraft absent significant horizontal structure, and characterized by the larger maximal wavenumber.

physics.flu-dyn↗

arXiv2014

A new class of finite element variational multiscale turbulence models for incompressible magnetohydrodynamics

New large eddy simulation (LES) turbulence models for incompressible magnetohydrodynamics (MHD) derived from the variational multiscale (VMS) formulation for finite element simulations are introduced. The new models include the variational multiscale formulation, a residual-based eddy viscosity model, and a mixed model that combines both of these component models. Each model contains terms that are proportional to the residual of the incompressible MHD equations and is therefore numerically consistent. Moreover, each model is also dynamic, in that its effect vanishes when this residual is small. The new models are tested on the decaying MHD Taylor Green vortex at low and high Reynolds numbers. The evaluation of the models is based on comparisons with available data from direct numerical simulations (DNS) of the time evolution of energies as well as energy spectra at various discrete times. A numerical study, on a sequence of meshes, is presented that demonstrates that the large eddy simulation approaches the DNS solution for these quantities with spatial mesh refinement.

physics.comp-ph↗