arXiv subjects

Markus Gross

Publications and source records attributed to Markus Gross.

At least 73 records · Page 4Linked to original sources

arXiv2019

Data-Driven Physical Face Inversion

Facial animation is one of the most challenging problems in computer graphics, and it is often solved using linear heuristics like blend-shape rigging. More expressive approaches like physical simulation have emerged, but these methods are very difficult to tune, especially when simulating a real actor's face. We propose to use a simple finite element simulation approach for face animation, and present a novel method for recovering the required simulation parameters in order to best match a real actor's face motion. Our method involves reconstructing a very small number of head poses of the actor in 3D, where the head poses span different configurations of force directions due to gravity. Our algorithm can then automatically recover both the gravity-free rest shape of the face as well as the spatially-varying physical material stiffness such that a forward simulation will match the captured targets as closely as possible. As a result, our system can produce actor-specific, physical parameters that can be immediately used in recent physical simulation methods for faces. Furthermore, as the simulation results depend heavily on the chosen spatial layout of material clusters, we analyze and compare different spatial layouts.

cs.GR↗

arXiv2019

Explaining Deep Neural Networks with a Polynomial Time Algorithm for Shapley Values Approximation

The problem of explaining the behavior of deep neural networks has recently gained a lot of attention. While several attribution methods have been proposed, most come without strong theoretical foundations, which raises questions about their reliability. On the other hand, the literature on cooperative game theory suggests Shapley values as a unique way of assigning relevance scores such that certain desirable properties are satisfied. Unfortunately, the exact evaluation of Shapley values is prohibitively expensive, exponential in the number of input features. In this work, by leveraging recent results on uncertainty propagation, we propose a novel, polynomial-time approximation of Shapley values in deep neural networks. We show that our method produces significantly better approximations of Shapley values than existing state-of-the-art attribution methods.

cs.LG↗

arXiv2019

Ensemble dependence of Critical Casimir Forces in Films with Dirichlet Boundary Conditions

In a recent study [Phys. Rev. E \textbf{94}, 022103 (2016)] it has been shown that, for a fluid film subject to critical adsorption, the resulting critical Casimir force (CCF) may significantly depend on the thermodynamic ensemble. Here, we extend that study by considering fluid films within the so-called ordinary surface universality class. We focus on mean-field theory, within which the OP profile satisfies Dirichlet boundary conditions and produces a nontrivial CCF in the presence of external bulk fields or, respectively, a nonzero total order parameter within the film. Our analytical results are supported by Monte Carlo simulations of the three-dimensional Ising model. We show that, in the canonical ensemble, i.e., when fixing the so-called total mass within the film, the CCF is typically repulsive instead of attractive as in the grand canonical ensemble. Based on the Landau-Ginzburg free energy, we furthermore obtain analytic expressions for the order parameter profiles and analyze the relation between the total mass in the film and the external bulk field.

cond-mat.stat-mech↗

arXiv2019

Generating Animations from Screenplays

Automatically generating animation from natural language text finds application in a number of areas e.g. movie script writing, instructional videos, and public safety. However, translating natural language text into animation is a challenging task. Existing text-to-animation systems can handle only very simple sentences, which limits their applications. In this paper, we develop a text-to-animation system which is capable of handling complex sentences. We achieve this by introducing a text simplification step into the process. Building on an existing animation generation system for screenwriting, we create a robust NLP pipeline to extract information from screenplays and map them to the system's knowledge base. We develop a set of linguistic transformation rules that simplify complex sentences. Information extracted from the simplified sentences is used to generate a rough storyboard and video depicting the text. Our sentence simplification module outperforms existing systems in terms of BLEU and SARI metrics.We further evaluated our system via a user study: 68 % participants believe that our system generates reasonable animation from input screenplays.

cs.CL↗

arXiv2019

Neural Sequential Phrase Grounding (SeqGROUND)

We propose an end-to-end approach for phrase grounding in images. Unlike prior methods that typically attempt to ground each phrase independently by building an image-text embedding, our architecture formulates grounding of multiple phrases as a sequential and contextual process. Specifically, we encode region proposals and all phrases into two stacks of LSTM cells, along with so-far grounded phrase-region pairs. These LSTM stacks collectively capture context for grounding of the next phrase. The resulting architecture, which we call SeqGROUND, supports many-to-many matching by allowing an image region to be matched to multiple phrases and vice versa. We show competitive performance on the Flickr30K benchmark dataset and, through ablation studies, validate the efficacy of sequential grounding as well as individual design choices in our model architecture.

cs.CV↗

arXiv2019

Deep Fluids: A Generative Network for Parameterized Fluid Simulations

This paper presents a novel generative model to synthesize fluid simulations from a set of reduced parameters. A convolutional neural network is trained on a collection of discrete, parameterizable fluid simulation velocity fields. Due to the capability of deep learning architectures to learn representative features of the data, our generative model is able to accurately approximate the training data set, while providing plausible interpolated in-betweens. The proposed generative model is optimized for fluids by a novel loss function that guarantees divergence-free velocity fields at all times. In addition, we demonstrate that we can handle complex parameterizations in reduced spaces, and advance simulations in time by integrating in the latent space with a second network. Our method models a wide variety of fluid behaviors, thus enabling applications such as fast construction of simulations, interpolation of fluids with different parameters, time re-sampling, latent space simulations, and compression of fluid simulation data. Reconstructed velocity fields are generated up to 700x faster than re-simulating the data with the underlying CPU solver, while achieving compression rates of up to 1300x.

cs.LG↗

arXiv2018

Disentangled Dynamic Representations from Unordered Data

We present a deep generative model that learns disentangled static and dynamic representations of data from unordered input. Our approach exploits regularities in sequential data that exist regardless of the order in which the data is viewed. The result of our factorized graphical model is a well-organized and coherent latent space for data dynamics. We demonstrate our method on several synthetic dynamic datasets and real video data featuring various facial expressions and head poses.

stat.ML↗

arXiv2018

Surface-induced non-equilibrium dynamics and critical Casimir forces for model B in film geometry

Using analytic and numerical approaches, we study the spatio-temporal evolution of a conserved order parameter of a fluid in film geometry, following an instantaneous quench to the critical temperature $T_c$ as well as to supercritical temperatures. The order parameter dynamics is chosen to be governed by model B within mean field theory and is subject to no-flux boundary conditions as well as to symmetric surface fields at the confining walls. The latter give rise to critical adsorption of the order parameter at both walls and provide the driving force for the non-trivial time evolution of the order parameter. During the dynamics, the order parameter is locally and globally conserved; thus, at thermal equilibrium, the system represents the canonical ensemble. We furthermore consider the dynamics of the nonequilibrium critical Casimir force, which we obtain based on the generalized force exerted by the order parameter field on the confining walls. We identify various asymptotic regimes concerning the time evolution of the order parameter and the critical Casimir force and we provide, within our approach, exact expressions of the corresponding dynamic scaling functions.

cond-mat.stat-mech↗

arXiv2018

Deep Video Color Propagation

Traditional approaches for color propagation in videos rely on some form of matching between consecutive video frames. Using appearance descriptors, colors are then propagated both spatially and temporally. These methods, however, are computationally expensive and do not take advantage of semantic information of the scene. In this work we propose a deep learning framework for color propagation that combines a local strategy, to propagate colors frame-by-frame ensuring temporal stability, and a global strategy, using semantics for color propagation within a longer range. Our evaluation shows the superiority of our strategy over existing video and image color propagation methods as well as neural photo-realistic style transfer approaches.

cs.CV↗

arXiv2018

Ten Years of Research on Intelligent Educational Games for Learning Spelling and Mathematics

In this article, we present our findings from ten years of research on intelligent educational games. We discuss the architecture of our training environments for learning spelling and mathematics, and specifically focus on the representation of the content and the controller that enables personalized trainings. We first show the multi-modal representation that reroutes information through multiple perceptual cues and discuss the game structure. We then present the data-driven student model that is used for a personalized, adaptive presentation of the content. We further leverage machine learning for analytics and visualization tools targeted at teachers and experts. A large data set consisting of training sessions of more than 20,000 children allows statistical interpretations and insights into the nature of learning.

cs.CY↗

arXiv2018

Shear-density coupling for a compressible single-component yield-stress fluid

Flow behavior of a single-component yield stress fluid is addressed on the hydrodynamic level. A basic ingredient of the model is a coupling between fluctuations of density and velocity gradient via a Herschel-Bulkley-type constitutive model. Focusing on the limit of low shear rates and high densities, the model approximates well---but is not limited to---gently sheared hard sphere colloidal glasses, where solvent effects are negligible. A detailed analysis of the linearized hydrodynamic equations for fluctuations and the resulting cubic dispersion relation reveals the existence of a range of densities and shear rates with growing flow heterogeneity. In this regime, after an initial transient, the velocity and density fields monotonically reach a spatially inhomogeneous stationary profile, where regions of high shear rate and low density coexist with regions of low shear rate and high density. The steady state is thus maintained by a competition between shear-induced enhancement of density inhomogeneities and relaxation via overdamped sound waves. An analysis of the mechanical equilibrium condition provides a criterion for the existence of steady state solutions. The dynamical evolution of the system is discussed in detail for various boundary conditions, imposing either a constant velocity, shear rate, or stress at the walls.

cond-mat.soft↗

arXiv2018

A Neural Multi-sequence Alignment TeCHnique (NeuMATCH)

The alignment of heterogeneous sequential data (video to text) is an important and challenging problem. Standard techniques for this task, including Dynamic Time Warping (DTW) and Conditional Random Fields (CRFs), suffer from inherent drawbacks. Mainly, the Markov assumption implies that, given the immediate past, future alignment decisions are independent of further history. The separation between similarity computation and alignment decision also prevents end-to-end training. In this paper, we propose an end-to-end neural architecture where alignment actions are implemented as moving data between stacks of Long Short-term Memory (LSTM) blocks. This flexible architecture supports a large variety of alignment tasks, including one-to-one, one-to-many, skipping unmatched elements, and (with extensions) non-monotonic alignment. Extensive experiments on semi-synthetic and real datasets show that our algorithm outperforms state-of-the-art baselines.

cs.CV↗

arXiv2018

PhaseNet for Video Frame Interpolation

Most approaches for video frame interpolation require accurate dense correspondences to synthesize an in-between frame. Therefore, they do not perform well in challenging scenarios with e.g. lighting changes or motion blur. Recent deep learning approaches that rely on kernels to represent motion can only alleviate these problems to some extent. In those cases, methods that use a per-pixel phase-based motion representation have been shown to work well. However, they are only applicable for a limited amount of motion. We propose a new approach, PhaseNet, that is designed to robustly handle challenging scenarios while also coping with larger motion. Our approach consists of a neural network decoder that directly estimates the phase decomposition of the intermediate frame. We show that this is superior to the hand-crafted heuristics previously used in phase-based methods and also compares favorably to recent deep learning based approaches for video frame interpolation on challenging datasets.

cs.CV↗

arXiv2018

First-passage dynamics of linear stochastic interface models: weak-noise theory and influence of boundary conditions

We consider a one-dimensional fluctuating interfacial profile governed by the Edwards-Wilkinson or the stochastic Mullins-Herring equation for periodic, standard Dirichlet and Dirichlet no-flux boundary conditions. The minimum action path of an interfacial fluctuation conditioned to reach a given maximum height $M$ at a finite (first-passage) time $T$ is calculated within the weak-noise approximation. Dynamic and static scaling functions for the profile shape are obtained in the transient and the equilibrium regime, i.e., for first-passage times $T$ smaller or lager than the characteristic relaxation time, respectively. In both regimes, the profile approaches the maximum height $M$ with a universal algebraic time dependence characterized solely by the dynamic exponent of the model. It is shown that, in the equilibrium regime, the spatial shape of the profile depends sensitively on boundary conditions and conservation laws, but it is essentially independent of them in the transient regime.

cond-mat.stat-mech↗

arXiv2018

First-passage dynamics of linear stochastic interface models: numerical simulations and entropic repulsion effect

A fluctuating interfacial profile in one dimension is studied via Langevin simulations of the Edwards-Wilkinson equation with non-conserved noise and the Mullins-Herring equation with conserved noise. The profile is subject to either periodic or Dirichlet (no-flux) boundary conditions. We determine the noise-driven time-evolution of the profile between an initially flat configuration and the instant at which the profile reaches a given height $M$ for the first time. The shape of the averaged profile agrees well with the prediction of weak-noise theory (WNT), which describes the most-likely trajectory to a fixed first-passage time. Furthermore, in agreement with WNT, on average the profile approaches the height $M$ algebraically in time, with an exponent that is essentially independent of the boundary conditions. However, the actual value of the dynamic exponent turns out to be significantly smaller than predicted by WNT. This "renormalization" of the exponent is explained in terms of the entropic repulsion exerted by the impenetrable boundary on the fluctuations of the profile around its most-likely path. The entropic repulsion mechanism is analyzed in detail for a single (fractional) Brownian walker, which describes the anomalous diffusion of a tagged monomer of the interface as it approaches the absorbing boundary. The present study sheds light on the accuracy and the limitations of the weak-noise approximation for the description of the full first-passage dynamics.

cond-mat.stat-mech↗

arXiv2018

Towards better understanding of gradient-based attribution methods for Deep Neural Networks

Understanding the flow of information in Deep Neural Networks (DNNs) is a challenging problem that has gain increasing attention over the last few years. While several methods have been proposed to explain network predictions, there have been only a few attempts to compare them from a theoretical perspective. What is more, no exhaustive empirical comparison has been performed in the past. In this work, we analyze four gradient-based attribution methods and formally prove conditions of equivalence and approximation between them. By reformulating two of these methods, we construct a unified framework which enables a direct comparison, as well as an easier implementation. Finally, we propose a novel evaluation metric, called Sensitivity-n and test the gradient-based attribution methods alongside with a simple perturbation-based attribution method on several datasets in the domains of image and text classification, using various network architectures.

cs.LG↗

arXiv2017

Deep Scattering: Rendering Atmospheric Clouds with Radiance-Predicting Neural Networks

We present a technique for efficiently synthesizing images of atmospheric clouds using a combination of Monte Carlo integration and neural networks. The intricacies of Lorenz-Mie scattering and the high albedo of cloud-forming aerosols make rendering of clouds---e.g. the characteristic silverlining and the "whiteness" of the inner body---challenging for methods based solely on Monte Carlo integration or diffusion theory. We approach the problem differently. Instead of simulating all light transport during rendering, we pre-learn the spatial and directional distribution of radiant flux from tens of cloud exemplars. To render a new scene, we sample visible points of the cloud and, for each, extract a hierarchical 3D descriptor of the cloud geometry with respect to the shading location and the light source. The descriptor is input to a deep neural network that predicts the radiance function for each shading configuration. We make the key observation that progressively feeding the hierarchical descriptor into the network enhances the network's ability to learn faster and predict with high accuracy while using few coefficients. We also employ a block design with residual connections to further improve performance. A GPU implementation of our method synthesizes images of clouds that are nearly indistinguishable from the reference solution within seconds interactively. Our method thus represents a viable solution for applications such as cloud design and, thanks to its temporal stability, also for high-quality production of animated content.

cs.LG↗

arXiv2017

Statistical field theory with constraints: application to critical Casimir forces in the canonical ensemble

The effect of imposing a constraint on a fluctuating scalar order parameter field in a system of finite volume is studied within statistical field theory. The canonical ensemble, corresponding to a fixed total integrated order parameter, is obtained as a special case of the theory. A perturbative expansion is developed which allows one to systematically determine the constraint-induced finite-volume corrections to the free energy and to correlation functions. In particular, we focus on the Landau-Ginzburg model in a film geometry (i.e., a rectangular parallelepiped with a small aspect ratio) with periodic, Dirichlet, or Neumann boundary conditions in the transverse direction and periodic boundary conditions in the remaining, lateral directions. Within the expansion in terms of $ε=4-d$, where $d$ is the spatial dimension of the bulk, the finite-size contribution to the free energy and the associated critical Casimir force are calculated to leading order in $ε$ and are compared to the corresponding expressions for an unconstrained (grand canonical) system. The constraint restricts the fluctuations within the system and it accordingly modifies the residual finite-size free energy. The resulting Casimir force is shown to depend on whether it is defined by assuming a fixed transverse area or a fixed total volume. In the former case, the constraint is typically found to significantly enhance the attractive character of the force as compared to the grand canonical case. In contrast to the grand canonical Casimir force, which, for supercritical temperatures, vanishes in the limit of thick films, the canonical Casimir force defined for fixed transverse area attains for thick films a negative value for all boundary conditions studied here. Typically, the dependence of the Casimir force both on the temperature- and on the field-like scaling variables is different in the two ensembles.

cond-mat.stat-mech↗