arXiv subjects

Michael P. Brenner

Publications and source records attributed to Michael P. Brenner.

At least 55 records · Page 3Linked to original sources

arXiv2022

Context-Aware Abbreviation Expansion Using Large Language Models

Motivated by the need for accelerating text entry in augmentative and alternative communication (AAC) for people with severe motor impairments, we propose a paradigm in which phrases are abbreviated aggressively as primarily word-initial letters. Our approach is to expand the abbreviations into full-phrase options by leveraging conversation context with the power of pretrained large language models (LLMs). Through zero-shot, few-shot, and fine-tuning experiments on four public conversation datasets, we show that for replies to the initial turn of a dialog, an LLM with 64B parameters is able to exactly expand over 70% of phrases with abbreviation length up to 10, leading to an effective keystroke saving rate of up to about 77% on these exact expansions. Including a small amount of context in the form of a single conversation turn more than doubles abbreviation expansion accuracies compared to having no context, an effect that is more pronounced for longer phrases. Additionally, the robustness of models against typo noise can be enhanced through fine-tuning on noisy data.

cs.CL↗

arXiv2022

Improved tracking of particles with highly correlated motion

Despite significant advances in particle imaging technologies over the past two decades, few advances have been made in particle tracking, i.e. linking individual particle positions across time series data. The state-of-the-art tracking algorithm is highly effective for systems in which the particles behave mostly independently. However, these algorithms become inaccurate when particle motion is highly correlated, such as in dense or strongly interacting systems. Accurate particle tracking is essential in the study of the physics of dense colloids, such as the study of dislocation formation, nucleation, and shear transformations. Here, we present a new method for particle tracking that incorporates information about the correlated motion of the particles. We demonstrate significant improvement over the state-of-the-art tracking algorithm in simulated data on highly correlated systems.

cond-mat.soft↗

arXiv2022

Optimal control of nonequilibrium systems through automatic differentiation

Controlling the evolution of nonequilibrium systems to minimize dissipated heat or work is a key goal for designing nanodevices, both in nanotechnology and biology. Progress in computing optimal protocols has thus far been limited to either simple systems or near-equilibrium evolution. Here, we present an approach for computing optimal protocols based on automatic differentiation. Our methodology is applicable to complex systems and multidimensional protocols and is valid arbitrarily far from equilibrium. We validate our method by reproducing theoretical optimal protocols for a Brownian particle in a time-varying harmonic trap. We also compute departures from near-equilibrium behaviour for magnetization reversal on an Ising lattice and for barrier crossing driven by a harmonic trap, which has been used to represent a range of biological processes including biomolecular unfolding reactions. Algorithms based on automatic differentiation outperform the near-equilibrium theory for far-from-equilibrium magnetization reversal and driven barrier crossing. The optimal protocol for crossing an energy landscape barrier of 10kT is found to hasten the approach to, and slow the departure from, the barrier region compared to the near-equilibrium theoretical protocol.

physics.comp-ph↗

arXiv2021

Programming Interactions in Magnetic Handshake Materials

The ability to rapidly manufacture building blocks with specific binding interactions is a key aspect of programmable assembly. Recent developments in DNA nanotechnology and colloidal particle synthesis have significantly advanced our ability to create particle sets with programmable interactions, based on DNA or shape complementarity. The increasing miniaturization underlying magnetic storage offers a new path for engineering programmable components for self assembly, by printing magnetic dipole patterns on substrates using nanotechnology. How to efficiently design dipole patterns for programmable assembly remains an open question as the design space is combinatorially large. Here, we present design rules for programming these magnetic interactions. By optimizing the structure of the dipole pattern, we demonstrate that the number of independent building blocks scales super linearly with the number of printed domains. We test these design rules using computational simulations of self assembled blocks, and experimental realizations of the blocks at the mm scale, demonstrating that the designed blocks give high yield assembly. In addition, our design rules indicate that with current printing technology, micron sized magnetic panels could easily achieve hundreds of different building blocks.

cond-mat.soft↗

arXiv2021

Using a Cross-Task Grid of Linear Probes to Interpret CNN Model Predictions On Retinal Images

We analyze a dataset of retinal images using linear probes: linear regression models trained on some "target" task, using embeddings from a deep convolutional (CNN) model trained on some "source" task as input. We use this method across all possible pairings of 93 tasks in the UK Biobank dataset of retinal images, leading to ~164k different models. We analyze the performance of these linear probes by source and target task and by layer depth. We observe that representations from the middle layers of the network are more generalizable. We find that some target tasks are easily predicted irrespective of the source task, and that some other target tasks are more accurately predicted from correlated source tasks than from embeddings trained on the same task.

cs.LG↗

arXiv2021

Comparing Supervised Models And Learned Speech Representations For Classifying Intelligibility Of Disordered Speech On Selected Phrases

Automatic classification of disordered speech can provide an objective tool for identifying the presence and severity of speech impairment. Classification approaches can also help identify hard-to-recognize speech samples to teach ASR systems about the variable manifestations of impaired speech. Here, we develop and compare different deep learning techniques to classify the intelligibility of disordered speech on selected phrases. We collected samples from a diverse set of 661 speakers with a variety of self-reported disorders speaking 29 words or phrases, which were rated by speech-language pathologists for their overall intelligibility using a five-point Likert scale. We then evaluated classifiers developed using 3 approaches: (1) a convolutional neural network (CNN) trained for the task, (2) classifiers trained on non-semantic speech representations from CNNs that used an unsupervised objective [1], and (3) classifiers trained on the acoustic (encoder) embeddings from an ASR system trained on typical speech [2]. We found that the ASR encoder's embeddings considerably outperform the other two on detecting and classifying disordered speech. Further analysis shows that the ASR embeddings cluster speech by the spoken phrase, while the non-semantic embeddings cluster speech by speaker. Also, longer phrases are more indicative of intelligibility deficits than single words.

eess.AS↗

arXiv2021

Variational Data Assimilation with a Learned Inverse Observation Operator

Variational data assimilation optimizes for an initial state of a dynamical system such that its evolution fits observational data. The physical model can subsequently be evolved into the future to make predictions. This principle is a cornerstone of large scale forecasting applications such as numerical weather prediction. As such, it is implemented in current operational systems of weather forecasting agencies across the globe. However, finding a good initial state poses a difficult optimization problem in part due to the non-invertible relationship between physical states and their corresponding observations. We learn a mapping from observational data to physical states and show how it can be used to improve optimizability. We employ this mapping in two ways: to better initialize the non-convex optimization problem, and to reformulate the objective function in better behaved physics space instead of observation space. Our experimental results for the Lorenz96 model and a two-dimensional turbulent fluid flow demonstrate that this procedure significantly improves forecast quality for chaotic systems.

cs.LG↗

arXiv2021

Topological Design of Heterogeneous Self-Assembly

Controlling the topology of structures self-assembled from a set of heterogeneous building blocks is highly desirable for many applications, but is poorly understood theoretically. Here we show that the thermodynamic theory of self-assembly involves an inevitable divergence in chemical potential. The divergence and its detailed structure are controlled by the spectrum of the transfer matrix, which summarizes all of self-assembly design degrees of freedom. By analyzing the transfer matrix, we map out the phase boundary between the designable structures and the unstructured aggregates, driven by the level of cross-talk.

cond-mat.soft↗

arXiv2021

Cascades and Reconnection in Interacting Vortex Filaments

At high Reynolds number, the interaction between two vortex tubes leads to intense velocity gradients, which are at the heart of fluid turbulence. This vorticity amplification comes about through two different instability mechanisms of the initial vortex tubes, assumed anti-parallel and with a mirror plane of symmetry. At moderate Reynolds number, the tubes destabilize via a Crow instability, with the nonlinear development leading to strong flattening of the cores into thin sheets. These sheets then break down into filaments which can repeat the process. At higher Reynolds number, the instability proceeds via the elliptical instability, producing vortex tubes that are perpendicular to the original tube directions. In this work, we demonstrate that these same transition between Crow and Elliptical instability occurs at moderate Reynolds number when we vary the initial angle $β$ between two straight vortex tubes. We demonstrate that when the angle between the two tubes is close to $π/2$, the interaction between tubes leads to the formation of thin vortex sheets. The subsequent breakdown of these sheets involves a twisting of the paired sheets, followed by the appearance of a localized cloud of small scale vortex structures. At smaller values of the angle $β$ between the two tubes, the breakdown mechanism changes to an elliptic cascade-like mechanism. Whereas the interaction of two vortices depends on the initial condition, the rapid formation of fine-scales vortex structures appears to be a robust feature, possibly universal at very high Reynolds numbers.

physics.flu-dyn↗

arXiv2021

Machine learning accelerated computational fluid dynamics

Numerical simulation of fluids plays an essential role in modeling many physical phenomena, such as weather, climate, aerodynamics and plasma physics. Fluids are well described by the Navier-Stokes equations, but solving these equations at scale remains daunting, limited by the computational cost of resolving the smallest spatiotemporal features. This leads to unfavorable trade-offs between accuracy and tractability. Here we use end-to-end deep learning to improve approximations inside computational fluid dynamics for modeling two-dimensional turbulent flows. For both direct numerical simulation of turbulence and large eddy simulation, our results are as accurate as baseline solvers with 8-10x finer resolution in each spatial dimension, resulting in 40-80x fold computational speedups. Our method remains stable during long simulations, and generalizes to forcing functions and Reynolds numbers outside of the flows where it is trained, in contrast to black box machine learning approaches. Our approach exemplifies how scientific computing can leverage machine learning and hardware accelerators to improve simulations without sacrificing accuracy or generalization.

physics.flu-dyn↗

arXiv2020

Learned discretizations for passive scalar advection in a 2-D turbulent flow

The computational cost of fluid simulations increases rapidly with grid resolution. This has given a hard limit on the ability of simulations to accurately resolve small scale features of complex flows. Here we use a machine learning approach to learn a numerical discretization that retains high accuracy even when the solution is under-resolved with classical methods. We apply this approach to passive scalar advection in a two-dimensional turbulent flow. The method maintains the same accuracy as traditional high-order flux-limited advection solvers, while using 4x lower grid resolution in each dimension. The machine learning component is tightly integrated with traditional finite-volume schemes and can be trained via an end-to-end differentiable programming framework. The solver can achieve near-peak hardware utilization on CPUs and accelerators via convolutional filters. Code is available at https://github.com/google-research/data-driven-pdes.

physics.comp-ph↗

arXiv2020

Revealing the state space of turbulence using machine learning

Despite the apparent complexity of turbulent flow, identifying a simpler description of the underlying dynamical system remains a fundamental challenge. Capturing how the turbulent flow meanders amongst unstable states (simple invariant solutions) in phase space, as envisaged by Hopf in 1948, using some efficient representation offers the best hope of doing this, despite the inherent difficulty in identifying these states. Here, we make a significant step towards this goal by demonstrating that deep convolutional autoencoders can identify low-dimensional representations of two-dimensional turbulence which are closely associated with the simple invariant solutions characterizing the turbulent attractor. To establish this, we develop latent Fourier analysis that decomposes the flow embedding into a set of orthogonal latent Fourier modes which decode into physically meaningful patterns resembling simple invariant solutions. The utility of this approach is highlighted by analysing turbulent Kolmogorov flow (flow on a 2D torus forced at large scale) at $Re=40$ where, in between intermittent bursts, the flow resides in the neighbourhood of an unstable state and is very low dimensional. Projections onto individual latent Fourier wavenumbers reveal the simple invariant solutions organising both the quiescent and bursting dynamics in a systematic way inaccessible to previous approaches.

physics.flu-dyn↗

arXiv2020

Predicting ground state configuration of energy landscape ensemble using graph neural network

Many scientific problems seek to find the ground state in a rugged energy landscape, a task that becomes prohibitively difficult for large systems. Within a particular class of problems, however, the short-range correlations within energy minima might be independent of system size. Can these correlations be inferred from small problems with known ground states to accelerate the search for the ground states of larger problems? Here, we demonstrate the strategy on Ising spin glasses, where the interaction matrices are drawn from protein contact maps. We use graph neural network to learn the mapping from an interaction matrix $J$ to a ground state configuration, yielding guesses for the set of most probable configurations. Given these guesses, we show that ground state configurations can be searched much faster than in vanilla simulated annealing. For large problems, a model trained on small $J$ matrices predicts a configurations whose energy is much lower than those obtained by simulated annealing, indicating the size generalizability of the strategy.

physics.comp-ph↗

arXiv2019

Robust increase in supply by vessel dilation in globally coupled microvasculature

Neuronal activity induces changes in blood flow by locally dilating vessels in the brain microvasculature. How can the local dilation of a single vessel increase flow-based metabolite supply, given that flows are globally coupled within microvasculature? Solving the supply dynamics for rat brain microvasculature, we find one parameter regime to dominate physiologically. This regime allows for robust increase in supply independent of the position in the network, which we explain analytically. We show that local coupling of vessels promotes spatially correlated increased supply by dilation.

physics.bio-ph↗

arXiv2019

Turbulence generation through an iterative cascade of the elliptical instability

The essence of turbulent flow is the conveyance of energy through the formation, interaction, and destruction of eddies over a wide range of spatial scales--from the largest scales where energy is injected, down to the smallest scales where it is dissipated through viscosity. Currently, there is no mechanistic framework that captures how the interactions of vortices drive this cascade. We show that iterations of the elliptical instability, arising from the interactions between counter-rotating vortices, lead to the emergence of turbulence. We demonstrate how the nonlinear development of the elliptical instability generates an ordered array of antiparallel secondary filaments. The secondary filaments mutually interact, leading to the formation of even smaller tertiary filaments. In experiments and simulations, we observe two and three iterations of this cascade, respectively. Our observations indicate that the elliptical instability could be one of the fundamental mechanisms by which the turbulent cascade develops.

physics.flu-dyn↗

arXiv2019

Learning data driven discretizations for partial differential equations

The numerical solution of partial differential equations (PDEs) is challenging because of the need to resolve spatiotemporal features over wide length and timescales. Often, it is computationally intractable to resolve the finest features in the solution. The only recourse is to use approximate coarse-grained representations, which aim to accurately represent long-wavelength dynamics while properly accounting for unresolved small scale physics. Deriving such coarse grained equations is notoriously difficult, and often \emph{ad hoc}. Here we introduce \emph{data driven discretization}, a method for learning optimized approximations to PDEs based on actual solutions to the known underlying equations. Our approach uses neural networks to estimate spatial derivatives, which are optimized end-to-end to best satisfy the equations on a low resolution grid. The resulting numerical methods are remarkably accurate, allowing us to integrate in time a collection of nonlinear equations in one spatial dimension at resolutions 4-8x coarser than is possible with standard finite difference methods.

cond-mat.dis-nn↗

arXiv2019

Using Attribution to Decode Dataset Bias in Neural Network Models for Chemistry

Deep neural networks have achieved state of the art accuracy at classifying molecules with respect to whether they bind to specific protein targets. A key breakthrough would occur if these models could reveal the fragment pharmacophores that are causally involved in binding. Extracting chemical details of binding from the networks could potentially lead to scientific discoveries about the mechanisms of drug actions. But doing so requires shining light into the black box that is the trained neural network model, a task that has proved difficult across many domains. Here we show how the binding mechanism learned by deep neural network models can be interrogated, using a recently described attribution method. We first work with carefully constructed synthetic datasets, in which the 'fragment logic' of binding is fully known. We find that networks that achieve perfect accuracy on held out test datasets still learn spurious correlations due to biases in the datasets, and we are able to exploit this non-robustness to construct adversarial examples that fool the model. The dataset bias makes these models unreliable for accurately revealing information about the mechanisms of protein-ligand binding. In light of our findings, we prescribe a test that checks for dataset bias given a hypothesis. If the test fails, it indicates that either the model must be simplified or regularized and/or that the training dataset requires augmentation.

cs.LG↗

arXiv2018

A Cascade Leading to the Emergence of Small Structures in Vortex Ring Collisions

When vortex rings collide head-on at high enough Reynolds numbers, they ultimately annihilate through a violent interaction which breaks down their cores into a turbulent cloud. We experimentally show that this very strong interaction, which leads to the production of fluid motion at very fine scales, uncovers direct evidence of a novel iterative cascade of instabilities in a bulk fluid. When the coherent vortex cores approach each other, they deform into tent-like structures, and the mutual strain causes them to locally flatten into extremely thin vortex sheets. These sheets then break down into smaller secondary vortex filaments, which themselves rapidly flatten and break down into even smaller tertiary filaments. By performing numerical simulations of the full Navier-Stokes equations, we also resolve one iteration of this instability and highlight the subtle role that viscosity must play in the rupturing of a vortex sheet. The concurrence of this observed iterative cascade of instabilities over various scales with those of recent theoretical predictions could provide a new mechanistic framework in which the evolution of turbulent flows can be examined in real-time as a series of discrete dynamic instabilities.

physics.flu-dyn↗