arXiv subjects

Joseph Kleinhenz

Publications and source records attributed to Joseph Kleinhenz.

14 recordsLinked to original sources

arXiv2026

Diffeomorphic Optimization

Generative models learn data distributions that reside on a low-dimensional manifold within a higher-dimensional ambient space. Optimizing differentiable objectives on this manifold is challenging: the ambient loss landscape is high-dimensional, rugged, and non-convex. Direct gradient descent, blind to the manifold's geometry, quickly drifts off it. Diffeomorphic optimization starts from the observation that diffusion and flow models provide a map from the data manifold to a much simpler base space in which we perform gradient descent. Using differential geometry, we show this is equivalent to Riemannian gradient descent on the data manifold up to $\mathcal{O}(\lambda^2)$ corrections, keeping trajectories on-manifold by construction and yielding a smoother optimization surface. For protein design, we extend diffeomorphic optimization to the matrix Lie groups $\mathrm{SO}(3)$ and $\mathrm{SE}(3)$, deriving an autograd-compatible $\mathrm{SO}(3)$ gradient and a generalized adjoint-state method for backpropagation through Lie-group ODE solvers. Diffeomorphic optimization improves over tuned guidance on secondary-structure targeting with FrameFlow ($91.3\%$ vs. $63.3\%$ of residues in the Ramachandran target), outperforms OC-Flow on peptide binding affinity at $2\times$ the speed, and reduces Rosetta energies by thousands of units across the PDB test set for structures with hundreds of residues.

cs.LG↗

arXiv2025

Unified all-atom molecule generation with neural fields

Generative models for structure-based drug design are often limited to a specific modality, restricting their broader applicability. To address this challenge, we introduce FuncBind, a framework based on computer vision to generate target-conditioned, all-atom molecules across atomic systems. FuncBind uses neural fields to represent molecules as continuous atomic densities and employs score-based generative models with modern architectures adapted from the computer vision literature. This modality-agnostic representation allows a single unified model to be trained on diverse atomic systems, from small to large molecules, and handle variable atom/residue counts, including non-canonical amino acids. FuncBind achieves competitive in silico performance in generating small molecules, macrocyclic peptides, and antibody complementarity-determining region loops, conditioned on target structures. FuncBind also generated in vitro novel antibody binders via de novo redesign of the complementarity-determining region H3 loop of two chosen co-crystal structures. As a final contribution, we introduce a new dataset and benchmark for structure-conditioned macrocyclic peptide generation. The code is available at https://github.com/prescient-design/funcbind.

cs.LG↗

arXiv2025

Matching the Optimal Denoiser in Point Cloud Diffusion with (Improved) Rotational Alignment

Diffusion models are a popular class of generative models trained to reverse a noising process starting from a target data distribution. Training a diffusion model consists of learning how to denoise noisy samples at different noise levels. When training diffusion models for point clouds such as molecules and proteins, there is often no canonical orientation that can be assigned. To capture this symmetry, the true data samples are often augmented by transforming them with random rotations sampled uniformly over $SO(3)$. Then, the denoised predictions are often rotationally aligned via the Kabsch-Umeyama algorithm to the ground truth samples before computing the loss. However, the effect of this alignment step has not been well studied. Here, we show that the optimal denoiser can be expressed in terms of a matrix Fisher distribution over $SO(3)$. Alignment corresponds to sampling the mode of this distribution, and turns out to be the zeroth order approximation for small noise levels, explaining its effectiveness. We build on this perspective to derive better approximators to the optimal denoiser in the limit of small noise. Our experiments highlight that alignment is often a `good enough' approximation for the noise levels that matter most for training diffusion models.

cs.LG↗

arXiv2024

Concept Bottleneck Language Models For protein design

We introduce Concept Bottleneck Protein Language Models (CB-pLM), a generative masked language model with a layer where each neuron corresponds to an interpretable concept. Our architecture offers three key benefits: i) Control: We can intervene on concept values to precisely control the properties of generated proteins, achieving a 3 times larger change in desired concept values compared to baselines. ii) Interpretability: A linear mapping between concept values and predicted tokens allows transparent analysis of the model's decision-making process. iii) Debugging: This transparency facilitates easy debugging of trained models. Our models achieve pre-training perplexity and downstream task performance comparable to traditional masked protein language models, demonstrating that interpretability does not compromise performance. While adaptable to any language model, we focus on masked protein language models due to their importance in drug discovery and the ability to validate our model's capabilities through real-world experiments and expert knowledge. We scale our CB-pLM from 24 million to 3 billion parameters, making them the largest Concept Bottleneck Models trained and the first capable of generative language modeling.

cs.LG↗

arXiv2024

JAMUN: Bridging Smoothed Molecular Dynamics and Score-Based Learning for Conformational Ensembles

Conformational ensembles of protein structures are immensely important both for understanding protein function and drug discovery in novel modalities such as cryptic pockets. Current techniques for sampling ensembles such as molecular dynamics (MD) are computationally inefficient, while many recent machine learning methods do not transfer to systems outside their training data. We propose JAMUN which performs MD in a smoothed, noised space of all-atom 3D conformations of molecules by utilizing the framework of walk-jump sampling. JAMUN enables ensemble generation for small peptides at rates of an order of magnitude faster than traditional molecular dynamics. The physical priors in JAMUN enables transferability to systems outside of its training data, even to peptides that are longer than those originally trained on. Our model, code and weights are available at https://github.com/prescient-design/jamun.

physics.bio-ph↗

arXiv2024

Protein Discovery with Discrete Walk-Jump Sampling

We resolve difficulties in training and sampling from a discrete generative model by learning a smoothed energy function, sampling from the smoothed data manifold with Langevin Markov chain Monte Carlo (MCMC), and projecting back to the true data manifold with one-step denoising. Our Discrete Walk-Jump Sampling formalism combines the contrastive divergence training of an energy-based model and improved sample quality of a score-based model, while simplifying training and sampling by requiring only a single noise level. We evaluate the robustness of our approach on generative modeling of antibody proteins and introduce the distributional conformity score to benchmark protein generative models. By optimizing and sampling from our models for the proposed distributional conformity score, 97-100% of generated samples are successfully expressed and purified and 70% of functional designs show equal or improved binding affinity compared to known functional antibodies on the first attempt in a single round of laboratory experiments. We also report the first demonstration of long-run fast-mixing MCMC chains where diverse antibody protein classes are visited in a single MCMC chain.

q-bio.BM↗

arXiv2024

3D molecule generation by denoising voxel grids

We propose a new score-based approach to generate 3D molecules represented as atomic densities on regular grids. First, we train a denoising neural network that learns to map from a smooth distribution of noisy molecules to the distribution of real molecules. Then, we follow the neural empirical Bayes framework (Saremi and Hyvarinen, 19) and generate molecules in two steps: (i) sample noisy density grids from a smooth distribution via underdamped Langevin Markov chain Monte Carlo, and (ii) recover the "clean" molecule by denoising the noisy grid with a single step. Our method, VoxMol, generates molecules in a fundamentally different way than the current state of the art (ie, diffusion models applied to atom point clouds). It differs in terms of the data representation, the noise model, the network architecture and the generative modeling algorithm. Our experiments show that VoxMol captures the distribution of drug-like molecules better than state of the art, while being faster to generate samples.

cs.LG↗

arXiv2023

Inchworm quasi Monte Carlo for quantum impurities

The inchworm expansion is a promising approach to solving strongly correlated quantum impurity models due to its reduction of the sign problem in real and imaginary time. However, inchworm Monte Carlo is computationally expensive, converging as $1/\sqrt{N}$ where $N$ is the number of samples. We show that the imaginary time integration is amenable to quasi Monte Carlo, with enhanced $1/N$ convergence, by mapping the Sobol low-discrepancy sequence from the hypercube to the simplex with the so-called Root transform. This extends the applicability of the inchworm method to, e.g., multi-orbital Anderson impurity models with off-diagonal hybridization, relevant for materials simulation, where continuous time hybridization expansion Monte Carlo has a severe sign problem.

cond-mat.str-el↗

arXiv2021

The Kondo Cloud in a 1D Nanowire

A recent experiment [Nature 579, 210--213 (2020)] probed the extent of the Kondo cloud in 1D by measuring the effect of electrostatic perturbations applied a distance $L$ away from the impurity on $T_K$. We study the Kondo cloud in a model proposed to describe this experimental setup, consisting of a single impurity Anderson model coupled to two semi-infinite 1D leads. In agreement with the experimental results, we find that $T_K$ is strongly affected by perturbations to the lead within the Kondo cloud. We obtain a complementary picture of the Kondo cloud in this system by observing how the Kondo state manifests itself in the local density of states of the leads, which may be observed experimentally via scanning tunneling microscopy. Our results support the existing experimental data and provide detailed predictions for future experiments seeking to characterize the Kondo cloud in this system.

cond-mat.str-el↗

arXiv2020

Dynamic Control of Nonequilibrium Metal-Insulator Transitions

We demonstrate a first order metal-insulator phase transition in the repulsive, fully frustrated, single-band Hubbard model as a function of the coupling to a fermion bath. Time dependent manipulation of the bath coupling allows switching between metallic and insulating states both across the phase transition and within the coexistence region. We propose a simple nanoelectronic device for experimentally realizing dynamic control of the bath coupling. Analysis of the device characteristics shows that it can act as a two-terminal memristor.

cond-mat.str-el↗

arXiv2019

Dynamics of Kondo voltage splitting after a quantum quench

We analyze the time-dependent formation of the spectral function of an Anderson impurity model in the Kondo regime within a numerically exact real-time quantum Monte Carlo framework. At steady state, splitting of the Kondo peak occurs with nontrivial dependence on voltage and temperature, and with little effect on the location or intensity of high-energy features. Examining the transient development of the Kondo peak after a quench from an initially uncorrelated state reveals a two-stage process where the initial formation of a single central Kondo peak is followed by splitting. We analyze the time dependence of splitting in detail and demonstrate a strong dependence of its characteristic timescale on the voltage. We expect both the steady state and the transient phenomenon to be experimentally observable.

cond-mat.str-el↗

arXiv2018

Updated Core Libraries of the ALPS Project

The open source ALPS (Algorithms and Libraries for Physics Simulations) project provides a collection of physics libraries and applications, with a focus on simulations of lattice models and strongly correlated electron systems. The libraries provide a convenient set of well-documented and reusable components for developing condensed matter physics simulation codes, and the applications strive to make commonly used and proven computational algorithms available to a non-expert community. In this paper we present an update of the core ALPS libraries. We present in particular new Monte Carlo libraries and new Green's function libraries.

physics.comp-ph↗

arXiv2017

Quantum Monte Carlo solution of the dynamical mean field equations in real time

We present real-time inchworm quantum Monte Carlo results for single-site dynamical mean field theory on an infinite coordination number Bethe lattice. Our numerically exact results are obtained on the L-shaped Keldysh contour and, being evaluated in real-time, avoid the analytic continuation issues typically encountered in Monte Carlo calculations. Our results show that inchworm Monte Carlo methods have now reached a state where they can be used as dynamical mean field impurity solvers and the dynamical sign problem can be overcome. As non-equilibrium problems can be simulated at the same cost, we envisage the main use of these methods as dynamical mean field solvers for time-dependent problems far from equilibrium.

cond-mat.str-el↗

arXiv2016

Currents and Green's functions of impurities out of equilibrium -- results from inchworm Quantum Monte Carlo

We generalize the recently developed inchworm quantum Monte Carlo method to the full Keldysh contour with forward, backward, and equilibrium branches to describe the dynamics of strongly correlated impurity problems with time dependent parameters. We introduce a method to compute Green's functions, spectral functions, and currents for inchworm Monte Carlo and show how systematic error assessments in real time can be obtained. We then illustrate the capabilities of the algorithm with a study of the behavior of quantum impurities after an instantaneous voltage quench from a thermal equilibrium state.

cond-mat.str-el↗