arXiv subjects

Ali Backour

Publications and source records attributed to Ali Backour.

4 recordsLinked to original sources

arXiv2026

Encrypt What Matters: When Selective Homomorphic Inference Is Efficient

Fully homomorphic encryption (FHE) enables inference on private data without revealing it to the server, but evaluating an entire input under FHE is expensive. We study \emph{selective homomorphic inference}, where only a sensitive region of interest (ROI) is encrypted, and computations independent of that region are performed in plaintext. Selective evaluation produces the same output as full FHE on the same model, without retraining. Its efficiency depends on how quickly encrypted dependencies spread through the network. For small encrypted ROIs, locality-preserving architectures can achieve order-of-magnitude homomorphic-evaluation speedups, whereas architectures with early global mixing provide essentially no speedup. These results identify locality as the key architectural property governing the benefit of selective homomorphic inference.

cs.CR↗

arXiv2026

The Dynamics of Continuous Mixture Collapse in Language Models

LLMs latent-state reasoning methods replace discrete intermediate tokens with continuous states, such as weighted mixtures of token embeddings, to retain multiple possible reasoning directions rather than committing to one. Yet pretrained language models often fail to preserve these mixtures. We study why through a combination of theoretical analysis and controlled empirical investigations on a variety of models. We identify three independent, distinct sources of failure. First, transformer architectures already distort mixture geometry, and training substantially amplifies this effect. Moreover, the failure can occur even if the model transports mixtures perfectly linearly: the softmax readout and autoregressive feedback form a dynamical system that either amplifies small differences until one component of the mixture dominates or contracts different mixtures until they become indistinguishable. We verify this theoretical prediction empirically: the observed transition between contraction and amplification occurs near the theoretical threshold derived by our analysis, and pretrained-model rollouts lie predominantly on the amplifying side. Finally, we generalize to mixtures of many components and show that exact preservation generally requires context-dependent correction, whose required dimensionality can grow with the number of components.

cs.LG↗

arXiv2026

Turbulence teaches equivariance to neural networks

We show that the rotational nature of turbulence affects how neural networks learn mappings between quantities governed by the Navier-Stokes equations. We train super-resolution models at different wall-normal locations in a turbulent channel flow, where anisotropy varies naturally, and test their generalization to new coordinate frames, new anisotropy regimes, and a higher Reynolds number. Our findings inform both the design of equivariant machine learning models for turbulence and our understanding of how turbulence shapes what those models learn. First, mappings that better respect the rotational symmetries of the Navier-Stokes equations generalize better to new flows. Coordinate-frame generalization is therefore a key part of the broader generalization problem, since turbulent flows contain a wide range of local orientations. Second, turbulence itself partially teaches equivariance to learned mappings, an effect we call implicit data augmentation. The effect strengthens with dataset size and with isotropy, since a more isotropic dataset samples more orientations under which the Navier-Stokes equations are covariant. Implicit augmentation is also scale-dependent, with smaller scales exhibiting lower equivariance error. This scale-dependency is consistent with Kolmogorov's hypothesis of local isotropy. Third, enforcing equivariance as an architectural inductive bias is the limit of these effects: an exactly equivariant network outperforms unconstrained CNNs on all generalization tests, with roughly an order of magnitude fewer parameters. We expect these effects to apply broadly to learned mappings between tensorial flow quantities, making them relevant to most machine learning applications in turbulence.

physics.flu-dyn↗

arXiv2025

Implicit Augmentation from Distributional Symmetry in Turbulence Super-Resolution

The immense computational cost of simulating turbulence has motivated the use of machine learning approaches for super-resolving turbulent flows. A central challenge is ensuring that learned models respect physical symmetries, such as rotational equivariance. We show that standard convolutional neural networks (CNNs) can partially acquire this symmetry without explicit augmentation or specialized architectures, as turbulence itself provides implicit rotational augmentation in both time and space. Using 3D channel-flow subdomains with differing anisotropy, we find that models trained on more isotropic mid-plane data achieve lower equivariance error than those trained on boundary layer data, and that greater temporal or spatial sampling further reduces this error. We show a distinct scale-dependence of equivariance error that occurs regardless of dataset anisotropy that is consistent with Kolmogorov's local isotropy hypothesis. These results clarify when rotational symmetry must be explicitly incorporated into learning algorithms and when it can be obtained directly from turbulence, enabling more efficient and symmetry-aware super-resolution.

physics.flu-dyn↗