arXiv subjects

Stephen Zhang

Publications and source records attributed to Stephen Zhang.

16 recordsLinked to original sources

arXiv2026

Crystal structure and basic properties of dirhenate quantum materials

The anhydrous divalent 3d-metal dirhenate quantum materials, M(ReO$_4$)$_2$,were synthesized using solid-state methods for M = Mn through Zn. Previously unreported Mg(ReO4)2 is also described. Their layered crystal structures, which feature an in-plane triangular lattice of M$^{2+}$, were refined using single crystal X-ray diffraction, and their optical absorption and several other physical properties were characterized. Their magnetism and heat capacity reveal long-range magnetic order at low temperatures in many of the M(ReO$_4$)$_2$ phases. Notably, many of these ordered states are sensitive to applied magnetic fields and can be readily suppressed by relatively small fields, suggesting competing magnetic interactions in a low-dimensional framework, which appear worthy of further study.

cond-mat.mtrl-sci↗

arXiv2026

Chirality Transfer to the Magnetic Sublattice in the Hybrid Perovskite (R)-/(S)-3-Fluoropyrrolidinium Copper(II) Chloride

Incorporating chiral organic cations into organic-inorganic hybrid materials has been shown to enable the inorganic sublattice to display chiroptical properties. We report a new two-dimensional magnetic ($S=1/2$) chiral metal halide perovskite, (R)- and (S)-$(C_4H_9FN)_2CuCl_4$ (where $(C_4H_9FN)^+$ is 3-fluoropyrrolidinium), which consists of Cu-Cl inorganic layers separated by $(C_4H_9FN)^+$ organic cations. The presence of the chiral $(C_4H_9FN)^+$ organic cation induces formation of chiral magnetic order, even though the inorganic sublattice itself is nearly structurally centrosymmetric. We also report the racemic variant, containing an equal amount of (R)- and (S)- cations, which shows no evidence of chiral magnetic order. When the magnetic susceptibility is measured perpendicular to inorganic Cu-Cl layer propagation direction, an antiferromagnetic phase transition at N\'eel temperature $T_N = 2.23~K$ is observed in both the chiral and racemic materials, and the existence of the magnetic phase transition is supported by specific heat capacity measurements. Field-induced magnetic chirality is observed through the existence of a second-order magnetoelectric effect in the chiral variant, while no magnetoelectric signal is observed for the racemic material, indicating the absence of magnetic chirality. Our findings demonstrate that materials exhibiting chiral magnetic order can be created through the incorporation of a chiral cation into an organic-inorganic hybrid magnetic material, potentially allowing for the design of tailored materials that combine chiral magnetism with other desirable optical and electronic properties that come from structural chirality.

cond-mat.mtrl-sci↗

arXiv2025

Suppression of Intertwined Density Waves in La$_4$Ni$_{3-x}$Cu$_x$O$_{10+\delta}$

Superconductivity in La$_{4}$Ni$_{3}$O$_{10}$ has been reported to emerge upon suppression of intertwined spin and charge density wave (SDW/CDW) order, suggesting a possible connection to the pairing mechanism. Here we report a systematic investigation of La$_{4}$Ni$_{3-x}$Cu$_{x}$O$_{10+\delta}$ ($0 \leq x \leq 0.7$), focusing on the evolution of the SDW/CDW order as a function of chemical substitution. Temperature-dependent resistivity, magnetic susceptibility, and Hall effect measurements reveal a linear suppression of density wave transition temperature $T{\text{dw}}$ and a concurrent enhancement of hole concentration with increasing Cu content. At higher substitution levels ($x > 0.15$), the transition-induced anomaly in the resistivity becomes undetectable while a magnetic signature persists, indicating a partial decoupling of spin and charge components and the possible survival of short-range spin correlations. The absence of superconductivity across the substitution series highlights the importance of additional factors in stabilizing the superconducting state in pressurized La$_{4}$Ni$_{3}$O$_{10}$.

cond-mat.str-el↗

arXiv2025

On Fitting Flow Models with Large Sinkhorn Couplings

Flow models transform data gradually from one modality (e.g. noise) onto another (e.g. images). Such models are parameterized by a time-dependent velocity field, trained to fit segments connecting pairs of source and target points. When the pairing between source and target points is given, training flow models boils down to a supervised regression problem. When no such pairing exists, as is the case when generating data from noise, training flows is much harder. A popular approach lies in picking source and target points independently. This can, however, lead to velocity fields that are slow to train, but also costly to integrate at inference time. In theory, one would greatly benefit from training flow models by sampling pairs from an optimal transport (OT) measure coupling source and target, since this would lead to a highly efficient flow solving the Benamou and Brenier dynamical OT problem. In practice, recent works have proposed to sample mini-batches of $n$ source and $n$ target points and reorder them using an OT solver to form better pairs. These works have advocated using batches of size $n\approx 256$, and considered OT solvers that return couplings that are either sharp (using e.g. the Hungarian algorithm) or blurred (using e.g. entropic regularization, a.k.a. Sinkhorn). We follow in the footsteps of these works by exploring the benefits of increasing $n$ by three to four orders of magnitude, and look more carefully on the effect of the entropic regularization $\varepsilon$ used in the Sinkhorn algorithm. Our analysis is facilitated by new scale invariant quantities to report the sharpness of a coupling, while our sharded computations across multiple GPU or GPU nodes allow scaling up $n$. We show that in both synthetic and image generation tasks, flow models greatly benefit when fitted with large Sinkhorn couplings, with a low entropic regularization $\varepsilon$.

cs.LG↗

arXiv2025

Inferring stochastic dynamics with growth from cross-sectional data

Time-resolved single-cell omics data offers high-throughput, genome-wide measurements of cellular states, which are instrumental to reverse-engineer the processes underpinning cell fate. Such technologies are inherently destructive, allowing only cross-sectional measurements of the underlying stochastic dynamical system. Furthermore, cells may divide or die in addition to changing their molecular state. Collectively these present a major challenge to inferring realistic biophysical models. We present a novel approach, unbalanced probability flow inference, that addresses this challenge for biological processes modelled as stochastic dynamics with growth. By leveraging a Lagrangian formulation of the Fokker-Planck equation, our method accurately disentangles drift from intrinsic noise and growth. We showcase the applicability of our approach through evaluation on a range of simulated and real single-cell RNA-seq datasets. Comparing to several existing methods, we find our method achieves higher accuracy while enjoying a simple two-step training scheme.

cs.LG↗

arXiv2025

Attention Sinks: A 'Catch, Tag, Release' Mechanism for Embeddings

Large language models (LLMs) often concentrate their attention on a few specific tokens referred to as attention sinks. Common examples include the first token, a prompt-independent sink, and punctuation tokens, which are prompt-dependent. While the tokens causing the sinks often lack direct semantic meaning, the presence of the sinks is critical for model performance, particularly under model compression and KV-caching. Despite their ubiquity, the function, semantic role, and origin of attention sinks -- especially those beyond the first token -- remain poorly understood. In this work, we conduct a comprehensive investigation demonstrating that attention sinks: catch a sequence of tokens, tag them using a common direction in embedding space, and release them back into the residual stream, where tokens are later retrieved based on the tags they have acquired. Probing experiments reveal these tags carry semantically meaningful information, such as the truth of a statement. These findings extend to reasoning models, where the mechanism spans more heads and explains greater variance in embeddings, or recent models with query-key normalization, where sinks remain just as prevalent. To encourage future theoretical analysis, we introduce a minimal problem which can be solved through the 'catch, tag, release' mechanism, and where it emerges through training.

cs.CL↗

arXiv2024

Identifying Drift, Diffusion, and Causal Structure from Temporal Snapshots

Stochastic differential equations (SDEs) are a fundamental tool for modelling dynamic processes, including gene regulatory networks (GRNs), contaminant transport, financial markets, and image generation. However, learning the underlying SDE from data is a challenging task, especially if individual trajectories are not observable. Motivated by burgeoning research in single-cell datasets, we present the first comprehensive approach for jointly identifying the drift and diffusion of an SDE from its temporal marginals. Assuming linear drift and additive diffusion, we show that non-identifiability can only arise if the initial distribution possesses generalized rotational symmetries. We further prove that even if this condition holds, the drift and diffusion can almost always be recovered from the marginals. Additionally, we show that the causal graph of any SDE with additive diffusion can be recovered from the identified SDE parameters. To complement this theory, we adapt entropy-regularized optimal transport to handle anisotropic diffusion, and introduce APPEX (Alternating Projection Parameter Estimation from $X_0$), an iterative algorithm designed to estimate the drift, diffusion, and causal graph of an additive noise SDE, solely from temporal marginals. We show that APPEX iteratively decreases Kullback-Leibler divergence to the true solution, and demonstrate its effectiveness on simulated data from linear additive noise SDEs.

stat.ML↗

arXiv2024

Pressure-Induced Phase Transitions in Bilayer La$_3$Ni$_2$O$_7$

La$_3$Ni$_2$O$_7$ exists in two polymorphs: an unconventional structure with alternating layers of single- and triple-layered nickel-oxygen octahedra, and a classical double-layered Ruddlesden-Popper phase. In this study, we report the growth of single crystals of classical double-layered La$_3$Ni$_2$O$_7$ using the floating zone method. Structural characterization under pressures up to 15.4 GPa reveals a gradual transition from orthorhombic to tetragonal symmetry near 12 GPa. Additionally, we present pressure and field-dependent electrical resistance measurements under pressures as high as 27.4 GPa, from which we construct a phase diagram.

cond-mat.mtrl-sci↗

arXiv2024

OATS: Outlier-Aware Pruning Through Sparse and Low Rank Decomposition

The recent paradigm shift to large-scale foundation models has brought about a new era for deep learning that, while has found great success in practice, has also been plagued by prohibitively expensive costs in terms of high memory consumption and compute. To mitigate these issues, there has been a concerted effort in post-hoc neural network pruning techniques that do not require costly retraining. Despite the considerable progress being made, existing methods often exhibit a steady drop in model performance as the compression increases. In this paper, we present a novel approach to compressing large transformers, coined OATS, that utilizes the second moment information in the input embeddings to decompose the model weights into a sum of sparse and low-rank matrices. Without any retraining, OATS achieves state-of-the-art performance when compressing models by up to $60\%$ on large language models such as Llama-3 and Phi-3 and vision transformers such as ViT and DINOv2 while delivering up to $1.37\times$ the CPU acceleration versus a model that was comparably pruned.

cs.LG↗

arXiv2024

Sparsest Models Elude Pruning: An Expos\'e of Pruning's Current Capabilities

Pruning has emerged as a promising approach for compressing large-scale models, yet its effectiveness in recovering the sparsest of models has not yet been explored. We conducted an extensive series of 485,838 experiments, applying a range of state-of-the-art pruning algorithms to a synthetic dataset we created, named the Cubist Spiral. Our findings reveal a significant gap in performance compared to ideal sparse networks, which we identified through a novel combinatorial search algorithm. We attribute this performance gap to current pruning algorithms' poor behaviour under overparameterization, their tendency to induce disconnected paths throughout the network, and their propensity to get stuck at suboptimal solutions, even when given the optimal width and initialization. This gap is concerning, given the simplicity of the network architectures and datasets used in our study. We hope that our research encourages further investigation into new pruning techniques that strive for true network sparsity.

cs.LG↗

arXiv2023

Manifold Learning with Sparse Regularised Optimal Transport

Manifold learning is a central task in modern statistics and data science. Many datasets (cells, documents, images, molecules) can be represented as point clouds embedded in a high dimensional ambient space, however the degrees of freedom intrinsic to the data are usually far fewer than the number of ambient dimensions. The task of detecting a latent manifold along which the data are embedded is a prerequisite for a wide family of downstream analyses. Real-world datasets are subject to noisy observations and sampling, so that distilling information about the underlying manifold is a major challenge. We propose a method for manifold learning that utilises a symmetric version of optimal transport with a quadratic regularisation that constructs a sparse and adaptive affinity matrix, that can be interpreted as a generalisation of the bistochastic kernel normalisation. We prove that the resulting kernel is consistent with a Laplace-type operator in the continuous limit, establish robustness to heteroskedastic noise and exhibit these results in numerical experiments. We identify a highly efficient computational scheme for computing this optimal transport for discrete data and demonstrate that it outperforms competing methods in a set of examples.

stat.ML↗

arXiv2022

Quadratically Regularized Optimal Transport: nearly optimal potentials and convergence of discrete Laplace operators

We consider the conjecture proposed in Matsumoto, Zhang and Schiebinger (2022) suggesting that optimal transport with quadratic regularisation can be used to construct a graph whose discrete Laplace operator converges to the Laplace--Beltrami operator. We derive first order optimal potentials for the problem under consideration and find that the resulting solutions exhibit a surprising resemblance to the well-known Barenblatt--Prattle solution of the porous medium equation. Then, relying on these first order optimal potentials, we derive the pointwise $L^2$-limit of such discrete operators built from an i.i.d. random sample on a smooth compact manifold. Simulation results complementing the limiting distribution results are also presented.

math.AP↗

arXiv2022

Beyond kNN: Adaptive, Sparse Neighborhood Graphs via Optimal Transport

Nearest neighbour graphs are widely used to capture the geometry or topology of a dataset. One of the most common strategies to construct such a graph is based on selecting a fixed number k of nearest neighbours (kNN) for each point. However, the kNN heuristic may become inappropriate when sampling density or noise level varies across datasets. Strategies that try to get around this typically introduce additional parameters that need to be tuned. We propose a simple approach to construct an adaptive neighbourhood graph from a single parameter, based on quadratically regularised optimal transport. Our numerical experiments show that graphs constructed in this manner perform favourably in unsupervised and semi-supervised learning applications.

stat.ML↗

arXiv2022

Trajectory Inference via Mean-field Langevin in Path Space

Trajectory inference aims at recovering the dynamics of a population from snapshots of its temporal marginals. To solve this task, a min-entropy estimator relative to the Wiener measure in path space was introduced by Lavenant et al. arXiv:2102.09204, and shown to consistently recover the dynamics of a large class of drift-diffusion processes from the solution of an infinite dimensional convex optimization problem. In this paper, we introduce a grid-free algorithm to compute this estimator. Our method consists in a family of point clouds (one per snapshot) coupled via Schr\"odinger bridges which evolve with noisy gradient descent. We study the mean-field limit of the dynamics and prove its global convergence to the desired estimator. Overall, this leads to an inference method with end-to-end theoretical guarantees that solves an interpretable model for trajectory inference. We also present how to adapt the method to deal with mass variations, a useful extension when dealing with single cell RNA-sequencing data where cells can branch and die.

math.OC↗

arXiv2021

Towards a mathematical theory of trajectory inference

We devise a theoretical framework and a numerical method to infer trajectories of a stochastic process from samples of its temporal marginals. This problem arises in the analysis of single cell RNA-sequencing data, which provide high dimensional measurements of cell states but cannot track the trajectories of the cells over time. We prove that for a class of stochastic processes it is possible to recover the ground truth trajectories from limited samples of the temporal marginals at each time-point, and provide an efficient algorithm to do so in practice. The method we develop, Global Waddington-OT (gWOT), boils down to a smooth convex optimization problem posed globally over all time-points involving entropy-regularized optimal transport. We demonstrate that this problem can be solved efficiently in practice and yields good reconstructions, as we show on several synthetic and real datasets.

stat.ML↗

arXiv2019

Persistent exclusion processes: inertia, drift, mixing and correlation

In many biological systems, motile agents exhibit random motion with short-term directional persistence, together with crowding effects arising from spatial exclusion. We formulate and study a class of lattice-based models for multiple walkers with motion persistence and spatial exclusion in one and two dimensions, and use a mean-field approximation to investigate relevant population-level partial differential equations in the continuum limit. We show that this model of a persistent exclusion process is in general well described by a nonlinear diffusion equation. With reference to results presented in the current literature, our results reveal that the nonlinearity arises from the combination of motion persistence and volume exclusion, with linearity in terms of the canonical diffusion or heat equation being recovered in either the case of persistence without spatial exclusion, or spatial exclusion without persistence. We generalise our results to include systems of multiple species of interacting, motion-persistent walkers, as well as to incorporate a global drift in addition to persistence. These models are shown to be governed approximately by systems of nonlinear advection-diffusion equations. By comparing the prediction of the mean-field approximation to stochastic simulation results, we assess the performance of our results. Finally, we also address the problem of inferring the presence of persistence from simulation results, with a view to application to experimental cell-imaging data.

physics.bio-ph↗