arXiv subjects

Leon Zhang

Publications and source records attributed to Leon Zhang.

11 recordsLinked to original sources

arXiv2025

Nanoscale Ultrafast Lattice Modulation with Hard X-ray Free Electron Laser

Understanding and controlling microscopic dynamics across spatial and temporal scales has driven major progress in science and technology over the past several decades. While ultrafast laser-based techniques have enabled probing nanoscale dynamics at their intrinsic temporal scales down to femto- and attoseconds, the long wavelengths of optical lasers have prevented the interrogation and manipulation of such dynamics with nanoscale spatial specificity. With advances in hard X-ray free electron lasers (FELs), significant progress has been made developing X-ray transient grating (XTG) spectroscopy, aiming at the coherent control of elementary excitations with nanoscale X-ray standing waves. So far, XTGs have been probed only at optical wavelengths, thus intrinsically limiting the achievable periodicities to several hundreds of nm. By achieving sub-femtosecond synchronization of two hard X-ray pulses at a controlled crossing angle, we demonstrate the generation of an XTG with spatial periods of 10 nm. The XTG excitation drives a thermal grating that drives coherent monochromatic longitudinal acoustic phonons in the cubic perovskite, SrTiO3 (STO). With a third X-ray pulse with the same photon energy, time-and-momentum resolved measurement of the XTG-induced scattering intensity modulation provides evidence of ballistic thermal transport at nanometer scale in STO. These results highlight the great potential of XTG for studying high-wave-vector excitations and nanoscale transport in condensed matter, and establish XTG as a powerful platform for the coherent control and study of nanoscale dynamics.

physics.optics↗

arXiv2024

Kernel Looping: Eliminating Synchronization Boundaries for Peak Inference Performance

Token generation speed is critical to power the next wave of AI inference applications. GPUs significantly underperform during token generation due to synchronization overheads at kernel boundaries, utilizing only 21% of their peak memory bandwidth. While recent dataflow architectures mitigate these overheads by enabling aggressive fusion of decoder layers into a single kernel, they too leave performance on the table due to synchronization penalties at layer boundaries. This paper presents kernel looping, a specialized global optimization technique which exploits an optimization opportunity brought by combining the unique layer-level fusion possible in modern dataflow architectures with the repeated layer structure found in language models. Kernel looping eliminates synchronization costs between consecutive calls to the same kernel by transforming these calls into a single call to a modified kernel containing a pipelined outer loop. We evaluate kernel looping on the SambaNova SN40L Reconfigurable Dataflow Unit (RDU), a commercial dataflow accelerator for AI. Experiments demonstrate that kernel looping speeds up the decode phase of a wide array of powerful open-source models by up to 2.2$\times$ on SN40L. Kernel looping allows scaling of decode performance over multiple SN40L sockets, achieving speedups of up to 2.5$\times$. Finally, kernel looping enables SN40L to achieve over 90% of peak performance on 8 and 16 sockets and achieve a speedup of up to 3.7$\times$ over DGX H100. Kernel looping, as well as the models evaluated in this paper, are deployed in production in a commercial AI inference cloud.

cs.CL↗

arXiv2024

SambaLingo: Teaching Large Language Models New Languages

Despite the widespread availability of LLMs, there remains a substantial gap in their capabilities and availability across diverse languages. One approach to address these issues has been to take an existing pre-trained LLM and continue to train it on new languages. While prior works have experimented with language adaptation, many questions around best practices and methodology have not been covered. In this paper, we present a comprehensive investigation into the adaptation of LLMs to new languages. Our study covers the key components in this process, including vocabulary extension, direct preference optimization and the data scarcity problem for human alignment in low-resource languages. We scale these experiments across 9 languages and 2 parameter scales (7B and 70B). We compare our models against Llama 2, Aya-101, XGLM, BLOOM and existing language experts, outperforming all prior published baselines. Additionally, all evaluation code and checkpoints are made public to facilitate future research.

cs.CL↗

arXiv2021

Sharp bounds for the number of regions of maxout networks and vertices of Minkowski sums

We present results on the number of linear regions of the functions that can be represented by artificial feedforward neural networks with maxout units. A rank-k maxout unit is a function computing the maximum of $k$ linear functions. For networks with a single layer of maxout units, the linear regions correspond to the upper vertices of a Minkowski sum of polytopes. We obtain face counting formulas in terms of the intersection posets of tropical hypersurfaces or the number of upper faces of partial Minkowski sums, along with explicit sharp upper bounds for the number of regions for any input dimension, any number of units, and any ranks, in the cases with and without biases. Based on these results we also obtain asymptotically sharp upper bounds for networks with multiple layers.

math.CO↗

arXiv2021

Tropical flag varieties

Flag matroids are combinatorial abstractions of flags of linear subspaces, just as matroids are of linear subspaces. We introduce the flag Dressian as a tropical analogue of the partial flag variety, and prove a correspondence between: (a) points on the flag Dressian, (b) valuated flag matroids, (c) flags of projective tropical linear spaces, and (d) coherent flag matroidal subdivisions. We introduce and characterize projective tropical linear spaces, which serve as a fundamental tool in our proof. We apply the correspondence to prove that all valuated flag matroids on ground set up to size 5 are realizable, and give an example where this fails for a flag matroid on 6 elements.

math.CO↗

arXiv2020

Multivariate volume, Ehrhart, and $h^*$-polynomials of polytropes

The univariate Ehrhart and $h^*$-polynomials of lattice polytopes have been widely studied. We describe methods from toric geometry for computing multivariate versions of volume, Ehrhart and $h^*$-polynomials of lattice polytropes, which are both tropically and classically convex. These algorithms are applied to all polytropes of dimensions 2,3 and 4, yielding a large class of integer polynomials. We give a complete combinatorial description of the coefficients of volume polynomials of 3-dimensional polytropes in terms of regular central subdivisions of the fundamental polytope. Finally, we provide a partial characterization of the analogous coefficients in dimension 4.

math.CO↗

arXiv2019

Tropical principal component analysis on the space of ultrametrics

In 2019, Yoshida et al. introduced a notion of tropical principal component analysis (PCA). The output is a tropical polytope with a fixed number of vertices that best fits the data. We here apply tropical PCA to dimension reduction and visualization of data sampled from the space of phylogenetic trees. Our main results are twofold: the existence of a tropical cell decomposition into regions of fixed tree topology and the development of a stochastic optimization method to estimate the tropical PCA using a Markov Chain Monte Carlo (MCMC) approach. This method performs well with simulation studies, and it is applied to three empirical datasets: Apicomplexa and African coelacanth genomes as well as sequences of hemagglutinin for influenza from New York.

math.CO↗

arXiv2019

Computing zero-dimensional tropical varieties via projections

We present an algorithm for computing zero-dimensional tropical varieties using projections. Our main tools are fast unimodular transforms of lexicographical Gröbner bases. We prove that our algorithm requires only a polynomial number of arithmetic operations if given a Gröbner basis, and we demonstrate that our implementation compares favourably to other existing implementations. Applying it to the computation of general positive-dimensional tropical varieties, we argue that the complexity for calculating tropical links is dominated by the complexity of the Gröbner walk.

math.AG↗

arXiv2018

Computing Convex Hulls in the Affine Building of SL_d

We describe an algorithm for computing the convex hull of a finite collection of points in the affine building of SL_d(K), for K a field with discrete valuation. These convex hulls describe the relations among a finite collection of invertible matrices over K. As a consequence, we bound the dimension of the tropical projective space needed to realize the convex hull as a tropical polytope.

math.CO↗

arXiv2018

Computing Unit Groups of Curves

The group of units modulo constants of an affine variety over an algebraically closed field is free abelian of finite rank. Computing this group is difficult but of fundamental importance in tropical geometry, where it is desirable to realize intrinsic tropicalizations. We present practical algorithms for computing unit groups of smooth curves of low genus. Our approach is rooted in divisor theory, based on interpolation in the case of rational curves and on methods from algebraic number theory in the case of elliptic curves.

math.AG↗

arXiv2017

Tropical Principal Component Analysis and its Application to Phylogenetics

Principal component analysis is a widely-used method for the dimensionality reduction of a given data set in a high-dimensional Euclidean space. Here we define and analyze two analogues of principal component analysis in the setting of tropical geometry. In one approach, we study the Stiefel tropical linear space of fixed dimension closest to the data points in the tropical projective torus; in the other approach, we consider the tropical polytope with a fixed number of vertices closest to the data points. We then give approximative algorithms for both approaches and apply them to phylogenetics, testing the methods on simulated phylogenetic data and on an empirical dataset of Apicomplexa genomes.

math.CO↗