arXiv subjects

Xiaoyu He

Publications and source records attributed to Xiaoyu He.

At least 37 records · Page 2Linked to original sources

arXiv2026

BAPS: A Fine-Grained Low-Precision Scheme for Softmax in Attention via Block-Aware Precision reScaling

As the performance gains from accelerating quantized matrix multiplication plateau, the softmax operation becomes the critical bottleneck in Transformer inference. This bottleneck stems from two hardware limitations: (1) limited data bandwidth between matrix and vector compute cores, and (2) the significant area cost of high-precision (FP32/16) exponentiation units (EXP2). To address these issues, we introduce a novel low-precision workflow that employs a specific 8-bit floating-point format (HiF8) and block-aware precision rescaling for softmax. Crucially, our algorithmic innovations make low-precision softmax feasible without the significant model accuracy loss that hampers direct low-precision approaches. Specifically, our design (i) halves the required data movement bandwidth by enabling matrix multiplication outputs constrained to 8-bit, and (ii) substantially reduces the EXP2 unit area by computing exponentiations in low (8-bit) precision. Extensive evaluation on language models and multi-modal models confirms the validity of our method. By alleviating the vector computation bottleneck, our work paves the way for doubling end-to-end inference throughput without increasing chip area, and offers a concrete co-design path for future low-precision hardware and software.

cs.LG↗

arXiv2025

A question of Erdős and Graham on Egyptian fractions

Answering a question of Erdős and Graham, we show that for each fixed positive rational number $x$ the number of ways to write $x$ as a sum of reciprocals of distinct positive integers each at most $n$ is $2^{(c_x + o(1))n}$ for an explicit constant $c_x$ increasing with $x$.

math.CO↗

arXiv2025

Alada: Alternating Adaptation of Momentum Method for Memory-Efficient Matrix Optimization

This work proposes Alada, an adaptive momentum method for stochastic optimization over large-scale matrices. Alada employs a rank-one factorization approach to estimate the second moment of gradients, where factors are updated alternatively to minimize the estimation error. Alada achieves sublinear memory overheads and can be readily extended to optimizing tensor-shaped variables.We also equip Alada with a first moment estimation rule, which enhances the algorithm's robustness without incurring additional memory overheads. The theoretical performance of Alada aligns with that of traditional methods such as Adam. Numerical studies conducted on several natural language processing tasks demonstrate the reduction in memory overheads and the robustness in training large models relative to Adam and its variants.

cs.LG↗

arXiv2025

New Sidorenko-type inequalities in tournaments

As a directed analog of Sidorenko's conjecture in extremal graph theory, Fox, Himwich, Zhou, and the second author defined an oriented graph $H$ to be tournament Sidorenko (anti-Sidorenko) if the random tournament asymptotically minimizes (maximizes) the number of copies of $H$ among all tournaments. We prove new inequalities of this form for oriented trees and cycles, considering both local and global notions of the Sidorenko property. We make progress on a conjecture of the aforementioned authors that every tree has an anti-Sidorenko direction, and give a characterization of short paths. For long paths we show that orientations are split symmetrically between being locally Sidorenko and anti-Sidorenko, yet almost all orientations are not globally Sidorenko. Finally, we give algorithms characterizing the local Sidorenko status of paths and cycles when the number of vertices is not divisible by four.

math.CO↗

arXiv2025

Asymptotically half of binary words are shuffle squares

A binary shuffle square is a binary word of even length that can be partitioned into two disjoint, identical subwords. Huang, Nam, Thaper, and the first author conjectured that as $n\rightarrow \infty$, asymptotically half of all binary words of length $2n$ are shuffle squares. We prove this conjecture in a strong form, by showing that the number of binary shuffle squares of length $2n$ is $(\frac{1}{2} - o(n^{-1/15})) 2^{2n}$.

math.CO↗

arXiv2025

SWiT-4D: Sliding-Window Transformer for Lossless and Parameter-Free Temporal 4D Generation

Despite significant progress in 4D content generation, the conversion of monocular videos into high-quality animated 3D assets with explicit 4D meshes remains considerably challenging. The scarcity of large-scale, naturally captured 4D mesh datasets further limits the ability to train generalizable video-to-4D models from scratch in a purely data-driven manner. Meanwhile, advances in image-to-3D generation, supported by extensive datasets, offer powerful prior models that can be leveraged. To better utilize these priors while minimizing reliance on 4D supervision, we introduce SWiT-4D, a Sliding-Window Transformer for lossless, parameter-free temporal 4D mesh generation. SWiT-4D integrates seamlessly with any Diffusion Transformer (DiT)-based image-to-3D generator, adding spatial-temporal modeling across video frames while preserving the original single-image forward process, enabling 4D mesh reconstruction from videos of arbitrary length. To recover global translation, we further introduce an optimization-based trajectory module tailored for static-camera monocular videos. SWiT-4D demonstrates strong data efficiency: with only a single short (<10s) video for fine-tuning, it achieves high-fidelity geometry and stable temporal consistency, indicating practical deployability under extremely limited 4D supervision. Comprehensive experiments on both in-domain zoo-test sets and challenging out-of-domain benchmarks (C4D, Objaverse, and in-the-wild videos) show that SWiT-4D consistently outperforms existing baselines in temporal smoothness. Project page: https://animotionlab.github.io/SWIT4D/

cs.CV↗

arXiv2025

Ramsey numbers of grid graphs

Let the grid graph $G_{M\times N}$ denote the Cartesian product $K_M \square K_N$. For a fixed subgraph $H$ of a grid, we study the off-diagonal Ramsey number $\operatorname{gr}(H, K_k)$, which is the smallest $N$ such that any red/blue edge coloring of $G_{N\times N}$ contains either a red copy of $H$ (a copy must preserve each edge's horizontal/vertical orientation), or a blue copy of $K_k$ contained inside a single row or column. Conlon, Fox, Mubayi, Suk, Verstraëte, and the first author recently showed that such grid Ramsey numbers are closely related to off-diagonal Ramsey numbers of bipartite $3$-uniform hypergraphs, and proved that $2^{Ω(\log ^2 k)} \le \operatorname{gr}(G_{2\times 2}, K_k) \le 2^{O(k^{2/3}\log k)}$. We prove that the square $G_{2\times 2}$ is exceptional in this regard, by showing that $\operatorname{gr}(C,K_k) = k^{O_C(1)}$ for any cycle $C \ne G_{2\times 2}$. We also obtain that a larger class of grid subgraphs $H$ obtained via a recursive blowup procedure satisfies $\operatorname{gr}(H,K_k) = k^{O_H(1)}$. Finally, we show that conditional on the multicolor Erdős-Hajnal conjecture, $\operatorname{gr}(H,K_k) = k^{O_H(1)}$ for any $H$ with two rows that does not contain $G_{2\times 2}$.

math.CO↗

arXiv2025

When are off-diagonal hypergraph Ramsey numbers polynomial?

A natural open problem in Ramsey theory is to determine those $3$-graphs $H$ for which the off-diagonal Ramsey number $r(H, K_n^{(3)})$ grows polynomially with $n$. We make substantial progress on this question by showing that if $H$ is tightly connected or has at most two tight components, then $r(H, K_n^{(3)})$ grows polynomially if and only if $H$ is contained in an iterated blowup of an edge.

math.CO↗

arXiv2025

Joint Problems in Learning Multiple Dynamical Systems

Clustering of time series is a well-studied problem, with applications ranging from quantitative, personalized models of metabolism obtained from metabolite concentrations to state discrimination in quantum information theory. We consider a variant, where given a set of trajectories and a number of parts, we jointly partition the set of trajectories and learn linear dynamical system (LDS) models for each part, so as to minimize the maximum error across all the models. We present globally convergent methods and EM heuristics, accompanied by promising computational results. The key highlight of this method is that it does not require a predefined hidden state dimension but instead provides an upper bound. Additionally, it offers guidance for determining regularization in the system identification.

math.OC↗

arXiv2025

Polynomial-to-exponential transition in 3-uniform Ramsey numbers

Let $r_k(s, e; t)$ denote the smallest $N$ such that any red/blue edge coloring of the complete $k$-uniform hypergraph on $N$ vertices contains either $e$ red edges among some $s$ vertices, or a blue clique of size $t$. Erd\H os and Hajnal introduced the study of this Ramsey number in 1972 and conjectured that for fixed $s>k\geq 3$, there is a well defined value $h_k(s)$ such that $r_k(s, h_k(s)-1; t)$ is polynomial in $t$, while $r_k(s, h_k(s); t)$ is exponential in a power of $t$. Erd\H os later offered \$500 for a proof. Conlon, Fox, and Sudakov proved the conjecture for $k=3$ and $3$-adically special values of $s$, and Mubayi and Razborov proved it for $s > k \geq 4$. We prove the conjecture for $k=3$ and all $s$, settling all remaining cases of the problem. We do this by solving a novel Turán-type problem: what is the maximum number of edges in an $n$-vertex $3$-uniform hypergraph in which all tight components are tripartite? We show that the balanced iterated blowup of an edge is an exact extremizer for this problem for all $n$.

math.CO↗

arXiv2025

Generative Retrieval and Alignment Model: A New Paradigm for E-commerce Retrieval

Traditional sparse and dense retrieval methods struggle to leverage general world knowledge and often fail to capture the nuanced features of queries and products. With the advent of large language models (LLMs), industrial search systems have started to employ LLMs to generate identifiers for product retrieval. Commonly used identifiers include (1) static/semantic IDs and (2) product term sets. The first approach requires creating a product ID system from scratch, missing out on the world knowledge embedded within LLMs. While the second approach leverages this general knowledge, the significant difference in word distribution between queries and products means that product-based identifiers often do not align well with user search queries, leading to missed product recalls. Furthermore, when queries contain numerous attributes, these algorithms generate a large number of identifiers, making it difficult to assess their quality, which results in low overall recall efficiency. To address these challenges, this paper introduces a novel e-commerce retrieval paradigm: the Generative Retrieval and Alignment Model (GRAM). GRAM employs joint training on text information from both queries and products to generate shared text identifier codes, effectively bridging the gap between queries and products. This approach not only enhances the connection between queries and products but also improves inference efficiency. The model uses a co-alignment strategy to generate codes optimized for maximizing retrieval efficiency. Additionally, it introduces a query-product scoring mechanism to compare product values across different codes, further boosting retrieval efficiency. Extensive offline and online A/B testing demonstrates that GRAM significantly outperforms traditional models and the latest generative retrieval models, confirming its effectiveness and practicality.

cs.IR↗

arXiv2025

Off-Diagonal Ramsey Numbers for Linear Hypergraphs

We study off-diagonal Ramsey numbers $r(H, K_n^{(k)})$ of $k$-uniform hypergraphs, where $H$ is a fixed linear $k$-uniform hypergraph and $K_n^{(k)}$ is complete on $n$ vertices. Recently, Conlon et al.\ disproved the folklore conjecture that $r(H, K_n^{(3)})$ always grows polynomially in $n$. In this paper we show that much larger growth rates are possible in higher uniformity. In uniformity $k\ge 4$, we prove that for any constant $C>0$, there exists a linear $k$-uniform hypergraph $H$ for which $$r(H,K_n^{(k)}) \geq \textup{twr}_{k-2}(2^{(\log n)^C}).$$

math.CO↗

arXiv2025

Generalized Erdős-Rogers problems for hypergraphs

Given $r$-uniform hypergraphs $G$ and $F$ and an integer $n$, let $f_{F,G}(n)$ be the maximum $m$ such that every $n$-vertex $G$-free $r$-graph has an $F$-free induced subgraph on $m$ vertices. We show that $f_{F,G}(n)$ is polynomial in $n$ when $G$ is a subgraph of an iterated blowup of $F$. As a partial converse, we show that if $G$ is not a subgraph of an $F$-iterated blowup and is $2$-tightly connected, then $f_{F,G}(n)$ is at most polylogarithmic in $n$. Our bounds generalize previous results of Dudek and Mubayi for the case when $F$ and $G$ are complete.

math.CO↗

arXiv2024

The Reverse Littlewood--Offord problem of Erdős

Let $ε_{1},\ldots,ε_{n}$ be a sequence of independent Rademacher random variables. We prove that there is a constant $c>0$ such that for any unit vectors $v_1,\ldots,v_n\in \mathbb{R}^2$, $$\Pr\left[||ε_1 v_1+\ldots+ε_n v_n||_2 \leq \sqrt{2}\right]\geq \frac{c}{n}.$$ This resolves the only remaining conjecture from the seminal paper of Erdős on the Littlewood--Offord problem, and it is sharp both in the sense that the constant $\sqrt{2}$ cannot be reduced and that the magnitude $n^{-1}$ is best possible. We also prove polynomial bounds for the analogous problem in higher dimensions.

math.PR↗

arXiv2024

Consistent Diffusion: Denoising Diffusion Model with Data-Consistent Training for Image Restoration

In this work, we address the limitations of denoising diffusion models (DDMs) in image restoration tasks, particularly the shape and color distortions that can compromise image quality. While DDMs have demonstrated a promising performance in many applications such as text-to-image synthesis, their effectiveness in image restoration is often hindered by shape and color distortions. We observe that these issues arise from inconsistencies between the training and testing data used by DDMs. Based on our observation, we propose a novel training method, named data-consistent training, which allows the DDMs to access images with accumulated errors during training, thereby ensuring the model to learn to correct these errors. Experimental results show that, across five image restoration tasks, our method has significant improvements over state-of-the-art methods while effectively minimizing distortions and preserving image fidelity.

cs.CV↗

arXiv2024

A Historical Trajectory Assisted Optimization Method for Zeroth-Order Federated Learning

Federated learning heavily relies on distributed gradient descent techniques. In the situation where gradient information is not available, the gradients need to be estimated from zeroth-order information, which typically involves computing finite-differences along isotropic random directions. This method suffers from high estimation errors, as the geometric features of the objective landscape may be overlooked during the isotropic sampling. In this work, we propose a non-isotropic sampling method to improve the gradient estimation procedure. Gradients in our method are estimated in a subspace spanned by historical trajectories of solutions, aiming to encourage the exploration of promising regions and hence improve the convergence. The proposed method uses a covariance matrix for sampling which is a convex combination of two parts. The first part is a thin projection matrix containing the basis of the subspace which is designed to improve the exploitation ability. The second part is the historical trajectories. We implement this method in zeroth-order federated settings, and show that the convergence rate aligns with existing ones while introducing no significant overheads in communication or local computation. The effectiveness of our proposal is verified on several numerical experiments in comparison to several commonly-used zeroth-order federated optimization algorithms.

cs.LG↗

arXiv2024

Cultural Adaptation of Menus: A Fine-Grained Approach

Machine Translation of Culture-Specific Items (CSIs) poses significant challenges. Recent work on CSI translation has shown some success using Large Language Models (LLMs) to adapt to different languages and cultures; however, a deeper analysis is needed to examine the benefits and pitfalls of each method. In this paper, we introduce the ChineseMenuCSI dataset, the largest for Chinese-English menu corpora, annotated with CSI vs Non-CSI labels and a fine-grained test set. We define three levels of CSI figurativeness for a more nuanced analysis and develop a novel methodology for automatic CSI identification, which outperforms GPT-based prompts in most categories. Importantly, we are the first to integrate human translation theories into LLM-driven translation processes, significantly improving translation accuracy, with COMET scores increasing by up to 7 points.

cs.CL↗

arXiv2024

Long-Range Vision-Based UAV-assisted Localization for Unmanned Surface Vehicles

The global positioning system (GPS) has become an indispensable navigation method for field operations with unmanned surface vehicles (USVs) in marine environments. However, GPS may not always be available outdoors because it is vulnerable to natural interference and malicious jamming attacks. Thus, an alternative navigation system is required when the use of GPS is restricted or prohibited. To this end, we present a novel method that utilizes an Unmanned Aerial Vehicle (UAV) to assist in localizing USVs in GNSS-restricted marine environments. In our approach, the UAV flies along the shoreline at a consistent altitude, continuously tracking and detecting the USV using a deep learning-based approach on camera images. Subsequently, triangulation techniques are applied to estimate the USV's position relative to the UAV, utilizing geometric information and datalink range from the UAV. We propose adjusting the UAV's camera angle based on the pixel error between the USV and the image center throughout the localization process to enhance accuracy. Additionally, visual measurements are integrated into an Extended Kalman Filter (EKF) for robust state estimation. To validate our proposed method, we utilize a USV equipped with onboard sensors and a UAV equipped with a camera. A heterogeneous robotic interface is established to facilitate communication between the USV and UAV. We demonstrate the efficacy of our approach through a series of experiments conducted during the ``Muhammad Bin Zayed International Robotic Challenge (MBZIRC-2024)'' in real marine environments, incorporating noisy measurements and ocean disturbances. The successful outcomes indicate the potential of our method to complement GPS for USV navigation.

cs.RO↗