arXiv subjects

Hongzhi Huang

Publications and source records attributed to Hongzhi Huang.

At least 19 recordsLinked to original sources

arXiv2026

Fundamental Groups in the Five Dimensional Pan-Rong Conjecture

Let $M$ be a complete open $5$-manifold with nonnegative Ricci curvature. If its universal cover $\tilde M$ has Euclidean volume growth, then $π_1(M)$ is finitely generated and virtually $\mathbb Z^k$ for some $0\le k\le4$. This confirms a conjecture of Pan and Rong \cite{PR18} in dimension five; see also \cite{BNS25,BrNaSe25}. In dimension six, under the same volume growth assumption, we also obtain a quantitative virtual abelianness result without assuming finite generation of the fundamental group.

math.DG↗

arXiv2026

Scaling Latent Reasoning via Looped Language Models

Modern LLMs are trained to "think" primarily via explicit text generation, such as chain-of-thought (CoT), which defers reasoning to post-training and under-leverages pre-training data. We present and open-source Ouro, named after the recursive Ouroboros, a family of pre-trained Looped Language Models (LoopLM) that instead build reasoning into the pre-training phase through (i) iterative computation in latent space, (ii) an entropy-regularized objective for learned depth allocation, and (iii) scaling to 7.7T tokens. Ouro 1.4B and 2.6B models enjoy superior performance that match the results of up to 12B SOTA LLMs across a wide range of benchmarks. Through controlled experiments, we show this advantage stems not from increased knowledge capacity, but from superior knowledge manipulation capabilities. We also show that LoopLM yields reasoning traces more aligned with final outputs than explicit CoT. We hope our results show the potential of LoopLM as a novel scaling direction in the reasoning era. Our model is available here: http://ouro-llm.github.io.

cs.CL↗

arXiv2026

Fibrations, the First Betti Number, and Almost Nonnegative Ricci Curvature

In this paper, we prove fibration theorems for manifolds with almost nonnegative Ricci curvature and certain extra regularity assumptions. We show that a closed $n$-manifold $M$ satisfying $\mathrm{diam}(M)^2\mathrm{sec}_M \geq -κ$ and $\mathrm{diam}(M)^2\mathrm{Ric}_M \geq -δ$, where $δ>0$ is sufficiently small depending only on $n$ and $κ$, fibers over a $b_1(M)$-torus. This removes the upper sectional curvature bound required in the earlier result of Yamaguchi \cite{Y88}. As a corollary, we obtain a refinement of Yamaguchi's smooth fibration theorem (\cite{Y91}), showing that the fiber itself (rather than a finite cover of it) fibers over a $b_1$-torus. Our results extend to manifolds satisfying a generalized Reifenberg condition introduced in \cite{HH24}, which encompasses both a lower bound on sectional curvature and the local rewinding Reifenberg condition. In the nonsmooth setting, a similar result also holds for a non-collapsed $\mathrm{RCD}(-ε(D,r,n),n)$ space whose diameter is bounded by $D$ and which satisfies the $(r,δ(n))$-local rewinding Reifenberg condition. The proofs rely on an equivariant regularity theorem for almost submetries under a lower Ricci curvature bound. In addition, we study the stability of rank of Abelian actions along equivariant Gromov-Hausdorff convergence in this paper.

math.DG↗

arXiv2025

Virtual Width Networks

We introduce Virtual Width Networks (VWN), a framework that delivers the benefits of wider representations without incurring the quadratic cost of increasing the hidden size. VWN decouples representational width from backbone width, expanding the embedding space while keeping backbone compute nearly constant. In our large-scale experiment, an 8-times expansion accelerates optimization by over 2 times for next-token and 3 times for next-2-token prediction. The advantage amplifies over training as both the loss gap grows and the convergence-speedup ratio increases, showing that VWN is not only token-efficient but also increasingly effective with scale. Moreover, we identify an approximately log-linear scaling relation between virtual width and loss reduction, offering an initial empirical basis and motivation for exploring virtual-width scaling as a new dimension of large-model efficiency.

cs.LG↗

arXiv2025

Splitting and Slow Volume Growth for Open Manifolds with Nonnegative Ricci Curvature

In \cite{NPZ24}, Navarro-Pan-Zhu proved that the fundamental group of an open manifold with nonnegative Ricci curvature and linear volume growth contains a subgroup isomorphic to $\mathbb{Z}^k$ with finite index. They further asked whether the existence of a torsion-free element in the fundamental group forces the universal cover to split off an isometric $\mathbb{R}$-factor (Question 1.3 of \cite{NPZ24}). In this article, we provide an affirmative answer to this question. Specifically, we prove that if an open manifold with nonnegative Ricci curvature has linear volume growth, then its universal cover is isometric to a metric product $\mathbb{R}^k \times N$, where $N$ is an open manifold with linear volume growth and $k$ is the integer such that $π_1(M)$ contains a $\mathbb{Z}^k$-subgroup of finite index. As a direct consequence, if the Ricci curvature is positive at some point, then the fundamental group is finite. We also establish that for an open manifold $M$ with nonnegative Ricci curvature, if the infimum of its volume growth order is strictly less than $3$ and $\tilde{M}$ has Euclidean volume growth, then the universal cover $\tilde{M}$ splits off an $\mathbb{R}^{n-2}$-factor. As an application, if $M$ has first Betti number $b_1 = n-2$ and $\tilde{M}$ has Euclidean volume growth, then its universal cover admits such a splitting. This result provides a partial answer to \cite[Question 1.6]{PY24}.

math.DG↗

arXiv2025

UltraMemV2: Memory Networks Scaling to 120B Parameters with Superior Long-Context Learning

While Mixture of Experts (MoE) models achieve remarkable efficiency by activating only subsets of parameters, they suffer from high memory access costs during inference. Memory-layer architectures offer an appealing alternative with very few memory access, but previous attempts like UltraMem have only matched the performance of 2-expert MoE models, falling significantly short of state-of-the-art 8-expert configurations. We present UltraMemV2, a redesigned memory-layer architecture that closes this performance gap. Our approach introduces five key improvements: integrating memory layers into every transformer block, simplifying value expansion with single linear projections, adopting FFN-based value processing from PEER, implementing principled parameter initialization, and rebalancing memory-to-FFN computation ratios. Through extensive evaluation, we demonstrate that UltraMemV2 achieves performance parity with 8-expert MoE models under same computation and parameters but significantly low memory access. Notably, UltraMemV2 shows superior performance on memory-intensive tasks, with improvements of +1.6 points on long-context memorization, +6.2 points on multi-round memorization, and +7.9 points on in-context learning. We validate our approach at scale with models up to 2.5B activated parameters from 120B total parameters, and establish that activation density has greater impact on performance than total sparse parameter count. Our work brings memory-layer architectures to performance parity with state-of-the-art MoE models, presenting a compelling alternative for efficient sparse computation.

cs.LG↗

arXiv2025

Over-Tokenized Transformer: Vocabulary is Generally Worth Scaling

Tokenization is a fundamental component of large language models (LLMs), yet its influence on model scaling and performance is not fully explored. In this paper, we introduce Over-Tokenized Transformers, a novel framework that decouples input and output vocabularies to improve language modeling performance. Specifically, our approach scales up input vocabularies to leverage multi-gram tokens. Through extensive experiments, we uncover a log-linear relationship between input vocabulary size and training loss, demonstrating that larger input vocabularies consistently enhance model performance, regardless of model size. Using a large input vocabulary, we achieve performance comparable to double-sized baselines with no additional cost. Our findings highlight the importance of tokenization in scaling laws and provide practical insight for tokenizer design, paving the way for more efficient and powerful LLMs.

cs.CL↗

arXiv2025

Seed1.5-Thinking: Advancing Superb Reasoning Models with Reinforcement Learning

We introduce Seed1.5-Thinking, capable of reasoning through thinking before responding, resulting in improved performance on a wide range of benchmarks. Seed1.5-Thinking achieves 86.7 on AIME 2024, 55.0 on Codeforces and 77.3 on GPQA, demonstrating excellent reasoning abilities in STEM and coding. Beyond reasoning tasks, the method demonstrates notable generalization across diverse domains. For instance, it surpasses DeepSeek R1 by 8% in win rate on non-reasoning tasks, indicating its broader applicability. Compared to other state-of-the-art reasoning models, Seed1.5-Thinking is a Mixture-of-Experts (MoE) model with a relatively small size, featuring 20B activated and 200B total parameters. As part of our effort to assess generalized reasoning, we develop two internal benchmarks, BeyondAIME and Codeforces, both of which will be publicly released to support future research. Model trial link: https://www.volcengine.com/experience/ark.

cs.CL↗

arXiv2025

BackMix: Regularizing Open Set Recognition by Removing Underlying Fore-Background Priors

Open set recognition (OSR) requires models to classify known samples while detecting unknown samples for real-world applications. Existing studies show impressive progress using unknown samples from auxiliary datasets to regularize OSR models, but they have proved to be sensitive to selecting such known outliers. In this paper, we discuss the aforementioned problem from a new perspective: Can we regularize OSR models without elaborately selecting auxiliary known outliers? We first empirically and theoretically explore the role of foregrounds and backgrounds in open set recognition and disclose that: 1) backgrounds that correlate with foregrounds would mislead the model and cause failures when encounters 'partially' known images; 2) Backgrounds unrelated to foregrounds can serve as auxiliary known outliers and provide regularization via global average pooling. Based on the above insights, we propose a new method, Background Mix (BackMix), that mixes the foreground of an image with different backgrounds to remove the underlying fore-background priors. Specifically, BackMix first estimates the foreground with class activation maps (CAMs), then randomly replaces image patches with backgrounds from other images to obtain mixed images for training. With backgrounds de-correlated from foregrounds, the open set recognition performance is significantly improved. The proposed method is quite simple to implement, requires no extra operation for inferences, and can be seamlessly integrated into almost all of the existing frameworks. The code is released on https://github.com/Vanixxz/BackMix.

cs.CV↗

arXiv2025

Hyper-Connections

We present hyper-connections, a simple yet effective method that can serve as an alternative to residual connections. This approach specifically addresses common drawbacks observed in residual connection variants, such as the seesaw effect between gradient vanishing and representation collapse. Theoretically, hyper-connections allow the network to adjust the strength of connections between features at different depths and dynamically rearrange layers. We conduct experiments focusing on the pre-training of large language models, including dense and sparse models, where hyper-connections show significant performance improvements over residual connections. Additional experiments conducted on vision tasks also demonstrate similar improvements. We anticipate that this method will be broadly applicable and beneficial across a wide range of AI problems.

cs.LG↗

arXiv2025

Frac-Connections: Fractional Extension of Hyper-Connections

Residual connections are central to modern deep learning architectures, enabling the training of very deep networks by mitigating gradient vanishing. Hyper-Connections recently generalized residual connections by introducing multiple connection strengths at different depths, thereby addressing the seesaw effect between gradient vanishing and representation collapse. However, Hyper-Connections increase memory access costs by expanding the width of hidden states. In this paper, we propose Frac-Connections, a novel approach that divides hidden states into multiple parts rather than expanding their width. Frac-Connections retain partial benefits of Hyper-Connections while reducing memory consumption. To validate their effectiveness, we conduct large-scale experiments on language tasks, with the largest being a 7B MoE model trained on up to 3T tokens, demonstrating that Frac-Connections significantly outperform residual connections.

cs.LG↗

arXiv2025

Ultra-Sparse Memory Network

It is widely acknowledged that the performance of Transformer models is logarithmically related to their number of parameters and computational complexity. While approaches like Mixture of Experts (MoE) decouple parameter count from computational complexity, they still face challenges in inference due to high memory access costs. This work introduces UltraMem, incorporating large-scale, ultra-sparse memory layer to address these limitations. Our approach significantly reduces inference latency while maintaining model performance. We also investigate the scaling laws of this new architecture, demonstrating that it not only exhibits favorable scaling properties but outperforms MoE. In experiments, the largest UltraMem we train has 20 million memory slots. The results show that our method achieves state-of-the-art inference speed and model performance within a given computational budget, paving the way for billions of slots or experts.

cs.LG↗

arXiv2025

Nonnegative Ricci Curvature, Euclidean Volume Growth, and the Fundamental Groups of Open $4$-Manifolds

Let $M$ be a 4-dimensional open manifold with nonnegative Ricci curvature. In this paper, we prove that if the universal cover of $M$ has Euclidean volume growth, then the fundamental group $π_1(M)$ is finitely generated. This result confirms Pan-Rong's conjecture \cite{PR18} for dimension $n = 4$. Additionally, we prove that there exists a universal constant $C>0$ such that $π_1(M)$ contains an abelian subgroup of index $\le C$. More specifically, if $π_1(M)$ is infinite, then $π_1(M)$ is a crystallographic group of rank $\le 3$. If $π_1(M)$ is finite, then $π_1(M)$ is isomorphic to a quotient of the fundamental group of a spherical 3-manifold.

math.DG↗

arXiv2024

A finite Topological Type Theorem for open manifolds with Non-negative Ricci Curvature and Almost Maximal Local Rewinding Volume

In this paper, we present finite topological type theorems for open manifolds with non-negative Ricci curvature, under almost maximal local rewinding volume. Unlike previous related research, our theorems remove the constraints of sectional curvature or conjugate radius, which were crucial additional assumptions on metric regularity in prior results. Notably, our settings do not necessarily satisfy a triangle comparison of Toponogov type. In fact, the method we adopt also extends to many previous related studies.

math.DG↗

arXiv2024

Almost splitting maps, transformation theorems and smooth fibration theorems

In this paper, we introduce a notion, called generalized Reifenberg condition, under which we prove a smooth fibration theorem for collapsed manifolds with Ricci curvature bounded below, which gives a unified proof of smooth fibration theorems in many previous works (including the ones proved by Fukaya and Yamaguchi respectively). A key tool in the proof of this fibration theorem is the transformation technique for almost splitting maps, which originates from Cheeger-Naber (\cite{CN}) and Cheeger-Jiang-Naber (\cite{CJN21}). More precisely, we show that a transformation theorem of Cheeger-Jiang-Naber (see Proposition 7.7 in \cite{CJN21}) holds for possibly collapsed manifolds. Some other applications of the transformation theorems are given in this paper.

math.DG↗

arXiv2024

Finite generation of fundamental groups for manifolds with nonnegative Ricci curvature whose universal cover is almost $k$-polar at infinity

In this article, we prove that the fundamental group $π_1(M)$ of a complete open manifold $M$ with nonnegative Ricci curvature is finitely generated, under the condition that the Riemannian universal cover $\tilde M$ satisfies an "almost $k$-polar at infinity" condition. Additionally, such $π_1(M)$ is virtually abelian. Furthermore, we demonstrate that the base point of any tangent cone at infinity of such a manifold is nearly a pole. In the case where $\tilde M$ exhibits almost maximal Euclidean volume growth, we prove that $M$ deformation retracts to a closed submanifold $F$ which is diffeomorphic to a flat manifold, provided $M$ is not simply connected.

math.DG↗

arXiv2022

Class-Specific Semantic Reconstruction for Open Set Recognition

Open set recognition enables deep neural networks (DNNs) to identify samples of unknown classes, while maintaining high classification accuracy on samples of known classes. Existing methods basing on auto-encoder (AE) and prototype learning show great potential in handling this challenging task. In this study, we propose a novel method, called Class-Specific Semantic Reconstruction (CSSR), that integrates the power of AE and prototype learning. Specifically, CSSR replaces prototype points with manifolds represented by class-specific AEs. Unlike conventional prototype-based methods, CSSR models each known class on an individual AE manifold, and measures class belongingness through AE's reconstruction error. Class-specific AEs are plugged into the top of the DNN backbone and reconstruct the semantic representations learned by the DNN instead of the raw image. Through end-to-end learning, the DNN and the AEs boost each other to learn both discriminative and representative information. The results of experiments conducted on multiple datasets show that the proposed method achieves outstanding performance in both close and open set recognition and is sufficiently simple and flexible to incorporate into existing frameworks.

cs.CV↗

arXiv2020

Fibrations, and stability for compact group actions on manifolds with local bounded Ricci covering geometry

In this work, we (partially) generalize two classical tools in study of collapsed manifolds with bounded sectional curvature: a (singular) fibration theorem by Fukaya (1987) and Cheeger-Fukaya-Gromov (1992), and the stability for isometric compact Lie group actions on manifolds by Palais (1961) and Grove-Karcher (1973), to manifolds with local bounded Ricci covering geometry. Our two generalized results have been used in a recent work of Xiaochun Rong in generalizing Gromov's almost flat manifolds theorem to maximally collapsed manifolds with local bounded Ricci covering geometry.

math.DG↗