arXiv subjects

Carol Wang

Publications and source records attributed to Carol Wang.

10 recordsLinked to original sources

arXiv2026

Yuan3.0 Ultra: A Trillion-Parameter Enterprise-Oriented MoE LLM

We introduce Yuan3.0 Ultra, an open-source Mixture-of-Experts (MoE) large language model featuring 68.8B activated parameters and 1010B total parameters, specially designed to enhance performance on enterprise scenarios tasks while maintaining competitive capabilities on general purpose tasks. We propose Layer-Adaptive Expert Pruning (LAEP) algorithm designed for the pre-training stage of MoE LLMs. In contrast to previous expert pruning approaches that operate primarily in the post-training phase, the proposed algorithm enhances training efficiency by selectively pruning underutilized experts and reorganizing experts across computing devices according to token distribution statistics. Comprehensive experiments demonstrate that LAEP effectively reduces model size and substantially improves pre-training efficiency. When pre-training Yuan3.0 Ultra from scratch original with 1515B parameters, this algorithm delivers a 49\% boost in pre-training efficiency and a 33.3\% reduction in total parameters, while preserving the model's outstanding multi-domain performance. On enterprise scenario benchmarks including Docmatix, ChatRAG, SummEval and MMTab, Yuan3.0 Ultra achieves leading accuracy. The model and codes are publicly available at https://github.com/Yuan-lab-LLM/Yuan3.0-Ultra.

cs.LG↗

arXiv2026

Yuan3.0 Flash: An Open Multimodal Large Language Model for Enterprise Applications

We introduce Yuan3.0 Flash, an open-source Mixture-of-Experts (MoE) MultiModal Large Language Model featuring 3.7B activated parameters and 40B total parameters, specifically designed to enhance performance on enterprise-oriented tasks while maintaining competitive capabilities on general-purpose tasks. To address the overthinking phenomenon commonly observed in Large Reasoning Models (LRMs), we propose Reflection-aware Adaptive Policy Optimization (RAPO), a novel RL training algorithm that effectively regulates overthinking behaviors. In enterprise-oriented tasks such as retrieval-augmented generation (RAG), complex table understanding, and summarization, Yuan3.0 Flash consistently achieves superior performance. Moreover, it also demonstrates strong reasoning capabilities in domains such as mathematics, science, etc., attaining accuracy comparable to frontier model while requiring only approximately 1/4 to 1/2 of the average tokens. Yuan3.0 Flash has been fully open-sourced to facilitate further research and real-world deployment: https://github.com/Yuan-lab-LLM/Yuan3.0.

cs.AI↗

arXiv2019

Fundamental Limits of Communication Over State-Dependent Channels With Feedback

The fundamental limits of communication over state-dependent discrete memoryless channels with noiseless feedback are studied, under the assumption that the communicating parties are allowed to use variable-length coding schemes. Various cases are analyzed, with the employed coding schemes having either bounded or unbounded codeword lengths, and with state information revealed to the encoder and/or decoder in a strictly causal, causal, or non-causal manner. In each of these settings, necessary and sufficient conditions for positivity of the zero-error capacity are obtained and it is shown that, whenever the zero-error capacity is positive, it equals the conventional vanishing-error capacity. Moreover, it is shown that the vanishing-error capacity of state-dependent channels is not increased by the use of feedback and variable-length coding. Both these kinds of capacities of state-dependent channels with feedback are thus fully characterized.

cs.IT↗

arXiv2018

On the Maximum Size of Block Codes Subject to a Distance Criterion

We establish a general formula for the maximum size of finite length block codes with minimum pairwise distance no less than $d$. The achievability argument involves an iterative construction of a set of radius-$d$ balls, each centered at a codeword. We demonstrate that the number of such balls that cover the entire code alphabet cannot exceed this maximum size. Our approach can be applied to codes $i)$ with elements over arbitrary code alphabets, and $ii)$ under a broad class of distance measures, thereby ensuring the generality of our formula. Our formula indicates that the maximum code size can be fully characterized by the cumulative distribution function of the distance measure evaluated at two independent and identically distributed random codewords. When the two random codewords assume a uniform distribution over the entire code alphabet, our formula recovers and obtains a natural generalization of the Gilbert-Varshamov (GV) lower bound. We also establish a general formula for the zero-error capacity of any sequence of channels. Finally, we extend our study to the asymptotic setting, where we establish first- and second-order bounds on the asymptotic code rate subject to a normalized minimum distance constraint.

cs.IT↗

arXiv2016

Maximally Recoverable Codes for Grid-like Topologies

The explosion in the volumes of data being stored online has resulted in distributed storage systems transitioning to erasure coding based schemes. Yet, the codes being deployed in practice are fairly short. In this work, we address what we view as the main coding theoretic barrier to deploying longer codes in storage: at large lengths, failures are not independent and correlated failures are inevitable. This motivates designing codes that allow quick data recovery even after large correlated failures, and which have efficient encoding and decoding. We propose that code design for distributed storage be viewed as a two-step process. The first step is choose a topology of the code, which incorporates knowledge about the correlated failures that need to be handled, and ensures local recovery from such failures. In the second step one specifies a code with the chosen topology by choosing coefficients from a finite field. In this step, one tries to balance reliability (which is better over larger fields) with encoding and decoding efficiency (which is better over smaller fields). This work initiates an in-depth study of this reliability/efficiency tradeoff. We consider the field-size needed for achieving maximal recoverability: the strongest reliability possible with a given topology. We propose a family of topologies called grid-like topologies which unify a number of topologies considered both in theory and practice, and prove a collection of results about maximally recoverable codes in such topologies including the first super-polynomial lower bound on the field size.

cs.IT↗

arXiv2016

On the capacity of the binary adversarial wiretap channel

New bounds on the semantic secrecy capacity of the binary adversarial wiretap channel are established . Against an adversary which reads a $ρ_r$ fraction of the transmitted codeword and modifies a $ρ_w$ fraction of the codeword, we show an achievable rate of $1-h(ρ_w)-ρ_r$, where $h(\cdot)$ is the binary entropy function. We also give an upper bound which is nearly matching when $ρ_r$ is small.

cs.IT↗

arXiv2014

Deletion codes in the high-noise and high-rate regimes

The noise model of deletions poses significant challenges in coding theory, with basic questions like the capacity of the binary deletion channel still being open. In this paper, we study the harder model of worst-case deletions, with a focus on constructing efficiently decodable codes for the two extreme regimes of high-noise and high-rate. Specifically, we construct polynomial-time decodable codes with the following trade-offs (for any eps > 0): (1) Codes that can correct a fraction 1-eps of deletions with rate poly(eps) over an alphabet of size poly(1/eps); (2) Binary codes of rate 1-O~(sqrt(eps)) that can correct a fraction eps of deletions; and (3) Binary codes that can be list decoded from a fraction (1/2-eps) of deletions with rate poly(eps) Our work is the first to achieve the qualitative goals of correcting a deletion fraction approaching 1 over bounded alphabets, and correcting a constant fraction of bit deletions with rate aproaching 1. The above results bring our understanding of deletion code constructions in these regimes to a similar level as worst-case errors.

cs.IT↗

arXiv2013

Explicit rank-metric codes list-decodable with optimal redundancy

We construct an explicit family of linear rank-metric codes over any field ${\mathbb F}_h$ that enables efficient list decoding up to a fraction $ρ$ of errors in the rank metric with a rate of $1-ρ-ε$, for any desired $ρ\in (0,1)$ and $ε> 0$. Previously, a Monte Carlo construction of such codes was known, but this is in fact the first explicit construction of positive rate rank-metric codes for list decoding beyond the unique decoding radius. Our codes are subcodes of the well-known Gabidulin codes, which encode linearized polynomials of low degree via their values at a collection of linearly independent points. The subcode is picked by restricting the message polynomials to an ${\mathbb F}_h$-subspace that evades the structured subspaces over an extension field ${\mathbb F}_{h^t}$ that arise in the linear-algebraic list decoder for Gabidulin codes due to Guruswami and Xing (STOC'13). This subspace is obtained by combining subspace designs contructed by Guruswami and Kopparty (FOCS'13) with subspace evasive varieties due to Dvir and Lovett (STOC'12). We establish a similar result for subspace codes, which are a collection of subspaces, every pair of which have low-dimensional intersection, and which have received much attention recently in the context of network coding. We also give explicit subcodes of folded Reed-Solomon (RS) codes with small folding order that are list-decodable (in the Hamming metric) with optimal redundancy, motivated by the fact that list decoding RS codes reduces to list decoding such folded RS codes. However, as we only list decode a subcode of these codes, the Johnson radius continues to be the best known error fraction for list decoding RS codes.

cs.IT↗

arXiv2012

List decoding subspace codes from insertions and deletions

We present a construction of subspace codes along with an efficient algorithm for list decoding from both insertions and deletions, handling an information-theoretically maximum fraction of these with polynomially small rate. Our construction is based on a variant of the folded Reed-Solomon codes in the world of linearized polynomials, and the algorithm is inspired by the recent linear-algebraic approach to list decoding. Ours is the first list decoding algorithm for subspace codes that can handle deletions; even one deletion can totally distort the structure of the basis of a subspace and is thus challenging to handle. When there are only insertions, we also present results for list decoding subspace codes that are the linearized analog of Reed-Solomon codes (proposed previously, and closely related to the Gabidulin codes for rank-metric), obtaining some improvements over similar results in previous work.

cs.IT↗

arXiv2011

Optimal rate list decoding via derivative codes

The classical family of $[n,k]_q$ Reed-Solomon codes over a field $\F_q$ consist of the evaluations of polynomials $f \in \F_q[X]$ of degree $< k$ at $n$ distinct field elements. In this work, we consider a closely related family of codes, called (order $m$) {\em derivative codes} and defined over fields of large characteristic, which consist of the evaluations of $f$ as well as its first $m-1$ formal derivatives at $n$ distinct field elements. For large enough $m$, we show that these codes can be list-decoded in polynomial time from an error fraction approaching $1-R$, where $R=k/(nm)$ is the rate of the code. This gives an alternate construction to folded Reed-Solomon codes for achieving the optimal trade-off between rate and list error-correction radius. Our decoding algorithm is linear-algebraic, and involves solving a linear system to interpolate a multivariate polynomial, and then solving another structured linear system to retrieve the list of candidate polynomials $f$. The algorithm for derivative codes offers some advantages compared to a similar one for folded Reed-Solomon codes in terms of efficient unique decoding in the presence of side information.

cs.IT↗