Searcharxiv⌕ Search

arXiv subjects

Zixiang Ni

Publications and source records attributed to Zixiang Ni.

10 recordsLinked to original sources

A generalized Hurwitz stability criterion via rectangular block Hankel matrices for nonmonic matrix polynomials

We develop a Hurwitz stability criterion for nonmonic matrix polynomials via column reduction, generalizing existing approaches constrained by the monic assumption and thus serving as a more natural extension of Gantmacher's classical stability criterion via Markov parameters. Starting from redefining the associated Markov parameters through an adaptive column-wise Lienard-Chipart splitting method, our framework constructs two structured matrices whose rectangular Hankel blocks are obtained via the extraction of these parameters. We establish an explicit interrelation between the inertia of column reduced matrix polynomials and the derived structured matrices. Furthermore, we demonstrate that the Hurwitz stability of column reduced matrix polynomials can be determined by the Hermitian positive definiteness of these rectangular block Hankel matrices.

math.OC↗

How Far Does Second-Order Linear Response Predict Quantization Mismatch? Pre-Registered Measurements on Five Language Models

Serving systems choose a weight format for each model and deployment. A low-cost screen predicts the resulting KL divergence from the unquantized model's Fisher information using $\tfrac12Δ^\top FΔ$, the expansion underlying Hessian-aware and second-order post-training quantizers. We assess the reliability of this screen in GPU batches with channel-wise int8, per-tensor fp8 and group-wise int4 on Qwen3-8B, Qwen3-4B, OLMo-2-7B, Qwen3-32B and Qwen3-30B-A3B. Relative predictions hold across a 16-fold range of perturbation scales, int4 group sizes and rounding modes, while absolute predictions are heavy-tailed. A trimmed average brings 10 of 12 cells on Qwen3-4B and OLMo-2 within 15% of measured KL. The MoE model exhibits nonquadratic response at small perturbations, with int8 KL scaling as $α^{0.63}$. Computing per-sequence gradients in bf16 can cause fourfold prediction errors. We trace these errors to arithmetic in the lower half of the backward pass and eliminate them by using fp32 there. Batch-invariant kernels run at 0.55 times the throughput, while changes in reduction order introduce a floor of 0.17-0.22 nats per sequence. We build a screener with one streamed fp32 Fisher pass, trimmed tensor-block predictions with a band, a flag for aligned tensors and per-layer format allocation. On the unseen Mistral-7B-v0.3 family, all six predictions fall within the band, all three KL budgets are met, and vLLM throughput is predicted within 0.4%.

cs.CL↗

What Sets the Serving Speed of Looped Language Models? Measurements and Optimizations Graded by Exactness

Looped language models apply one block of layers many times per token, so a serving engine pays every per-layer overhead once per loop; we measure serving for Ouro and Huginn-0125 in an open-source looped-model engine and grade our optimizations from bitwise-identical changes to those that require an accuracy check. At batch 1, fusing all loop submissions leaves Ouro throughput at 0.99x; the 5689 small kernels per token are the real cost, and fusing them behind rounding fences raises throughput 1.11-1.59x while 1014/1014 requests remain identical. An approximate fusion is no faster and misses a one-point GSM8K rule on 87 questions (61 and 62 correct versus 64), so we do not ship it. Huginn's released defaults avoid dead KV slots, reuse prefix-cache pages, and serve 1.77-3.16x faster than the base engine; Ouro reaches 1.65-2.15x and agent sessions 1.91-3.21x, while Huginn's noise and prefix-reuse changes stay within one point on all 1319 Huginn test questions; Huginn's packed prefill, -0.61 points on the same questions, meets that rule only on its point estimate. Position-keyed noise makes CPU cache reuse exact but still leaves 20 of 128 GPU full hits different, while loop fusion delays prefill admission and a 64-graph cache prevents eager fallbacks. Under prefill-decode disaggregation ExactLoop reaches up to 3.3x the base throughput with peer error handling disabled, and 3.1x in two of three runs with it enabled. Its transfer budget, sized to hold one request's KV in flight, cuts the commit tail from about one second under the earlier budget to 16 ms (base: 2.8 s), removing the polling rounds that the engine's interval would otherwise impose.

cs.DC↗

Where KV-Cache Error Lives in Looped Language Models: A Diagnostic Study of Ouro and Huginn

A looped language model applies one block of layers 4 times per token and keeps a key/value (KV) plane for each loop. We study where storage error changes the next-token distribution and how shared-weight error accumulates, using Ouro-1.4B (4 loops) on GSM8K and MATH and Huginn-0125 (32 loops) on GSM8K-derived chains, with dense, simulated quantization. Per unit of error, values carry 94% of the diagonal Fisher mass, but measured KL is dominated by keys with larger quantization errors. Local second-order prediction follows the measured KL of typical sequences, including 136 held-out chains, but misses a few confident rows, and a local second-order factor for shared weights at 32 loops ranges from 1.65 (INT8) to 4.09 (INT4), so it depends on the error direction. A cell-level allocator lowers one-step KL at 25-48% of BF16 bytes, while paired task-accuracy comparisons depend on budget and the 25% gap against K4V2 shrinks to near zero under a post hoc answer-extraction check. A later Ouro-2.6B test retains the key/value split but falsifies measured-cost allocation's advantage over Fisher at 31%; across all 21 ledger registrations, 60 of 176 criteria failed.

cs.DC↗

Recency Is the Wrong Axis: A Negative Result on Windowed Deepest-Plane KV Caching in Looped Language Models

Looped language models reuse one block of layers over $R$ recurrent steps but store a key-value (KV) plane for each step. Keeping only the deepest plane shrinks the cache up to $R$-fold. A natural repair, which we call RIPE, keeps every plane for the $W$ most recent positions and shares the deepest plane elsewhere. This repair acts on the wrong axis: across three looped models, the cost of sharing is set by how far the late loops' planes have converged (ripeness) and by which positions share, not by recency. On Raven-Llama the deep planes sit within the bf16 rounding floor, so no window is needed. On Huginn-0125 a 64-position window removes most of a concentrated replay-error tail (9 to 1 positions over 48 prompts), yet in a real self-speculative decoder the policy is slower and larger than the deepest plane alone and not measurably closer to full-cache decoding. On Ouro-1.4B the prompt lies outside any 16-64-position window: on 24 GSM8K questions with a 2048-token budget, the full cache and a FlashLoop baseline finish 23 answers and RIPE at most 2, although physical storage matches the emulated policy bitwise. Sensitivity and ripening models fitted on Huginn and Raven explain their contrast, and Ouro's own report that prompt-side sharing is harmful explains the third. Separately, bf16 execution-shape effects are as large as the cache effects being measured. Recent-window repair of shared-plane caches should not be adopted without a per-model ripeness test. We release ten pre-registered batches with every failed prediction.

cs.DC↗

ScaleRLT: Exact Log-Probabilities and Fast Deferred Rollouts for Reinforcement Learning on Looped Transformers

Looped language models such as Ouro reuse one block of layers several times per token. A rollout engine can batch the deeper loops of several response positions and run them only when a deep readout is due. This deferred schedule is fast, but the learner of an on-policy RL algorithm recomputes the sampled tokens' log-probabilities with a dense pass, and the two computations differ in floating point: in FP32 at 4096 tokens about 18% of the sampled tokens miss a $10^{-5}$ same-weight gate, and with bf16 weights nearly every token does. We use a hybrid learner log-probability, $\ell=\ell_{\mathrm{dense}}+\mathrm{sg}(\ell_{\mathrm{sched}}-\ell_{\mathrm{dense}})$, where $\mathrm{sg}$ denotes stop-gradient. Its value is a no-grad replay through the rollout's own schedule, and its gradient is the dense learner's. The replay reproduces the rollout bit for bit in all 167 FP32 and 28 bf16 real-weight cells we ran on three platforms, including rollouts whose batch shrinks at stop tokens, and the gradient has the same FP32 error as other dense implementations in all 102 pre-registered comparisons from 96 to 4096 tokens; the full GRPO loss of a training step matches a float64 reference to $3.1\times10^{-4}$. On one GPU, deferral raises rollout throughput by $1.47$ to $1.56\times$, and by $1.18$ to $1.54\times$ once every step runs from CUDA graphs, bit for bit. In a two-node GRPO loop, one-wave KV admission and CUDA graphs shorten an update by $2.71\times$ with the gate bitwise at every update, and four nodes keep the update time within 2% at the same per-GPU load. In 100-update runs held-out GSM8K accuracy rose from 23% to 82% with the mixed-depth readout that deferral relies on and from 34% to 90% with the native readout, which reached a higher accuracy in less time. For looped models beyond one node we analyse a ring mapping of layers to GPUs and measure its communication paths.

cs.LG↗

ReTaCo: Residual-Target Control for On-Policy Distillation

On-policy distillation (OPD) trains a student on its own generated prefixes with token-level teacher feedback, but transmitting or storing the teacher's full-vocabulary distribution at every token is costly. Entropy-aware OPD (EOPD) adds forward supervision to reverse KL to help the student recover plausible tokens it underestimates, using only the teacher's top-$k$ probabilities to limit cost. Because EOPD renormalizes these probabilities, its target assigns no mass to the omitted vocabulary. We prove that the resulting loss keeps pushing the student's top-$k$ mass toward one even after the student matches the teacher's relative probabilities within the top-$k$ set, so the teacher itself is not a stationary point whenever the omitted tokens have positive teacher probability. We propose ReTaCo (Residual-Target Control), which keeps the top-$k$ tokens individually and groups the remaining tokens into one residual symbol, and pairs this forward target with a single-sample estimator whose expectation equals the full-vocabulary reverse KL. With teacher top-$k$ mass $m$, the residual target is $(1-β)(1-m)$ for $β\in[0,1]$: $β=0$ preserves the teacher's mass, and larger $β$ moves more mass onto the top-$k$ tokens without changing their relative probabilities. At a fixed prefix, we prove that the population objective has a unique optimum whose top-$k$ mass lies between $m$ and $m+β(1-m)$ and increases monotonically with $β$; at $β=0$, underestimated top-$k$ tokens still receive non-vanishing recovery gradients. Numerical optimization confirms these predictions, and across three teacher-student pairs, ReTaCo outperforms EOPD on most mathematics and code benchmarks.

cs.LG↗

CARAT: Do Materials LLMs Reason or Recite?

When a materials LLM answers a question about crystal structure, does it reason from the structure or copy an answer already printed in its input? Accuracy cannot tell: a structural description often prints the very field it is scored against. CARAT holds question and gold answer fixed across eight matched views, names each structural relation separately in GraphSpace, and adds matched fine-tuning, answer masking, evidence injection, paired inference, and a rule that can withhold claims. First, on the benchmark's hardest families the grounded view is worth 17.3 points over formula inputs. Second, we turn that scrutiny on ourselves. GraphSpace beats a plain periodic graph by 19.3 points, but that margin is two effects at once: where the plain rendering carries everything the question needs it is 1.96 points, and where it omits those fields entirely, 46.7 points. The headline mostly measures what the baseline lacked, not how evidence is presented. Third, we attack our own benchmark. A rule that skips the link and reads the list directly answers four of seven hardened families, so we rebuilt it until eleven such shortcuts sat near chance. The frozen model quotes that link yet answers the same when we redirect it, on 95.6% of paired cases: it repeats the relation without using it. After matched supervision it reaches 99.8%, and deleting the link drops it to 23.4%, below the 27.0% the best shortcut reaches: both steps are learnable.

cs.AI↗

The Quartic Hessian Conjecture in Dimension Four

The Hessian conjecture asks whether a polynomial with nonzero constant Hessian determinant has a polynomial gradient inverse. It is known in dimensions at most three, false in dimensions at least five, and open in dimension four. We prove its four-variable quartic case. The top homogeneous part has zero Hessian determinant and, by the four-dimensional homogeneous Hesse theorem, is a cone. We divide its cone representative into three exhaustive types: a genuinely ternary quartic with nonzero ternary Hessian, a genuinely binary quartic, and a fourth power of a linear form. In the first type, the degree-seven determinant equation forces the cubic part to be affine-linear in the cone direction. In the binary type, the degree-six equation gives a constant null direction in a two-variable Hessian of the cubic part. In the unary type, the degree-five equation and a constant-direction lemma give the same conclusion. Every type therefore reduces to \[ f=P(x_1,x_2,x_3)+x_4Q(x_1,x_2,x_3)+a x_4^2, \qquad °Q\leq2. \] We prove, independently of the degree or top part of \(P\), that every constant-Hessian polynomial of this form has a polynomial gradient inverse. The branch \(a\ne0\) descends from the known three-dimensional Hessian conjecture after a Schur complement. When \(a=0\), an isotropic-cone rank analysis eliminates rank two, solves the rank-one exception by an explicit triangular inverse, and reduces rank zero to the two-dimensional Hessian conjecture. The coupled degree-six identity is retained throughout; no component with respect to a fixed quadratic form is separated.

math.AG↗

Read Between the Stickers: Sentiment-Prior Reasoning with Learnable Verbalized Rules for Multimodal Chat Analysis

Multimodal chat analysis of social media stickers (MCAS) benefits from jointly modeling text and sticker semantics, yet it is inherently challenged by the interference between sentiment and intent recognition. Although existing multi-task approaches achieve competitive performance, they largely ignore this inter-task interference and offer little explicit reasoning about how these two predictions are made. To address this issue, we propose \textbf{ExCoVer}, an \textbf{Ex}plicit \textbf{C}hain-\textbf{o}f-Thought framework with \textbf{Ver}balized rules learning that integrates sentiment-prior reasoning with learnable discrimination rules to produce explicit reasoning chains for sentiment and intent predictions. Specifically, ExCoVer consists of two components: (1) Sentiment-Prior Chain-of-Thought (SP-CoT), which detects cross-modal sentiment conflicts and uses the dominant sentiment as a prior to mitigate inter-task interference and narrow the candidate intent space; and (2) Verbalized Rules Learning for Confusing Intent Discrimination (VRLCID), which treats discrimination rules as learnable parameters and optimizes them via learner, optimizer, and regularizer agents to suppress spurious correlations and distinguish confusing intents. Extensive experiments on CSMSA and MSAIRS datasets demonstrate that ExCoVer achieves state-of-the-art performance while providing explicit reasoning chains.

cs.HC↗