Searcharxiv⌕ Search

arXiv subjects

Nitya Mani

Publications and source records attributed to Nitya Mani.

At least 19 recordsLinked to original sources
arXiv2026

Rewriting Scaling Laws for Synthetic Data with Compute and Data Constraints

Synthetic rewrites, where a language model rewrites a natural document e.g. into a target style, are now a standard ingredient of pretraining corpora. However, critical aspects of their use remain poorly understood, such as how much compute should be spent rewriting training data and whether a model trained on rewrites can outperform the rewriter. We study these two questions by proposing a rewriting scaling law that predicts loss from model size, training tokens, the fraction of training tokens that are rewrites, and the number of epochs over the natural data, charging the compute spent generating rewrites. We fit this scaling law in two settings. In the compute-limited setting, natural data is plentiful and rewrites come from a third-party rewriter trained with more compute than our largest run. We find that the estimated optimal rewrite fraction increases with compute, from under $10\%$ at $3\times10^{18}$ FLOPs to $32\%$ at $10^{22}$ FLOPs. Our scaling forecast extrapolates $33\times$ in FLOPs to train a 22B-parameter model, whose final loss is within $1.1\%$ of our prediction. This model achieves a better BPB than its rewriter on several web domains at one-tenth the rewriter's training compute. In the data-limited setting, we assume we only have a fixed seed set of natural data and that our rewriter is a model of the same size as the target, pretrained on that seed set. Again, we find the optimal rewrite fraction increases with the compute budget. We also demonstrate that a model trained on rewrites of its own pretraining data, generated by a rewriter of the same size, beats the best run on natural data alone at every model size up to 3B. Our results show that synthetic rewriting can be studied through the scaling law lens: that optimal methodological choices are predictable and scaling does not necessarily break down beyond the rewriter's capabilities.

cs.LG↗
arXiv2026

LEAPS: Long-context Evaluation of Adaptive Pattern Learning and State Tracking

Long-context language models (LCLMs) power agents that act over long trajectories, from improving a codebase to running research experiments to forecasters that condition on lengthy histories. These applications draw on three distinct long-context capabilities: recall, where the answer is a fact present in the context; state tracking, where the outcome can be computed by performing a series of transformations to a specified state; and pattern learning, where the goal is to make a prediction by sampling from a distribution inferred from many observations. Existing benchmarks primarily focus on the first capability. In this work, we introduce LEAPS (Long-context Evaluation of Adaptive Pattern learning and State tracking), a collection of 12 novel datasets, spanning agentic workflows, behavioral modeling, and forecasting of market and world events. By construction, LEAPS is resistant to data contamination, extends to millions of input tokens, and uses reliable metrics. We evaluate 30 API and open models from 8K to 512K input tokens. LEAPS distinguishes models that perform near perfectly on recall tasks, finding that such models' state tracking and pattern learning abilities can vary significantly and degrade much more dramatically with increasing input lengths--even for pattern learning, where more observations should make learning a theoretically easier task. Moreover, thinking models often underperform their base counterparts, exhausting their generation budget or oversimplifying the task. We hope LEAPS can guide the development of the next generation of LCLMs.

cs.CL↗
arXiv2025

New Sidorenko-type inequalities in tournaments

As a directed analog of Sidorenko's conjecture in extremal graph theory, Fox, Himwich, Zhou, and the second author defined an oriented graph $H$ to be tournament Sidorenko (anti-Sidorenko) if the random tournament asymptotically minimizes (maximizes) the number of copies of $H$ among all tournaments. We prove new inequalities of this form for oriented trees and cycles, considering both local and global notions of the Sidorenko property. We make progress on a conjecture of the aforementioned authors that every tree has an anti-Sidorenko direction, and give a characterization of short paths. For long paths we show that orientations are split symmetrically between being locally Sidorenko and anti-Sidorenko, yet almost all orientations are not globally Sidorenko. Finally, we give algorithms characterizing the local Sidorenko status of paths and cycles when the number of vertices is not divisible by four.

math.CO↗
arXiv2025

Random local access for sampling k-SAT solutions

We present a sublinear time algorithm that gives random local access to the uniform distribution over satisfying assignments to an arbitrary k-SAT formula $Φ$, at exponential clause density. Our algorithm provides memory-less query access to variable assignments, such that the output variable assignments consistently emulate a single global satisfying assignment whose law is close to the uniform distribution over satisfying assignments to $Φ$. Random local access and related models have been studied for a wide variety of natural Gibbs distributions and random graphical processes. Here, we establish feasibility of random local access models for one of the most canonical such sample spaces, the set of satisfying assignments to a k-SAT formula. Our algorithm proceeds by leveraging the local uniformity of the uniform distribution over satisfying assignments to $Φ$. We randomly partition the variables into two subsets, so that each clause has sufficiently many variables from each set to preserve local uniformity. We then sample some variables by simulating a systematic scan Glauber dynamics backward in time, greedily constructing the necessary intermediate steps. We sample the other variables by first conducting a search for a polylogarithmic-sized local component, which we iteratively grow to identify a small subformula from which we can efficiently sample using the appropriate marginal distribution. This two-pronged approach enables us to sample individual variable assignments without constructing a full solution.

cs.DS↗
arXiv2025

Counterexamples to a Weitz-Style Reduction for Multispin Systems

In a seminal paper, Weitz showed that for two-state spin systems, such as the Ising and hardcore models from statistical physics, correlation decay on trees implies correlation decay on arbitrary graphs. The key gadget in Weitz's reduction has been instrumental in recent advances in approximate counting and sampling, from analysis of local Markov chains like Glauber dynamics to the design of deterministic algorithms for estimating the partition function. A longstanding open problem in the field has been to find such a reduction for more general multispin systems like the uniform distribution over proper colorings of a graph. In this paper, we show that for a rich class of multispin systems, including the ferromagnetic Potts model, there are fundamental obstacles to extending Weitz's reduction to the multispin setting. A central component of our investigation is establishing nonconvexity of the image of the belief propagation functional, the standard tool for analyzing spin systems on trees. On the other hand, we provide evidence of convexity for the antiferromagnetic Potts model.

cs.DS↗
arXiv2024

On monochromatic solutions to linear equations over the integers

We study the number of monochromatic solutions to linear equations in a $2$-coloring of $\{1,\ldots,n\}$. We show that any nontrivial linear equation has a constant fraction of solutions that are monochromatic in any $2$-coloring of $\{1,\ldots,n\}$. We further study commonness of four-term equations and disprove a conjecture of Costello and Elvin by showing that, unlike over $\mathbb{F}_p$, the four-term equation $x_1 + 2x_2 - x_3 - 2x_4 = 0$ is uncommon over $\{1,\ldots,n\}$.

math.CO↗
arXiv2024

Fast sampling of satisfying assignments from random $k$-SAT with applications to connectivity

We give a nearly linear-time algorithm to approximately sample satisfying assignments in the random $k$-SAT model when the density of the formula scales exponentially with $k$. The best previously known sampling algorithm for the random $k$-SAT model applies when the density $α=m/n$ of the formula is less than $2^{k/300}$ and runs in time $n^{\exp(Θ(k))}$. Here $n$ is the number of variables and $m$ is the number of clauses. Our algorithm achieves a significantly faster running time of $n^{1 + o_k(1)}$ and samples satisfying assignments up to density $α\leq 2^{0.039 k}$. The main challenge in our setting is the presence of many variables with unbounded degree, which causes significant correlations within the formula and impedes the application of relevant Markov chain methods from the bounded-degree setting. Our main technical contribution is a $o_k(\log n )$ bound of the sum of influences in the $k$-SAT model which turns out to be robust against the presence of high-degree variables. This allows us to apply the spectral independence framework and obtain fast mixing results of a uniform-block Glauber dynamics on a carefully selected subset of the variables. The final key ingredient in our method is to take advantage of the sparsity of logarithmic-sized connected sets and the expansion properties of the random formula, and establish relevant connectivity properties of the set of satisfying assignments that enable the fast simulation of this Glauber dynamics. Our results also allow us to conclude that, with high probability, a random $k$-CNF formula with density at most $2^{0.227 k}$ has a giant component of solutions that are connected in a graph where solutions are adjacent if they have Hamming distance $O_k(\log n)$. We are also able to deduce looseness results for random $k$-CNFs in the same regime.

cs.DS↗
arXiv2024

Clustering in typical unit-distance avoiding sets

In the 1960s Moser asked how dense a subset of $\mathbb{R}^d$ can be if no pairs of points in the subset are exactly distance 1 apart. There has been a long line of work showing upper bounds on this density. One curious feature of dense unit distance avoiding sets is that they appear to be ``clumpy,'' i.e. forbidding unit distances comes hand in hand with having more than the expected number distance $\approx 2$ pairs. In this work we rigorously establish this phenomenon in $\mathbb{R}^2$. We show that dense unit distance avoiding sets have over-represented distance $\approx 2$ pairs, and that this clustering extends to typical unit distance avoiding sets. To do so, we build off of the linear programming approach used previously to prove upper bounds on the density of unit distance avoiding sets.

math.MG↗
arXiv2024

Turán Problems for Mixed Graphs

We investigate natural Turán problems for mixed graphs, generalizations of graphs where edges can be either directed or undirected. We study a natural \textit{Turán density coefficient} that measures how large a fraction of directed edges an $F$-free mixed graph can have; we establish an analogue of the Erdős-Stone-Simonovits theorem and give a variational characterization of the Turán density coefficient of any mixed graph (along with an associated extremal $F$-free family). This characterization enables us to highlight an important divergence between classical extremal numbers and the Turán density coefficient. We show that Turán density coefficients can be irrational, but are always algebraic; for every positive integer $k$, we construct a family of mixed graphs whose Turán density coefficient has algebraic degree $k$.

math.CO↗
arXiv2024

Multidisperse Random Sequential Adsorption and Generalizations

In this paper, we present a unified study of the limiting density in one-dimensional random sequential adsorption (RSA) processes where segment lengths are drawn from a given distribution. In addition to generic bounds, we are also able to characterize specific cases, including multidisperse RSA, in which we draw from a finite set of lengths, and power-law RSA, in which we draw lengths from a power-law distribution.

math.PR↗
arXiv2024

Characterizing the fourth-moment phenomenon of monochromatic subgraph counts via influences

We investigate the distribution of monochromatic subgraph counts in random vertex $2$-colorings of large graphs. We give sufficient conditions for the asymptotic normality of these counts and demonstrate their essential necessity (particularly for monochromatic triangles). Our approach refines the fourth-moment theorem to establish new, local influence-based conditions for asymptotic normality; these findings more generally provide insight into fourth-moment phenomena for a broader class of Rademacher and Gaussian polynomials.

math.PR↗
arXiv2024

When will (game) wars end?

We study several variants of the classical card game war. As anyone who played this game knows, the game can take some time to terminate, but it usually does. Here, we analyze a number of asymptotic variants of the game, where the number of cards is $n$, and show that all have expected termination time of order $n^2$. This is the same expected termination time as in the game where at each turn a fair coin toss decides which player wins a card, known as Gambler's Ruin and studied by Pascal, Fermat and others in the seventeenth century.

math.CO↗
arXiv2024

Strong spatial mixing for colorings on trees and its algorithmic applications

Strong spatial mixing (SSM) is an important quantitative notion of correlation decay for Gibbs distributions arising in statistical physics, probability theory, and theoretical computer science. A longstanding conjecture is that the uniform distribution on proper $q$-colorings on a $Δ$-regular tree exhibits SSM whenever $q \ge Δ+1$. Moreover, it is widely believed that as long as SSM holds on bounded-degree trees with $q$ colors, one would obtain an efficient sampler for $q$-colorings on all bounded-degree graphs via simple Markov chain algorithms. It is surprising that such a basic question is still open, even on trees, but then again it also highlights how much we still have to learn about random colorings. In this paper, we show the following: (1) For any $Δ\ge 3$, SSM holds for random $q$-colorings on trees of maximum degree $Δ$ whenever $q \ge Δ+ 3$. Thus we almost fully resolve the aforementioned conjecture. Our result substantially improves upon the previously best bound which requires $q \ge 1.59Δ+γ^*$ for an absolute constant $γ^* > 0$. (2) For any $Δ\ge 3$ and girth $g = Ω_Δ(1)$, we establish optimal mixing of the Glauber dynamics for $q$-colorings on graphs of maximum degree $Δ$ and girth $g$ whenever $q \ge Δ+3$. Our approach is based on a new general reduction from spectral independence on large-girth graphs to SSM on trees that is of independent interest. Using the same techniques, we also prove near-optimal bounds on weak spatial mixing (WSM), a closely-related notion to SSM, for the antiferromagnetic Potts model on trees.

cs.DS↗
arXiv2024

Variations on Sidorenko's conjecture in tournaments

We study variants of Sidorenko's conjecture in tournaments, where new phenomena arise that do not have clear analogues in the setting of undirected graphs. We first consider oriented graphs that are systematically under-represented in tournaments (called tournament anti-Sidorenko). We prove that such oriented graphs must be quite sparse; specifically, the maximum number of edges of a $k$-vertex oriented graph which is tournament anti-Sidorenko is $(1+o(1))k\log_2 k$. We also give several novel constructions of oriented graphs that are systematically over-represented in tournaments (tournament Sidorenko); as a representative example, we show that most ways to delete an edge from a transitive tournament yield a tournament Sidorenko oriented graph. As an illustration of our methods, we characterize which orientations of stars are tournament Sidorenko and which are tournament anti-Sidorenko.

math.CO↗
arXiv2023

Nearly all $k$-SAT functions are unate

We prove that $1-o(1)$ fraction of all $k$-SAT functions on $n$ Boolean variables are unate (i.e., monotone after first negating some variables), for any fixed positive integer $k$ and as $n \to \infty$. This resolves a conjecture by Bollobás, Brightwell, and Leader from 2003.

math.CO↗
arXiv2023

On sum-intersecting families of positive integers

We study the following natural arithmetic question regarding intersecting families: how large can a family of subsets of integers from $\{1, \ldots n\}$ be such that, for every pair of subsets in the family, the intersection contains a sum $x + y = z$? We conjecture that any such sum-intersecting family must have size at most $\frac14 \cdot 2^{n}$ (which would be tight if correct). Towards this conjecture, we show that every sum-intersecting family has at most $0.32 \cdot 2^n$ subsets.

math.CO↗
arXiv2023

A fourth moment phenomenon for asymptotic normality of monochromatic subgraphs

Given a graph sequence $\{G_n\}_{n\ge1}$ and a simple connected subgraph $H$, we denote by $T(H,G_n)$ the number of monochromatic copies of $H$ in a uniformly random vertex coloring of $G_n$ with $c \ge 2$ colors. In this article, we prove a central limit theorem for $T(H,G_n)$ with explicit error rates. The error rates arise from graph counts of collections formed by joining copies of $H$ that we call good joins. Counts of good joins are closely related to the fourth moment of a normalized version of $T(H,G_{n})$, and that connection allows us to show a fourth moment phenomenon for the central limit theorem. Precisely, for $c\ge 30$, we show that $T(H,G_n)$ (appropriately centered and rescaled) converges in distribution to $\mathcal{N}(0,1)$ whenever its fourth moment converges to 3 (the fourth moment of the standard normal distribution). We show the convergence of the fourth moment is necessary to obtain a normal limit when $c\ge 2$. The combination of these results implies that the fourth moment condition characterizes the limiting normal distribution of $T(H,G_n)$ for all subgraphs $H$, whenever $c\ge 30$.

math.PR↗