Searcharxiv⌕ Search

arXiv subjects

Arash Gholami Davoodi

Publications and source records attributed to Arash Gholami Davoodi.

15 recordsLinked to original sources

The Shape of Overthinking: Backtracking Bursts in Long Reasoning Traces

Reasoning models often generate long traces in which useful self-correction and unproductive revision are hard to distinguish. We study this distinction through backtracking dynamics: local reconsideration, retraction, or re-derivation inside long-form reasoning traces. On 6{,}000 Qwen3-8B AIME traces, we annotate segment-level backtrack severity and analyze event timing, normalized depth, and local burst structure. We find that early isolated repair is often compatible with correct reasoning, whereas incorrect traces more often show moderate-to-severe backtracks that persist and cluster late. Cross-corpus checks show the same qualitative asymmetry across additional model/domain pairs. Filtering analyses instantiate the signal as a prefix-causal selective early-exit policy: at shallow and intermediate depths, burst-aware filtering outperforms fixed length-based filtering while using only prefix-available features. Moderate length cutoffs remain strong completed-trace baselines, but burst-aware control provides a deployable mechanism for separating recoverable repair from likely instability.

cs.AI↗

Geometry-Aware Decoding with Wasserstein-Regularized Truncation and Mass Penalties for Large Language Models

Large language models (LLMs) must balance diversity and creativity against logical coherence in open-ended generation. Existing truncation-based samplers are effective but largely heuristic, relying mainly on probability mass and entropy while ignoring semantic geometry of the token space. We present Top-W, a geometry-aware truncation rule that uses Wasserstein distance-defined over token-embedding geometry-to keep the cropped distribution close to the original, while explicitly balancing retained probability mass against the entropy of the kept set. Our theory yields a simple closed-form structure for the fixed-potential subset update: depending on the mass-entropy trade-off, the optimal crop either collapses to a single token or takes the form of a one-dimensional prefix that can be found efficiently with a linear scan. We implement Top-W using efficient geometry-based potentials (nearest-set or k-NN) and pair it with an alternating decoding routine that keeps the standard truncation-and-sampling interface unchanged. Extensive experiments on four benchmarks (GSM8K, GPQA, AlpacaEval, and MT-Bench) across three instruction-tuned models show that Top-W consistently outperforms prior state-of-the-art decoding approaches achieving up to 33.7% improvement. Moreover, we find that Top-W not only improves accuracy-focused performance, but also boosts creativity under judge-based open-ended evaluation.

cs.CL↗

Vertex-Softmax: Tight Transformer Verification via Exact Softmax Optimization

Certified verification of transformer attention requires bounding the softmax function over interval constraints on the pre-softmax scores. Existing verifiers relax softmax ndependently of the downstream objective, leaving avoidable slack. We prove that the exact optimum of this score-box problem is attained at a vertex of the constraint box, and establish a threshold structure theorem showing that, after sorting the objective coefficients, the optimum lies among only linearly many candidates, yielding the Vertex-Softmax primitive with log-linear complexity in the sequence length. We further prove a formal optimality result showing that Vertex-Softmax is the tightest sound bound obtainable from score intervals alone, characterizing precisely what additional structure (score correlations, score-value coupling) is needed for further improvement. Integrated into a CROWN Convex Relaxation based Optimization for Worst-case Neurons)-style verifier with a formal soundness guarantee, Vertex-Softmax significantly improves certified rates and substantially tightens lower bounds across MNIST, Fashion-MNIST, and CIFAR-10 attention models, while consistently matching or outperforming alpha-CROWN and branch-and-bound baselines at a fraction of their cost.

cs.LG↗

LLMs Are Not Intelligent Thinkers: Introducing Mathematical Topic Tree Benchmark for Comprehensive Evaluation of LLMs

Large language models (LLMs) demonstrate impressive capabilities in mathematical reasoning. However, despite these achievements, current evaluations are mostly limited to specific mathematical topics, and it remains unclear whether LLMs are genuinely engaging in reasoning. To address these gaps, we present the Mathematical Topics Tree (MaTT) benchmark, a challenging and structured benchmark that offers 1,958 questions across a wide array of mathematical subjects, each paired with a detailed hierarchical chain of topics. Upon assessing different LLMs using the MaTT benchmark, we find that the most advanced model, GPT-4, achieved a mere 54\% accuracy in a multiple-choice scenario. Interestingly, even when employing Chain-of-Thought prompting, we observe mostly no notable improvement. Moreover, LLMs accuracy dramatically reduced by up to 24.2 percentage point when the questions were presented without providing choices. Further detailed analysis of the LLMs' performance across a range of topics showed significant discrepancy even for closely related subtopics within the same general mathematical area. In an effort to pinpoint the reasons behind LLMs performances, we conducted a manual evaluation of the completeness and correctness of the explanations generated by GPT-4 when choices were available. Surprisingly, we find that in only 53.3\% of the instances where the model provided a correct answer, the accompanying explanations were deemed complete and accurate, i.e., the model engaged in genuine reasoning.

cs.CL↗

ForestDSH: A Universal Hash Design for Discrete Probability Distributions

In this paper, we consider the problem of classification of $M$ high dimensional queries $y^1,\cdots,y^M\in B^S$ to $N$ high dimensional classes $x^1,\cdots,x^N\in A^S$ where $A$ and $B$ are discrete alphabets and the probabilistic model that relates data to the classes $P(x,y)$ is known. This problem has applications in various fields including the database search problem in mass spectrometry. The problem is analogous to the nearest neighbor search problem, where the goal is to find the data point in a database that is the most similar to a query point. The state of the art method for solving an approximate version of the nearest neighbor search problem in high dimensions is locality sensitive hashing (LSH). LSH is based on designing hash functions that map near points to the same buckets with a probability higher than random (far) points. To solve our high dimensional classification problem, we introduce distribution sensitive hashes that map jointly generated pairs $(x,y)\sim P$ to the same bucket with probability higher than random pairs $x\sim P^A$ and $y\sim P^B$, where $P^A$ and $P^B$ are the marginal probability distributions of $P$. We design distribution sensitive hashes using a forest of decision trees and we show that the complexity of search grows with $O(N^{λ^*(P)})$ where $λ^*(P)$ is expressed in an analytical form. We further show that the proposed hashes perform faster than state of the art approximate nearest neighbor search methods for a range of probability distributions, in both theory and simulations. Finally, we apply our method to the spectral library search problem in mass spectrometry, and show that it is an order of magnitude faster than the state of the art methods.

cs.LG↗

Degrees of Freedom Region of the $(M,N_1,N_2)$ MIMO Broadcast Channel with Partial CSIT: An Application of Sum-set Inequalities Based on Aligned Image Sets

The degrees of freedom (DoF) region is characterized for the $2$-user multiple input multiple output (MIMO) broadcast channel (BC), where the transmitter is equipped with $M$ antennas, the two receivers are equipped with $N_1$ and $N_2$ antennas, and the levels of channel state information at the transmitter (CSIT) for the two users are parameterized by $β_1, β_2$, respectively. The achievability of the DoF region was established by Hao, Rassouli and Clerckx, but no proof of optimality was heretofore available. The proof of optimality is provided in this work with the aid of sum-set inequalities based on the aligned image sets (AIS) approach.

cs.IT↗

Optimality of Simple Layered Superposition Coding in the 3 User MISO BC with Finite Precision CSIT

We study the $K=3$ user multiple input single output (MISO) broadcast channel (BC) with $M=3$ antennas at the transmitter and $1$ antenna at each receiver, from the generalized degrees of freedom (GDoF) perspective, under the assumption that the channel state information at the transmitter (CSIT) is limited to finite precision. In particular, our goal is to identify a parameter regime where a simple layered superposition (SLS) coding scheme achieves the entire GDoF region. With $α_{ij}$ representing the channel strength parameter for the link from the $j^{th}$ antenna of the transmitter to the $i^{th}$ receiver, we prove that SLS is GDoF optimal without the need for time-sharing if $\max(α_{ki},α_{im})\leqα_{ii}$ and $α_{ki}+α_{im}\leα_{ii}+α_{km}$ for all $i,k\in[3],m\in[M]$. The GDoF region under this condition is a convex polyhedron. The result generalizes to arbitrary $M\geq 3$.

cs.IT↗

$K$-User Symmetric M$\times$N MIMO Interference Channel under Finite Precision CSIT: A GDoF perspective

Generalized Degrees of Freedom (GDoF) are characterized for the symmetric $K$-user Multiple Input Multiple Output (MIMO) Interference Channel (IC) under the assumption that the channel state information at the transmitters (CSIT) is limited to finite precision. In this symmetric setting, each transmitter is equipped with $M$ antennas, each receiver is equipped with $N$ antennas, each desired channel (i.e., a channel between a transmit antenna and a receive antenna belonging to the same user) has strength $\sim P$, while each undesired channel has strength $\sim P^α$, where $P$ is a nominal SNR parameter. The result generalizes a previous GDoF characterization for the SISO setting $(M=N=1)$ and is enabled by a significant extension of the Aligned Image Sets bound that is broadly useful. GDoF per user take the form of a $W$-curve with respect to $α$ for fixed values of $M$ and $N$. Under finite precision CSIT, in spite of the presence of multiple antennas, all the benefits of interference alignment are lost.

cs.IT↗

Sum-set Inequalities from Aligned Image Sets: Instruments for Robust GDoF Bounds

We present sum-set inequalities specialized to the generalized degrees of freedom (GDoF) framework. These are information theoretic lower bounds on the entropy of bounded density linear combinations of discrete, power-limited dependent random variables in terms of the joint entropies of arbitrary linear combinations of new random variables that are obtained by power level partitioning of the original random variables. These bounds generalize the aligned image sets approach, and are useful instruments to obtain GDoF characterizations for wireless networks, especially with multiple antenna nodes, subject to arbitrary channel strength and channel uncertainty levels. To demonstrate the utility of these bounds, we consider a non-trivial instance of wireless networks - a two user interference channel with different number of antennas at each node, and different levels of partial channel knowledge available to the transmitters. We obtain tight GDoF characterization for specific instance of this channel with the aid of sum-set inequalities.

cs.IT↗

Network Coherence Time Matters - Aligned Image Sets and the Degrees of Freedom of Interference Networks with Finite Precision CSIT and Perfect CSIR

This work obtains the first bound that is provably sensitive to network coherence time, i.e., coherence time in an interference network where all channels experience the same coherence patterns. This is accomplished by a novel adaptation of the aligned image sets bound, and settles various open problems noted previously by Naderi and Avestimehr and by Gou et al. For example, a necessary and sufficient condition is obtained for the optimality of 1/2 DoF per user in a partially connected interference network where the channel state information at the receivers (CSIR) is perfect, the channel state information at the transmitters (CSIT) is instantaneous but limited to finite precision, and the network coherence time is T_c= 1. The surprising insight that emerges is that even with perfect CSIR and instantaneous finite precision CSIT, network coherence time matters, i.e., it has a DoF impact.

cs.IT↗

Aligned Image Sets and the Generalized Degrees of Freedom of Symmetric MIMO Interference Channel with Partial CSIT

The generalized degrees of freedom (GDoF) of the two user symmetric multiple input multiple output (MIMO) interference channel (IC) are characterized as a function of the channel strength levels and the level of channel state information at the transmitters (CSIT). In this symmetric setting, each transmitter is equipped with M antennas, each receiver is equipped with N antennas, and both cross links have the same strength parameter $α$ and the same channel uncertainty parameter $β$. The main challenge resides in the proof of the outer bound which is accomplished by a generalization of the aligned image sets approach.

cs.IT↗

GDoF of the MISO BC: Bridging the Gap between Finite Precision and Perfect CSIT

For the $K=2$ user MISO BC, i.e., the wireless broadcast channel where a transmitter equipped with $K=2$ antennas sends independent messages to $K=2$ receivers each of which is equipped with a single antenna, the sum generalized degrees of freedom (GDoF) are characterized for arbitrary channel strength and channel uncertainty levels for each of the channel coefficients. The result is extended to $K>2$ users under additional restrictions which include the assumption of symmetry.

cs.IT↗

Generalized Degrees of Freedom of the Symmetric K-User Interference Channel under Finite Precision CSIT

The generalized degrees of freedom (GDoF) characterization of the symmetric K-user interference channel is obtained under finite precision channel state information at the transmitters (CSIT). The symmetric setting is where each cross channel is capable of carrying degrees of freedom (DoF) while each direct channel is capable of carrying 1 DoF. Remarkably, under finite precision CSIT the symmetric K-user interference channel loses all the GDoF benefits of interference alignment. The GDoF per user diminish with the number of users everywhere except in the very strong (optimal for every receiver to decode all messages) and very weak (optimal to treat all interference as noise) interference regimes. The result stands in sharp contrast to prior work on the symmetric setting under perfect CSIT, where the GDoF per user remain undiminished due to interference alignment. The result also stands in contrast to prior work on a subclass of asymmetric settings under finite precision CSIT, i.e., the topological interference management problem, where interference alignment plays a crucial role and provides substantial GDoF benefits.

cs.IT↗

Aligned Image Sets under Channel Uncertainty: Settling a Conjecture by Lapidoth, Shamai and Wigger on the Collapse of Degrees of Freedom under Finite Precision CSIT

A conjecture made by Lapidoth, Shamai and Wigger at Allerton 2005 (also an open problem presented at ITA 2006) states that the DoF of a 2 user broadcast channel, where the transmitter is equipped with 2 antennas and each user is equipped with 1 antenna, must collapse under finite precision CSIT. In this work we prove that the conjecture is true in all non-degenerate settings (e.g., where the probability density function of unknown channel coefficients exists and is bounded). The DoF collapse even when perfect channel knowledge for one user is available to the transmitter. This also settles a related recent conjecture by Tandon et al. The key to our proof is a bound on the number of codewords that can cast the same image (within noise distortion) at the undesired receiver whose channel is subject to finite precision CSIT, while remaining resolvable at the desired receiver whose channel is precisely known by the transmitter. We are also able to generalize the result along two directions. First, if the peak of the probability density function is allowed to scale as O(P^(α/2)), representing the concentration of probability density (improving CSIT) due to, e.g., quantized feedback at rate (α/2)\log(P), then the DoF are bounded above by 1+α, which is also achievable under quantized feedback. Second, we generalize the result to the K user broadcast channel with K antennas at the transmitter and a single antenna at each receiver. Here also the DoF collapse under non-degenerate channel uncertainty. The result directly implies a collapse of DoF to unity under non-degenerate channel uncertainty for the general K-user interference and MxN user X networks as well.

cs.IT↗

Optimum Power Allocations for Fading Decode-and-Forward Relay Channel

In this paper, for a fading decode-and-forward full-duplex relay channel, we analytically derive optimum power allocations. Individual power constraints for the source and the relay are assumed and the related optimization problem is analyzed for two scenarios. First, optimization is taken over the source power, the relay power, and the correlation coefficient between the transmitted signals of the source and the relay. Then, for a fixed value of correlation coefficient, the optimization problem is analyzed. It is also proven that the optimization problems are convex for these two scenarios. Finally, implications of theoretical results are discussed through simulations for each scenario.

cs.IT↗