arXiv subjects

Hamed Omidvar

Publications and source records attributed to Hamed Omidvar.

5 recordsLinked to original sources

arXiv2026

A Communication-Theoretic Framework for LLM Agents: Cost-Aware Adaptive Reliability

Agents built on large language models (LLMs) rely on a range of reliability techniques, including retry, majority voting, and self-consistency, that have been developed in parallel rather than within a common analytical framework. We observe that an LLM sampled at temperature $T$ is a discrete stochastic channel $p(y \mid x)$ in the sense of Shannon's coding theory, and use this identity as the entry point for such a framework grounded in communication theory. Each of these techniques is a special case of one of six classical reliability operators: diversity combining, hybrid retransmission, iterative generator-critic decoding, rateless sampling, structured redundant verification, and difficulty-adaptive routing. Within the framework we give two closed-form results: a noise-variance threshold above which uniform averaging beats quality-weighted averaging, and a contractivity criterion for generator-critic refinement, consistent with a contractive-to-divergent transition we observe between 3B- and 14B-parameter models. We further introduce a cost-aware semantic-nearest-neighbor router whose single Lagrangian knob traverses the quality-cost frontier without retraining. Across six channel configurations spanning local and cloud models on 69 hard tasks, no fixed model-technique-budget choice dominates, motivating per-task allocation. On a 300-item hard split of MMLU, GSM8K, and HumanEval, our router occupies the full empirical Pareto frontier: at matched quality, its normalized cost is ${\approx}56$\% lower than the strongest fixed technique; at matched normalized cost, it improves quality by ${\approx}7$\% ($26$\% over single-shot decoding). These results argue for consolidating these reliability techniques into a single tunable layer informed by channel coding.

cs.LG↗

arXiv2020

Evolution and Limiting Configuration of a Long-Range Schelling-Type Spin System

We consider a long-range interacting particle system in which binary particles -- whose initial states are chosen uniformly at random -- are located at the nodes of a flat torus $(\mathbb{Z}/h\mathbb{Z})^2$. Each node of the torus is connected to all the nodes located in an $l_\infty$-ball of radius $w$ in the toroidal space centered at itself and we assume that $h$ is exponentially larger than $w^2$. Based on the states of the neighboring particles and on the value of a common intolerance threshold $τ$, every particle is labeled "stable," or "unstable." Every unstable particle that can become stable by flipping its state is labeled "p-stable." Finally, unstable particles that remained p-stable for a random, independent and identically distributed waiting time, flip their state and become stable. When the waiting times have an exponential distribution and $τ\le 1/2$, this model is equivalent to a Schelling model of self-organized segregation in an open system, a zero-temperature Ising model with Glauber dynamics, or an Asynchronous Cellular Automaton (ACA) with extended Moore neighborhoods. We first prove a shape theorem for the spreading of the "affected" nodes of a given state -- namely nodes on which a particle of a given state would be p-stable. As $w \rightarrow \infty$, this spreading starts with high probability (w.h.p.) from any $l_\infty$-ball in the torus having radius $w/2$ and containing only affected nodes, and continues for a time that is at least exponential in the cardinalilty of the neighborhood of interaction $N = (2w+1)^2$. Second, we show that when the process reaches a limiting configuration and no more state changes occur, for all ${τ\in (τ^*,1-τ^*) \setminus \{1/2\}}$ where ${τ^* \approx 0.488}$, w.h.p. any particle is contained in a large "monochromatic ball" of cardinality exponential in $N$.

math.PR↗

arXiv2019

Associative Convolutional Layers

Motivated by the necessity for parameter efficiency in distributed machine learning and AI-enabled edge devices, we provide a general and easy to implement method for significantly reducing the number of parameters of Convolutional Neural Networks (CNNs), during both the training and inference phases. We introduce a simple auxiliary neural network which can generate the convolutional filters of any CNN architecture from a low dimensional latent space. This auxiliary neural network, which we call "Convolutional Slice Generator" (CSG), is unique to the network and provides the association between its convolutional layers. During the training of the CNN, instead of training the filters of the convolutional layers, only the parameters of the CSG and their corresponding "code vectors" are trained. This results in a significant reduction of the number of parameters due to the fact that the CNN can be fully represented using only the parameters of the CSG, the code vectors, the fully connected layers, and the architecture of the CNN. We evaluate our approach by applying it to ResNet and DenseNet models when trained on CIFAR-10 and ImageNet datasets. While reducing the number of parameters by $\approx 2 \times$ on average, the accuracies of these networks remain within 1$\%$ of their original counterparts and in some cases there is an increase in the accuracy.

cs.LG↗

arXiv2018

Improved Intolerance Intervals and Size Bounds for a Schelling-Type Spin System

We consider a Schelling model of self-organized segregation in an open system that is equivalent to a zero-temperature Ising model with Glauber dynamics, or an Asynchronous Cellular Automaton (ACA) with extended Moore neighborhoods. Previous work has shown that if the intolerance parameter of the model $τ\in (\sim 0.488,\sim 0.512) \setminus \{1/2\}$, then for a sufficiently large neighborhood of interaction $N$, any particle will end up in an exponentially large monochromatic region almost surely. This paper extends the above result to the interval $τ\in (\sim 0.433,\sim 0.567) \setminus \{1/2\}$. We also improve the bounds on the size of the monochromatic region by exponential factors in $N$. Finally, we show that when particles are placed on the infinite lattice $\mathbb{Z}^2$ rather than on a flat torus, for the values of $τ$ mentioned above, sufficiently large $N$, and after a sufficiently long evolution time, any particle is contained in a large monochromatic region of size exponential in $N$, almost surely. The new proof, critically relies on a novel geometric construction related to the formation of the monochromatic region.

cs.SI↗

arXiv2018

Self-organized Segregation on the Grid

We consider an agent-based model in which two types of agents interact locally over a graph and have a common intolerance threshold $τ$ for changing their types with exponentially distributed waiting times. The model is equivalent to an unperturbed Schelling model of self-organized segregation, an Asynchronous Cellular Automata (ACA) with extended Moore neighborhoods, or a zero-temperature Ising model with Glauber dynamics, and has applications in the analysis of social and biological networks, and spin glasses systems. Some rigorous results were recently obtained in the theoretical computer science literature, and this work provides several extensions. We enlarge the intolerance interval leading to the formation of large segregated regions of agents of a single type from the known size $ε>0$ to size $\approx 0.134$. Namely, we show that for $0.433 < τ< 1/2$ (and by symmetry $1/2<τ<0.567$), the expected size of the largest segregated region containing an arbitrary agent is exponential in the size of the neighborhood. We further extend the interval leading to large segregated regions to size $\approx 0.312$ considering "almost segregated" regions, namely regions where the ratio of the number of agents of one type and the number of agents of the other type vanishes quickly as the size of the neighborhood grows. In this case, we show that for $0.344 < τ\leq 0.433$ (and by symmetry for $0.567 \leq τ<0.656$) the expected size of the largest almost segregated region containing an arbitrary agent is exponential in the size of the neighborhood. The exponential bounds that we provide also imply that complete segregation, where agents of a single type cover the whole grid, does not occur with high probability for $p=1/2$ and the range of tolerance considered.

cs.SI↗