arXiv subjects

Semih Kara

Publications and source records attributed to Semih Kara.

8 recordsLinked to original sources

arXiv2026

The Role of Feedback Alignment in Self-Distillation

Conditioning a language model on additional context, such as feedback on a previous attempt, typically improves its response. Self-distillation trains the model to retain this improvement when the context is not present. The method works by matching the model's output distribution under two settings: a student that sees only the question, and a self-teacher that also sees the context. What the model learns therefore depends on what context the self-teacher receives, yet the design of this context remains largely unexplored. We study context design for self-distillation by training a solver on feedback from a frozen critic. We compare three conditions: (i) a binary reward (GRPO), (ii) the reference solution, and (iii) a step-by-step critique aligned to the solver's reasoning trace. Step-aligned critique yields the largest gains, outperforming GRPO by 16.11 points and reference-solution-conditioned self-distillation by 5.27 points (Avg@12). Per-token advantage analysis reveals why: step-aligned feedback targets only the tokens where reasoning fails, leaving correct behavior intact. Conditioning on the reference solution, by contrast, pressures the model to change its behavior at every token (even correct steps) because an alternative derivation inevitably differs in phrasing and approach. This suggests that structural alignment between feedback and the solver's reasoning is a key driver of self-distillation effectiveness.

cs.AI↗

arXiv2025

Congestion Reduction in EV Charger Placement Using Traffic Equilibrium Models

Growing EV adoption can worsen traffic conditions if chargers are sited without regard to their impact on congestion. We study how to strategically place EV chargers to reduce congestion using two equilibrium models: one based on congestion games and one based on an atomic queueing simulation. We apply both models within a scalable greedy station-placement algorithm. Experiments show that this greedy scheme yields optimal or near-optimal congestion outcomes in realistic networks, even though global optimality is not guaranteed as we show with a counterexample. We also show that the queueing-based approach yields more realistic results than the congestion-game model, and we present a unified methodology that calibrates congestion delays from queue simulation and solves equilibrium in link-space.

eess.SY↗

arXiv2025

Sharing is Caring: Efficient LM Post-Training with Collective RL Experience Sharing

Post-training language models (LMs) with reinforcement learning (RL) can enhance their complex reasoning capabilities without supervised fine-tuning, as demonstrated by DeepSeek-R1-Zero. However, effectively utilizing RL for LMs requires significant parallelization to scale-up inference, which introduces non-trivial technical challenges (e.g. latency, memory, and reliability) alongside ever-growing financial costs. We present Swarm sAmpling Policy Optimization (SAPO), a fully decentralized and asynchronous RL post-training algorithm. SAPO is designed for decentralized networks of heterogenous compute nodes, where each node manages its own policy model(s) while "sharing" rollouts with others in the network; no explicit assumptions about latency, model homogeneity, or hardware are required and nodes can operate in silo if desired. As a result, the algorithm avoids common bottlenecks in scaling RL post-training while also allowing (and even encouraging) new possibilities. By sampling rollouts "shared" across the network, it enables "Aha moments" to propagate, thereby bootstrapping the learning process. In this paper we show SAPO achieved cumulative reward gains of up to 94% in controlled experiments. We also share insights from tests on a network with thousands of nodes contributed by Gensyn community members running the algorithm on diverse hardware and models during an open-source demo.

cs.LG↗

arXiv2025

Aggregate Fictitious Play for Learning in Anonymous Polymatrix Games (Extended Version)

Fictitious play (FP) is a well-studied algorithm that enables agents to learn Nash equilibrium in games with certain reward structures. However, when agents have no prior knowledge of the reward functions, FP faces a major challenge: the joint action space grows exponentially with the number of agents, which slows down reward exploration. Anonymous games offer a structure that mitigates this issue. In these games, the rewards depend only on the actions taken; not on who is taking which action. Under such a structure, we introduce aggregate fictitious play (agg-FP), a variant of FP where each agent tracks the frequency of the number of other agents playing each action, rather than these agents' individual actions. We show that in anonymous polymatrix games, agg-FP converges to a Nash equilibrium under the same conditions as classical FP. In essence, by aggregating the agents' actions, we reduce the action space without losing the convergence guarantees. Using simulations, we provide empirical evidence on how this reduction accelerates convergence.

cs.GT↗

arXiv2025

NoLoCo: No-all-reduce Low Communication Training Method for Large Models

Training large language models is generally done on clusters containing thousands of accelerators, communicating over a high-bandwidth interconnect. Scaling up these clusters is expensive and can become impractical, imposing limits on the size of models that can be trained. Several recent studies have proposed training methods that are less communication intensive, avoiding the need for compute clusters with extremely high interconnect speeds. These low communication training methods still employ a global synchronization step for model parameters, which can be too costly with a high number of participants, as the communication cost scales quadratically with group size. In this work, we propose a novel optimization method, NoLoCo, that does not explicitly synchronize all model parameters during training and does not require any collective communication. NoLoCo implicitly synchronizes model weights via a novel variant of the Nesterov momentum optimizer by partially averaging model weights within randomly selected subgroups. We provide both a theoretical convergence analysis of our optimizer and empirical results from language model training. Our method requires significantly less communication than fully sharded data parallel training and DiLoCo, a widely used low-communication baseline. Moreover, our method avoids global blocking communication, thereby reducing accelerator idle time. Our experiments show that NoLoCo is more communication-efficient than DiLoCo, improving final perplexity by up to $4\%$ and converging up to $4\times$ faster in wall-clock time across a range of worker counts, model sizes, and communication bandwidths.

cs.LG↗

arXiv2023

Differential Equation Approximations for Population Games using Elementary Probability

Population games model the evolution of strategic interactions among a large number of uniform agents. Due to the agents' uniformity and quantity, their aggregate strategic choices can be approximated by the solutions of a class of ordinary differential equations. This mean-field approach has found to be an effective tool of analysis. However its current proofs rely on advanced mathematical techniques, making them less accessible. In this article, we present a simpler derivation, using only undergraduate-level probability.

cs.GT↗

arXiv2022

Population Games With Erlang Clocks: Convergence to Nash Equilibria For Pairwise Comparison Dynamics

The prevailing methodology for analyzing population games and evolutionary dynamics in the large population limit assumes that a Poisson process (or clock) inherent to each agent determines when the agent can revise its strategy. Hence, such an approach presupposes exponentially distributed inter-revision intervals, and is inadequate for cases where each strategy entails a sequence of sub-tasks (sub-strategies) that must be completed before a new revision time occurs. This article proposes a methodology for such cases under the premise that a sub-strategy's duration is exponentially-distributed, leading to Erlang distributed inter-revision intervals. We assume that a so-called pairwise-comparison protocol captures the agents' revision preferences to render our analysis concrete. The presence of sub-strategies brings on additional dynamics that is incompatible with existing models and results. Our main contributions are twofold, both derived for a deterministic approximation valid for large populations. We prove convergence of the population's state to the Nash equilibrium set when a potential game generates a payoff for the strategies. We use system-theoretic passivity to determine conditions under which this convergence is guaranteed for contractive games.

eess.SY↗

arXiv2021

Pairwise Comparison Evolutionary Dynamics with Strategy-Dependent Revision Rates: Stability and Delta-Passivity (Expanded Version)

We report on new stability conditions for evolutionary dynamics in the context of population games. We adhere to the prevailing framework consisting of many agents, grouped into populations, that interact noncooperatively by selecting strategies with a favorable payoff. Each agent is repeatedly allowed to revise its strategy at a rate referred to as revision rate. Previous stability results considered either that the payoff mechanism was a memoryless potential game, or allowed for dynamics (in the payoff mechanism) at the expense of precluding any explicit dependence of the agents' revision rates on their current strategies. Allowing the dependence of revision rates on strategies is relevant because the agents' strategies at any point in time are generally unequal. To allow for strategy-dependent revision rates and payoff mechanisms that are dynamic (or memoryless games that are not potential), we focus on an evolutionary dynamics class obtained from a straightforward modification of one that stems from the so-called impartial pairwise comparison strategy revision protocol. Revision protocols consistent with the modified class retain from those in the original one the advantage that the agents operate in a fully decentralized manner and with minimal information requirements - they need to access only the payoff values (not the mechanism) of the available strategies. Our main results determine conditions under which system-theoretic passivity properties are assured, which we leverage for stability analysis.

q-bio.PE↗