Searcharxiv⌕ Search

arXiv subjects

Tomasz Odrzygóźdź

Publications and source records attributed to Tomasz Odrzygóźdź.

11 recordsLinked to original sources
arXiv2026

Random groups in the square model have property (T) at densities over $\frac{5}{12}$

A random group in the square model $\mathcal{Q}(n,d)$ is given by a presentation with $n$ generators and a uniformly random set of $(2n-1)^{4d}$ cyclically reduced relators of length 4 over them. We prove that if $d>\frac{5}{12}$, then a random group in $\mathcal{Q}(n,d)$ has Kazhdan's property (T) asymptotically almost surely (a.a.s.). This provides a new construction of infinite word-hyperbolic Kazhdan groups when $d\in\left(\frac{5}{12}, \frac{1}{2}\right)$. Moreover, also for $d>\frac{5}{12}$, we show that a random group in the square model a.a.s. has property $(\textrm{F}L^p)$ for an increasing range of values of $p$, and property $(\textrm{F}_X)$ for any fixed uniformly curved Banach space $X$. These results lead to bounds on the conformal dimension of the boundary of a random group in the square model. In the process we devise a method of applying the spectral link criteria for property (T), and other fixed point properties, to the groups presented by relations of length 4. In order to use it for the square model, we verify the required spectral gap hypothesis by the trace method, in the spirit of the work of Broder-Shamir.

math.GR↗
arXiv2026

Subgoal Search For Complex Reasoning Tasks

Humans excel in solving complex reasoning tasks through a mental process of moving from one idea to a related one. Inspired by this, we propose Subgoal Search (kSubS) method. Its key component is a learned subgoal generator that produces a diversity of subgoals that are both achievable and closer to the solution. Using subgoals reduces the search space and induces a high-level search graph suitable for efficient planning. In this paper, we implement kSubS using a transformer-based subgoal module coupled with the classical best-first search framework. We show that a simple approach of generating $k$-th step ahead subgoals is surprisingly efficient on three challenging domains: two popular puzzle games, Sokoban and the Rubik's Cube, and an inequality proving benchmark INT. kSubS achieves strong results including state-of-the-art on INT within a modest computational budget.

cs.AI↗
arXiv2024

Mixture of Tokens: Continuous MoE through Cross-Example Aggregation

Mixture of Experts (MoE) models based on Transformer architecture are pushing the boundaries of language and vision tasks. The allure of these models lies in their ability to substantially increase the parameter count without a corresponding increase in FLOPs. Most widely adopted MoE models are discontinuous with respect to their parameters - often referred to as sparse. At the same time, existing continuous MoE designs either lag behind their sparse counterparts or are incompatible with autoregressive decoding. Motivated by the observation that the adaptation of fully continuous methods has been an overarching trend in deep learning, we develop Mixture of Tokens (MoT), a simple, continuous architecture that is capable of scaling the number of parameters similarly to sparse MoE models. Unlike conventional methods, MoT assigns mixtures of tokens from different examples to each expert. This architecture is fully compatible with autoregressive training and generation. Our best models not only achieve a 3x increase in training speed over dense Transformer models in language pretraining but also match the performance of state-of-the-art MoE architectures. Additionally, a close connection between MoT and MoE is demonstrated through a novel technique we call transition tuning.

cs.CL↗
arXiv2024

Fast and Precise: Adjusting Planning Horizon with Adaptive Subgoal Search

Complex reasoning problems contain states that vary in the computational cost required to determine a good action plan. Taking advantage of this property, we propose Adaptive Subgoal Search (AdaSubS), a search method that adaptively adjusts the planning horizon. To this end, AdaSubS generates diverse sets of subgoals at different distances. A verification mechanism is employed to filter out unreachable subgoals swiftly, allowing to focus on feasible further subgoals. In this way, AdaSubS benefits from the efficiency of planning with longer subgoals and the fine control with the shorter ones, and thus scales well to difficult planning problems. We show that AdaSubS significantly surpasses hierarchical planning algorithms on three complex reasoning tasks: Sokoban, the Rubik's Cube, and inequality proving benchmark INT.

cs.AI↗
arXiv2024

Scaling Laws for Fine-Grained Mixture of Experts

Mixture of Experts (MoE) models have emerged as a primary solution for reducing the computational cost of Large Language Models. In this work, we analyze their scaling properties, incorporating an expanded range of variables. Specifically, we introduce a new hyperparameter, granularity, whose adjustment enables precise control over the size of the experts. Building on this, we establish scaling laws for fine-grained MoE, taking into account the number of training tokens, model size, and granularity. Leveraging these laws, we derive the optimal training configuration for a given computational budget. Our findings not only show that MoE models consistently outperform dense Transformers but also highlight that the efficiency gap between dense and MoE models widens as we scale up the model size and training budget. Furthermore, we demonstrate that the common practice of setting the size of experts in MoE to mirror the feed-forward layer is not optimal at almost any computational budget.

cs.LG↗
arXiv2022

Thor: Wielding Hammers to Integrate Language Models and Automated Theorem Provers

In theorem proving, the task of selecting useful premises from a large library to unlock the proof of a given conjecture is crucially important. This presents a challenge for all theorem provers, especially the ones based on language models, due to their relative inability to reason over huge volumes of premises in text form. This paper introduces Thor, a framework integrating language models and automated theorem provers to overcome this difficulty. In Thor, a class of methods called hammers that leverage the power of automated theorem provers are used for premise selection, while all other tasks are designated to language models. Thor increases a language model's success rate on the PISA dataset from $39\%$ to $57\%$, while solving $8.2\%$ of problems neither language models nor automated theorem provers are able to solve on their own. Furthermore, with a significantly smaller computational budget, Thor can achieve a success rate on the MiniF2F dataset that is on par with the best existing methods. Thor can be instantiated for the majority of popular interactive theorem provers via a straightforward protocol we provide.

cs.AI↗
arXiv2019

Bent walls for random groups in the square and hexagonal model

We consider two random group models: the hexagonal model and the square model, defined as the quotient of a free group by a random set of reduced words of length four and six respectively. Our first main result is that in this model there exists a sharp density threshold for Kazhdan's Property (T) and it equals 1/3. Our second main result is that for densities < 3/8 a random group in the square model with overwhelming probability does not have Property (T). Moreover, we provide a new version of the Isoperimetric Inequality that concerns non-planar diagrams and we introduce new geometrical tools to investigate random groups: trees of loops, diagrams collared by a tree of loops and specific codimension one structures in the Cayley complex, called bent hypergraphs.

math.GR↗
arXiv2016

Cubulating random groups in the square model

Our main result is that for densities $<\frac{3}{10}$ a random group in the square model has the Haagerup property and is residually finite. Moreover, we generalize the Isoperimetric Inequality, to some class of non-planar diagrams and, using this, we introduce a system of modified hypergraphs providing the structure of a space with walls on the Cayley complex of a random group. Then we show that the natural action of a random group on this space with walls is proper, which gives the proper action of a random group on a CAT(0) cube complex.

math.GR↗
arXiv2014

The square model for random groups

We introduce a new random group model called the square model: we quotient a free group on $n$ generators by a random set of relations, each of which is a reduced word of length four. We prove, as in the Gromov density model, that for densities $> \frac{1}{2}$ a random group in the square model is trivial with overwhelming probability and for densities $<\frac{1}{2}$ a random group is with overwhelming probability hyperbolic. Moreover we show that for densities $\frac{1}{4} < d < \frac{1}{3}$ a random group in the square model does not have Property (T). Inspired by the results for the triangular model we prove that for densities $<\frac{1}{4}$ in the square model, a random group is free with overwhelming probability. We also introduce abstract diagrams with fixed edges and prove a generalization of the isoperimetric inequality.

math.GR↗