arXiv subjects

Jonas Mayer Martins

Publications and source records attributed to Jonas Mayer Martins.

7 recordsLinked to original sources

arXiv2026

Structural priors for data-efficient language learning

Efficient language learning requires methods to reduce the reliance on large data and computational resources. We investigate structural transfer: First training models on non-language data to induce useful priors for natural language. This approach is a form of weight initialization for multilingual language modeling. We evaluate transfer via next-token-prediction loss, weight shifts in the model, and downstream linguistic benchmarks. Several symbolic data types - notably music, probabilistic grammars, and cellular automata - yield lower language-modeling loss than random initialization. These gains coincide with smaller weight shifts during subsequent language training, suggesting that structural transfer positions models in a more favorable region of the parameter space. However, a lower loss does not translate consistently into better downstream linguistic performance, and transfer from non-language data is less efficient than additional language data. We conclude that non-language data can serve as a partial substitute for language data for the training objective of next-token prediction but does not reliably support broader linguistic generalization.

cs.CL↗

arXiv2026

What makes a word hard to learn? Modeling L1 influence on English vocabulary difficulty

What makes a word difficult to learn, and how does the difficulty depend on the learner's native language? We computationally model vocabulary difficulty for English learners whose first language is Spanish, German, or Chinese with gradient-boosted models trained on features related to a word's familiarity (e.g., frequency), meaning, surface form, and cross-linguistic transfer. Using Shapley values, we determine the importance of each feature group. Word familiarity is the dominant feature group shared by all three languages. However, predictions for Spanish- and German-speaking learners rely additionally on orthographic transfer. This transfer mechanism is unavailable to Chinese learners, whose difficulty is shaped by a combination of familiarity and surface features alone. Our models provide interpretable, L1-tailored difficulty estimates that can be used to design vocabulary curricula.

cs.CL↗

arXiv2026

Vocabulary shapes cross-lingual variation of word-order learnability in language models

Why do some languages like Czech permit free word order, while others like English do not? We address this question by pretraining transformer language models on a spectrum of synthetic word-order variants of natural languages. We observe that greater word-order irregularity consistently raises model surprisal, indicating reduced learnability. Sentence reversal, however, affects learnability only weakly. A coarse distinction of free- (e.g., Czech and Finnish) and fixed-word-order languages (e.g., English and French) does not explain cross-lingual variation. Instead, the structure of the word and subword vocabulary strongly predicts the model surprisal. Overall, vocabulary structure emerges as a key driver of computational word-order learnability across languages.

cs.CL↗

arXiv2025

Active pattern formation emergent from single-species nonreciprocity

Nonreciprocal interactions violating Newton's third law are common in a plethora of nonequilibrium situations ranging from predator-prey systems to the swarming of birds and effective colloidal interactions under flow. While many recent studies have focused on two species with nonreciprocal coupling, less is examined for the basic single-component system breaking the actio and reactio equality of force within the same species. Here, we systematically derive a field theory for the case of single-species nonreciprocal interactions from the microscopic particle dynamics, leading to a generic continuum model termed Active Model N (N denoting nonreciprocity). We explore the rich dynamics of pattern formation in this nonreciprocal system and the emergence of self-traveling states with persistent variation and flowing of active branched patterns. One particular new characteristic pattern is an interwoven self-knitting "yarn" structure with a unique feature of simultaneous development of micro- and bulk phase separations. The growth dynamics of a "ball-of-wool" active droplet towards these self-knitted yarn or branched states exhibits a crossover between different scaling behaviors. The mechanism underlying this distinct class of active phase separation is attributed to the interplay between nonreciprocity and competition of interparticle forces. Our predictions can be applied to various biological and artificial active matter systems controlled by single-species nonreciprocity.

cond-mat.soft↗

arXiv2025

Once Upon a Time: Interactive Learning for Storytelling with Small Language Models

Children efficiently acquire language not just by listening, but by interacting with others in their social environment. Conversely, large language models are typically trained with next-word prediction on massive amounts of text. Motivated by this contrast, we investigate whether language models can be trained with less data by learning not only from next-word prediction but also from high-level, cognitively inspired feedback. We train a student model to generate stories, which a teacher model rates on readability, narrative coherence, and creativity. By varying the amount of pretraining before the feedback loop, we assess the impact of this interactive learning on formal and functional linguistic competence. We find that the high-level feedback is highly data efficient: With just 1 M words of input in interactive learning, storytelling skills can improve as much as with 410 M words of next-word prediction.

cs.CL↗

arXiv2024

Excitability and memory in a time-delayed optoelectronic neuron

We study the dynamics of an optoelectronic circuit composed of an excitable nanoscale resonant-tunneling diode (RTD) driving a nanolaser diode (LD) coupled via time-delayed feedback. Using a combination of numerical path-continuation methods and time simulations, we demonstrate that this RTD-LD system can serve as an artificial neuron, generating pulses in the form of temporal localized states (TLSs) that can be employed as memory for neuromorphic computing. In particular, our findings reveal that the prototypical delayed FitzHugh-Nagumo model previously employed to model the RTD-LD resembles our more realistic model only in the limit of a slow RTD. We show that the RTD time scale plays a critical role in memory capacity as it governs a shift in pulse interaction from repulsive to attractive, leading to a transition from stable to unstable multi-pulse TLSs. Our theoretical analysis uncovers features and challenges previously unknown for the RTD-LD system, including the multistability of TLSs and attractive interaction forces, stemming from the previously neglected intrinsic dynamics of the laser. These effects are crucial to consider since they define the memory properties of the RTD-LD.

nlin.PS↗

arXiv2022

Inertial dynamics of an active Brownian particle

Active Brownian motion commonly assumes spherical overdamped particles. However, self-propelled particles are often neither symmetric nor overdamped yet underlie random fluctuations from their surroundings. Active Brownian motion has already been generalized to include asymmetric particles. Separately, recent findings have shown the importance of inertial effects for particles of macroscopic size or in low-friction environments. We aim to consolidate the previous findings into the general description of a self-propelled asymmetric particle with inertia. We derive the Langevin equation of such a particle as well as the corresponding Fokker-Planck equation. Furthermore, a formula is presented that allows reconstructing the hydrodynamic resistance matrix of the particle by measuring its trajectory. Numerical solutions of the Langevin equation show that, independently of the particle's shape, the noise-free trajectory at zero temperature starts with an inertial transition phase and converges to a circular helix. We discuss this universal convergence with respect to the helical motion that many microorganisms exhibit.

cond-mat.soft↗