Searcharxiv⌕ Search

arXiv subjects

Simon Thel

Publications and source records attributed to Simon Thel.

2 recordsLinked to original sources

Kolibri: A Sovereign European Model on the Pareto Frontier

This report introduces Kolibri, an English-German mixture-of-experts transformer with 78.1B total parameters and 3.46B active parameters per token, released as open weights under the Apache 2.0 licence. We built Kolibri for sovereign and specialised deployment, with a particular focus on German and on regulated domains such as public administration, industry, and aerospace. To support these settings, we control the full model-development process, including data, architecture, training infrastructure, post-training, and evaluation. We train Kolibri on 24T tokens across pre-training, mid-training, and long-context extension, with German accounting for more than 20% of the data mix by bytes. To reach this share, we complement German datasets with more than 2T German tokens that we curate from the web with our own pipeline and generate synthetically. Post-training combines supervised fine-tuning on curated German and English data with reinforcement learning on more than 1.2M internally curated tasks, which cover reasoning, tool use, instruction following, code, and retrieval. The model activates 4.4% of its parameters per token, and among the open models evaluated in this report, it lies on the Pareto frontier of quality and serving cost in English and German. Kolibri builds on Kolibri Origin, an earlier iteration that validated our full pipeline at scale. Both models come from our Model Factory, which connects data, experimentation, training, evaluation, and deployment through version-controlled workflows.

cs.CL↗

Adapting Deep Variational Bayes Filter for Enhanced Confidence Estimation in Finite Element Method Integrated Networks (FEMIN)

The Finite Element Method (FEM) is a widely used technique for simulating crash scenarios with high accuracy and reliability. To reduce the significant computational costs associated with FEM, the Finite Element Method Integrated Networks (FEMIN) framework integrates neural networks (NNs) with FEM solvers. However, this integration can introduce errors and deviations from full-FEM simulations, highlighting the need for an additional metric to assess prediction confidence, especially when no ground truth data is available. In this study, we adapt the Deep Variational Bayes Filter (DVBF) to the FEMIN framework, incorporating a probabilistic approach to provide qualitative insights into prediction confidence during FEMIN simulations. The adaptation involves using the learned transition model for a predictive decoding step, generating a preliminary force prediction. This predictive force is used alongside the displacement and the velocity data from the FEM solver as input for the encoder model. The decoder reconstructs the likelihood distribution based on the posterior. The mean force of this distribution is applied to the FEM solver, while the predicted standard deviation can be used for uncertainty estimation. Our findings demonstrate that the DVBF outperforms deterministic NN architectures in terms of accuracy. Furthermore, the standard deviation derived from the decoder serves as a valuable qualitative metric for assessing the confidence in FEMIN simulations. This approach enhances the robustness of FEMIN by providing a measure of reliability alongside the simulation results.

cs.CE↗