arXiv subjects

Meliksah Turker

Publications and source records attributed to Meliksah Turker.

4 recordsLinked to original sources

arXiv2024

VBART: The Turkish LLM

We present VBART, the first Turkish sequence-to-sequence Large Language Models (LLMs) pre-trained on a large corpus from scratch. VBART are compact LLMs based on good ideas leveraged from BART and mBART models and come in two sizes, Large and XLarge. Fine-tuned VBART models surpass the prior state-of-the-art results in abstractive text summarization, title generation, text paraphrasing, question answering and question generation tasks. They allow fine-tuning for future text generation tasks and datasets, carving a new path for Turkish Natural Language Processing (NLP) research. Our work shows that having a pre-trained LLM for Turkish outperforms up to 3x multilingual models, improving existing results and providing efficient models for training and inference. Moreover, we show that our monolingual tokenizer is up to 11x more efficient than multilingual tokenizers. Last but not least, we introduce a method to enlarge an existing pre-trained LLM and question the relevancy of Chinchilla Scaling Law to sequence-to-sequence masked language models. Our fine-tuned models, tokenizer and cleaned vngrs-web-corpus of 135 GB are publicly available at huggingface.co/vngrs-ai.

cs.CL↗

arXiv2024

VNLP: Turkish NLP Package

In this work, we present VNLP: the first dedicated, complete, open-source, well-documented, lightweight, production-ready, state-of-the-art Natural Language Processing (NLP) package for the Turkish language. It contains a wide variety of tools, ranging from the simplest tasks, such as sentence splitting and text normalization, to the more advanced ones, such as text and token classification models. Its token classification models are based on "Context Model", a novel architecture that is both an encoder and an auto-regressive model. NLP tasks solved by VNLP models include but are not limited to Sentiment Analysis, Named Entity Recognition, Morphological Analysis \& Disambiguation and Part-of-Speech Tagging. Moreover, it comes with pre-trained word embeddings and corresponding SentencePiece Unigram tokenizers. VNLP has an open-source GitHub repository, ReadtheDocs documentation, PyPi package for convenient installation, Python and command-line API and a demo page to test all the functionality. Consequently, our main contribution is a complete, compact, easy-to-install and easy-to-use NLP package for Turkish.

cs.CL↗

arXiv2023

Multi-layer network approach in modeling epidemics in an urban town

The last three years have been an extraordinary time with the Covid-19 pandemic killing millions, affecting and distressing billions of people worldwide. Authorities took various measures such as turning school and work to remote and prohibiting social relations via curfews. In order to mitigate the negative impact of the epidemics, researchers tried to estimate the future of the pandemic for different scenarios, using forecasting techniques and epidemics simulations on networks. Intending to better represent the real-life in an urban town in high resolution, we propose a novel multi-layer network model, where each layer corresponds to a different interaction that occurs daily, such as "household", "work" or "school". Our simulations indicate that locking down "friendship" layer has the highest impact on slowing down epidemics. Hence, our contributions are twofold, first we propose a parametric network generator model; second, we run SIR simulations on it and show the impact of layers.

cs.CY↗

arXiv2021

Forgiveness is an Adaptation in Iterated Prisoner's Dilemma with Memory

The Prisoner's Dilemma is used to represent many real life phenomena whether from the civilized world of humans or from the wild life of the other living. Researchers working on iterated prisoner's dilemma (IPD) with limited memory inspected the outcome of different forgetting strategies in homogeneous environment, within which all agents adopt the same forgetting strategy at a time. In this work, with the intention to represent real life more realistically, we improve existing forgetting strategies, offer new ones, and conduct experiments in heterogeneous environment that contains mixed agents and compare the results with previous research as well as homogeneous environment. Our findings show that the outcome depends on the type of the environment, and is just the opposite for homogeneous and heterogeneous ones, opposing the existing literature in IPD. Consequently, forgetting and forgiving defectors is the supreme memory management strategy in a competitive, heterogeneous environment. Therefore, forgiveness is an adaptation.

cs.GT↗