arXiv subjects

Orhan Demirci

Publications and source records attributed to Orhan Demirci.

3 recordsLinked to original sources

arXiv2026

ADE: Adaptive Dictionary Embeddings -- Scaling Multi-Anchor Representations to Large Language Models

Word embeddings are fundamental to natural language processing, yet traditional approaches represent each word with a single vector, creating representational bottlenecks for polysemous words and limiting semantic expressiveness. While multi-anchor representations have shown promise by representing words as combinations of multiple vectors, they have been limited to small-scale models due to computational inefficiency and lack of integration with modern transformer architectures. We introduce Adaptive Dictionary Embeddings (ADE), a framework that successfully scales multi-anchor word representations to large language models. ADE makes three key contributions: (1) Vocabulary Projection (VP), which transforms the costly two-stage anchor lookup into a single efficient matrix operation; (2) Grouped Positional Encoding (GPE), a novel positional encoding scheme where anchors of the same word share positional information, preserving semantic coherence while enabling anchor-level variation; and (3) context-aware anchor reweighting, which leverages self-attention to dynamically compose anchor contributions based on sequence context. We integrate these components into the Segment-Aware Transformer (SAT), which provides context-aware reweighting of anchor contributions at inference time. We evaluate ADE on AG News and DBpedia-14 text classification benchmarks. With 98.7% fewer trainable parameters than DeBERTa-v3-base, ADE surpasses DeBERTa on DBpedia-14 (98.06% vs. 97.80%) and approaches it on AG News (90.64% vs. 94.50%), while compressing the embedding layer over 40x -- demonstrating that multi-anchor representations are a practical and parameter-efficient alternative to single-vector embeddings in modern transformer architectures.

cs.CL↗

arXiv2025

Tree-NET: Enhancing 2D Medical Image Segmentation Through Efficient Low-Level Feature Training

This paper introduces Tree-NET, a novel framework for medical image segmentation that leverages bottleneck supervision to enhance both segmentation accuracy and computational efficiency. While previous studies have applied bottleneck feature supervision to segmentation tasks, it has typically been limited to the training phase, offering no computational benefits during inference. To the best of our knowledge, this is the first framework to employ dual bottleneck supervision for segmentation, leveraging latent space features at both the input and output stages. This approach reduces input and label dimensions with minimal parameter overhead while preserving accuracy. Tree-NET features a three-component architecture: Encoder-Net and Decoder-Net, which compress input and label data via autoencoding, and Bridge-Net, a segmentation model trained on these compressed representations. By operating entirely on dense, low-dimensional features, Tree-NET improves runtime efficiency and can be integrated into existing segmentation models without modifying their internal structures or increasing model size. We evaluate Tree-NET on two key segmentation tasks: skin lesion and polyp segmentation using various backbone models, including U-NET, U-NET++, and Polyp-PVT. Experimental results show that Tree-NET reduces FLOPs by a factor of 4 to 13 and decreases memory usage while maintaining segmentation accuracy comparable to baseline models. For example, with an untrained U-NET++ backbone, Tree-NET improves the Dice score on ISIC 2018 from 0.829 to 0.862 and the IoU from 0.736 to 0.787. On CVC-ClinicDB, it achieves a Dice score of 0.946 and an IoU of 0.901 using a Polyp-PVT backbone, matching or surpassing baseline performance. These findings underscore Tree-NET's potential as a robust and efficient solution for medical image segmentation.

eess.IV↗

arXiv2024

When IoT Meet LLMs: Applications and Challenges

Recent advances in Large Language Models (LLMs) have positively and efficiently transformed workflows in many domains. One such domain with significant potential for LLM integration is the Internet of Things (IoT), where this integration brings new opportunities for improved decision making and system interaction. In this paper, we explore the various roles of LLMs in IoT, with a focus on their reasoning capabilities. We show how LLM-IoT integration can facilitate advanced decision making and contextual understanding in a variety of IoT scenarios. Furthermore, we explore the integration of LLMs with edge, fog, and cloud computing paradigms, and show how this synergy can optimize resource utilization, enhance real-time processing, and provide scalable solutions for complex IoT applications. To the best of our knowledge, this is the first comprehensive study covering IoT-LLM integration between edge, fog, and cloud systems. Additionally, we propose a novel system model for industrial IoT applications that leverages LLM-based collective intelligence to enable predictive maintenance and condition monitoring. Finally, we highlight key challenges and open issues that provide insights for future research in the field of LLM-IoT integration.

cs.DC↗