LLM-assisted Agentic Edge Intelligence Framework

Abstract

Edge intelligence delivers low-latency inference, yet most edge analytics remain hard-coded and must be redeployed as conditions change. When data patterns shift or new questions arise, engineers often need to write new scripts and push updates to devices, which slows iteration and raises operating costs. This limited adaptability reduces scalability and autonomy in large, heterogeneous, and resource-constrained edge deployments, and it increases reliance on human oversight. Meanwhile, large language models (LLMs) can interpret instructions and generate code, but their compute and memory requirements typically prevent direct deployment on edge devices. We address this gap with the LLM-assisted Edge Intelligence (LEI) framework, which removes the need for manually specified business logic. In LEI, a cloud-hosted LLM coordinates the creation and update of device-side logic as requirements evolve. The system generates candidate lightweight programs, checks them against available data and constraints, and then deploys the selected version to each device. This lets each device receive a tailored program based on sample data, metadata, context, and current resource limits. We evaluate LEI on four heterogeneous datasets, including air quality, temperature \& humidity, wind, and soil datasets using multiple LLM backends. The experimental results show that the framework maintains low average CPU and memory utilization during the execution. These results indicate that the framework adapts efficiently to changing conditions while maintaining resource efficiency.