Botfip-LLM: An Enhanced Multimodal Scientific Computing Framework Leveraging Knowledge Distillation from Large Language Models

Abstract

In recent years, the introduction of AI technologies has brought transformative changes to scientific computing. However, AI models typically focus on single-task and single-modal data processing, limiting their application. To address this, multimodal scientific computing frameworks have become a trend. The Botfip framework aligns function images with symbolic operation trees through multimodal training, extracting deep scientific information. However, Botfip struggles with processing Formula Strings, leading to inadequate understanding in multimodal learning. To enhance Botfip's learning of Formula Strings and expand its applicability to related tasks, we propose the Botfip-LLM framework based on knowledge distillation, incorporating pre-trained large language models for aligning symbolic tree data. Experimental analysis shows that the choice of LLM is crucial, with ChatGLM-2 outperforming others in training and testing. Botfip-LLM not only improves performance, generalization, and extrapolation over the original Botfip model but also significantly enhances applicability to Formula String-related tasks, enabling more diverse task handling.