SearcharxivSearch

arXiv subjects

Adekunle Ajibode

Publications and source records attributed to Adekunle Ajibode.

6 recordsLinked to original sources

When Model Release Meets Model Reuse: Producer-Consumer Misalignment in Hugging Face

Pre-trained Language Models (PTLMs) are increasingly reused as dependencies in modern software systems, even though prior work has documented persistent structural problems in PTLM supply chains, such as inconsistent release practices, incomplete metadata, and divergence between Hugging Face and GitHub repositories. One previously unexplored angle on these problems is that misalignment between PTLM producers' release practices and consumers' model reuse needs may explain several of these challenges, yet the human expectations, processes and interactions underneath this misalignment have never been studied. As such, we conducted the first dual-perspective survey of 50 Hugging Face producers and 95 GitHub consumers to examine producer-consumer misalignments across four AI supply chain dimensions: model discovery, documentation practices, lineage tracing, and model governance adoption. We find that, although producers and consumers rely on the same documentation artifacts, they disagree on where critical metadata should be recorded. Even though 27.9% of producers and 31.4% of consumers trace lineage beyond the immediate parent model, producers primarily trace model lineage for provenance and reproducibility, whereas consumers are mainly driven by quality and reliability concerns. Furthermore, producers believe that governance mechanisms that streamline model release would have the greatest positive impact, whereas consumers prioritize documentation and dependency transparency. These findings highlight opportunities to improve model documentation conventions, lineage visibility, and governance support in PTLM supply chains.

cs.SE

Towards Imputation of Pre-Trained Language Model Metadata using Semantic Fingerprinting

Pre-trained language models (PTLMs) hosted on platforms such as Hugging Face form complex lineage structures similar to software dependency graphs. However, unlike traditional software ecosystems, PTLM repositories often lack reliable provenance due to missing metadata, such as licenses, reuse methods, pipeline tags, model types, and training libraries. To address this gap, we introduce Semantic Fingerprinting (SemFin), a lightweight approach that combines Hugging Face (HF) configuration files with model repository tags to automatically impute missing model metadata fields and reconstruct model lineage chains. We evaluate SemFin on a large-scale dataset of 317,133 PTLMs. Our results show that configuration files typically encode the technical requirements necessary to instantiate and reuse models, enabling them to serve as a structural blueprint for model reuse, particularly for transformer-based architectures. By combining these configuration files with model repository tags, SemFin significantly outperforms the existing propagation-based imputation approaches, improving prediction accuracy by up to 31.4% and 26.6% compared to Graph Avg and Hub Avg baselines. Importantly, SemFin also imputes metadata for 16.6% of isolated models where propagation-based methods fail. Applying SemFin to impute missing reuse-method and license metadata for 167,089 unlabeled models reveals that traceable reuse method chains expand by 75.9% and license lineage chains by 53.6%, uncovering 86 previously invisible reuse method patterns, while the proportion of incompatible license patterns only increases from 34.8% to 36.8%. These findings demonstrate how automatically derived structural signals can support the automated construction of AI Bills of Materials (AIBOMs), helping transform metadata from an error-prone manual declaration into information inferred directly from model artifacts.

cs.SE

On the synchronization between Hugging Face pre-trained language models and their upstream GitHub repository

Pre-trained language models (PTLMs) have transformed natural language processing (NLP), enabling major advances in tasks such as text generation and translation. Similar to software package management, PTLMs are developed using code and environment scripts hosted in upstream repositories (e.g., GitHub), while families of trained model variants are distributed through downstream platforms such as Hugging Face (HF). Despite this similarity, coordinating development and release activities across these platforms remains challenging, leading to misaligned timelines, inconsistent versioning practices, and barriers to effective reuse. To examine how commit activities are coordinated between GitHub and HF, we conducted an in-depth mixed-method study of 325 PTLM families comprising 904 HF model variants. Our findings show that GitHub contributors primarily focus on model version specification, code quality improvements, performance optimization, and dependency management, whereas HF contributors mainly address model documentation, dataset handling, and inference setup. We further analyze synchronization across three dimensions -- lag, type, and intensity -- revealing eight distinct synchronization patterns. The dominance of partially synchronized patterns, such as Disperse and Sparse synchronization, highlights structural disconnects in cross-platform release practices. These disconnects often result in isolated or abandoned updates, increasing the risk of incomplete, outdated, or behaviorally inconsistent models being exposed to end users. Recognizing these synchronization patterns is essential for improving oversight and traceability in PTLM release workflows.

cs.SE

Towards Semantic Versioning of Open Pre-trained Language Model Releases on Hugging Face

The proliferation of open Pre-trained Language Models (PTLMs) on model registry platforms like Hugging Face (HF) presents both opportunities and challenges for companies building products around them. Similar to traditional software dependencies, PTLMs continue to evolve after a release. However, the current state of release practices of PTLMs on model registry platforms are plagued by a variety of inconsistencies, such as ambiguous naming conventions and inaccessible model training documentation. Given the knowledge gap on current PTLM release practices, our empirical study uses a mixed-methods approach to analyze the releases of 52,227 PTLMs on the most well-known model registry, HF. Our results reveal 148 different naming practices for PTLM releases, with 40.87% of changes to model weight files not represented in the adopted name-based versioning practice or their documentation. In addition, we identified that the 52,227 PTLMs are derived from only 299 different base models (the modified original models used to create 52,227 PTLMs), with Fine-tuning and Quantization being the most prevalent modification methods applied to these base models. Significant gaps in release transparency, in terms of training dataset specifications and model card availability, still exist, highlighting the need for standardized documentation. While we identified a model naming practice explicitly differentiating between major and minor PTLM releases, we did not find any significant difference in the types of changes that went into either type of releases, suggesting that major/minor version numbers for PTLMs often are chosen arbitrarily. Our findings provide valuable insights to improve PTLM release practices, nudging the field towards more formal semantic versioning practices.

cs.SE

Systematic literature review on forecasting and prediction of technical debt evolution

Context: Technical debt (TD) refers to the additional costs incurred due to compromises in software quality, providing short-term advantages during development but potentially compromising long-term quality. Accurate TD forecasting and prediction are vital for informed software maintenance and proactive management. However, this research area lacks comprehensive documentation on the available forecasting techniques. Objective: This study aims to explore existing knowledge in software engineering to gain insights into approaches proposed in research and industry for forecasting TD evolution. Methods: To achieve this objective, we conducted a Systematic Literature Review encompassing 646 distinct papers published until 2023. Following established methodology in software engineering, we identified and included 14 primary studies for analysis. Result: Our analysis unveiled various approaches for TD evolution forecasting. Notably, random forest and temporal convolutional networks demonstrated superior performance compared to other methods based on the result from the primary studies. However, these approaches only address two of the fifteen identified TD types, specifically Code debt and Architecture debt, while disregarding the remaining types. Conclusion: Our findings indicate that research on TD evolution forecasting is still in its early stages, leaving numerous challenges unaddressed. Therefore, we propose several research directions that require further investigation to bridge the existing gaps. Keywords: Systematic literature review, Technical debt, Technical debt prediction, Technical debt forecasting, Technical debt metrics

cs.SE

Software issues report for bug fixing process: An empirical study of machine-learning libraries

Issue resolution and bug-fixing processes are essential in the development of machine-learning libraries, similar to software development, to ensure well-optimized functions. Understanding the issue resolution and bug-fixing process of machine-learning libraries can help developers identify areas for improvement and optimize their strategies for issue resolution and bug-fixing. However, detailed studies on this topic are lacking. Therefore, we investigated the effectiveness of issue resolution for bug-fixing processes in six machine-learning libraries: Tensorflow, Keras, Theano, Pytorch, Caffe, and Scikit-learn. We addressed seven research questions (RQs) using 16,921 issues extracted from the GitHub repository via the GitHub Rest API. We employed several quantitative methods of data analysis, including correlation, OLS regression, percentage and frequency count, and heatmap to analyze the RQs. We found the following through our empirical investigation: (1) The most common categories of issues that arise in machine-learning libraries are bugs, documentation, optimization, crashes, enhancement, new feature requests, build/CI, support, and performance. (2) Effective strategies for addressing these problems include fixing critical bugs, optimizing performance, and improving documentation. (3) These categorized issues are related to testing and runtime and are common among all six machine-learning libraries. (4) Monitoring the total number of comments on issues can provide insights into the duration of the issues. (5) It is crucial to strike a balance between prioritizing critical issues and addressing other issues in a timely manner. Therefore, this study concludes that efficient issue-tracking processes, effective communication, and collaboration are vital for effective resolution of issues and bug fixing processes in machine-learning libraries.

cs.SE