arXiv subjects

Thomas Allen

Publications and source records attributed to Thomas Allen.

6 recordsLinked to original sources

arXiv2026

The Riemann Problem for a 3x3 Generalized Chaplygin Gas System with Variable Pressure

We consider the Riemann problem for a 3x3 system of conservation laws with generalized Chaplygin pressure $p(\rho,v)=-\frac{A(v)}{\rho^\alpha}$, $0<\alpha\leq 1$, where the pressure depends on an additional transported variable. We analyze the system's wave structure and classify the Riemann solutions. Whenever classical solutions consisting of shocks, rarefaction waves, and contact discontinuities fail to exist, singular solutions arise. We verify that these satisfy the conservation laws in the distributional sense within the classical Dirac delta framework, and compare them with Nedeljkov's shadow-wave construction, giving two complementary descriptions of the same singular solution. We further study admissibility via the Dafermos maximum entropy dissipation principle, with several examples showing how it selects the physically relevant solution. Lax-Friedrichs simulations illustrate the Riemann wave patterns and provide a comparison with the analytical results. To construct viscous profiles for the isolated overcompressive $\delta$-shock, we assume $\alpha\in\mathbb{Q}\cap(0,1]$ and apply the Dafermos regularization together with a spherical blow-up. Working in three directional charts, we construct the reduced singular concatenation consisting of the left outer orbit, the middle orbit on the blown-up boundary, and the right outer orbit. We then prove that, for sufficiently small positive viscosity, this singular concatenation perturbs to a heteroclinic orbit. Consequently, the isolated overcompressive $\delta$-shock is realized as the zero-viscosity limit of a family of self-similar Dafermos viscous profiles.

math.AP↗

arXiv2025

Intrinsic Fingerprint of LLMs: Continue Training is NOT All You Need to Steal A Model!

Large language models (LLMs) face significant copyright and intellectual property challenges as the cost of training increases and model reuse becomes prevalent. While watermarking techniques have been proposed to protect model ownership, they may not be robust to continue training and development, posing serious threats to model attribution and copyright protection. This work introduces a simple yet effective approach for robust LLM fingerprinting based on intrinsic model characteristics. We discover that the standard deviation distributions of attention parameter matrices across different layers exhibit distinctive patterns that remain stable even after extensive continued training. These parameter distribution signatures serve as robust fingerprints that can reliably identify model lineage and detect potential copyright infringement. Our experimental validation across multiple model families demonstrates the effectiveness of our method for model authentication. Notably, our investigation uncovers evidence that a recently Pangu Pro MoE model released by Huawei is derived from Qwen-2.5 14B model through upcycling techniques rather than training from scratch, highlighting potential cases of model plagiarism, copyright violation, and information fabrication. These findings underscore the critical importance of developing robust fingerprinting methods for protecting intellectual property in large-scale model development and emphasize that deliberate continued training alone is insufficient to completely obscure model origins.

cs.CR↗

arXiv2024

INDUS: Effective and Efficient Language Models for Scientific Applications

Large language models (LLMs) trained on general domain corpora showed remarkable results on natural language processing (NLP) tasks. However, previous research demonstrated LLMs trained using domain-focused corpora perform better on specialized tasks. Inspired by this insight, we developed INDUS, a comprehensive suite of LLMs tailored for the closely-related domains of Earth science, biology, physics, heliophysics, planetary sciences and astrophysics, and trained using curated scientific corpora drawn from diverse data sources. The suite of models include: (1) an encoder model trained using domain-specific vocabulary and corpora to address NLP tasks, (2) a contrastive-learning based text embedding model trained using a diverse set of datasets to address information retrieval tasks and (3) smaller versions of these models created using knowledge distillation for applications which have latency or resource constraints. We also created three new scientific benchmark datasets, CLIMATE-CHANGE NER (entity-recognition), NASA-QA (extractive QA) and NASA-IR (IR) to accelerate research in these multi-disciplinary fields. We show that our models outperform both general-purpose (RoBERTa) and domain-specific (SCIBERT) encoders on these new tasks as well as existing tasks in the domains of interest. Furthermore, we demonstrate the use of these models in two industrial settings -- as a retrieval model for large-scale vector search applications and in automatic content tagging systems.

cs.CL↗

arXiv2023

Experimenting with Large Language Models and vector embeddings in NASA SciX

Open-source Large Language Models enable projects such as NASA SciX (i.e., NASA ADS) to think out of the box and try alternative approaches for information retrieval and data augmentation, while respecting data copyright and users' privacy. However, when large language models are directly prompted with questions without any context, they are prone to hallucination. At NASA SciX we have developed an experiment where we created semantic vectors for our large collection of abstracts and full-text content, and we designed a prompt system to ask questions using contextual chunks from our system. Based on a non-systematic human evaluation, the experiment shows a lower degree of hallucination and better responses when using Retrieval Augmented Generation. Further exploration is required to design new features and data augmentation processes at NASA SciX that leverages this technology while respecting the high level of trust and quality that the project holds.

cs.CL↗

arXiv2022

Improving astroBERT using Semantic Textual Similarity

The NASA Astrophysics Data System (ADS) is an essential tool for researchers that allows them to explore the astronomy and astrophysics scientific literature, but it has yet to exploit recent advances in natural language processing. At ADASS 2021, we introduced astroBERT, a machine learning language model tailored to the text used in astronomy papers in ADS. In this work we: - announce the first public release of the astroBERT language model; - show how astroBERT improves over existing public language models on astrophysics specific tasks; - and detail how ADS plans to harness the unique structure of scientific papers, the citation graph and citation context, to further improve astroBERT.

cs.CL↗

arXiv2020

A highly scalable Met Office NERC Cloud model

Large Eddy Simulation is a critical modelling tool for scientists investigating atmospheric flows, turbulence and cloud microphysics. Within the UK, the principal LES model used by the atmospheric research community is the Met Office Large Eddy Model (LEM). The LEM was originally developed in the late 1980s using computational techniques and assumptions of the time, which means that the it does not scale beyond 512 cores. In this paper we present the Met Office NERC Cloud model, MONC, which is a re-write of the existing LEM. We discuss the software engineering and architectural decisions made in order to develop a flexible, extensible model which the community can easily customise for their own needs. The scalability of MONC is evaluated, along with numerous additional customisations made to further improve performance at large core counts. The result of this work is a model which delivers to the community significant new scientific modelling capability that takes advantage of the current and future generation HPC machines.

cs.SE↗