Transfer learning for solvation free energies: from quantum chemistry to experiments

Abstract

Data scarcity, bias, and experimental noise are all frequently encountered problems in the application of deep learning to chemical and material science disciplines. Transfer learning has proven effective in compensating for the lack in data. The use of quantum calculations in machine learning enables the generation of a diverse dataset and ensures that learning is less affected by noise inherent to experimental databases. In this work, we propose a transfer learning approach for the prediction of solvation free energies that combines fundamentals from quantum calculations with the higher accuracy of experimental measurements. The employed model architecture is based on the directed-message passing neural network for the molecular embedding of solvent and solute molecules. A significant advantage of models pre-trained on quantum calculations is demonstrated for small experimental datasets and for out-of-sample predictions. The improved out-of-sample performance is shown for new solvents, for new solute elements, and for the extension to higher molar mass solutes. The overall performance of the pre-trained models is limited by the noise in the experimental test data, known as the aleatoric uncertainty. On a random test split, a mean absolute error of 0.21 kcal/mol is achieved. This is a significant improvement compared to the mean absolute error of the quantum calculations (0.40 kcal/mol). The error can be further reduced to 0.09 kcal/mol if the model performance is assessed on a more accurate subset of the experimental data.