Fitting, comparison and selection of different calmodulin kinetic schemes on a single data set using non-linear mixed effects modelling

Calmodulin (CaM) is one of the key neuronal calcium binding proteins that is essential in calcium signalling (Xia and Storm, 2005; https://doi.org/10.1038/nrn1647). Reflecting its importance, there are many different computational kinetic calcium-CaM models (Heil et al., 2018; https://doi.org/10.1101/254094). However, there has been no quantitative comparison and evaluation of these models, with some publications not reporting their training and validation performance. Moreover, there is no comparison on a common benchmark data set as is common practice in other modeling domains.We quantitatively compare how well different binding schemes, with either published parameters or parameters we find ourselves, fit the data from calcium uncaging experiments (Faas et al. 2011, https://doi.org/10.1038/nn.2746). To fit the model parameters we used the Julia programming language and the Pumas.jl software package which implements powerful training algorithms for non-linear mixed effects modeling.The published model parameters have vanishingly small relative likelihoods compared to ones we obtained from fitting to the dataset. Moreover, a kinetic scheme with independent lobes and unique, rather than identical, binding sites performed best. We conclude that more attention should be given to validation and comparison of models of individual molecules. If models composed of many molecular species are to be used in posing quantitative hypotheses, their component parts should be accurate and well-validated.