Fidelity and Replication: Modelling the Impact of Protocol Deviations on Effect Size
Cognitive science and cognitive neuroscience researchers have agreed that the replication of findings is important for establishing which ideas (or theories) are integral to the study of cognition across the lifespan. Recently, high-profile papers have called into question findings that were once thought to be unassailable. Much attention has been paid to how p-hacking, publication bias, and sample size are responsible for failed replications. However, much less attention has been paid to the fidelity by which researchers enact study protocols. Researchers conducting education or clinical trials are aware of the importance in fidelity – or the extent to which the protocols are delivered in the same way across participants. Nevertheless, this idea has not been applied to cognitive contexts. This seminar discusses factors that impact the replicability of findings alongside recent models suggesting that even small fidelity deviations have real impacts on the data collected.
Neuroimaging reproducibility - pain points and roadmap for solid and reusable results
There is a growing body of evidence that reproducibility or replication is low in neuroscience and in neuroimaging in particular, but the factors affecting studies solidity are still generally poorly understood, and the solutions are not clearly exposed to the neuroimaging scientific community. In this talk, I will review the key factors contributing to irreproducible results in neuroimaging specifically in the context of explanatory or prediction studies and propose a series of practical steps to improve the neuroimaging (and neuroscience) results robustness and re-usability.