ePoster · Deep Learning
Stochastic Process Model derived indicators of overfitting for deep architectures: Applicability to small sample recalibration of sEMG decoders
Stephan Lehmler, Muhammad Saif-Ur-Rehman, Ioannis Iossifidis · Bernstein Conference 2024
Our recent work presents a stochastic process model of the activations within an ANN and shows a promising indicator to distinguish memorizing from generalizing ANNs. The average λ, or mean firing rate (MFR), of a hidden layer, shows stable differences between memorizing and generalizing networks, comparatively independent of the underlying data used for evaluation. We first show the performance of this indicator during training on benchmark computer vision datasets such as MNIST and CIFAR-10. In a second step, we extend the work to the real-life use case of calibrating a pre-trained model to