Understanding neural information processing is extremely challenging. This is because often, it is not even clear what information neurons are really processing. However, we can access the dynamic fingerprint of the processing with information-theoretic and statistical methods. Recent work has thus focused on the autocorrelation time of neural activity, also termed intrinsic timescale, which serves as a proxy for how long information is stored in the activity of the network. This work revealed a hierarchy of intrinsic timescales, which suggests that along the processing hierarchy, the brain forms higher-level representations through an enhanced and long-lasting integration or maintenance of past information. Intuitively, one could expect that an enhanced integration of past information does not only affect the timescale, but also increases the "predictability", i.e. the proportion of predictable information in neural spiking. To test this hypothesis, we estimated predictable information in highly parallel electrophysiological recordings of the mouse visual cortex. We could recover the result that the intrinsic timescale (measured both in terms of the autocorrelation time and as generalized timescale of predictable information) increases for higher cortical areas. Surprisingly, however, we found that the predictability decreases along the cortical hierarchy. Although surprising at first, this decrease in predictable information is in line with hierarchical predictive coding, where, at each processing stage, predictable information is cancelled by internal predictions. Thus, our results provide a new perspective on hierarchical processing in mouse visual cortex, where higher-level representations of inputs are formed through an enhanced and long-lasting integration of past information, which is accompanied by a predictive coding scheme to implement inference in a hierarchical internal model.