Perceptual memories store our experiences; they are the basis for our understanding the external world and guiding our decisions. Despite fast-paced research in this field, behavioral and cognitive constructs are often custom built around the investigators’ preferred task, and general principles across tasks are still missing. To address these issues, we performed psychophysical experiments aiming to build a computational model comprised of interconnected functional units, each performing a specific task-independent operation. The interaction between units and the readout of the system can be controlled by a top-down mechanism, depending on task-defined goals. Rats and human subjects performed each of two different tasks requiring the elaboration of vibrotactile stimuli: a categorization task, where a single stimulus must be judged (“strong” or “weak”) according to an implicit boundary, and a delayed comparison task, where two stimuli are delivered in each trial, the first of which must be stored in short-term memory to be compared to the second. We demonstrated how aspects of trial history, such as recent stimuli and recent choice outcomes, factored into each trial’s choice. Neuronal recordings from posterior parietal cortex (PPC) and medial prefrontal cortex (mPFC) in rats revealed how this network encoded task variables and carried information from one trial to the next. By extending a previously published model, we managed to describe stimulus history temporal dynamics in a task-independent manner, in both rats and humans; we also showed how the stimulus carryover in the PPC-mPFC network predicts the magnitude of the stimulus serial dependence.