In order to investigate the involvement of primary visual cortex (V1) in working memory (WM), parallel, multisite recordings of multiunit activity were obtained from monkey V1 while the animals performed a delayed match-to-sample (DMS) task. During the delay period, V1 population firing rate vectors maintained a lingering trace of the sample stimulus that could be reactivated by intervening impulse stimuli that enhanced neuronal firing. This fading trace of the sample did not require active engagement of the monkeys in the DMS task and was also observed in a WM-independent passive viewing task, therefore likely reflecting the intrinsic dynamics of recurrent cortical networks in lower visual areas. This renders an active, attention-dependent involvement of V1 in the maintenance of working memory contents unlikely. By contrast, population responses to the test stimulus depended on the probabilistic contingencies between sample and test stimuli, and responses to tests that matched expectations were reduced. This prior expectation-dependent firing rate difference emerged in late sessions, implying gradual learning of the probabilistic contingencies. The effect was only visible when we gradually ramped up stimulus intensity, to dampen the sharp onset transient produced by abrupt stimulus onset. Visual system likely used prior probability in WM to improve coding efficiency.