The exclusive attribution of working memory (WM) retention to persistent frontal activity [1,2] has been questioned by implications of spike-silent WM [3] in early visual cortex (EVC) [4,5,6]. Spike-silent WM might partly be explained by short-term plasticity (STP) [7], where perceptual encoding leaves a neural trace that can be retrieved using perturbation methods [8,9]. However, STP traces cannot easily withstand continual visual stimulation. In contrast, $modulatory$ top-down feedback [10] (i.e., altering spiking propensity without inducing spiking directly [11]) from frontal regions to EVC might explain spike-silent WM while being robust against new inputs. We demonstrate these effects in two neural network simulations. EVC was modeled as a population of orientation-selective leaky integrate-and-fire rate neurons. We simulated neural firing rates in EVC in response to Gabor patches followed by a delay period, distraction, and ‘perturbation’ (weak current injected into each EVC neuron encoding sensory input; Fig 1$A$). In the $STP$ $model$ (Fig 1$B$), neurons' synaptic efficacy depends on firing history [7,9]: firing accumulates calcium (facilitation) but depletes neurotransmitter resources (depression). Given faster recovery of neurotransmitters than breakdown of calcium, active neurons become facilitated and increase their firing rate upon reactivation. In the $Feedback$ $model$ (Fig 1$D$), EVC consists of feedforward (EVC-FF) and feedback (EVC-FB) layers. EVC-FF encodes input and feeds into a ring attractor [12] that self-excites recurrently and connects back to EVC-FB. ECV-FB provides weak, excitatory feedback to ECV-FF, resulting in spike-silence in the absence of external drive. Orientation decoding [4] from EVC firing rates demonstrates that both STP and feedback can explain emergence of WM traces following perturbation in otherwise spike-silent neurons (Fig 1$C/E$, $left$). Presenting novel input (distractor) to EVC weakens traces maintained through STP but not feedback (Fig 1$C/E$, $right$). Although both passive STP and active feedback mechanisms can account for reactivation of WM traces from spike-silent states, STP might be too inflexible to support WM. Being purely bottom-up, STP cannot protect WM representations from the constant visual stimulation in natural environments. In contrast, top-down feedback can instill robust WM traces in EVC even in the presence of distraction, thereby allowing WM contents to guide perception.