Recent experiments that stimulate a small number of excitatory neurons appear to offer conflicting results regarding the role of recurrent circuitry in the mouse primary visual cortex. Marshel et al. (2019) found that stimulating 20 neurons could drive behavior, suggesting that this circuitry has strong amplifying qualities. However, Chettih and Harvey (2019) show that single-neuron perturbations drive network suppression and feature-specific competition. Here, we attempt to reconcile these results using multiphoton holographic optogenetics and mathematical modeling. Specifically, we use spatially and temporally precise 3D-SHOT to drive ensembles of ten excitatory neurons in L2/3 of mouse primary visual cortex to spike a total of exactly one hundred times. We find that most ensembles reliably drive significant network suppression. However, we also observe that individual responding neurons may be excited or inhibited depending on their physical proximity to the stimulated ensemble. Furthermore, the selection of which cells make up a stimulated ensemble impacts the inhibition observed. Close together, co-tuned ensembles recruit more inhibition than far apart ensembles. To explain these results, we create a spiking network model suited to mimicking this insertion of precisely one hundred spikes. While previous models are capable of producing a realistic salt-and-pepper orientation map through the interactions of their feedforward and recurrent connections when they have spatial connectivity rules [3], such models are unable to capture this transition of nearby excitation to nearby suppression observed experimentally. Through the use of a novel remapping algorithm, we add in feature-dependent wiring that maintains the salt-and-pepper map. This biologically-realistic network is able to recapitulate our results, and hence refine our understanding of the cortical microcircuit, reconciling the Hebbian ‘like-to-like’ connectivity, with smooth spatial rules. Further, the model calls attention to the previously undervalued trade-off occurring between the excitatory E->E and suppressive E->I->E recurrent pathways.