Artificial neural networks are frequently used to model neural systems, and have been shown to recapitulate features of biological networks, including in the hippocampus [1] and visual cortex [2] [3]. However, despite it’s success in neural modelling, the predominant learning algorithm, backpropagation, has long been considered biologically implausible [4]. One reason is the weight-transport problem - or the problem of how a global error signal can be accurately transmitted across the network to minimize the error resulting from each layer’s parameters. Previous solutions to the weight-transport problem have proposed using a separate, error-feedback network to transport an error signal across layers [5]. However, more recent experimental work indicates that cortical feedback connections are more likely to be involved in reconstructing lower-level activity based on activity from higher layers, rather than being exclusively used to transmit top-level error [6] [7] [8]. Under the predictive coding framework, the bidirectional, cortical processing hierarchy is more appropriately modeled as a multi-layered autoencoder [9]. Here, we attempt to unify these two views by showing how autoencoder-like, inverse feedback connections may be used to minimize top-level error in neural networks. Our proposed mechanism, Reciprocal Feedback, consists of two contributions: first we show how a modification of the Recirculation autoencoder algorithm [10] is equivalent to learning the Moore-Penrose pseudoinverse. Then, we will show how, using a Newton-like method [11], locally-learned pseudoinverse feedback connections may be used to facilitate an alternative, more biologically-realistic optimization method to gradient descent, by relying on the reciprocal of the forward network rather than its gradient. Overall, we provide a mathematical framework for understanding how the hierarchical, autoencoder-like feedback connections observed in the layers of the cortex may additionally be used as a mechanism for minimizing a global error signal, using only local activity.