Understanding synaptic plasticity is critical for elucidating mechanisms for neural development, learning and memory consolidation. Synaptic plasticity is modelled by simulating the activity of interacting neural populations, using hypothesized functions (plasticity rules) that modify the connections between neurons. Plasticity rules are typically hand-crafted from single-synapse experiment data, but have shown limited success in understanding network-level properties. Recent studies aim to discover rules using supervised learning rather than hand-crafting them. However, these approaches still require hand-crafted loss functions. We here propose to jointly learn the plasticity rule and the loss with an unsupervised approach using generative adversarial networks (GANs). We approximate the plasticity rule (`generator' in GAN parlance) and the loss (`discriminator') with deep neural networks. The inputs to the discriminator are recorded neuron activities and simulated activities from the biological network model with the generator-based plasticity rule. We train the generator and discriminator adversarially: minimizing a cross-entropy loss to train the generator; maximizing it to train the discriminator. At convergence, we expect the generator to have learnt a plasticity rule such that the biological network model produces activity resembling the observed data, and the two are indistinguishable to the discriminator. We test our set-up on simulated data from a two-layer linear network updated using Oja's rule. We learn plasticity rules that generate qualitatively similar activities to ground-truth data. However, the learnt rules do not resemble Oja's rule. Hence, even in a simple model, a wide variety of update rules could potentially explain the same observed activity. This suggests that shifting focus away from individual plasticity rules to manifolds of rules eliciting similar network dynamics may lead to a better understanding of neural circuits and associated functions. Unsupervised data-driven methods to learn update rules may allow for a more extensive, robust exploration of synaptic plasticity mechanisms.