Numerous computational models of the cochlea have been developed that faithfully capture the transformation of incoming sound into basilar membrane motion and auditory nerve spike patterns, with widespread uptake across academia and industry. Models of sensory processing in the brain, however, are generally much less accurate, with even the best failing to explain most of the variance in sub-cortical and cortical neural activity.Here, we used large-scale neural recordings from measurement sites spanning the inferior colliculus (IC) of normal-hearing gerbils to develop a deep learning model of central auditory encoding. Our model (ICNet) takes as input a raw audio waveform and produces as output spiking patterns across hundreds of units with millisecond precision. ICNet is fast to execute and can cover a wide range of applications. It captures the full statistics of neural spiking and generalizes well to a wide range of sounds.To evaluate our model, we use sounds that were not part of the training dataset and compare the difference between simulated and recorded activity with the difference between recorded activity across repeated trials. We show that the model performs well across qualitatively different sound classes (music, speech, pure tones) when assessed by metrics such as coherence, correlation and explainable variance explained. We hope that our model will be only the first of many that are created to replace in vivo experiments across neuroscience research.Work funded by EPSRC EP/W004275/1 and MRC MR/W019787/1.