How does the motor cortex achieve generalizable and purposeful movements from the complex, nonlinear musculoskeletal system? ​​Previous research in this field has focused on developing dimensionality reduction and modeling techniques to elucidate the structure in high-dimensional neural activity, and relate this directly to kinematic outcomes. However, these models typically do not consider the biophysical underpinnings of the musculoskeletal system, nor do they allow us to understand the role of sensory feedback in motor control. These models thus fail to elucidate the computational role of neural activity in driving the musculoskeletal system such that the body reaches a desired state. Recent advances have led to vast improvements in powerful physics-based engines for efficient rigid body simulations, allowing us to efficiently simulate and analyze musculoskeletal motion. However, these techniques do not allow any insight into the neural strategies that underlie motor control, nor allow for prediction of neural strategies in novel environments. Here, we develop a neuromechanical control model using deep reinforcement learning (DRL) for a monkey limb model. We adapted an established 39-muscle anatomically accurate monkey limb model for DRL-applications and designed a maximum-entropy based actor-critic algorithm with the goal of tracking a rotating target by issuing appropriate muscle signals, resulting in the cycling motion of the limb model at different speeds. We analyzed the trained actor-network’s activity and observed high correlations and consistency with the recorded motor cortex (M1) data. Moreover, perturbations in the muscle and kinematic space led to the accurate generalization of the observed response to novel movements, and produced accurate and reasonable responses in unobserved conditions. Thus, the DRL framework for anatomically accurate limb models can mimic biologically observed neural strategies, and enables hypothesis generation for prediction and analysis of novel movements and neural strategies.