Most walking animals can maintain motor control after limb injury or amputation, where coordinating movements of a drastically altered body requires dynamic interactions between the nervous system, the biomechanics of the body, and the physical environment. Until recently, it has been challenging to link neural and biomechanical models to investigate adaptive motor recovery because it requires coupling neural control in closed-loop with the environment. Further, while previous approaches have been able to simulate walking in biomechanically realistic bodies, the joint kinematics and ground reaction forces have been unrealistic or unvalidated. In this project, we develop and train an agent with deep reinforcement learning (DRL) to imitate real Drosophila walking using a biomechanically realistic fly body model in the physics simulator MuJoCo. For training data, we use inverse kinematics to transform high-fidelity 3D keypoint data into 36 joint angle trajectories. We show that our model closely resembles real fly walking while reproducing accurate movement dynamics (i.e. ground reaction forces). Historically, measuring forces produced by such small animals has been impossible. We validate our MuJoCo model with the first-of-their-kind measurements of ground reaction forces in freely walking fruit flies, demonstrating that simulated ground contact forces during walking closely match experimental measurements. Using our validated walking model, we simulate locomotion after a front left leg amputation, and we show that force distribution per leg during walking significantly increases in the z-direction compared to normal locomotion. More broadly, this work is a key step in using embodied agents to understand the neural mechanisms controlling robust movement with a dynamically changing body and environment.