Efficient Learning for Quadruped Fault Recovery
A legged robot should keep walking when an actuator degrades. Starting from MathWorks’ DDPG quadruped-locomotion example, I injected actuator signal attenuation into one leg in Simulink and compared two ways of learning to walk through it: training directly on severe faults, or a curriculum that raises fault severity step by step.
- Episodes to reach the target under severe faults
- 3,900 → 850
- Faster convergence with curriculum learning
- 4.6×
- Reduction in training episodes
- ≈ 80%
Controller
Joint torques are continuous, so I used DDPG, an off-policy actor-critic algorithm for continuous action spaces. The agent observes 44 state values and outputs 8 normalized joint torques in [−1, 1]; it learns without a hand-written kinematic model.
Training scenario
- Baseline: verify the walking performance of the official example and obtain a pre-trained agent.
- Intermediate faults: multiply the rear-right leg’s actuator signal by a random gain of 0.5–1.0.
- Severe faults: extend the random gain to 0.0–1.0.
- Compare direct transfer from the baseline with curriculum transfer through the intermediate stage.
Results
Three workstations with similar specifications were used, so episode count—not wall-clock time—is the primary comparison metric.
| Stage | Starting agent | Episodes | Training time |
|---|---|---|---|
| Gain 0.5–1.0 | Baseline | 325 | 1:18:08 |
| Gain 0.0–1.0 | Intermediate | 525 | 2:00:40 |
| Curriculum total | — | 850 | 3:18:48 |
| Gain 0.0–1.0 (direct) | Baseline | 3,900 | 16:57:10 |
What I learned along the way
- Modeling the fault with a normal random block made training fail early; the final setup draws a random gain from a fixed range instead.
- Asynchronous parallel training converged less stably than serial training in this setup, and serial runs also allowed the gait to be monitored during training.
- Simulink also made further fault scenarios easy to build: independent hip or knee failure, a complete signal cut-off after a set time, and mixed faults.
Next steps
- Simultaneous failures across multiple joints, such as hip and knee together.
- Narrowing the sim-to-real gap with realistic sensor noise and hardware constraints.
Resources
Technologies: MATLAB · Simulink · Reinforcement Learning Toolbox · Simscape Multibody