This project demonstrates how a Deep Q-Network (DQN) agent learns to land a rocket with precision on a sea platform, inspired by real-world Falcon 9 landings.
| Basic Lunar Lander | Perfect Sea Rocket |
|---|---|
| Watch | Watch |
- Built a custom rocket lander simulator with realistic physics and visuals
- Trained a DQN agent with experience replay, target networks, and ε-greedy exploration
- Achieved >200 reward, indicating stable and soft landings
- Designed PerfectSeaRocket, featuring:
- Thruster effects, segmented rocket body
- Dynamic sea background
- Landing pad with side supports and navigation lights
- Recorded and annotated full landing videos with mission phases and commentary
This project uses a Deep Q-Network (DQN) to train an autonomous agent for rocket landing. The training loop follows the standard reinforcement learning pipeline:
Training is done via experience replay and target network updates, following this algorithm:
#Example: State-Action Transition
Below is a transition sampled from the replay buffer, showing how the agent uses state features to select an action:
#Deep Q-Network Architecture
Architecture of the Q-Network and its target counterpart:
- Input: 8-dimensional state vector
- Hidden Layers: 2 × Dense(64), ReLU activation
- Output: 4-dimensional Q-values (one per action)
The agent successfully solved the Lunar Lander environment in 544 episodes, achieving a 100-episode moving average of >200 points.
- Languages/Libraries: Python, TensorFlow, Keras, Gymnasium, Pygame
- Reinforcement Learning: Deep Q-Learning (DQN)
- Visualization: Custom rendering with RGB array and video output
Due to project scope and academic policies, the full source code is not publicly released.
To request access for academic, research, or hiring purposes, please contact:
#Project Report
See full methodology and visuals in the project summary PDF.
This project builds upon the final project of the Stanford/DeepLearning.AI Deep Machine Learning Learning Specialization on Coursera.
Original inspiration: Lunar Lander with Deep Q-Learning.
All enhancements including the environment, custom rendering, video generation, and advanced RL pipeline were implemented independently as an extension of that foundational work.
To dive deeper into the algorithms behind this work, explore these foundational papers:
-
Mnih, V., Kavukcuoglu, K., Silver, D., et al. (2015).
Human-level control through deep reinforcement learning. Nature, 518(7540), 529–533. -
Lillicrap, T. P., Hunt, J. J., Pritzel, A., et al. (2016).
Continuous Control with Deep Reinforcement Learning. ICLR. -
Mnih, V., Kavukcuoglu, K., Silver, D., et al. (2013).
Playing Atari with Deep Reinforcement Learning.
#🔗 Related
Portfolio available on request