Deep Reinforcement Learning & Meta-Learning Series

Photo by Tyler Nix

Deep Reinforcement Learning is about making the best decisions for what we see and what we hear. It sounds simple but making a decision is never easy. This subject is one of the hardest and one most rewarding. I try to explain things with an easy to understand angle. I don’t want to fill my readers with fancy talks that feel good but learn nothing. In reality, simplicity makes me see through the subject in better clarity. But I don’t want to skip the equations either. It just needs to be introduced in the proper manner. Understand them helps us to go deeper.

While there are still many articles need to be reviewed before publishing, the published one should give you enough details to start your journey. For the remaining articles, I will try to release them in 2019 Spring. So stay tuned.

Overview

RL— Introduction to Deep Reinforcement Learning

RL — Deep Reinforcement Learning (Learn effectively like a human)

RL — Transfer Learning (Learn from the Past)

Value-learning

RL — Value Learning

RL — Value Fitting & Q-Learning

Monte Carlo Tree Search (MCTS) in AlphaGo Zero

Q-learning

RL — DQN Deep Q-network

Policy Gradients

RL — Policy Gradients Explained

RL — Policy Gradients Explained (Part 2)

RL — Actor-Critic Methods: A3C, GAE, DDPG, Q-prop

RL — Natural Policy Gradient Explained

RL — Trust Region Policy Optimization (TRPO) Explained

RL — Trust Region Policy Optimization (TRPO) Part 2

RL — Appendix: Proof for the article in TRPO & PPO

RL — Actor-Critic using Kronecker-Factored Trust Region (ACKTR) Explained

RL — Proximal Policy Optimization (PPO) Explained

RL — The Math behind TRPO & PPO

Model-based RL

RL — LQR & iLQR Linear Quadratic Regulator

RL — Model-based Reinforcement Learning

RL — Guided Policy Search (GPS)

RL — Guided Policy Search (A walkthrough)

RL — Model-Based Learning with Raw Videos

Technologies

RL — Imitation learning

RL — Transfer Learning

RL — Inverse Reinforcement Learning

RL — Exploration

RL — Prediction

RL — PLATO Policy Learning using Adaptive Trajectory Optimization

Comparison & Tips

RL — Reinforcement Learning Algorithms Overview

RL — Tips on Reinforcement Learning

RL — Reinforcement Learning Algorithms Comparison

Meta-learning

Meta-Learning (Learn how to Learn)

Meta-Learning (Bayesian Meta-Learning & Weak Supervision)

RL — Meta-Learning

Neural Turing Machines: a fundamental approach to access memory in deep learning

Cheat sheet

RL — Reinforcement Learning Algorithms Quick Overview

RL — Reinforcement Learning Terms

Applications

AlphaGo Zero — a game changer. (How it works?)

AlphaGo: How it works technically?

Basic

RL — Optimization Algorithms

RL — Importance Sampling

RL — Dual Gradient Descent

RL — Conjugate Gradient

Credit and references

Reinforcement learning is a huge topic and I owe a lot of debt to many professors, researchers, and bloggers. It is impossible to quote all videos, classes, research papers, and blog that I read. In fact, there are other university courses that help me a lot but I cannot recall the institutes anymore.

For here, I want to list a few that has the biggest impacts on me.

UCL reinforcement Learning

UC Berkeley Reinforcement Learning Bootcamp

Reinforcement Learning: An introduction

Nando de Freitas class

But I want to single out the UC Berkeley Reinforcement Learning course which offers every year for now. I start watching it in 2015. It is a tough course. The lesson on LQR almost made me give up RL. But with some perseverance, that makes the biggest impact on me. I hope it can have the same impact on you too.