Pinned

A Listing of my Article Series in Deep Learning, Reinforcement Learning & Machine Learning

Welcome to my articles on Deep Learning, Reinforcement Learning, Meta-Learning, and Machine Learning. The purpose of this article is to index the series and articles I wrote so far. If you want to reach me or have non-technical comments regarding the articles, please read the F.A.Q. …

RL — Reinforcement Learning Algorithms Comparison

Choosing an RL algorithm can be confusing. In this article, we will focus on different decision factors in choosing your algorithms.

Sample efficiency — how many samples are needed to train a good policy?
Stability & convergence — how easy & how fast the model converges.
Generalization — Will the…

RL — Value Fitting & Q-Learning

We can learn the value function and the Q-value function iteratively. However, it cannot scale well to large state space. In practice, we don’t have enough memory for all the states. But we can approximate it through the model fitting. …

RL — Exploration with Deep Learning

Exploration is an art. In the high-tech world, we value bold ideas but yet we are risk-averse in reality. Releases are filled with low-hanging fruit. We keep repeating our past successes without exploring what the next success should be.

We focus on judging the ideas rather than developing them. We…

RL — Actor-Critic Methods: A3C, GAE, DDPG, Q-prop

Value-learning, including Q-learning, is believed to be unstable when a large and non-linear function approximator, like the deep network, is used. This is a curse for many reinforcement learning training (RL). To address these problems, DQN applies experience replay and the target network. However, DQN handles discrete and low-dimensional action…

Applications of Graph Neural Networks (GNN)

In the two previous articles on GCN and GNN networks, we present a design overview for Graph Neural Networks. In our final article, we will focus on GNN applications. Since the basic GNN theory is already covered by the mentioned articles, we will not repeat it here. …

Graph Neural Networks (GNN, GAE, STGNN)

In general, Graph Neural Networks (GNN) refer to the general concept of applying neural networks (NNs) on graphs. In a previous article, we cover GCN which is one of the popular approaches in GNN. But in some literature, GNN may refer to a more specific approach that the hidden state…