On March 14th, GPT-4 was released. It gave us some insight into its progress toward achieving superhuman proficiency. Nonetheless, you may feel let down by the lack of technical information on the GPT-4 model, such as the speculated model size. In fact, AI companies have become increasingly secretive about their…

Deep Learning (DL) is hard to train and reinforcement learning (RL) is much worse. In early development, follow the same strategy for DL: keep things simple! Remove any bell and whistle that get in your way and reduce the uncertainty to a minimum. In specific for the RL, For new…

Andy Rachleff, an entrepreneur and investor, famously said that you know you have a strong product-market fit when your customers are so attached to your product that they scream when you try to take it away from them. ChatGPT is a product that seems to have this kind of attachment…

We have examined many Reinforcement Learning (RL) algorithms in this series, for instance, Policy Gradient methods for the MoJoCo tasks, DQN for Atari games, and Model-based RL for the robotic controls. While many algorithms are introduced with specific domains, such ties can simply be legacy only. …

Humans are explorers and we do it smartly. In reinforcement learning RL, model-free methods search the solution space millions of times. This is hardly efficient. Humans also learn from experience which is an integral part of intelligence. We modify skills to tackle new problems. But in many deep learning or…

Imitation plays a major role in learning. In RL, it reduces the amount of time in searching for solutions and it is sample efficient. And imitation usually uses supervised learning for training which has good stability. DAgger: Dataset Aggregation DAgger is an algorithm to augment the training dataset with expert demonstrations for states…

How can we learn better? This is something we struggle with in real life also. Besides meta-learning, humans make predictions. Even if we don’t know the rules, we make predictions based on instincts. Prediction Prediction plays a major role in decision-making. Sometimes, we predict the state after taking an action. …

We learn from past experiences. We apply learned knowledge to solve new tasks. In Deep Learning, training a deep network takes a lot of time and samples. This sounds awfully inefficient.

Vision is a critical part of intelligence and the decision-making process. Many toy experiments avoid raw image processing and handcraft features to simplify the task. But for real-life tasks, such handcrafting is labor-intensive and not necessarily transferable to other tasks. Performing reinfocement learning from observing raw videos are critical. Training v.s. Testing During…