Andy Rachleff, an entrepreneur and investor, famously said that you know you have a strong product-market fit when your customers are so attached to your product that they scream when you try to take it away from them. ChatGPT is a product that seems to have this kind of attachment…

We have examined many Reinforcement Learning (RL) algorithms in this series, for instance, Policy Gradient methods for the MoJoCo tasks, DQN for Atari games, and Model-based RL for the robotic controls. While many algorithms are introduced with specific domains, such ties can simply be legacy only. …

Humans are explorers and we do it smartly. In reinforcement learning RL, model-free methods search the solution space millions of times. This is hardly efficient. Humans also learn from experience which is an integral part of intelligence. We modify skills to tackle new problems. But in many deep learning or…

Imitation plays a major role in learning. In RL, it reduces the amount of time in searching for solutions and it is sample efficient. And imitation usually uses supervised learning for training which has good stability. DAgger: Dataset Aggregation DAgger is an algorithm to augment the training dataset with expert demonstrations for states…

How can we learn better? This is something we struggle with in real life also. Besides meta-learning, humans make predictions. Even if we don’t know the rules, we make predictions based on instincts. Prediction Prediction plays a major role in decision-making. Sometimes, we predict the state after taking an action. Otherwise…

We learn from past experiences. We apply learned knowledge to solve new tasks. In Deep Learning, training a deep network takes a lot of time and samples. This sounds awfully inefficient.

Vision is a critical part of intelligence and the decision-making process. Many toy experiments avoid raw image processing and handcraft features to simplify the task. But for real-life tasks, such handcrafting is labor-intensive and not necessarily transferable to other tasks. Performing reinfocement learning from observing raw videos are critical. Training v.s. Testing During…

In the previous article, we discuss the concept of the Guided Policy Search. Now we look into how it is trained. Partially observable MDP (POMDP) During training, we can establish a well-controlled environment to collect state information. However, to deploy the solution in real life, we don’t have such luxury.

For the past decade, we were building the Lamborghini in AI. The key focus was on the engine and the model, with fairness on the backburner. When the public and the media start the scrutiny, issues are exposed. Public images are damaged. From the previous article, we discuss how broad…