Welcome to my articles on Deep Learning, Reinforcement Learning, Meta-Learning, and Machine Learning. The purpose of this article is to index the series and articles I wrote so far. If you want to reach me or have non-technical comments regarding the articles, please read the F.A.Q. first for the most common questions and contact methods.
NLP
In this article, we cover the TensorFlow generative models includes:
DCGAN is one popular design for GAN. It composes of convolution and transposed convolutional layers without max pooling or fully connected layers. The figure below is the network design for the generator. This example will be trained to generate MNIST digits. We will start the article with this most basic GAN model.
As part of the TensorFlow series, this article focuses on coding examples on BERT and Transformer. These examples are:
In this example, we use a pre-trained TensorFlow Hub model for BERT and an AdamW optimizer. Because most of the heavy work is done by the TF Hub model, we will keep the explanation simple for this example.
First, we download and prepare IMDB files.
In the last article, we present TensorFlow coding examples on computer vision. Now, we will focus on NLP classification and BERT. Here is the list of examples, tested with TF 2.4.0 released in Dec 2020. These examples are originated from the TensorFlow Tutorial.
This example performs sentimental analysis on IMDB…
Reading code is one effective way to get professional in TensorFlow (TF). In this article, we reuse the examples in the TensorFlow tutorial but we keep them condense and take away codes that are for tracing or demonstration purposes. We also keep our discussion minimum so you can browse through as many examples as possible to get a complete picture. If you have problems to follow, in particular after reading the first example, please read the articles in this series first.
Yet, in most examples, we keep all the boilerplate codes required by the examples. So skip them if needed…
In this series, we cover major topics in deep learning coding with TensorFlow 2 — for computer vision, NLP, and generative models. In the end, we go through coding examples categorized by each type of application.
In this article, we discuss lower-level APIs in modeling, building, and training a model. This gives us more flexibility, like those required for GAN. We will also show examples of performing transfer learning.
Keras provides training and evaluation methods, fit() and evaluate(), out-of-the-box. By declaring the optimizer and the loss function to be used, the gradient descent will be performed automatically. However, for advanced optimization algorithms, we need access to the inner mechanism of computing gradients and performing gradient descent. In TensorFlow (TF), this can be done using tf.GradientTape.
The example in this section uses a lower level APIs to…
Early versions of TensorFlow have major identity crises. So many APIs are proposed with split personalities. Thankfully, the adoption of the Keras API in TensorFlow 2 (TF 2) has changed the picture. More significantly, the API becomes more programmer-friendly, easier to trace, and less tedious. In this article, we show how to use this API to build deep learning models.
Keras API provides a high-level built-in class “models.Sequential” to create common deep learning (DL) models. In this section, we will train a CIFAR10 classifier and make predictions from it. The first part of the code below prepares the CIFAR10 dataset…
Deep Learning