Image for post
Image for post
Photo by Tim Mossholder

Welcome to my articles on Deep Learning, Reinforcement Learning, Meta-Learning, and Machine Learning. The purpose of this article is to index the series and articles I wrote so far. If you want to reach me or have non-technical comments regarding the articles, please read the F.A.Q. first for the most common questions and contact methods.

Series

Not part of a series yet

NLP


Image for post
Image for post

You know, who you choose to be around you, let’s you know who you are. — The Fast and the Furious: Tokyo Drift.

What is the challenge?

In social networks, friend connections can be realized by a social graph.


Image for post
Image for post

In this article, we cover the TensorFlow generative models includes:

  • DCGAN
  • CycleGAN
  • Pix2Pix
  • Neural style transfer
  • Adversarial FGSM
  • Autoencoder
  • Autoencoder Denoising
  • Autoencoder Anomaly detection
  • Convolutional Variational Autoencoder (VAE)
  • DeepDream

DCGAN

DCGAN is one popular design for GAN. It composes of convolution and transposed convolutional layers without max pooling or fully connected layers. The figure below is the network design for the generator. This example will be trained to generate MNIST digits. We will start the article with this most basic GAN model.


Image for post
Image for post

As part of the TensorFlow series, this article focuses on coding examples on BERT and Transformer. These examples are:

  • IMDB files: Sentimental analysis with pre-trained TF Hub BERT model and AdamW,
  • GLUE/MRPC BERT finetuning,
  • Transformer for language translation.

IMDB files: Sentimental analysis with pre-trained TF Hub BERT model and AdamW

In this example, we use a pre-trained TensorFlow Hub model for BERT and an AdamW optimizer. Because most of the heavy work is done by the TF Hub model, we will keep the explanation simple for this example.

First, we download and prepare IMDB files.


Image for post
Image for post

Sequence-to-sequence models are particularly popular in NLP. This article, as part of the TensorFlow series, will cover examples for the sequence to sequence model.


Image for post
Image for post

In the last article, we present TensorFlow coding examples on computer vision. Now, we will focus on NLP classification and BERT. Here is the list of examples, tested with TF 2.4.0 released in Dec 2020. These examples are originated from the TensorFlow Tutorial.

  • IMDB files: Sentimental analysis with dataset mapping & Embedding,
  • IMDB files: TensorBoard & Sentimental analysis with Embedding & a data preprocessing layer,
  • IMDB TF Dataset: Sentimental analysis with pre-trained TF Hub Embedding,
  • IMDB TF Dataset: Sentimental analysis with Bi-directional LSTM,
  • Illiad files classification using data processing with tf.text & TextLineDataset,

IMDB files: Sentimental analysis with dataset mapping & Embedding

This example performs sentimental analysis on IMDB…


Image for post
Image for post

Reading code is one effective way to get professional in TensorFlow (TF). In this article, we reuse the examples in the TensorFlow tutorial but we keep them condense and take away codes that are for tracing or demonstration purposes. We also keep our discussion minimum so you can browse through as many examples as possible to get a complete picture. If you have problems to follow, in particular after reading the first example, please read the articles in this series first.

Yet, in most examples, we keep all the boilerplate codes required by the examples. So skip them if needed…


Image for post
Image for post

In this series, we cover major topics in deep learning coding with TensorFlow 2 — for computer vision, NLP, and generative models. In the end, we go through coding examples categorized by each type of application.


Image for post
Image for post

In this article, we discuss lower-level APIs in modeling, building, and training a model. This gives us more flexibility, like those required for GAN. We will also show examples of performing transfer learning.

Model Training with GradientTape

Keras provides training and evaluation methods, fit() and evaluate(), out-of-the-box. By declaring the optimizer and the loss function to be used, the gradient descent will be performed automatically. However, for advanced optimization algorithms, we need access to the inner mechanism of computing gradients and performing gradient descent. In TensorFlow (TF), this can be done using tf.GradientTape.

The example in this section uses a lower level APIs to…


Image for post
Image for post

Early versions of TensorFlow have major identity crises. So many APIs are proposed with split personalities. Thankfully, the adoption of the Keras API in TensorFlow 2 (TF 2) has changed the picture. More significantly, the API becomes more programmer-friendly, easier to trace, and less tedious. In this article, we show how to use this API to build deep learning models.

Sequential Model with CIFAR10

Keras API provides a high-level built-in class “models.Sequential” to create common deep learning (DL) models. In this section, we will train a CIFAR10 classifier and make predictions from it. The first part of the code below prepares the CIFAR10 dataset…

Jonathan Hui

Deep Learning

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store