Welcome to my articles on Deep Learning, Reinforcement Learning, Meta-Learning, and Machine Learning. The purpose of this article is to index the series and articles I wrote so far. I also include a F.A.Q. for the most common questions and contact methods.
Early versions of TensorFlow have major identity crises. So many APIs are proposed with split personalities. Thankfully, the adoption of the Keras API in TensorFlow 2 (TF 2) has changed the picture. More significantly, the API becomes more programmer-friendly, easier to trace, and less tedious. In this article, we show how to use this API to build deep learning models.
Keras API provides a high-level built-in class “models.Sequential” to create common deep learning (DL) models. In this section, we will train a CIFAR10 classifier and make predictions from it. The first part of the code below prepares the CIFAR10 dataset for training. Then, we build a CNN classifier and configure a training that uses an Adam optimizer with the cross-entropy loss function (model.compile). We also configure the training to collect data on accuracy. Finally, we call “model.fit” to train the model which returns history that contains the performance metric. …
While Keras provides deep learning layers to create models, it also provides APIs to preprocessing data. For example, preprocessing.Normalization() normalizes features with the feature means and the variances of the training dataset. These parameters are not part of the trainable parameters. Instead, in line 18, we “adapt” the normalization layer to the training samples (“data”). This method will calculate the means and the variances automatically.
In this article, we discuss how to use TensorFlow (TF) Dataset to build efficient data pipelines for training and evaluation. But, if the training data is small, we can fit the data into memory and preprocess them as Numpy ndarry. However, many real-life datasets are too large. To scale the solution, we create data pipelines (tf.data.Dataset) using features like preprocessing, prefetching, and caching in scaling up the solution.
Nevertheless, toy datasets remain important in developing novel models and algorithms. So, let’s start our discussion with these small datasets, like MNIST.
Keras provides APIs in loading popular datasets into memory as NumPy ndarray. When a ndarray is passed into a TF operation, it will be converted to a TF Tensor automatically (or vice versa). If possible, TF maintains the same underlying memory representation for Numpy ndarray and Tensors. So the conversion is cheap. Nevertheless, if the Tensor is hosted in GPU, the conversion requires a data copy to the CPU host since a NumPy ndarray always has a copy in the host memory. …
Eager execution is highly promoted in TF 2. It makes coding and debugging easier. But that is not necessarily suggested for real training or production. In this article, we will talk about the two options: the eager execution mode and the graph mode, as well as its pro and con. In particular, running code in graph mode is not as seamless as it was projected. It can have significant performance impacts if some issues are ignored.
By default, TF operations in 2.x are run in eager execution mode. For example, tf.matmul (matrix multiplication) below executes immediately and returns a tf.Tensor object containing the result [[4.]]. This is what we usually expect in running Python codes. …
Keras API can perform backpropagation easily using built-in optimizers and loss functions. However, there are cases where we want to manipulate or apply gradient specifically. For example, to avoid explosive gradient, we may want to clip the gradient.
In this article, we will overview some of the key extensions and libraries in TensorFlow 2.x (TF 2.x). This will include TF Datasets, TF Hub, XLA, model optimization, TensorBoard, TF Probability, Neural Structured Learning, TF Serving, TF Federated, TF Graphics, and MLIR.
It supports the loading of many popular datasets. For a complete list, checkout out the TF dataset category. Here is the code sample for loading MNINST data.
import tensorflow as tf
import tensorflow_datasets as tfds
import tensorflow.data as tfd
# Construct a tf.data.Dataset
# Data loaded into ~/tensorflow_datasets/mnist
ds = tfds.load('mnist', split='train', shuffle_files=True)
# Build your input pipeline
ds = ds.shuffle(1024).batch(32).prefetch(tfd.experimental.AUTOTUNE)
for example in ds.take(1):
image, label = example["image"], example["label"]
assert image.shape == (32, 28, 28, 1)
assert label.shape …
CPU provides a generic set of instructions for general computing. To modify or optimize applications, we change the code but the hardware is fixed. Nevertheless, this generalization comes at the cost of complexity in hardware. Without complex hardware optimizations, like speculative execution, it hurts performance. However, these optimizations increase die size, and power consumption.