# Using tensorboard with Keras - DL
Data Science for Political and Social Phenomena
Author: Guillaume Redoulès

# Using tensorboard with Keras

## general workflow

Let's see how we can get tensorboard to work with a Keras-based Tensorflow code.

    import tensorflow as tf
    import keras
    import numpy as np

In this example, we will be using the fashion MNIST dataset to do some basic computer vision, where we will train a Keras neural network to classify items of clothing.

In order to import the data we will be using the built in function in Keras :

    keras.datasets.fashion_mnist.load_data()

The model is a very simple neural network consisting in 2 fully connected layers. The model loss function is chosen in order to have a multiclass classifier : "sparse_categorical_crossentropy"

Finally, let's train the model for 5 epochs

    ##get and preprocess the data
    fashion_mnist = keras.datasets.fashion_mnist
    (train_images, train_labels), (test_images, test_labels) = fashion_mnist.load_data()
    train_images = train_images / 255.0
    test_images = test_images / 255.0

    ## define the model

    model = keras.Sequential([
        keras.layers.Flatten(input_shape=(28,28)),
        keras.layers.Dense(128, activation="relu"),
        keras.layers.Dense(10, activation="softmax")
    ])

    model.compile(optimizer="adam",
                 loss = "sparse_categorical_crossentropy",
                 metrics = ["accuracy"])

    model.fit(train_images, train_labels, epochs=5)

    Epoch 1/5
    60000/60000 [==============================] - 9s 143us/step - loss: 0.4939 - acc: 0.8254
    Epoch 2/5
    60000/60000 [==============================] - 11s 182us/step - loss: 0.3688 - acc: 0.8661
    Epoch 3/5
    60000/60000 [==============================] - 10s 169us/step - loss: 0.3305 - acc: 0.8798
    Epoch 4/5
    60000/60000 [==============================] - 21s 350us/step - loss: 0.3079 - acc: 0.8874
    Epoch 5/5
    60000/60000 [==============================] - 18s 302us/step - loss: 0.2889 - acc: 0.8927

    <keras.callbacks.History at 0x235c1bc1be0>

During the training we can see the process, including the loss and the accuracy in the output.

    test_loss, test_acc = model.evaluate(test_images, test_labels)
    print(f"Test accuracy : {test_acc}")

    10000/10000 [==============================] - 1s 67us/step
    Test accuracy : 0.8763

When the model finishes training, we get an accuracy of about 87%, and we output some sample predictions

    predictions = model.predict(test_images)
    print(predictions[0])

    [1.8075149e-05 3.6810281e-08 6.3094416e-07 5.1111499e-07 1.6264809e-06
     3.5973577e-04 1.0840570e-06 3.1453002e-02 1.7062060e-06 9.6816361e-01]

This kind of process only gives us minimal information during the training process.

## Setting up tensorboard

To make it easier to understand, debug, and optimize TensorFlow programs, a suite of visualization tools called TensorBoard is included. You can use TensorBoard to visualize your TensorFlow graph, plot quantitative metrics about the execution of your graph, and show additional data like images that pass through it. When TensorBoard is fully configured, it looks like this:

Let's start by importing the time library and tensorboard itself. It can be found in tensorflow.python.keras.callbacks.

    from time import time
    from tensorflow.python.keras.callbacks import TensorBoard

After having imported our data and defined our model, we specify a log directory where the training information will get written to.

    #keep in mind that we already imported the data and defined the model.

    tensorboard = TensorBoard(log_dir=f"logs/{time()}")

Finally, to tell Keras to call back to TensorBoard we refer to the instant of TensorBoard we created.

    model.compile(optimizer="adam",
                  loss = "sparse_categorical_crossentropy",
                  metrics = ["accuracy"])

Now, we need to execute the TensorBoard command pointing at the log directory previously specified.

    tensorboard --logdir=logs/

TensorBoard will return a http address

    TensorBoard 1.12.0 at http://localhost:6006 (Press CTRL+C to quit)

Now, if we retrain again, we can take a look in TensorBoard and start investigating the loss and accuracy

    model.fit(train_images, train_labels, epochs=5, callbacks=[tensorboard])

    Epoch 1/5
    60000/60000 [==============================] - 41s 684us/step - loss: 0.4990 - acc: 0.8241
    Epoch 2/5
    60000/60000 [==============================] - 49s 812us/step - loss: 0.3765 - acc: 0.8648
    Epoch 3/5
    60000/60000 [==============================] - 46s 765us/step - loss: 0.3392 - acc: 0.8766
    Epoch 4/5
    60000/60000 [==============================] - 48s 794us/step - loss: 0.3135 - acc: 0.8836
    Epoch 5/5
    60000/60000 [==============================] - 49s 813us/step - loss: 0.2971 - acc: 0.8897

    <keras.callbacks.History at 0x235be1c76d8>

TensorBoard also give access to a dynamic visualization of the graph

### Find an error or bug? Have a suggestion?

Everything on this site is avaliable on GitHub. Head on over and submit an issue. You can also message me directly by email.