Book a Demo!
CoCalc Logo Icon
StoreFeaturesDocsShareSupportNewsAboutPoliciesSign UpSign In
codebasics
GitHub Repository: codebasics/deep-learning-keras-tf-tutorial
 Path: blob/master/13_dropout_layer/dropout_regularization_ann.ipynb
1141 views
Kernel: Python 3

Dropout Regularization In Deep Neural Network

This is a dataset that describes sonar chirp returns bouncing off different services. The 60 input variables are the strength of the returns at different angles. It is a binary classification problem that requires a model to differentiate rocks from metal cylinders.

Dataset information: https://archive.ics.uci.edu/ml/datasets/Connectionist+Bench+(Sonar,+Mines+vs.+Rocks) Download it from here: https://archive.ics.uci.edu/ml/machine-learning-databases/undocumented/connectionist-bench/sonar/sonar.all-data

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns

import warnings
warnings.filterwarnings('ignore')

df = pd.read_csv("./sonar_dataset.csv", header=None)
df.sample(5)

df.shape
(208, 61)
# check for nan values
df.isna().sum()
0 0 1 0 2 0 3 0 4 0 .. 56 0 57 0 58 0 59 0 60 0 Length: 61, dtype: int64
df.columns
Int64Index([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60], dtype='int64')
df[60].value_counts() # label is not skewed
M 111 R 97 Name: 60, dtype: int64
X = df.drop(60, axis=1)
y = df[60]
y.head()
0 R 1 R 2 R 3 R 4 R Name: 60, dtype: object
y = pd.get_dummies(y, drop_first=True)
y.sample(5) # R --> 1 and M --> 0

y.value_counts()
R 0 111 1 97 dtype: int64
X.head()

from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=1)

X_train.head()

Using Deep Learning Model

Model without Dropout Layer

import tensorflow as tf
from tensorflow import keras

model = keras.Sequential([
    keras.layers.Dense(60, input_dim=60, activation='relu'),
    keras.layers.Dense(30, activation='relu'),
    keras.layers.Dense(15, activation='relu'),
    keras.layers.Dense(1, activation='sigmoid')
])

model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

model.fit(X_train, y_train, epochs=100, batch_size=8)
Epoch 1/100 20/20 [==============================] - 0s 2ms/step - loss: 0.6895 - accuracy: 0.5321 Epoch 2/100 20/20 [==============================] - 0s 2ms/step - loss: 0.6669 - accuracy: 0.5833 Epoch 3/100 20/20 [==============================] - 0s 1ms/step - loss: 0.6512 - accuracy: 0.6474 Epoch 4/100 20/20 [==============================] - 0s 2ms/step - loss: 0.6189 - accuracy: 0.6923 Epoch 5/100 20/20 [==============================] - 0s 1ms/step - loss: 0.5780 - accuracy: 0.7564 Epoch 6/100 20/20 [==============================] - 0s 1ms/step - loss: 0.5420 - accuracy: 0.8141 Epoch 7/100 20/20 [==============================] - 0s 1ms/step - loss: 0.5194 - accuracy: 0.7756 Epoch 8/100 20/20 [==============================] - 0s 1ms/step - loss: 0.4909 - accuracy: 0.7885 Epoch 9/100 20/20 [==============================] - 0s 1ms/step - loss: 0.4450 - accuracy: 0.8013 Epoch 10/100 20/20 [==============================] - 0s 1ms/step - loss: 0.4145 - accuracy: 0.8462 Epoch 11/100 20/20 [==============================] - 0s 1ms/step - loss: 0.3802 - accuracy: 0.8333 Epoch 12/100 20/20 [==============================] - 0s 1ms/step - loss: 0.3669 - accuracy: 0.8590 Epoch 13/100 20/20 [==============================] - 0s 1ms/step - loss: 0.3465 - accuracy: 0.8654 Epoch 14/100 20/20 [==============================] - 0s 2ms/step - loss: 0.3427 - accuracy: 0.8397 Epoch 15/100 20/20 [==============================] - 0s 1ms/step - loss: 0.3336 - accuracy: 0.8654 Epoch 16/100 20/20 [==============================] - 0s 1ms/step - loss: 0.3140 - accuracy: 0.8846 Epoch 17/100 20/20 [==============================] - 0s 1ms/step - loss: 0.3288 - accuracy: 0.8397 Epoch 18/100 20/20 [==============================] - 0s 1ms/step - loss: 0.2863 - accuracy: 0.9038 Epoch 19/100 20/20 [==============================] - 0s 1ms/step - loss: 0.2882 - accuracy: 0.9038 Epoch 20/100 20/20 [==============================] - 0s 1ms/step - loss: 0.2654 - accuracy: 0.9231 Epoch 21/100 20/20 [==============================] - 0s 1ms/step - loss: 0.2573 - accuracy: 0.8974 Epoch 22/100 20/20 [==============================] - 0s 1ms/step - loss: 0.2967 - accuracy: 0.8846 Epoch 23/100 20/20 [==============================] - 0s 1ms/step - loss: 0.2742 - accuracy: 0.9038 Epoch 24/100 20/20 [==============================] - 0s 1ms/step - loss: 0.2375 - accuracy: 0.9038 Epoch 25/100 20/20 [==============================] - 0s 1ms/step - loss: 0.2559 - accuracy: 0.9103 Epoch 26/100 20/20 [==============================] - 0s 1ms/step - loss: 0.2348 - accuracy: 0.9103 Epoch 27/100 20/20 [==============================] - 0s 1ms/step - loss: 0.2922 - accuracy: 0.8718 Epoch 28/100 20/20 [==============================] - 0s 1ms/step - loss: 0.2213 - accuracy: 0.9167 Epoch 29/100 20/20 [==============================] - 0s 1ms/step - loss: 0.2142 - accuracy: 0.9103 Epoch 30/100 20/20 [==============================] - 0s 1ms/step - loss: 0.1910 - accuracy: 0.9167 Epoch 31/100 20/20 [==============================] - 0s 1ms/step - loss: 0.1929 - accuracy: 0.9103 Epoch 32/100 20/20 [==============================] - 0s 1ms/step - loss: 0.2011 - accuracy: 0.9359 Epoch 33/100 20/20 [==============================] - 0s 1ms/step - loss: 0.1888 - accuracy: 0.9103 Epoch 34/100 20/20 [==============================] - 0s 1ms/step - loss: 0.1721 - accuracy: 0.9423 Epoch 35/100 20/20 [==============================] - 0s 1ms/step - loss: 0.1699 - accuracy: 0.9423 Epoch 36/100 20/20 [==============================] - 0s 1ms/step - loss: 0.1504 - accuracy: 0.9487 Epoch 37/100 20/20 [==============================] - 0s 1ms/step - loss: 0.1447 - accuracy: 0.9551 Epoch 38/100 20/20 [==============================] - 0s 1ms/step - loss: 0.1388 - accuracy: 0.9615 Epoch 39/100 20/20 [==============================] - 0s 1ms/step - loss: 0.1512 - accuracy: 0.9423 Epoch 40/100 20/20 [==============================] - 0s 1ms/step - loss: 0.1380 - accuracy: 0.9551 Epoch 41/100 20/20 [==============================] - 0s 1ms/step - loss: 0.1287 - accuracy: 0.9615 Epoch 42/100 20/20 [==============================] - 0s 1ms/step - loss: 0.1336 - accuracy: 0.9551 Epoch 43/100 20/20 [==============================] - 0s 1ms/step - loss: 0.1299 - accuracy: 0.9551 Epoch 44/100 20/20 [==============================] - 0s 1ms/step - loss: 0.1019 - accuracy: 0.9679 Epoch 45/100 20/20 [==============================] - 0s 1ms/step - loss: 0.1005 - accuracy: 0.9872 Epoch 46/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0891 - accuracy: 0.9744 Epoch 47/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0912 - accuracy: 0.9744 Epoch 48/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0860 - accuracy: 0.9744 Epoch 49/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0740 - accuracy: 0.9872 Epoch 50/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0779 - accuracy: 0.9808 Epoch 51/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0629 - accuracy: 0.9936 Epoch 52/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0605 - accuracy: 0.9872 Epoch 53/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0540 - accuracy: 0.9872 Epoch 54/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0493 - accuracy: 0.9936 Epoch 55/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0537 - accuracy: 0.9936 Epoch 56/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0506 - accuracy: 0.9872 Epoch 57/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0512 - accuracy: 0.9936 Epoch 58/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0435 - accuracy: 0.9936 Epoch 59/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0396 - accuracy: 0.9936 Epoch 60/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0395 - accuracy: 0.9936 Epoch 61/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0355 - accuracy: 1.0000 Epoch 62/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0268 - accuracy: 1.0000 Epoch 63/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0310 - accuracy: 1.0000 Epoch 64/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0255 - accuracy: 1.0000 Epoch 65/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0297 - accuracy: 1.0000 Epoch 66/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0263 - accuracy: 1.0000 Epoch 67/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0224 - accuracy: 1.0000 Epoch 68/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0198 - accuracy: 1.0000 Epoch 69/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0169 - accuracy: 1.0000 Epoch 70/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0149 - accuracy: 1.0000 Epoch 71/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0146 - accuracy: 1.0000 Epoch 72/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0164 - accuracy: 1.0000 Epoch 73/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0128 - accuracy: 1.0000 Epoch 74/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0130 - accuracy: 1.0000 Epoch 75/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0205 - accuracy: 1.0000 Epoch 76/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0210 - accuracy: 1.0000 Epoch 77/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0109 - accuracy: 1.0000 Epoch 78/100 20/20 [==============================] - 0s 2ms/step - loss: 0.0095 - accuracy: 1.0000 Epoch 79/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0126 - accuracy: 1.0000 Epoch 80/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0111 - accuracy: 1.0000 Epoch 81/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0089 - accuracy: 1.0000 Epoch 82/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0076 - accuracy: 1.0000 Epoch 83/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0071 - accuracy: 1.0000 Epoch 84/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0067 - accuracy: 1.0000 Epoch 85/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0064 - accuracy: 1.0000 Epoch 86/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0061 - accuracy: 1.0000 Epoch 87/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0063 - accuracy: 1.0000 Epoch 88/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0057 - accuracy: 1.0000 Epoch 89/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0057 - accuracy: 1.0000 Epoch 90/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0053 - accuracy: 1.0000 Epoch 91/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0055 - accuracy: 1.0000 Epoch 92/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0049 - accuracy: 1.0000 Epoch 93/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0043 - accuracy: 1.0000 Epoch 94/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0043 - accuracy: 1.0000 Epoch 95/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0040 - accuracy: 1.0000 Epoch 96/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0041 - accuracy: 1.0000 Epoch 97/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0040 - accuracy: 1.0000 Epoch 98/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0040 - accuracy: 1.0000 Epoch 99/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0039 - accuracy: 1.0000 Epoch 100/100 20/20 [==============================] - 0s 1ms/step - loss: 0.0039 - accuracy: 1.0000
<tensorflow.python.keras.callbacks.History at 0x21797a18070>
model.evaluate(X_test, y_test)
2/2 [==============================] - 0s 1ms/step - loss: 0.8629 - accuracy: 0.7692
[0.8629363775253296, 0.7692307829856873]

Training Accuracy >>> Test Accuracy

y_pred = model.predict(X_test).reshape(-1)
print(y_pred[:10])

# round the values to nearest integer ie 0 or 1
y_pred = np.round(y_pred)
print(y_pred[:10])
[7.1083555e-08 3.7778303e-01 9.9051148e-01 2.6755422e-05 9.9999976e-01 9.9999321e-01 6.4399661e-03 9.9999988e-01 9.1076899e-06 9.9999988e-01] [0. 0. 1. 0. 1. 1. 0. 1. 0. 1.] 
y_test[:10]

from sklearn.metrics import confusion_matrix , classification_report

print(classification_report(y_test, y_pred))
precision recall f1-score support 0 0.71 0.93 0.81 27 1 0.88 0.60 0.71 25 accuracy 0.77 52 macro avg 0.80 0.76 0.76 52 weighted avg 0.80 0.77 0.76 52

Model with Dropout Layer

modeld = keras.Sequential([
    keras.layers.Dense(60, input_dim=60, activation='relu'),
    keras.layers.Dropout(0.5),
    keras.layers.Dense(30, activation='relu'),
    keras.layers.Dropout(0.5),
    keras.layers.Dense(15, activation='relu'),
    keras.layers.Dropout(0.5),
    keras.layers.Dense(1, activation='sigmoid')
])

modeld.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

modeld.fit(X_train, y_train, epochs=100, batch_size=8)
Epoch 1/100 20/20 [==============================] - 0s 2ms/step - loss: 0.7653 - accuracy: 0.4744 Epoch 2/100 20/20 [==============================] - 0s 2ms/step - loss: 0.6923 - accuracy: 0.5256 Epoch 3/100 20/20 [==============================] - 0s 2ms/step - loss: 0.7149 - accuracy: 0.4872 Epoch 4/100 20/20 [==============================] - 0s 2ms/step - loss: 0.7439 - accuracy: 0.4423 Epoch 5/100 20/20 [==============================] - 0s 2ms/step - loss: 0.6974 - accuracy: 0.5064 Epoch 6/100 20/20 [==============================] - 0s 2ms/step - loss: 0.7053 - accuracy: 0.5000 Epoch 7/100 20/20 [==============================] - 0s 2ms/step - loss: 0.7035 - accuracy: 0.4808 Epoch 8/100 20/20 [==============================] - 0s 2ms/step - loss: 0.6994 - accuracy: 0.4679 Epoch 9/100 20/20 [==============================] - 0s 2ms/step - loss: 0.6984 - accuracy: 0.5385 Epoch 10/100 20/20 [==============================] - 0s 2ms/step - loss: 0.6833 - accuracy: 0.5000 Epoch 11/100 20/20 [==============================] - 0s 2ms/step - loss: 0.6910 - accuracy: 0.4679 Epoch 12/100 20/20 [==============================] - 0s 2ms/step - loss: 0.6803 - accuracy: 0.5641 Epoch 13/100 20/20 [==============================] - 0s 2ms/step - loss: 0.6887 - accuracy: 0.5385 Epoch 14/100 20/20 [==============================] - 0s 1ms/step - loss: 0.6772 - accuracy: 0.5769 Epoch 15/100 20/20 [==============================] - 0s 2ms/step - loss: 0.6914 - accuracy: 0.5064 Epoch 16/100 20/20 [==============================] - 0s 1ms/step - loss: 0.6757 - accuracy: 0.6026 Epoch 17/100 20/20 [==============================] - 0s 2ms/step - loss: 0.6685 - accuracy: 0.5962 Epoch 18/100 20/20 [==============================] - 0s 2ms/step - loss: 0.6687 - accuracy: 0.5897 Epoch 19/100 20/20 [==============================] - 0s 2ms/step - loss: 0.6872 - accuracy: 0.5192 Epoch 20/100 20/20 [==============================] - 0s 2ms/step - loss: 0.6616 - accuracy: 0.6346 Epoch 21/100 20/20 [==============================] - 0s 2ms/step - loss: 0.6522 - accuracy: 0.6346 Epoch 22/100 20/20 [==============================] - 0s 2ms/step - loss: 0.6605 - accuracy: 0.5833 Epoch 23/100 20/20 [==============================] - 0s 1ms/step - loss: 0.6714 - accuracy: 0.5897 Epoch 24/100 20/20 [==============================] - 0s 2ms/step - loss: 0.6519 - accuracy: 0.6538 Epoch 25/100 20/20 [==============================] - 0s 2ms/step - loss: 0.6577 - accuracy: 0.6282 Epoch 26/100 20/20 [==============================] - 0s 2ms/step - loss: 0.6252 - accuracy: 0.6987 Epoch 27/100 20/20 [==============================] - 0s 2ms/step - loss: 0.6241 - accuracy: 0.6603 Epoch 28/100 20/20 [==============================] - 0s 2ms/step - loss: 0.6093 - accuracy: 0.6731 Epoch 29/100 20/20 [==============================] - 0s 2ms/step - loss: 0.6306 - accuracy: 0.6538 Epoch 30/100 20/20 [==============================] - 0s 1ms/step - loss: 0.6008 - accuracy: 0.6923 Epoch 31/100 20/20 [==============================] - 0s 2ms/step - loss: 0.5760 - accuracy: 0.7500 Epoch 32/100 20/20 [==============================] - 0s 2ms/step - loss: 0.6171 - accuracy: 0.6603 Epoch 33/100 20/20 [==============================] - 0s 2ms/step - loss: 0.6315 - accuracy: 0.6603 Epoch 34/100 20/20 [==============================] - 0s 2ms/step - loss: 0.5704 - accuracy: 0.6859 Epoch 35/100 20/20 [==============================] - 0s 2ms/step - loss: 0.5848 - accuracy: 0.6859 Epoch 36/100 20/20 [==============================] - 0s 2ms/step - loss: 0.5785 - accuracy: 0.6987 Epoch 37/100 20/20 [==============================] - 0s 2ms/step - loss: 0.5493 - accuracy: 0.7372 Epoch 38/100 20/20 [==============================] - 0s 2ms/step - loss: 0.5312 - accuracy: 0.7500 Epoch 39/100 20/20 [==============================] - 0s 2ms/step - loss: 0.5531 - accuracy: 0.7308 Epoch 40/100 20/20 [==============================] - 0s 2ms/step - loss: 0.5497 - accuracy: 0.7500 Epoch 41/100 20/20 [==============================] - 0s 2ms/step - loss: 0.5843 - accuracy: 0.7372 Epoch 42/100 20/20 [==============================] - 0s 1ms/step - loss: 0.5336 - accuracy: 0.7372 Epoch 43/100 20/20 [==============================] - 0s 2ms/step - loss: 0.4994 - accuracy: 0.7628 Epoch 44/100 20/20 [==============================] - 0s 2ms/step - loss: 0.5225 - accuracy: 0.7756 Epoch 45/100 20/20 [==============================] - 0s 2ms/step - loss: 0.5242 - accuracy: 0.7436 Epoch 46/100 20/20 [==============================] - 0s 2ms/step - loss: 0.4783 - accuracy: 0.7692 Epoch 47/100 20/20 [==============================] - 0s 2ms/step - loss: 0.5040 - accuracy: 0.7564 Epoch 48/100 20/20 [==============================] - 0s 2ms/step - loss: 0.5070 - accuracy: 0.7692 Epoch 49/100 20/20 [==============================] - 0s 2ms/step - loss: 0.4904 - accuracy: 0.8013 Epoch 50/100 20/20 [==============================] - 0s 2ms/step - loss: 0.4318 - accuracy: 0.8333 Epoch 51/100 20/20 [==============================] - 0s 2ms/step - loss: 0.4261 - accuracy: 0.8013 Epoch 52/100 20/20 [==============================] - 0s 2ms/step - loss: 0.5395 - accuracy: 0.7436 Epoch 53/100 20/20 [==============================] - 0s 2ms/step - loss: 0.4932 - accuracy: 0.7564 Epoch 54/100 20/20 [==============================] - 0s 2ms/step - loss: 0.4554 - accuracy: 0.7949 Epoch 55/100 20/20 [==============================] - 0s 2ms/step - loss: 0.5198 - accuracy: 0.7500 Epoch 56/100 20/20 [==============================] - 0s 2ms/step - loss: 0.4238 - accuracy: 0.8205 Epoch 57/100 20/20 [==============================] - 0s 2ms/step - loss: 0.4558 - accuracy: 0.7821 Epoch 58/100 20/20 [==============================] - 0s 2ms/step - loss: 0.4618 - accuracy: 0.7756 Epoch 59/100 20/20 [==============================] - 0s 2ms/step - loss: 0.4335 - accuracy: 0.7949 Epoch 60/100 20/20 [==============================] - 0s 2ms/step - loss: 0.4415 - accuracy: 0.8141 Epoch 61/100 20/20 [==============================] - 0s 2ms/step - loss: 0.3873 - accuracy: 0.8333 Epoch 62/100 20/20 [==============================] - 0s 2ms/step - loss: 0.4032 - accuracy: 0.8269 Epoch 63/100 20/20 [==============================] - 0s 2ms/step - loss: 0.3989 - accuracy: 0.8141 Epoch 64/100 20/20 [==============================] - 0s 2ms/step - loss: 0.4432 - accuracy: 0.8077 Epoch 65/100 20/20 [==============================] - 0s 2ms/step - loss: 0.4252 - accuracy: 0.8205 Epoch 66/100 20/20 [==============================] - 0s 2ms/step - loss: 0.4451 - accuracy: 0.8013 Epoch 67/100 20/20 [==============================] - 0s 2ms/step - loss: 0.3673 - accuracy: 0.8590 Epoch 68/100 20/20 [==============================] - 0s 2ms/step - loss: 0.3783 - accuracy: 0.8526 Epoch 69/100 20/20 [==============================] - 0s 2ms/step - loss: 0.4337 - accuracy: 0.8269 Epoch 70/100 20/20 [==============================] - 0s 2ms/step - loss: 0.3985 - accuracy: 0.8269 Epoch 71/100 20/20 [==============================] - 0s 2ms/step - loss: 0.3747 - accuracy: 0.8526 Epoch 72/100 20/20 [==============================] - 0s 2ms/step - loss: 0.3114 - accuracy: 0.8846 Epoch 73/100 20/20 [==============================] - 0s 2ms/step - loss: 0.3631 - accuracy: 0.8718 Epoch 74/100 20/20 [==============================] - 0s 1ms/step - loss: 0.3989 - accuracy: 0.8205 Epoch 75/100 20/20 [==============================] - 0s 2ms/step - loss: 0.3730 - accuracy: 0.8269 Epoch 76/100 20/20 [==============================] - 0s 2ms/step - loss: 0.3628 - accuracy: 0.8590 Epoch 77/100 20/20 [==============================] - 0s 1ms/step - loss: 0.3613 - accuracy: 0.8397 Epoch 78/100 20/20 [==============================] - 0s 2ms/step - loss: 0.4003 - accuracy: 0.8269 Epoch 79/100 20/20 [==============================] - 0s 2ms/step - loss: 0.4395 - accuracy: 0.8141 Epoch 80/100 20/20 [==============================] - 0s 2ms/step - loss: 0.3944 - accuracy: 0.8269 Epoch 81/100 20/20 [==============================] - 0s 2ms/step - loss: 0.4066 - accuracy: 0.8205 Epoch 82/100 20/20 [==============================] - 0s 2ms/step - loss: 0.3896 - accuracy: 0.8654 Epoch 83/100 20/20 [==============================] - 0s 2ms/step - loss: 0.3426 - accuracy: 0.8590 Epoch 84/100 20/20 [==============================] - 0s 2ms/step - loss: 0.3245 - accuracy: 0.9038 Epoch 85/100 20/20 [==============================] - 0s 2ms/step - loss: 0.3396 - accuracy: 0.8526 Epoch 86/100 20/20 [==============================] - 0s 2ms/step - loss: 0.3587 - accuracy: 0.8590 Epoch 87/100 20/20 [==============================] - 0s 2ms/step - loss: 0.3705 - accuracy: 0.8205 Epoch 88/100 20/20 [==============================] - 0s 2ms/step - loss: 0.2835 - accuracy: 0.8654 Epoch 89/100 20/20 [==============================] - 0s 2ms/step - loss: 0.3712 - accuracy: 0.8013 Epoch 90/100 20/20 [==============================] - 0s 1ms/step - loss: 0.3640 - accuracy: 0.8333 Epoch 91/100 20/20 [==============================] - 0s 2ms/step - loss: 0.3402 - accuracy: 0.8974 Epoch 92/100 20/20 [==============================] - 0s 1ms/step - loss: 0.3445 - accuracy: 0.8654 Epoch 93/100 20/20 [==============================] - 0s 1ms/step - loss: 0.2922 - accuracy: 0.8910 Epoch 94/100 20/20 [==============================] - 0s 1ms/step - loss: 0.3192 - accuracy: 0.8782 Epoch 95/100 20/20 [==============================] - 0s 2ms/step - loss: 0.2977 - accuracy: 0.8846 Epoch 96/100 20/20 [==============================] - 0s 2ms/step - loss: 0.3482 - accuracy: 0.8269 Epoch 97/100 20/20 [==============================] - 0s 2ms/step - loss: 0.3393 - accuracy: 0.8269 Epoch 98/100 20/20 [==============================] - 0s 2ms/step - loss: 0.3268 - accuracy: 0.8526 Epoch 99/100 20/20 [==============================] - 0s 2ms/step - loss: 0.2882 - accuracy: 0.8974 Epoch 100/100 20/20 [==============================] - 0s 2ms/step - loss: 0.2939 - accuracy: 0.9167
<tensorflow.python.keras.callbacks.History at 0x21799056c40>
modeld.evaluate(X_test, y_test)
2/2 [==============================] - 0s 1ms/step - loss: 0.3962 - accuracy: 0.8077
[0.39620405435562134, 0.807692289352417]

Training Accuracy is still good but Test Accuracy Improved

y_pred = modeld.predict(X_test).reshape(-1)
print(y_pred[:10])

# round the values to nearest integer ie 0 or 1
y_pred = np.round(y_pred)
print(y_pred[:10])
[0.00204679 0.855612 0.87832576 0.02817929 0.99947876 0.92398506 0.24040593 0.99971884 0.04001041 0.99982685] [0. 1. 1. 0. 1. 1. 0. 1. 0. 1.] 
from sklearn.metrics import confusion_matrix , classification_report

print(classification_report(y_test, y_pred))
precision recall f1-score support 0 0.76 0.93 0.83 27 1 0.89 0.68 0.77 25 accuracy 0.81 52 macro avg 0.83 0.80 0.80 52 weighted avg 0.82 0.81 0.80 52

You can see that by using dropout layer test accuracy increased from 0.77 to 0.81