Book a Demo!
CoCalc Logo Icon
StoreFeaturesDocsShareSupportNewsAboutPoliciesSign UpSign In
codebasics
GitHub Repository: codebasics/deep-learning-keras-tf-tutorial
 Path: blob/master/14_imbalanced/Handling Imbalanced Data In Customer Churn Using ANN/Bank Turnover Customer Churn Using ANN.ipynb
1141 views
Kernel: Python 3.9.5 64-bit

Customer churn prediction is to measure why customers are leaving a business. In this tutorial we will be looking at customer churn in Bank business. We will build a deep learning model to predict the churn and use precision, recall, f1-score to measure performance of our model

import pandas as pd
from matplotlib import pyplot as plt
import numpy as np
%matplotlib inline

LOAD THE DATA

df = pd.read_csv('Churn_Modelling.csv')
df.head()

DROP UNNECCESSARY COLUMNS

df.drop(['RowNumber','CustomerId','Surname'], axis= 'columns', inplace=True)
df.head()

FEATURE ENGINEERING

df.dtypes
CreditScore int64 Geography object Gender object Age int64 Tenure int64 Balance float64 NumOfProducts int64 HasCrCard int64 IsActiveMember int64 EstimatedSalary float64 Exited int64 dtype: object
def print_unique_col_values(df):
    for column in df:
        if df[column].dtypes == 'object':
            print(f'{column}: {df[column].unique()}')

print_unique_col_values(df)
Geography: ['France' 'Spain' 'Germany'] Gender: ['Female' 'Male']

ONE HOT ENCODING CATEGORICAL VALUES

df1 = pd.get_dummies(data=df, columns=['Geography', 'Gender'])
df1.columns
Index(['CreditScore', 'Age', 'Tenure', 'Balance', 'NumOfProducts', 'HasCrCard', 'IsActiveMember', 'EstimatedSalary', 'Exited', 'Geography_France', 'Geography_Germany', 'Geography_Spain', 'Gender_Female', 'Gender_Male'], dtype='object')
print_unique_col_values(df1)

df1.head()

SCALING THE DATASET

from sklearn.preprocessing import MinMaxScaler
scaler = MinMaxScaler()

col_to_scale=['CreditScore', 'Age', 'Tenure','Balance', 'NumOfProducts', 'EstimatedSalary']

df1[col_to_scale] = scaler.fit_transform(df1[col_to_scale])

df1.head()

X = df1.drop('Exited',axis='columns')
y = df1.Exited

SPLITTING THE DATASET INTO TRAINING AND TEST SET

from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=10, stratify=y)


X_train.shape
(8000, 13)
X_test.shapeb
(2000, 13)

IMPORTING TENSORFLOW LIBRARIES

import tensorflow as tf
from tensorflow import keras
from sklearn.metrics import confusion_matrix, classification_report

BUILD THE MODEL(ANN)

def ANN(X_train, y_train, X_test, y_test, loss):
    model = keras.Sequential([
        keras.layers.Dense(13, input_dim=13, activation='relu'),
        keras.layers.Dense(10, activation='relu'),
        keras.layers.Dense(1, activation='sigmoid')
    ])

    model.compile(optimizer='adam', loss=loss, metrics=['accuracy'])

    
    model.fit(X_train, y_train, epochs=100)

    print(model.evaluate(X_test, y_test))

    y_preds = model.predict(X_test)
    y_preds = np.round(y_preds)

    print("Classification Report: \n", classification_report(y_test, y_preds))

    return y_preds

y_preds = ANN(X_train, y_train, X_test, y_test, 'binary_crossentropy')
Epoch 1/100 250/250 [==============================] - 2s 854us/step - loss: 0.5041 - accuracy: 0.7944 Epoch 2/100 250/250 [==============================] - 0s 718us/step - loss: 0.4742 - accuracy: 0.7964 Epoch 3/100 250/250 [==============================] - 0s 677us/step - loss: 0.4673 - accuracy: 0.7972 Epoch 4/100 250/250 [==============================] - 0s 730us/step - loss: 0.4588 - accuracy: 0.7981 Epoch 5/100 250/250 [==============================] - 0s 851us/step - loss: 0.4508 - accuracy: 0.8046 Epoch 6/100 250/250 [==============================] - 0s 687us/step - loss: 0.4417 - accuracy: 0.8111 Epoch 7/100 250/250 [==============================] - 0s 671us/step - loss: 0.4337 - accuracy: 0.8155 Epoch 8/100 250/250 [==============================] - 0s 727us/step - loss: 0.4280 - accuracy: 0.8186 Epoch 9/100 250/250 [==============================] - 0s 667us/step - loss: 0.4237 - accuracy: 0.8158 Epoch 10/100 250/250 [==============================] - 0s 679us/step - loss: 0.4205 - accuracy: 0.8207 Epoch 11/100 250/250 [==============================] - 0s 695us/step - loss: 0.4176 - accuracy: 0.8224 Epoch 12/100 250/250 [==============================] - 0s 699us/step - loss: 0.4156 - accuracy: 0.8245 Epoch 13/100 250/250 [==============================] - 0s 703us/step - loss: 0.4126 - accuracy: 0.8265 Epoch 14/100 250/250 [==============================] - 0s 715us/step - loss: 0.4107 - accuracy: 0.8267 Epoch 15/100 250/250 [==============================] - 0s 703us/step - loss: 0.4077 - accuracy: 0.8285 Epoch 16/100 250/250 [==============================] - 0s 743us/step - loss: 0.4037 - accuracy: 0.8317 Epoch 17/100 250/250 [==============================] - 0s 683us/step - loss: 0.3994 - accuracy: 0.8313 Epoch 18/100 250/250 [==============================] - 0s 699us/step - loss: 0.3938 - accuracy: 0.8341 Epoch 19/100 250/250 [==============================] - 0s 727us/step - loss: 0.3872 - accuracy: 0.8369 Epoch 20/100 250/250 [==============================] - 0s 723us/step - loss: 0.3805 - accuracy: 0.8414 Epoch 21/100 250/250 [==============================] - 0s 685us/step - loss: 0.3739 - accuracy: 0.8450 Epoch 22/100 250/250 [==============================] - 0s 723us/step - loss: 0.3684 - accuracy: 0.8499 Epoch 23/100 250/250 [==============================] - 0s 759us/step - loss: 0.3622 - accuracy: 0.8508 Epoch 24/100 250/250 [==============================] - 0s 667us/step - loss: 0.3586 - accuracy: 0.8514 Epoch 25/100 250/250 [==============================] - 0s 848us/step - loss: 0.3571 - accuracy: 0.8549 Epoch 26/100 250/250 [==============================] - 0s 687us/step - loss: 0.3541 - accuracy: 0.8518 Epoch 27/100 250/250 [==============================] - 0s 683us/step - loss: 0.3522 - accuracy: 0.8553 Epoch 28/100 250/250 [==============================] - 0s 695us/step - loss: 0.3517 - accuracy: 0.8536 Epoch 29/100 250/250 [==============================] - 0s 703us/step - loss: 0.3505 - accuracy: 0.8537 Epoch 30/100 250/250 [==============================] - 0s 679us/step - loss: 0.3492 - accuracy: 0.8558 Epoch 31/100 250/250 [==============================] - 0s 735us/step - loss: 0.3481 - accuracy: 0.8549 Epoch 32/100 250/250 [==============================] - 0s 719us/step - loss: 0.3466 - accuracy: 0.8545 Epoch 33/100 250/250 [==============================] - 0s 687us/step - loss: 0.3468 - accuracy: 0.8560 Epoch 34/100 250/250 [==============================] - 0s 691us/step - loss: 0.3451 - accuracy: 0.8576 Epoch 35/100 250/250 [==============================] - 0s 692us/step - loss: 0.3452 - accuracy: 0.8575 Epoch 36/100 250/250 [==============================] - 0s 702us/step - loss: 0.3448 - accuracy: 0.8569 Epoch 37/100 250/250 [==============================] - 0s 783us/step - loss: 0.3430 - accuracy: 0.8564 Epoch 38/100 250/250 [==============================] - 0s 703us/step - loss: 0.3436 - accuracy: 0.8593 Epoch 39/100 250/250 [==============================] - 0s 700us/step - loss: 0.3430 - accuracy: 0.8568 Epoch 40/100 250/250 [==============================] - 0s 711us/step - loss: 0.3431 - accuracy: 0.8566 Epoch 41/100 250/250 [==============================] - 0s 707us/step - loss: 0.3419 - accuracy: 0.8581 Epoch 42/100 250/250 [==============================] - 0s 735us/step - loss: 0.3418 - accuracy: 0.8593 Epoch 43/100 250/250 [==============================] - 0s 747us/step - loss: 0.3407 - accuracy: 0.8604 Epoch 44/100 250/250 [==============================] - 0s 745us/step - loss: 0.3406 - accuracy: 0.8624 Epoch 45/100 250/250 [==============================] - 0s 715us/step - loss: 0.3402 - accuracy: 0.8601 Epoch 46/100 250/250 [==============================] - 0s 727us/step - loss: 0.3397 - accuracy: 0.8605 Epoch 47/100 250/250 [==============================] - 0s 707us/step - loss: 0.3393 - accuracy: 0.8594 Epoch 48/100 250/250 [==============================] - 0s 904us/step - loss: 0.3385 - accuracy: 0.8619 Epoch 49/100 250/250 [==============================] - 0s 711us/step - loss: 0.3384 - accuracy: 0.8601 Epoch 50/100 250/250 [==============================] - 0s 763us/step - loss: 0.3387 - accuracy: 0.8611 Epoch 51/100 250/250 [==============================] - 0s 723us/step - loss: 0.3377 - accuracy: 0.8609 Epoch 52/100 250/250 [==============================] - 0s 715us/step - loss: 0.3368 - accuracy: 0.8599 Epoch 53/100 250/250 [==============================] - 0s 837us/step - loss: 0.3369 - accuracy: 0.8593 Epoch 54/100 250/250 [==============================] - 0s 699us/step - loss: 0.3373 - accuracy: 0.8596 Epoch 55/100 250/250 [==============================] - 0s 723us/step - loss: 0.3361 - accuracy: 0.8599 Epoch 56/100 250/250 [==============================] - 0s 719us/step - loss: 0.3364 - accuracy: 0.8622 Epoch 57/100 250/250 [==============================] - 0s 715us/step - loss: 0.3360 - accuracy: 0.8597 Epoch 58/100 250/250 [==============================] - 0s 707us/step - loss: 0.3350 - accuracy: 0.8622 Epoch 59/100 250/250 [==============================] - 0s 731us/step - loss: 0.3356 - accuracy: 0.8608 Epoch 60/100 250/250 [==============================] - 0s 711us/step - loss: 0.3342 - accuracy: 0.8618 Epoch 61/100 250/250 [==============================] - 0s 827us/step - loss: 0.3342 - accuracy: 0.8612 Epoch 62/100 250/250 [==============================] - 0s 735us/step - loss: 0.3344 - accuracy: 0.8621 Epoch 63/100 250/250 [==============================] - 0s 756us/step - loss: 0.3348 - accuracy: 0.8618 Epoch 64/100 250/250 [==============================] - 0s 747us/step - loss: 0.3345 - accuracy: 0.8614 Epoch 65/100 250/250 [==============================] - 0s 726us/step - loss: 0.3337 - accuracy: 0.8620 Epoch 66/100 250/250 [==============================] - 0s 699us/step - loss: 0.3334 - accuracy: 0.8618 Epoch 67/100 250/250 [==============================] - 0s 711us/step - loss: 0.3335 - accuracy: 0.8590 Epoch 68/100 250/250 [==============================] - 0s 683us/step - loss: 0.3337 - accuracy: 0.8622 Epoch 69/100 250/250 [==============================] - 0s 699us/step - loss: 0.3323 - accuracy: 0.8620 Epoch 70/100 250/250 [==============================] - 0s 727us/step - loss: 0.3326 - accuracy: 0.8625 Epoch 71/100 250/250 [==============================] - 0s 691us/step - loss: 0.3320 - accuracy: 0.8635 Epoch 72/100 250/250 [==============================] - 0s 679us/step - loss: 0.3332 - accuracy: 0.8629 Epoch 73/100 250/250 [==============================] - 0s 703us/step - loss: 0.3319 - accuracy: 0.8611 Epoch 74/100 250/250 [==============================] - 0s 731us/step - loss: 0.3318 - accuracy: 0.8611 Epoch 75/100 250/250 [==============================] - 0s 707us/step - loss: 0.3315 - accuracy: 0.8614 Epoch 76/100 250/250 [==============================] - 0s 775us/step - loss: 0.3319 - accuracy: 0.8639 Epoch 77/100 250/250 [==============================] - 0s 751us/step - loss: 0.3314 - accuracy: 0.8626 Epoch 78/100 250/250 [==============================] - 0s 679us/step - loss: 0.3308 - accuracy: 0.8615 Epoch 79/100 250/250 [==============================] - 0s 699us/step - loss: 0.3313 - accuracy: 0.8626 Epoch 80/100 250/250 [==============================] - 0s 848us/step - loss: 0.3308 - accuracy: 0.8614 Epoch 81/100 250/250 [==============================] - 0s 707us/step - loss: 0.3294 - accuracy: 0.8666 Epoch 82/100 250/250 [==============================] - 0s 711us/step - loss: 0.3307 - accuracy: 0.8635 Epoch 83/100 250/250 [==============================] - 0s 736us/step - loss: 0.3299 - accuracy: 0.8636 Epoch 84/100 250/250 [==============================] - 0s 698us/step - loss: 0.3303 - accuracy: 0.8615 Epoch 85/100 250/250 [==============================] - 0s 715us/step - loss: 0.3302 - accuracy: 0.8631 Epoch 86/100 250/250 [==============================] - 0s 730us/step - loss: 0.3299 - accuracy: 0.8596 Epoch 87/100 250/250 [==============================] - 0s 695us/step - loss: 0.3294 - accuracy: 0.8641 Epoch 88/100 250/250 [==============================] - 0s 703us/step - loss: 0.3296 - accuracy: 0.8633 Epoch 89/100 250/250 [==============================] - 0s 715us/step - loss: 0.3299 - accuracy: 0.8627 Epoch 90/100 250/250 [==============================] - 0s 730us/step - loss: 0.3291 - accuracy: 0.8655 Epoch 91/100 250/250 [==============================] - 0s 675us/step - loss: 0.3296 - accuracy: 0.8636 Epoch 92/100 250/250 [==============================] - 0s 727us/step - loss: 0.3286 - accuracy: 0.8634 Epoch 93/100 250/250 [==============================] - 0s 719us/step - loss: 0.3290 - accuracy: 0.8635 Epoch 94/100 250/250 [==============================] - 0s 695us/step - loss: 0.3300 - accuracy: 0.8639 Epoch 95/100 250/250 [==============================] - 0s 747us/step - loss: 0.3295 - accuracy: 0.8634 Epoch 96/100 250/250 [==============================] - 0s 702us/step - loss: 0.3286 - accuracy: 0.8643 Epoch 97/100 250/250 [==============================] - 0s 707us/step - loss: 0.3292 - accuracy: 0.8654 Epoch 98/100 250/250 [==============================] - 0s 723us/step - loss: 0.3282 - accuracy: 0.8648 Epoch 99/100 250/250 [==============================] - 0s 716us/step - loss: 0.3284 - accuracy: 0.8645 Epoch 100/100 250/250 [==============================] - 0s 735us/step - loss: 0.3289 - accuracy: 0.8639 63/63 [==============================] - 0s 532us/step - loss: 0.3331 - accuracy: 0.8630 [0.33314570784568787, 0.8629999756813049] Classification Report: precision recall f1-score support 0 0.88 0.97 0.92 1593 1 0.77 0.46 0.58 407 accuracy 0.86 2000 macro avg 0.82 0.71 0.75 2000 weighted avg 0.85 0.86 0.85 2000

As We See, the precision, recall and f1 score of Class 1 is very low due to imbalanced dataset

Mitigating Skewdness of Data

Method 1: Undersampling

reference: https: // www.kaggle.com/rafjaa/resampling-strategies-for-imbalanced-datasets

# Class count
count_class_0, count_class_1 = df1.Exited.value_counts()

# Divide by class
df_class_0 = df1[df1['Exited'] == 0]
df_class_1 = df1[df1['Exited'] == 1]

df_class_0_under = df_class_0.sample(count_class_1)
df_test_under = pd.concat([df_class_0_under, df_class_1], axis=0)

print('Random under-sampling:')
print(df_test_under.Exited.value_counts())
Random under-sampling: 0 2037 1 2037 Name: Exited, dtype: int64 
from sklearn.model_selection import train_test_split
X = df_test_under.drop('Exited', axis='columns')
y = df_test_under['Exited']

X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.2, random_state=15, stratify=y)

y_train.value_counts()
1 1630 0 1629 Name: Exited, dtype: int64
y_preds = ANN(X_train, y_train, X_test, y_test, 'binary_crossentropy')
Epoch 1/100 102/102 [==============================] - 1s 2ms/step - loss: 0.6774 - accuracy: 0.5802 Epoch 2/100 102/102 [==============================] - 0s 1ms/step - loss: 0.6464 - accuracy: 0.6349 Epoch 3/100 102/102 [==============================] - 0s 1ms/step - loss: 0.6310 - accuracy: 0.6542 Epoch 4/100 102/102 [==============================] - 0s 2ms/step - loss: 0.6239 - accuracy: 0.6646 Epoch 5/100 102/102 [==============================] - 0s 2ms/step - loss: 0.6175 - accuracy: 0.6720 Epoch 6/100 102/102 [==============================] - 0s 1ms/step - loss: 0.6120 - accuracy: 0.6744 Epoch 7/100 102/102 [==============================] - 0s 1ms/step - loss: 0.6056 - accuracy: 0.6793 Epoch 8/100 102/102 [==============================] - 0s 1ms/step - loss: 0.5993 - accuracy: 0.6870 Epoch 9/100 102/102 [==============================] - 0s 1ms/step - loss: 0.5936 - accuracy: 0.6910 Epoch 10/100 102/102 [==============================] - 0s 1ms/step - loss: 0.5881 - accuracy: 0.7005 Epoch 11/100 102/102 [==============================] - 0s 2ms/step - loss: 0.5818 - accuracy: 0.7014 Epoch 12/100 102/102 [==============================] - 0s 1ms/step - loss: 0.5777 - accuracy: 0.7036 Epoch 13/100 102/102 [==============================] - 0s 1ms/step - loss: 0.5731 - accuracy: 0.7079 Epoch 14/100 102/102 [==============================] - 0s 2ms/step - loss: 0.5716 - accuracy: 0.7085 Epoch 15/100 102/102 [==============================] - 0s 1ms/step - loss: 0.5669 - accuracy: 0.7125 Epoch 16/100 102/102 [==============================] - 0s 1ms/step - loss: 0.5641 - accuracy: 0.7128 Epoch 17/100 102/102 [==============================] - 0s 1ms/step - loss: 0.5611 - accuracy: 0.7156 Epoch 18/100 102/102 [==============================] - 0s 1ms/step - loss: 0.5560 - accuracy: 0.7245 Epoch 19/100 102/102 [==============================] - 0s 1ms/step - loss: 0.5513 - accuracy: 0.7288 Epoch 20/100 102/102 [==============================] - 0s 1ms/step - loss: 0.5453 - accuracy: 0.7315 Epoch 21/100 102/102 [==============================] - 0s 1ms/step - loss: 0.5387 - accuracy: 0.7373 Epoch 22/100 102/102 [==============================] - 0s 1ms/step - loss: 0.5349 - accuracy: 0.7407 Epoch 23/100 102/102 [==============================] - 0s 2ms/step - loss: 0.5267 - accuracy: 0.7447 Epoch 24/100 102/102 [==============================] - 0s 1ms/step - loss: 0.5220 - accuracy: 0.7496 Epoch 25/100 102/102 [==============================] - 0s 2ms/step - loss: 0.5154 - accuracy: 0.7548 Epoch 26/100 102/102 [==============================] - 0s 1ms/step - loss: 0.5104 - accuracy: 0.7579 Epoch 27/100 102/102 [==============================] - 0s 2ms/step - loss: 0.5026 - accuracy: 0.7668 Epoch 28/100 102/102 [==============================] - 0s 967us/step - loss: 0.5006 - accuracy: 0.7650 Epoch 29/100 102/102 [==============================] - 0s 812us/step - loss: 0.4968 - accuracy: 0.7683 Epoch 30/100 102/102 [==============================] - 0s 1ms/step - loss: 0.4941 - accuracy: 0.7683 Epoch 31/100 102/102 [==============================] - 0s 891us/step - loss: 0.4919 - accuracy: 0.7659 Epoch 32/100 102/102 [==============================] - 0s 891us/step - loss: 0.4883 - accuracy: 0.7671 Epoch 33/100 102/102 [==============================] - 0s 812us/step - loss: 0.4889 - accuracy: 0.7665 Epoch 34/100 102/102 [==============================] - 0s 901us/step - loss: 0.4852 - accuracy: 0.7665 Epoch 35/100 102/102 [==============================] - 0s 921us/step - loss: 0.4821 - accuracy: 0.7659 Epoch 36/100 102/102 [==============================] - 0s 862us/step - loss: 0.4819 - accuracy: 0.7668 Epoch 37/100 102/102 [==============================] - 0s 822us/step - loss: 0.4801 - accuracy: 0.7659 Epoch 38/100 102/102 [==============================] - 0s 891us/step - loss: 0.4806 - accuracy: 0.7693 Epoch 39/100 102/102 [==============================] - 0s 862us/step - loss: 0.4788 - accuracy: 0.7699 Epoch 40/100 102/102 [==============================] - 0s 822us/step - loss: 0.4778 - accuracy: 0.7683 Epoch 41/100 102/102 [==============================] - 0s 901us/step - loss: 0.4784 - accuracy: 0.7671 Epoch 42/100 102/102 [==============================] - 0s 802us/step - loss: 0.4792 - accuracy: 0.7680 Epoch 43/100 102/102 [==============================] - 0s 871us/step - loss: 0.4770 - accuracy: 0.7689 Epoch 44/100 102/102 [==============================] - 0s 1ms/step - loss: 0.4756 - accuracy: 0.7732 Epoch 45/100 102/102 [==============================] - 0s 2ms/step - loss: 0.4773 - accuracy: 0.7705 Epoch 46/100 102/102 [==============================] - 0s 772us/step - loss: 0.4747 - accuracy: 0.7714 Epoch 47/100 102/102 [==============================] - 0s 842us/step - loss: 0.4745 - accuracy: 0.7686 Epoch 48/100 102/102 [==============================] - 0s 802us/step - loss: 0.4742 - accuracy: 0.7720 Epoch 49/100 102/102 [==============================] - 0s 842us/step - loss: 0.4732 - accuracy: 0.7702 Epoch 50/100 102/102 [==============================] - 0s 832us/step - loss: 0.4742 - accuracy: 0.7732 Epoch 51/100 102/102 [==============================] - 0s 792us/step - loss: 0.4728 - accuracy: 0.7729 Epoch 52/100 102/102 [==============================] - 0s 832us/step - loss: 0.4717 - accuracy: 0.7723 Epoch 53/100 102/102 [==============================] - 0s 812us/step - loss: 0.4714 - accuracy: 0.7751 Epoch 54/100 102/102 [==============================] - 0s 822us/step - loss: 0.4709 - accuracy: 0.7739 Epoch 55/100 102/102 [==============================] - 0s 971us/step - loss: 0.4710 - accuracy: 0.7760 Epoch 56/100 102/102 [==============================] - 0s 911us/step - loss: 0.4699 - accuracy: 0.7717 Epoch 57/100 102/102 [==============================] - 0s 921us/step - loss: 0.4692 - accuracy: 0.7760 Epoch 58/100 102/102 [==============================] - 0s 822us/step - loss: 0.4696 - accuracy: 0.7739 Epoch 59/100 102/102 [==============================] - 0s 782us/step - loss: 0.4690 - accuracy: 0.7729 Epoch 60/100 102/102 [==============================] - 0s 831us/step - loss: 0.4694 - accuracy: 0.7739 Epoch 61/100 102/102 [==============================] - 0s 852us/step - loss: 0.4685 - accuracy: 0.7766 Epoch 62/100 102/102 [==============================] - 0s 881us/step - loss: 0.4696 - accuracy: 0.7736 Epoch 63/100 102/102 [==============================] - 0s 901us/step - loss: 0.4696 - accuracy: 0.7736 Epoch 64/100 102/102 [==============================] - 0s 862us/step - loss: 0.4691 - accuracy: 0.7708 Epoch 65/100 102/102 [==============================] - 0s 860us/step - loss: 0.4681 - accuracy: 0.7717 Epoch 66/100 102/102 [==============================] - 0s 862us/step - loss: 0.4678 - accuracy: 0.7714 Epoch 67/100 102/102 [==============================] - 0s 832us/step - loss: 0.4681 - accuracy: 0.7748 Epoch 68/100 102/102 [==============================] - 0s 1ms/step - loss: 0.4677 - accuracy: 0.7751 Epoch 69/100 102/102 [==============================] - 0s 871us/step - loss: 0.4696 - accuracy: 0.7766 Epoch 70/100 102/102 [==============================] - 0s 891us/step - loss: 0.4694 - accuracy: 0.7693 Epoch 71/100 102/102 [==============================] - 0s 782us/step - loss: 0.4669 - accuracy: 0.7751 Epoch 72/100 102/102 [==============================] - 0s 862us/step - loss: 0.4668 - accuracy: 0.7769 Epoch 73/100 102/102 [==============================] - 0s 881us/step - loss: 0.4676 - accuracy: 0.7723 Epoch 74/100 102/102 [==============================] - 0s 821us/step - loss: 0.4669 - accuracy: 0.7729 Epoch 75/100 102/102 [==============================] - 0s 971us/step - loss: 0.4657 - accuracy: 0.7788 Epoch 76/100 102/102 [==============================] - 0s 852us/step - loss: 0.4666 - accuracy: 0.7736 Epoch 77/100 102/102 [==============================] - 0s 911us/step - loss: 0.4654 - accuracy: 0.7760 Epoch 78/100 102/102 [==============================] - 0s 910us/step - loss: 0.4656 - accuracy: 0.7757 Epoch 79/100 102/102 [==============================] - 0s 1ms/step - loss: 0.4645 - accuracy: 0.7797 Epoch 80/100 102/102 [==============================] - 0s 1ms/step - loss: 0.4667 - accuracy: 0.7782 Epoch 81/100 102/102 [==============================] - 0s 1ms/step - loss: 0.4678 - accuracy: 0.7754 Epoch 82/100 102/102 [==============================] - 0s 2ms/step - loss: 0.4653 - accuracy: 0.7772 Epoch 83/100 102/102 [==============================] - 0s 881us/step - loss: 0.4658 - accuracy: 0.7763 Epoch 84/100 102/102 [==============================] - 0s 862us/step - loss: 0.4645 - accuracy: 0.7751 Epoch 85/100 102/102 [==============================] - 0s 852us/step - loss: 0.4639 - accuracy: 0.7748 Epoch 86/100 102/102 [==============================] - 0s 822us/step - loss: 0.4645 - accuracy: 0.7760 Epoch 87/100 102/102 [==============================] - 0s 911us/step - loss: 0.4648 - accuracy: 0.7785 Epoch 88/100 102/102 [==============================] - 0s 1ms/step - loss: 0.4649 - accuracy: 0.7766 Epoch 89/100 102/102 [==============================] - 0s 1ms/step - loss: 0.4642 - accuracy: 0.7754 Epoch 90/100 102/102 [==============================] - 0s 842us/step - loss: 0.4639 - accuracy: 0.7748 Epoch 91/100 102/102 [==============================] - 0s 842us/step - loss: 0.4627 - accuracy: 0.7775 Epoch 92/100 102/102 [==============================] - 0s 842us/step - loss: 0.4653 - accuracy: 0.7699 Epoch 93/100 102/102 [==============================] - 0s 921us/step - loss: 0.4634 - accuracy: 0.7791 Epoch 94/100 102/102 [==============================] - 0s 891us/step - loss: 0.4626 - accuracy: 0.7806 Epoch 95/100 102/102 [==============================] - 0s 921us/step - loss: 0.4642 - accuracy: 0.7800 Epoch 96/100 102/102 [==============================] - 0s 920us/step - loss: 0.4628 - accuracy: 0.7766 Epoch 97/100 102/102 [==============================] - 0s 951us/step - loss: 0.4662 - accuracy: 0.7714 Epoch 98/100 102/102 [==============================] - 0s 812us/step - loss: 0.4652 - accuracy: 0.7711 Epoch 99/100 102/102 [==============================] - 0s 812us/step - loss: 0.4626 - accuracy: 0.7788 Epoch 100/100 102/102 [==============================] - 0s 883us/step - loss: 0.4622 - accuracy: 0.7809 26/26 [==============================] - 0s 1ms/step - loss: 0.4573 - accuracy: 0.7730 [0.4572858512401581, 0.7730061411857605] Classification Report: precision recall f1-score support 0 0.77 0.78 0.77 408 1 0.78 0.77 0.77 407 accuracy 0.77 815 macro avg 0.77 0.77 0.77 815 weighted avg 0.77 0.77 0.77 815

As we see, there is considerable Improvement in the f1, recall and precision scores of Class 1 Value. The f1 score has improved from 0.58 to 0.77.

Method2: Oversampling

# Oversample 1-class and concat the DataFrames of both classes
df_class_1_over = df_class_1.sample(count_class_0, replace=True)
df_test_over = pd.concat([df_class_0, df_class_1_over], axis=0)

print('Random over-sampling:')
print(df_test_over.Exitedb.value_counts())
Random over-sampling: 0 7963 1 7963 Name: Exited, dtype: int64 
from sklearn.model_selection import train_test_split
X = df_test_over.drop('Exited', axis='columns')
y = df_test_over['Exited']

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=15, stratify=y)


y_preds = ANN(X_train, y_train, X_test, y_test, 'binary_crossentropy')
Epoch 1/100 399/399 [==============================] - 1s 919us/step - loss: 0.6620 - accuracy: 0.6154 Epoch 2/100 399/399 [==============================] - 0s 920us/step - loss: 0.6135 - accuracy: 0.6735 Epoch 3/100 399/399 [==============================] - 0s 900us/step - loss: 0.5829 - accuracy: 0.7017 Epoch 4/100 399/399 [==============================] - 0s 857us/step - loss: 0.5554 - accuracy: 0.7208 Epoch 5/100 399/399 [==============================] - 0s 799us/step - loss: 0.5319 - accuracy: 0.7394 Epoch 6/100 399/399 [==============================] - 0s 747us/step - loss: 0.5101 - accuracy: 0.7530 Epoch 7/100 399/399 [==============================] - 0s 829us/step - loss: 0.4970 - accuracy: 0.7612 Epoch 8/100 399/399 [==============================] - 0s 862us/step - loss: 0.4893 - accuracy: 0.7596 Epoch 9/100 399/399 [==============================] - 0s 777us/step - loss: 0.4855 - accuracy: 0.7601 Epoch 10/100 399/399 [==============================] - 0s 792us/step - loss: 0.4815 - accuracy: 0.7644 Epoch 11/100 399/399 [==============================] - 0s 782us/step - loss: 0.4798 - accuracy: 0.7648 Epoch 12/100 399/399 [==============================] - 0s 782us/step - loss: 0.4781 - accuracy: 0.7655 Epoch 13/100 399/399 [==============================] - 0s 749us/step - loss: 0.4769 - accuracy: 0.7637 Epoch 14/100 399/399 [==============================] - 0s 792us/step - loss: 0.4751 - accuracy: 0.7664 Epoch 15/100 399/399 [==============================] - 0s 789us/step - loss: 0.4737 - accuracy: 0.7661 Epoch 16/100 399/399 [==============================] - 0s 857us/step - loss: 0.4730 - accuracy: 0.7680 Epoch 17/100 399/399 [==============================] - 0s 819us/step - loss: 0.4720 - accuracy: 0.7670 Epoch 18/100 399/399 [==============================] - 0s 769us/step - loss: 0.4706 - accuracy: 0.7658 Epoch 19/100 399/399 [==============================] - 0s 782us/step - loss: 0.4701 - accuracy: 0.7694 Epoch 20/100 399/399 [==============================] - 0s 792us/step - loss: 0.4702 - accuracy: 0.7681 Epoch 21/100 399/399 [==============================] - 0s 779us/step - loss: 0.4693 - accuracy: 0.7703 Epoch 22/100 399/399 [==============================] - 0s 777us/step - loss: 0.4684 - accuracy: 0.7690 Epoch 23/100 399/399 [==============================] - 0s 781us/step - loss: 0.4680 - accuracy: 0.7703 Epoch 24/100 399/399 [==============================] - 0s 789us/step - loss: 0.4680 - accuracy: 0.7701 Epoch 25/100 399/399 [==============================] - 0s 777us/step - loss: 0.4667 - accuracy: 0.7703 Epoch 26/100 399/399 [==============================] - 0s 784us/step - loss: 0.4654 - accuracy: 0.7710 Epoch 27/100 399/399 [==============================] - 0s 852us/step - loss: 0.4655 - accuracy: 0.7713 Epoch 28/100 399/399 [==============================] - 0s 789us/step - loss: 0.4648 - accuracy: 0.7710 Epoch 29/100 399/399 [==============================] - 0s 807us/step - loss: 0.4643 - accuracy: 0.7685 Epoch 30/100 399/399 [==============================] - 0s 789us/step - loss: 0.4637 - accuracy: 0.7703 Epoch 31/100 399/399 [==============================] - 0s 798us/step - loss: 0.4636 - accuracy: 0.7724 Epoch 32/100 399/399 [==============================] - 0s 751us/step - loss: 0.4633 - accuracy: 0.7703 Epoch 33/100 399/399 [==============================] - 0s 907us/step - loss: 0.4629 - accuracy: 0.7712 Epoch 34/100 399/399 [==============================] - 0s 784us/step - loss: 0.4625 - accuracy: 0.7706 Epoch 35/100 399/399 [==============================] - 0s 761us/step - loss: 0.4611 - accuracy: 0.7717 Epoch 36/100 399/399 [==============================] - 0s 854us/step - loss: 0.4618 - accuracy: 0.7718 Epoch 37/100 399/399 [==============================] - 0s 814us/step - loss: 0.4605 - accuracy: 0.7714 Epoch 38/100 399/399 [==============================] - 0s 865us/step - loss: 0.4623 - accuracy: 0.7724 Epoch 39/100 399/399 [==============================] - 0s 832us/step - loss: 0.4598 - accuracy: 0.7728 Epoch 40/100 399/399 [==============================] - 0s 807us/step - loss: 0.4599 - accuracy: 0.7733 Epoch 41/100 399/399 [==============================] - 0s 845us/step - loss: 0.4597 - accuracy: 0.7693 Epoch 42/100 399/399 [==============================] - 0s 807us/step - loss: 0.4594 - accuracy: 0.7724 Epoch 43/100 399/399 [==============================] - 0s 784us/step - loss: 0.4579 - accuracy: 0.7737 Epoch 44/100 399/399 [==============================] - 0s 789us/step - loss: 0.4596 - accuracy: 0.7746 Epoch 45/100 399/399 [==============================] - 0s 812us/step - loss: 0.4583 - accuracy: 0.7734 Epoch 46/100 399/399 [==============================] - 0s 803us/step - loss: 0.4580 - accuracy: 0.7729 Epoch 47/100 399/399 [==============================] - 0s 789us/step - loss: 0.4571 - accuracy: 0.7718 Epoch 48/100 399/399 [==============================] - 0s 790us/step - loss: 0.4575 - accuracy: 0.7744 Epoch 49/100 399/399 [==============================] - 0s 792us/step - loss: 0.4562 - accuracy: 0.7751 Epoch 50/100 399/399 [==============================] - 0s 769us/step - loss: 0.4578 - accuracy: 0.7732 Epoch 51/100 399/399 [==============================] - 0s 955us/step - loss: 0.4555 - accuracy: 0.7756 Epoch 52/100 399/399 [==============================] - 0s 821us/step - loss: 0.4559 - accuracy: 0.7737 Epoch 53/100 399/399 [==============================] - 0s 772us/step - loss: 0.4552 - accuracy: 0.7757 Epoch 54/100 399/399 [==============================] - 0s 777us/step - loss: 0.4564 - accuracy: 0.7733 Epoch 55/100 399/399 [==============================] - 0s 925us/step - loss: 0.4552 - accuracy: 0.7746 Epoch 56/100 399/399 [==============================] - 0s 925us/step - loss: 0.4566 - accuracy: 0.7725 Epoch 57/100 399/399 [==============================] - 0s 882us/step - loss: 0.4534 - accuracy: 0.7751 Epoch 58/100 399/399 [==============================] - 0s 809us/step - loss: 0.4548 - accuracy: 0.7769 Epoch 59/100 399/399 [==============================] - 0s 814us/step - loss: 0.4548 - accuracy: 0.7768 Epoch 60/100 399/399 [==============================] - 0s 910us/step - loss: 0.4530 - accuracy: 0.7785 Epoch 61/100 399/399 [==============================] - 0s 859us/step - loss: 0.4530 - accuracy: 0.7760 Epoch 62/100 399/399 [==============================] - 0s 847us/step - loss: 0.4538 - accuracy: 0.7757 Epoch 63/100 399/399 [==============================] - 0s 849us/step - loss: 0.4532 - accuracy: 0.7784 Epoch 64/100 399/399 [==============================] - 0s 862us/step - loss: 0.4529 - accuracy: 0.7778 Epoch 65/100 399/399 [==============================] - 0s 837us/step - loss: 0.4523 - accuracy: 0.7780 Epoch 66/100 399/399 [==============================] - 0s 827us/step - loss: 0.4519 - accuracy: 0.7797 Epoch 67/100 399/399 [==============================] - 0s 811us/step - loss: 0.4523 - accuracy: 0.7795 Epoch 68/100 399/399 [==============================] - 0s 857us/step - loss: 0.4524 - accuracy: 0.7793 Epoch 69/100 399/399 [==============================] - 0s 821us/step - loss: 0.4514 - accuracy: 0.7799 Epoch 70/100 399/399 [==============================] - 0s 799us/step - loss: 0.4510 - accuracy: 0.7781 Epoch 71/100 399/399 [==============================] - 0s 799us/step - loss: 0.4513 - accuracy: 0.7810 Epoch 72/100 399/399 [==============================] - 0s 839us/step - loss: 0.4514 - accuracy: 0.7802 Epoch 73/100 399/399 [==============================] - 0s 808us/step - loss: 0.4508 - accuracy: 0.7809 Epoch 74/100 399/399 [==============================] - 0s 975us/step - loss: 0.4497 - accuracy: 0.7790 Epoch 75/100 399/399 [==============================] - 0s 800us/step - loss: 0.4497 - accuracy: 0.7845 Epoch 76/100 399/399 [==============================] - 0s 812us/step - loss: 0.4505 - accuracy: 0.7788 Epoch 77/100 399/399 [==============================] - 0s 804us/step - loss: 0.4490 - accuracy: 0.7824 Epoch 78/100 399/399 [==============================] - 0s 789us/step - loss: 0.4487 - accuracy: 0.7797 Epoch 79/100 399/399 [==============================] - 0s 978us/step - loss: 0.4500 - accuracy: 0.7812 Epoch 80/100 399/399 [==============================] - 0s 930us/step - loss: 0.4484 - accuracy: 0.7812 Epoch 81/100 399/399 [==============================] - 0s 761us/step - loss: 0.4482 - accuracy: 0.7815 Epoch 82/100 399/399 [==============================] - 0s 724us/step - loss: 0.4480 - accuracy: 0.7828 Epoch 83/100 399/399 [==============================] - 0s 816us/step - loss: 0.4485 - accuracy: 0.7816 Epoch 84/100 399/399 [==============================] - 0s 807us/step - loss: 0.4482 - accuracy: 0.7827 Epoch 85/100 399/399 [==============================] - 0s 739us/step - loss: 0.4488 - accuracy: 0.7827 Epoch 86/100 399/399 [==============================] - 0s 807us/step - loss: 0.4481 - accuracy: 0.7830 Epoch 87/100 399/399 [==============================] - 0s 832us/step - loss: 0.4478 - accuracy: 0.7827 Epoch 88/100 399/399 [==============================] - 0s 764us/step - loss: 0.4480 - accuracy: 0.7811 Epoch 89/100 399/399 [==============================] - 0s 772us/step - loss: 0.4470 - accuracy: 0.7835 Epoch 90/100 399/399 [==============================] - 0s 757us/step - loss: 0.4479 - accuracy: 0.7845 Epoch 91/100 399/399 [==============================] - 0s 727us/step - loss: 0.4488 - accuracy: 0.7809 Epoch 92/100 399/399 [==============================] - 0s 754us/step - loss: 0.4474 - accuracy: 0.7830 Epoch 93/100 399/399 [==============================] - 0s 749us/step - loss: 0.4478 - accuracy: 0.7835 Epoch 94/100 399/399 [==============================] - 0s 754us/step - loss: 0.4468 - accuracy: 0.7841 Epoch 95/100 399/399 [==============================] - 0s 668us/step - loss: 0.4463 - accuracy: 0.7827 Epoch 96/100 399/399 [==============================] - 0s 731us/step - loss: 0.4473 - accuracy: 0.7814 Epoch 97/100 399/399 [==============================] - 0s 725us/step - loss: 0.4470 - accuracy: 0.7829 Epoch 98/100 399/399 [==============================] - 0s 634us/step - loss: 0.4473 - accuracy: 0.7820 Epoch 99/100 399/399 [==============================] - 0s 769us/step - loss: 0.4469 - accuracy: 0.7840 Epoch 100/100 399/399 [==============================] - 0s 734us/step - loss: 0.4470 - accuracy: 0.7829 100/100 [==============================] - 0s 465us/step - loss: 0.4378 - accuracy: 0.7897 [0.4378308355808258, 0.7897049784660339] Classification Report: precision recall f1-score support 0 0.79 0.79 0.79 1593 1 0.79 0.79 0.79 1593 accuracy 0.79 3186 macro avg 0.79 0.79 0.79 3186 weighted avg 0.79 0.79 0.79 3186

f1-score for minority class 1 improved from 0.58 to 0.79.

Method3: SMOTE

To install imbalanced-learn library use pip install imbalanced-learn command

X = df1.drop('Exited', axis='columns')
y = df1['Exited']

from imblearn.over_sampling import SMOTE

smote = SMOTE(sampling_strategy='minority')
X_sm, y_sm = smote.fit_resample(X, y)

y_sm.value_counts()
0 7963 1 7963 Name: Exited, dtype: int64
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X_sm, y_sm, test_size=0.2, random_state=15, stratify=y_sm)

y_preds = ANN(X_train, y_train, X_test, y_test, 'binary_crossentropy')
Epoch 1/100 399/399 [==============================] - 1s 611us/step - loss: 0.6671 - accuracy: 0.5758 Epoch 2/100 399/399 [==============================] - 0s 621us/step - loss: 0.6068 - accuracy: 0.6684 Epoch 3/100 399/399 [==============================] - 0s 586us/step - loss: 0.5796 - accuracy: 0.6971 Epoch 4/100 399/399 [==============================] - 0s 596us/step - loss: 0.5640 - accuracy: 0.7059 Epoch 5/100 399/399 [==============================] - 0s 596us/step - loss: 0.5538 - accuracy: 0.7171 Epoch 6/100 399/399 [==============================] - 0s 679us/step - loss: 0.5424 - accuracy: 0.7272 Epoch 7/100 399/399 [==============================] - 0s 641us/step - loss: 0.5301 - accuracy: 0.7365 Epoch 8/100 399/399 [==============================] - 0s 597us/step - loss: 0.5195 - accuracy: 0.7414 Epoch 9/100 399/399 [==============================] - 0s 570us/step - loss: 0.5094 - accuracy: 0.7480 Epoch 10/100 399/399 [==============================] - 0s 638us/step - loss: 0.5004 - accuracy: 0.7567 Epoch 11/100 399/399 [==============================] - 0s 709us/step - loss: 0.4935 - accuracy: 0.7600 Epoch 12/100 399/399 [==============================] - 0s 590us/step - loss: 0.4861 - accuracy: 0.7643 Epoch 13/100 399/399 [==============================] - 0s 583us/step - loss: 0.4803 - accuracy: 0.7680 Epoch 14/100 399/399 [==============================] - 0s 590us/step - loss: 0.4759 - accuracy: 0.7720 Epoch 15/100 399/399 [==============================] - 0s 585us/step - loss: 0.4722 - accuracy: 0.7761 Epoch 16/100 399/399 [==============================] - 0s 583us/step - loss: 0.4704 - accuracy: 0.7756 Epoch 17/100 399/399 [==============================] - 0s 580us/step - loss: 0.4666 - accuracy: 0.7784 Epoch 18/100 399/399 [==============================] - 0s 570us/step - loss: 0.4627 - accuracy: 0.7780 Epoch 19/100 399/399 [==============================] - 0s 626us/step - loss: 0.4618 - accuracy: 0.7786 Epoch 20/100 399/399 [==============================] - 0s 598us/step - loss: 0.4591 - accuracy: 0.7789 Epoch 21/100 399/399 [==============================] - 0s 558us/step - loss: 0.4582 - accuracy: 0.7820 Epoch 22/100 399/399 [==============================] - 0s 583us/step - loss: 0.4562 - accuracy: 0.7818 Epoch 23/100 399/399 [==============================] - 0s 578us/step - loss: 0.4553 - accuracy: 0.7835 Epoch 24/100 399/399 [==============================] - 0s 614us/step - loss: 0.4538 - accuracy: 0.7834 Epoch 25/100 399/399 [==============================] - 0s 580us/step - loss: 0.4530 - accuracy: 0.7795 Epoch 26/100 399/399 [==============================] - 0s 586us/step - loss: 0.4510 - accuracy: 0.7847 Epoch 27/100 399/399 [==============================] - 0s 593us/step - loss: 0.4511 - accuracy: 0.7840 Epoch 28/100 399/399 [==============================] - 0s 591us/step - loss: 0.4509 - accuracy: 0.7867 Epoch 29/100 399/399 [==============================] - 0s 573us/step - loss: 0.4505 - accuracy: 0.7830 Epoch 30/100 399/399 [==============================] - 0s 578us/step - loss: 0.4483 - accuracy: 0.7868 Epoch 31/100 399/399 [==============================] - 0s 570us/step - loss: 0.4475 - accuracy: 0.7863 Epoch 32/100 399/399 [==============================] - 0s 606us/step - loss: 0.4474 - accuracy: 0.7856 Epoch 33/100 399/399 [==============================] - 0s 571us/step - loss: 0.4466 - accuracy: 0.7863 Epoch 34/100 399/399 [==============================] - 0s 576us/step - loss: 0.4463 - accuracy: 0.7861 Epoch 35/100 399/399 [==============================] - 0s 580us/step - loss: 0.4460 - accuracy: 0.7866 Epoch 36/100 399/399 [==============================] - 0s 586us/step - loss: 0.4448 - accuracy: 0.7891 Epoch 37/100 399/399 [==============================] - 0s 576us/step - loss: 0.4438 - accuracy: 0.7878 Epoch 38/100 399/399 [==============================] - 0s 563us/step - loss: 0.4435 - accuracy: 0.7885 Epoch 39/100 399/399 [==============================] - 0s 570us/step - loss: 0.4423 - accuracy: 0.7895 Epoch 40/100 399/399 [==============================] - 0s 673us/step - loss: 0.4433 - accuracy: 0.7874 Epoch 41/100 399/399 [==============================] - 0s 566us/step - loss: 0.4426 - accuracy: 0.7900 Epoch 42/100 399/399 [==============================] - 0s 565us/step - loss: 0.4418 - accuracy: 0.7889 Epoch 43/100 399/399 [==============================] - 0s 555us/step - loss: 0.4407 - accuracy: 0.7921 Epoch 44/100 399/399 [==============================] - 0s 560us/step - loss: 0.4404 - accuracy: 0.7924 Epoch 45/100 399/399 [==============================] - 0s 560us/step - loss: 0.4387 - accuracy: 0.7916 Epoch 46/100 399/399 [==============================] - 0s 555us/step - loss: 0.4392 - accuracy: 0.7907 Epoch 47/100 399/399 [==============================] - 0s 570us/step - loss: 0.4389 - accuracy: 0.7911 Epoch 48/100 399/399 [==============================] - 0s 551us/step - loss: 0.4382 - accuracy: 0.7933 Epoch 49/100 399/399 [==============================] - 0s 555us/step - loss: 0.4382 - accuracy: 0.7931 Epoch 50/100 399/399 [==============================] - 0s 558us/step - loss: 0.4375 - accuracy: 0.7938 Epoch 51/100 399/399 [==============================] - 0s 563us/step - loss: 0.4368 - accuracy: 0.7929 Epoch 52/100 399/399 [==============================] - 0s 565us/step - loss: 0.4363 - accuracy: 0.7940 Epoch 53/100 399/399 [==============================] - 0s 568us/step - loss: 0.4368 - accuracy: 0.7943 Epoch 54/100 399/399 [==============================] - 0s 573us/step - loss: 0.4358 - accuracy: 0.7959 Epoch 55/100 399/399 [==============================] - 0s 560us/step - loss: 0.4354 - accuracy: 0.7944 Epoch 56/100 399/399 [==============================] - 0s 573us/step - loss: 0.4349 - accuracy: 0.7949 Epoch 57/100 399/399 [==============================] - 0s 565us/step - loss: 0.4330 - accuracy: 0.7943 Epoch 58/100 399/399 [==============================] - 0s 565us/step - loss: 0.4344 - accuracy: 0.7972 Epoch 59/100 399/399 [==============================] - 0s 570us/step - loss: 0.4323 - accuracy: 0.7980 Epoch 60/100 399/399 [==============================] - 0s 586us/step - loss: 0.4322 - accuracy: 0.7959 Epoch 61/100 399/399 [==============================] - 0s 598us/step - loss: 0.4323 - accuracy: 0.7973 Epoch 62/100 399/399 [==============================] - 0s 581us/step - loss: 0.4319 - accuracy: 0.7929 Epoch 63/100 399/399 [==============================] - 0s 593us/step - loss: 0.4299 - accuracy: 0.7976 Epoch 64/100 399/399 [==============================] - 0s 593us/step - loss: 0.4306 - accuracy: 0.7955 Epoch 65/100 399/399 [==============================] - 0s 601us/step - loss: 0.4291 - accuracy: 0.7957 Epoch 66/100 399/399 [==============================] - 0s 573us/step - loss: 0.4280 - accuracy: 0.7990 Epoch 67/100 399/399 [==============================] - 0s 586us/step - loss: 0.4281 - accuracy: 0.7977 Epoch 68/100 399/399 [==============================] - 0s 580us/step - loss: 0.4261 - accuracy: 0.7985 Epoch 69/100 399/399 [==============================] - 0s 699us/step - loss: 0.4259 - accuracy: 0.8013 Epoch 70/100 399/399 [==============================] - 0s 588us/step - loss: 0.4251 - accuracy: 0.8013 Epoch 71/100 399/399 [==============================] - 0s 585us/step - loss: 0.4243 - accuracy: 0.8002 Epoch 72/100 399/399 [==============================] - 0s 603us/step - loss: 0.4250 - accuracy: 0.8009 Epoch 73/100 399/399 [==============================] - 0s 568us/step - loss: 0.4227 - accuracy: 0.8014 Epoch 74/100 399/399 [==============================] - 0s 588us/step - loss: 0.4215 - accuracy: 0.8008 Epoch 75/100 399/399 [==============================] - 0s 581us/step - loss: 0.4213 - accuracy: 0.8016 Epoch 76/100 399/399 [==============================] - 0s 597us/step - loss: 0.4208 - accuracy: 0.8036 Epoch 77/100 399/399 [==============================] - 0s 591us/step - loss: 0.4206 - accuracy: 0.8020 Epoch 78/100 399/399 [==============================] - 0s 574us/step - loss: 0.4198 - accuracy: 0.8039 Epoch 79/100 399/399 [==============================] - 0s 576us/step - loss: 0.4186 - accuracy: 0.8054 Epoch 80/100 399/399 [==============================] - 0s 565us/step - loss: 0.4171 - accuracy: 0.8048 Epoch 81/100 399/399 [==============================] - 0s 568us/step - loss: 0.4168 - accuracy: 0.8049 Epoch 82/100 399/399 [==============================] - 0s 565us/step - loss: 0.4155 - accuracy: 0.8040 Epoch 83/100 399/399 [==============================] - 0s 569us/step - loss: 0.4149 - accuracy: 0.8068 Epoch 84/100 399/399 [==============================] - 0s 571us/step - loss: 0.4161 - accuracy: 0.8037 Epoch 85/100 399/399 [==============================] - 0s 562us/step - loss: 0.4146 - accuracy: 0.8043 Epoch 86/100 399/399 [==============================] - 0s 566us/step - loss: 0.4146 - accuracy: 0.8073 Epoch 87/100 399/399 [==============================] - 0s 577us/step - loss: 0.4127 - accuracy: 0.8082 Epoch 88/100 399/399 [==============================] - 0s 568us/step - loss: 0.4124 - accuracy: 0.8115 Epoch 89/100 399/399 [==============================] - 0s 593us/step - loss: 0.4115 - accuracy: 0.8087 Epoch 90/100 399/399 [==============================] - 0s 603us/step - loss: 0.4115 - accuracy: 0.8064 Epoch 91/100 399/399 [==============================] - 0s 601us/step - loss: 0.4104 - accuracy: 0.8081 Epoch 92/100 399/399 [==============================] - 0s 588us/step - loss: 0.4123 - accuracy: 0.8082 Epoch 93/100 399/399 [==============================] - 0s 595us/step - loss: 0.4109 - accuracy: 0.8078 Epoch 94/100 399/399 [==============================] - 0s 591us/step - loss: 0.4092 - accuracy: 0.8104 Epoch 95/100 399/399 [==============================] - 0s 593us/step - loss: 0.4084 - accuracy: 0.8115 Epoch 96/100 399/399 [==============================] - 0s 603us/step - loss: 0.4086 - accuracy: 0.8115 Epoch 97/100 399/399 [==============================] - 0s 590us/step - loss: 0.4076 - accuracy: 0.8104 Epoch 98/100 399/399 [==============================] - 0s 586us/step - loss: 0.4085 - accuracy: 0.8100 Epoch 99/100 399/399 [==============================] - 0s 596us/step - loss: 0.4088 - accuracy: 0.8094 Epoch 100/100 399/399 [==============================] - 0s 696us/step - loss: 0.4089 - accuracy: 0.8114 100/100 [==============================] - 0s 445us/step - loss: 0.4078 - accuracy: 0.8142 [0.4077799916267395, 0.8141870498657227] Classification Report: precision recall f1-score support 0 0.81 0.83 0.82 1593 1 0.82 0.80 0.81 1593 accuracy 0.81 3186 macro avg 0.81 0.81 0.81 3186 weighted avg 0.81 0.81 0.81 3186

SMOT Oversampling increases f1 score of minority class 1 from 0.58 to 0.81.

Method4: Use of Ensemble with undersampling

df1.Exited.value_counts()
0 7963 1 2037 Name: Exited, dtype: int64
X = df1.drop('Exited', axis='columns')
y = df1['Exited']

from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.2, random_state=15, stratify=y)

df2 = X_train.copy()
df2['Exited'] = y_train

df2.head()

df2_class0 = df2[df2.Exited == 0]
df2_class1 = df2[df2.Exited == 1]

def get_train_batch(df_majority, df_minority, start, end):
    df_train = pd.concat([df_majority[start:end], df_minority], axis=0)

    X_train = df_train.drop('Exited', axis='columns')
    y_train = df_train.Exited
    return X_train, y_train

X_train, y_train = get_train_batch(df2_class0, df2_class1, 0, 1495)

y_pred1 = ANN(X_train, y_train, X_test, y_test, 'binary_crossentropy')
Epoch 1/100 98/98 [==============================] - 1s 866us/step - loss: 0.6631 - accuracy: 0.6243 Epoch 2/100 98/98 [==============================] - 0s 907us/step - loss: 0.6369 - accuracy: 0.6493 Epoch 3/100 98/98 [==============================] - 0s 804us/step - loss: 0.6259 - accuracy: 0.6518 Epoch 4/100 98/98 [==============================] - 0s 866us/step - loss: 0.6186 - accuracy: 0.6605 Epoch 5/100 98/98 [==============================] - 0s 773us/step - loss: 0.6107 - accuracy: 0.6650 Epoch 6/100 98/98 [==============================] - 0s 815us/step - loss: 0.6043 - accuracy: 0.6765 Epoch 7/100 98/98 [==============================] - 0s 804us/step - loss: 0.5949 - accuracy: 0.6854 Epoch 8/100 98/98 [==============================] - 0s 1ms/step - loss: 0.5863 - accuracy: 0.6906 Epoch 9/100 98/98 [==============================] - 0s 1ms/step - loss: 0.5769 - accuracy: 0.7050 Epoch 10/100 98/98 [==============================] - 0s 835us/step - loss: 0.5658 - accuracy: 0.7152 Epoch 11/100 98/98 [==============================] - 0s 2ms/step - loss: 0.5557 - accuracy: 0.7235 Epoch 12/100 98/98 [==============================] - 0s 2ms/step - loss: 0.5450 - accuracy: 0.7322 Epoch 13/100 98/98 [==============================] - 0s 1ms/step - loss: 0.5345 - accuracy: 0.7389 Epoch 14/100 98/98 [==============================] - 0s 990us/step - loss: 0.5268 - accuracy: 0.7418 Epoch 15/100 98/98 [==============================] - 0s 1ms/step - loss: 0.5189 - accuracy: 0.7494 Epoch 16/100 98/98 [==============================] - 0s 887us/step - loss: 0.5133 - accuracy: 0.7501 Epoch 17/100 98/98 [==============================] - 0s 990us/step - loss: 0.5083 - accuracy: 0.7514 Epoch 18/100 98/98 [==============================] - 0s 877us/step - loss: 0.5048 - accuracy: 0.7549 Epoch 19/100 98/98 [==============================] - 0s 989us/step - loss: 0.4989 - accuracy: 0.7626 Epoch 20/100 98/98 [==============================] - 0s 918us/step - loss: 0.4991 - accuracy: 0.7549 Epoch 21/100 98/98 [==============================] - 0s 2ms/step - loss: 0.4967 - accuracy: 0.7635 Epoch 22/100 98/98 [==============================] - 0s 1ms/step - loss: 0.4951 - accuracy: 0.7578 Epoch 23/100 98/98 [==============================] - 0s 1ms/step - loss: 0.4935 - accuracy: 0.7590 Epoch 24/100 98/98 [==============================] - 0s 1ms/step - loss: 0.4908 - accuracy: 0.7661 Epoch 25/100 98/98 [==============================] - 0s 991us/step - loss: 0.4900 - accuracy: 0.7603 Epoch 26/100 98/98 [==============================] - 0s 1ms/step - loss: 0.4883 - accuracy: 0.7654 Epoch 27/100 98/98 [==============================] - 0s 976us/step - loss: 0.4885 - accuracy: 0.7661 Epoch 28/100 98/98 [==============================] - 0s 1ms/step - loss: 0.4865 - accuracy: 0.7667 Epoch 29/100 98/98 [==============================] - 0s 887us/step - loss: 0.4901 - accuracy: 0.7658 Epoch 30/100 98/98 [==============================] - 0s 969us/step - loss: 0.4865 - accuracy: 0.7622 Epoch 31/100 98/98 [==============================] - 0s 959us/step - loss: 0.4863 - accuracy: 0.7654 Epoch 32/100 98/98 [==============================] - 0s 980us/step - loss: 0.4856 - accuracy: 0.7667 Epoch 33/100 98/98 [==============================] - 0s 1ms/step - loss: 0.4836 - accuracy: 0.7699 Epoch 34/100 98/98 [==============================] - 0s 1ms/step - loss: 0.4839 - accuracy: 0.7696 Epoch 35/100 98/98 [==============================] - 0s 938us/step - loss: 0.4817 - accuracy: 0.7683 Epoch 36/100 98/98 [==============================] - 0s 846us/step - loss: 0.4830 - accuracy: 0.7651 Epoch 37/100 98/98 [==============================] - 0s 1ms/step - loss: 0.4816 - accuracy: 0.7696 Epoch 38/100 98/98 [==============================] - 0s 887us/step - loss: 0.4807 - accuracy: 0.7693 Epoch 39/100 98/98 [==============================] - 0s 975us/step - loss: 0.4807 - accuracy: 0.7712 Epoch 40/100 98/98 [==============================] - 0s 835us/step - loss: 0.4808 - accuracy: 0.7674 Epoch 41/100 98/98 [==============================] - 0s 1ms/step - loss: 0.4801 - accuracy: 0.7664 Epoch 42/100 98/98 [==============================] - 0s 1ms/step - loss: 0.4795 - accuracy: 0.7690 Epoch 43/100 98/98 [==============================] - 0s 866us/step - loss: 0.4799 - accuracy: 0.7699 Epoch 44/100 98/98 [==============================] - 0s 928us/step - loss: 0.4793 - accuracy: 0.7683 Epoch 45/100 98/98 [==============================] - 0s 1ms/step - loss: 0.4782 - accuracy: 0.7706 Epoch 46/100 98/98 [==============================] - 0s 990us/step - loss: 0.4782 - accuracy: 0.7693 Epoch 47/100 98/98 [==============================] - 0s 990us/step - loss: 0.4786 - accuracy: 0.7715 Epoch 48/100 98/98 [==============================] - 0s 1ms/step - loss: 0.4776 - accuracy: 0.7683 Epoch 49/100 98/98 [==============================] - 0s 959us/step - loss: 0.4772 - accuracy: 0.7734 Epoch 50/100 98/98 [==============================] - 0s 1ms/step - loss: 0.4772 - accuracy: 0.7686 Epoch 51/100 98/98 [==============================] - 0s 1ms/step - loss: 0.4762 - accuracy: 0.7683 Epoch 52/100 98/98 [==============================] - 0s 1ms/step - loss: 0.4761 - accuracy: 0.7702 Epoch 53/100 98/98 [==============================] - 0s 1ms/step - loss: 0.4746 - accuracy: 0.7686 Epoch 54/100 98/98 [==============================] - 0s 947us/step - loss: 0.4748 - accuracy: 0.7693 Epoch 55/100 98/98 [==============================] - 0s 970us/step - loss: 0.4757 - accuracy: 0.7718 Epoch 56/100 98/98 [==============================] - 0s 868us/step - loss: 0.4753 - accuracy: 0.7677 Epoch 57/100 98/98 [==============================] - 0s 801us/step - loss: 0.4750 - accuracy: 0.7693 Epoch 58/100 98/98 [==============================] - 0s 866us/step - loss: 0.4752 - accuracy: 0.7693 Epoch 59/100 98/98 [==============================] - 0s 897us/step - loss: 0.4743 - accuracy: 0.7725 Epoch 60/100 98/98 [==============================] - 0s 907us/step - loss: 0.4736 - accuracy: 0.7715 Epoch 61/100 98/98 [==============================] - 0s 856us/step - loss: 0.4737 - accuracy: 0.7696 Epoch 62/100 98/98 [==============================] - 0s 897us/step - loss: 0.4733 - accuracy: 0.7728 Epoch 63/100 98/98 [==============================] - 0s 959us/step - loss: 0.4726 - accuracy: 0.7747 Epoch 64/100 98/98 [==============================] - 0s 856us/step - loss: 0.4726 - accuracy: 0.7763 Epoch 65/100 98/98 [==============================] - 0s 907us/step - loss: 0.4716 - accuracy: 0.7715 Epoch 66/100 98/98 [==============================] - 0s 846us/step - loss: 0.4721 - accuracy: 0.7738 Epoch 67/100 98/98 [==============================] - 0s 856us/step - loss: 0.4714 - accuracy: 0.7728 Epoch 68/100 98/98 [==============================] - 0s 918us/step - loss: 0.4713 - accuracy: 0.7712 Epoch 69/100 98/98 [==============================] - 0s 856us/step - loss: 0.4704 - accuracy: 0.7734 Epoch 70/100 98/98 [==============================] - 0s 897us/step - loss: 0.4718 - accuracy: 0.7725 Epoch 71/100 98/98 [==============================] - 0s 856us/step - loss: 0.4718 - accuracy: 0.7754 Epoch 72/100 98/98 [==============================] - 0s 866us/step - loss: 0.4713 - accuracy: 0.7706 Epoch 73/100 98/98 [==============================] - 0s 794us/step - loss: 0.4721 - accuracy: 0.7715 Epoch 74/100 98/98 [==============================] - 0s 897us/step - loss: 0.4711 - accuracy: 0.7744 Epoch 75/100 98/98 [==============================] - 0s 1ms/step - loss: 0.4712 - accuracy: 0.7725 Epoch 76/100 98/98 [==============================] - 0s 969us/step - loss: 0.4706 - accuracy: 0.7757 Epoch 77/100 98/98 [==============================] - 0s 980us/step - loss: 0.4707 - accuracy: 0.7725 Epoch 78/100 98/98 [==============================] - 0s 1ms/step - loss: 0.4696 - accuracy: 0.7738 Epoch 79/100 98/98 [==============================] - 0s 1ms/step - loss: 0.4707 - accuracy: 0.7757 Epoch 80/100 98/98 [==============================] - 0s 938us/step - loss: 0.4711 - accuracy: 0.7699 Epoch 81/100 98/98 [==============================] - 0s 1ms/step - loss: 0.4700 - accuracy: 0.7770 Epoch 82/100 98/98 [==============================] - 0s 990us/step - loss: 0.4700 - accuracy: 0.7699 Epoch 83/100 98/98 [==============================] - 0s 1ms/step - loss: 0.4727 - accuracy: 0.7722 Epoch 84/100 98/98 [==============================] - 0s 959us/step - loss: 0.4700 - accuracy: 0.7731 Epoch 85/100 98/98 [==============================] - 0s 928us/step - loss: 0.4688 - accuracy: 0.7738 Epoch 86/100 98/98 [==============================] - 0s 980us/step - loss: 0.4695 - accuracy: 0.7760 Epoch 87/100 98/98 [==============================] - 0s 1ms/step - loss: 0.4699 - accuracy: 0.7744 Epoch 88/100 98/98 [==============================] - 0s 897us/step - loss: 0.4690 - accuracy: 0.7741 Epoch 89/100 98/98 [==============================] - 0s 980us/step - loss: 0.4694 - accuracy: 0.7731 Epoch 90/100 98/98 [==============================] - 0s 1ms/step - loss: 0.4711 - accuracy: 0.7747 Epoch 91/100 98/98 [==============================] - 0s 1ms/step - loss: 0.4679 - accuracy: 0.7763 Epoch 92/100 98/98 [==============================] - 0s 1ms/step - loss: 0.4685 - accuracy: 0.7722 Epoch 93/100 98/98 [==============================] - 0s 1ms/step - loss: 0.4696 - accuracy: 0.7744 Epoch 94/100 98/98 [==============================] - 0s 1ms/step - loss: 0.4707 - accuracy: 0.7709 Epoch 95/100 98/98 [==============================] - 0s 969us/step - loss: 0.4691 - accuracy: 0.7731 Epoch 96/100 98/98 [==============================] - 0s 949us/step - loss: 0.4685 - accuracy: 0.7725 Epoch 97/100 98/98 [==============================] - 0s 877us/step - loss: 0.4684 - accuracy: 0.7766 Epoch 98/100 98/98 [==============================] - 0s 928us/step - loss: 0.4672 - accuracy: 0.7754 Epoch 99/100 98/98 [==============================] - 0s 876us/step - loss: 0.4672 - accuracy: 0.7731 Epoch 100/100 98/98 [==============================] - 0s 877us/step - loss: 0.4683 - accuracy: 0.7702 63/63 [==============================] - 0s 549us/step - loss: 0.4943 - accuracy: 0.7660 [0.4943360388278961, 0.765999972820282] Classification Report: precision recall f1-score support 0 0.93 0.76 0.84 1593 1 0.46 0.77 0.57 407 accuracy 0.77 2000 macro avg 0.69 0.77 0.71 2000 weighted avg 0.83 0.77 0.78 2000 
X_train, y_train = get_train_batch(df2_class0, df2_class1, 1495, 2990)

y_pred2 = ANN(X_train, y_train, X_test, y_test, 'binary_crossentropy')
Epoch 1/100 98/98 [==============================] - 1s 1ms/step - loss: 0.6884 - accuracy: 0.5488 Epoch 2/100 98/98 [==============================] - 0s 938us/step - loss: 0.6573 - accuracy: 0.6077 Epoch 3/100 98/98 [==============================] - 0s 980us/step - loss: 0.6390 - accuracy: 0.6320 Epoch 4/100 98/98 [==============================] - 0s 1ms/step - loss: 0.6256 - accuracy: 0.6406 Epoch 5/100 98/98 [==============================] - 0s 1ms/step - loss: 0.6156 - accuracy: 0.6630 Epoch 6/100 98/98 [==============================] - 0s 918us/step - loss: 0.6066 - accuracy: 0.6749 Epoch 7/100 98/98 [==============================] - 0s 866us/step - loss: 0.5996 - accuracy: 0.6835 Epoch 8/100 98/98 [==============================] - 0s 876us/step - loss: 0.5937 - accuracy: 0.6906 Epoch 9/100 98/98 [==============================] - 0s 1ms/step - loss: 0.5910 - accuracy: 0.6893 Epoch 10/100 98/98 [==============================] - 0s 990us/step - loss: 0.5856 - accuracy: 0.6944 Epoch 11/100 98/98 [==============================] - 0s 825us/step - loss: 0.5827 - accuracy: 0.6970 Epoch 12/100 98/98 [==============================] - 0s 949us/step - loss: 0.5818 - accuracy: 0.7037 Epoch 13/100 98/98 [==============================] - 0s 928us/step - loss: 0.5801 - accuracy: 0.6989 Epoch 14/100 98/98 [==============================] - 0s 897us/step - loss: 0.5788 - accuracy: 0.7062 Epoch 15/100 98/98 [==============================] - 0s 1ms/step - loss: 0.5761 - accuracy: 0.6992 Epoch 16/100 98/98 [==============================] - 0s 938us/step - loss: 0.5750 - accuracy: 0.7050 Epoch 17/100 98/98 [==============================] - 0s 959us/step - loss: 0.5746 - accuracy: 0.7024 Epoch 18/100 98/98 [==============================] - 0s 856us/step - loss: 0.5745 - accuracy: 0.7014 Epoch 19/100 98/98 [==============================] - 0s 866us/step - loss: 0.5756 - accuracy: 0.7088 Epoch 20/100 98/98 [==============================] - 0s 897us/step - loss: 0.5719 - accuracy: 0.7066 Epoch 21/100 98/98 [==============================] - 0s 815us/step - loss: 0.5709 - accuracy: 0.7059 Epoch 22/100 98/98 [==============================] - 0s 887us/step - loss: 0.5708 - accuracy: 0.7091 Epoch 23/100 98/98 [==============================] - 0s 938us/step - loss: 0.5694 - accuracy: 0.7075 Epoch 24/100 98/98 [==============================] - 0s 949us/step - loss: 0.5684 - accuracy: 0.7059 Epoch 25/100 98/98 [==============================] - 0s 804us/step - loss: 0.5691 - accuracy: 0.7088 Epoch 26/100 98/98 [==============================] - 0s 897us/step - loss: 0.5674 - accuracy: 0.7110 Epoch 27/100 98/98 [==============================] - 0s 825us/step - loss: 0.5671 - accuracy: 0.7158 Epoch 28/100 98/98 [==============================] - 0s 856us/step - loss: 0.5667 - accuracy: 0.7123 Epoch 29/100 98/98 [==============================] - 0s 856us/step - loss: 0.5650 - accuracy: 0.7101 Epoch 30/100 98/98 [==============================] - 0s 959us/step - loss: 0.5656 - accuracy: 0.7072 Epoch 31/100 98/98 [==============================] - 0s 980us/step - loss: 0.5637 - accuracy: 0.7155 Epoch 32/100 98/98 [==============================] - 0s 1ms/step - loss: 0.5634 - accuracy: 0.7114 Epoch 33/100 98/98 [==============================] - 0s 907us/step - loss: 0.5629 - accuracy: 0.7155 Epoch 34/100 98/98 [==============================] - 0s 918us/step - loss: 0.5616 - accuracy: 0.7174 Epoch 35/100 98/98 [==============================] - 0s 835us/step - loss: 0.5621 - accuracy: 0.7149 Epoch 36/100 98/98 [==============================] - 0s 1ms/step - loss: 0.5610 - accuracy: 0.7165 Epoch 37/100 98/98 [==============================] - 0s 1ms/step - loss: 0.5595 - accuracy: 0.7206 Epoch 38/100 98/98 [==============================] - 0s 1ms/step - loss: 0.5600 - accuracy: 0.7133 Epoch 39/100 98/98 [==============================] - 0s 1ms/step - loss: 0.5589 - accuracy: 0.7168 Epoch 40/100 98/98 [==============================] - 0s 1ms/step - loss: 0.5578 - accuracy: 0.7194 Epoch 41/100 98/98 [==============================] - 0s 1ms/step - loss: 0.5574 - accuracy: 0.7178 Epoch 42/100 98/98 [==============================] - 0s 990us/step - loss: 0.5568 - accuracy: 0.7194 Epoch 43/100 98/98 [==============================] - 0s 1ms/step - loss: 0.5566 - accuracy: 0.7136 Epoch 44/100 98/98 [==============================] - 0s 815us/step - loss: 0.5557 - accuracy: 0.7200 Epoch 45/100 98/98 [==============================] - 0s 895us/step - loss: 0.5547 - accuracy: 0.7184 Epoch 46/100 98/98 [==============================] - 0s 1ms/step - loss: 0.5556 - accuracy: 0.7178 Epoch 47/100 98/98 [==============================] - 0s 1ms/step - loss: 0.5550 - accuracy: 0.7149 Epoch 48/100 98/98 [==============================] - 0s 969us/step - loss: 0.5529 - accuracy: 0.7206 Epoch 49/100 98/98 [==============================] - 0s 1ms/step - loss: 0.5528 - accuracy: 0.7216 Epoch 50/100 98/98 [==============================] - 0s 887us/step - loss: 0.5525 - accuracy: 0.7194 Epoch 51/100 98/98 [==============================] - 0s 1ms/step - loss: 0.5509 - accuracy: 0.7226 Epoch 52/100 98/98 [==============================] - 0s 1ms/step - loss: 0.5508 - accuracy: 0.7200 Epoch 53/100 98/98 [==============================] - 0s 918us/step - loss: 0.5505 - accuracy: 0.7165 Epoch 54/100 98/98 [==============================] - 0s 942us/step - loss: 0.5497 - accuracy: 0.7229 Epoch 55/100 98/98 [==============================] - 0s 798us/step - loss: 0.5483 - accuracy: 0.7235 Epoch 56/100 98/98 [==============================] - 0s 951us/step - loss: 0.5500 - accuracy: 0.7216 Epoch 57/100 98/98 [==============================] - 0s 825us/step - loss: 0.5475 - accuracy: 0.7219 Epoch 58/100 98/98 [==============================] - 0s 846us/step - loss: 0.5464 - accuracy: 0.7296 Epoch 59/100 98/98 [==============================] - 0s 1ms/step - loss: 0.5467 - accuracy: 0.7306 Epoch 60/100 98/98 [==============================] - 0s 897us/step - loss: 0.5474 - accuracy: 0.7232 Epoch 61/100 98/98 [==============================] - 0s 1ms/step - loss: 0.5451 - accuracy: 0.7242 Epoch 62/100 98/98 [==============================] - 0s 1ms/step - loss: 0.5450 - accuracy: 0.7210 Epoch 63/100 98/98 [==============================] - 0s 866us/step - loss: 0.5444 - accuracy: 0.7270 Epoch 64/100 98/98 [==============================] - 0s 876us/step - loss: 0.5427 - accuracy: 0.7251 Epoch 65/100 98/98 [==============================] - 0s 959us/step - loss: 0.5419 - accuracy: 0.7226 Epoch 66/100 98/98 [==============================] - 0s 846us/step - loss: 0.5418 - accuracy: 0.7235 Epoch 67/100 98/98 [==============================] - 0s 825us/step - loss: 0.5396 - accuracy: 0.7293 Epoch 68/100 98/98 [==============================] - 0s 907us/step - loss: 0.5386 - accuracy: 0.7293 Epoch 69/100 98/98 [==============================] - 0s 1ms/step - loss: 0.5326 - accuracy: 0.7322 Epoch 70/100 98/98 [==============================] - 0s 1ms/step - loss: 0.5276 - accuracy: 0.7376 Epoch 71/100 98/98 [==============================] - 0s 825us/step - loss: 0.5246 - accuracy: 0.7398 Epoch 72/100 98/98 [==============================] - 0s 846us/step - loss: 0.5207 - accuracy: 0.7443 Epoch 73/100 98/98 [==============================] - 0s 815us/step - loss: 0.5147 - accuracy: 0.7488 Epoch 74/100 98/98 [==============================] - 0s 887us/step - loss: 0.5093 - accuracy: 0.7587 Epoch 75/100 98/98 [==============================] - 0s 1ms/step - loss: 0.5058 - accuracy: 0.7584 Epoch 76/100 98/98 [==============================] - 0s 1ms/step - loss: 0.5028 - accuracy: 0.7571 Epoch 77/100 98/98 [==============================] - 0s 938us/step - loss: 0.4994 - accuracy: 0.7600 Epoch 78/100 98/98 [==============================] - 0s 897us/step - loss: 0.4975 - accuracy: 0.7658 Epoch 79/100 98/98 [==============================] - 0s 959us/step - loss: 0.4960 - accuracy: 0.7606 Epoch 80/100 98/98 [==============================] - 0s 928us/step - loss: 0.4948 - accuracy: 0.7658 Epoch 81/100 98/98 [==============================] - 0s 928us/step - loss: 0.4922 - accuracy: 0.7661 Epoch 82/100 98/98 [==============================] - 0s 926us/step - loss: 0.4896 - accuracy: 0.7670 Epoch 83/100 98/98 [==============================] - 0s 813us/step - loss: 0.4897 - accuracy: 0.7626 Epoch 84/100 98/98 [==============================] - 0s 825us/step - loss: 0.4876 - accuracy: 0.7738 Epoch 85/100 98/98 [==============================] - 0s 854us/step - loss: 0.4850 - accuracy: 0.7642 Epoch 86/100 98/98 [==============================] - 0s 844us/step - loss: 0.4861 - accuracy: 0.7690 Epoch 87/100 98/98 [==============================] - 0s 897us/step - loss: 0.4853 - accuracy: 0.7680 Epoch 88/100 98/98 [==============================] - 0s 937us/step - loss: 0.4825 - accuracy: 0.7680 Epoch 89/100 98/98 [==============================] - 0s 897us/step - loss: 0.4810 - accuracy: 0.7709 Epoch 90/100 98/98 [==============================] - 0s 980us/step - loss: 0.4800 - accuracy: 0.7715 Epoch 91/100 98/98 [==============================] - 0s 825us/step - loss: 0.4777 - accuracy: 0.7731 Epoch 92/100 98/98 [==============================] - 0s 1ms/step - loss: 0.4788 - accuracy: 0.7664 Epoch 93/100 98/98 [==============================] - 0s 1ms/step - loss: 0.4750 - accuracy: 0.7718 Epoch 94/100 98/98 [==============================] - 0s 1ms/step - loss: 0.4737 - accuracy: 0.7722 Epoch 95/100 98/98 [==============================] - 0s 1ms/step - loss: 0.4717 - accuracy: 0.7766 Epoch 96/100 98/98 [==============================] - 0s 969us/step - loss: 0.4713 - accuracy: 0.7766 Epoch 97/100 98/98 [==============================] - 0s 876us/step - loss: 0.4707 - accuracy: 0.7722 Epoch 98/100 98/98 [==============================] - 0s 887us/step - loss: 0.4694 - accuracy: 0.7731 Epoch 99/100 98/98 [==============================] - 0s 866us/step - loss: 0.4695 - accuracy: 0.7741 Epoch 100/100 98/98 [==============================] - 0s 825us/step - loss: 0.4687 - accuracy: 0.7738 63/63 [==============================] - 0s 790us/step - loss: 0.4739 - accuracy: 0.7680 [0.4739205837249756, 0.7680000066757202] Classification Report: precision recall f1-score support 0 0.92 0.77 0.84 1593 1 0.46 0.75 0.57 407 accuracy 0.77 2000 macro avg 0.69 0.76 0.70 2000 weighted avg 0.83 0.77 0.79 2000 
X_train, y_train = get_train_batch(df2_class0, df2_class1, 2990, 4130)

y_pred3 = ANN(X_train, y_train, X_test, y_test, 'binary_crossentropy')
Epoch 1/100 87/87 [==============================] - 1s 861us/step - loss: 0.6886 - accuracy: 0.5347 Epoch 2/100 87/87 [==============================] - 0s 839us/step - loss: 0.6398 - accuracy: 0.6484 Epoch 3/100 87/87 [==============================] - 0s 1ms/step - loss: 0.6135 - accuracy: 0.6704 Epoch 4/100 87/87 [==============================] - 0s 885us/step - loss: 0.5996 - accuracy: 0.6827 Epoch 5/100 87/87 [==============================] - 0s 861us/step - loss: 0.5903 - accuracy: 0.6870 Epoch 6/100 87/87 [==============================] - 0s 849us/step - loss: 0.5821 - accuracy: 0.6971 Epoch 7/100 87/87 [==============================] - 0s 1ms/step - loss: 0.5762 - accuracy: 0.6982 Epoch 8/100 87/87 [==============================] - 0s 2ms/step - loss: 0.5701 - accuracy: 0.7079 Epoch 9/100 87/87 [==============================] - 0s 1ms/step - loss: 0.5646 - accuracy: 0.7094 Epoch 10/100 87/87 [==============================] - 0s 1ms/step - loss: 0.5586 - accuracy: 0.7130 Epoch 11/100 87/87 [==============================] - 0s 942us/step - loss: 0.5530 - accuracy: 0.7227 Epoch 12/100 87/87 [==============================] - 0s 930us/step - loss: 0.5481 - accuracy: 0.7242 Epoch 13/100 87/87 [==============================] - 0s 954us/step - loss: 0.5451 - accuracy: 0.7292 Epoch 14/100 87/87 [==============================] - 0s 1ms/step - loss: 0.5407 - accuracy: 0.7354 Epoch 15/100 87/87 [==============================] - 0s 960us/step - loss: 0.5383 - accuracy: 0.7372 Epoch 16/100 87/87 [==============================] - 0s 873us/step - loss: 0.5322 - accuracy: 0.7451 Epoch 17/100 87/87 [==============================] - 0s 838us/step - loss: 0.5296 - accuracy: 0.7484 Epoch 18/100 87/87 [==============================] - 0s 919us/step - loss: 0.5273 - accuracy: 0.7516 Epoch 19/100 87/87 [==============================] - 0s 933us/step - loss: 0.5268 - accuracy: 0.7509 Epoch 20/100 87/87 [==============================] - 0s 898us/step - loss: 0.5219 - accuracy: 0.7552 Epoch 21/100 87/87 [==============================] - 0s 847us/step - loss: 0.5166 - accuracy: 0.7542 Epoch 22/100 87/87 [==============================] - 0s 922us/step - loss: 0.5120 - accuracy: 0.7599 Epoch 23/100 87/87 [==============================] - 0s 1ms/step - loss: 0.5076 - accuracy: 0.7614 Epoch 24/100 87/87 [==============================] - 0s 897us/step - loss: 0.5035 - accuracy: 0.7635 Epoch 25/100 87/87 [==============================] - 0s 919us/step - loss: 0.4997 - accuracy: 0.7646 Epoch 26/100 87/87 [==============================] - 0s 1ms/step - loss: 0.4954 - accuracy: 0.7668 Epoch 27/100 87/87 [==============================] - 0s 941us/step - loss: 0.4922 - accuracy: 0.7708 Epoch 28/100 87/87 [==============================] - 0s 815us/step - loss: 0.4893 - accuracy: 0.7733 Epoch 29/100 87/87 [==============================] - 0s 861us/step - loss: 0.4871 - accuracy: 0.7708 Epoch 30/100 87/87 [==============================] - 0s 854us/step - loss: 0.4844 - accuracy: 0.7758 Epoch 31/100 87/87 [==============================] - 0s 965us/step - loss: 0.4810 - accuracy: 0.7765 Epoch 32/100 87/87 [==============================] - 0s 954us/step - loss: 0.4797 - accuracy: 0.7718 Epoch 33/100 87/87 [==============================] - 0s 896us/step - loss: 0.4784 - accuracy: 0.7747 Epoch 34/100 87/87 [==============================] - 0s 919us/step - loss: 0.4763 - accuracy: 0.7787 Epoch 35/100 87/87 [==============================] - 0s 837us/step - loss: 0.4742 - accuracy: 0.7747 Epoch 36/100 87/87 [==============================] - 0s 896us/step - loss: 0.4732 - accuracy: 0.7791 Epoch 37/100 87/87 [==============================] - 0s 826us/step - loss: 0.4727 - accuracy: 0.7773 Epoch 38/100 87/87 [==============================] - 0s 884us/step - loss: 0.4714 - accuracy: 0.7744 Epoch 39/100 87/87 [==============================] - 0s 826us/step - loss: 0.4708 - accuracy: 0.7780 Epoch 40/100 87/87 [==============================] - 0s 896us/step - loss: 0.4690 - accuracy: 0.7827 Epoch 41/100 87/87 [==============================] - 0s 884us/step - loss: 0.4697 - accuracy: 0.7733 Epoch 42/100 87/87 [==============================] - 0s 942us/step - loss: 0.4692 - accuracy: 0.7798 Epoch 43/100 87/87 [==============================] - 0s 1ms/step - loss: 0.4669 - accuracy: 0.7780 Epoch 44/100 87/87 [==============================] - 0s 1ms/step - loss: 0.4667 - accuracy: 0.7816 Epoch 45/100 87/87 [==============================] - 0s 837us/step - loss: 0.4659 - accuracy: 0.7798 Epoch 46/100 87/87 [==============================] - 0s 849us/step - loss: 0.4659 - accuracy: 0.7809 Epoch 47/100 87/87 [==============================] - 0s 907us/step - loss: 0.4653 - accuracy: 0.7783 Epoch 48/100 87/87 [==============================] - 0s 1ms/step - loss: 0.4655 - accuracy: 0.7830 Epoch 49/100 87/87 [==============================] - 0s 907us/step - loss: 0.4635 - accuracy: 0.7845 Epoch 50/100 87/87 [==============================] - 0s 1ms/step - loss: 0.4634 - accuracy: 0.7801 Epoch 51/100 87/87 [==============================] - 0s 1ms/step - loss: 0.4622 - accuracy: 0.7805 Epoch 52/100 87/87 [==============================] - 0s 1ms/step - loss: 0.4614 - accuracy: 0.7841 Epoch 53/100 87/87 [==============================] - 0s 965us/step - loss: 0.4618 - accuracy: 0.7848 Epoch 54/100 87/87 [==============================] - 0s 884us/step - loss: 0.4610 - accuracy: 0.7866 Epoch 55/100 87/87 [==============================] - 0s 837us/step - loss: 0.4600 - accuracy: 0.7888 Epoch 56/100 87/87 [==============================] - 0s 826us/step - loss: 0.4597 - accuracy: 0.7888 Epoch 57/100 87/87 [==============================] - 0s 814us/step - loss: 0.4592 - accuracy: 0.7884 Epoch 58/100 87/87 [==============================] - 0s 826us/step - loss: 0.4595 - accuracy: 0.7823 Epoch 59/100 87/87 [==============================] - 0s 872us/step - loss: 0.4579 - accuracy: 0.7845 Epoch 60/100 87/87 [==============================] - 0s 849us/step - loss: 0.4577 - accuracy: 0.7841 Epoch 61/100 87/87 [==============================] - 0s 861us/step - loss: 0.4591 - accuracy: 0.7884 Epoch 62/100 87/87 [==============================] - 0s 872us/step - loss: 0.4572 - accuracy: 0.7866 Epoch 63/100 87/87 [==============================] - 0s 861us/step - loss: 0.4564 - accuracy: 0.7859 Epoch 64/100 87/87 [==============================] - 0s 954us/step - loss: 0.4555 - accuracy: 0.7906 Epoch 65/100 87/87 [==============================] - 0s 861us/step - loss: 0.4573 - accuracy: 0.7848 Epoch 66/100 87/87 [==============================] - 0s 896us/step - loss: 0.4561 - accuracy: 0.7874 Epoch 67/100 87/87 [==============================] - 0s 907us/step - loss: 0.4558 - accuracy: 0.7866 Epoch 68/100 87/87 [==============================] - 0s 896us/step - loss: 0.4565 - accuracy: 0.7884 Epoch 69/100 87/87 [==============================] - 0s 814us/step - loss: 0.4548 - accuracy: 0.7874 Epoch 70/100 87/87 [==============================] - 0s 1ms/step - loss: 0.4553 - accuracy: 0.7913 Epoch 71/100 87/87 [==============================] - 0s 886us/step - loss: 0.4534 - accuracy: 0.7921 Epoch 72/100 87/87 [==============================] - 0s 859us/step - loss: 0.4544 - accuracy: 0.7838 Epoch 73/100 87/87 [==============================] - 0s 882us/step - loss: 0.4542 - accuracy: 0.7856 Epoch 74/100 87/87 [==============================] - 0s 966us/step - loss: 0.4532 - accuracy: 0.7866 Epoch 75/100 87/87 [==============================] - 0s 899us/step - loss: 0.4529 - accuracy: 0.7892 Epoch 76/100 87/87 [==============================] - 0s 921us/step - loss: 0.4533 - accuracy: 0.7899 Epoch 77/100 87/87 [==============================] - 0s 931us/step - loss: 0.4522 - accuracy: 0.7917 Epoch 78/100 87/87 [==============================] - 0s 901us/step - loss: 0.4530 - accuracy: 0.7888 Epoch 79/100 87/87 [==============================] - 0s 907us/step - loss: 0.4522 - accuracy: 0.7863 Epoch 80/100 87/87 [==============================] - 0s 849us/step - loss: 0.4514 - accuracy: 0.7884 Epoch 81/100 87/87 [==============================] - 0s 837us/step - loss: 0.4505 - accuracy: 0.7888 Epoch 82/100 87/87 [==============================] - 0s 907us/step - loss: 0.4519 - accuracy: 0.7906 Epoch 83/100 87/87 [==============================] - 0s 1ms/step - loss: 0.4535 - accuracy: 0.7892 Epoch 84/100 87/87 [==============================] - 0s 861us/step - loss: 0.4501 - accuracy: 0.7892 Epoch 85/100 87/87 [==============================] - 0s 814us/step - loss: 0.4506 - accuracy: 0.7917 Epoch 86/100 87/87 [==============================] - 0s 837us/step - loss: 0.4494 - accuracy: 0.7910 Epoch 87/100 87/87 [==============================] - 0s 896us/step - loss: 0.4497 - accuracy: 0.7863 Epoch 88/100 87/87 [==============================] - 0s 1ms/step - loss: 0.4497 - accuracy: 0.7888 Epoch 89/100 87/87 [==============================] - 0s 896us/step - loss: 0.4491 - accuracy: 0.7899 Epoch 90/100 87/87 [==============================] - 0s 989us/step - loss: 0.4485 - accuracy: 0.7906 Epoch 91/100 87/87 [==============================] - 0s 930us/step - loss: 0.4484 - accuracy: 0.7845 Epoch 92/100 87/87 [==============================] - 0s 837us/step - loss: 0.4517 - accuracy: 0.7823 Epoch 93/100 87/87 [==============================] - 0s 884us/step - loss: 0.4489 - accuracy: 0.7888 Epoch 94/100 87/87 [==============================] - 0s 872us/step - loss: 0.4485 - accuracy: 0.7877 Epoch 95/100 87/87 [==============================] - 0s 965us/step - loss: 0.4484 - accuracy: 0.7895 Epoch 96/100 87/87 [==============================] - 0s 872us/step - loss: 0.4479 - accuracy: 0.7881 Epoch 97/100 87/87 [==============================] - 0s 977us/step - loss: 0.4491 - accuracy: 0.7881 Epoch 98/100 87/87 [==============================] - 0s 989us/step - loss: 0.4484 - accuracy: 0.7935 Epoch 99/100 87/87 [==============================] - 0s 977us/step - loss: 0.4480 - accuracy: 0.7834 Epoch 100/100 87/87 [==============================] - 0s 907us/step - loss: 0.4470 - accuracy: 0.7903 63/63 [==============================] - 0s 597us/step - loss: 0.5767 - accuracy: 0.7020 [0.5767270922660828, 0.7020000219345093] Classification Report: precision recall f1-score support 0 0.95 0.66 0.78 1593 1 0.39 0.86 0.54 407 accuracy 0.70 2000 macro avg 0.67 0.76 0.66 2000 weighted avg 0.84 0.70 0.73 2000 
y_pred_final = y_pred1.copy()
for i in range(len(y_pred1)):
    n_ones = y_pred1[i] + y_pred2[i] + y_pred3[i]
    if n_ones > 1:
        y_pred_final[i] = 1
    else:
        y_pred_final[i] = 0

cl_rep = classification_report(y_test, y_pred_final)
print(cl_rep)
precision recall f1-score support 0 0.94 0.73 0.82 1593 1 0.44 0.81 0.57 407 accuracy 0.75 2000 macro avg 0.69 0.77 0.69 2000 weighted avg 0.83 0.75 0.77 2000

f1-score for minority class 1 is 0.57