CoCalc -- handling_imbalanced

GitHub Repository: codebasics/deep-learning-keras-tf-tutorial
Path: blob/master/14_imbalanced/handling_imbalanced_data.ipynb
¹¹⁴¹ views

Kernel: Python 3

Handling imbalanced data in customer churn prediction

Customer churn prediction is to measure why customers are leaving a business. In this tutorial we will be looking at customer churn in telecom business. We will build a deep learning model to predict the churn and use precision,recall, f1-score to measure performance of our model. We will then handle imbalance in data using various techniques and improve f1-score

In [5]:

import pandas as pd
from matplotlib import pyplot as plt
import numpy as np
%matplotlib inline

In [6]:

import warnings
warnings.filterwarnings('ignore')

Load the data

In [7]:

df = pd.read_csv("customer_churn.csv")
df.sample(5)

Out[7]:

In [8]:

df.Churn.value_counts()

Out[8]:

No     5174
Yes    1869
Name: Churn, dtype: int64

In [9]:

517400/df.shape[0]

Out[9]:

73.46301292063042

First of all, drop customerID column as it is of no use

In [10]:

df.drop('customerID',axis='columns',inplace=True)

In [11]:

df.dtypes

Out[11]:

gender               object
SeniorCitizen         int64
Partner              object
Dependents           object
tenure                int64
PhoneService         object
MultipleLines        object
InternetService      object
OnlineSecurity       object
OnlineBackup         object
DeviceProtection     object
TechSupport          object
StreamingTV          object
StreamingMovies      object
Contract             object
PaperlessBilling     object
PaymentMethod        object
MonthlyCharges      float64
TotalCharges         object
Churn                object
dtype: object

Quick glance at above makes me realize that TotalCharges should be float but it is an object. Let's check what's going on with this column

In [12]:

df.TotalCharges.values

Out[12]:

array(['29.85', '1889.5', '108.15', ..., '346.45', '306.6', '6844.5'],
      dtype=object)

Ahh... it is string. Lets convert it to numbers

In [13]:

pd.to_numeric(df.TotalCharges,errors='coerce').isnull()

Out[13]:

     False
     False
     False
     False
     False
        ...  
  False
  False
  False
  False
  False
Name: TotalCharges, Length: 7043, dtype: bool

In [14]:

df[pd.to_numeric(df.TotalCharges,errors='coerce').isnull()]

Out[14]:

In [15]:

df.shape

Out[15]:

(7043, 20)

In [16]:

df.iloc[488].TotalCharges

Out[16]:

' '

In [17]:

df[df.TotalCharges!=' '].shape

Out[17]:

(7032, 20)

Remove rows with space in TotalCharges

In [18]:

df1 = df[df.TotalCharges!=' ']
df1.shape

Out[18]:

(7032, 20)

In [19]:

df1.dtypes

Out[19]:

gender               object
SeniorCitizen         int64
Partner              object
Dependents           object
tenure                int64
PhoneService         object
MultipleLines        object
InternetService      object
OnlineSecurity       object
OnlineBackup         object
DeviceProtection     object
TechSupport          object
StreamingTV          object
StreamingMovies      object
Contract             object
PaperlessBilling     object
PaymentMethod        object
MonthlyCharges      float64
TotalCharges         object
Churn                object
dtype: object

In [20]:

df1.TotalCharges = pd.to_numeric(df1.TotalCharges)

In [21]:

df1.TotalCharges.values

Out[21]:

array([  29.85, 1889.5 ,  108.15, ...,  346.45,  306.6 , 6844.5 ])

In [22]:

df1[df1.Churn=='No']

Out[22]:

Data Visualization

In [23]:

tenure_churn_no = df1[df1.Churn=='No'].tenure
tenure_churn_yes = df1[df1.Churn=='Yes'].tenure

plt.xlabel("tenure")
plt.ylabel("Number Of Customers")
plt.title("Customer Churn Prediction Visualiztion")

blood_sugar_men = [113, 85, 90, 150, 149, 88, 93, 115, 135, 80, 77, 82, 129]
blood_sugar_women = [67, 98, 89, 120, 133, 150, 84, 69, 89, 79, 120, 112, 100]

plt.hist([tenure_churn_yes, tenure_churn_no], rwidth=0.95, color=['green','red'],label=['Churn=Yes','Churn=No'])
plt.legend()

Out[23]:

<matplotlib.legend.Legend at 0x25057e46190>

In [24]:

mc_churn_no = df1[df1.Churn=='No'].MonthlyCharges      
mc_churn_yes = df1[df1.Churn=='Yes'].MonthlyCharges      

plt.xlabel("Monthly Charges")
plt.ylabel("Number Of Customers")
plt.title("Customer Churn Prediction Visualiztion")

blood_sugar_men = [113, 85, 90, 150, 149, 88, 93, 115, 135, 80, 77, 82, 129]
blood_sugar_women = [67, 98, 89, 120, 133, 150, 84, 69, 89, 79, 120, 112, 100]

plt.hist([mc_churn_yes, mc_churn_no], rwidth=0.95, color=['green','red'],label=['Churn=Yes','Churn=No'])
plt.legend()

Out[24]:

<matplotlib.legend.Legend at 0x2505b006a30>

Many of the columns are yes, no etc. Let's print unique values in object columns to see data values

In [25]:

def print_unique_col_values(df):
       for column in df:
            if df[column].dtypes=='object':
                print(f'{column}: {df[column].unique()}')

In [26]:

print_unique_col_values(df1)

Out[26]:

gender: ['Female' 'Male']
Partner: ['Yes' 'No']
Dependents: ['No' 'Yes']
PhoneService: ['No' 'Yes']
MultipleLines: ['No phone service' 'No' 'Yes']
InternetService: ['DSL' 'Fiber optic' 'No']
OnlineSecurity: ['No' 'Yes' 'No internet service']
OnlineBackup: ['Yes' 'No' 'No internet service']
DeviceProtection: ['No' 'Yes' 'No internet service']
TechSupport: ['No' 'Yes' 'No internet service']
StreamingTV: ['No' 'Yes' 'No internet service']
StreamingMovies: ['No' 'Yes' 'No internet service']
Contract: ['Month-to-month' 'One year' 'Two year']
PaperlessBilling: ['Yes' 'No']
PaymentMethod: ['Electronic check' 'Mailed check' 'Bank transfer (automatic)'
 'Credit card (automatic)']
Churn: ['No' 'Yes']

Some of the columns have no internet service or no phone service, that can be replaced with a simple No

In [27]:

df1.replace('No internet service','No',inplace=True)
df1.replace('No phone service','No',inplace=True)

In [28]:

print_unique_col_values(df1)

Out[28]:

gender: ['Female' 'Male']
Partner: ['Yes' 'No']
Dependents: ['No' 'Yes']
PhoneService: ['No' 'Yes']
MultipleLines: ['No' 'Yes']
InternetService: ['DSL' 'Fiber optic' 'No']
OnlineSecurity: ['No' 'Yes']
OnlineBackup: ['Yes' 'No']
DeviceProtection: ['No' 'Yes']
TechSupport: ['No' 'Yes']
StreamingTV: ['No' 'Yes']
StreamingMovies: ['No' 'Yes']
Contract: ['Month-to-month' 'One year' 'Two year']
PaperlessBilling: ['Yes' 'No']
PaymentMethod: ['Electronic check' 'Mailed check' 'Bank transfer (automatic)'
 'Credit card (automatic)']
Churn: ['No' 'Yes']

Convert Yes and No to 1 or 0

In [29]:

yes_no_columns = ['Partner','Dependents','PhoneService','MultipleLines','OnlineSecurity','OnlineBackup',
                  'DeviceProtection','TechSupport','StreamingTV','StreamingMovies','PaperlessBilling','Churn']
for col in yes_no_columns:
    df1[col].replace({'Yes': 1,'No': 0},inplace=True)

In [30]:

for col in df1:
    print(f'{col}: {df1[col].unique()}')

Out[30]:

gender: ['Female' 'Male']
SeniorCitizen: [0 1]
Partner: [1 0]
Dependents: [0 1]
tenure: [ 1 34  2 45  8 22 10 28 62 13 16 58 49 25 69 52 71 21 12 30 47 72 17 27
  5 46 11 70 63 43 15 60 18 66  9  3 31 50 64 56  7 42 35 48 29 65 38 68
 32 55 37 36 41  6  4 33 67 23 57 61 14 20 53 40 59 24 44 19 54 51 26 39]
PhoneService: [0 1]
MultipleLines: [0 1]
InternetService: ['DSL' 'Fiber optic' 'No']
OnlineSecurity: [0 1]
OnlineBackup: [1 0]
DeviceProtection: [0 1]
TechSupport: [0 1]
StreamingTV: [0 1]
StreamingMovies: [0 1]
Contract: ['Month-to-month' 'One year' 'Two year']
PaperlessBilling: [1 0]
PaymentMethod: ['Electronic check' 'Mailed check' 'Bank transfer (automatic)'
 'Credit card (automatic)']
MonthlyCharges: [29.85 56.95 53.85 ... 63.1  44.2  78.7 ]
TotalCharges: [  29.85 1889.5   108.15 ...  346.45  306.6  6844.5 ]
Churn: [0 1]

In [31]:

df1['gender'].replace({'Female':1,'Male':0},inplace=True)

In [32]:

df1.gender.unique()

Out[32]:

array([1, 0], dtype=int64)

One hot encoding for categorical columns

In [33]:

df2 = pd.get_dummies(data=df1, columns=['InternetService','Contract','PaymentMethod'])
df2.columns

Out[33]:

Index(['gender', 'SeniorCitizen', 'Partner', 'Dependents', 'tenure',
       'PhoneService', 'MultipleLines', 'OnlineSecurity', 'OnlineBackup',
       'DeviceProtection', 'TechSupport', 'StreamingTV', 'StreamingMovies',
       'PaperlessBilling', 'MonthlyCharges', 'TotalCharges', 'Churn',
       'InternetService_DSL', 'InternetService_Fiber optic',
       'InternetService_No', 'Contract_Month-to-month', 'Contract_One year',
       'Contract_Two year', 'PaymentMethod_Bank transfer (automatic)',
       'PaymentMethod_Credit card (automatic)',
       'PaymentMethod_Electronic check', 'PaymentMethod_Mailed check'],
      dtype='object')

In [34]:

df2.sample(5)

Out[34]:

In [35]:

df2.dtypes

Out[35]:

gender                                       int64
SeniorCitizen                                int64
Partner                                      int64
Dependents                                   int64
tenure                                       int64
PhoneService                                 int64
MultipleLines                                int64
OnlineSecurity                               int64
OnlineBackup                                 int64
DeviceProtection                             int64
TechSupport                                  int64
StreamingTV                                  int64
StreamingMovies                              int64
PaperlessBilling                             int64
MonthlyCharges                             float64
TotalCharges                               float64
Churn                                        int64
InternetService_DSL                          uint8
InternetService_Fiber optic                  uint8
InternetService_No                           uint8
Contract_Month-to-month                      uint8
Contract_One year                            uint8
Contract_Two year                            uint8
PaymentMethod_Bank transfer (automatic)      uint8
PaymentMethod_Credit card (automatic)        uint8
PaymentMethod_Electronic check               uint8
PaymentMethod_Mailed check                   uint8
dtype: object

In [36]:

cols_to_scale = ['tenure','MonthlyCharges','TotalCharges']

from sklearn.preprocessing import MinMaxScaler
scaler = MinMaxScaler()
df2[cols_to_scale] = scaler.fit_transform(df2[cols_to_scale])

In [37]:

for col in df2:
    print(f'{col}: {df2[col].unique()}')

Out[37]:

gender: [1 0]
SeniorCitizen: [0 1]
Partner: [1 0]
Dependents: [0 1]
tenure: [0.         0.46478873 0.01408451 0.61971831 0.09859155 0.29577465
 0.12676056 0.38028169 0.85915493 0.16901408 0.21126761 0.8028169
 0.67605634 0.33802817 0.95774648 0.71830986 0.98591549 0.28169014
 0.15492958 0.4084507  0.64788732 1.         0.22535211 0.36619718
 0.05633803 0.63380282 0.14084507 0.97183099 0.87323944 0.5915493
 0.1971831  0.83098592 0.23943662 0.91549296 0.11267606 0.02816901
 0.42253521 0.69014085 0.88732394 0.77464789 0.08450704 0.57746479
 0.47887324 0.66197183 0.3943662  0.90140845 0.52112676 0.94366197
 0.43661972 0.76056338 0.50704225 0.49295775 0.56338028 0.07042254
 0.04225352 0.45070423 0.92957746 0.30985915 0.78873239 0.84507042
 0.18309859 0.26760563 0.73239437 0.54929577 0.81690141 0.32394366
 0.6056338  0.25352113 0.74647887 0.70422535 0.35211268 0.53521127]
PhoneService: [0 1]
MultipleLines: [0 1]
OnlineSecurity: [0 1]
OnlineBackup: [1 0]
DeviceProtection: [0 1]
TechSupport: [0 1]
StreamingTV: [0 1]
StreamingMovies: [0 1]
PaperlessBilling: [1 0]
MonthlyCharges: [0.11542289 0.38507463 0.35422886 ... 0.44626866 0.25820896 0.60149254]
TotalCharges: [0.0012751  0.21586661 0.01031041 ... 0.03780868 0.03321025 0.78764136]
Churn: [0 1]
InternetService_DSL: [1 0]
InternetService_Fiber optic: [0 1]
InternetService_No: [0 1]
Contract_Month-to-month: [1 0]
Contract_One year: [0 1]
Contract_Two year: [0 1]
PaymentMethod_Bank transfer (automatic): [0 1]
PaymentMethod_Credit card (automatic): [0 1]
PaymentMethod_Electronic check: [1 0]
PaymentMethod_Mailed check: [0 1]

Train test split

In [38]:

X = df2.drop('Churn',axis='columns')
y = testLabels = df2.Churn.astype(np.float32)

from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=15, stratify=y)

In [39]:

y_train.value_counts()

Out[39]:

0.0    4130
1.0    1495
Name: Churn, dtype: int64

In [40]:

y.value_counts()

Out[40]:

0.0    5163
1.0    1869
Name: Churn, dtype: int64

In [41]:

5163/1869

Out[41]:

2.7624398073836276

In [42]:

y_test.value_counts()

Out[42]:

0.0    1033
1.0     374
Name: Churn, dtype: int64

In [43]:

X_train.shape

Out[43]:

(5625, 26)

In [44]:

X_test.shape

Out[44]:

(1407, 26)

In [45]:

X_train[:10]

Out[45]:

In [46]:

len(X_train.columns)

Out[46]:

26

Build a model (ANN) in tensorflow/keras

In [47]:

from tensorflow_addons import losses

In [48]:

import tensorflow as tf
from tensorflow import keras
from sklearn.metrics import confusion_matrix , classification_report

In [49]:

def ANN(X_train, y_train, X_test, y_test, loss, weights):
    model = keras.Sequential([
        keras.layers.Dense(26, input_dim=26, activation='relu'),
        keras.layers.Dense(15, activation='relu'),
        keras.layers.Dense(1, activation='sigmoid')
    ])

    model.compile(optimizer='adam', loss=loss, metrics=['accuracy'])
    
    if weights == -1:
        model.fit(X_train, y_train, epochs=100)
    else:
        model.fit(X_train, y_train, epochs=100, class_weight = weights)
    
    print(model.evaluate(X_test, y_test))
    
    y_preds = model.predict(X_test)
    y_preds = np.round(y_preds)
    
    print("Classification Report: \n", classification_report(y_test, y_preds))
    
    return y_preds

In [50]:

y_preds = ANN(X_train, y_train, X_test, y_test, 'binary_crossentropy', -1)

Out[50]:

Epoch 1/100
176/176 [==============================] - 0s 1ms/step - loss: 0.4940 - accuracy: 0.7593
Epoch 2/100
176/176 [==============================] - 0s 1ms/step - loss: 0.4263 - accuracy: 0.7988
Epoch 3/100
176/176 [==============================] - 0s 1ms/step - loss: 0.4201 - accuracy: 0.7984
Epoch 4/100
176/176 [==============================] - 0s 1ms/step - loss: 0.4177 - accuracy: 0.7995
Epoch 5/100
176/176 [==============================] - 0s 2ms/step - loss: 0.4121 - accuracy: 0.8041
Epoch 6/100
176/176 [==============================] - 0s 1ms/step - loss: 0.4113 - accuracy: 0.8060
Epoch 7/100
176/176 [==============================] - 0s 1ms/step - loss: 0.4088 - accuracy: 0.8080
Epoch 8/100
176/176 [==============================] - 0s 1ms/step - loss: 0.4075 - accuracy: 0.8064
Epoch 9/100
176/176 [==============================] - 0s 1ms/step - loss: 0.4058 - accuracy: 0.8059
Epoch 10/100
176/176 [==============================] - 0s 1ms/step - loss: 0.4047 - accuracy: 0.8089
Epoch 11/100
176/176 [==============================] - 0s 1ms/step - loss: 0.4039 - accuracy: 0.8071
Epoch 12/100
176/176 [==============================] - 0s 1ms/step - loss: 0.4023 - accuracy: 0.8105
Epoch 13/100
176/176 [==============================] - 0s 1ms/step - loss: 0.4014 - accuracy: 0.8110
Epoch 14/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3996 - accuracy: 0.8068
Epoch 15/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3990 - accuracy: 0.8094
Epoch 16/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3974 - accuracy: 0.8110
Epoch 17/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3961 - accuracy: 0.8108
Epoch 18/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3944 - accuracy: 0.8151
Epoch 19/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3951 - accuracy: 0.8119
Epoch 20/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3932 - accuracy: 0.8119
Epoch 21/100
176/176 [==============================] - 0s 2ms/step - loss: 0.3917 - accuracy: 0.8171
Epoch 22/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3918 - accuracy: 0.8153
Epoch 23/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3908 - accuracy: 0.8165
Epoch 24/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3901 - accuracy: 0.8135
Epoch 25/100
176/176 [==============================] - 0s 2ms/step - loss: 0.3891 - accuracy: 0.8130
Epoch 26/100
176/176 [==============================] - 0s 2ms/step - loss: 0.3876 - accuracy: 0.8158
Epoch 27/100
176/176 [==============================] - 0s 2ms/step - loss: 0.3871 - accuracy: 0.8183
Epoch 28/100
176/176 [==============================] - 0s 2ms/step - loss: 0.3860 - accuracy: 0.8190
Epoch 29/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3868 - accuracy: 0.8180
Epoch 30/100
176/176 [==============================] - 0s 2ms/step - loss: 0.3843 - accuracy: 0.8181
Epoch 31/100
176/176 [==============================] - 0s 2ms/step - loss: 0.3845 - accuracy: 0.8155
Epoch 32/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3828 - accuracy: 0.8213
Epoch 33/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3836 - accuracy: 0.8203
Epoch 34/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3822 - accuracy: 0.8180
Epoch 35/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3810 - accuracy: 0.8194
Epoch 36/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3805 - accuracy: 0.8197
Epoch 37/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3807 - accuracy: 0.8204
Epoch 38/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3799 - accuracy: 0.8187
Epoch 39/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3778 - accuracy: 0.8229
Epoch 40/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3768 - accuracy: 0.8210
Epoch 41/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3762 - accuracy: 0.8199
Epoch 42/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3758 - accuracy: 0.8226
Epoch 43/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3767 - accuracy: 0.8233
Epoch 44/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3753 - accuracy: 0.8260
Epoch 45/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3732 - accuracy: 0.8222
Epoch 46/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3740 - accuracy: 0.8235
Epoch 47/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3729 - accuracy: 0.8244
Epoch 48/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3723 - accuracy: 0.8249
Epoch 49/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3721 - accuracy: 0.8254
Epoch 50/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3710 - accuracy: 0.8265
Epoch 51/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3712 - accuracy: 0.8265
Epoch 52/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3688 - accuracy: 0.8274
Epoch 53/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3692 - accuracy: 0.8242
Epoch 54/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3693 - accuracy: 0.8260
Epoch 55/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3684 - accuracy: 0.8242
Epoch 56/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3680 - accuracy: 0.8244
Epoch 57/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3664 - accuracy: 0.8272
Epoch 58/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3670 - accuracy: 0.8238
Epoch 59/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3670 - accuracy: 0.8238
Epoch 60/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3661 - accuracy: 0.8265
Epoch 61/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3654 - accuracy: 0.8288
Epoch 62/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3663 - accuracy: 0.8290
Epoch 63/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3644 - accuracy: 0.8265
Epoch 64/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3635 - accuracy: 0.8290
Epoch 65/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3630 - accuracy: 0.8272
Epoch 66/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3615 - accuracy: 0.8279
Epoch 67/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3630 - accuracy: 0.8274
Epoch 68/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3625 - accuracy: 0.8265
Epoch 69/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3612 - accuracy: 0.8277
Epoch 70/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3607 - accuracy: 0.8284
Epoch 71/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3604 - accuracy: 0.8309
Epoch 72/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3604 - accuracy: 0.8295
Epoch 73/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3590 - accuracy: 0.8284
Epoch 74/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3586 - accuracy: 0.8300
Epoch 75/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3583 - accuracy: 0.8300
Epoch 76/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3575 - accuracy: 0.8318
Epoch 77/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3575 - accuracy: 0.8334
Epoch 78/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3563 - accuracy: 0.8345
Epoch 79/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3580 - accuracy: 0.8309
Epoch 80/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3554 - accuracy: 0.8320
Epoch 81/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3563 - accuracy: 0.8322
Epoch 82/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3546 - accuracy: 0.8345
Epoch 83/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3562 - accuracy: 0.8293
Epoch 84/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3536 - accuracy: 0.8322
Epoch 85/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3544 - accuracy: 0.8308
Epoch 86/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3522 - accuracy: 0.8350
Epoch 87/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3527 - accuracy: 0.8329
Epoch 88/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3517 - accuracy: 0.8329
Epoch 89/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3513 - accuracy: 0.8334
Epoch 90/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3515 - accuracy: 0.8332
Epoch 91/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3509 - accuracy: 0.8354
Epoch 92/100
176/176 [==============================] - 0s 2ms/step - loss: 0.3506 - accuracy: 0.8375
Epoch 93/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3503 - accuracy: 0.8336
Epoch 94/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3490 - accuracy: 0.8338
Epoch 95/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3481 - accuracy: 0.8331
Epoch 96/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3481 - accuracy: 0.8377
Epoch 97/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3473 - accuracy: 0.8380
Epoch 98/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3471 - accuracy: 0.8359
Epoch 99/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3475 - accuracy: 0.8343
Epoch 100/100
176/176 [==============================] - 0s 1ms/step - loss: 0.3479 - accuracy: 0.8334
44/44 [==============================] - 0s 1ms/step - loss: 0.4865 - accuracy: 0.7861
[0.48652538657188416, 0.7860696315765381]
Classification Report: 
               precision    recall  f1-score   support

         0.0       0.84      0.88      0.86      1033
         1.0       0.61      0.54      0.57       374

    accuracy                           0.79      1407
   macro avg       0.73      0.71      0.71      1407
weighted avg       0.78      0.79      0.78      1407

Mitigating Skewdness of Data

Method 1: Undersampling

reference: https://www.kaggle.com/rafjaa/resampling-strategies-for-imbalanced-datasets

In [51]:

# Class count
count_class_0, count_class_1 = df1.Churn.value_counts()

# Divide by class
df_class_0 = df2[df2['Churn'] == 0]
df_class_1 = df2[df2['Churn'] == 1]

In [52]:

# Undersample 0-class and concat the DataFrames of both class
df_class_0_under = df_class_0.sample(count_class_1)
df_test_under = pd.concat([df_class_0_under, df_class_1], axis=0)

print('Random under-sampling:')
print(df_test_under.Churn.value_counts())

Out[52]:

Random under-sampling:
1    1869
0    1869
Name: Churn, dtype: int64

In [53]:

X = df_test_under.drop('Churn',axis='columns')
y = df_test_under['Churn']

from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=15, stratify=y)

In [54]:

# Number of classes in training Data
y_train.value_counts()

Out[54]:

1    1495
0    1495
Name: Churn, dtype: int64

Printing Classification in the last, Scroll down till the last epoch to watch the classification report

In [55]:

y_preds = ANN(X_train, y_train, X_test, y_test, 'binary_crossentropy', -1)

Out[55]:

Epoch 1/100
94/94 [==============================] - 0s 1ms/step - loss: 0.5709 - accuracy: 0.7375
Epoch 2/100
94/94 [==============================] - 0s 1ms/step - loss: 0.5009 - accuracy: 0.7656
Epoch 3/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4873 - accuracy: 0.7692
Epoch 4/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4828 - accuracy: 0.7669
Epoch 5/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4808 - accuracy: 0.7652
Epoch 6/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4781 - accuracy: 0.7706
Epoch 7/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4763 - accuracy: 0.7716
Epoch 8/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4739 - accuracy: 0.7709
Epoch 9/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4721 - accuracy: 0.7759
Epoch 10/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4717 - accuracy: 0.7736
Epoch 11/100
94/94 [==============================] - 0s 2ms/step - loss: 0.4690 - accuracy: 0.7756
Epoch 12/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4676 - accuracy: 0.7742
Epoch 13/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4682 - accuracy: 0.7709
Epoch 14/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4664 - accuracy: 0.7786
Epoch 15/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4644 - accuracy: 0.7793
Epoch 16/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4626 - accuracy: 0.7809
Epoch 17/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4620 - accuracy: 0.7756
Epoch 18/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4604 - accuracy: 0.7773
Epoch 19/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4585 - accuracy: 0.7799
Epoch 20/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4578 - accuracy: 0.7789
Epoch 21/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4566 - accuracy: 0.7836
Epoch 22/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4551 - accuracy: 0.7866
Epoch 23/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4529 - accuracy: 0.7796
Epoch 24/100
94/94 [==============================] - 0s 2ms/step - loss: 0.4525 - accuracy: 0.7826
Epoch 25/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4525 - accuracy: 0.7870
Epoch 26/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4501 - accuracy: 0.7826
Epoch 27/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4480 - accuracy: 0.7843
Epoch 28/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4505 - accuracy: 0.7829
Epoch 29/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4462 - accuracy: 0.7839
Epoch 30/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4448 - accuracy: 0.7893
Epoch 31/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4423 - accuracy: 0.7883
Epoch 32/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4422 - accuracy: 0.7870
Epoch 33/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4402 - accuracy: 0.7890
Epoch 34/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4394 - accuracy: 0.7923
Epoch 35/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4395 - accuracy: 0.7930
Epoch 36/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4389 - accuracy: 0.7910
Epoch 37/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4347 - accuracy: 0.7957
Epoch 38/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4343 - accuracy: 0.7963
Epoch 39/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4328 - accuracy: 0.7936
Epoch 40/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4322 - accuracy: 0.7953
Epoch 41/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4302 - accuracy: 0.7930
Epoch 42/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4299 - accuracy: 0.7977
Epoch 43/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4292 - accuracy: 0.7930
Epoch 44/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4270 - accuracy: 0.7967
Epoch 45/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4266 - accuracy: 0.8003
Epoch 46/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4269 - accuracy: 0.7946
Epoch 47/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4241 - accuracy: 0.7963
Epoch 48/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4233 - accuracy: 0.8017
Epoch 49/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4221 - accuracy: 0.7987
Epoch 50/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4202 - accuracy: 0.7970
Epoch 51/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4200 - accuracy: 0.7993
Epoch 52/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4187 - accuracy: 0.7997
Epoch 53/100
94/94 [==============================] - 0s 2ms/step - loss: 0.4177 - accuracy: 0.8037
Epoch 54/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4168 - accuracy: 0.8043
Epoch 55/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4161 - accuracy: 0.8013
Epoch 56/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4175 - accuracy: 0.8057
Epoch 57/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4140 - accuracy: 0.8033
Epoch 58/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4125 - accuracy: 0.8057
Epoch 59/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4120 - accuracy: 0.8054
Epoch 60/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4130 - accuracy: 0.8070
Epoch 61/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4098 - accuracy: 0.8087
Epoch 62/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4107 - accuracy: 0.8077
Epoch 63/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4083 - accuracy: 0.8017
Epoch 64/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4074 - accuracy: 0.8084
Epoch 65/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4079 - accuracy: 0.8043
Epoch 66/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4064 - accuracy: 0.8060
Epoch 67/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4041 - accuracy: 0.8094
Epoch 68/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4060 - accuracy: 0.8120
Epoch 69/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4027 - accuracy: 0.8161
Epoch 70/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4020 - accuracy: 0.8140
Epoch 71/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4013 - accuracy: 0.8104
Epoch 72/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4015 - accuracy: 0.8117
Epoch 73/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4016 - accuracy: 0.8161
Epoch 74/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3985 - accuracy: 0.8147
Epoch 75/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3975 - accuracy: 0.8130
Epoch 76/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3976 - accuracy: 0.8107
Epoch 77/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3988 - accuracy: 0.8154
Epoch 78/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3983 - accuracy: 0.8130
Epoch 79/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3957 - accuracy: 0.8154
Epoch 80/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3926 - accuracy: 0.8151
Epoch 81/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3934 - accuracy: 0.8171
Epoch 82/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3923 - accuracy: 0.8167
Epoch 83/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3910 - accuracy: 0.8161
Epoch 84/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3908 - accuracy: 0.8201
Epoch 85/100
94/94 [==============================] - 0s 2ms/step - loss: 0.3898 - accuracy: 0.8191
Epoch 86/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3885 - accuracy: 0.8184
Epoch 87/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3921 - accuracy: 0.8174
Epoch 88/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3883 - accuracy: 0.8204
Epoch 89/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3863 - accuracy: 0.8234
Epoch 90/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3856 - accuracy: 0.8231
Epoch 91/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3846 - accuracy: 0.8204
Epoch 92/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3847 - accuracy: 0.8211
Epoch 93/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3831 - accuracy: 0.8214
Epoch 94/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3829 - accuracy: 0.8247
Epoch 95/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3820 - accuracy: 0.8221
Epoch 96/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3827 - accuracy: 0.8237
Epoch 97/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3819 - accuracy: 0.8227
Epoch 98/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3805 - accuracy: 0.8271
Epoch 99/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3802 - accuracy: 0.8261
Epoch 100/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3781 - accuracy: 0.8264
24/24 [==============================] - 0s 1ms/step - loss: 0.5917 - accuracy: 0.7246
[0.5916573405265808, 0.7245989441871643]
Classification Report: 
               precision    recall  f1-score   support

           0       0.74      0.68      0.71       374
           1       0.71      0.76      0.74       374

    accuracy                           0.72       748
   macro avg       0.73      0.72      0.72       748
weighted avg       0.73      0.72      0.72       748

Check classification report above. f1-score for minority class 1 improved from 0.57 to 0.76. Score for class 0 reduced to 0.75 from 0.85 but that's ok. We have more generalized classifier which classifies both classes with similar prediction score

Method2: Oversampling

In [56]:

# Oversample 1-class and concat the DataFrames of both classes
df_class_1_over = df_class_1.sample(count_class_0, replace=True)
df_test_over = pd.concat([df_class_0, df_class_1_over], axis=0)

print('Random over-sampling:')
print(df_test_over.Churn.value_counts())

Out[56]:

Random over-sampling:
1    5163
0    5163
Name: Churn, dtype: int64

In [57]:

X = df_test_over.drop('Churn',axis='columns')
y = df_test_over['Churn']

from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=15, stratify=y)

In [58]:

# Number of classes in training Data
y_train.value_counts()

Out[58]:

1    4130
0    4130
Name: Churn, dtype: int64

In [59]:

loss = keras.losses.BinaryCrossentropy()
weights = -1
y_preds = ANN(X_train, y_train, X_test, y_test, 'binary_crossentropy', -1)

Out[59]:

Epoch 1/100
259/259 [==============================] - 0s 1ms/step - loss: 0.5655 - accuracy: 0.7068
Epoch 2/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4985 - accuracy: 0.7529
Epoch 3/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4913 - accuracy: 0.7607
Epoch 4/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4854 - accuracy: 0.7650
Epoch 5/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4814 - accuracy: 0.7655
Epoch 6/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4787 - accuracy: 0.7688
Epoch 7/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4769 - accuracy: 0.7703
Epoch 8/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4742 - accuracy: 0.7714
Epoch 9/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4717 - accuracy: 0.7703
Epoch 10/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4693 - accuracy: 0.7748
Epoch 11/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4676 - accuracy: 0.7745
Epoch 12/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4665 - accuracy: 0.7769
Epoch 13/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4631 - accuracy: 0.7805
Epoch 14/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4612 - accuracy: 0.7776
Epoch 15/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4597 - accuracy: 0.7824
Epoch 16/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4578 - accuracy: 0.7795
Epoch 17/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4556 - accuracy: 0.7809
Epoch 18/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4544 - accuracy: 0.7806
Epoch 19/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4524 - accuracy: 0.7843
Epoch 20/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4505 - accuracy: 0.7827
Epoch 21/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4484 - accuracy: 0.7862
Epoch 22/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4454 - accuracy: 0.7858
Epoch 23/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4441 - accuracy: 0.7900
Epoch 24/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4427 - accuracy: 0.7916
Epoch 25/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4402 - accuracy: 0.7904
Epoch 26/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4385 - accuracy: 0.7941
Epoch 27/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4377 - accuracy: 0.7944
Epoch 28/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4348 - accuracy: 0.7948
Epoch 29/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4335 - accuracy: 0.7962
Epoch 30/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4313 - accuracy: 0.7977
Epoch 31/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4301 - accuracy: 0.7988
Epoch 32/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4290 - accuracy: 0.7983
Epoch 33/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4271 - accuracy: 0.8023
Epoch 34/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4253 - accuracy: 0.8023
Epoch 35/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4245 - accuracy: 0.8039
Epoch 36/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4225 - accuracy: 0.8012
Epoch 37/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4215 - accuracy: 0.8048
Epoch 38/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4197 - accuracy: 0.8025
Epoch 39/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4183 - accuracy: 0.8065
Epoch 40/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4174 - accuracy: 0.8035
Epoch 41/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4157 - accuracy: 0.8061
Epoch 42/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4144 - accuracy: 0.8051
Epoch 43/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4131 - accuracy: 0.8068
Epoch 44/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4109 - accuracy: 0.8087
Epoch 45/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4117 - accuracy: 0.8084
Epoch 46/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4090 - accuracy: 0.8114
Epoch 47/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4080 - accuracy: 0.8126
Epoch 48/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4059 - accuracy: 0.8111
Epoch 49/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4067 - accuracy: 0.8087
Epoch 50/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4037 - accuracy: 0.8120
Epoch 51/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4044 - accuracy: 0.8111
Epoch 52/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4008 - accuracy: 0.8125
Epoch 53/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3982 - accuracy: 0.8116
Epoch 54/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3986 - accuracy: 0.8153
Epoch 55/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3970 - accuracy: 0.8142
Epoch 56/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3944 - accuracy: 0.8182
Epoch 57/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3945 - accuracy: 0.8138
Epoch 58/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3908 - accuracy: 0.8203
Epoch 59/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3898 - accuracy: 0.8177
Epoch 60/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3908 - accuracy: 0.8195
Epoch 61/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3892 - accuracy: 0.8189
Epoch 62/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3875 - accuracy: 0.8189
Epoch 63/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3866 - accuracy: 0.8202
Epoch 64/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3848 - accuracy: 0.8208
Epoch 65/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3846 - accuracy: 0.8211
Epoch 66/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3830 - accuracy: 0.8223
Epoch 67/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3814 - accuracy: 0.8223
Epoch 68/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3799 - accuracy: 0.8247
Epoch 69/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3787 - accuracy: 0.8243
Epoch 70/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3787 - accuracy: 0.8262
Epoch 71/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3781 - accuracy: 0.8265
Epoch 72/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3746 - accuracy: 0.8293
Epoch 73/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3753 - accuracy: 0.8248
Epoch 74/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3749 - accuracy: 0.8274
Epoch 75/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3737 - accuracy: 0.8268
Epoch 76/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3720 - accuracy: 0.8291
Epoch 77/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3722 - accuracy: 0.8308
Epoch 78/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3713 - accuracy: 0.8285
Epoch 79/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3713 - accuracy: 0.8289
Epoch 80/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3688 - accuracy: 0.8316
Epoch 81/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3677 - accuracy: 0.8338
Epoch 82/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3674 - accuracy: 0.8322
Epoch 83/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3671 - accuracy: 0.8316
Epoch 84/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3669 - accuracy: 0.8300
Epoch 85/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3656 - accuracy: 0.8322
Epoch 86/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3653 - accuracy: 0.8352
Epoch 87/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3641 - accuracy: 0.8355
Epoch 88/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3639 - accuracy: 0.8341
Epoch 89/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3616 - accuracy: 0.8360
Epoch 90/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3614 - accuracy: 0.8379
Epoch 91/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3613 - accuracy: 0.8345
Epoch 92/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3595 - accuracy: 0.8362
Epoch 93/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3586 - accuracy: 0.8364
Epoch 94/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3584 - accuracy: 0.8364
Epoch 95/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3586 - accuracy: 0.8392
Epoch 96/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3582 - accuracy: 0.8378
Epoch 97/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3555 - accuracy: 0.8370
Epoch 98/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3562 - accuracy: 0.8402
Epoch 99/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3543 - accuracy: 0.8397
Epoch 100/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3543 - accuracy: 0.8400
65/65 [==============================] - 0s 1ms/step - loss: 0.4601 - accuracy: 0.7851
[0.4600695073604584, 0.7850919365882874]
Classification Report: 
               precision    recall  f1-score   support

           0       0.84      0.70      0.77      1033
           1       0.74      0.87      0.80      1033

    accuracy                           0.79      2066
   macro avg       0.79      0.79      0.78      2066
weighted avg       0.79      0.79      0.78      2066

Method3: SMOTE

To install imbalanced-learn library use pip install imbalanced-learn command

In [60]:

X = df2.drop('Churn',axis='columns')
y = df2['Churn']

In [61]:

from imblearn.over_sampling import SMOTE

smote = SMOTE(sampling_strategy='minority')
X_sm, y_sm = smote.fit_sample(X, y)

y_sm.value_counts()

Out[61]:

1    5163
0    5163
Name: Churn, dtype: int64

In [62]:

from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X_sm, y_sm, test_size=0.2, random_state=15, stratify=y_sm)

In [63]:

# Number of classes in training Data
y_train.value_counts()

Out[63]:

1    4130
0    4130
Name: Churn, dtype: int64

In [64]:

y_preds = ANN(X_train, y_train, X_test, y_test, 'binary_crossentropy', -1)

Out[64]:

Epoch 1/100
259/259 [==============================] - 0s 1ms/step - loss: 0.5222 - accuracy: 0.7470
Epoch 2/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4716 - accuracy: 0.7712
Epoch 3/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4627 - accuracy: 0.7798
Epoch 4/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4552 - accuracy: 0.7824
Epoch 5/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4518 - accuracy: 0.7852
Epoch 6/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4472 - accuracy: 0.7860
Epoch 7/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4443 - accuracy: 0.7904
Epoch 8/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4396 - accuracy: 0.7953
Epoch 9/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4367 - accuracy: 0.7944
Epoch 10/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4350 - accuracy: 0.7961
Epoch 11/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4290 - accuracy: 0.7985
Epoch 12/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4271 - accuracy: 0.8008
Epoch 13/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4228 - accuracy: 0.8059
Epoch 14/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4198 - accuracy: 0.8058
Epoch 15/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4192 - accuracy: 0.8061
Epoch 16/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4133 - accuracy: 0.8126
Epoch 17/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4124 - accuracy: 0.8120
Epoch 18/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4081 - accuracy: 0.8151
Epoch 19/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4056 - accuracy: 0.8165
Epoch 20/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4029 - accuracy: 0.8178
Epoch 21/100
259/259 [==============================] - 0s 1ms/step - loss: 0.4013 - accuracy: 0.8194
Epoch 22/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3991 - accuracy: 0.8203
Epoch 23/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3975 - accuracy: 0.8224
Epoch 24/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3943 - accuracy: 0.8185
Epoch 25/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3933 - accuracy: 0.8229
Epoch 26/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3937 - accuracy: 0.8219
Epoch 27/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3907 - accuracy: 0.8236
Epoch 28/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3883 - accuracy: 0.8224
Epoch 29/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3866 - accuracy: 0.8259
Epoch 30/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3846 - accuracy: 0.8301
Epoch 31/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3825 - accuracy: 0.8285
Epoch 32/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3822 - accuracy: 0.8308
Epoch 33/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3812 - accuracy: 0.8301
Epoch 34/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3801 - accuracy: 0.8306
Epoch 35/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3761 - accuracy: 0.8341
Epoch 36/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3765 - accuracy: 0.8333
Epoch 37/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3766 - accuracy: 0.8326
Epoch 38/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3750 - accuracy: 0.8327
Epoch 39/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3739 - accuracy: 0.8358
Epoch 40/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3717 - accuracy: 0.8368
Epoch 41/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3716 - accuracy: 0.8368
Epoch 42/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3711 - accuracy: 0.8361
Epoch 43/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3689 - accuracy: 0.8387
Epoch 44/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3673 - accuracy: 0.8377
Epoch 45/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3695 - accuracy: 0.8370
Epoch 46/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3676 - accuracy: 0.8374
Epoch 47/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3660 - accuracy: 0.8391
Epoch 48/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3646 - accuracy: 0.8427
Epoch 49/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3627 - accuracy: 0.8392
Epoch 50/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3634 - accuracy: 0.8374
Epoch 51/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3616 - accuracy: 0.8402
Epoch 52/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3614 - accuracy: 0.8400
Epoch 53/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3608 - accuracy: 0.8412
Epoch 54/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3598 - accuracy: 0.8404
Epoch 55/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3576 - accuracy: 0.8431
Epoch 56/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3621 - accuracy: 0.8400
Epoch 57/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3570 - accuracy: 0.8421
Epoch 58/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3606 - accuracy: 0.8401
Epoch 59/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3551 - accuracy: 0.8452
Epoch 60/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3561 - accuracy: 0.8413
Epoch 61/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3559 - accuracy: 0.8444
Epoch 62/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3540 - accuracy: 0.8419
Epoch 63/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3543 - accuracy: 0.8454
Epoch 64/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3548 - accuracy: 0.8410
Epoch 65/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3542 - accuracy: 0.8415
Epoch 66/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3553 - accuracy: 0.8439
Epoch 67/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3500 - accuracy: 0.8472
Epoch 68/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3506 - accuracy: 0.8444
Epoch 69/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3506 - accuracy: 0.8494
Epoch 70/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3528 - accuracy: 0.8465
Epoch 71/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3497 - accuracy: 0.8470
Epoch 72/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3502 - accuracy: 0.8449
Epoch 73/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3490 - accuracy: 0.8492
Epoch 74/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3478 - accuracy: 0.8466
Epoch 75/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3464 - accuracy: 0.8489
Epoch 76/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3480 - accuracy: 0.8467
Epoch 77/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3465 - accuracy: 0.8492
Epoch 78/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3469 - accuracy: 0.8502
Epoch 79/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3470 - accuracy: 0.8492
Epoch 80/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3460 - accuracy: 0.8515
Epoch 81/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3432 - accuracy: 0.8523
Epoch 82/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3458 - accuracy: 0.8471
Epoch 83/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3453 - accuracy: 0.8487
Epoch 84/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3439 - accuracy: 0.8519
Epoch 85/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3423 - accuracy: 0.8472
Epoch 86/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3437 - accuracy: 0.8483
Epoch 87/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3411 - accuracy: 0.8500
Epoch 88/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3436 - accuracy: 0.8512
Epoch 89/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3416 - accuracy: 0.8515
Epoch 90/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3405 - accuracy: 0.8539
Epoch 91/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3417 - accuracy: 0.8531
Epoch 92/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3403 - accuracy: 0.8518
Epoch 93/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3422 - accuracy: 0.8513
Epoch 94/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3377 - accuracy: 0.8542
Epoch 95/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3394 - accuracy: 0.8522
Epoch 96/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3396 - accuracy: 0.8561
Epoch 97/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3357 - accuracy: 0.8521
Epoch 98/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3370 - accuracy: 0.8511
Epoch 99/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3393 - accuracy: 0.8524
Epoch 100/100
259/259 [==============================] - 0s 1ms/step - loss: 0.3389 - accuracy: 0.8544
65/65 [==============================] - 0s 1ms/step - loss: 0.4252 - accuracy: 0.8088
[0.425162136554718, 0.8088092803955078]
Classification Report: 
               precision    recall  f1-score   support

           0       0.84      0.76      0.80      1033
           1       0.78      0.85      0.82      1033

    accuracy                           0.81      2066
   macro avg       0.81      0.81      0.81      2066
weighted avg       0.81      0.81      0.81      2066

SMOT Oversampling increases f1 score of minority class 1 from 0.57 to 0.81 (huge improvement) Also over all accuracy improves from 0.78 to 0.80

Method4: Use of Ensemble with undersampling

In [65]:

df2.Churn.value_counts()

Out[65]:

0    5163
1    1869
Name: Churn, dtype: int64

In [66]:

# Regain Original features and labels
X = df2.drop('Churn',axis='columns')
y = df2['Churn']

In [67]:

from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=15, stratify=y)

In [68]:

y_train.value_counts()

Out[68]:

0    4130
1    1495
Name: Churn, dtype: int64

model1 --> class1(1495) + class0(0, 1495)

model2 --> class1(1495) + class0(1496, 2990)

model3 --> class1(1495) + class0(2990, 4130)

In [73]:

df3 = X_train.copy()
df3['Churn'] = y_train

In [74]:

df3.head()

Out[74]:

In [75]:

df3_class0 = df3[df3.Churn==0]
df3_class1 = df3[df3.Churn==1]

In [76]:

def get_train_batch(df_majority, df_minority, start, end):
    df_train = pd.concat([df_majority[start:end], df_minority], axis=0)

    X_train = df_train.drop('Churn', axis='columns')
    y_train = df_train.Churn
    return X_train, y_train

In [77]:

X_train, y_train = get_train_batch(df3_class0, df3_class1, 0, 1495)

y_pred1 = ANN(X_train, y_train, X_test, y_test, 'binary_crossentropy', -1)

Out[77]:

Epoch 1/100
94/94 [==============================] - 0s 1ms/step - loss: 0.5840 - accuracy: 0.7164
Epoch 2/100
94/94 [==============================] - 0s 1ms/step - loss: 0.5094 - accuracy: 0.7542
Epoch 3/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4964 - accuracy: 0.7656
Epoch 4/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4893 - accuracy: 0.7649
Epoch 5/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4844 - accuracy: 0.7672
Epoch 6/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4817 - accuracy: 0.7689
Epoch 7/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4767 - accuracy: 0.7719
Epoch 8/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4758 - accuracy: 0.7722
Epoch 9/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4733 - accuracy: 0.7753
Epoch 10/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4702 - accuracy: 0.7753
Epoch 11/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4703 - accuracy: 0.7736
Epoch 12/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4659 - accuracy: 0.7769
Epoch 13/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4648 - accuracy: 0.7796
Epoch 14/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4631 - accuracy: 0.7813
Epoch 15/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4614 - accuracy: 0.7826
Epoch 16/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4597 - accuracy: 0.7819
Epoch 17/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4579 - accuracy: 0.7806
Epoch 18/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4578 - accuracy: 0.7829
Epoch 19/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4559 - accuracy: 0.7863
Epoch 20/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4534 - accuracy: 0.7866
Epoch 21/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4526 - accuracy: 0.7846
Epoch 22/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4506 - accuracy: 0.7896
Epoch 23/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4491 - accuracy: 0.7916
Epoch 24/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4489 - accuracy: 0.7896
Epoch 25/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4469 - accuracy: 0.7953
Epoch 26/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4469 - accuracy: 0.7923
Epoch 27/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4449 - accuracy: 0.7943
Epoch 28/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4434 - accuracy: 0.7940
Epoch 29/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4443 - accuracy: 0.7906
Epoch 30/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4419 - accuracy: 0.7963
Epoch 31/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4397 - accuracy: 0.7983
Epoch 32/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4391 - accuracy: 0.7983
Epoch 33/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4401 - accuracy: 0.7973
Epoch 34/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4377 - accuracy: 0.7997
Epoch 35/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4356 - accuracy: 0.8000
Epoch 36/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4350 - accuracy: 0.7990
Epoch 37/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4334 - accuracy: 0.8000
Epoch 38/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4343 - accuracy: 0.7973
Epoch 39/100
94/94 [==============================] - 0s 2ms/step - loss: 0.4311 - accuracy: 0.7997
Epoch 40/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4299 - accuracy: 0.8007
Epoch 41/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4300 - accuracy: 0.8010
Epoch 42/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4279 - accuracy: 0.8050
Epoch 43/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4286 - accuracy: 0.8000
Epoch 44/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4264 - accuracy: 0.8027
Epoch 45/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4262 - accuracy: 0.8007
Epoch 46/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4243 - accuracy: 0.8030
Epoch 47/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4227 - accuracy: 0.8067
Epoch 48/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4208 - accuracy: 0.8084
Epoch 49/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4233 - accuracy: 0.8037
Epoch 50/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4203 - accuracy: 0.8037
Epoch 51/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4182 - accuracy: 0.8080
Epoch 52/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4191 - accuracy: 0.8067
Epoch 53/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4155 - accuracy: 0.8050
Epoch 54/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4164 - accuracy: 0.8104
Epoch 55/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4142 - accuracy: 0.8130
Epoch 56/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4146 - accuracy: 0.8087
Epoch 57/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4137 - accuracy: 0.8087
Epoch 58/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4106 - accuracy: 0.8130
Epoch 59/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4108 - accuracy: 0.8114
Epoch 60/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4090 - accuracy: 0.8137
Epoch 61/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4086 - accuracy: 0.8157
Epoch 62/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4081 - accuracy: 0.8127
Epoch 63/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4070 - accuracy: 0.8134
Epoch 64/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4054 - accuracy: 0.8140
Epoch 65/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4047 - accuracy: 0.8137
Epoch 66/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4043 - accuracy: 0.8161
Epoch 67/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4050 - accuracy: 0.8107
Epoch 68/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4023 - accuracy: 0.8161
Epoch 69/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4015 - accuracy: 0.8161
Epoch 70/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4007 - accuracy: 0.8174
Epoch 71/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4011 - accuracy: 0.8161
Epoch 72/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3984 - accuracy: 0.8187
Epoch 73/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3982 - accuracy: 0.8191
Epoch 74/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3968 - accuracy: 0.8201
Epoch 75/100
94/94 [==============================] - 0s 2ms/step - loss: 0.3968 - accuracy: 0.8207
Epoch 76/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3980 - accuracy: 0.8154
Epoch 77/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3943 - accuracy: 0.8214
Epoch 78/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3947 - accuracy: 0.8191
Epoch 79/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3927 - accuracy: 0.8207
Epoch 80/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3918 - accuracy: 0.8224
Epoch 81/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3912 - accuracy: 0.8207
Epoch 82/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3897 - accuracy: 0.8211
Epoch 83/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3921 - accuracy: 0.8244
Epoch 84/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3879 - accuracy: 0.8247
Epoch 85/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3908 - accuracy: 0.8284
Epoch 86/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3880 - accuracy: 0.8221
Epoch 87/100
94/94 [==============================] - 0s 2ms/step - loss: 0.3894 - accuracy: 0.8258
Epoch 88/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3862 - accuracy: 0.8271
Epoch 89/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3881 - accuracy: 0.8284
Epoch 90/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3843 - accuracy: 0.8268
Epoch 91/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3847 - accuracy: 0.8268
Epoch 92/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3865 - accuracy: 0.8268
Epoch 93/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3832 - accuracy: 0.8271
Epoch 94/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3834 - accuracy: 0.8268
Epoch 95/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3814 - accuracy: 0.8284
Epoch 96/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3802 - accuracy: 0.8324
Epoch 97/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3800 - accuracy: 0.8268
Epoch 98/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3799 - accuracy: 0.8314
Epoch 99/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3783 - accuracy: 0.8324
Epoch 100/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3779 - accuracy: 0.8288
44/44 [==============================] - 0s 955us/step - loss: 0.6137 - accuracy: 0.7114
[0.6136508584022522, 0.711442768573761]
Classification Report: 
               precision    recall  f1-score   support

           0       0.89      0.70      0.78      1033
           1       0.47      0.76      0.58       374

    accuracy                           0.71      1407
   macro avg       0.68      0.73      0.68      1407
weighted avg       0.78      0.71      0.73      1407

In [78]:

X_train, y_train = get_train_batch(df3_class0, df3_class1, 1495, 2990)

y_pred2 = ANN(X_train, y_train, X_test, y_test, 'binary_crossentropy', -1)

Out[78]:

Epoch 1/100
94/94 [==============================] - 0s 1ms/step - loss: 0.6486 - accuracy: 0.6328
Epoch 2/100
94/94 [==============================] - 0s 1ms/step - loss: 0.5236 - accuracy: 0.7522
Epoch 3/100
94/94 [==============================] - 0s 1ms/step - loss: 0.5005 - accuracy: 0.7572
Epoch 4/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4935 - accuracy: 0.7592
Epoch 5/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4883 - accuracy: 0.7562
Epoch 6/100
94/94 [==============================] - 0s 2ms/step - loss: 0.4845 - accuracy: 0.7605
Epoch 7/100
94/94 [==============================] - 0s 2ms/step - loss: 0.4795 - accuracy: 0.7625
Epoch 8/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4778 - accuracy: 0.7642
Epoch 9/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4747 - accuracy: 0.7659
Epoch 10/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4719 - accuracy: 0.7692
Epoch 11/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4682 - accuracy: 0.7696
Epoch 12/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4659 - accuracy: 0.7689
Epoch 13/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4679 - accuracy: 0.7659
Epoch 14/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4625 - accuracy: 0.7712
Epoch 15/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4596 - accuracy: 0.7776
Epoch 16/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4590 - accuracy: 0.7722
Epoch 17/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4556 - accuracy: 0.7799
Epoch 18/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4539 - accuracy: 0.7783
Epoch 19/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4519 - accuracy: 0.7806
Epoch 20/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4499 - accuracy: 0.7819
Epoch 21/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4497 - accuracy: 0.7873
Epoch 22/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4473 - accuracy: 0.7803
Epoch 23/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4466 - accuracy: 0.7836
Epoch 24/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4439 - accuracy: 0.7860
Epoch 25/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4429 - accuracy: 0.7876
Epoch 26/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4435 - accuracy: 0.7843
Epoch 27/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4405 - accuracy: 0.7843
Epoch 28/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4381 - accuracy: 0.7906
Epoch 29/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4364 - accuracy: 0.7896
Epoch 30/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4377 - accuracy: 0.7870
Epoch 31/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4347 - accuracy: 0.7946
Epoch 32/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4325 - accuracy: 0.7963
Epoch 33/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4315 - accuracy: 0.7946
Epoch 34/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4293 - accuracy: 0.7946
Epoch 35/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4299 - accuracy: 0.7943
Epoch 36/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4280 - accuracy: 0.7963
Epoch 37/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4253 - accuracy: 0.8003
Epoch 38/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4271 - accuracy: 0.7953
Epoch 39/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4233 - accuracy: 0.8010
Epoch 40/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4234 - accuracy: 0.7993
Epoch 41/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4198 - accuracy: 0.7993
Epoch 42/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4186 - accuracy: 0.8007
Epoch 43/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4188 - accuracy: 0.8020
Epoch 44/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4167 - accuracy: 0.8050
Epoch 45/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4152 - accuracy: 0.8057
Epoch 46/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4139 - accuracy: 0.8033
Epoch 47/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4134 - accuracy: 0.8060
Epoch 48/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4112 - accuracy: 0.8090
Epoch 49/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4093 - accuracy: 0.8057
Epoch 50/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4099 - accuracy: 0.8067
Epoch 51/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4090 - accuracy: 0.8057
Epoch 52/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4065 - accuracy: 0.8054
Epoch 53/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4050 - accuracy: 0.8084
Epoch 54/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4031 - accuracy: 0.8064
Epoch 55/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4024 - accuracy: 0.8140
Epoch 56/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4005 - accuracy: 0.8151
Epoch 57/100
94/94 [==============================] - 0s 1ms/step - loss: 0.4020 - accuracy: 0.8074
Epoch 58/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3997 - accuracy: 0.8140
Epoch 59/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3982 - accuracy: 0.8140
Epoch 60/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3974 - accuracy: 0.8154
Epoch 61/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3969 - accuracy: 0.8114
Epoch 62/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3955 - accuracy: 0.8140
Epoch 63/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3961 - accuracy: 0.8134
Epoch 64/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3938 - accuracy: 0.8161
Epoch 65/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3912 - accuracy: 0.8177
Epoch 66/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3922 - accuracy: 0.8127
Epoch 67/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3899 - accuracy: 0.8174
Epoch 68/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3909 - accuracy: 0.8147
Epoch 69/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3891 - accuracy: 0.8194
Epoch 70/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3878 - accuracy: 0.8207
Epoch 71/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3870 - accuracy: 0.8214
Epoch 72/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3870 - accuracy: 0.8224
Epoch 73/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3852 - accuracy: 0.8214
Epoch 74/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3847 - accuracy: 0.8247
Epoch 75/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3846 - accuracy: 0.8191
Epoch 76/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3835 - accuracy: 0.8201
Epoch 77/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3825 - accuracy: 0.8264
Epoch 78/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3830 - accuracy: 0.8184
Epoch 79/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3813 - accuracy: 0.8244
Epoch 80/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3784 - accuracy: 0.8278
Epoch 81/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3794 - accuracy: 0.8227
Epoch 82/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3788 - accuracy: 0.8224
Epoch 83/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3779 - accuracy: 0.8227
Epoch 84/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3769 - accuracy: 0.8221
Epoch 85/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3758 - accuracy: 0.8281
Epoch 86/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3749 - accuracy: 0.8308
Epoch 87/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3730 - accuracy: 0.8258
Epoch 88/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3737 - accuracy: 0.8231
Epoch 89/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3789 - accuracy: 0.8227
Epoch 90/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3718 - accuracy: 0.8318
Epoch 91/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3700 - accuracy: 0.8284
Epoch 92/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3703 - accuracy: 0.8268
Epoch 93/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3694 - accuracy: 0.8308
Epoch 94/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3675 - accuracy: 0.8281
Epoch 95/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3682 - accuracy: 0.8301
Epoch 96/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3670 - accuracy: 0.8271
Epoch 97/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3647 - accuracy: 0.8308
Epoch 98/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3653 - accuracy: 0.8355
Epoch 99/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3659 - accuracy: 0.8355
Epoch 100/100
94/94 [==============================] - 0s 1ms/step - loss: 0.3648 - accuracy: 0.8294
44/44 [==============================] - 0s 978us/step - loss: 0.6379 - accuracy: 0.7079
[0.6378723382949829, 0.7078891396522522]
Classification Report: 
               precision    recall  f1-score   support

           0       0.88      0.70      0.78      1033
           1       0.47      0.74      0.57       374

    accuracy                           0.71      1407
   macro avg       0.67      0.72      0.68      1407
weighted avg       0.77      0.71      0.72      1407

In [79]:

X_train, y_train = get_train_batch(df3_class0, df3_class1, 2990, 4130)

y_pred3 = ANN(X_train, y_train, X_test, y_test, 'binary_crossentropy', -1)

Out[79]:

Epoch 1/100
83/83 [==============================] - 0s 1ms/step - loss: 0.6392 - accuracy: 0.6676
Epoch 2/100
83/83 [==============================] - 0s 1ms/step - loss: 0.5260 - accuracy: 0.7609
Epoch 3/100
83/83 [==============================] - 0s 1ms/step - loss: 0.4914 - accuracy: 0.7674
Epoch 4/100
83/83 [==============================] - 0s 1ms/step - loss: 0.4835 - accuracy: 0.7715
Epoch 5/100
83/83 [==============================] - 0s 1ms/step - loss: 0.4793 - accuracy: 0.7738
Epoch 6/100
83/83 [==============================] - 0s 1ms/step - loss: 0.4751 - accuracy: 0.7787
Epoch 7/100
83/83 [==============================] - 0s 1ms/step - loss: 0.4736 - accuracy: 0.7731
Epoch 8/100
83/83 [==============================] - 0s 1ms/step - loss: 0.4708 - accuracy: 0.7818
Epoch 9/100
83/83 [==============================] - 0s 1ms/step - loss: 0.4700 - accuracy: 0.7772
Epoch 10/100
83/83 [==============================] - 0s 1ms/step - loss: 0.4679 - accuracy: 0.7810
Epoch 11/100
83/83 [==============================] - 0s 1ms/step - loss: 0.4670 - accuracy: 0.7799
Epoch 12/100
83/83 [==============================] - 0s 1ms/step - loss: 0.4653 - accuracy: 0.7791
Epoch 13/100
83/83 [==============================] - 0s 2ms/step - loss: 0.4632 - accuracy: 0.7822
Epoch 14/100
83/83 [==============================] - 0s 2ms/step - loss: 0.4607 - accuracy: 0.7810
Epoch 15/100
83/83 [==============================] - 0s 1ms/step - loss: 0.4590 - accuracy: 0.7894
Epoch 16/100
83/83 [==============================] - 0s 1ms/step - loss: 0.4573 - accuracy: 0.7867
Epoch 17/100
83/83 [==============================] - 0s 1ms/step - loss: 0.4561 - accuracy: 0.7879
Epoch 18/100
83/83 [==============================] - 0s 1ms/step - loss: 0.4548 - accuracy: 0.7863
Epoch 19/100
83/83 [==============================] - 0s 1ms/step - loss: 0.4535 - accuracy: 0.7894
Epoch 20/100
83/83 [==============================] - 0s 2ms/step - loss: 0.4506 - accuracy: 0.7901
Epoch 21/100
83/83 [==============================] - 0s 1ms/step - loss: 0.4518 - accuracy: 0.7890
Epoch 22/100
83/83 [==============================] - 0s 1ms/step - loss: 0.4487 - accuracy: 0.7947
Epoch 23/100
83/83 [==============================] - 0s 1ms/step - loss: 0.4471 - accuracy: 0.7917
Epoch 24/100
83/83 [==============================] - 0s 1ms/step - loss: 0.4458 - accuracy: 0.7909
Epoch 25/100
83/83 [==============================] - 0s 1ms/step - loss: 0.4430 - accuracy: 0.7958
Epoch 26/100
83/83 [==============================] - 0s 1ms/step - loss: 0.4423 - accuracy: 0.7935
Epoch 27/100
83/83 [==============================] - 0s 1ms/step - loss: 0.4421 - accuracy: 0.7920
Epoch 28/100
83/83 [==============================] - 0s 1ms/step - loss: 0.4421 - accuracy: 0.7992
Epoch 29/100
83/83 [==============================] - 0s 1ms/step - loss: 0.4383 - accuracy: 0.7951
Epoch 30/100
83/83 [==============================] - 0s 1ms/step - loss: 0.4389 - accuracy: 0.7996
Epoch 31/100
83/83 [==============================] - 0s 1ms/step - loss: 0.4387 - accuracy: 0.7966
Epoch 32/100
83/83 [==============================] - 0s 1ms/step - loss: 0.4352 - accuracy: 0.7996
Epoch 33/100
83/83 [==============================] - 0s 1ms/step - loss: 0.4330 - accuracy: 0.8000
Epoch 34/100
83/83 [==============================] - 0s 1ms/step - loss: 0.4317 - accuracy: 0.8027
Epoch 35/100
83/83 [==============================] - 0s 1ms/step - loss: 0.4307 - accuracy: 0.8015
Epoch 36/100
83/83 [==============================] - 0s 1ms/step - loss: 0.4275 - accuracy: 0.8023
Epoch 37/100
83/83 [==============================] - 0s 1ms/step - loss: 0.4275 - accuracy: 0.8053
Epoch 38/100
83/83 [==============================] - 0s 1ms/step - loss: 0.4239 - accuracy: 0.8008
Epoch 39/100
83/83 [==============================] - 0s 1ms/step - loss: 0.4252 - accuracy: 0.8061
Epoch 40/100
83/83 [==============================] - 0s 1ms/step - loss: 0.4227 - accuracy: 0.8083
Epoch 41/100
83/83 [==============================] - 0s 1ms/step - loss: 0.4201 - accuracy: 0.8053
Epoch 42/100
83/83 [==============================] - 0s 1ms/step - loss: 0.4207 - accuracy: 0.8053
Epoch 43/100
83/83 [==============================] - 0s 1ms/step - loss: 0.4192 - accuracy: 0.8110
Epoch 44/100
83/83 [==============================] - 0s 1ms/step - loss: 0.4184 - accuracy: 0.8121
Epoch 45/100
83/83 [==============================] - 0s 1ms/step - loss: 0.4156 - accuracy: 0.8099
Epoch 46/100
83/83 [==============================] - 0s 1ms/step - loss: 0.4138 - accuracy: 0.8125
Epoch 47/100
83/83 [==============================] - 0s 1ms/step - loss: 0.4118 - accuracy: 0.8137
Epoch 48/100
83/83 [==============================] - 0s 1ms/step - loss: 0.4120 - accuracy: 0.8144
Epoch 49/100
83/83 [==============================] - 0s 2ms/step - loss: 0.4104 - accuracy: 0.8110
Epoch 50/100
83/83 [==============================] - 0s 2ms/step - loss: 0.4097 - accuracy: 0.8106
Epoch 51/100
83/83 [==============================] - 0s 1ms/step - loss: 0.4075 - accuracy: 0.8125
Epoch 52/100
83/83 [==============================] - 0s 1ms/step - loss: 0.4049 - accuracy: 0.8163
Epoch 53/100
83/83 [==============================] - 0s 1ms/step - loss: 0.4051 - accuracy: 0.8171
Epoch 54/100
83/83 [==============================] - 0s 1ms/step - loss: 0.4039 - accuracy: 0.8152
Epoch 55/100
83/83 [==============================] - 0s 1ms/step - loss: 0.4023 - accuracy: 0.8152
Epoch 56/100
83/83 [==============================] - 0s 1ms/step - loss: 0.4002 - accuracy: 0.8159
Epoch 57/100
83/83 [==============================] - 0s 1ms/step - loss: 0.4014 - accuracy: 0.8171
Epoch 58/100
83/83 [==============================] - 0s 1ms/step - loss: 0.3987 - accuracy: 0.8171
Epoch 59/100
83/83 [==============================] - 0s 1ms/step - loss: 0.3957 - accuracy: 0.8216
Epoch 60/100
83/83 [==============================] - 0s 1ms/step - loss: 0.3991 - accuracy: 0.8220
Epoch 61/100
83/83 [==============================] - 0s 1ms/step - loss: 0.3950 - accuracy: 0.8209
Epoch 62/100
83/83 [==============================] - 0s 1ms/step - loss: 0.3934 - accuracy: 0.8258
Epoch 63/100
83/83 [==============================] - 0s 1ms/step - loss: 0.3938 - accuracy: 0.8239
Epoch 64/100
83/83 [==============================] - 0s 1ms/step - loss: 0.3892 - accuracy: 0.8228
Epoch 65/100
83/83 [==============================] - 0s 1ms/step - loss: 0.3902 - accuracy: 0.8254
Epoch 66/100
83/83 [==============================] - 0s 1ms/step - loss: 0.3891 - accuracy: 0.8247
Epoch 67/100
83/83 [==============================] - 0s 1ms/step - loss: 0.3864 - accuracy: 0.8288
Epoch 68/100
83/83 [==============================] - 0s 1ms/step - loss: 0.3850 - accuracy: 0.8296
Epoch 69/100
83/83 [==============================] - 0s 1ms/step - loss: 0.3872 - accuracy: 0.8235
Epoch 70/100
83/83 [==============================] - 0s 1ms/step - loss: 0.3834 - accuracy: 0.8315
Epoch 71/100
83/83 [==============================] - 0s 1ms/step - loss: 0.3826 - accuracy: 0.8300
Epoch 72/100
83/83 [==============================] - 0s 1ms/step - loss: 0.3809 - accuracy: 0.8304
Epoch 73/100
83/83 [==============================] - 0s 1ms/step - loss: 0.3823 - accuracy: 0.8323
Epoch 74/100
83/83 [==============================] - 0s 1ms/step - loss: 0.3792 - accuracy: 0.8330
Epoch 75/100
83/83 [==============================] - 0s 1ms/step - loss: 0.3771 - accuracy: 0.8342
Epoch 76/100
83/83 [==============================] - 0s 1ms/step - loss: 0.3757 - accuracy: 0.8406
Epoch 77/100
83/83 [==============================] - 0s 1ms/step - loss: 0.3753 - accuracy: 0.8383
Epoch 78/100
83/83 [==============================] - 0s 1ms/step - loss: 0.3751 - accuracy: 0.8357
Epoch 79/100
83/83 [==============================] - 0s 1ms/step - loss: 0.3746 - accuracy: 0.8338
Epoch 80/100
83/83 [==============================] - 0s 1ms/step - loss: 0.3718 - accuracy: 0.8372
Epoch 81/100
83/83 [==============================] - 0s 1ms/step - loss: 0.3725 - accuracy: 0.8387
Epoch 82/100
83/83 [==============================] - 0s 1ms/step - loss: 0.3690 - accuracy: 0.8406
Epoch 83/100
83/83 [==============================] - 0s 1ms/step - loss: 0.3709 - accuracy: 0.8357
Epoch 84/100
83/83 [==============================] - 0s 1ms/step - loss: 0.3704 - accuracy: 0.8387
Epoch 85/100
83/83 [==============================] - 0s 1ms/step - loss: 0.3687 - accuracy: 0.8391
Epoch 86/100
83/83 [==============================] - 0s 1ms/step - loss: 0.3675 - accuracy: 0.8444
Epoch 87/100
83/83 [==============================] - 0s 1ms/step - loss: 0.3649 - accuracy: 0.8410
Epoch 88/100
83/83 [==============================] - 0s 1ms/step - loss: 0.3669 - accuracy: 0.8440
Epoch 89/100
83/83 [==============================] - 0s 1ms/step - loss: 0.3677 - accuracy: 0.8387
Epoch 90/100
83/83 [==============================] - 0s 1ms/step - loss: 0.3633 - accuracy: 0.8459
Epoch 91/100
83/83 [==============================] - 0s 1ms/step - loss: 0.3626 - accuracy: 0.8463
Epoch 92/100
83/83 [==============================] - 0s 1ms/step - loss: 0.3614 - accuracy: 0.8433
Epoch 93/100
83/83 [==============================] - 0s 1ms/step - loss: 0.3595 - accuracy: 0.8452
Epoch 94/100
83/83 [==============================] - 0s 1ms/step - loss: 0.3614 - accuracy: 0.8448
Epoch 95/100
83/83 [==============================] - 0s 1ms/step - loss: 0.3615 - accuracy: 0.8436
Epoch 96/100
83/83 [==============================] - 0s 1ms/step - loss: 0.3600 - accuracy: 0.8448
Epoch 97/100
83/83 [==============================] - 0s 1ms/step - loss: 0.3566 - accuracy: 0.8459
Epoch 98/100
83/83 [==============================] - 0s 1ms/step - loss: 0.3600 - accuracy: 0.8467
Epoch 99/100
83/83 [==============================] - 0s 1ms/step - loss: 0.3575 - accuracy: 0.8478
Epoch 100/100
83/83 [==============================] - 0s 1ms/step - loss: 0.3571 - accuracy: 0.8482
44/44 [==============================] - 0s 978us/step - loss: 0.6610 - accuracy: 0.6915
[0.6610018014907837, 0.6915422677993774]
Classification Report: 
               precision    recall  f1-score   support

           0       0.89      0.66      0.76      1033
           1       0.45      0.78      0.57       374

    accuracy                           0.69      1407
   macro avg       0.67      0.72      0.67      1407
weighted avg       0.78      0.69      0.71      1407

In [80]:

len(y_pred1)

Out[80]:

1407

In [81]:

y_pred_final = y_pred1.copy()
for i in range(len(y_pred1)):
    n_ones = y_pred1[i] + y_pred2[i] + y_pred3[i]
    if n_ones>1:
        y_pred_final[i] = 1
    else:
        y_pred_final[i] = 0

In [82]:

cl_rep = classification_report(y_test, y_pred_final)
print(cl_rep)

Out[82]:

              precision    recall  f1-score   support

           0       0.88      0.69      0.77      1033
           1       0.46      0.75      0.57       374

    accuracy                           0.70      1407
   macro avg       0.67      0.72      0.67      1407
weighted avg       0.77      0.70      0.72      1407

f1-score for minority class 1 improved to 0.62 from 0.57. The score for majority class 0 is suffering and reduced to 0.80 from 0.85 but at least there is some balance in terms of prediction accuracy across two classes

Handling imbalanced data in customer churn prediction

Mitigating Skewdness of Data

Method 1: Undersampling

Method2: Oversampling

Method3: SMOTE

Method4: Use of Ensemble with undersampling

Product

Resources

Company