5_loss_or_cost_function.ipynb

Kernel: Python 3

import numpy as np

y_predicted = np.array([1,1,0,0,1])
y_true = np.array([0.30,0.7,1,0,0.5])

Implement Mean Absolute Error

def mae(y_predicted, y_true):
    total_error = 0
    for yp, yt in zip(y_predicted, y_true):
        total_error += abs(yp - yt)
    print("Total error is:",total_error)
    mae = total_error/len(y_predicted)
    print("Mean absolute error is:",mae)
    return mae

mae(y_predicted, y_true)

Total error is: 2.5
Mean absolute error is: 0.5

0.5

Implement same thing using numpy in much easier way

np.abs(y_predicted-y_true)

array([0.5, 0.5, 0. , 0. , 0.3])

np.mean(np.abs(y_predicted-y_true))

0.26

def mae_np(y_predicted, y_true):
    return np.mean(np.abs(y_predicted-y_true))

mae_np(y_predicted, y_true)

0.26

Implement Log Loss or Binary Cross Entropy

np.log([0])

<ipython-input-79-faee82fd9f21>:1: RuntimeWarning: divide by zero encountered in log
  np.log([0])

array([-inf])

epsilon = 1e-15

np.log([1e-15])

array([-34.53877639])

y_predicted

array([1, 1, 0, 0, 1])

y_predicted_new = [max(i,epsilon) for i in y_predicted]
y_predicted_new

[1, 1, 1e-15, 1e-15, 1]

1-epsilon

0.999999999999999

y_predicted_new = [min(i,1-epsilon) for i in y_predicted_new]
y_predicted_new

[0.999999999999999, 0.999999999999999, 1e-15, 1e-15, 0.999999999999999]

y_predicted_new = np.array(y_predicted_new)

np.log(y_predicted_new)

array([-9.99200722e-16, -9.99200722e-16, -3.45387764e+01, -3.45387764e+01,
       -9.99200722e-16])

-np.mean(y_true*np.log(y_predicted_new)+(1-y_true)*np.log(1-y_predicted_new))

17.2696280766844

def log_loss(y_true, y_predicted):
    y_predicted_new = [max(i,epsilon) for i in y_predicted]
    y_predicted_new = [min(i,1-epsilon) for i in y_predicted_new]
    y_predicted_new = np.array(y_predicted_new)
    return -np.mean(y_true*np.log(y_predicted_new)+(1-y_true)*np.log(1-y_predicted_new))

log_loss(y_true, y_predicted)

17.2696280766844

Exercise

Implement mean squared error (or MSE) in two ways,

1. Without using numpy (i.e. using plain python)

2. With the use of numpy

Solution