EigenFace.ipynb

Kernel: Python 3

Eigen Faces

In this notebook, we will learn about Eigenface — a very interesting application of Principal Component Analysis (PCA) for human faces.

What are EigenFaces ?

Eigenfaces are images that can be added to a mean (average) face to create new facial images. We can write this mathematically as,

$F = F_{m} + \sum \limits _{i=1} ^{n} \alpha_{i}F_{i}$

where,

• $F$ is a new face.

• $F_{m}$ is the mean or the average face.

• $F_{i}$ is an EigenFace.

• $\alpha_{i}$ are scalar multipliers we can choose to create new faces. They can be positive or negative.

Eigenfaces are calculated by estimating the principal components of the dataset of facial images. They are used for applications like Face Recognition and Facial Landmark Detection.

How to calculate EigenFaces?

To calculate EigenFaces, we need to go through the following steps.

• Obtain a facial image dataset.

• Align and resize images.

• Create a data matrix.

• Calculate Mean Vector [Optional].

• Calculate Principal Components.

• Reshape Eigenvectors to obtain EigenFaces.

Import Libraries

# Import necessary packages.
from __future__ import print_function
import os
import sys
import cv2
import numpy as np

Read Images

The dataset must be processed so that the centre of the eyes are same and each image has same size. In our case we are using pre-processed images from calebA, which do not require alignment and resizing. Here, we read all images in the specified directory using the defined function readImages. The directory contains images that are aligned. The center of the left and the right eyes in all images are the same. We add these images to a list ( or vector ). We also flip the images vertically and add them to the list. Because the mirror image of a valid facial image, we just doubled the size of our dataset and made it symmetric at that same time.

# Read images from the directory.
def readImages(path):
    print("Reading images from " + path, end = "...")
    # Create array of array of images.
    images = []
    # List all files in the directory and read points from text files one by one.
    for filePath in sorted(os.listdir(path)):
        fileExt = os.path.splitext(filePath)[1]
        if fileExt in [".jpg", ".jpeg"]:

            # Add to array of images.
            imagePath = os.path.join(path, filePath)
            im = cv2.imread(imagePath)

            if im is None :
                print("image:{} not read properly".format(imagePath))
            else :
                # Convert image to floating point.
                im = np.float32(im)/255.0
                # Add image to list.
                images.append(im)
                # Flip image.
                imFlip = cv2.flip(im, 1);
                # Append flipped image.
                images.append(imFlip)
    numImages = int(len(images) / 2)
    # Exit if no image found.
    if numImages == 0 :
        print("No images found")
        sys.exit(0)

    print(str(numImages) + " files read.")
    return images

Create Data Matrix

Create a data matrix containing all images as a row vector. Next, we use the function createDataMatrix to assemble the images into a data matrix. Each row of the data matrix is one image. Let’s look into the createDataMatrix function. If all the images in the dataset are of size 100 x 100 and there are 1000 images, we will have a data matrix of size 30k x 1000. So, according to our example, numImages = 1000, sz[0] = 100, sz[1] = 100 and sz[2] = 3. flatten returns a copy of the array collapsed into one dimension.

# Create data matrix from a list of images.
def createDataMatrix(images):
    print("Creating data matrix", end = " ... ")

    numImages = len(images)
    sz = images[0].shape
    # Data matrix.
    data = np.zeros((numImages, sz[0] * sz[1] * sz[2]), dtype = np.float32)
    for i in range(0, numImages):
        image = images[i].flatten()
        # Each row get replaced with one flattened image.
        data[i,:] = image

    print("DONE")
    return data

Generate New Face

The averageFace is calculated below in the main function. We add the output and the weighted eigen faces to generate different results. The weight parameter is acquired from the trackbar position. We use the logic, weight = sliderValues[i] - MAX_SLIDER_VALUE/2 as OpenCV does not allow slider values to be negative. Finally we resize the image to double of its original size.

OpenCV Documentation

getTrackbarPos()

resize()

def createNewFace(*args):
    # Start with the mean image.
    output = averageFace

    # Add the eigen faces with the weights.
    for i in range(0, NUM_EIGEN_FACES):
        # Get trackbar position.
        sliderValues[i] = cv2.getTrackbarPos("Weight" + str(i), "Trackbars");
        weight = sliderValues[i] - MAX_SLIDER_VALUE/2
        # Add the weighted eigen face to the mean face.
        output = np.add(output, eigenFaces[i] * weight)

    # Display Result at 2x size.
    output = cv2.resize(output, (0,0), fx = 2, fy = 2)
    cv2.imshow("Result", output)

Reset Sliders

This is the callback function for mouse hover on the Average named window. By doing so, we reset the sliders and at the same time reset the output to its preveous state.

OpenCV Documentation

setTrackbarPos()

def resetSliderValues(*args):
    for i in range(0, NUM_EIGEN_FACES):
        cv2.setTrackbarPos("Weight" + str(i), "Trackbars", int(MAX_SLIDER_VALUE/2));
    createNewFace()

Main Function

This is the main function. As mentioned earlier, the workflow starts with creation of data matrix, then Principal Component Analysis, followed by reshaping of eigen vectors to obtain eigen faces. OpenCV has built-in function for PCA calculation, PCACompute.

Function Syntax

mean, eigenvectors  =  cv.PCACompute( data, mean[, maxComponents] )

Parameters:

• data : The data matrix containing every data point as either a row or a column vector. If our data consists of 1000 images, and each image is a 30k long row vector, the data matrix will of size 30k x 1000.

• mean : The average of the data. If every data point in the data matrix is a 30k long row vector, the mean will also be a vector of the same size. This parameter is optional and is calculated internally if it is not supplied.

• maxComponents : The maximum number of principal components is usually the smaller of the two values 1) Dimensionality of the original data ( in our case it is 30k ) 2) The number of data points ( e.g. 1000 in the above example ). However, we can explicity fix the maximum number of components we want to calculate by setting this argument. For example, we may be interested in only the first 50 principal components. Calculating fewer principal components is cheaper than calculating the theoretical max.

Flowchart

if __name__ == '__main__':

    # Number of EigenFaces.
    NUM_EIGEN_FACES = 10

    # Maximum weight.
    MAX_SLIDER_VALUE = 255

    # Directory containing images.
    dirName = "images"

    # Read images.
    images = readImages(dirName)

    # Size of images.
    sz = images[0].shape

    # Create data matrix for PCA.
    data = createDataMatrix(images)

    # Compute the eigenvectors from the stack of images created.
    print("Calculating PCA ", end = "...")
    
    mean, eigenVectors = cv2.PCACompute(data, mean = None, maxComponents = NUM_EIGEN_FACES)
    
    print ("DONE")

    averageFace = mean.reshape(sz)

    # Create a container to hold eigen faces.
    eigenFaces  = []

    # Reshape eigen vectors to eigen faces.
    for eigenVector in eigenVectors:
        # REshape.
        eigenFace = eigenVector.reshape(sz)
        # Append eigen faces to the container.
        eigenFaces.append(eigenFace)

    # Create window for displaying result.
    cv2.namedWindow("Result", cv2.WINDOW_NORMAL)
    # Create window for displaying mean face.
    cv2.namedWindow("Average", cv2.WINDOW_NORMAL)

    # Upscale by a factor of two.
    output = cv2.resize(averageFace, (0,0), fx = 2, fy = 2)
    
    # Display.
    cv2.imshow("Result", output)
    cv2.imshow("Average", averageFace)

    # Create Window for trackbars.
    cv2.namedWindow("Trackbars", cv2.WINDOW_NORMAL)

    # Create a list to contain slider values.
    sliderValues = []

    # Create Trackbars.
    for i in range(0, NUM_EIGEN_FACES):
        sliderValues.append(int(MAX_SLIDER_VALUE/2))
        cv2.createTrackbar( "Weight" + str(i), "Trackbars", int(MAX_SLIDER_VALUE/2), MAX_SLIDER_VALUE, createNewFace)

    # You can reset the sliders by clicking on the mean image.
    cv2.setMouseCallback("Average", resetSliderValues);

    print('''Usage:
    Change the weights using the sliders.
    Mouse hover on the result window to reset sliders.
    Press q to terminate.''')

    key = cv2.waitKey(0)
    if key == ord('q'):
        cv2.destroyAllWindows()