Defining Correspondences.
In this project, we were tasked with creating a face morphing application. The first step in this process was to define correspondences between two images, which was done by manually selecting points on the two images that corresponded to each other. This was done by using the tool made by a previous student that was linked in the project description. The picture of me and Clooney together with the data points are shown below.
I then calculated the mean of the points from the two images. Then I used the Delanuay function from scipy to calculate the triangulation of the mean points. The mean triangulation is shown below.
Computing the "Mid-way Face"
The next process was to compute the "mid-way face" between the two images. This is done by warping the two images onto the average image, then averaging the colors of the two morphed images on the average image. The images are warped to the average image triangle by triangle. Each triangle from the original images is warped to the corresponding triangle in the average image by applying an affine transformation matrix. The transformation matrix is found by soling the following set of linear equations:
\[ \begin{bmatrix} x_1 & y_1 & 1 & 0 & 0 & 0 \\ 0 & 0 & 0 & x_1 & y_1 & 1 \\ x_2 & y_2 & 1 & 0 & 0 & 0 \\ 0 & 0 & 0 & x_2 & y_2 & 1 \\ x_3 & y_3 & 1 & 0 & 0 & 0 \\ 0 & 0 & 0 & x_3 & y_3 & 1 \end{bmatrix} \begin{bmatrix} a \\ b \\ c \\ d \\ e \\ f \end{bmatrix} = \begin{bmatrix} x_1' \\ y_1' \\ x_2' \\ y_2' \\ x_3' \\ y_3' \end{bmatrix} \]
Where (x1, y1) corresponds to a point in the original image, and (x1', y1') corresponds to the same point in the average image.
To then find the pixel values on the mean image, we use the inverse of the transformation matrix. The inverse of the transformation matrix will take us from a pixel in the mean image to a pixel in the image correspodning to the transformation matrix. However, we may land between pixels(at most between 4 pixels). To solve this, we use bilinear interpolation to find the pixel value. That is essentialy a weighted average of the pixels, where the weights are the distance to the pixel. Finally, to compute the mid-way face, we average the pixel values of the two images. The mid-way face is shown below.
The morph sequence
The next step was to create a morph sequence between the two images. This is done by computing a series of intermediate images between the two original images. The intermediate images are created by computing both the weighted average shape and color. The weight is determined by the parameters wrap_frac and dissolve_frac. wrap_frac = 0.25 means that the image is 25% the shape and color of one image, and 75% the shape and color of another image. This essentialy mean that the mid-way points are 0.25*image1 + 0.75*image2. The same applies for dissolve_frac, just that it decides the pixel-mix instead of shape. The morph sequence is shown below. NB, you need to refresh the page to see the animation.
The "Mean face" of a population
The next part of the project was to compute the "mean face" of a population. This was done by computing the average of the shapes and colors of all the images in the population. I used the images from the database frontalimages_manuallyaligned_part1 frontalimages_manuallyaligned_part1 from this link. To find the mean image, I first found the mean shape by averaging the points of all the images. I then found the triangulation of the mean shape. Then, I looped over all the images, found the transformation matrix from the original images to the mean images, and then applied the inverse transformation matrix from the mean image to the original image to find the pixel values. I used the same interpolation method as before to find the pixel values. I summed all the pixel values and divided by the number of images to find the mean image. The mean face is shown below.
Here is a collection of some of the images from the population and the images warped to the average shape. As you can see, when the triangles form sharp angles, the warping leads to some visible lines along the line of the triangles. This is probably becuase the boundary pixels of adjacent triangles are not exactly the same due to rounding errors. If f.ex two triangles overlap, the overlapping pixels will be averaged twice, leading to a visible white line. If there is a gap, then there will be no averaging, leading to a visible black line.
Here is an image of my face warped into the average face of the population:
Here is an image of my the average face warped into my face:
Caricatures: Extrapolating from the mean
The final part of the project was to create a caricatur of my face by extrapolating from the mean face of the population. This was done by computing the difference between points on my face and points on the mean face, and then adding a multiple (alpha) of this difference to the points from my face, and use this as the shape of the caricature. The carricatures with alpha betweeen -1.5 and 1.5 is shown below.
Bells and whistles:
I added a Bells and whistles to the project. I found the average face of a chinese mean here I then morphed my face into the average chinese face. I morphed just the shape, just the apperance and a combination of both. For each example, I used a morphing value of 0.5. The morphed images are shown below.
