Possible enhancements for the two basic steps of our procedure, Key point localization and Classification:

Key point localization:

The localization of key points has emerged as the most problematic step. Feature points are difficult to detect, since the associated local image information varies due to 

-   the proband (size, skin color, and other personal features like a beard or glasses)

-   head orientation

-   motion variations like bride grins and closed eyes

-   changing illumination

-   varying distance from the camera

Our experimental results point out clearly that the simple approach used so far needs to be enhanced to achieve higher robustness. Furthermore, the current performance is far from real-time, since the detection of a feature took in the order of 6 seconds.

In the following, several possibilities to enhance Key point detection are outlined:

Correlation in the frequency domain

Correlation between the input image and the key point pattern is so far computed in the image space. A much faster approach would be to transform both the image and the pattern into the Fourier Space, where the computation of correlation –which is basically a convolution – turns out to be a fast multiplication.

This approach has not lead us to any results so far, since we are still looking for ways to overcome influences of varying image intensity overemphasizing bright areas. Maybe a local normalization procedure could solve this problem.

Using side information about key point locations

Side knowledge on the location of keypoints – for example that the right eye is located right from the left eye, but above the tip of the nose – has been neglected so far. It could be exploited so remove localization errors and limit the search area, thus leading to a better performance.

Classification:

One improvement for classification might be to switch to a more elaborated Pattern Recognition concept, e.g. a Neural Network like a Multilayer Perceptron.

Although preliminary results with a simple Nearest Neighbor classification were satisfying, another classification concept might for example help to perform classification for less features or incomplete data, and thus compensate for weaknesses of Key Point localization.

We started working on a Matlab implementation, but coult not finish so far due to time pressure.