The objective of this project is to develop an advanced automatic vision system for detecting, locating and tracking human faces in highly complex and dynamic environments. Both analytic and holistic face detection techniques will be developed and combined to create a novel and efficient system that is invariant to rotation, scale, lighting conditions, and facial expressions. Our approach relies on skin colour information and other facial features to reduce the search space and accurately locate the position and orientation of the face in the image. The outcomes of this project will lead to innovative commercial applications of face detection and tracking including perceptual human-computer interaction, content-based image retrieval, video indexing, teleconferencing, and electronic surveillance. Such a system will be a key component in the emerging face biometric technologies for personal identification and human activity monitoring.

Automatic face detection and tracking is a crucial first step in any application that involves facial image analysis. For example, in a face recognition application, before a face can be compared to stored faces, it must be extracted automatically from a visual scene. Robust real-time detection of faces is needed in a video surveillance system where dynamic scenes are scanned for known faces. Furthermore, face tracking is required for service robots operating in a natural environment in identification and interaction with humans. In these applications, accurate and fast human face detection is the key to a successful operation.

The proposed system will be the enabling technology for many applications of facial image analysis. Recently, government agencies in Australia and around the world have taken strong steps towards introducing biometric identification documents and adopting facial biometrics technology for passport fraud prevention and airport/border security [10]. The new face detection and tracking system can enhance existing face biometric technology in three ways: (i) improving recognition accuracy through precise detection and localisation of the face, (ii) supporting a more user-friendly technology by alleviating the constraints on the user when taking live images, (iii) enabling video surveillance to be applied in unconstrained environments. Therefore, the proposed research will play a part in safeguarding and protecting Australia from terrorism and crime, which is one of the ARC designated research priorities. Besides these existing applications, several innovative applications of face detection and tracking are emerging such as human-computer interaction and video indexing. These new applications belong to smart information use, which is part of another ARC designated research priority.

The proposed system consists of four major components: skin detection, face candidate selection, face verification, and face tracking (Fig. 1). First, skin regions are identified using skin colour information. Second, potential face candidates are selected in each skin region based on analytic features of the human face. Third, face candidate is verified if it is a true face using holistic face and non-face pattern classifiers. Finally, face movements through the video sequence are monitored through motion tracking.

Figure 1: Components of the proposed face detection and tracking system.

Developing each component of the system in its own is a challenging problem. We expect that collaboration with other research network participant will benefit this project significantly; a number of collaboration possibilities are listed below.

Skin detection involves the classification of colour pixels into skin colour and nonskin colour (at pixel level), and the segmentation of skin-coloured pixels into homogenous skin regions (at region level). Therefore, pattern classification and image segmentation techniques are required. Furthermore, lighting correction techniques for colour images are needed to develop skin detection algorithm that can cope with strong lighting variations.

Face candidate selection involves the identification of potential eye regions, which are then used to construct face candidates. The eye regions must be identified regardless of it orientation facial expression. Hence, rotation-invariant eye detection techniques are required.

Face candidate verification focuses on the elimination of false face candidates using face and nonface classifiers. In the past, classifiers such as neural networks[2, 3, 7], Gaussian distributions [8], support vector machines [5], naïve Bayes model, classifier cascade with AdaBoost [4, 9] have been used. In this project, we aim to develop two new pattern classifiers; one is based on nonlinear convolutional neural networks, and the other is based on independent component analysis (ICA) for density estimation. Hence, research collaboration in machine learning, neural networks, and independent component analysis will be valuable to this part of the project.

Face tracking is accomplished by estimating the motion vectors in the facial region and using them to track the face pattern. Here we also plan to investigate particle filters [1] for face tracking. Research collaboration in real-time object tracking and video processing can facilitate the development of efficient solution to this problem.

We have developed a preliminary face detector, and its performance compared to state-of-the-art face detectors is promising [6]. On a test set of 200 images containing 231 faces, the face detector could locate 90% of the faces and made 10 false detections. A sample result of face detection on an input images are shown in Fig. 2. The face detector can detect in-plane rotated faces of arbitrary angles, and produce normalized faces for recognition purpose. In addition, the face detector can determine precisely spatial coordinates of the faces and eye points. In developing this face detector, we have constructed and compared several skin colour pixel classifiers and face/nonface classifiers. Figure 3a shows that the Bayesian and neural network classifiers have good performances in skin detection task. Figure 3b shows that combination of multiple classifiers, each using a different feature vector, can improve classification. The shunting inhibitory convolutional neural network was found to have better performance among the tested face/nonface classifiers.

(a) input image       (b) skin detection output        (c) face detection output
Figure 2: Example of face detection.

Figure 3: Classifier comparison: (a) skin colour pixel classifiers, (b) face/nonface classifiers including the naïve Bayes (different feature vectors), shunting inhibitory CoNN classifiers, and template matching.

The proposed face detection and tracking system is an enabling technology with a huge potential in surveillance and security, law enforcement, and information and communications technology (ICT), to name a few. This research will contribute to building a knowledge economy in Australia and help safeguard and protect Australia from terrorism and crime, ARC Designated Research Priority 4. Recent world events have underlined the importance of security and safety in the world. Australia is a major player in the war on terror, and hence must adapt modern and innovative solutions to safeguard its people and resources. The proposed system will play a key role in personal identification and human activity monitoring in highly complex and dynamic environments. Furthermore, this research will enhance the reputation of Australia as a leader in frontier technologies and smart information use, two priority goals for research in Designated Research Priority 3. The outcomes of the project are expected to contribute to the building of a knowledge economy in Australia and help safeguard its people and resources by enhancing safety and security.

[1] S. Arulampalam, S. Maskell, N. Gordon, and T. Clapp, "A tutorial on particle filters for on-line nonlinear/non-Gaussian Bayesian tracking," IEEE Transactions on Signal Processing, vol. 50, no. 2, pp. 174-188, Feb. 2002.

[2] R. Féraud, O. J. Bernier, J.-E. Viallet, and M. Collobert, "A fast and accurate face detector based on neural networks," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, no. 1, pp. 42-53, Jan. 2001.

[3] C. Garcia and M. Delakis, "A neural architecture for fast and robust face detection," Proc. IEEE International Conference on Pattern Recognition, pp. 44-47, Quebec, Canada, Aug. 2002.

[4] R. Lienhart, A. Kuranov, and V. Pisarevsky, "An extended set of Haar-like features for rapid object detection," Proc. IEEE International Conference on Image Processing, pp. 900-903, Rochester, NY, Sep. 2002.

[5] E. Osuna, R. Freund, and F. Girosi, "Training support vector machines: an application to face detection," Proc. IEEE Conference on Computer Vision and Pattern Recognition, pp. 130 -136, Puerto Rico, June 1997.

[6] S. L. Phung, A. Bouzerdoum, D. Chai, and W. Kuczborski, "A color-based approach to automatic face detection," Proc. IEEE International Symposium on Signal Processing and Information Technology, Darmstadt, Germany, Dec. 2003.

[7] H. A. Rowley, S. Baluja, and T. Kanade, "Neural network-based face detection," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 20, no. 1, pp. 23-38, Jan. 1998.

[8] K. K. Sung and T. Poggio, "Example-based learning for view-based human face detection," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 20, no. 1, pp. 39-51, 1998.

[9] P. Viola and M. Jones, "Rapid object detection using a boosted cascade of simple features," Proc. Computer Vision and Pattern Recognition, pp. 511-518, Kauai, Hawaii, Dec. 2001.

[10] J. L. Wayman and A. J. Mansfield, "Technical analysis of the SmartGate project: summary and report," United Kingdom Biometrics Working Group, Technical report, December 6, 2003.