1996 IEEE International Conference on Systems, Man and Cybernetics

Invariant Face Recognition by Gabor Wavelets and Neural Network Matching nadet Pramadihanta

Iiaiyuan Wu

Masahiko Yachida

Department of Systems Engineering, Faculty of Engineering Science, Osaka University 1-3 Machikaneyama, Toyonaka, Osaka, Japan 560 (&de t ,whu y ,y achi da)@sy s.es.0 saka-u.ac.j p from single example, where the input face may include some transformations.

Abstract This paper presents a model-based face recognition approach that uses a hierarchical Gabor wavelet representation and neural network matching. Local features of gmy level images a& extracted by multiresolutions of Gabw wavelets, which are scaled and rotated versions of each other. The Gabor wavelet representation is use in a innovative neural network matching approach that can provide robust recognition. Neural network matching between a model and a input image is to fmd out the exact correspondence of local features and to map the model to the input image based on local similarity and neighborhood grouping of local features. The results on face recognition are presented, where the objects undergo rotation, translation, local distortions,and deformation caused by facial expression.

1. Introduction Object recognition systems which can robustly deal with the presence of some possible changes included in the input image, such as geometric transformations (translation, rotation, scaling, etc.), distortion, changes in lighting intensities and so on in parallel are difficult problems in the field of pattern recognition. Face recognition is one of them which also includes variations of deformation caused by facial expressions. Current approaches use shape primitives, silhouette and contour, colors, and invariant object feahues for matching. The performance of these methods is acceptable when the objects are well defined, and have high contrast. There are some recent works to achieve invariance with different approaches such as view-based recognition, deformable templates, etc. [IJ, and generic 3-Dmodels 121. In the flexible matching approach [3], the input image is deformed in 2-D to match the example view. The deformation, which is like a local 2-D warp of the image, allows the matching of input and example view even though they may differ in expression or out-of-plane rotation. The goal of this research is to recognize human face A

0-7803-3280-6/96/$5.00 k~ 1996 IEEE

- 59

2. Approach in Face Recognition The problem to he solvexi in a face recognition system can be stated as follows. Given a still images of a face, identify the person in the image using a stored data base of faces. The problems involve feature extraction from the images and matching processes between the input image and images stored in the data base. The matchjng process should be flexible enough to deal with some possihle changes included in input image. The general scheme of oiu face recognition system is shown in figure 1. The model base is generated offline based on frontal views of faces and application of Gabor wavelets to the image area which contain the faces. Face models are represented by the magnitude of multiscale Gabor wavelets filter. Faces in the input images are recognized when they successfully match with a specific model based on distinctive local features in the Gahor wavelet representation.

Lnral Fraturcs

Dctcctitin Multi-rrsnlii lion

r-7

hlatcliing Evaluation

1

Ilccopnition llrsiilt

Figure 1. Our face recognition approach

-

The main contributions of our work are the use of a hexagonal grid on the Gabor wavelet centex placement, neuron arrangement in the neural network matching process and the use of a self-organizing matching process for invariant face recognition. Our nemal network model is differs from methods previously discussed in [41, because in our model neuron grid position is allowed to deform, which implies more flexibility in the mapping or matching process. We used hexagonal grid tessellation, because it bas been considered an optimal way to distribute picture elements on the screen surface. The advantages in using a hexagonal grid instead of a conventionalrectangular grid include uniform connections property, greater angular resolution, higher degree of symmetry, and savings in storage and computations for various processing operation [5].

3. Gabor Wavelets The general form of a two dimensional Gabor wavelet function is given as

In this function, (x,y) is the spatial centroid of the Gaussian window and digital pixel coordinates m the image. (T determine the width of the Gaussian window and 0 determines the orientation. k is the frequency aspect which determines the shape of the wavelet. Multiscale Gabor wavelets are given as follows

where j is the number of scales, j=( 1,2, ...}.

Figure 2. An illustration of Gabor multiscale wavelet for local feature extraction and center placement in hexagonal

grias

Gabor features at each spatial location are extracted by taking the convolution of the grey level image with the family of Gabor wavelets at different scales and orientations

XY

Here (WI), symbolizes a convolution of an image with Gabor wavelets and I(x,y) are pixels in grey level images. 'zhe Gabor wavelet decomposition of an object image I(x,y) is m iconic multiscale template. Its sparse center position in the image is arranged according to hexagonal grids.

4. Local Similarity of Gabor Features Gabor features at each spatial location in the input image and the model image are given as vectors of Gabor wavelet decomposition of magnitude, P and Q, respectively. Local features between two objects corresponds to a search of maximum similarity between a Gabor features of model image and Gabor features of input image. The similarity between two Gabor features is computed as follows

This similarity implies initial guessing of the matching process, which may contain several local ambiguities. It appears because in an object there are some local Gabor features may be vety similar to each other. This ambiguity problem is left to be solved in flexible matching process to find the exact CofTespondmce of local Gabor features. The basic idea of the flexible matching process is to use the neighborhood of local Gabor feature information to find exact the counterpart of local features by flexiblely updating the weight of weighted similarity local features and to find a flexible mapping of local features in the model image to input image by allowing grid deformation.

5. NeuraI Network Matching Model After defining the similarity of local features of Gabor wavelets, which may contain local ambiguities, the flexible matching starts to further verify the hypothesis of local similarity and solving the ambiguities. Our matching model discussed further seems to be similar with dynamic link matching discussed in [4]. The differences are first the neural arrangement in planar space;in our model, neurons are arranged in hexagonal grids instead of rectangular grids, which incorporate a higher degree of symmetry. The second is that our model allows grids position to deform in order to find the best

- 60 -

-

mapping process in a sparse grid matching process. And the third is the difference in application. The flexible matching model composed of two neural Iayeq which represent an input image and model image, respectively. The input image layet, the X-layer is define by a dynamic equation

and the model image layer, which is influenced by activation of the X-layer through interlayer connections. The incorporated weighted local feature similarity of both layers is defined as

appear in the Y-layer as an equilibrium solution, centered on the neuron which receives the largest activity flow. -Updating pn>cedure , Ap the activity in both layers X and Y have converged to eqylilibrium blob solution, the weight strength J is updated a d i n g to

a'

- Grids Deformation Sparse grid mapping of neurons in the Y-layer to corresponding neurons in the X-layer are imply that in general, neurons in the Y-layer does not fall exactly on the neurons in the X-layers, but may lie somewhere between them.

Here, S(x) and S(y) are sigmoidal nonlinearity function,

a={al,a2}, b={bl,b2} are coordinateposition at X and Ylayer and p is constant excitation. The interaction kernels K(d) in both layers are Gaussian type weighted function, which d e f i i as follows,

This central moment defines the location of mapping neurons of the Y-laver in the X-laver sDam

K aa'(d)=G aa'(d)-o Gm,(d)= yexp( - d2/2s2)

(7)

which consist of short-range excitatory connections s and global inhibitory connections 0. The distance d is the distance in hexagonal grids.

This matching model encapsulates the mapping process of neurons in the Y-layer to their best counterparts in the X-layer by incorporating their neighborhood feature information. For a given randomly generated constant activation in the X-layer, a blob will appear in the X-layer as an equilibrium state. This blob activation appears at different positions at each iteration step depending on the input excitation. The activity flow from an active blob in the X-layer to the Y-layer is proportional to weighted local similarity features connected them. As a result a corresponding a blob will

- 61

Figure 3. The grey level shows the strength of neuron activations. Depending on the interlayer connection weights the mapping from the model layer to the input layer can be decided basedon equation (1 1).

-

6. Evaluation Success of the matching process is measured based on the activity correlation of active neurons in X-layer and Y-layer. The correlationof two neurons in X-layer and Ylayer is computed over iteration steps as follows

ba

where <> denotes averaging over the iteration steps. Their values range from - 1 to 1, which means as anticorrelated to correlated, the value of 0 is uncorrelatd. The discrimination of successful and unsuccessful matches is defmed as ‘=

blCba? 0.9) The higher C implies the successful match.

7. Experimental Results In this paper, Gabor wavelet filters are defintxl by 4 scales and 8 orientations. The parameters of neural network matching are set that at the equlibrium state there are about 40% of neurons in a layers activated. Grey level images in the databaqe and input images have the size of 256 x 256 pixels. They aere taken from original images of size 640 x 480 pixels by cutting them to square dimension of 480 x 480 pixels and then decimated down to 256 x 256 pixels. Figure 3. show an example of matching results where the percieved image undergoes out of plane and planar rotation. Correlation based matching evaluation for two input images shown in figure 4b and c is presented in figure 4d, where the x axis is matching steps, A is correlation result from matching figure 4a and 4b, and B is the correlation of matching of figure 4a and 4c.

only one example view of each person is available. This system is based on multiscale Gabor wavelets and neural network matching techniques. The features of faces are extracted by applying multiscale Gabor wavelets transformations. Flexible neural network matching techmques, which also allow for gnd deformations are good techniques for sparse grid matching. The system discussed in this paper able to recognize faces which include several variations such as rotation, shift, local distortion, and deformation caused by facial expression. REFERENCES [ 11 Chellapa R. et.all.,”Human and Machine Recognition

of Faces: A Survey”, Roc. of the IEEE, ~01.83, No.5, 1995. [2] Beymer D. and Poggio T. ”Face Recognition from ine Example View”, International Conference on , 1995 [3] Lades, M. et.al., ”Distortion Invariant Object Recognition in the Dynamic Link Architecture”, IEEE Trans. on Computers, vo1.42, 300-3 1 1, 1993. 141 Konen, W. and von der Malsburg C.,”Learning to Generalize from Single Examples in the Dynamic Link Architecture”, Neural Computation, vo1.5, 7 19735, 1993. 151 Her I., “A Symmetrical Coordinate Frame on the Hexagonal Grid for Computer Graphics and vision“, Trans. of the ASME, J. of Mech. Design, vo1.155, 447-449, 1993. [6] Lampinen J. et& ”Distortion Tolerant Pattern Rtxogmtion Based on Self-organizing Feature Extraction”, IEEE Trans. on Neural Network, ~ 0 1 . 639-547, 3 1995. 171 Daugman, J.,”Complete discrete 2-D Gabor transform by neural networks for image analysis and compression”, IEEE Trans. ASSP, vo1.36, 11691179, 1988. [SI Mallat S.G.,“A Theory for Multiresolution Signal Decomposition: The Wavelet Representation”, IEEE Trans on PAM1 vol.l1,No.7, 1989.

8. Future work For future work in our approach to face recognition from one example view, we now deal with minimizing the computational complexity of the matching model, and constructing a flexible fuzzy neural matching methods. We plan to apply our approach to a larger database.

9. Conclusion In ths paper we have addressed the problem of recognizing faces under various changes in parallel when

- 62 -

Correlation

0

20

40

60

80 100 120 1 4 0 160

(4

Figure 4. Result of matching ptocess. (a) a face in the data base. (b) and (c) are input images. (d) is matching evaluation based on cmelation as in equation (1 2) and (1 3).

- 63 -