Combining Heterogeneous Features for Facial Action Recognition Using Multikernel Learning

Abstract

Facial action recognition (FAR) is an important but challenging task in computer vision due to the high variability of facial expressions. To capture the complex facial features, diverse face representations need to be employed. In this paper, we propose a novel multi-kernel learning algorithm to effectively integrate heterogeneous features for FAR. We employ seven types of facial features, including Local Binary Patterns (LBP), Histograms of Oriented Gradients (HOG), convolutional neural network (CNN) features, Gabor Wavelets, and so on, and learn a combination of these different feature types in a unified way. Two single kernel support vector machine (SVM) models and a multiple kernel learning SVM model are compared in our experiments. By combining all seven feature types, we achieve an accuracy of 62.02% on the JAFFE dataset, which is far better than the performance of any single feature type. Moreover, our proposed multi-kernel learning method achieves a competitive accuracy