A bi-modal face recognition framework integrating facial expression with facial appearance

Pohsiang Tsai, Longbing Cao, Tom Hintz, Tony Jan

Research output: Contribution to journalArticlepeer-review

8 Citations (Scopus)


Among many biometric characteristics, the facial biometric is considered to be the least intrusive technology that can be deployed in the real-world visual surveillance environment. However, in facial biometric, little research attention has been paid to facial expression changes. In fact, facial expression changes have often been treated as noise that would degrade the recognition performance. This paper studies an innovative viewpoint: (1) whether facial expression changes, namely facial behavior, can be positively used for face recognition or not? (2) furthermore, can facial behavior be integrated with facial appearance for assisting the extra-personal separation to enhance face recognition performance? We propose a bi-modal face recognition framework which integrates facial expression with facial appearance. Substantial experiments on multiple facial appearance and facial expression data have been conducted. Our experimental results have validated that facial behavior can play a positive role in face recognition and can assist facial appearance in extra-personal separation in multiple modalities for personal identification improvement.

Original languageEnglish
Pages (from-to)1096-1109
Number of pages14
JournalPattern Recognition Letters
Issue number12
Publication statusPublished - 1 Sept 2009
Externally publishedYes


  • Biometrics
  • Facial appearance
  • Facial expression
  • Fusion techniques
  • Human Computer Interface (HCI)
  • Multimodal biometrics


Dive into the research topics of 'A bi-modal face recognition framework integrating facial expression with facial appearance'. Together they form a unique fingerprint.

Cite this