Facial Expression Recognition Using Prior Attention

Table of contents

• General info
• Background
• Repository Description
• Attention Mechanism
• Architecture
• Requirement
• Further development ideas
• References
• Notes

General info

In this project, we propose an approach based on an attention mechanism in the prepossessing phase, even before training our network. which requires low resource consumption and is still able to achieve great performance over previous models on the FERG database.

Background

Over the last few years, there is an increase in the desire to use deep learning tools to understand human behavior patterns. One of the most trendy challenges in this field is facial expression recognition. Facial expressions are a form of nonverbal communication. The immediate association when talking about human behavior is facial expression and, it's can tell us a lot about human emotions. The challenge is predicting the underlying image to one of seven different emotions (happiness, sadness, anger, fear, disgust, surprise, neutral).

Repository Description

Filename	description
emotion-recognision.ipynb	The main file in google colab format, including the prepossessing. to open import to google colab
emotion-recognision.py	The main file in Python format
prepossessing.py	The prepossessing only in a python format
dataset.py	Python file consists of the implementation of the dataset object.
architecture.py	Python file consists of the implementation of the proposed architecture.
res	Folder consists of all the images from the project
requirement.txt	File containing all the packages we used in this project
FinalProjectDeepLearning.pdf	The report of the project

Attention Mechanism

We proposed an attention mechanism that relies on a prior belief that much of the clues come from two major regions- the mouse region and the eyes region. Considering this fact we divide the images into two blocks.

• Upper Block consists of the eyes and eyebrows.
• Lower Block consists of the mouse.
  We extract those two blocks by a pretrained model from OpenCv library and others helper functions we implemented.

The fully explenation is in the report.

Architecture

The proposed architecture used for this work is further explained in the report

Requirement

To run this project, you need to install several packages. For convenience, we created a requirement.txt file consists of all the packages used in this projcet.

In order to install all the packages in the requirement.txt file, simply use to command pip install -r requirements.txt.

Further development ideas

1. Try to expand our work to more facial expression datasets.
2. Try different uses of the attention mechanism.

References

• FERG dataset source: FERG dataset

Notes

• The data, and the predictor are too big to be uploaded to GitHub. You can found them in the following links: train validation, test, raw-images, predictor (for the raw images and the predictor you need to download the files and put them in a folder with the exact same name as in the original folder)