Facial Expression Recognition is the application of computer software to glean facial expression information from video and/or images. Computer Vision and Machine Learning techniques are used in combination to identify facial action units and other facial expressions. Facial expression recognition, or face recognition for short, has been studied for decades and commercially available now for a long time, mostly in a surveillance context (as a way to detect unauthorized access to a facility using video cameras). That said, it is only beginning to make its way into consumer products, with some apps supporting it on mobile devices and Facebookâs DeepFace research project being one of the most well-known research projects today.
In the past years, deep learning has been the driving force behind a number of breakthroughs in image processing and recognition. Deep convolutional neural networks (CNNs) trained with millions of examples have achieved unprecedented accuracy for recognizing objects, faces, and even activities in images. The accuracy of these models is not limited to images but can be extended to videos and audio. Inspired by these successes, there has been an increasing interest in applying similar techniques to human-robot interaction. In particular, one of the main challenges for human-robot interaction is how to interpret people’s facial expression signals from their face images or videos and use them to drive robot behaviours towards more natural and effective interactions. Although there has been a lot of work on this topic, most of these methods are only designed for recognizing a specific emotion or expression such as anger or happiness. However, it is well known that different emotions can share similar facial expressions (e.g., happiness and surprise). Therefore it is essential to be able to recognize different emotions simultaneously instead of only single emotion at a time.
Facial expression recognition is a new way to interact with machines and has many applications in human-robot interaction (HRI). This technology can be found in the following fields: Automotive industry, where it is used to assess driver’s attentiveness and fatigue level; Health care, to monitor patient’s vital signs through facial expression analysis of video recordings; Retail, for monitoring customers’ habits and mood as well as their reactions to marketing materials and advertisements. This technology provides a lot of possibilities for everyday life, but it also raises several privacy concerns. Facial expression recognition is usually performed by cameras or other sensors that capture people’s images without their knowledge. It is very important to emphasize that this technology can reveal a lot of sensitive information about a person and it should be used only with people’s consent.
Facial expression recognition is the next frontier of Human-Robot Interaction. We will see a lot of developments in this area in the next 5 years from both industry and academia. While current applications are centred around entertainment, I believe that facial expression recognition will be used for a lot more than just entertainment.