Covid-19 has impacted our lives in many ways, but one of the most noticeable is the wearing of face masks.

The World Health Organisation encourages mask-wearing as a way to prevent the spread of the virus and many governments around the world have made wearing a mask compulsory in public spaces.

However, researchers from Cornell University have developed a new device which is able to identify facial expressions even if someone is wearing a mask. These new facial devices, known as C-Face, can track movements by monitoring the contour of the cheeks, and then translate this into emojis or silent speech commands.

The C-Face

This innovative device is placed on the ear using earphones. Then, it tracks facial expressions and translates them, which allows communication even when the user is wearing a mask.

This is a particularly exciting development for those who have hearing issues and rely on lipreading, as it offers a new way to communicate. This facial device technology is still in the study phase, but recently produced prototypes have been shown to work effectively.

The technology of the future

The C-Face uses two miniature RGB cameras that sit below each ear. They use machine learning to understand the contours of the face and which expression this movement relates to.

When facial muscles move, the cameras record this and feed the changes onto a machine learning model. This model continuously outputs 42 facial feature points such as the eyes, eyebrows and mouth. Cheng Zhang, who is leading the project, stated:

“With previous technology to reconstruct facial expression, you had to put a camera in front of you. But that brings a lot of limitation. Right now, many people are wearing a face mask, and standard facial tracking will not work. Our technology still works because it doesn’t rely on what your face looks like.”

The research

The facial device was tested on nine volunteers. This research shows that the device accurately predicted facial arrangements for different expressions to within 1 millimetre when compared to a library of images taken with a front-positioned camera.

With more research, it is hoped that the C-Face could be rolled out worldwide and help improve the mask-wearing experience.