During the COVID-19 pandemic, requiring people to wear facemasks in public places and shops was an important measure taken by governments all over the world. The requirement is no longer in place, but in Japan and elsewhere, many people still choose to wear a facemask as a precaution, and it is expected that facemasks will remain commonplace. This poses a problem to security and identification systems, as it is difficult to correctly automatically identify a person whose half face is covered by a mask. To address this situation, computational methods capable of ‘inpainting’ masked face images have been developed, with some degree of success. Now, Yuichi Sei and colleagues from The University of Electro-Communications, Tokyo, have created a method that not only uses visual information, but also audio — the person’s voice — to reconstruct a masked face. The approach enables face shape restoration with improved quality.

Read more