Apple is facing our face-masked future. This week, the company began testing new software for the iPhone that will allow device owners to unlock the handset while wearing a face cover. There is one caveat, however, that aligns with Apple’s strategy of holding people to different Apple products and shows how challenging it can be to develop accurate facial recognition technology: the new face unlock feature requires an Apple Watch.
The first beta of iOS 14.5 will include updates to app tracking controls and Siri in addition to the face mask feature. App makers typically get early access to the latest version of iOS to launch or reimagine their apps well in advance of formal software release. (Brave souls who don’t mind the risk of potentially bricking their iPhones can also sign up for public beta versions.) The fully baked version of the software is expected to be made available to the general public this spring.
That means that by the time most people install the latest version of iOS on their iPhones, we have been wearing masks for a year or more to help prevent the spread of the coronavirus. Compared to all the other ways the pandemic has changed our lives, using any method other than Apple’s Face ID to unlock your iPhone isn’t a major inconvenience. Still, holding your phone up to your face is frustrating to remember that Face ID isn’t working because of your mask. The promise of facial recognition technology – which comes with very well-founded concerns about its misuse and error rates for dark-skinned people – is that it would get smarter and better over time.
With the upcoming software update, Apple more or less transfers the authentication payload to the Apple Watch. If you are using a newer iPhone model (one with Face ID) and have iOS 14.5 beta software installed, and you are wearing an apple watch with watchOS 7.4, if you hold the locked phone up to your face, a little communication between the phone and watch is triggered. The phone will be unlocked. The watch will also display a notification that the phone has been unlocked. An iOS developer described it to WIRED as a similar experience to unlocking a Mac with an Apple Watch.
As 9to5 Mac notes, this is the second change Apple has made to its Face ID face mask authentication system. Last spring, the company released software that made it easier not to use Face ID while wearing a mask by displaying the iPhone’s password screen after Face ID first failed. Still, these updates have their limitations. The new Face ID-with-face-mask feature only works with the phone unlock. So, if you’re using Face ID for Apple Pay transactions or to sign in to third-party apps, you still need to authenticate in another way.
But the bigger question is why Apple relies on the Apple Watch to unlock iPhones with face masks, rather than releasing software that simply recognizes the uncovered part of someone’s face. At the time of publication, Apple had not yet answered WIRED’s questions about this. Experts say there are plenty of ethical and technical considerations when it comes to using facial recognition technology, but when it comes to performing facial recognition on partially obscured faces, it’s particularly challenging.
Anil K. Jain, who researches computer vision, machine learning, and biometric recognition at Michigan State University, says that despite the advancements in facial recognition over the past five to 10 years, it is still “ prone to occlusion – that is, which part of the face is not visible. In most cases, the technology assumes that the person takes off their glasses and face cover, that the lighting is homogeneous and the expression is neutral, as in a passport photo. “