Pan Shot Face Unlock: Towards Unlocking Personal Mobile Devices using Stereo Vision and Biometric Face Information from multiple Perspectives

Personal mobile devices hold a vast amount of private and sensitive data and can e. g. be used to access services with associated cost. For security reasons, most mobile platforms therefore implement automatic device locking after a period of inactivity. Unlocking them using approaches like PIN, password or an unlock pattern is both problematic in terms of usability and potentially insecure, as it is prone to the shoulder surfing attack: an attacker watching the display during user authentication. Hence, face unlock – using biometric face information for authentication – was developed as a more secure as well as more usable personal device unlock. Unfortunately, when using frontal face information only, authentication can still be circumvented by a photo attack: presenting a photo/video of the authorized person to the camera. In this work we present a variant of face unlock which is harder to circumvent than with using frontal face information only by using more facial information, available during a 180∘ pan shot around the user’s head. We develop and evaluate our mobile device pan shot face unlock in four different stages in order to identify conceptual weaknesses and do improvements within the next stage. In the first stage we present a proof-of-concept prototype based on Android, which uses different Viola and Jones Haar-cascades for face detection and Eigenfaces for face recognition. We identify Eigenfaces as being insufficient for usage in a mobile device unlocking scenario. Therefore, we utilize neural networks and support vector machines for face recognition in the next stage, with which we identify using Viola and Jones based face detection as being insufficient for usage in a mobile device pan shot unlocking scenario based on multiple perspectives. Hence, we develop a novel face detection and segmentation approach based on stereo vision and range template matching in the next stage, which we find to deliver promising results and consequently focus on improving details of the range template generation and matching within the fourth and last stage. Parallel to developing and evaluating our approach we build up the u’smile face database containing grayscale and stereo vision pan shot test data. Concluding, our results indicate that a mobile device pan shot face unlock is a viable approach to unlocking mobile devices and that using range information might in general be an effective approach for incorporated face detection and segmentation.