Automated Border Control Research Articles

ENTRY-EXIT SYSTEM SCOPE, API/PNR CAPABILITY WEAKNESS

The practical recommendations for the application of Advance Passenger Information and Passenger Name Record System (API / PNR) were described in the article with the aim to use such system as a counteracting cross-border threats tool. Biometrics establishes a strong link between a document and its holder only if all electronic security features (signatures) are reviewed. That is the critical point of all system. Many EU countries don’t have Public key infrastructure in place and can’t use the mentioned security functions. There no joint EU system. Only ICAO has a commercial Public key directory, which provides PKI functions to some countries. Specific digital algorithms know and do many things better, like people, and can make a significant improvement. The order of actions for system implementing is proposed. He will be responsible for Risk assessment, Biometric verification accuracy, and Document verification. It is mandatory to improve data management, mostly to cover all fields and flows of data. Improve legislation related to data management for the mentioned purpose and GDPR. Besides, we have to invest more in automatic border control systems infrastructure. It is mandatory to centralize Public Key Infrastructure and Certificates master list management and validation. All measures will increase public security and save innocent lives from cross border crimes.

ADFF: Adaptive de‐morphing factor framework for restoring accomplice's facial image

AbstractMorphing attacks (MAs) pose a substantial security threat to the Automatic Border Control (ABC) system. While a few morphing attack detection (MAD) methods have been proposed, the face morphing accomplice's facial restoration has not received sufficient attention. Due to the inability to foresee the morphing factor used for a particular morphed image, selecting the appropriate de‐morphing factor becomes a challenging problem in the restoration of the accomplice's facial image. If the morphing factor cannot be chosen reasonably, achieving the desired restoration effect is difficult. Therefore, this paper presents an adaptive de‐morphing factor framework (ADFF) architecture for restoring the accomplice's facial image. By exploiting the morphed images stored in the electronic passport system and the real‐time captured criminal's images, ADFF can effectively restore the accomplice's facial image. Experimental results and analysis show that ADFF can significantly reduce the security threats of MAs on ABC.

Survey on Different Morphology Detection Techniques

There is a lot of data traffic that translates and transports data in the digital environment of the World Wide Web. The data is presented as files and photos. Data can morph, so it's important to detect these instances. The suggested method will identify modified photographs and alert the user to the validity of the images. In recent years, the research community has paid a great deal of attention to the problem of morph attack detection. To accurately detect morph attacks, various studies have been done in this area and various methods have been used. Although enough morph images are not readily available for research purposes, a variety of face databases are used to create morph image databases. Attack detection via face morphing is difficult. Automated border control gates utilize both manual inspection and automatic categorization methods to identify morphing attacks. It is vital to comprehend how a machine learning system can recognise altered faces and the most pertinent facial regions.

Analyzing Human Observer Ability in Morphing Attack Detection—Where Do We Stand?

Morphing attacks are based on the technique of digitally fusing two (or more) face images into one, with the final visage resembling the contributing faces. Morphed images not only pose a challenge to Face-Recognition Systems (FRS) but also challenge experienced human observers due to high quality, postprocessing eliminating any visible artifacts, and further the printing and scanning process. Few studies have concentrated on examining how human observers can recognize morphing attacks, even as several publications have examined the susceptibility of automated FRS to morphing attacks and offered morphing attack detection (MAD) approaches. MAD approaches base their decisions either on a single image with no reference to compare against (Single-Image MAD (S-MAD)) or using a reference image (Differential MAD (D-MAD)). One prevalent misconception is that an examiner’s or observer’s capacity for facial morph detection depends on their subject expertise, experience, and familiarity with the issue. No works have reported the specific results of observers who regularly verify identity (ID) documents for their jobs. As human observers are involved in checking ID documents having facial images, a lapse in their competence can result in significant societal challenges. To assess the observers’ proficiency, this research first builds a new benchmark database of realistic morphing attacks from 48 different subjects, resulting in 400 morphed images. Unlike the previous works, we also capture images from Automated Border Control (ABC) gates to mimic realistic border-crossing scenarios in the D-MAD setting with 400 probe images, to study the ability of human observers to detect morphed images. A new dataset of 180 morphing images is also produced to research human capacity in the S-MAD environment. In addition to creating a new evaluation platform to conduct S-MAD and D-MAD analysis, the study employs 469 observers for D-MAD and 410 observers for S-MAD who are primarily governmental employees from more than 40 countries, along with 103 control group members who are not examiners. The analysis offers intriguing insights and highlights the lack of expertise and failure to recognize a sizable number of morphing attacks by experienced observers. Human observers tend to detect morphed images to a lower accuracy as compared to the four automated MAD algorithms evaluated in this work. The results of this study are intended to aid in the development of training programs that will prevent security failures while determining whether an image is bona fide or altered.

Survey on Different Morphology Detection Techniques

Abstract: There is a lot of data traffic that translates and transports data in the digital environment of the World Wide Web. The data is presented as files and photos. Data can morph, so it's important to detect these instances. The suggested method will identify modified photographs and alert the user to the validity of the images. In recent years, the research community has paid a great deal of attention to the problem of morph attack detection. To accurately detect morph attacks, various studies have been done in this area and various methods have been used. Although enough morph images are not readily available for research purposes, a variety of face databases are used to create morph image databases. Attack detection via face morphing is difficult. Automated border control gates utilize both manual inspection and automatic categorization methods to identify morphing attacks. It is vital to comprehend how a machine learning system can recognise altered faces and the most pertinent facial regions.

Testing and evaluation of the Automated Border Control System

AbstractAim: The aim of the study is to present the circumstances and results of the operational test and evaluation of the Automated Border Control System to be implemented in 2021 on the Hungarian-Ukrainian (Schengen external) border. The focus is on the operational efficiency of the system.Methodology: Through document and content analysis, we examined the theoretical and legal background of the system, its regulatory environment, and in order to achieve the results, we carried out a qualification measurement based on tool- and method-specific criteria.Findings: Measurements and evaluations have shown that, in terms of the ratio between the human resources used in border checks and the time taken to carry out the checks, the checks carried out by the automated border control system have proved to be more efficient.Value: The testing and evaluation of the kiosks and automated (fixed) gates that make up the components of the Automated Border Control System at Schengen external land border crossing points has not been carried out before in Hungary, and therefore this study and the study based on it are unique i n the border policing area.

Techniques and Challenges for Generation and Detection Face Morphing Attacks: A Survey

Face recognition system is the most widely used application in the field of security and especially in border control. This system may be exposed to direct or indirect attacks through the use of face morphing attacks (FMAs). Face morphing attacks is the process of producing a passport photo resulting from a mixture of two images, one of which is for an ordinary person and the other is a judicially required. In this case, a face recognition system may allow travel of persons not permitted to travel through face morphing image in a Machine-Readable Electronic Travel Document (eMRTD) or electronic passport at Automatic Border Control (ABC) gates. In creating an electronic passport, most countries rely on applicant to submit images in a form of a document or via the Internet, and this allows applicants to manipulate the images to produce morphing images. These photos allow both beneficial and harmful partners to cross borders using the same passport. This is considered a major threat to the security systems that allow them to travel without revealing their true identity. This paper aims to provide a comprehensive overview of face morphing attacks and the development taking place in this specialty. This paper describes the techniques for generating metamorphic images and challenges they face, in addition to the advantages and disadvantages of these techniques. It also dealt with types of techniques used in detecting and determining the attack of mutant faces in the field of deep learning or machine learning, in addition to the laws and criteria for measuring the efficiency of the algorithms used. It provides a general summary of the work that has been produced in this field.

Adversarial Patch Attacks on Deep-Learning-Based Face Recognition Systems Using Generative Adversarial Networks.

Deep learning technology has developed rapidly in recent years and has been successfully applied in many fields, including face recognition. Face recognition is used in many scenarios nowadays, including security control systems, access control management, health and safety management, employee attendance monitoring, automatic border control, and face scan payment. However, deep learning models are vulnerable to adversarial attacks conducted by perturbing probe images to generate adversarial examples, or using adversarial patches to generate well-designed perturbations in specific regions of the image. Most previous studies on adversarial attacks assume that the attacker hacks into the system and knows the architecture and parameters behind the deep learning model. In other words, the attacked model is a white box. However, this scenario is unrepresentative of most real-world adversarial attacks. Consequently, the present study assumes the face recognition system to be a black box, over which the attacker has no control. A Generative Adversarial Network method is proposed for generating adversarial patches to carry out dodging and impersonation attacks on the targeted face recognition system. The experimental results show that the proposed method yields a higher attack success rate than previous works.

KurdFace Morph Dataset Creation Using OpenCV

Automated facial recognition is rapidly being used to reliably identify the identities of individuals for a variety of applications, from automated border control to unlocking mobile phones. The attack of Morphing has presented a significant risk to the face recognition system (FRS) at automated border control. Face morphing is a technique for blending the facial images of two or more people such that the outcome looks like both of them. For example, a morphing attack may be used to get a fake passport by using a morphed image. This passport can be used by both the modified image contributors while crossing the border. Due to the publicly available digital altering tools that criminals may use to carry out face morphing attacks. Morph Attack Detection (MAD) systems have received a lot of attention in recent years. In the absence of automated morphing detection, Face Recognition Systems (FRS) are extremely susceptible to morphing attacks. Due to the limited number of publicly available face morph datasets to investigate, especially to our knowledge, there is no Kurdish morph dataset. In this work, we decided to generate a new face dataset, including morphed images which we named as "KurdFace" dataset. OpenCV was used to generate morphed images. Then we study the susceptibility of biometric systems to such morphed face attacks by designing and creating a Morph Attack Detection model to distinguish morphed images from genuine ones. To evaluate the robustness of our dataset regarding morphing attack detection, we compare it with the AMSL dataset to determine the classification error rate on both datasets to see how our dataset is different from others. Local Binary Pattern and Uniform Local Binary Pattern are used as feature extraction techniques, and as a classifier, SVM is utilized. The experimental result shows that our dataset is suitable for research purposes.

A multi-agent based enhancement for multimodal biometric system at border control

Border criminals and unauthorized immigrants are dramatically increasing within the last few years in globally due to the absence of proper authorization methods at the border locations. Usually, passenger controls done by trained immigration officers who compare the passport and the physical appearance at the border while some of the countries done by automated border control (ABC) systems. ABC is one of the applicable real-world applications of the biometric domain which commonly implements with fingerprint and face (multimodal) biometric authorization. However, selecting an appropriate classification method is a challenging task at the decision-making stage. This paper proposes a novel architecture for multimodal biometric authorization engaged with the multi-agent system (MAS) to come up with the optimal solution by using the co-feature of MAS, such as coordination, corporation, and negotiation features. The experiment was done with four available multimodal datasets, namely, the National Institute of Standard Technology (NIST) multimodal, SDUMLA-HMT multimodal, BANCA and PRIVATE databases have been reported to demonstrate the efficiency of the proposed method. The experimental result delivers an excellent performance comparing with previous ABC systems at the authentication phase and computationally fast.

FaceHack: Attacking Facial Recognition Systems Using Malicious Facial Characteristics

Recent advances in machine learning have opened up new avenues for its extensive use in real-world applications. Facial recognition, specifically, is used from simple friend suggestions in social-media platforms to critical security applications for biometric validation in automated border control at airports. Considering these scenarios, security vulnerabilities of such facial recognition systems pose serious threats with severe outcomes. Recent work demonstrated that Deep Neural Networks (DNNs), typically used in facial recognition systems, are susceptible to backdoor attacks; in other words, the DNNs turn malicious in the presence of a unique trigger. Detection mechanisms have focused on identifying these distinct trigger-based outliers statistically or through reconstructing them. In this work, we propose the use of facial characteristics as triggers to backdoored facial recognition systems. Additionally, we demonstrate that these attacks can be realised on real-time facial recognition systems. Depending on the attack scenario, the changes in the facial attributes may be embedded artificially using social-media filters or introduced naturally through facial muscle movements. We evaluate the success of the attack and validate that it does not interfere with the performance criteria of the model. We also substantiate that our triggers are undetectable by thoroughly testing them on state-of-the-art defense and detection mechanisms.

Face morphing attack detection and attacker identification based on a watchlist

Face verification is widely used for Automated Border Control (ABC) in many countries. But such ABC systems are vulnerable to Morphing Attacks (MAs), where a morphed face image is submitted to apply for a passport. To counter face MAs, this paper proposes to detect morphed face images and identify morphing attackers by the use of a watchlist. It is carried out by comparing a suspect image with the biometric references contained in a watchlist, and its detection process is accomplished by analyzing the results of face comparison. Once a morphed image is detected, its morphing attacker is also identified. Meanwhile, a database with different morphing methods, image qualities, facial expressions, and face angles is collected. Experimental results and analysis show that it can achieve stable detection and attacker identification performance for 4 different face MAs, and it can well generalize to unseen morphing types and weights. Moreover, it has good robustness to variations of image qualities, facial expressions, and face angles.

Feature Focus: Towards Explainable and Transparent Deep Face Morphing Attack Detectors

Detecting morphed face images has become an important task to maintain the trust in automated verification systems based on facial images, e.g., at automated border control gates. Deep Neural Network (DNN)-based detectors have shown remarkable results, but without further investigations their decision-making process is not transparent. In contrast to approaches based on hand-crafted features, DNNs have to be analyzed in complex experiments to know which characteristics or structures are generally used to distinguish between morphed and genuine face images or considered for an individual morphed face image. In this paper, we present Feature Focus, a new transparent face morphing detector based on a modified VGG-A architecture and an additional feature shaping loss function, as well as Focused Layer-wise Relevance Propagation (FLRP), an extension of LRP. FLRP in combination with the Feature Focus detector forms a reliable and accurate explainability component. We study the advantages of the new detector compared to other DNN-based approaches and evaluate LRP and FLRP regarding their suitability for highlighting traces of image manipulation from face morphing. To this end, we use partial morphs which contain morphing artifacts in predefined areas only and analyze how much of the overall relevance each method assigns to these areas.

Face Morphing Attack Generation and Detection: A Comprehensive Survey

Face recognition has been successfully deployed in real-time applications, including secure applications such as border control. The vulnerability of face recognition systems (FRSs) to various kinds of attacks (both direct and indirect attacks) and face morphing attacks has received great interest from the biometric community. The goal of a morphing attack is to subvert an FRS at an automatic border control (ABC) gate by presenting an electronic machine-readable travel document (eMRTD) or e-passport that is obtained based on a morphed face image. Since the application process for an e-passport in the majority of countries requires a passport photograph to be presented by the applicant, a malicious actor and an accomplice can generate a morphed face image to obtain the e-passport. An e-passport with a morphed face image can be used by both the malicious actor and the accomplice to cross a border, as the morphed face image can be verified against both of them. This can result in a significant threat, as a malicious actor can cross the border without revealing the trace of his/her criminal background, while the details of the accomplice are recorded in the log of the access control system. This survey aims to present a systematic overview of the progress made in the area of face morphing in terms of both morph generation and morph detection. In this article, we describe and illustrate various aspects of face morphing attacks, including different techniques for generating morphed face images and state-of-the-art morph attack detection (MAD) algorithms based on a stringent taxonomy as well as the availability of public databases, which allow us to benchmark new MAD algorithms in a reproducible manner. The outcomes of competitions and benchmarking, vulnerability assessments, and performance evaluation metrics are also provided in a comprehensive manner. Furthermore, we discuss the open challenges and potential future areas that need to be addressed in the evolving field of biometrics.

Suspicious Infrastructures: Automating Border Control and the Multiplication of Mistrust through Biometric E-Gates

ABSTRACT In recent years, practices of control at EU border crossing points have been progressively transferred to electronic gates, or what is often called Automated Border Control (ABC). In this paper, I unpack ABC’s infrastructure and argue that e-gates enact three distinct modes of suspicion: First, they address the mistrust towards travellers’ identity claims and promise to better detect identity fraud and the misuse of other persons’ ID documents. Second, they replace the manual work of border guards, which has itself been suspected of being unreliable and error-prone. Third, however, ABC has also raised suspicion among border guards and data protection advocates alike, due to its opaque mode of operation. To examine how these three modes of suspicion unfold, I first show that automating border control relies on a heterogeneous entanglement of material devices, calculative practices and new forms of data. Drawing on document analysis and participant observation of ABC, I then trace the socio-technical controversies that its proliferation has sparked, arguing that e-gates have significantly reconfigured how suspicion at the EU borders is enacted and led to a multiplication of mistrust in the European border regime.

Preventing face morphing attacks by using legacy face images

Countries allow citizens to upload a face image or provide printed copies to authorities to issue their passport. This allows prior image manipulation with criminal intent. A composite image can be created by blending the images of two individuals before submitting the composite image to the authorities. Depending on several factors, the submitted morphed face image can fool the issuing officer to issue a legitimate document. The document can then be successfully used by either contributor to attack the automatic Face Recognition Systems (FRS) operating, for example, at Automatic Border Control (ABC) airport gates. This is known as a Morphing Attack (MA), an identity sharing scheme with serious consequences. Here, the security vulnerabilities due to MAs are identified and analysed, and an additional security measure that allows mitigating the risk or preventing MAs in certain scenarios is proposed. The measure introduces more comparisons by keeping the old passport or ID card image in the chip, in passport renewal applications or first time passport applications, respectively. This approach is implemented with two FRSs on a challenging dataset and the dramatic decrease in the vulnerability is shown. Finally, their performance is compared with a state-of-the-art MA detection algorithm on the same dataset.

Extended evaluation of the effect of real and simulated masks on face recognition performance.

Face recognition is an essential technology in our daily lives as a contactless and convenient method of accurate identity verification. Processes such as secure login to electronic devices or identity verification at automatic border control gates are increasingly dependent on such technologies. The recent COVID‐19 pandemic has increased the focus on hygienic and contactless identity verification methods. The pandemic has led to the wide use of face masks, essential to keep the pandemic under control. The effect of mask‐wearing on face recognition in a collaborative environment is currently a sensitive yet understudied issue. Recent reports have tackled this by using face images with synthetic mask‐like face occlusions without exclusively assessing how representative they are of real face masks. These issues are addressed by presenting a specifically collected database containing three sessions, each with three different capture instructions, to simulate real use cases. The data are augmented to include previously used synthetic mask occlusions. Further studied is the effect of masked face probes on the behaviour of four face recognition systems—three academic and one commercial. This study evaluates both masked‐to‐non‐masked and masked‐to‐masked face comparisons. In addition, real masks in the database are compared with simulated masks to determine their comparative effects on face recognition performance.

Face Morphing, a Modern Threat to Border Security: Recent Advances and Open Challenges

Face morphing poses a serious threat to Automatic Border Control (ABC) and Face Recognition Systems (FRS) in general. The aim of this paper is to present a qualitative assessment of the morphing attack issue, and the challenges it entails, highlighting both the technological and human aspects of the problem. Here, after the face morphing attack scenario is presented, the paper provides an overview of the relevant bibliography and recent advances towards two central directions. First, the morphing of face images is outlined with a particular focus on the three main steps that are involved in the process, namely, landmark detection, face alignment and blending. Second, the detection of morphing attacks is presented under the prism of the so-called on-line and off-line detection scenarios and whether the proposed techniques employ handcrafted features, using classical methods, or automatically generated features, using deep-learning-based methods. The paper, then, presents the evaluation metrics that are employed in the corresponding bibliography and concludes with a discussion on open challenges that need to be address for further advancing automatic detection of morphing attacks. Despite the progress being made, the general consensus of the research community is that significant effort and resources are needed in the near future for the mitigation of the issue, especially, towards the creation of datasets capturing the full extent of the problem at hand and the availability of reference evaluation procedures for comparing novel automatic attack detection algorithms.

Adversarial Attacks Against Face Recognition: A Comprehensive Study

Face recognition (FR) systems have demonstrated reliable verification performance, suggesting suitability for real-world applications ranging from photo tagging in social media to automated border control (ABC). In an advanced FR system with deep learning-based architecture, however, promoting the recognition efficiency alone is not sufficient, and the system should also withstand potential kinds of attacks. Recent studies show that (deep) FR systems exhibit an intriguing vulnerability to imperceptible or perceptible but natural-looking adversarial input images that drive the model to incorrect output predictions. In this article, we present a comprehensive survey on adversarial attacks against FR systems and elaborate on the competence of new countermeasures against them. Further, we propose a taxonomy of existing attack and defense methods based on different criteria. We compare attack methods on the orientation, evaluation process, and attributes, and defense approaches on the category. Finally, we discuss the challenges and potential research direction.

Morphing Attack Detection-Database, Evaluation Platform, and Benchmarking

Morphing attacks have posed a severe threat to Face Recognition System (FRS). Despite the number of advancements reported in recent works, we note serious open issues such as independent benchmarking, generalizability challenges and considerations to age, gender, ethnicity that are inadequately addressed. Morphing Attack Detection (MAD) algorithms often are prone to generalization challenges as they are database dependent. The existing databases, mostly of semi-public nature, lack in diversity in terms of ethnicity, various morphing process and post-processing pipelines. Further, they do not reflect a realistic operational scenario for Automated Border Control (ABC) and do not provide a basis to test MAD on unseen data, in order to benchmark the robustness of algorithms. In this work, we present a new sequestered dataset for facilitating the advancements of MAD where the algorithms can be tested on unseen data in an effort to better generalize. The newly constructed dataset consists of facial images from 150 subjects from various ethnicities, age-groups and both genders. In order to challenge the existing MAD algorithms, the morphed images are with careful subject pre-selection created from the contributing images, and further post-processed to remove morphing artifacts. The images are also printed and scanned to remove all digital cues and to simulate a realistic challenge for MAD algorithms. Further, we present a new online evaluation platform to test algorithms on sequestered data. With the platform we can benchmark the morph detection performance and study the generalization ability. This work also presents a detailed analysis on various subsets of sequestered data and outlines open challenges for future directions in MAD research.