Biometric Performance Metrics Can Help You Select the Right Biometric Solution

Growing popularity of biometrics in mainstream identification and authentication has made it vitally important to ensure that the performance of biometric systems, solutions and applications rates at an acceptable level. In any system or solution, performance evaluation and metrics boost confidence of prospective adopters. Performance evaluation particularly becomes important for systems that are going serve crucial or high security applications. Biometric identification and authentication methods have been employed at various high security facilities and applications for quite a while. Governments of many countries are using citizens’ biometrics for identification, verification and distribution of government funded facilities as well as non-government benefits. This level of usage demands biometric systems, solutions and applications to be a consistent performer and requires that performance evaluation has been executed prior to the deployment.

Biometric performance metrics rates performance of a biometric system, solution or application. Different metrics can be used for the purpose. The most common performance metrics are the FAR (false acceptance rate) and the FRR (false rejection rate).

Biometric systems first require a person to establish his or her identity in the system. This process is called enrollment the person has to present his or her biometric for imaging/scanning. Captured biometrics is processed by biometric and encryption algorithms. A biometric template is generated as a result of processing, which is stored in a database and associated with identity data of that person. The procedure so far is called enrollment. Next time, when this person requires verification of his or her identity, biometrics are re-scanned, processed and the new template is compared with the existing one in the database. The matching algorithm returns a match in case of acceptance or no-match in case of rejection.

The FAR (false acceptance rate) is the probability of cases for which a biometric system fallaciously authorizes an unauthorized person. It happens when a biometric system, solution or application inaccurately matches a biometric input with a stored template, fallaciously returning a match and granting access to an unauthorized person. It is one of the commonly used metrics in biometric recognition systems for assessing the performance of the system.

The FAR measures false acceptances by a biometric system, for example: typically, biometric systems are used to allow access to restricted areas only to authorized persons. Suppose there are two people X and Y, Y has access to the system while X has no permission. A false acceptance is obtained when X is recognized as Y (or any other person with permission) at the time of the verification by granting him permission to access reserved areas even if he or she is not entitled to it. It can be easily imagined what consequences a false acceptance can bring in, specially in a system with a high level of security, such as a military base, or a nuclear power plant.

False acceptance is an undesirable result from a biometric system. It is expressed as the percentage of instances where system will authorize an unauthorized person. For example: if FAR = 0.1% that means that in 1 out of 1000 cases, a biometric system, solution of application have probability to grant access to an unauthorized individual. In systems where the level of security is high, the existence of false positives is a serious problem; for these systems FAR = 0 is will be a desirable scenario, i.e. there should be no false positives.

The FRR (false rejection rate), on the contrary, is the probability of cases for which a biometric system fallaciously denies access to an authorized person. It happens when a biometric system, solution or application fails to match the biometric input with a stored template, fallaciously returning a no-match and denying access to an authorized person. The False Rejection Rate (FRR) is one of the important metrics along with FAR and commonly used to for assessing the performance of a biometric systems, solutions and applications. Like FAR, it is also expressed as percentage of probability, in which a system will fallaciously deny access to an authorized person. For example if FRR = 0.01%, it means that 1 out of 10000 cases, a biometric system, solution of application have probability to deny access to an authorized individual.

For more information on FAR and FRR read this article: False Acceptance Rate and False Recognition Rate (https://www.bayometric.com/false-acceptance-rate-far-false-recognition-rate-frr/)

FAR and FRR are most common metrics used to evaluate performance of a biometric systems, solutions or applications. During the performance evaluation of a biometric system, many genuine and impostor attempts are made to collect data of false acceptance and false rejections. Genuine attempts are attempts made by an individual to match with his/her stored template, while an imposter attempt is made to match with someone else’s biometric template. Genuine attempts help measure false rejections while imposter attempts are made to count false acceptance.

Precise extraction of features is the key to improve performance of a biometric system; performance does not solely depend on the biometric algorithm. Biometric algorithm only makes use of what presented to it, i.e. raw data. Poorly scanned biometric features will make it hard for biometric algorithm to identify and calculate available minutia that will affect overall performance of a biometric system. Biometric algorithm does play a role in performance of a biometric system, but precise extraction of biometric features remains the important factor.

FMR is an empirical estimate of the probability (the percentage of times) at which the system incorrectly declares that a biometric sample belongs to the claimed identity when the sample actually belongs to a different subject. During the performance evaluation, pairs of FRR and FAR (or FNMR and FMR) are calculated.

An empirical estimate of the probability at which the system incorrectly rejects a claimed identity when the sample actually belongs to the subject (genuine user).

The rate at which FMR is Equal to FNMR.

FAR and FMR are often used interchangeably in literature, so as FNMR and FRR. However, their subtle difference is that FAR and FRR are system level errors which include sample failed to be acquired or compared.

It is defined as 1 – FRR. This measure represents the degree that the biometric system is able to correctly match the biometric information from the same person. Developers of biometric systems attempt to maximize this measure.

It is defined as the weighted sum between FNMR (FRR) and FMR (FAR).

Different biometric systems have different capacity to store biometric templates. Template capacity tells the maximum number of sets of data that can be stored in the system.

The time it takes for an individual to be authenticated or identified using the application.

Performance rates of a biometric system can be expressed in many ways, e.g. in decimal format (0.05), in percent (1%), as fractions (1/100) or by using powers of ten (10­2). When comparing two systems, the more accurate one would show lower FRR at the same level of FAR.

FAR and FRR are dependent on many factors including technical implementation, quality of biometrics sampled, environmental factors, etc. FRR significantly depends on user behavior and quality of biometrics presented. A person successfully enrolled earlier, may have a number of conditions at the time of verification, affecting extraction of biometric features and inhibiting verification process, for example: sticky, wet, dirty or worn out fingertips, diseased eye, hoarseness in voice, pain in arm affecting ability to signature, etc. Some of these conditions may be temporary; others can be permanent, affecting performance of a biometric system.

Amount of useful information in samples and capability of a biometric system, these two are the main categories of factors that can affect performance of a biometric system. Technical ability of a biometric system will fluctuate with the quality of biometric raw data presented; it affects both, enrollment as well as verification. Poor biometric samples increase the false acceptance and rejection percentage, decreasing the system performance. Hence, controlling the quality of the acquired biometric raw data is essential in order to have an efficient biometric authentication system. There are other factors that can affect performance of a biometric system during the verification process, for example: quality and type of biometric features, user behavior, sensor quality, environmental factors, effectiveness of biometric algorithm, etc. Understanding biometric performance metrics not only helps you select the right biometric solution, it also gives you a chance to understand the challenges current biometric technology is facing. Despite the current challenges to biometric system performance, studies are underway to address shortcomings of these systems.