Accuracy by Dataset
Principle evaluation was performed over four datasets briefly described below. Further details can be found in the NIST report.
-
Visa Images were collected at United States consular facilities in Mexico. They have pixel dimensions 252x300 with a mean interocular distance of 69 pixels. Most are live captures although some are photographs of printed photographs. These images were used in NIST’s Performance of Automated Age Estimation Algorithms study.
-
Mugshots were collected post-arrest by law enforcement in the United States. They have reasonable compliance with the ANSI/NIST ITL 1-2011 Type 10 standard’s subject acquisition profile levels 10 - 20 for frontal images. The images are JPEG compressed and vary in pixel dimensions, with 480x640 being the most common.
-
Application Images were collected during interviews at immigration offices in the United States. The images are 300x300 pixels and have a uniform (typically white) background. Eyeglasses were removed prior to capture. The images are generally conformant to the ISO/IEC 19794-5 Face Image Data standard.
-
Border Crossing Images were acquired using a webcam operated by an immigration officer. The images are 240x300 with a mean interocular distance of 38 pixels. They frequently suffer from quality-related problems such as off-angle head orientations and poor frontal illumination.
Table Summary
The table below summarizes the performance of the age estimation algorithm. It measures accuracy using the following metrics.
-
Mean Absolute Error (MAE): The average (absolute) difference between the estimated age and the actual age. Lower values are more desirable.
-
Acc(x): The proportion of face images for which the estimated age is within x years of the actual age. Higher values are more desirable.
-
FPR(x, y): The proportion of x year-olds whose ages are estimated to be at least y years old. When x < y this is a false positive rate.
-
FNR(x, y): The proportion of x year-olds who are estimated to be less than y years old. When x > y this is a false negative rate.
-
Failure to Process (FTP): The proportion of images for which an age estimate could not be calculated.
For more detailed definitions of the error metrics see Section 2 of the NIST report.
Figures
The plots below show error rates as a function of threshold (i.e., challenge age). The grey vertical line shows the actual age (16 or 25 years).