Face Detection¶

SDK.detect_faces(self: tfsdk.SDK) → List[tfsdk.FaceBoxAndLandmarks]¶

Detect all the faces in the image and return the bounding boxes and facial landmarks. This method has a small false positive rate. To reduce the false positive rate to near zero, filter out faces with score lower than 0.90. Alternatively, you can use the FACEDETECTIONFILTER configuration option to filter the detected faces. The face detector has a detection scale range of about 5 octaves. tfsdk.ConfigurationOptions.smallest_face_height determines the lower of the detection scale range. E.g., setting tfsdk.ConfigurationOptions.smallest_face_height to 40 pixels yields the detection scale range of ~40 pixels to 1280 (=40x2^5) pixels.

Returns: A list of FaceBoxAndLandmarks representing each of the detected faces. If not faces are found, the list will be empty. The detected faces are sorted in order of descending face score.

The recall and precision of the face detection algorithm on the WIDER FACE dataset:

The effect of face height on similarity score:

_images/face_height_match_score_FULL_model.png

_images/face_height_match_score_LITE_model.png

SDK.detect_largest_face(self: tfsdk.SDK) → Tuple[bool, tfsdk.FaceBoxAndLandmarks]¶

Detect the largest face in the image. This method has a small false positive rate. To reduce the false positive rate to near zero, filter out faces with score lower than 0.90. Alternatively, you can use the FACEDETECTIONFILTER configuration option to filter the detected faces. See tfsdk.SDK.detect_faces() for detection range.

Returns: A bool indicating if a face was detected and the corresponding tfsdk.FaceBoxAndLandmarks, in that order.

SDK.get_face_landmarks(self: tfsdk.SDK, face_box_and_landmarks: tfsdk.FaceBoxAndLandmarks) → Tuple[tfsdk.ERRORCODE, List[tfsdk.Point[106]]]¶

Obtain the 106 face landmarks.

Parameters: face_box_and_landmarks - tfsdk.FaceBoxAndLandmarks returned by tfsdk.SDK.detect_faces() or tfsdk.SDK.detect_largest_face().
Returns: The tfsdk.ERRORCODE and list of the 106 face landmark points, returned in that order.

Obtain the 106 face landmarks.

The order of the face landmarks:

SDK.extract_aligned_face(*args, **kwargs)¶

Overloaded function.

extract_aligned_face(self: tfsdk.SDK, face_box_and_landmarks: tfsdk.FaceBoxAndLandmarks, margin_left: int = 0, margin_top: int = 0, margin_right: int = 0, margin_bottom: int = 0, scale: float = 1.0) -> numpy.ndarray[numpy.uint8]
Extract the aligned face chip in a Numpy array. Changing the margins and scale will change the face chip size. If using the face chip with Trueface algorithms (ex face recognition), do not change the default margin and scale values.
Parameters
face_box_and_landmarks - the tfsdk.FaceBoxAndLandmarks returned by tfsdk.SDK.detect_largest_face() or tfsdk.SDK.detect_faces().

margin_left - adds a margin to the left side of the face chip (default = 0).

margin_top - adds a margin to the top side of the face chip (default = 0).

margin_right - adds a margin to the right side of the face chip (default = 0).

margin_bottom - adds a margin to the bottom side of the face chip (default = 0).

scale - changes the scale of the face chip (default = 1).
Returns

Returns a numpy array containing the face chip.
extract_aligned_face(self: tfsdk.SDK, buffer_pointer: int, face_box_and_landmarks: tfsdk.FaceBoxAndLandmarks, margin_left: int = 0, margin_top: int = 0, margin_right: int = 0, margin_bottom: int = 0, scale: float = 1.0) -> tfsdk.ERRORCODE
Extract the aligned face chip in a Numpy array. Changing the margins and scale will change the face chip size. If using the face chip with Trueface algorithms (ex face recognition), do not change the default margin and scale values. This function override requires the caller to allocate the memory required for the face chip. The buffer size can be computed as follows: width = int((112+margin_left+margin_right)*scale), height = int((112+margin_top+margin_bottom)*scale), and therefore the buffer size is computed as: width * height * 3
Parameters
buffer_pointer - a buffer allocated by the caller which the face chip will be written to.

face_box_and_landmarks - the tfsdk.FaceBoxAndLandmarks returned by tfsdk.SDK.detect_largest_face() or tfsdk.SDK.detect_faces().

margin_left - adds a margin to the left side of the face chip (default = 0).

margin_top - adds a margin to the top side of the face chip (default = 0).

margin_right - adds a margin to the right side of the face chip (default = 0).

margin_bottom - adds a margin to the bottom side of the face chip (default = 0).

scale - changes the scale of the face chip (default = 1).
Returns

The tfsdk.ERRORCODE.

SDK.save_face_image(self: tfsdk.SDK, face_image: numpy.ndarray[numpy.uint8], filepath: str, height: int = 112, width: int = 112) → None¶

Store the face image in JPEG file. If face image has non-standard size (112x112), must specify width and height.

Parameters

face_image - the face chip as a numpy array, obtained from tfsdk.SDK.extract_aligned_face().
filepath - relative or absolute file path without a file extension.
height - the image height, computed as int((112+margin_top+margin_bottom)*scale).
width - the image width, computed as int((112+margin_left+margin_right)*scale).

Returns

Error code, see ERRORCODE

SDK.estimate_head_orientation(self: tfsdk.SDK, face_box_and_landmarks: tfsdk.FaceBoxAndLandmarks) → Tuple[tfsdk.ERRORCODE, float, float, float]¶

Estimate the head orientation using the detected facial landmarks.

Parameters: face_box_and_landmarks - the tfsdk.FaceBoxAndLandmarks returned by tfsdk.SDK.detect_largest_face() or tfsdk.SDK.detect_faces().
Returns: The ERRORCODE, yaw, pitch, roll, in that order. Angles are in radians.

The accuracy of this method is estimated using 1920x1080 pixel test images. A test image:

The accuracy of the head orientation estimation:

The effect of the face yaw angle on match similarity can be seen in the following figure:

The effect of the face pitch angle on match similarity can be seen in the following figure:

class tfsdk.Point¶

property x¶: Coordinate along the horizontal axis, or pixel column.

property y¶: Coordinate along the vertical axis, or pixel row.

class tfsdk.FaceBoxAndLandmarks¶

property bottom_right¶: The bottom-right corner Point of the bounding box.

property landmarks¶: The list of facial landmark points (Point) in this order: left eye, right eye, nose, left mouth corner, right mouth corner.

property score¶: Likelihood of this being a true positive; a value lower than 0.85 indicates a high chance of being a false positive.

property top_left¶: The top-left corner Point of the bounding box.