Face Detection

class Facechip

Class which is used to manage a face chip image. A face chip represents and aligned and cropped face image. A Facechip can be in CPU memory or GPU memory.

Trueface::Facechip::Facechip() = default

Default constructor.

void Trueface::Facechip::saveImage(const std::string &filepath)

Save the face chip to disk.

Parameters

filepath – the filepath where the face chip should be saved, including the image extension.

ErrorCode Trueface::Facechip::loadImage(const std::string &filepath, bool gpuMemory = false, int gpuIndex = 0)

Load the face chip from disk.

Parameters

  • filepath – the filepath of the facechip to load.

  • gpuMemory – read the image into GPU memory. This should be set to true when running GPU inference.

  • gpuIndex – the GPU index.

unsigned int Trueface::Facechip::getHeight()

Get the face chip height in pixels.

Returns

Returns the face chip height in pixels.

unsigned int Trueface::Facechip::getWidth()

Get the face chip width in pixels.

Returns

Returns the face chip width in pixels.

bool Trueface::Facechip::isGPUImage()

Is the face chip in GPU memory.

Returns

Returns true if the face chip is in GPU memory.

uint8_t *Trueface::Facechip::getData()

Get the decoded image buffer.

Returns

Returns the decoded image buffer.

using Trueface::TFFacechip = std::shared_ptr<Facechip>
ErrorCode Trueface::SDK::detectFaces(const TFImage &tfImage, std::vector<FaceBoxAndLandmarks> &faceBoxAndLandmarks)

Detect all the faces in the image. This method has a small false positive rate. To reduce the false positive rate to near zero, filter out faces with score lower than 0.90. Alternatively, you can use the Trueface::FaceDetectionFilter configuration option to filter the detected faces.

The face detector has a detection scale range of about 5 octaves. ConfigurationOptions::smallestFaceHeight determines the lower of the detection scale range. E.g., setting ConfigurationOptions::smallestFaceHeight to 40 pixels yields the detection scale range of ~40 pixels to 1280 (=40x2^5) pixels.

Parameters

  • tfImage – [in] the input image returned by preprocessImage().

  • faceBoxAndLandmarks – [out] a vector of FaceBoxAndLandmarks representing each of the detected faces. If not faces are found, the vector will be empty. The detected faces are sorted in order of descending face score.

Returns

error code, see ErrorCode.

The recall and precision of the face detection algorithm on the WIDER FACE dataset:

../_images/face_detection_roc.png

The effect of face height on similarity score:

../_images/face_height_match_score_FULL_model.png ../_images/face_height_match_score_LITE_model.png
ErrorCode Trueface::SDK::detectLargestFace(const TFImage &tfImage, FaceBoxAndLandmarks &faceBoxAndLandmarks, bool &found)

Detect the largest face in the image. This method has a small false positive rate. To reduce the false positive rate to near zero, filter out faces with score lower than 0.90. Alternatively, you can use the Trueface::FaceDetectionFilter configuration option to filter the detected faces. See detectFaces() for the detection scale range.

Parameters

  • tfImage – [in] the input image returned by preprocessImage().

  • faceBoxAndLandmarks – [out] the FaceBoxAndLandmarks containing the landmarks and bounding box of the largest detected face in the image.

  • found – [out] whether a face was found in the image.

Returns

error code, see ErrorCode.

ErrorCode Trueface::SDK::getFaceLandmarks(const TFImage &tfImage, const FaceBoxAndLandmarks &faceBoxAndLandmarks, Landmarks &landmarks)

Obtain the 106 face landmarks.

Parameters
Returns

error code, see ErrorCode.

The order of the face landmarks:

../_images/landmarks.png
ErrorCode Trueface::SDK::extractAlignedFace(const TFImage &tfImage, const FaceBoxAndLandmarks &faceBoxAndLandmarks, TFFacechip &tfFacechip, int marginLeft = 0, int marginTop = 0, int marginRight = 0, int marginBottom = 0, float scale = 1.0)

Align the the detected face to be optimized for passing to feature extraction. If using the face chip with Trueface algorithms (ex face recognition), do not change the default margin and scale values.

Parameters

  • tfImage – [in] the input image returned by preprocessImage().

  • faceBoxAndLandmarks – [in] the FaceBoxAndLandmarks returned by detectLargestFace() or detectFaces().

  • tfFacechip – [out] A TFFacechip to store the output facechip.

  • marginLeft – [in] adds a margin to the left side of the face chip.

  • marginTop – [in] adds a margin to the top side of the face chip.

  • marginRight – [in] adds a margin to the right side of the face chip.

  • marginBottom – [in] adds a margin to the bottom side of the face chip.

  • scale – [in] changes the scale of the face chip.

Returns

error code, see ErrorCode.

ErrorCode Trueface::SDK::estimateHeadOrientation(const TFImage &tfImage, const FaceBoxAndLandmarks &faceBoxAndLandmarks, float &yaw, float &pitch, float &roll)

Estimate the head pose. Only works with images where the face is at least 100 pixels in height, otherwise returns ErrorCode::FACE_TOO_SMALL.

Parameters

  • tfImage – [in] the input image returned by preprocessImage().

  • faceBoxAndLandmarks – [in] FaceBoxAndLandmarks returned by detectFaces() or detectLargestFace().

  • yaw – [out] the rotation angle around the image’s vertical axis, in radians.

  • pitch – [out] the rotation angle around the image’s transverse axis, in radians.

  • roll – [out] the rotation angle around the image’s longitudinal axis, in radians.

Returns

error code, see ErrorCode.

The accuracy of this method is estimated using 1920x1080 pixel test images. A test image:

../_images/yaw_positive_20.jpg

The accuracy of the head orientation estimation:

../_images/yaw_estimation_accuracy.png

The effect of the face yaw angle on match similarity can be seen in the following figure:

../_images/yaw_vs_sim_score.png

The effect of the face pitch angle on match similarity can be seen in the following figure:

../_images/pitch_vs_sim_score.png
template<typename T>
struct Trueface::Point

Public Members

T x

Coordinate along the horizontal axis, or pixel column.

T y

Coordinate along the vertical axis, or pixel row.

struct Trueface::FaceBoxAndLandmarks

Public Members

Point<float> topLeft

The top left corner Point of the bounding box.

Point<float> bottomRight

The bottom right corner Point of the bounding box.

std::vector<Point<float>> landmarks

A vector of Point representing the facial landmarks: subject right eye, subject left eye, nose, subject right mouth corner, subject left mouth corner.

float score

Likelihood of this being a true positive; a value lower than 0.85 indicates a high chance of being a false positive.

The order of the face landmarks:

../_images/landmarks2.jpg
using Trueface::Landmarks = std::array<Point<float>, 106>

Array of 106 face points.