Face Detection and Recognition

ErrorCode Trueface::SDK::detectFaces(std::vector<FaceBoxAndLandmarks> &faceBoxAndLandmarks)

Detect all the faces in the image. This method has a small false positive rate. To reduce the false positive rate to near zero, filter out faces with score lower than 0.90. Alternatively, you can use the Trueface::FaceDetectionFilter configuration option to filter the detected faces.

The face detector has a detection scale range of about 5 octaves. ConfigurationOptions::smallestFaceHeight determines the lower of the detection scale range. E.g., setting ConfigurationOptions::smallestFaceHeight to 40 pixels yields the detection scale range of ~40 pixels to 1280 (=40x2^5) pixels.

Parameters

faceBoxAndLandmarks – [out] a vector of FaceBoxAndLandmarks representing each of the detected faces. If not faces are found, the vector will be empty. The detected faces are sorted in order of descending face score.

Returns

error code, see ErrorCode.

The recall and precision of the face detection algorithm on the WIDER FACE dataset:

../_images/face_detection_roc.png

The effect of face height on similarity score:

../_images/face_height_match_score_FULL_model.png ../_images/face_height_match_score_LITE_model.png
ErrorCode Trueface::SDK::detectLargestFace(FaceBoxAndLandmarks &faceBoxAndLandmarks, bool &found)

Detect the largest face in the image. This method has a small false positive rate. To reduce the false positive rate to near zero, filter out faces with score lower than 0.90. Alternatively, you can use the Trueface::FaceDetectionFilter configuration option to filter the detected faces. See detectFaces() for the detection scale range.

Parameters

  • faceBoxAndLandmarks – [out] the FaceBoxAndLandmarks containing the landmarks and bounding box of the largest detected face in the image.

  • found – [out] whether a face was found in the image.

Returns

error code, see ErrorCode.

ErrorCode Trueface::SDK::getFaceLandmarks(const FaceBoxAndLandmarks &faceBoxAndLandmarks, std::array<Point<int>, 106> &landmarks)

Obtain the 106 face landmarks.

Parameters
Returns

error code, see ErrorCode.

The order of the face landmarks:

../_images/landmarks.png
ErrorCode Trueface::SDK::extractAlignedFace(const FaceBoxAndLandmarks &faceBoxAndLandmarks, uint8_t *faceImage, int marginLeft = 0, int marginTop = 0, int marginRight = 0, int marginBottom = 0, float scale = 1.0)

Align the the detected face to be optimized for passing to feature extraction. If using the face chip with Trueface algorithms (ex face recognition), do not change the default margin and scale values.

Parameters

  • faceBoxAndLandmarks – [in] the FaceBoxAndLandmarks returned by detectLargestFace() or detectFaces().

  • faceImage – [out] the pointer to a uint8_t buffer of 112x112x3 = 37632 bytes (when using default margins and scale). The aligned face image is stored in this buffer. The memory must be allocated by the user. If using non-default margin and scale (again, non-standard face chip sizes will not work with Trueface algorithms), the faceImage will be of size: width = int((112+marginLeft+marginRight)*scale), height = int((112+marginTop+marginBottom)*scale), and therefore the buffer size is computed as: width * height * 3

  • marginLeft – [in] adds a margin to the left side of the face chip.

  • marginTop – [in] adds a margin to the top side of the face chip.

  • marginRight – [in] adds a margin to the right side of the face chip.

  • marginBottom – [in] adds a margin to the bottom side of the face chip.

  • scale – [in] changes the scale of the face chip.

Returns

error code, see ErrorCode.

void Trueface::SDK::saveFaceImage(uint8_t *faceImage, const std::string &filepath, unsigned int height = 112, unsigned int width = 112)

Store the face image in JPEG file. If face image has non-standard size (112x112), must specify width and height.

Parameters

  • faceImage – [in] the extracted image by extractAlignedFace().

  • filepath – [in] relative or absolute file path without a file extension.

  • height – [in] the image height, computed as int((112+marginTop+marginBottom)*scale).

  • width – [in] the image width, computed as int((112+marginLeft+marginRight)*scale).

ErrorCode Trueface::SDK::getFaceFeatureVector(const FaceBoxAndLandmarks &faceBoxAndLandmarks, Faceprint &faceprint)

Extract the face feature vector from the face box.

Parameters
Returns

error code, see ErrorCode.

ErrorCode Trueface::SDK::getFaceFeatureVector(uint8_t *alignedFaceImage, Faceprint &faceprint)

Extract the face feature vector from an aligned face image.

Parameters

  • alignedFaceImage – [in] buffer returned by extractAlignedFace(). Face image must be have size of 112x112 pixels (default extractAlignedFace() margin and scale values).

  • faceprint – [out] a Faceprint object which will contain the face feature vector.

Returns

error code, see ErrorCode.

ErrorCode Trueface::SDK::getFaceFeatureVectors(std::vector<uint8_t*> &alignedFaceImages, std::vector<Faceprint> &faceprints)

Extract the face feature vectors from the aligned face images. This batch processing method increases throughput on platforms such as cuda.

Parameters

  • alignedFaceImages – [in] vector of aligned face image buffers.

  • faceprints – [out] vector of Faceprint object which will contain the face feature vectors.

Returns

error code, see ErrorCode.

ErrorCode Trueface::SDK::getLargestFaceFeatureVector(Faceprint &faceprint, bool &foundFace)

Detect the largest face in the image and return its feature vector.

Parameters

  • faceprint – [out] a Faceprint object which will contain the face feature vector.

  • foundFace – [out] indicates if a face was detected in the image. If no face was detected, then the faceprint will be empty.

Returns

error code, see ErrorCode.

static std::string Trueface::SDK::faceprintToJson(const Faceprint &faceprint)

Convert a Faceprint into a json string.

Parameters

faceprint – [in] The Faceprint to stringify.

Returns

The json string representation of the Faceprint.

static ErrorCode Trueface::SDK::jsonToFaceprint(const std::string &jsonStr, Faceprint &faceprint)

Populate a Faceprint from a json string.

Parameters

  • jsonStr – [in] The json string representation of a Faceprint, generated from the faceprintToJson() function.

  • faceprint – [out] The Faceprint which will be populated from the json string.

Returns

error code, see ErrorCode.

static ErrorCode Trueface::SDK::getSimilarity(const Faceprint &faceprint1, const Faceprint &faceprint2, float &matchProbability, float &similarityMeasure)

Compute the similarity between two feature vectors, or how similar two faces are.

See

getLargestFaceFeatureVector

Parameters

  • faceprint1 – [in] the first Faceprint to be compared.

  • faceprint2 – [in] the second Faceprint to be compared.

  • matchProbability – [out] the probability the two face feature vectors are a match.

  • similarityMeasure – [out] the computed similarity measure.

Returns

error code, see ErrorCode.

ErrorCode Trueface::SDK::estimateFaceImageQuality(uint8_t *alignedFaceImage, float &quality)

Estimate the quality of the face image for recognition.

Parameters

  • alignedFaceImage – [in] The array returned by extractAlignedFace().

  • quality – [out] a value between 0 to 1, 1 being prefect quality for recognition. We suggest using a threshold of 0.999 as a filter for enrollment images.

Returns

error code, see ErrorCode.

ErrorCode Trueface::SDK::estimateHeadOrientation(const FaceBoxAndLandmarks &faceBoxAndLandmarks, float &yaw, float &pitch, float &roll)

Estimate the head pose.

Parameters

  • faceBoxAndLandmarks – [in] FaceBoxAndLandmarks returned by detectFaces() or detectLargestFace().

  • yaw – [out] the rotation angle around the image’s vertical axis, in radians.

  • pitch – [out] the rotation angle around the image’s transverse axis, in radians.

  • roll – [out] the rotation angle around the image’s longitudinal axis, in radians.

Returns

error code, see ErrorCode.

The accuracy of this method is estimated using 1920x1080 pixel test images. A test image:

../_images/yaw_positive_20.jpg

The accuracy of the head orientation estimation:

../_images/yaw_estimation_accuracy.png

The effect of the face yaw angle on match similarity can be seen in the following figure:

../_images/yaw_vs_sim_score.png

The effect of the face pitch angle on match similarity can be seen in the following figure:

../_images/pitch_vs_sim_score.png
struct Trueface::FaceBoxAndLandmarks

Public Members

Point<float> topLeft

The top left corner Point of the bounding box

Point<float> bottomRight

The bottom right corner Point of the bounding box

std::vector<Point<float>> landmarks

A vector of Point representing the facial landmarks: left eye, right eye, nose, left mouth corner, right mouth corner

float score

Likelihood of this being a true positive; a value lower than 0.85 indicates a high chance of being a false positive

struct Trueface::Faceprint

Face feature vector and meta data

Public Members

std::vector<float> featureVector

Vector of floats which describe the face

std::string sdkVersion

SDK version used to generate feature vector

std::string modelName

Name of model used to generate feature vector

ModelOptions modelOptions

Additional options used when generating the feature vector

struct Trueface::ModelOptions

Options used when generating the feature vector

Public Members

bool frVectorCompression = false

Indicates if the Trueface::ConfigurationOptions.frVectorCompression was enabled when generating the template

template<typename T>
struct Trueface::Point

Public Members

T x

Coordinate along the horizontal axis, or pixel column

T y

Coordinate along the vertical axis, or pixel row