Face Detection and Recognition

ErrorCode Trueface::SDK::detectFaces(std::vector<FaceBoxAndLandmarks> &faceBoxAndLandmarks)

Detect all the faces in the image. This method has a small false positive rate. To reduce the false positive rate to near zero, filter out faces with score lower than 0.90.

The face detector has a detection scale range of about 5 octaves. ConfigurationOptions::smallestFaceHeight determines the lower of the detection scale range. E.g., setting ConfigurationOptions::smallestFaceHeight to 40 pixels yields the detection scale range of ~40 pixels to 1280 (=40x2^5) pixels.

Return

error code, see ErrorCode. structure.

Parameters

  • [out] faceBoxAndLandmarks: a vector of bounding box and landmarks.

The recall and precision of the face detection algorithm on the WIDER FACE dataset:

../_images/face_detection_roc.png

The effect of face height on similarity score:

../_images/face_height_match_score_FULL_model.png ../_images/face_height_match_score_LITE_model.png
ErrorCode Trueface::SDK::detectLargestFace(FaceBoxAndLandmarks &faceBoxAndLandmarks, bool &found)

Detect the largest face in the image. See detectFaces() for the detection scale range.

Return

error code, see ErrorCode.

Parameters

  • [out] faceBoxAndLandmarks: the bounding box and landmarks structure.

  • [out] found: whether a face was found in the image.

ErrorCode Trueface::SDK::getFaceLandmarks(const FaceBoxAndLandmarks &faceBoxAndLandmarks, std::vector<Point<int>> &points)

Obtain the 106 face landmarks.

Return

error code, see ErrorCode.

Parameters

The order of the face landmarks:

../_images/landmarks.png
ErrorCode Trueface::SDK::extractAlignedFace(const FaceBoxAndLandmarks &faceBoxAndLandmarks, uint8_t *faceImage, int marginLeft = 0, int marginTop = 0, int marginRight = 0, int marginBottom = 0, float scale = 1.0)

Align the the detected face to be optimized for passing to feature extraction. If using the face chip with Trueface algorithms (ex face recognition), do NOT change the default margin and scale values.

Return

error code, see ErrorCode.

Parameters

  • [in] faceBoxAndLandmarks: the FaceBoxAndLandmarks returned by detectLargestFace().

  • [out] faceImage: the pointer to a uint8_t buffer of 112x112x3 = 37632 bytes (when using default margins and scale). The aligned face image is stored in this buffer. The memory must be allocated by the user. If using non-default margin and scale (again, non-standard face chip sizes will NOT work with Trueface algorithms), the faceImage will be of size: width = int((112+marginLeft+marginRight)*scale), height = int((112+marginTop+marginBottom)*scale), and therefore the buffer size is computed as: width * height * 3

  • [in] marginLeft: adds a margin to the left side of the face chip.

  • [in] marginTop: adds a margin to the top side of the face chip.

  • [in] marginRight: adds a margin to the right side of the face chip.

  • [in] marginBottom: adds a margin to the bottom side of the face chip.

  • [in] scale: changes the scale of the face chip.

void Trueface::SDK::saveFaceImage(uint8_t *faceImage, std::string filepath)

Store the face image in JPEG file.

Parameters

  • [in] faceImage: the extracted image by extractAlignedFace(). Face image must have size of 112x112 pixels (default extractAlignedFace margin and scale values).

  • [in] filepath: relative or absolute file path without a file extension.

ErrorCode Trueface::SDK::getFaceFeatureVector(const FaceBoxAndLandmarks &faceBoxAndLandmarks, Faceprint &faceprint)

Extract the face feature vector from the face box.

Return

error code, see ErrorCode.

Parameters

ErrorCode Trueface::SDK::getFaceFeatureVector(uint8_t *alignedFaceImage, Faceprint &faceprint)

Extract the face feature vector from an aligned face image.

Return

error code, see ErrorCode. Note, if no face is detected in the image, then ErrorCode::NO_FACE_IN_FRAME will be returned.

Parameters

  • [in] alignedFaceImage: buffer returned by extractAlignedFace(). Face image must be have size of 112x112 pixels (default extractAlignedFace margin and scale values).

  • [out] faceprint: a Faceprint object which will contain the face feature vector.

ErrorCode Trueface::SDK::getFaceFeatureVectors(std::vector<uint8_t*> &alignedFaceImages, std::vector<Faceprint> &faceprints)

Extract the face feature vectors from the aligned face images. This batch processing method increases throughput on platforms such as cuda.

Return

error code, see ErrorCode.

Parameters

  • [in] alignedFaceImages: vector of aligned face image buffers.

  • [out] faceprints: vector of Faceprint object which will contain the face feature vectors.

ErrorCode Trueface::SDK::getLargestFaceFeatureVector(Faceprint &faceprint, bool &foundFace)

Detect the largest face in the image and return its feature vector.

Return

error code, see ErrorCode.

Parameters

  • [out] faceprint: a Faceprint object which will contain the face feature vector.

  • [out] foundFace: Indicates if a face was detected in the image. If no face was detected, then the faceprint will be empty.

std::string Trueface::SDK::faceprintToJson(const Faceprint &faceprint)

Convert a Faceprint into a json string.

Return

The json string representation of the Faceprint.

Parameters

  • [in] faceprint: The Faceprint to stringify.

ErrorCode Trueface::SDK::jsonToFaceprint(const std::string &jsonStr, Faceprint &faceprint)

Populate a Faceprint from a json string.

Return

error code, see ErrorCode.

Parameters

  • [in] jsonStr: The json string which contains the Faceprint data.

  • [out] faceprint: The Faceprint which will be populated from the json string.

ErrorCode Trueface::SDK::getSimilarity(const Faceprint &faceprint1, const Faceprint &faceprint2, float &matchProbability, float &similarityMeasure)

Compute the similarity between two feature vectors, or how similar two faces are.

Return

error code, see ErrorCode.

See

getLargestFaceFeatureVector

Parameters

  • [in] faceprint1: the first Faceprint to be compared.

  • [in] faceprint2: the second Faceprint to be compared.

  • [out] matchProbability: the probability the two face feature vectors are a match.

  • [out] similarityMeasure: the computed similarity measure.

ErrorCode Trueface::SDK::estimateFaceImageQuality(uint8_t *alignedFaceImage, float &quality)

Estimate the quality of the face image for recognition.

Return

error code, see ErrorCode.

Parameters

  • [in] alignedFaceImage: The array returned by extractAlignedFace().

  • [out] quality: a value between 0 to 1, 1 being prefect quality for recognition.

ErrorCode Trueface::SDK::estimateHeadOrientation(const FaceBoxAndLandmarks &faceBoxAndLandmarks, float &yaw, float &pitch, float &roll)

Estimate the head pose.

Return

error code, see ErrorCode.

Parameters

  • [in] faceBoxAndLandmarks: FaceBoxAndLandmarks returned by detectFaces() or detectLargestFace().

  • [out] yaw: the rotation angle around the image’s vertical axis, in radians.

  • [out] pitch: the rotation angle around the image’s transverse axis, in radians.

  • [out] roll: the rotation angle around the image’s longitudinal axis, in radians.

The accuracy of this method is estimated using 1920x1080 pixel test images. A test image:

../_images/yaw_positive_20.jpg

The accuracy of the head orientation estimation:

../_images/yaw_estimation_accuracy.png

The effect of the face yaw angle on match similarity can be seen in the following figure:

../_images/yaw_vs_sim_score.png

The effect of the face pitch angle on match similarity can be seen in the following figure:

../_images/pitch_vs_sim_score.png
struct Trueface::FaceBoxAndLandmarks

Public Members

Point<float> topLeft

The top left corner of the bounding box

Point<float> bottomRight

The bottom right corner of the bounding box

std::vector<Point<float>> landmarks

The facial landmark points: left eye, right eye, nose, left mouth corner, right mouth corner

float score

Likelihood of this being a true positive; a value lower than 0.85 indicates a high chance of being a false positive

struct Trueface::Faceprint

Face feature vector and meta data

Public Members

std::vector<float> featureVector

Vector of floats which describe the face

std::string sdkVersion

SDK version used to generate feature vector

std::string modelName

Name of model used to generate feature vector

ModelOptions modelOptions

Additional options used when generating the feature vector

struct Trueface::ModelOptions

Options used when generating the feature vector

Public Members

bool frVectorCompression = false

Indicates if the frVectorCompression was enabled when generating the template

template<typename T>
struct Trueface::Point

Public Members

T x

Coordinate along the horizontal axis, or pixel column

T y

Coordinate along the vertical axis, or pixel row