Face Detection and Recognition

ErrorCode Trueface::SDK::detectFaces(std::vector<FaceBoxAndLandmarks> &faceBoxAndLandmarks)

Detect all the faces in the image. This method has a small false positive rate. To reduce the false positive rate to near zero, filter out faces with score lower than 0.90.

The face detector has a detection scale range of about 5 octaves. ConfigurationOptions::smallestFaceHeight determines the lower of the detection scale range. E.g., setting ConfigurationOptions::smallestFaceHeight to 40 pixels yields the detection scale range of ~40 pixels to 1280 (=40x2^5) pixels.

Return

error code, see ErrorCode. structure.

Parameters

• [out] faceBoxAndLandmarks: a vector of bounding box and landmarks.

The recall and precision of the face detection algorithm on the WIDER FACE dataset:

The effect of face height on similarity score:

ErrorCode Trueface::SDK::detectLargestFace(FaceBoxAndLandmarks &faceBoxAndLandmarks, bool &found)

Detect the largest face in the image. See detectFaces() for the detection scale range.

Return

error code, see ErrorCode.

Parameters

• [out] faceBoxAndLandmarks: the bounding box and landmarks structure.

• [out] found: whether a face was found in the image.

ErrorCode Trueface::SDK::getFaceLandmarks(const FaceBoxAndLandmarks &faceBoxAndLandmarks, std::vector<Point<int>> &points)

Obtain the 106 face landmarks.

Return

error code, see ErrorCode.

Parameters

• [in] faceBoxAndLandmarks: FaceBoxAndLandmarks returned by detectFaces() or detectLargestFace().

• [out] points: a vector of 106 face landmark points.

The order of the face landmarks:

ErrorCode Trueface::SDK::extractAlignedFace(const FaceBoxAndLandmarks &faceBoxAndLandmarks, uint8_t *faceImage, int marginLeft = 0, int marginTop = 0, int marginRight = 0, int marginBottom = 0, float scale = 1.0)

Align the the detected face to be optimized for passing to feature extraction.

Return

error code, see ErrorCode.

Parameters

• [in] faceBoxAndLandmarks: the FaceBoxAndLandmarks returned by detectLargestFace().

• [out] faceImage: the pointer to a uint8_t buffer of 112x112x3 (=37632) bytes. The aligned face image is stored in this buffer.

void Trueface::SDK::saveFaceImage(uint8_t *faceImage, std::string filepath)

Store the face image in JPEG file.

Parameters

• [in] faceImage: the extracted image by extractAlignedFace().

• [in] filepath: relative or absolute file path without a file extension.

ErrorCode Trueface::SDK::getFaceFeatureVector(const FaceBoxAndLandmarks &faceBoxAndLandmarks, Faceprint &faceprint)

Extract the face feature vector from the face box.

Return

error code, see ErrorCode.

Parameters

• [in] faceBoxAndLandmarks: face box returned by detectFaces() or detectLargestFace().

• [out] featureVector: vector of floats to contain the features.

ErrorCode Trueface::SDK::getFaceFeatureVector(uint8_t *alignedFaceImage, Faceprint &faceprint)

Extract the face feature vector from an aligned face image.

Return

error code, see ErrorCode.

Parameters

• [in] alignedFaceImage: buffer returned by extractAlignedFace().

• [out] faceprint: a Faceprint object which will contain the face feature vector.

ErrorCode Trueface::SDK::getFaceFeatureVectors(std::vector<uint8_t*> &alignedFaceImages, std::vector<Faceprint> &faceprints)

Extract the face feature vectors from the aligned face images. This batch processing method increases throughput on platforms such as cuda.

Return

error code, see ErrorCode.

Parameters

• [in] alignedFaceImages: vector of aligned face image buffers.

• [out] faceprints: vector of Faceprint object which will contain the face feature vectors.

ErrorCode Trueface::SDK::getLargestFaceFeatureVector(Faceprint &faceprint)

Detect the largest face in the image and return its feature vector.

Return

error code, see ErrorCode.

Parameters

• [out] faceprint: a Faceprint object which will contain the face feature vector.

std::string Trueface::SDK::faceprintToJson(const Faceprint &faceprint)

Convert a Faceprint into a json string.

Return

The json string representation of the Faceprint.

Parameters

• [in] faceprint: The Faceprint to stringify.

ErrorCode Trueface::SDK::jsonToFaceprint(const std::string &jsonStr, Faceprint &faceprint)

Populate a Faceprint from a json string.

Return

error code, see ErrorCode.

Parameters

• [in] jsonStr: The json string which contains the Faceprint data.

• [out] faceprint: The Faceprint which will be populated from the json string.

ErrorCode Trueface::SDK::getSimilarity(const Faceprint &faceprint1, const Faceprint &faceprint2, float &matchProbability, float &similarityMeasure)

Compute the similarity between two feature vectors, or how similar two faces are.

Return

error code, see ErrorCode.

See

getLargestFaceFeatureVector

Parameters

• [in] faceprint1: Faceprint of first template to be compared.

• [in] faceprint2: Faceprint of second template to be compared.

• [out] matchProbability: the probability the two face feature vectors are a match.

• [out] similarityMeasure: the computed similarity measure.

ErrorCode Trueface::SDK::estimateFaceImageQuality(uint8_t *alignedFaceImage, float &quality)

Estimate the quality of the face image for recognition.

Return

error code, see ErrorCode.

Parameters

• [in] alignedFaceImage: The array returned by extractAlignedFace().

• [out] quality: a value between 0 to 1, 1 being prefect quality for recognition.

ErrorCode Trueface::SDK::estimateHeadOrientation(const FaceBoxAndLandmarks &faceBoxAndLandmarks, float &yaw, float &pitch, float &roll)

Estimate the head pose.

Return

error code, see ErrorCode.

Parameters

• [in] faceBoxAndLandmarks: FaceBoxAndLandmarks returned by detectFaces() or detectLargestFace().

• [out] yaw: the rotation angle around the image’s vertical axis, in radians.

• [out] pitch: the rotation angle around the image’s transverse axis, in radians.

• [out] roll: the rotation angle around the image’s longitudinal axis, in radians.

The accuracy of this method is estimated using 1920x1080 pixel test images. A test image:

The accuracy of the head orientation estimation:

The effect of the face yaw angle on match similarity can be seen in the following figures:

The effect of the face pitch angle on match similarity can be seen in the following figures:

struct Trueface::FaceBoxAndLandmarks

Public Members

Point<float> topLeft

The top left corner of the bounding box

Point<float> bottomRight

The bottom right corner of the bounding box

std::vector<Point<float>> landmarks

The facial landmark points: left eye, right eye, nose, left mouth corner, right mouth corner

float score

Likelihood of this being a true positive; a value lower than 0.85 indicates a high chance of being a false positive

struct Trueface::Faceprint

Face feature vector and meta data

Public Members

std::vector<float> featureVector

Vector of floats which describe the face

std::string sdkVersion

SDK version used to generate feature vector

std::string modelName

Name of model used to generate feature vector

ModelOptions modelOptions

Additional options used when generating the feature vector

struct Trueface::ModelOptions

Options used when generating the feature vector

Public Members

bool frVectorCompression

Indicates if the frVectorCompression was enabled when generating the template

template<typename T>
struct Trueface::Point

Public Members

T x

Coordinate along the horizontal axis, or pixel column

T y

Coordinate along the vertical axis, or pixel row