In my previous post, I compared emotion recognition performance between OpenFace and FACET which suggested that OpenFace provides a better face detection algorithm but slightly less accurate action unit detection.
Face Detection Success
I was surprised to see that OpenFace and iMotions both outperformed Affectiva in face detection. Affectiva’s success rate of 72% meant that emotion extraction through Affectiva provided 25% less data than through OpenFace or iMotions. Although changing the parameters might improve Affectiva’s detection, the maximum frame rate I could process was 20hz but it seemed to provide worse results.
Emotion Detection Comparison
Each software provides slightly different emotion predictions but smiling is a universal feature that can be compared across all three algorithms. From here I only compare the results from Affectiva’s frame detector as it provided the better result. Here is a simple timeseries plot of smile detection for each algorithm (units are standardized for comparison).
Here we can see that Affectiva definitely seems to miss out on many timepoints at which OpenFace or FACET predicts a smile. All values are standardize for plotting but it is pretty clear that Affectiva’s algorithm is less continuous than the other algorithm (Affectiva predictions are given on a 0-100 probability scale) suggesting that Affectiva might be missing the nuanced intensity differences of smiles.
We can also zoom in on times where the predictions diverge between Affectiva and other algorithms.
In the figure above, the left plots show a case in which Affectiva detects a smile while OpenFace and FACET predict a moderate smile. This is in contrast to the plot on the right which is clearly a full blown smile that is detected by OpenFace and FACET. Affectiva, does seem to get it right but for some reason cannot continuously detect the smile surrounding the instance.
We can also look at times when Affectiva is missing detections.
In the left plot, Affectiva fails to detect a face and a smile while a moderate smile is detected by OpenFace and FACET. In the plot on the right, Affectiva once again misses out on a subtle smile that OpenFace and FACET detects.
Also see this article in Behavior Research Methods that also compares Affectiva to FACET.
If you are interested in validation of the algorithms, check out this paper by Stockli and colleagues (2017) in Behavior Research Methods. They also find that the FACET engine outperforms Affectiva’s AFFDEX.