Content of review 1, reviewed on January 06, 2021

The paper title: Identifying Individual Facial Expressions by Deconstructing a Neural Network DOI: 10.1007/978-3-319-45886-1_28

This paper answers the question of what makes neural networks assign features to identify facial expressions (attractiveness, happiness, confidence, and intelligence) on the new test images so that they are consistent with human judgment. The study was conducted on 2,222 images using neural networks. Since most of the labeled attribute data sets are rather small, this paper retrains the two basic models (age and gender) in two modes (the dense layer and the entire network). Layer-wise Relevance Propagation (LRP) was applied to understand neural networks assessing multifaceted properties in images (age, gender, attractiveness, happiness, confidence, and intelligence). The resulting condition is that LRP can extract relevant specific features per image from convolutive neural networks. In addition, of the two models used (age and gender), the multi-class model (age) is more feature-rich than the binary model (gender).

The purpose of the study is written clearly as well as the ideas and method used. The title is short, concise, and clear. All references are relevant to the research topic. The dataset used is the 10k US faces dataset, which is clearly explained including pre-processing. The selection of the base model for transfer learning tasks is better supported by another evaluation score (such as MSE and/or MSE) than just relying on the Mean Absolute Error so that the conclusion for selecting the base model is more powerful.

Tables and most figures are relevant and clearly presented. However, figure label 2 is out of sync with the picture displayed. The middle label should be the heatmap for the age-based model and the right label for the heatmap of the gender-based model.

The statement in chapter 3.3, sentences no.3 is appropriate for dense layers only. However, for the full network, age and gender are overlapping. So that authors can consider this or add an annotation by this information. The authors might be added an explanation for confidence and intelligence attributes instead of only happiness and attractiveness in chapter 3, in order, the readers can see the features easily for both attributes well.

Overall, this study brings additional insights into recognizing emotions through facial expressions. The study also addresses a problem at the outset, namely what makes neural networks assign certain psychological attributes (attractiveness, happiness, confidence, intelligence) to new test images and their tasks according to human perception. I believe this work makes advancement in the field and may be very beneficial for future research or readers of this periodical.

Minor revisions: 1. Add another evaluation score (MSE and/or MSE) to support the conclusion for selecting the base model. 2. Figure label 2: the middle label should be the heatmap for the age-based model and the right-label for the gender-based model heatmap. 3. In chapter 3, better to add an explanation for the confidence and intelligence attributes.

Recommendations: The paper can be accepted for publication after the minor revisions

Source

    © 2021 the Reviewer.

References

    Farhad, A., Gregoire, M., Klaus-Robert, M., Wojciech, S. 2016. Identifying Individual Facial Expressions by Deconstructing a Neural Network. Lecture Notes in Computer Science.