• pre-publication peer review (FINAL ROUND)
    Decision Letter
    2021/07/07

    07-Jul-2021

    Dear Dr. Meho:

    It is a pleasure to accept your manuscript entitled "Gender gap in highly prestigious international research awards, 2001-2020" for publication in Quantitative Science Studies.

    I would like to request you to prepare the final version of your manuscript using the checklist available at https://tinyurl.com/qsschecklist. Please also sign the publication agreement, which can be downloaded from https://tinyurl.com/qssagreement. The final version of your manuscript, along with the completed checklist and the signed publication agreement, can be returned to qss@issi-society.org.

    Thank you for your contribution. On behalf of the Editors of Quantitative Science Studies, I look forward to your continued contributions to the journal.

    Best wishes,
    Dr. Ludo Waltman
    Editor, Quantitative Science Studies
    qss@issi-society.org

    Decision letter by
    Cite this decision letter
    Author Response
    2021/07/07

    EDITOR: Reviewer 1 maintains that it is essential to consider the statistical significance of your results.
    AUTHOR REPLY: I used t-test to determine the significance of the similarity/difference between two means. I modified the related section. Please see the highlighted text in red font and yellow background on page 11 in the revised version of the manuscript).

    EDITOR: I agree with reviewer 1 that the issue of age differences between men and women has only partially been addressed.
    AUTHOR REPLY: I agree with both of you. I modified the related section. Please see the highlighted in red font and yellow background on page 11 in the revised version of the manuscript.

    EDITOR: Like reviewer 1, I want to urge you to make your data set available in a data repository. In my view, the fact that you are working on another paper based on the same data set does not justify postponing the publication of the data set. Postponing the release of your data set goes against widely accepted open science practices, and Quantitative Science Studies requests authors to support such practices as much as possible. I kindly ask you to take this into account.
    AUTHOR REPLY: I agree with both of you. I deposited the data in our institutional repository in OA format. Please see: https://scholarworks.aub.edu.lb/handle/10938/22921

    REVIEWER 1: Only the age of the recipients has been considered. The age of the faculty has to be considered to clarify to which extent this is a gender gap or an age gap.
    AUTHOR REPLY: I agree with you. I modified the related section to read as follows (highlighted text in red font and yellow background on pages 11 and 12 in the revised version of the manuscript). I also added Table 2 with some relevant data.
    To determine whether the gender gap in awards may be explained by age gap between men and women professors, we examined their average age during 2001-2020. Because we do not have access to the average age of men and women professors in the U.S., we used those of Canada as a proxy for the world (Statistics Canada, 2021). Results showed that the difference in the average age between men and women professors has decreased from 1.8 years during 2001-2005 to only 0.6 years during 2016-2020. For the average age of award recipients at the time of receiving the awards, results showed fluctuations from one five-year period to another. Overall, men received their highly prestigious awards at an average age of 65.0 compared to 62.3 for women (see Table 2 for more details). These results suggest that women professors are not only, on average, younger than their male counterparts, but they receive highly prestigious awards at a younger age, too. The increase in the average age of women professors (moving from 53.8 during 2001-2005 to 57.0 during 2016-2020) may have been a factor in the increase in the number and proportion of awards they received from as we progress in time during 2001-2020; however, this suggestion could not be verified from our data. Note here that the average age of men professors also increased over time, yet, men received fewer awards as time progressed: the share of awards by men declined from 89% during 2011-2015 to 81% during 2016-2020 and the count of awards from 844 to 814 during the same period (see Figures 2 and 3).

    REVIEWER 1: I still think it is important to consider the ‘significance’, for instance is 11% and 15% basically the same number? Yes, or no?
    AUTHOR REPLY: I used the t-test to determine the significance of the similarity/difference between the two means. I modified the related text to read as follows (highlighted text in red font and yellow background on page 11 in the revised version of the manuscript):
    To determine whether the difference between 11% and 15% is statistically significant, we performed an independent-samples t-test. Our results showed no significant difference in the average number of awards received by the 29 internationally mobile women scientists (M=1.6, SD=2.61) compared to the 307 internationally mobile men scientists (M=1.5, SD=1.24); t=2.048, p=0.756. These results suggest that international mobility has a negligible impact on the number of awards received by elite scientists. We further verified this outcome by comparing the population of internationally mobile award recipients with non-internationally mobile award recipients. Here, too, our results showed no significant difference in the average number of awards received by the 336 internationally mobile scientists (M=1.50, SD=1.40) versus the 1,937 non-internationally mobile scientists (M=1.5, SD=1.21); t=1.965, p=0.767 (see also Netz, Hampel, & Aman, 2020).

    REVIEWER 1: The lack of clearly indicating the magnitude of the numbers lying behind the calculated percentages could be misleading. E.g., are the discussed fluctuations in Computer Science simply due to a low number of awards and/or temporally fluctuating number of female senior faculty members?
    AUTHOR REPLY: To address your comment, I changed the figure and the associated text entirely in the previously revised version of the manuscript. Despite the changes, you claimed that the point is still not clear. Again, I agree, especially for computer science. Accordingly, for this revision, I modified the text again regarding computer science to read as follows (highlighted text in red font and yellow background on page 7 in the revised version of the manuscript):
    Unlike all other fields, where increases in the number of awards received by women were in line with increases in the number of women professors, in computer science, women’s share of awards fluctuated markedly in the past 20 years even though: (1) the number of women professors has increased from 140 in 2001-2005 to 600 in 2016-2020; (2) the proportion of women professors increased from 12% to 20% during the same period; and (2) the number of available awards in the field remained relatively stable from 2001 to 2015 or increased considerably during 2016-2020. Women received a single computer science award during 2001-2005, followed by 8 during 2006-2010, then one award again during 2011-2015 when there was an average of 47 awards (+/-3) available in each of these five-year periods. These results suggest that the fluctuation in the number and proportion of awards received by women in computer science is a result of variables other than the count and proportion of women professors and the number of available awards in the field.

    REVIEWER 1: I recommend that the editor insists that the data is published publicly available on a data repository, prior to publication.
    AUTHOR REPLY: I agree with you. I deposited the data in our institutional repository in OA format. Please see: https://scholarworks.aub.edu.lb/handle/10938/22921

    Author response by


    Cite this author response
  • pre-publication peer review (ROUND 2)
    Decision Letter
    2021/07/05

    05-Jul-2021

    Dear Dr. Meho:

    Your manuscript QSS-2021-0002.R1 entitled "Gender gap in highly prestigious international research awards, 2001-2020", which you submitted to Quantitative Science Studies, has been reviewed. The comments of the reviewers are included at the bottom of this letter.

    Reviewer 1 (previously reviewer 2) recommends another revision of your manuscript, while reviewer 2 (previously reviewer 1) recommends acceptance of your work. I would like to invite you to prepare a second revised version of your manuscript in which the remaining comments of reviewer 1 are taken into consideration. More specifically, I have the following requests:

    1. Reviewer 1 maintains that it is essential to consider the statistical significance of your results. Given the ongoing debates about the pros and cons of statistical significance testing, I personally do not find this essential. I leave the decision to you. You may consider providing confidence intervals for some of the most important comparisons. You could for instance provide a confidence interval for the difference between 11% and 15% that is mentioned by the reviewer.

    2. I agree with reviewer 1 that the issue of age differences between men and women has only partially been addressed. This is an important issue. Please try to find a way to address this issue or, if you don’t have the data needed to address this issue, make sure to clearly acknowledge this as a limitation of your work.

    3. Like reviewer 1, I want to urge you to make your data set available in a data repository. In my view, the fact that you are working on another paper based on the same data set does not justify postponing the publication of the data set. Postponing the release of your data set goes against widely accepted open science practices, and Quantitative Science Studies requests authors to support such practices as much as possible. I kindly ask you to take this into account.

    To revise your manuscript, log into https://mc.manuscriptcentral.com/qss and enter your Author Center, where you will find your manuscript title listed under "Manuscripts with Decisions." Under "Actions," click on "Create a Revision." Your manuscript number has been appended to denote a revision.

    You may also click the below link to start the revision process (or continue the process if you have already started your revision) for your manuscript. If you use the below link you will not be required to login to ScholarOne Manuscripts.

    PLEASE NOTE: This is a two-step process. After clicking on the link, you will be directed to a webpage to confirm.

    https://mc.manuscriptcentral.com/qss?URL_MASK=6908e82226a043e08a5950fe2c41bd04

    You will be unable to make your revisions on the originally submitted version of the manuscript. Instead, revise your manuscript using a word processing program and save it on your computer. Please also highlight the changes to your manuscript within the document by using the track changes mode in MS Word or by using bold or colored text.

    Once the revised manuscript is prepared, you can upload it and submit it through your Author Center.

    When submitting your revised manuscript, you will be able to respond to the comments made by the reviewers in the space provided. You can use this space to document any changes you make to the original manuscript. In order to expedite the processing of the revised manuscript, please be as specific as possible in your response to the reviewers.

    IMPORTANT: Your original files are available to you when you upload your revised manuscript. Please delete any redundant files before completing the submission.

    If possible, please try to submit your revised manuscript by 03-Sep-2021. Let me know if you need more time to revise your work.

    Once again, thank you for submitting your manuscript to Quantitative Science Studies and I look forward to receiving your revision.

    Best wishes,
    Dr. Ludo Waltman
    Editor, Quantitative Science Studies
    qss@issi-society.org

    Reviewers' Comments to Author:

    Reviewer: 1

    Comments to the Author
    Please see the attached PDF file.

    Reviewer: 2

    Comments to the Author
    The author addressed all my questions. No more questions,Thanks.

    Decision letter by
    Cite this decision letter
    Reviewer report
    2021/06/24

    The author addressed all my questions. No more questions,Thanks.

    Reviewed by
    Cite this review
    Reviewer report
    2021/06/01

    This reviewer report was submitted to the journal in an attached file. Its contents are not displayed directly.

    Reviewed by
    Cite this review
    Author Response
    2021/05/31

    Dear Dr. Meho:

    Your manuscript QSS-2021-0002 entitled "Gender gap in highly prestigious international research awards, 2001-2020", which you submitted to Quantitative Science Studies, has been reviewed. The comments of the reviewers are included at the bottom of this letter.

    Based on the comments of the reviewers as well as my own reading of your manuscript, I would like to invite you to prepare a revised version of your manuscript. There are a few issues that require special attention:

    1. Reviewer 1 argues that you need to use a statistical test to demonstrate “significant similarities”. From my point of view, the use of a statistical test is not necessary. However, in order to avoid confusion, my suggestion is to consider replacing ‘significant’ by an alternative adjective.
      AUTHOR REPLY: Done.

    2. Reviewer 2 considers your conclusions to be overstated.
      AUTHOR REPLY: Fixed

    3. The issue of the age of award recipients, mentioned by reviewer 2, is an important one that requires careful consideration.
      AUTHOR REPLY: Done

    4. As suggested by reviewer 2, could you please make your data set available in a data repository.
      AUTHOR REPLY: Please see my note to Reviewer 2 in this regard.

    To revise your manuscript, log into https://mc.manuscriptcentral.com/qss and enter your Author Center, where you will find your manuscript title listed under "Manuscripts with Decisions." Under "Actions," click on "Create a Revision." Your manuscript number has been appended to denote a revision.

    You may also click the below link to start the revision process (or continue the process if you have already started your revision) for your manuscript. If you use the below link you will not be required to login to ScholarOne Manuscripts.

    PLEASE NOTE: This is a two-step process. After clicking on the link, you will be directed to a webpage to confirm.

    https://mc.manuscriptcentral.com/qss?URL_MASK=d454060fe72f447a98d8427ef1ac47d7

    You will be unable to make your revisions on the originally submitted version of the manuscript. Instead, revise your manuscript using a word processing program and save it on your computer. Please also highlight the changes to your manuscript within the document by using the track changes mode in MS Word or by using bold or colored text.

    Once the revised manuscript is prepared, you can upload it and submit it through your Author Center.

    When submitting your revised manuscript, you will be able to respond to the comments made by the reviewers in the space provided. You can use this space to document any changes you make to the original manuscript. In order to expedite the processing of the revised manuscript, please be as specific as possible in your response to the reviewers.

    IMPORTANT: Your original files are available to you when you upload your revised manuscript. Please delete any redundant files before completing the submission.

    If possible, please try to submit your revised manuscript by 14-Jul-2021. Let me know if you need more time to revise your work.

    Once again, thank you for submitting your manuscript to Quantitative Science Studies and I look forward to receiving your revision.

    Best wishes,
    Dr. Ludo Waltman
    Editor, Quantitative Science Studies
    qss@issi-society.org

    Reviewers' Comments to Author:

    Reviewer: 1
    AUTHOR REPLY: Thank you very much for all the suggestions you made and the questions you raised. I agree with all. I hope you find my edits, changes, and additions satisfactory.

    Comments to the Author

    The work by Lokman Meho explored the gender disparities in research award winners. The author checked the growth of numbers of women prizewinners by time and by research fields. The whole paper is self-structured and convincing, I have some points to improve the paper:

    REVIEWER 1: A lack of systematic literature review in the introduction, although there are few studies about research awards as the author mentioned, there are lots of work about gender gap in research, funding, collaboration and mobility (the author also mentions these in the last part), which are highly correlated with the content in this paper.
    AUTHOR REPLY: I agree with you. As a result, I revised and expanded the whole Introduction and Discussion and Conclusion sections (text in red font).

    REVIEWER 1: In the methods part, the author said, "We verified the accuracy of the field classification of each award recipient by examining their publication record in Scopus and ESI...", so how this process conducted, and what's the verified accuracy? According to my experience, not all scientists can find a unique ID in Scopus.
    AUTHOR REPLY: I added the following section to the manuscript: "We verified the accuracy of the Wikipedia field classification by examining the publication records of the award recipients in both Scopus and Essential Science Indicators (ESI) databases. These databases classify publications into 27 and 22 subject categories, respectively (e.g., chemistry, engineering, mathematics, and medicine/clinical medicine). For each award recipient, we retrieved all of their publications in the database and examined the 1-3 subject categories with the largest number of publications. Results showed over 99% match between Wikipedia field description of the award recipients and the subject categorization of awardees’ publications by Scopus and ESI. We subsequently used these two databases as follows to determine the field classification of the 127 awardees who lacked Wikipedia entries:
     If the great majority of an author’s publications were classified by Scopus or ESI under a single subject category (e.g., immunology) or under two or more related subject categories (e.g., medicine and neuroscience), we recorded this author under the broad field to which these subjects belong (in this case, biological and life sciences).
     In the cases where the great majority of an author’s publications were classified under two or more different fields (e.g., engineering and mathematics), and where Scopus and ESI differed as to which field was most representative, we searched the web for additional information on the awardees to decide which of these two fields to assign to the author.

    We used Scopus (author name search) to determine the institutional and country affiliation of all 2,273 award recipients. For each award recipient, we recorded all the institutions and countries with which the author had at least one-fourth of her/his publications and/or h-index papers. In cases where the award recipient had very few or no records in Scopus, we recorded all the institutions and countries with which the person was affiliated for more than 10 years. Overall, 52% of the award recipients had multiple affiliations or have changed affiliations during their careers (Schlagberger, Bornmann, & Bauer, 2016).
    We used professors for comparison because over 94% of the 2,273 award recipients are or were affiliated with universities and research institutions, 96% have doctoral degrees, the average age of the individuals at the time of receiving the award was 66 (well above the average age of professors, which stands at around 55 in the U.S.), and researchers receive these awards an average of 37 years after their PhDs (EUI, 2018). We used the proportion equation to determine the magnitude of the gap between men and women while taking into consideration their respective population sizes within the scientific community. We analyze data in five-year intervals (2001-2005, 2006-2010, 2011-2015, and 2016-2020) because of the small number of awards received by women per year."

    REVIEWER 1: For the comparison of research performance in Table 1, the author used "significant similarities" without any strictly statistical test. The author needs a statistical test if she/he want to show the significance.
    AUTHOR REPLY: I believe the use of a statistical test is not necessary here. However, since I agree with your comment and in order to avoid confusion, I replaced the term ‘significant’ by other adjectives throughout the mansucript.

    REVIEWER 2: Figure 1, the orange line should be indicated in the figure caption, this reference line is import for the readership.
    AUTHOR REPLY: Fixed (please see Figure 2).

    Reviewer: 2
    AUTHOR REPLY: Thank you very much for all the suggestions you made and the questions you raised. I agree with all. I hope you find my edits, changes, and additions satisfactory.

    Comments to the Author

    REVIEWER 2: The authors’ main hypothesis is that there is a gender difference in prestigious research awards. They find that there is a gender gap in the sense that women’s ratio of prices is lower than what the gender ratio among faculty members suggests. They furthermore look for differences among the awardees on the basis of gender and conclude that only the rate of mobility is different.
    AUTHOR REPLY: This is no longer the case as can be determined from the revised version of the manuscript.

    REVIEWER 2: I acknowledge the work that has been put into producing the dataset. I am sure that this comprise a valuable data set to investigate potential racial and gender biases among the awards. However, I think the analysis is vague and the conclusions are overstated. For instance, from the abstract (p.2 line 27): “The study concludes that the gender gap in highly prestigious research awards is a result of a number of factors, including implicit biases and a lack of proactive efforts to address inequities within the larger scientific community.” In my reading, the analysis does not consider “implicit biases” nor “proactive efforts”!
    AUTHOR REPLY: I agree with you. As a result, I revised and expanded the whole Discussion and Conclusion section.

    REVIEWER 2: I recommend that the editor insists that the data is published publicly available on a data repository, prior to publication.
    AUTHOR REPLY: Because I am working on another manuscript using the same data, I will provide the data upon request (as noted in the manuscript). I promise to place the data in a repository and make them easily discoverable as soon as I am done with the other manuscript.

    REVIEWER 2: The introduction section lacks state-of-the-art within the literature on the subject instead they claim (p. 3 line 21): “Quantitative studies of gender disparities in highly prestigious research awards are very few and have been shaped primarily by anecdotal reports and highly localized, monodisciplinary, and dated studies”. This reviewer can think of several fairly quantitative studies, so I suggest rewriting the introduction.
    AUTHOR REPLY: I agree with you. As a result, I revised and expanded the whole Introduction section.

    REVIEWER 2: The comparison of awardees on p. 3 line 18 shows no gender differences. This is actually very interesting; I think this a point to consider further. Does the Matheus effect affect women and men in the same way?
    AUTHOR REPLY: I revised and expanded the text to read as follows: "Our data show that in both cases awards are concentrated among a small group of elite researchers: 23% of women and 25% of men received 50% of all awards in their respective gender categories (Ma & Uzzi, 2018). Moreover, among women, 23% received more than one award, 12% received more than two awards, and 6% received more than three awards during 2001-2020 (Appendix B); among the men, the ratios were 26%, 11%, and 6%, respectively (Appendix C), suggesting great similarities between both groups with regard to potentials of the presence of a Matthew Effect (Azoulay, Stuart, & Wang, 2014; Chan, Gleeson, & Torgler, 2014; Merton, 1968)."

    REVIEWER 2: The following sentence on p. 5 line 49: “…our data showed that this significant increase was largely a result of six prizes in which women received 23 (or 50%) of all awards given in 2016- 2020 compared to only 13 (or 16% ) of all awards given during the previous 15 years”. This is super interesting, what is different for these prices? Do they have a clear policy of gender parity?
    AUTHOR REPLY: Unfortunately, I could not find any information that could help address your question. However, I changed the text to read as follows: (highlighted in red font): ". . . this sudden substantial increase was largely a result of five awards that women received 19 times (out of a total of 37) during 2016-2020 in comparison to seven times (out of a total of 75) during the previous 15 years: the Berggruen Prize for Philosophy and Culture (first awarded in 2016), Dan David Prize (2002-), Holberg Prize (2004-), John von Neumann Award (2001-), and the John W. Kluge Prize for Achievement in the Study of Humanity (2003-). Apart from the Berggruen Prize which started in 2016 and conferred its prize to women three out of five times (through 2020), we could not find any information on the websites of the other four awards or in the literature that could explain the reasons for the shift in the number and percentage of awards going towards women.

    REVIEWER 2: Note on p. 7 line 39: Comment about women affiliated with institutions located in the US received twice as many as their counterpart. What information does this give the reader? Maybe more prizes in general are more often given to US researchers, which seems to be the case judging from figure 5.
    AUTHOR REPLY: Fixed. I changed the text to read as follows (highlighted in red font): "Geographically, from 2001 to 2020, women affiliated with institutions located in the U.S. received nearly twice as many awards (n=272) as their counterparts in other high-income countries (n=140) and considerably more than the rest of the world (n=15) (Figure 4). During these 20 years, women in the U.S. always received a higher proportion of all awards received by scientists affiliated with institutions located in the country compared to women in other high-income countries; however, women in the latter group substantially increased their share of all awards in their respective countries during 2016-2020 bringing them closer to achieving parity with their women colleagues in the U.S. (Heinze, Jappe, & Pithan, 2019). Although women in the rest of the world enjoyed a better success rate during 2016-2020 compared to women in the U.S. and other high-income countries (21% vs. 18% and 17%, respectively), they received only nine of the 187 awards given to women worldwide during these five years. Overall, as mentioned earlier, women everywhere continue to face the challenge of achieving parity with men that is comparable with their numbers within the elite scientific community."

    REVIEWER 2: In general, all along the manuscript, the authors give percentages without indicating any data set sizes, N. If dealing with small numbers, does stochasticity matter here?
    AUTHOR REPLY: Fixed throughout the manuscript.

    REVIEWER 2: The authors do not discuss the age of award recipients nor the age distribution among the faculty members. However, this is needed to provide the claimed “much-needed empirical evidence of the extent of the gap”. Let me illustrate, if the women faculty age distribution is skewed a lot towards younger women (more renewal among women) compared to male faculty (old males) and the average age of recipient is high (as you could imagine for very prestigious prices), then the gender gap is explained by an age gap. For the Nobel Prize has been thoroughly investigated by both Bjørk and Lunnemann and co-workers.
    AUTHOR REPLY: The age of women award recipients was very close to those of men with a three-year difference. The number of women award winners is too small in several fields to allow making accurate comparisons. However, I agree with you that age is too important to be excluded. I, therefore, added the following statements on page 11 (last paragraph before the Discussion Section): "We also compared the average age of men and women award recipients upon receiving their PhDs and their average age at the time they received the awards during 2001-2020. On average, women award recipients in this study have completed their PhD degrees at the age of 28.1 compared to 27.6 for men and women received their highly prestigious awards at an average age of 62.3 compared to 65.0 for men. Whether the nearly three years in age difference between women and women have an impact on the gender gap in awards could not be determined from our data. In three years, however, women award recipients would have, on average, published 14 more articles, increased their citation count, and, ultimately, had more time to receive more visibility and recognition."

    REVIEWER 2: Table 1. This data seems to reinforce the result from p. 3 line 18 shows, i.e., almost no gender differences between the few that made it to the top of pops. The mobility, however, is different but please elaborate on whether this difference is significant or not.
    AUTHOR REPLY: I changed and expanded the text (pp. 10-11) to the following: "The only notable difference between men and women award recipients was in international mobility (proportion of authors changing their primary affiliation from one institution in one to another institution in another country). Results showed that only 11% of the women award recipients had changed their primary affiliations from one country to another compared to 15% among men. Our data, however, showed negligible impact of international mobility on award winning; on average, the 336 internationally mobile award recipients received 1.49 awards per scientist compared to 1.52 awards among the 1,937 non-internationally mobile scientists (Netz, Hampel, & Aman, 2020)."

    REVIEWER 2: The discussion is very normative, and it is unclear to this reviewer how the conclusions, e.g. about “implicit biases” not “proactive efforts”, follow from the present analysis.
    AUTHOR REPLY: All discussion and conclusions now are limited to those supported by the findings of the study (please see text marked in red font).

    REVIEWER 2: Figure 1. Missing legend for all three graph objects. Caption should have a clear description of what is shown. What is the dashed line? It should be stated clearly that this is a linear fit. Finally, the figure has a confusing use of bar chart and line. Rather stick to markers or bars instead of line. Typo in caption: Change “Percent of awards…” to “Percentage of awards…”
    AUTHOR REPLY: Fixed (please see Figure 2)

    REVIEWER 2: Figure 2. The difference between this figure and figure 1 is not clear. It should be clearly stated, how the numbers are calculated. Caption should change “Percent” to “Percentage”
    AUTHOR REPLY: Fixed (please see Figure 1)

    REVIEWER 2: Figure 3. The lack of clearly indicating the magnitude of the numbers lying behind the calculated percentages could be misleading. E.g., are the discussed fluctuations in Computer Science simply due to a low number of awards and/or temporally fluctuating number of female senior faculty members?
    AUTHOR REPLY: Fixed (please see Figure 3)

    REVIEWER 2: Figure 4. The grey and orange lines are entirely confusing, both using different scales as the bar plots and not helping in emphasizing the point of disproportions between awards and faculty member. For the bar plot, it would be helpful to know what numbers lie behind these fractions. How big an uncertainty can we expect? Also, a more descriptive captions would be helpful.
    AUTHOR REPLY: Fixed (please see Figure 3)

    REVIEWER 2: In figure 5, the gray bar for years 2001-2005 is missing.
    AUTHOR REPLY: Fixed (please see Figure 4).

    Author response by


    Cite this author response
  • pre-publication peer review (ROUND 1)
    Decision Letter
    2021/03/16

    16-Mar-2021

    Dear Dr. Meho:

    Your manuscript QSS-2021-0002 entitled "Gender gap in highly prestigious international research awards, 2001-2020", which you submitted to Quantitative Science Studies, has been reviewed. The comments of the reviewers are included at the bottom of this letter.

    Based on the comments of the reviewers as well as my own reading of your manuscript, I would like to invite you to prepare a revised version of your manuscript. There are a few issues that require special attention:

    1. Reviewer 1 argues that you need to use a statistical test to demonstrate “significant similarities”. From my point of view, the use of a statistical test is not necessary. However, in order to avoid confusion, my suggestion is to consider replacing ‘significant’ by an alternative adjective.

    2. Reviewer 2 considers your conclusions to be overstated. Please make sure that all conclusions are fully supported by the findings of your research.

    3. The issue of the age of award recipients, mentioned by reviewer 2, is an important one that requires careful consideration.

    4. As suggested by reviewer 2, could you please make your data set available in a data repository.

    To revise your manuscript, log into https://mc.manuscriptcentral.com/qss and enter your Author Center, where you will find your manuscript title listed under "Manuscripts with Decisions." Under "Actions," click on "Create a Revision." Your manuscript number has been appended to denote a revision.

    You may also click the below link to start the revision process (or continue the process if you have already started your revision) for your manuscript. If you use the below link you will not be required to login to ScholarOne Manuscripts.

    PLEASE NOTE: This is a two-step process. After clicking on the link, you will be directed to a webpage to confirm.

    https://mc.manuscriptcentral.com/qss?URL_MASK=d454060fe72f447a98d8427ef1ac47d7

    You will be unable to make your revisions on the originally submitted version of the manuscript. Instead, revise your manuscript using a word processing program and save it on your computer. Please also highlight the changes to your manuscript within the document by using the track changes mode in MS Word or by using bold or colored text.

    Once the revised manuscript is prepared, you can upload it and submit it through your Author Center.

    When submitting your revised manuscript, you will be able to respond to the comments made by the reviewers in the space provided. You can use this space to document any changes you make to the original manuscript. In order to expedite the processing of the revised manuscript, please be as specific as possible in your response to the reviewers.

    IMPORTANT: Your original files are available to you when you upload your revised manuscript. Please delete any redundant files before completing the submission.

    If possible, please try to submit your revised manuscript by 14-Jul-2021. Let me know if you need more time to revise your work.

    Once again, thank you for submitting your manuscript to Quantitative Science Studies and I look forward to receiving your revision.

    Best wishes,
    Dr. Ludo Waltman
    Editor, Quantitative Science Studies
    qss@issi-society.org

    Reviewers' Comments to Author:

    Reviewer: 1

    Comments to the Author

    The work by Lokman Meho explored the gender disparities in research award winners. The author checked the growth of numbers of women prizewinners by time and by research fields. The whole paper is self-structured and convincing, I have some points to improve the paper:

    1. A lack of systematic literature review in the introduction, although there are few studies about research awards as the author mentioned, there are lots of work about gender gap in research, funding, collaboration and mobility (the author also mentions these in the last part), which are highly correlated with the content in this paper.

    2. In the methods part, the author said, "We verified the accuracy of the field classification of each award recipient by examining their publication record in Scopus and ESI...", so how this process conducted, and what's the verified accuracy? According to my experience, not all scientists can find a unique ID in Scopus.

    3. For the comparison of research performance in Table 1, the author used "significant similarities" without any strictly statistical test. The author needs a statistical test if she/he want to show the significance.

    Minor:
    Figure 1, the orange line should be indicated in the figure caption, this reference line is import for the readership.

    Reviewer: 2

    Comments to the Author

    The authors’ main hypothesis is that there is a gender difference in prestigious research awards. They find that there is a gender gap in the sense that women’s ratio of prices is lower than what the gender ratio among faculty members suggests. They furthermore look for differences among the awardees on the basis of gender and conclude that only the rate of mobility is different.

    I acknowledge the work that has been put into producing the dataset. I am sure that this comprise a valuable data set to investigate potential racial and gender biases among the awards. However, I think the analysis is vague and the conclusions are overstated. For instance, from the abstract (p.2 line 27): “The study concludes that the gender gap in highly prestigious research awards is a result of a number of factors, including implicit biases and a lack of proactive efforts to address inequities within the larger scientific community.” In my reading, the analysis does not consider “implicit biases” nor “proactive efforts”!

    I recommend that the editor insists that the data is published publicly available on a data repository, prior to publication.

    Major issues

    The introduction section lacks state-of-the-art within the literature on the subject instead they claim (p. 3 line 21): “Quantitative studies of gender disparities in highly prestigious research awards are very few and have been shaped primarily by anecdotal reports and highly localized, monodisciplinary, and dated studies”. This reviewer can think of several fairly quantitative studies, so I suggest rewriting the introduction.

    The comparison of awardees on p. 3 line 18 shows no gender differences. This is actually very interesting; I think this a point to consider further. Does the Matheus effect affect women and men in the same way?

    The following sentence on p. 5 line 49: “…our data showed that this significant increase was largely a result of six prizes in which women received 23 (or 50%) of all awards given in 2016- 2020 compared to only 13 (or 16% ) of all awards given during the previous 15 years”. This is super interesting, what is different for these prices? Do they have a clear policy of gender parity?

    Note on p. 7 line 39: Comment about women affiliated with institutions located in the US received twice as many as their counterpart. What information does this give the reader? Maybe more prizes in general are more often given to US researchers, which seems to be the case judging from figure 5.

    In general, all along the manuscript, the authors give percentages without indicating any data set sizes, N. If dealing with small numbers, does stochasticity matter here?

    The authors do not discuss the age of award recipients nor the age distribution among the faculty members. However, this is needed to provide the claimed “much-needed empirical evidence of the extent of the gap”. Let me illustrate, if the women faculty age distribution is skewed a lot towards younger women (more renewal among women) compared to male faculty (old males) and the average age of recipient is high (as you could imagine for very prestigious prices), then the gender gap is explained by an age gap. For the Nobel Prize has been thoroughly investigated by both Bjørk and Lunnemann and co-workers.

    Table 1. This data seems to reinforce the result from p. 3 line 18 shows, i.e., almost no gender differences between the few that made it to the top of pops. The mobility, however, is different but please elaborate on whether this difference is significant or not.

    The discussion is very normative, and it is unclear to this reviewer how the conclusions, e.g. about “implicit biases” not “proactive efforts”, follow from the present analysis.

    Major issues, figures

    Figure 1. Missing legend for all three graph objects. Caption should have a clear description of what is shown. What is the dashed line? It should be stated clearly that this is a linear fit. Finally, the figure has a confusing use of bar chart and line. Rather stick to markers or bars instead of line. Typo in caption: Change “Percent of awards…” to “Percentage of awards…”

    Figure 2. The difference between this figure and figure 1 is not clear. It should be clearly stated, how the numbers are calculated. Caption should change “Percent” to “Percentage”

    Figure 3. The lack of clearly indicating the magnitude of the numbers lying behind the calculated percentages could be misleading. E.g., are the discussed fluctuations in Computer Science simply due to a low number of awards and/or temporally fluctuating number of female senior faculty members?

    Figure 4. The grey and orange lines are entirely confusing, both using different scales as the bar plots and not helping in emphasizing the point of disproportions between awards and faculty member. For the bar plot, it would be helpful to know what numbers lie behind these fractions. How big an uncertainty can we expect? Also, a more descriptive captions would be helpful.

    In figure 5, the gray bar for years 2001-2005 is missing.

    Decision letter by
    Cite this decision letter
    Reviewer report
    2021/02/17

    The authors’ main hypothesis is that there is a gender difference in prestigious research awards. They find that there is a gender gap in the sense that women’s ratio of prices is lower than what the gender ratio among faculty members suggests. They furthermore look for differences among the awardees on the basis of gender and conclude that only the rate of mobility is different.

    I acknowledge the work that has been put into producing the dataset. I am sure that this comprise a valuable data set to investigate potential racial and gender biases among the awards. However, I think the analysis is vague and the conclusions are overstated. For instance, from the abstract (p.2 line 27): “The study concludes that the gender gap in highly prestigious research awards is a result of a number of factors, including implicit biases and a lack of proactive efforts to address inequities within the larger scientific community.” In my reading, the analysis does not consider “implicit biases” nor “proactive efforts”!

    I recommend that the editor insists that the data is published publicly available on a data repository, prior to publication.

    Major issues

    The introduction section lacks state-of-the-art within the literature on the subject instead they claim (p. 3 line 21): “Quantitative studies of gender disparities in highly prestigious research awards are very few and have been shaped primarily by anecdotal reports and highly localized, monodisciplinary, and dated studies”. This reviewer can think of several fairly quantitative studies, so I suggest rewriting the introduction.

    The comparison of awardees on p. 3 line 18 shows no gender differences. This is actually very interesting; I think this a point to consider further. Does the Matheus effect affect women and men in the same way?

    The following sentence on p. 5 line 49: “…our data showed that this significant increase was largely a result of six prizes in which women received 23 (or 50%) of all awards given in 2016- 2020 compared to only 13 (or 16% ) of all awards given during the previous 15 years”. This is super interesting, what is different for these prices? Do they have a clear policy of gender parity?

    Note on p. 7 line 39: Comment about women affiliated with institutions located in the US received twice as many as their counterpart. What information does this give the reader? Maybe more prizes in general are more often given to US researchers, which seems to be the case judging from figure 5.

    In general, all along the manuscript, the authors give percentages without indicating any data set sizes, N. If dealing with small numbers, does stochasticity matter here?

    The authors do not discuss the age of award recipients nor the age distribution among the faculty members. However, this is needed to provide the claimed “much-needed empirical evidence of the extent of the gap”. Let me illustrate, if the women faculty age distribution is skewed a lot towards younger women (more renewal among women) compared to male faculty (old males) and the average age of recipient is high (as you could imagine for very prestigious prices), then the gender gap is explained by an age gap. For the Nobel Prize has been thoroughly investigated by both Bjørk and Lunnemann and co-workers.

    Table 1. This data seems to reinforce the result from p. 3 line 18 shows, i.e., almost no gender differences between the few that made it to the top of pops. The mobility, however, is different but please elaborate on whether this difference is significant or not.

    The discussion is very normative, and it is unclear to this reviewer how the conclusions, e.g. about “implicit biases” not “proactive efforts”, follow from the present analysis.

    Major issues, figures

    Figure 1. Missing legend for all three graph objects. Caption should have a clear description of what is shown. What is the dashed line? It should be stated clearly that this is a linear fit. Finally, the figure has a confusing use of bar chart and line. Rather stick to markers or bars instead of line. Typo in caption: Change “Percent of awards…” to “Percentage of awards…”

    Figure 2. The difference between this figure and figure 1 is not clear. It should be clearly stated, how the numbers are calculated. Caption should change “Percent” to “Percentage”

    Figure 3. The lack of clearly indicating the magnitude of the numbers lying behind the calculated percentages could be misleading. E.g., are the discussed fluctuations in Computer Science simply due to a low number of awards and/or temporally fluctuating number of female senior faculty members?

    Figure 4. The grey and orange lines are entirely confusing, both using different scales as the bar plots and not helping in emphasizing the point of disproportions between awards and faculty member. For the bar plot, it would be helpful to know what numbers lie behind these fractions. How big an uncertainty can we expect? Also, a more descriptive captions would be helpful.

    In figure 5, the gray bar for years 2001-2005 is missing.

    Reviewed by
    Cite this review
    Reviewer report
    2021/02/01

    The work by Lokman Meho explored the gender disparities in research award winners. The author checked the growth of numbers of women prizewinners by time and by research fields. The whole paper is self-structured and convincing, I have some points to improve the paper:

    1. A lack of systematic literature review in the introduction, although there are few studies about research awards as the author mentioned, there are lots of work about gender gap in research, funding, collaboration and mobility (the author also mentions these in the last part), which are highly correlated with the content in this paper.

    2. In the methods part, the author said, "We verified the accuracy of the field classification of each award recipient by examining their publication record in Scopus and ESI...", so how this process conducted, and what's the verified accuracy? According to my experience, not all scientists can find a unique ID in Scopus.

    3. For the comparison of research performance in Table 1, the author used "significant similarities" without any strictly statistical test. The author needs a statistical test if she/he want to show the significance.

    Minor:
    Figure 1, the orange line should be indicated in the figure caption, this reference line is import for the readership.

    Reviewed by
    Cite this review
All peer review content displayed here is covered by a Creative Commons CC BY 4.0 license.