Abstract

This paper studies a selection of eleven Norwegian journals in the humanities and social sciences and their conversion from subscription to Open Access, a move heavily incentivized by governmental mandates and Open Access policies. By investigating the journals’ visiting logs in the period 2014-2019, the study finds that a conversion to Open Access induces higher visiting numbers; all journals in the study had a significant increase which can be attributed to the conversion. Converting a journal had no spillover in terms of increased visits to previously published articles still behind the paywall in the same journals. Visits from previously subscribing Norwegian higher education institutions did not account for the increase in visits, indicating that the increase must be accounted for by visitors from other sectors. The results could be relevant for policymakers concerning the effects of strict polices targeting economically vulnerable national journals, and could further inform journal owners and editors on the effects of converting to Open Access.


Authors

Lars Wenaas

Publons users who've claimed - I am an author

No Publons users have claimed this paper.

Contributors on Publons
  • 2 reviewers
  • pre-publication peer review (FINAL ROUND)
    Decision Letter
    2021/02/09

    09-Feb-2021

    Dear Mr. Wenaas:

    Thank you for the careful revision of your manuscript entitled "Attracting new users or business as usual? A case study of converting academic subscription based journals to open access". It is a pleasure to accept your work for publication in Quantitative Science Studies.

    I would like to request you to prepare the final version of your manuscript using the checklist available at https://bit.ly/2QW3uV5. Please also sign the publication agreement, which can be downloaded from https://bit.ly/2QYuW4w. The final version of your manuscript, along with the completed checklist and the signed publication agreement, can be returned to qss@issi-society.org.

    Thank you for your contribution. On behalf of the Editors of Quantitative Science Studies, I look forward to your continued contributions to the journal.

    Best wishes,
    Dr. Ludo Waltman
    Editor, Quantitative Science Studies
    qss@issi-society.org

    Decision letter by
    Cite this decision letter
    Author Response
    2021/02/08

    Dear Editor and reviewers

    I have condensed the feedback from the reviewers and editor into a smaller list below. All revisions that are textual or minor and in my view not in need of comments, are found in the revised manuscript with tracked changes. The more substantial (structural or content-wise) revisions are commented below. Please note that there are no points raised by the referees or editor that I disagree with, so I have tried to deal with all the points accordingly. I wish to put emphasis on one change that was not asked for. It concerns the fourth plot (articles published before 2017), which has been revised. The analysis in the manuscript was based on a smaller set of data (articles published 5 years before 2017, as opposed to all articles published before 2017). This escaped my notice since the resulting plots were almost identical. Altough the first analysis was valid and with the same outcome, it was not in accordance with the text, this is now corrected.

    A final point about publishing of data. The publisher has allowed for publishing of data, but they wanted to notify all the journals in the study and ask for a “silent concent”. The journals has been given until 12th of February to respond with any concerns, so I won’t be able to publish data and scripts for the analysis until a few days after this. In case any journals raise concerns, journals will be anonymized, so I see no reason that I should not be able to publish data and scripts and link these to the article. I hope the slight delay won’t be troublesome for the publishing process, assuming the revised manuscript is accepted.

    SHORTLIST OF REVISIONS

    General point raised by the editor:
    • When reporting numerical values, please make sure to report only informative decimals.
    o Comment: I have edited numerical values to try to remedy this.

    Reviewer 1:
    • The heading division of the manuscript is a bit unconventional
    There are now 4 top-level headings before the methodology section. I think it would make sense both organisation-wise and for guiding the reader if this number was reduced.
    o "3. Research question and framing" could go under "Introduction".
    o 4. Could also be renamed to "Literature Review" so also the current "
     5. History and background of the support to Norwegian SSH" could fit under it.
    o "6. Methodology and results" currently combines both methodology description. I would argue that it would be better to cut of a "Methodology" section at 6.2.3.1 Result: The effect using number of visits, and then dedicate the rest to a "Results" section up until the "Discussion" section.

     Comment: I have restructured the headings as suggested and moved the research question and hypothesis at the end of the review. I have kept the sub-headings in the literature review, as this section is a bit long. These can of course easily be removed.
    I have also rearranged the Methodology-section and created a main heading with the results and removed the third level of sub-headings as it seemed excessive. The procedure of creating the dataset to investigate where the visits took place (HEI’s non-HEI’s) is moved up in the Methodology-section, while results are reported in the Result-section.

    • Page 6 Lines 6-8
    "This was the case with 226 Glossa, a journal in linguistics formerly known as Lingua (Rooryck, 2017) and Quantitative Science studies (QSS) a journal in the field of scientometrics formerly known as Journal of Informetrics (Waltman et al., 2020)"
    Here I don´t have a direct replacement to suggest but I think the current formulation is a bit misleading. The original journals still exists and continue to publish so writing "formerly known as" for both of the old journals is not really correct as the journals were not replaced or renamed.

    o Comment: This is an important point, I have tried to reformulate this part to make it correct.

    • Page 6 Lines

    "Of the 15 conversion scenarios elaborated in the review by Laakso et.al, the one best resembling the 233 Norwegian initiative is the scenario where the flipping strategy is a result of joining a ‘Consortium or Library Partnership Subsidy’"

    Here again I don´t have a pre-prepared slot-in replacement to suggest but I think this interpretation of the report is incorrect. Would not the "National Journal Subsidies" scenario be more fitting for the Norwegian context? The Norwegian initiative is mentioned as an example in the report. If you think this is not the case perhaps a sentence or two to explain why the consortium scenario is more suitable in your view would be useful.

    o Comment: An important point, I agree that this part is confusing. The Norwegian case of is indeed a part of the "National Journal Subsidies" scenario in the report. I suggested ‘Consortium or Library Partnership Subsidy’ as appropriate because I focused on the financial arrangement after conversion, where financing takes place partly by governmental funds directly from the ministry, partly from the Research Council Norway and partly with funds from the University library consortia. I also suspect I suffered from “tunnel-vision” when writing this part; I have now noticed that "National Journal Subsidies" is downplayed in the review as opposed to in the report.
    I have rewritten the section and included both paths to reflect this point better.

    Reviewer 2
    • Table 5, 6, 7: Please refer to as figure, move data into text or below figure to increase size of figure (better readability). Also provide caption explaining the figure, allowing it to stand for itself. Some of this can be moved from the text to the caption.

    o Comment: I have reorganized the presentations of the plots and moved most of the statistics in to the captions of the figures. The main plots are also made bigger for readability. Please also note that the fourth plot (articles published before 2017), has been revised. It contained a smaller set of data (articles published 5 years before 2017) which escaped my notice since the plots were almost identical. It is now corrected and in correspondence with the text.

    • Figure 2: The boxplot doesn't display properly. This could be an issue with PDF conversion of ggplot figures - I have previously seen the same kind of display error for my own figures.

    o Comment: I have reorganized the plot/presentation with a better and ‘cleaner’ figure and moved the information about the plot/statistics into the caption and text.

    • 4. The discussion addresses most of my concerns about limitations in a clear and honest fashion. I do however miss a discussion
    o about whether the results are influenced by other properties of the journals: they are Norwegian, they are in the social sciences and humanities, they are funded in a very specific manner. What does this mean in terms of generalisability?

    o Comment: Another good (but also somewhat difficult) point. I have tried to remedy this in the discussion.

    Best regards Lars Wenaas

    Author response by


    Cite this author response
  • pre-publication peer review (ROUND 1)
    Decision Letter
    2020/12/11

    11-Dec-2020

    Dear Mr. Wenaas:

    Your manuscript QSS-2020-0083 entitled "Attracting new users or business as usual? A case study of converting academic subscription based journals to open access", which you submitted to Quantitative Science Studies, has been reviewed. The comments of the reviewers are included at the bottom of this letter.

    I am pleased to let you know that both reviewers are positive about your manuscript. The reviewers both recommend a minor revision. Based on my own reading of your work, I agree with the recommendation made by the reviewers. I therefore would like to invite you to prepare a minor revision of your manuscript.

    As indicated by the reviewers, some of the tables in your manuscript should be referred to as figures. Furthermore, as pointed out by reviewer 2, your manuscript would benefit from fixing some language issues. However, there is no need for extensive language editing. When your manuscript has been accepted for publication in Quantitative Science Studies, the publisher will take care of further language editing. Finally, when reporting numerical values, please make sure to report only informative decimals. For instance, some numerical values in your manuscript are reported with five decimals, while there seems to be no need for more than two decimals.

    To revise your manuscript, log into https://mc.manuscriptcentral.com/qss and enter your Author Center, where you will find your manuscript title listed under "Manuscripts with Decisions." Under "Actions," click on "Create a Revision." Your manuscript number has been appended to denote a revision.

    You may also click the below link to start the revision process (or continue the process if you have already started your revision) for your manuscript. If you use the below link you will not be required to login to ScholarOne Manuscripts.

    PLEASE NOTE: This is a two-step process. After clicking on the link, you will be directed to a webpage to confirm.

    https://mc.manuscriptcentral.com/qss?URL_MASK=003d6633127a43168f8a568ffda3cbc9

    You will be unable to make your revisions on the originally submitted version of the manuscript. Instead, revise your manuscript using a word processing program and save it on your computer. Please also highlight the changes to your manuscript within the document by using the track changes mode in MS Word or by using bold or colored text.

    Once the revised manuscript is prepared, you can upload it and submit it through your Author Center.

    When submitting your revised manuscript, you will be able to respond to the comments made by the reviewers in the space provided. You can use this space to document any changes you make to the original manuscript. In order to expedite the processing of the revised manuscript, please be as specific as possible in your response to the reviewers.

    IMPORTANT: Your original files are available to you when you upload your revised manuscript. Please delete any redundant files before completing the submission.

    If possible, please try to submit your revised manuscript by 09-Feb-2021. Let me know if you need more time to revise your work.

    Once again, thank you for submitting your manuscript to Quantitative Science Studies and I look forward to receiving your revision.

    Best wishes,
    Dr. Ludo Waltman
    Editor, Quantitative Science Studies
    qss@issi-society.org

    Reviewers' Comments to Author:

    Reviewer: 1

    Comments to the Author
    The article manuscript "Attracting new users or business as usual? A case study of converting academic subscription-based journals to open access" provides an interesting a relevant study in the domain of open access impacts, more specifically if and how the number of visits to journal content are impacted by a journal converting to open access publishing.

    The introduction and literature review is comprehensive and well written, the key previous studies are included and summarized well. It is clear that there is a gap in research when it comes to understanding how change of publishing model influences visitor counts, studies so far have focused either on individual journals or open access models other than full open access publishing.

    The study has a very strong methodological design both in terms of the data used (11 journals that all converted to OA at the same time three years ago) as well as in the techniques used to analyze the data. Since the journals are so homogenous when it comes to their profiles and OA conversion it makes a lot of sense that the data for them was aggregated in order to gain more statistical power to the tests. The methodology section is also well written and detailed in its description.

    The results section is comprehensive and weaves both informative figures and interpretative text together in a very reader-friendly way. The conclusions are supported by the findings. As it stands this study is likely to become somewhat of a benchmark for future OA impact studies, as far as I know no prior studies have gone to this extent in-depth and polish to explore OA impacts to visit counts.

    The discussion section rounds out the manuscript well and brings forth many future avenues for studies in this vein. It is likely that interest in research in this area will only increase in the coming years as many countries are ramping up OA conversions of national journals, particularly in the Nordics. As such this study is well-positioned to inform decision making about the potential direct public value of OA SSH journals to society, as well as functioning as a baseline for monitoring OA conversions as they happen and reporting on them in a similar style.

    It is rare to get such a polished manuscript for review, I have no major requests for revision and can give my support for accepting it for publication with minor revisions. Below are some of the smaller things I noticed while going through the text that could be corrected at the author´s discretion.

    --- Minor suggestions for revision --

    The heading division of the manuscript is a bit unconventional and I think it would be a benefit to make it more in line with reader expectations for research articles.

    There are now 4 top-level headings before the methodology section. I think it would make sense both organisation-wise and for guiding the reader if this number was reduced. "3. Research question and framing" could go under "Introduction". 4. Could also be renamed to "Literature Review" so also the current "5. History and background of the support to Norwegian SSH" could fit under it. "6. Methodology and results" currently combines both methodology description. I would argue that it would be better to cut of a "Methodology" section at 6.2.3.1 Result: The effect using number of visits, and then dedicate the rest to a "Results" section up until the "Discussion" section.

    Page 4 Line 20

    "Studies of economic gains of open access..." -> "Studies of economic impacts of open access..."

    Gains is related to perspective, customers or sellers, something more neutral like impacts would be more suitable.

    Page 5 Line 34

    "The final report was published in 2016 which included commentaries by a panelist..." ---> "The final report was published in 2016 which included commentaries by a panel of experts..."

    The report had multiple panelists so the original formulation is incorrect

    Page 5 Line 42

    "After Plan" -> Remove

    Probably just an unfinished sentence.

    Page 6 Lines 6-8

    "This was the case with 226 Glossa, a journal in linguistics formerly known as Lingua (Rooryck, 2017) and Quantitative Science studies (QSS) a journal in the field of scientometrics formerly known as Journal of Informetrics (Waltman et al., 2020)"

    Here I don´t have a direct replacement to suggest but I think the current formulation is a bit misleading. The original journals still exists and continue to publish so writing "formerly known as" for both of the old journals is not really correct as the journals were not replaced or renamed.

    Page 6 Lines

    "Of the 15 conversion scenarios elaborated in the review by Laakso et.al, the one best resembling the 233 Norwegian initiative is the scenario where the flipping strategy is a result of joining a ‘Consortium or Library Partnership Subsidy’"

    Here again I don´t have a pre-prepared slot-in replacement to suggest but I think this interpretation of the report is incorrect. Would not the "National Journal Subsidies" scenario be more fitting for the Norwegian context? The Norwegian initiative is mentioned as an example in the report. If you think this is not the case perhaps a sentence or two to explain why the consortium scenario is more suitable in your view would be useful.

    Page 6 Line 56

    "Open Access"

    Would suggest to do a Ctrl+ F search through the manuscript to see that capitalization is consistent, now it jumps a bit back and forth.

    Page 7 Line 44

    "...in need of life support. "

    In my view not the best way to express what is "baseline funding" or "funds to secure continued operation" or something similar. The term also occurs elsewhere in the manuscript, would suggest to edit there as well.

    Page 8 Line 28

    "journal ratings"

    In my view "journal metrics" would be a more accurate term here

    The results section

    Most of the figures are named and numbered as Tables which seems a bit unconventional, my suggestion would be to treat them as figures instead

    6.4 Summary of findings

    I would suggest that this section would be presented as flowing text rather than individual bullet points.

    In several places in the manuscript

    PDF and HTML are abbreviations but are written in small caps in the manuscript, would suggest changing.

    Reviewer: 2

    Comments to the Author
    This is a peer review of the manuscript "Attracting new users or business as usual? A case study of converitng academic subscription based journals to open access" (QSS-2020-0083) by Lars Wenaas.

    The manuscript explores changes in readership to existing and new articles published by 11 Norweigan journals, after converting from a subscription-based to an open access model. The study finds increased readership for open access articles, no spill-over for older articles which remain behind paywall, and increased access from users not at a Norwegian higher education institution.

    The topic of the manuscript is of high relevance to QSS readers, and the design of the study is extraordinarily well thought out. I highly commend the author for the selection criteria they have used for constructing the data set, and the great match between research questions, analytical method and interpretation. This is a great piece of research, with a high degree of cohesion from beginning to end.

    I strongly recommend this paper for publication. I only have some minor comments for possible improvements:

    1. Generally the manuscript is well written, but there are some language glitches here and there. I would recommend following up on this.

    2. Table 5, 6, 7: Please refer to as figure, move data into text or below figure to increase size of figure (better readability). Also provide caption explaining the figure, allowing it to stand for itself. Some of this can be moved from the text to the caption.

    3. Figure 2: The boxplot doesn't display properly. This could be an issue with PDF conversion of ggplot figures - I have previously seen the same kind of display error for my own figures.

    4. The discussion addresses most of my concerns about limitations in a clear and honest fashion. I do however miss a discussion about whether the results are influenced by other properties of the journals: they are Norwegian, they are in the social sciences and humanities, they are funded in a very specific manner. What does this mean in terms of generalisability?

    Decision letter by
    Cite this decision letter
    Reviewer report
    2020/12/02

    This is a peer review of the manuscript "Attracting new users or business as usual? A case study of converitng academic subscription based journals to open access" (QSS-2020-0083) by Lars Wenaas.

    The manuscript explores changes in readership to existing and new articles published by 11 Norweigan journals, after converting from a subscription-based to an open access model. The study finds increased readership for open access articles, no spill-over for older articles which remain behind paywall, and increased access from users not at a Norwegian higher education institution.

    The topic of the manuscript is of high relevance to QSS readers, and the design of the study is extraordinarily well thought out. I highly commend the author for the selection criteria they have used for constructing the data set, and the great match between research questions, analytical method and interpretation. This is a great piece of research, with a high degree of cohesion from beginning to end.

    I strongly recommend this paper for publication. I only have some minor comments for possible improvements:

    1. Generally the manuscript is well written, but there are some language glitches here and there. I would recommend following up on this.

    2. Table 5, 6, 7: Please refer to as figure, move data into text or below figure to increase size of figure (better readability). Also provide caption explaining the figure, allowing it to stand for itself. Some of this can be moved from the text to the caption.

    3. Figure 2: The boxplot doesn't display properly. This could be an issue with PDF conversion of ggplot figures - I have previously seen the same kind of display error for my own figures.

    4. The discussion addresses most of my concerns about limitations in a clear and honest fashion. I do however miss a discussion about whether the results are influenced by other properties of the journals: they are Norwegian, they are in the social sciences and humanities, they are funded in a very specific manner. What does this mean in terms of generalisability?

    Cite this review
    Reviewer report
    2020/12/01

    The article manuscript "Attracting new users or business as usual? A case study of converting academic subscription-based journals to open access" provides an interesting a relevant study in the domain of open access impacts, more specifically if and how the number of visits to journal content are impacted by a journal converting to open access publishing.

    The introduction and literature review is comprehensive and well written, the key previous studies are included and summarized well. It is clear that there is a gap in research when it comes to understanding how change of publishing model influences visitor counts, studies so far have focused either on individual journals or open access models other than full open access publishing.

    The study has a very strong methodological design both in terms of the data used (11 journals that all converted to OA at the same time three years ago) as well as in the techniques used to analyze the data. Since the journals are so homogenous when it comes to their profiles and OA conversion it makes a lot of sense that the data for them was aggregated in order to gain more statistical power to the tests. The methodology section is also well written and detailed in its description.

    The results section is comprehensive and weaves both informative figures and interpretative text together in a very reader-friendly way. The conclusions are supported by the findings. As it stands this study is likely to become somewhat of a benchmark for future OA impact studies, as far as I know no prior studies have gone to this extent in-depth and polish to explore OA impacts to visit counts.

    The discussion section rounds out the manuscript well and brings forth many future avenues for studies in this vein. It is likely that interest in research in this area will only increase in the coming years as many countries are ramping up OA conversions of national journals, particularly in the Nordics. As such this study is well-positioned to inform decision making about the potential direct public value of OA SSH journals to society, as well as functioning as a baseline for monitoring OA conversions as they happen and reporting on them in a similar style.

    It is rare to get such a polished manuscript for review, I have no major requests for revision and can give my support for accepting it for publication with minor revisions. Below are some of the smaller things I noticed while going through the text that could be corrected at the author´s discretion.

    --- Minor suggestions for revision --

    The heading division of the manuscript is a bit unconventional and I think it would be a benefit to make it more in line with reader expectations for research articles.

    There are now 4 top-level headings before the methodology section. I think it would make sense both organisation-wise and for guiding the reader if this number was reduced. "3. Research question and framing" could go under "Introduction". 4. Could also be renamed to "Literature Review" so also the current "5. History and background of the support to Norwegian SSH" could fit under it. "6. Methodology and results" currently combines both methodology description. I would argue that it would be better to cut of a "Methodology" section at 6.2.3.1 Result: The effect using number of visits, and then dedicate the rest to a "Results" section up until the "Discussion" section.

    Page 4 Line 20

    "Studies of economic gains of open access..." -> "Studies of economic impacts of open access..."

    Gains is related to perspective, customers or sellers, something more neutral like impacts would be more suitable.

    Page 5 Line 34

    "The final report was published in 2016 which included commentaries by a panelist..." ---> "The final report was published in 2016 which included commentaries by a panel of experts..."

    The report had multiple panelists so the original formulation is incorrect

    Page 5 Line 42

    "After Plan" -> Remove

    Probably just an unfinished sentence.

    Page 6 Lines 6-8

    "This was the case with 226 Glossa, a journal in linguistics formerly known as Lingua (Rooryck, 2017) and Quantitative Science studies (QSS) a journal in the field of scientometrics formerly known as Journal of Informetrics (Waltman et al., 2020)"

    Here I don´t have a direct replacement to suggest but I think the current formulation is a bit misleading. The original journals still exists and continue to publish so writing "formerly known as" for both of the old journals is not really correct as the journals were not replaced or renamed.

    Page 6 Lines

    "Of the 15 conversion scenarios elaborated in the review by Laakso et.al, the one best resembling the 233 Norwegian initiative is the scenario where the flipping strategy is a result of joining a ‘Consortium or Library Partnership Subsidy’"

    Here again I don´t have a pre-prepared slot-in replacement to suggest but I think this interpretation of the report is incorrect. Would not the "National Journal Subsidies" scenario be more fitting for the Norwegian context? The Norwegian initiative is mentioned as an example in the report. If you think this is not the case perhaps a sentence or two to explain why the consortium scenario is more suitable in your view would be useful.

    Page 6 Line 56

    "Open Access"

    Would suggest to do a Ctrl+ F search through the manuscript to see that capitalization is consistent, now it jumps a bit back and forth.

    Page 7 Line 44

    "...in need of life support. "

    In my view not the best way to express what is "baseline funding" or "funds to secure continued operation" or something similar. The term also occurs elsewhere in the manuscript, would suggest to edit there as well.

    Page 8 Line 28

    "journal ratings"

    In my view "journal metrics" would be a more accurate term here

    The results section

    Most of the figures are named and numbered as Tables which seems a bit unconventional, my suggestion would be to treat them as figures instead

    6.4 Summary of findings

    I would suggest that this section would be presented as flowing text rather than individual bullet points.

    In several places in the manuscript

    PDF and HTML are abbreviations but are written in small caps in the manuscript, would suggest changing.

    Reviewed by
    Cite this review
All peer review content displayed here is covered by a Creative Commons CC BY 4.0 license.