Content of review 1, reviewed on January 15, 2024

The comments to the authors are all in an attached file which I uploaded. Please contact me via the editor if the file is not readable or if clarifications are needed. I hope you choose to resubmit and that your research goes will.

Source

    © 2024 the Reviewer.

Content of review 2, reviewed on March 21, 2024

Overview
I thank the authors for taking a positive view of my suggestions; I was worried that they would feel that I was unrealistic and hypercritical. I am very impressed with what they have done in the time and endorse their new clarity of focus on research environments for biomedical cohort studies. This is still a major topic and they are right to draw attention to the strategic importance of improving those environments. It is crucial that the appropriate decision makers take this on board, particularly those who shape funding and those who lead organisations housing or supporting such research. They paper is a first step, but the challenge of changing their behaviour to improve the support for and prioritisation of multi-disciplinary, multi-cultural, multi-national collaboration require a campaign (not a ‘monograph’). I hope they and RS will launch such a campaign. I now strongly support the paper, Inevitably, as I read and re-read it, I have a few suggestions, but these should not impede or delay publication.
Paper’s story
1 Introduction
Improved substantially to define the focus and explain the role of club theory.
2 Methods
Some exploration of further papers, but still not an explicit statement of the tool used to aid thematic review, and to count frequency, once the themes have emerged from the text.
3 Results
Little changed apart from promise of available tabulation of result data once publication has happened.
4 Model development
This now reads well, interpreted in the tighter scope.
5 Discussion
Revised version reads well, but maybe it would be good to flag more explicitly what should be done next. I’d expect this in two parts: (10 studies to refine scope, increase precision and to validate or refute the value of providing research environments with the properties you espouse; (2) wide collaboration to build a community and develop a strategy for delivering such environments. Maybe you could say you enjoy collaborating in this?
Conceptual issues
Scope
No clearly and appropriately specified.
All other conceptual issues consequently disappear for this paper.
You have made some revisions in referencing research repositories.

Diversity and evolution
You allude to it being uncertain how long your analysis will remain valid.

The value of executable precision.
You have alluded to this when you discuss workflow use in your “trust by design” context.
Minor technical suggestions
P9 l44-58 In my experience the data was often created and managed by a PhD student or RA/junior colleague who has since moved on and the contacted author doesn’t know how to deal with the request and it becomes low priority == never done! Even when someone is willing to supply the data, many technical changes since it was saved make this too difficult. Your research environments procedures, technology and provenance records should overcome these factors, so that the apparent reluctance is a result of unexpected difficulty. In the last 15 years, many research councils, e.g. NERC and many institutions, have set up such a research infrastructure, so in theory this should be less of a problem now. However, I doubt whether it has been well enough tested by demand to make it effective. The funders and the institutions say you should use this infrastructure, but I wonder whether it rises high enough on a researcher’s agenda. Not sure what you should say or do about this, but some caveat may make sense – and commitment to address this may be part of your governance?

P6 l46 I think it would be more informative if you also showed percentage of the total in Table 1. I don’t think it would be too cluttered and I found myself estimating percentages, to understand the popularity of each topic.

P10 44 “Figure 2” looks like “Table 2” to me, but you’re not incorrect.

P11 l17 “technology” it may be new organisations, new work practices, changes of scale, etc. I prefer “methods”

P14 l29-35 What you are talking about is not “standards” but organisations helping to develop standards and to encourage adherence to those standards. Perhaps “Standards are supported elsewhere”
CSCW does not do this for the whole of computing, perhaps “computing support for collaboration”
SLAC doesn’t do it for physics, they just do it for scientific databases, but I think you could identify NIST for that https://www.nist.gov/ .
IVOA is correct.
ESFRI is all of the European research infrastructures. I would choose ELIXIR https://elixir-europe.org/ but the environmental sciences also have significant life-science data ENVRI https://envri.eu/ you might mention both of these.

P26 citation 24 Does this have a date? Does your correspondent have a role in biobank? Etc.
P26 citation 31 For a paper in preparation, perhaps you should give an institution and maybe a date – these things are hard to identify when you’re trying to revisit research.
Trivial picky points
P4 l44 “applied to” -> “covered
P5 l29.5 “review” -> “revision”
P6 l41 Hard to understand how motivation includes mental health.
P10 l60 comma after “technical”
P14 l53 200pa or 2000 cumulatively?
P18 57.5 can you really achieve “optimal” in such complex contexts? “effective”?
P19 insert “aligned” or “related” before “disciplines”
P20 l50 “collaboration” -> “collaborations”
P21 59.65 after “scientific needs” and “and available resources”?
Summary of recommendations
The paper should be published. The few suggestions above could be included.
The authors should be encouraged to take this work further.

Source

    © 2024 the Reviewer.

References

    John, G., Chris, W. 2024. We must discuss research environments. Royal Society Open Science.

Would you like to get recognition for your own reviews?
Click or tap here to register.