I will start by saying I read this paper with great interest, but I am unsure I am the appropriate person to comment on the proofs or “math”-ness of the paper. I understood the arguments but in general do not do a lot of that type of work.

While I think this work is worthwhile and should be examined closely by those who like to model, I find this type of paper to be difficult to fully realize in real life. It assumes that all things modeling, data generation, and likewise are perfect representations of researcher reality. However, these things rarely are – just as you note in one of your first pages, that heterogeneity exists. You do show that openness is important for some of these components, but clearly, we don’t often know what others don’t often know about our work. I could write what I perceive to be a perfectly clear script and description of my model (Ma) but it could be misinterpreted or misunderstood by others reading of my work. Further, I doubt many researchers, including myself could likely provide the data generating information required to do this work. I agree with your assessment that we are likely “not in crisis” so much as we have an opportunity or “revolution” to be more informative about our work so as to situate it for interpretation. What do I gain by having these formulas and descriptions? I’m not sure I could create a likelihood of replication or if it’s even useful because of the substantial heterogeneity in every point of the research process that just cannot be controlled or explained. I appreciate that you tried to keep the example simple but I’m not positive it helped me interpret much since most models are rarely reduceable to simple proportion estimates.

I read the comments to my reviews and the other reviewer. I appreciate the care they took to clarify and strengthen the manuscript.