Content of review 1, reviewed on April 09, 2022

There is a quite large literature on sampling design, and this paper focuses on one aspect – how aggregation impacts the precision of three designs: simple random sampling, systematic and spatially balanced designs. Not surprisingly they find that simple random sampling leads to lower precision than the two other designs, but they investigate the differences for a large spectrum of densities, aggregations, and sample sizes.
I have two main concerns with the paper:
1) I am not sure I understood the way simulations were done. It seems they are done unconditional of the population realization, that is the sampling precision is assessed across (marginally) all simulated populations, not across samples conditional on a realized (simulated) population. This is not the way precision (or equivalently) variance is assessed, i.e. you want to know the precision of a sampling design given N and its spatial distribution. I understand that it can be relevant to calculate the unconditional variance but in practice it is the conditional variance which matters (i.e. you are estimating one population). At least this needs to be argued carefully.
2) A main difficulty (discussed shortly at the end of the paper) of systematic and spatially balanced design is to obtain a good variance estimator (as is well known, the variance estimator based on SRS will be biased high – which is of course directly related to the higher precision of the two other designs). Forest scientists have worked extensively on this topic (in addition to some references you provide, you have Stevens and Olsen 2003 on the spatially balanced design, and Magnussen et al. 2020 for examples with forest inventories). To me this is a critical aspect – a “precise” point estimate without a reliable estimate of its associated uncertainty is not very useful. Running the simulations but adding different variance estimators (and ideally coverage) would make this paper a valuable contribution.
One can also add that the authors consider a spatially homogenous population, and that having trends for example can affect the efficiency of different designs and in particular variance estimators (see Magnussen’s papers).

Details: l. 69 ff. The formula given here are given in terms of expectations, not in terms of design-based sampling estimators and sample values (e.g. Cochran 1977). For SRS without replacement, the variance would be (S^2/n)/(N-n), where N is the number of units in the population (eg quadrats) and n the sample size, i.e. you need to account for the proportion sampled (assuming you sample without replacement which I guess you do). Also S^2 is the unbiased estimator of the variance (i.e. in 1/(n-1)). I think you need to make differences explicit (cf my comments above).
L. 101 ff: I did recommend once to use (stratified) adaptive sampling (Shackleton et al. 2020), and the issue was not the implementation in the field, but that for a very aggregated and rare species, the estimates were very uncertain and hardly useful. Clearly, we should have had a much large sample size…

Nigel G. Yoccoz

Additional Refs:
Magnussen, S., R. E. McRoberts, J. Breidenbach, T. Nord-Larsen, G. Ståhl, L. Fehrmann, and S. Schnell. 2020. Comparison of estimators of variance for forest inventories with systematic sampling - results from artificial populations. Forest Ecosystems 7:17.
Shackleton, R. T., B. Petitpierre, M. Pajkovic, F. Dessimoz, O. Brönnimann, L. Cattin, Š. Čejková, C. A. Kull, J. Pergl, P. Pyšek, N. Yoccoz, and A. Guisan. 2020. Integrated Methods for Monitoring the Invasive Potential and Management of Heracleum mantegazzianum (giant hogweed) in Switzerland. Environmental Management 65:829-842.
Stevens Jr, D. L., and A. R. Olsen. 2003. Variance estimation for spatially balanced samples of environmental resources. Environmetrics 14:593-610.

Source

    © 2022 the Reviewer.

Content of review 2, reviewed on September 08, 2022

I apologize for the delay, but I had to read the just-published paper by Dumelle et al. in MEE, which address some of the same issues, at least with regards to comparing simple random sampling to spatially balanced designs. Given that Dumelle et al. (2022) was available online after this paper was initially submitted, the results presented here are still of interest but there should be some ways to refer to this other work.
In terms of terminology, it could help to add when you mention conditional variance that it refers also to design-based inference (see Dumelle et al. 2022 and particularly the very clear paper by Brus 2021 they cited). I liked in particular in Brus 2021 the discussion of model-assisted inference (when the population is fixed), something you could mention in the discussion in terms of improving accuracy of estimates.

Nigel G. Yoccoz

References
Brus, D. J. 2021. Statistical approaches for spatial sample survey: Persistent misconceptions and new developments. European Journal of Soil Science 72:686-703.
Dumelle, M., M. Higham, J. M. Ver Hoef, A. R. Olsen, and L. Madsen. 2022. A comparison of design-based and model-based approaches for finite population spatial sampling and inference. Methods in Ecology and Evolution 13:2018-2029.

Source

    © 2022 the Reviewer.

References

    Jan, P., Anne, C., Roger, P., Guillaume, P., Aurelien, B. 2022. Spatially balanced sampling methods are always more precise than random ones for estimating the size of aggregated populations. Methods in Ecology and Evolution.