Content of review 1, reviewed on December 23, 2021
The manuscript tests and interesting question, form the point of animal welfare. The authors put significant effort in data collection. This work would make a valuable contribution if it was performed and reported to a much higher standard. In its current form it could serve as a pilot for developing a protocol for a meta-analysis and meta-regression to answer the study questions.
It seems like the authors are aware of the methodology of meta-analysis (including systematic review), since they present PRISMA diagram in their SI. It would be beneficial if they would should follow reporting guideline, such as PRISMA-EcoEvo (O’Dea et al. 2021), and provide filled-in PRISMA-EcoEvo checklist with their manuscript.
I only focus on the major shortcomings below.
- The study is not based on pre-defined protocol/preregistration.
- Effect size used is not explained. The authors claim to use “log-odd ratios”, but this does not fit with the percentage data presented in their results and in the extracted data. Odds Ratio (OR) is usually applied to determine whether the probability of an event is the same or differs between two groups. Here, it is unclear how this would be calculated from the prevalence data (I imagine it is usually reported for a single group of animals, and there is no need to compare two groups). Please provide a formula or a clear description what “prevalence” and “log-odd ratios” mean in this context and how they should be interpreted.
- Authors claim to use novel methodology, but neither systematic review or multilevel meta-regression are novel (even when we use two types of moderators, biological and study settings). Authors did not record the sample sizes (number of animals observed) and do not use these values as weights in the analyses.
- Not clear why aquatic ungulates (e.g. captive dolphins) are not included neither in searches nor in the dataset.
- Not clear why data for different sexes were pulled together. This does not allow testing whether some of the moderators may have sex-specific effects (e.g., mating strategy). Extracted data on sex is not provided in the dataset.
- Search strategy is somewhat strange, with limited database use and patchy keyword choice. It is not clear why wildcards were not used for EBESCO, why not searching using higher taxa names (order, family) or common words (such as cattle, dolphins, etc.), and only a handful of only selected genera names. Search strategy has not been validated.
- It is hard to search comprehensively for literature on such large and diverse group of animals. It might be worth focusing on non-intensive livestock animals only, but searching more thoroughly. Or doing a separate study on livestock only, with a more comprehensive search.
- Systematic search is not really needed for finding biological characteristics of the species – some of these can be found in databases (e.g. AnAge, IUCN) and ad hoc searches are usually acceptable for searching for the remaining (with predefined strategy on which data sources to choose), ideally recording each used data source and assumptions/decisions made.
- List of included studies is missing. Ideally, a list of studies excluded at full-text screening stage should be provided in SI, with reasons for exclusion.
- Please report how many people performed each screening stage, data-extraction, and how. Especially how data integrity was assured (e.g. via independent screening/extraction, cross-checking).
- Summary of the dataset is not presented in the main text – it is important to present distribution of the data on the phylogenetic tree and across tested moderators (at least sample size for the levels of categorical moderators and distributions of points along continuous ones), to assess how balanced the dataset is or presence of outliers.
- Although main extracted data files are provided, detailed meta-data is missing (table in Supplementary Methods is not detailed enough to understand how all extracted data was coded, some columns are not explained at all or not matching manuscript text). Phylogenetic tree data file is missing.
- Associations among levels of categorical moderators and categorical versus continuous, have not been assessed. No justification is provided for not using models with multiple moderators and model selection.
- The “study” variable has a unique value for each row of the dataset, and as such does not represent separate studies – the data looks like there are multiple data points taken from the same group of animals, likely within the same study, so there is an issue of data non-independence to be dealt with and acknowledged. Currently, the analyses use each data row as an independent data point (except for phylogeny), which is incorrect given the repeated patterns clearly visible in the dataset, suggesting same cohorts of animals are present multiple times.
- There is some missing data, which should be acknowledged and considered in the analyses and interpretations of the results.
- Unfortunately, data on the location of the observations (e.g. which zoo, farm) is missing – this information should be recorded and used in analyses as random factors. No information is recorded on how the data was extracted and from where (article or dataset reference, which specific figure/table/page/raw dataset), also for biological data for each species. Please include notes on contacting authors, using raw data, pooling/subsetting, etc. – it would be extremely hard to get the same numbers and coded moderators to replicate this work.
- Figures: please plot actual data points on top of predicted distributions.
Source
© 2021 the Reviewer.
References
Kate, L., O., P. M., Leanne, P., D., M. S. 2022. Risk factors for stereotypic behaviour in captive ungulates. Proceedings of the Royal Society B: Biological Sciences.
