Reviewer: Liam Shaw, University of Oxford

The authors present an approach (or two approaches) to rank species according to their sensitivity to perturbations. They use simulated data to validate the method and then apply to empirical data. I understand the motivation, like the idea, and I found the article clear and reasonable. I cannot claim to have reviewed the copious supplementary info or every equation – although I’m glad they are included – and view my role as a peer reviewer not to go through the mathematical derivations line-by-line but to assess whether the manuscript merits publication. I therefore mostly restrict my comments to the main text.

Major comments:
My two major comments are about how the authors analyse the performance of methods.

1. Spearman’s rank correlation as a performance metric:
   p9 L250 onwards: using the rank correlation sort of ‘works’. However, it is slightly strange to me because e.g. for the 3-species community, the rho=0.5 cases should be distinct. The authors are trying to understand the performance of these ranking approaches, so I’d recommend using a metric that can actually distinguish between all the possible different rankings rather than equating different rankings. For example, in Figure 2, I would wish to penalise the second 0.5 case (going down the order of possible rankings) as ‘worse’ than the first one because of the ordering. Given that it these are permutations, why not use the permutation rank? i.e. if the observed sensitivities are defined as the sorted permutation,
   then if s1=A, s3=B, s2=C then sorted permutation is ABC. Then the ranks are ABC=0, ACB=1, BAC=2, BCA=3, CAB=4, CBA=5 (I think, excuse typos if I made them). So, as the authors have listed, but now one can distinguish the rho=0.5 cases. This also means that one can define the performance in terms of a median permutation rank difference (with 0 being perfect performance) which is more precise than e.g. ‘positive rho’. This applies to Figure 3 and Figure 4 as well. I appreciate the value of rho as a summary statistic, but maybe the patterns in e.g. Figure 4B would be clearer based on a true permutation ranking statistic. And might give a better performance than e.g. nonsignificant mean rank correlation ~0 for rocky intertidal community.

2. Tolerance of small deviations in sensitivity
   The authors focus on the ranking rather than the values of sensitivity, but: within some experimental tolerance of sensitivities who really cares if species X and Y have a tiny difference in sensitivity less than this? In particular, if one has a situation as in the rocky intertidal community where it seems one species is ‘the sensitive one’ and the others are more similar in sensitivity. I didn’t really feel the authors had engaged with this issue. It might be worth exploring this notion because in practice experimental noise is key; giving some idea of at what sensitivity margin one should just treat species as having ‘the same’ sensitivity seems important to me, rather than just talking always about ranks.

Minor comments
Data and code: I note the authors state they will be archived on Github upon acceptance. They could already be archived and available. I’d also recommend getting a doi on something like zenodo / figshare so there is a permanent fixed record, even if having a github repo is good too.

p2 L9 – the authors’ definition of sensitivity is not the same as vulnerable, so I wouldn’t be seen to claim that the definition is ‘better’ at identifying endangered species. It identifies ones sensitive to perturbation, but this can be increasing or decreasing abundance. So other information is needed to comment on endangered.

p3 L15 – ‘are under the constant threat of external perturbations, which are increasing in magnitude’ -> ‘are subject to external perturbations which are increasing in magnitude…’.
This is pedantic of me, but ‘constant threat’ is a colloquial phrase that isn’t right here. The perturbations are not constant and I suspect the probability is not constant either. I also think framing all perturbations as ‘threat’ is not necessary or correct because a) not all external perturbations need be a threat and b) the next sentence makes very clear why some of these perturbations can be extremely damaging. (I’m not denying external perturbations wreak disastrous change on ecosystems)

p13 L20 – ‘but also of its’ -> ‘but also the response of its’ [clarity]

p5 equation 2 - Definition of sensitivity. Not criticisms, but I think readers would appreciate more clarity here, given the whole article hinges on this definition.
First, this sensitivity is a kind of ‘instantaneous sensitivity’, which makes sense with the authors’ aims, but I’d clarify that. Why did the authors choose this specific definition of sensitivity? What is the sensitivity that is trying to be captured mathematically here? My edition of Strogatz (2015) has two mentions of ‘sensitivity’ (p212 and p453, both in the context of sensitivity to initial conditions). Vallejo (2019 edition) has more mentions, most also are the usual chaos phrase of ‘sensitivity to initial conditions’ excepting Chapter 2 on Lyapunov exponents. In summary, I could not understand what about this specific definition is ‘in line with the study of sensitivity to perturbations in nonlinear dynamics’ that is not the case for other possible definitions. I hope this can be clarified, perhaps with mention of relation to Lyapunov exponents which are currently not discussed in the main text apart from Box 2 (I see the multiple mentions in Supporting Information!). Not a criticism, just a point of clarification.
Second, is a function of k, the time interval chosen. In the 3-species example given, the authors use k=1 but don’t specify why this is chosen. For different values of k, couldn’t you get a different order of sensitivities? e.g. In Figure 1B, the blue lines cross at ~t+0.8 so s2=0. So if I choose this value of k, can’t I get the order of sensitivities to change from s3>s2>s1 to s3>s1>s2 ?
Third, I understand normalising by the initial average squared difference of abundances. Maybe specify that is always greater than or equal to 0 - and is not bounded? Can one make numerator arbitrarily big and denominator arbitrarily small? Are abundances normalised?
Fourth, I know the definition is motivated by non-equilibrium conditions, but how does it apply to equilibrium ones? I thought of a species which has constant abundance until time t and would continue to do so unless perturbed by a pulse p=+1. Two edge cases, one with (new) constant abundance and one where it decays to same original abundance:
___|‾‾‾‾ ___|-----___
=1 =?
For the first case, some might wish to say that the species is actually insensitive to perturbation – it is perturbed in abundance terms, but this has no effect on its dynamics afterwards i.e. value remains constant. For the second case, the choice of k to measure could produce ~1 for the second case if small enough, or =0. (Hope I’m making sense here…) This is just to point out that ‘sensitivity’ is a fuzzy word that means different things to different people, so it could be worth clarifying why the authors’ definition is the best one to connect with existing work nonlinear dynamics.

p7 L161 – ‘because J [is] evaluated at an equilibrium..’
p7 L162 – why can one assume that p(t) follows a distribution with mean zero – because it is normalised abundances and everything must cancel out? I don’t understand because in the Fig 1 example, p(t) was [7,7,7].

p11 L320 onwards – the variation of the training set proportion is interesting (S17 and S18), rather than using just one division of the data. It would be interesting (for example) to use the more recent 70% as a training set and the oldest 30% as the test set. This should work?

p13 L376 and following – sensitivity is not vulnerability. By the authors’ definition, the most sensitive species can be one that becomes much more abundant after a perturbation. This is the opposite of vulnerable. The least sensitive by this definition could actually be the most endangered one, which goes from a low abundance to a lower one very close to zero (which is sadly an absorbing boundary at a global scale). I think this is worth making clear.

Figure 1: This gets the motivation of sensitivity across well, thanks. I would put the food web diagram at the top, rather than as part of panel A, or remove it. It might be helpful to make the abundance y-axis in A and B the same scale. Sp 1 Sp 2 etc is a bit messy to connect with s1 s2, please be consistent.

Figure 2: In panel C and D, it would be interesting to connect the points in time with a line. It’s not clear whether the behaviour jumps around randomly or whether there are patterns of ‘flipping’ where you go intermediately through the values. And these probably correspond to ‘crossing’ of trajectories as I discussed for Figure 1B?

Figure 4: I really can’t tell any difference in expected sensitivity from the colouring of points in A, sorry. Is it possible to instead have a plot of the four species’ expected sensitivities over time for the test set? Ditto for the equivalent figures elsewhere (e.g. S15).
I appreciate this is real data and I’m not a tidal ecologist! But…in ~1992 there are some strange barnacle measurements that look wrong? And a few other places. Which variables should sum to 100%? Or because it’s just how much space is covered they don’t? Dumb questions I know but I suspect other readers might not be familiar with this system.

The authors responded well to the reviewer comments. Both in making changes, but also giving clearer explanations of why they had chosen particular methodologies that cleared things up for me. Thanks!