Content of review 1, reviewed on November 13, 2014

Basic reporting

This paper presents an open-source implementation of a convergent
cross mapping (CCM) - a previously-published method for causal
analysis of time series data. The authors discuss an implementation of
this algorithm as a Julia package, along with a presentation of some
of the key parameters and how they might best be adjusted. An example
of the impact of these parameters, and of the length of the time
series, on the analysis of the relationship between two
micro-organisms is presented.

This topic seems to be of interesting, and the CauseMap
implementation might prove very useful. Unfortunately, the
presentation of this work leaves much to be desired. There was no
discussion of the underlying algorithm -readers are simply referred to
the original paper for details. The description of the parameters was
particularly opaque, as we are told only that "E is related to the
dimensionality of the full causal system". The discussion of the
predictive skill does not define the method that is used, and the
example provided in Figure 1 does not demonstrate the inference of
causality that is presumably the point of the example provided.

I have no doubt that the implementation might be of real value, but a
clearer presentation is needed to make this paper more compelling.

Experimental design

The experimental design seems to be based on the evaluation of the
values of the parameters and of the impact of the length of the time
series. These analyses seem useful in terms of tuning the algorithm,
but there is nothing in the way of validation of the causality
algorithm itself.

Validity of the findings

Although the discussion of the values of the parameters and of the
effect of the time series seems plausible enough, the conclusions that
might be draw from the example in Figure 1 are unclear, leaving the
validity in question.

Source

    © 2014 the Reviewer (CC-BY 4.0 - source).

Content of review 2, reviewed on February 01, 2015

Basic reporting

Although this version of the paper is much improved, I feel that this revision lags in terms of clarity and detail - as Reviewer 2 said, more detail would be needed to understand the utility and applicability of the tool.

Specifically, the expanded description of the E parameter, no guidance is given regarding the value of Emax. How should this parameter be chosen?

I am also curious about the lengths of relevant time series - this paper talks in terms of 25 time points as "relatively long" , but many time series (for example, EKG) are much longer - aside from expected scaling issues, does the algorithm still work?

Experimental design

The evaluation seems reasonable, if a bit preliminary. I was frankly confused by much of the discussion of the parameter values. I was also left unclear as to the limitations of the algorithm - can CauseMap be applied to time series with multiple potential causal relationships, or is it simply a comparison of two inputs?

Validity of the findings

I see no particular questions regarding the validity.

Comments for the author

Given that the purpose of this paper is to present an implementation of a potentially useful algorithm, I certainly do not expect the authors to provide either a full description of all of the details, or a defense of all of the shortcomings. However, the paper as presented does not describe the parametrization clearly, and the limits and the strengths of the algorithm are not clearly presented. Clarification of these issues would greatly increase the value of this paper.

Source

    © 2015 the Reviewer (CC-BY 4.0 - source).