General Comments This study is an incremental study that is based on the previously presented LOGISMOS framework. Main contributions on the existing LOGISMOS framework are computation of the graph costs with hierarchical random forest classification-based approach considering spatial context, and extension of the framework into 4D using temporal context. For most of the cartilage sub-regions improvement of the hierarchical RF classifier in segmentation accuracies is significant. However quantitative improvement of 4D segmentation over 3D segmentation for the cartilage sub-regions does not seem convincing. The results were not discussed with respect to LOGISMOS framework or other studies in the literature in terms of segmentation accuracies or computational performance. The authors need to revise their manuscript according to the issues explained in the following paragraphs. I. Introduction - Authors overlooked some recent studies on automatic segmentation of cartilage a subset of which are listed below. Some of these and the referenced studies already used longitudinal (4-D) data or random forest classifiers segment knee MR images, so please review these studies accordingly. • Vincent et al., Fully automatic segmentation of the knee joint using active appearance models • Tamez-Pena et al., Unsupervised segmentation and quantification of anatomical knee features: data from the osteoarthritis initiative • Zhang et al., Automatic knee cartilage segmentation from multi-contrast mr images using support vector machine classification with spatial dependencies • Shan et al., Automatic atlas-based three label cartilage segmentation from MR knee images • Dam et al., Automatic segmentation of high-and low-field knee MRIs using knee image quantification with data from the osteoarthritis initiative - It is not clear what is meant by simple hand-tuned cost functions in third paragraph. Because the work is based on LOGISMOS framework and a novelty of it is considered as cost function design, Section II and the general information in Section III are better to be integrated into Section I as subsections. Already II is too short to be a section. - Are not the automated cartilage segmentation methods in [3],[4], or [7] use learning based cost function design? Why do only [5],[6],[8],[9] extensively use this design? - Mentioning of how LOGISMOS segmentation provides global optimality with respect to the provided cost functions would be useful. - It needs to be stressed in what respect the RF classifiers are novel in paragraph 4. - The inclusion of the flow of the article in Section I would attract the attention of the readers. II. LOGISMOS Segmentation - This section is better to stress in a paragraph that in what ways LOGISMOS segmentation was strong or weak so that the authors proposed their incremental work based on LOGISMOS, what kind of changes were made over the framework, and how and if its weaknesses were eliminated with the proposed system. Some of the answers to these questions exist in different sections of the text, and are better to be integrated in this section to make motivations clearer. III. Cost Function Design - Fig 1 needs explanation of the sub-figures. Left one seemingly shows the cartilage probability map of femur and right one that of tibia. - How the feature values were interpolated to the search column points from corresponding feature volumes can be briefly explained. - Did k-means algorithm run only according to the coordinates of the mesh points without using any other features? How was the value of k (40) used in k-means to account for variable local anatomy appearances determined? IV. 4D Longitudinal Segmentation of Knee MRI - How was the same configuration maintained for graph parameters and topology of the different time points? Do not graph parameters of different time points need to have already different values? - It would be elegant if the authors give an example figure to misregistered surface meshes and the eliminated version of it with their 2-step registration approach. - The value of inter-time point min value is better to be given where inter-time point max value is mentioned, and please further explain the equation of Etemporal. V. Experimental Methods - How was the correlation coefficient R computed (give its equation)? Do the authors think that R is a reasonable measure to evaluate success of 4D segmentation? Since cartilage thickness tends to decrease with OA, even if segmentation results are fairly accurate, the correlation may yield low. - Could not patellar cartilage be segmented with the proposed system? OAI data enables its segmentation. VI. Results - Presegmentation of each image volume at a time point was indicated to be performed by 3D LOGISMOS. Then how long does it take to fully segment an image volume in 4D LOGISMOS? The durations of both 3D and 4D LOGISMOS would be useful along with noting the processing capability of the experimental system. - For table IV, please explicitly write that the results of which approach t-tested with those of which. What is intuitively understood is the testing of (NAF+RF vs. Gradient) and (NAF+RF vs. RF only), which is already what is expected - Unsigned error values seem to be more meaningful to evaluate the success of the approaches, and Table V is alone more meaningful than Table VI to evaluate 3-D versus 4-D. Despite of shorter time-interval comparison was made with respect to the independent standard in Table V, so it is reliable. - R values are assessed as more consistent for MF and MT for 4-D w.r.t both 3-D and (LF\LT 4D). Should not R values for LT and LF be also consistent (at high frequency) because these sub-parts are known to be less affected by OA? Depending on the results in Table VI, is it possible to make the conclusion of “4D segmentation helped improve the accuracy over multiple time-points and better captured the cartilage losses” VII. Discussions and Conclusion - Authors are expected to qualitatively evaluate their studies with respect to the previous LOGISMOS framework and other fully automated 3-D and 4-D segmentation studies in the literature in this section. Use of other metrics such as volume overlap error, dice similarity coefficient, sensitivity, or specificity would be useful for qualitative comparison, since most of other studies did not estimate signed or unsigned surface positioning errors. Appendix B - How were the proportions of 60% and 20% useful in isolation of central load-bearing regions of femoral and tibial cartilage determined? Authors are expected to make a few words on the consistency of the resulting sub-regions with the nomenclature. Sentence\expression\punctuation\numbering that seems to need corrections: - References are better to be ordered (i.e. [5],[6], [8],[9]) (Section I) - For readability reorganize the sentence that ends with “object boundaries and/or are infeasible to be computed in close-to-real time”, in multiple sentences if required. (Section I) - “JEI [14] was used [15] to substantially”, there seems a problem with referencing (Section I) - “neighborhood of information” or “information of neighborhood” (Section III paragraph 2) - some figures are not placed in the order they referenced and need to be replaced accordingly (some of them are Fig.1, Fig 4, Fig 7, Fig 8). - “Along.”, remove dot (Section IV) - “digraph [20]”, put a dot (Section IV) - “for the the entire femur”, remove one of ‘the’s, (Section V paragraph 5) - The costs by the proposed … cost function was compared against the gradient based costs, (Section V paragraph 6) - load(-)bearing, gradient(-)based etc., use of dash is better - Fig.5 shows validation work-flow or testing work-flow, (Section V paragraph. 6) - Is “cartilage border positioning errors” in Table IV different from “cartilage surface positioning errors” in Table V. If not, please use one of the expression in both tables. - hierarchal, correct (paragraph 3, Section VI) - “and its performance (was) demonstrated”, “fast (and) accurate” (Section VII) - “of the of the automated sub-plate”, remove one of “of the” (Section VII) - graph frame-work, remove dash (Appendix A) - “The JEI interaction…although under…”, correct sentence (Appendix A) - reference in order [23],[22] (Appendix B)