Content of review 1, reviewed on April 02, 2025

Concerning data and code: the minimum requirements are almost met, however, I suggest several improvements:

  • The data should be made available through a data repository (e.g. Zenodo), so that it gets an associated version, is immutable and gets an assigned DOI. Ideally, a hash of the data is shared in the paper so that others can verify that what is downloaded is the same artifact. A public google drive link is not guaranteed to be available forever nor is it immutable.

  • The code relies on many dependencies (torch, opencv, python, ...) but the versions of these dependencies are not shared. Since the environment in which the code was ran is not shared, it means that the code is by definition not reproducible. I highly recommend using tools like uv or pixi to create a reproducible environment, and share the pyproject.toml and lock files in the repository. Sharing a container image would also be an option to guarantee reproducibility.

  • the code quality is in general quite poor. The scripts contain lots of code that is commented out, hard-coded absolute file paths, non-english comments. Hard-coded absolute file paths mean that it is impossible to run the scripts on another machine without modifying them. It is also hard to know what code that is commented out is relevant to reproduce a result. The code is repetitive, not modular and difficult to read. While this is not a hard requirement, it means that re-use of the code is nearly impossible, which is a pity. I highly recommend the authors make some effort to clean up, properly document the code, and refactor it such that it is reproducible on any system without modification to the code. Results of the code should not be committed to the repository, as is the case now.

Concerning the paper:
- The introduction mentions PCA as a dimensionality reduction technique, however it is not explored further. The key contribution of the paper seems to be automatically detecting the number of clusters based on the image quality metrics. I'm wondering whether PCA decomposition may not give very comparable results.
- NMF + clustering/segmentation is not new in 4D-STEM. It has been implemented in Pyxem and Hyperspy, see https://pyxem.readthedocs.io/en/stable/tutorials/pyxem-demos/06%20Nanocrystal%20segmentation%20in%20SPED%20data%20-%20Demonstration%20on%20partly%20overlapping%20MgO%20cubes.html#3.-NMF-Based-Segmentation and https://hyperspy.org/hyperspy-doc/current/user_guide/mva/decomposition.html#non-negative-matrix-factorization-nmf respectively. I'm wondering why the authors go through the trouble of re-implementing from scratch what could be a one-liner.

Source

    © 2025 the Reviewer.

Content of review 2, reviewed on July 01, 2025

The authors have sufficiently addressed the data and code concerns in this revision.

This reviewer highly appreciates the effort put into cleaning up the code for readability and documenting with relevant docstrings. There are still hard-coded paths in the scripts, meaning that the code is not re-usable as-is, but a motivated reader/user has a starting point. For future work, I highly recommend making the effort to integrate your algorithm/solution into an existing package like HyperSpy, Pyxem, Py4dstem, ... This will make it much easier for others to install your code and use your method on their own data, which will increase citations on your work.

A small correction that still needs to be made: when referring to where data can be obtained (beginning and end of the text), please cite the doi associated with the Zenodo entry. The text still refers to google drive and github.

Source

    © 2025 the Reviewer.

Content of review 3, reviewed on September 19, 2025

As stated last time, I believe the minimum requirements for code and data availability were met. However, I have an additional recommendation for a small change. Also there appears to be a regression compared to the last version.

Firstly, a small change: the link provided to the data on Zenodo is a direct URL, which is not guaranteed to stay functional forever. Highly recommended is to instead use the corresponding DOI link: https://doi.org/10.5281/zenodo.15492699, which provides greater guarantees of remaining the same and accessible forever.

Secondly, the regression: it seems the link to the code has been changed to another repository that is not publicly accessible. The original repository https://github.com/KIRA9359/00_nmf_analsysis_4dstem/tree/main is still available. I recommend linking to the original repository, and perhaps mentioning you are refactoring your code in another package/repository which will become available at a later date. Even better is to use Zenodo's Github integration to store versioned archives of your code and link to these in the paper using the DOI. See the following documentation: https://help.zenodo.org/docs/github/

Source

    © 2025 the Reviewer.

References

    Junhao, C., Nicolas, F., Gozde, O., Edgar, R., Stavros, N., Pratim, D. P., Arnaud, D. 2025. Unsupervised multi-clustering and decision-making strategies for 4D-STEM orientation mapping. Digital Discovery.