Content of review 1, reviewed on August 29, 2022

the author presented an interesting paper about a tool to obtain luminance and color measurement with consumer hardware. Overall the manuscript is well written, and the introduction highlights the gap the tool is filling in a great way. The provided supplementary materials are helpful in helping setting up the environment and trying the tool. The authors also provided a template for reporting measurement that I think it’s nice to have and a good addition to the manuscript.
I only have one big critique and two smaller concerns, that I detailed below.

The bigger problem I see with this is in terms of Licensing. The authors reused others’ code, such as from Argyll CMS, and published it on Github. Because of the original License of Argyll, the portion of codes drawn and edited from the package should follow the same License which is AGPL v.3 (https://www.argyllcms.com/commercialuse.html). Additionally, a file edited from Mcalibrator2 is present, and the original package is released under a BSD License as indicated in https://github.com/hiroshiban/Mcalibrator2#-license-. I believe that before being published, the article needs to address the licensing of PsyCalibrator, and consider what to do. Especially because of what indicated in the summary section (““This will facilitate the community to build on it and make adaptations for richer functionality and applicability, such as support for other platforms (e.g., Python)”) knowing to which extent adaptation and re-release of the code is possible is necessary.

For what concerns the usage of Psycalibrator, the abstract indicates “supporting Windows, macOS, and Linux (Ubuntu) systems”. I have tried on a non-Ubuntu distribution and although I don’t have a spyder device, so I am not 100% sure about it, the package seems to work. I would suggest removing the name Ubuntu from the abstract. Maybe It would be more meaningful to indicate that “the package works in distributions that support either Matlab or GNU Octave (Windows, Mac Os, Linux).

For what concerns the step-by-step tutorial, it is not clear whether the Mcalibrator2 mentioned in Step 6 refers to the modified file included in Psycalibrator, in which case the sentence would benefit from rephrasing, or if users are supposed to download the full package, in which case I would suggest to add it in the list of requirements in Step 1.

Lastly, I have tested what I could test without the device in MATLAB R2019b and R2022a. I would suggest the authors to indicate in which version they performed the analysis reported in the tutorial and also the analysis reported in the validation (I am not sure the same R2016 used for generating the visual stimuli is the same used for the analysis). This would help for reproducibility and also to avoid issues due to the presence/absence of certain methods in older or future versions of Matlab and Octave.

Overall, I think the paper is great and it deserves to be published. It is definitely a great addition to have for both psychological experiments, but I can also imagined it being used in different fields to test instruments and setup. My suggestion for the editor is to accept the paper for publication after edits, and especially clarifications on the licensing of the package, are reported in both the manuscript and in the repository.

Source

    © 2022 the Reviewer.

Content of review 2, reviewed on November 01, 2022

I am grateful to the authors for taking mine and the other reviewer's concern into consideration. I think the manuscript is now acceptable for publication. Also, the GitHub repository looks appropriate now, with the licensing and the bug tracking.

Source

    © 2022 the Reviewer.

References

    Zhicheng, L., Qi, M., Yang, Z. 2023. PsyCalibrator: An Open-Source Package for Display Gamma Calibration and Luminance and Color Measurement. Advances in Methods and Practices in Psychological Science.