aboutsummaryrefslogtreecommitdiff
path: root/peer-review
diff options
context:
space:
mode:
authorBoud Roukema <boud@cosmo.torun.pl>2020-11-25 19:00:01 +0100
committerBoud Roukema <boud@cosmo.torun.pl>2020-11-25 19:00:01 +0100
commitab0260d370c3c4b96f0f1fd7cde0b50f0b6d7d22 (patch)
tree0c27824f3ef7ca7b364f8d52313e634531f69681 /peer-review
parent5af1406a499ed3936fea303a532015e90370c4b6 (diff)
Reviewer points 16 to 32
Copyediting of points 16 to 32 (paper.tex + peer-review/1-answer.txt) is done in this commit. TODO list: 2. paper lacking focus 9. tidy up README-hacking.md for appearance on website App B.G. similar to Figure ?? - ref missing 29. website: README-hacking.md and tutorial "on same page"
Diffstat (limited to 'peer-review')
-rw-r--r--peer-review/1-answer.txt73
1 files changed, 42 insertions, 31 deletions
diff --git a/peer-review/1-answer.txt b/peer-review/1-answer.txt
index 91cf3d8..da92bd3 100644
--- a/peer-review/1-answer.txt
+++ b/peer-review/1-answer.txt
@@ -357,7 +357,7 @@ with the earlier sets of criteria.
ANSWER: In the submitted version we had stated that "Ideally, it is
possible to precisely identify the Docker “images” that are imported with
-their checksums ...". But to be more clear and directly to the point, it
+their checksums ...". But to be more clear and go directly to the point, it
has been edited to explicity say "... to recreate an identical OS image
later".
@@ -375,11 +375,11 @@ later".
ANSWER: Thank you very much for raising this important point. We hadn't
seen other reproducibility papers mention this important point and missed
it. In the acknowledgments (where we also mention the commit hashes) we now
-explicity mention the exact CPU architecture used to build this paper:
+explicitly mention the exact CPU architecture used to build this paper:
"This project was built on an x86_64 machine with Little Endian byte-order
and address sizes 39 bits physical, 48 bits virtual.". This is because we
have already seen cases where the architecture is the same, but programs
-fail because of the byte-order.
+fail because of the byte order.
Generally, Maneage will now extract this information from the running
system during its configuration phase and provide the users with three
@@ -396,7 +396,7 @@ different LaTeX macros that they can use anywhere in their paper.
ANSWER: This has been clarified with the short extra statement "a minimal
Unix-like standard that is shared between many operating systems". We would
-have liked to explain this more, but the word-limit is very constraining.
+have liked to explain this more, but the word limit is very constraining.
------------------------------
@@ -411,7 +411,7 @@ have liked to explain this more, but the word-limit is very constraining.
with the measurement tool.
ANSWER: Thank you very much for this good point. A description of a
-possible solution to this has been added after criteria 8.
+possible solution to this has been added after criterion 8.
------------------------------
@@ -430,23 +430,25 @@ Maneage'd software on Zenodo (https://doi.org/10.5281/zenodo.3883409) and
also downloading from there.
Until recently we would directly access each software's own webpage to
-download the files, and this caused many problems like this. In other
+download the source files, and this caused frequent problems of this sort. In other
cases, we were very frustrated when a software's webpage would temporarily
-be unavailable (for maintainance reasons), this wouldn't allow us to build
-new projects.
-
-Since all the software are free, we are allowed to re-distribute them and
-Zenodo is defined for long-term archival of academic artifacts, so we
-figured that a software source code repository on Zenodo would be the most
-reliable solution. At configure time, Maneage now accesses Zenodo's DOI and
-resolves the most recent URL to automatically download any necessary
-software source code that the project needs from there.
+be unavailable (for maintenance reasons); this would be a hindrance in
+trying to build new projects.
+
+Since all the software is free-licensed, we are legally allowed to
+re-distribute it (within the conditions, such as not removing copyright
+notices) and Zenodo is defined for long-term archival of
+academic digital objects, so we decided that a software source code
+repository on Zenodo would be the most reliable solution. At configure
+time, Maneage now accesses Zenodo's DOI and resolves the most recent
+URL to automatically download any necessary software source code that
+the project needs from there.
Generally, we also keep all software in a Git repository on our own
webpage: http://git.maneage.org/tarballs-software.git/tree. Also, Maneage
users can also identify their own custom URLs for downloading software,
which will be given higher priority than Zenodo (useful for situations when
-a custom software is downloaded and built in a project branch (not the core
+custom software is downloaded and built in a project branch (not the core
'maneage' branch).
------------------------------
@@ -467,7 +469,7 @@ building of GCC can optionally be disabled with the '--host-cc' option to
significantly speed up the build when the host's GCC is
similar). Furthermore, Maneage can be built within a Docker container.
-Generally, a paragraph has been added in Section IV on this issue (the
+A paragraph has been added in Section IV on this issue (the
build time and building within a Docker container). We have also defined
task #15818 [1] to have our own core Docker image that is ready to build a
Maneaged project and will be adding it shortly.
@@ -499,9 +501,9 @@ ANSWER: "Reproducibility" has been defined along with "Longevity" and
ANSWER: Ansible and Helm are primarily designed for distributed
computing. For example Helm is just a high-level package manager for a
-Kubernetes cluster that is based on containers. A review of them can be
-added in the Appendix, but we feel they may not be too relevant for this
-paper.
+Kubernetes cluster that is based on containers. A review of them could be
+added to the Appendices, but we feel they this would distract somewhat
+from the main points of our current paper.
------------------------------
@@ -514,8 +516,9 @@ paper.
the maintaining community, which creates another problem within the
perspective of the article.
-ANSWER: Thank you very much for highlighting that this point was not included
-for the sake of length, it has been fitted into the introduction now.
+ANSWER: Thank you very much for highlighting this point. We had excluded
+this point for the sake of article length, but we have restored it in
+the introduction of the revised version.
------------------------------
@@ -527,7 +530,7 @@ for the sake of length, it has been fitted into the introduction now.
the discussion presented by THAIN et al., 2015 is interesting to
increase essential points of the current work.
-ANSWER: Thank you very much for pointing this the works by Thain. We
+ANSWER: Thank you very much for pointing out the works by Thain. We
couldn't find any first-author papers in 2015, but found Meng & Thain
(https://doi.org/10.1016/j.procs.2017.05.116) which had a related
discussion of why they didn't use Docker containers in their work. That
@@ -542,9 +545,10 @@ paper is now cited in the discussion of Containers in Appendix A.
26. [Reviewer 3] About the Singularity, the description article was missing
(Kurtzer GM, Sochat V, Bauer MW, 2017).
-ANSWER: Thank you for the reference, we could not put it in the main body
-of the paper (like many others) due to the strict bibliography limit of 12,
-but it has been cited in Appendix A (where we discuss Singularity).
+ANSWER: Thank you for the reference. We are restricted in the main
+body of the paper due to the strict bibliography limit of 12
+references; we have included Kurtzer et al 2017 in Appendix A (where
+we discuss Singularity).
------------------------------
@@ -556,8 +560,8 @@ but it has been cited in Appendix A (where we discuss Singularity).
(WILKINSON et al., 2016).
ANSWER: The FAIR principles have been mentioned in the main body of the
-paper, but unfortunately we had to remove its citation the main paper (like
-many others) within the maximum limit 12 references. We have cited it in
+paper, but unfortunately we had to remove its citation in the main paper (like
+many others) to keep to the maximum of 12 references. We have cited it in
Appendix B.
------------------------------
@@ -571,9 +575,16 @@ Appendix B.
reproducibility of a publication could be better explored, which would
further enrich the tool presented.
-#####################################
-ANSWER:
-#####################################
+
+ANSWER: Our section II discussing existing tools seems to be the most
+appropriate place to mention IPOL, so we have retained its position at
+the end of this section.
+
+We have indeed included an in-depth discussion of IPOL in Appendix B.
+We recommend it to the reader for any project written uniquely in C,
+and we comment on the readiness of Maneage'd projects for a similar
+level of peer-review control.
+
------------------------------