diff options
| -rw-r--r-- | peer-review/2-review.txt | 147 |
1 files changed, 147 insertions, 0 deletions
diff --git a/peer-review/2-review.txt b/peer-review/2-review.txt new file mode 100644 index 0000000..9f8cdd8 --- /dev/null +++ b/peer-review/2-review.txt @@ -0,0 +1,147 @@ +From: Computing in Science and Engineering <onbehalfof@manuscriptcentral.com> +To: mohammad akhlaghi org, + infantesainz gmail com, + boud astro uni torun pl, + mkhellat ideal-information.com, + david.valls-gabaud observatoiredeparis psl eu, + rbaena iac es +Cc: cise@computer.org, + cise-rr@computer.org +Received: Wed, 7 Apr 2021 19:39:59 +0000 +Subject: Decision - Computing in Science and Engineering, CiSESI-2020-06-0048.R1 + +-------------------------------------------------- + +Computing in Science and Engineering, CiSESI-2020-06-0048.R1 +"Towards Long-term and Archivable Reproducibility" +manuscript type: Reproducible Research + +Dear Dr. Mohammad Akhlaghi: + +Congratulations! Your manuscript, "Towards Long-term and Archivable +Reproducibility," CiSESI-2020-06-0048.R1, has been accepted for publication +in an upcoming issue of Computing in Science and Engineering, subject to a +final light copyedit. Do note the editors' comments below. + +Thank you, +Lorena A. Barba +Editor in Chief, Computing in Science and Engineering +labarba gwu edu +********** + +Editor-in-Chief's Comments +********** +- I am processing this as an "accept" to expedite, but please take + notice/care of the following items before submitting your final files. +- Note particularly that you have to edit any usage of a reference (like + [1]) as part of speech in a sentence. Use alternatives like "Smith et + al. [1]." The article template uses superscripts for references! +- I strongly recommend that you deposit the appendices in arXiv, separately + from the main preprint. This way, all the cited works will get their + citation indexed by Google Scholar, which the authors will likely + appreciate. +- You may add the arXiv id for the manuscript and the appendices in your + reproducibility statement at the end of the Abstract. +- I have manually edited the due date for your final files for Friday April + 9 because we are finalizing the next issue. If we received your files + ASAP we can include your article in the next issue. + +Associate Editor Comments: +********** +(There are no comments) + +Reviewers' Comments +********** + +Reviewer: 1 + +Recommendation: Accept With No Changes + +Comments: +A more in depth evaluation of different options from a technical standpoint +rather than just discussion would really strengthen the paper showing that +the idea isn't just good enough to generate this paper that does not +contain experimental code executions and benchmarking demonstrating +different system characteristics. Deeper acknowledgement of the differences +each system brings and over time as they are improved could be greatly +expanded. This is an area of extreme concern. + +Additional Questions: + +1. How relevant is this manuscript to the readers of this periodical? +Please explain your rating in the Detailed Comments section.: Very Relevant + +2. To what extent is this manuscript relevant to readers around the world?: +The manuscript is of interest to readers throughout the world + +1. Please summarize what you view as the key point(s) of the manuscript and +the importance of the content to the readers of this periodical: This +paper presents necessary requirements for reproducible work. This is aimed +at being able to generate the same output from a workflow of inputs, test +code, and data processing into a text report format. + +The concepts presented cover a wide variety of topics hitting on the vast +majority of cases. The kinds of things not really addressed, such as minor +hardware version differences not evident except in physical stamps on the +part itself, but yielding slightly different behavior, are mentioned at a +high level and left unaddressed. These concerns are part of the longevity +discussion instead and limit the lifetime of an artifact. + +The topics presented cover the topic reasonably well and offer a good guide +for people to think about how best to approach providing a reproducible +scientific system. Most favorably, the paper itself is offered as an +example with an embedded, machine generated version ID and link to the +source materials. This is truly putting your money where your mouth is, so +to speak. + +2. Is the manuscript technically sound? Please explain your answer in the +Detailed Comments section.: Yes + +3. What do you see as this manuscript's contribution to the literature in +this field?: Clearly defining what the longevity limitations are for +reproducible work is crucial for us as a community to have effective +discussions about what we mean for something to be reproducible. Unless we +can agree what acceptable longevity is, we cannot agree if something is +reproducible or not. The other factors listed for different components are +important things to consider and cover basically everything necessary. + +4. What do you see as the strongest aspect of this manuscript?: A simple, +easy to follow discussion of what reproducibility really means and how to +achieve it. + +5. What do you see as the weakest aspect of this manuscript?: Few examples +of the process itself and no testing/comparison of any of the variants in +the supplemental materials. + +1. Does the manuscript contain title, abstract, and/or keywords?: Yes + +2. Are the title, abstract, and keywords appropriate? Please elaborate in +the Detailed Comments section.: Yes + +3. Does the manuscript contain sufficient and appropriate references +(maximum 12-unless the article is a survey or tutorial in scope)? Please +elaborate in the Detailed Comments section.: References are sufficient and +appropriate + +4. Does the introduction clearly state a valid thesis? Please explain your +answer in the Detailed Comments section.: Yes + +5. How would you rate the organization of the manuscript? Please elaborate +in the Detailed Comments section.: Satisfactory + +6. Is the manuscript focused? Please elaborate in the Detailed Comments +section.: Satisfactory + +7. Is the length of the manuscript appropriate for the topic? Please +elaborate in the Detailed Comments section.: Satisfactory + +8. Please rate and comment on the readability of this manuscript in the +Detailed Comments section.: Easy to read + +9. Please rate and comment on the timeliness and long term interest of this +manuscript to CiSE readers in the Detailed Comments section. Select all +that apply.: Topic and content are of immediate and continuing interest to +CiSE readers + +Please rate the manuscript. Explain your choice in the Detailed Comments +section.: Good |