diff options
| -rw-r--r-- | paper.tex | 5 | ||||
| -rw-r--r-- | peer-review/1-answer.txt | 12 |
2 files changed, 9 insertions, 8 deletions
@@ -287,7 +287,7 @@ In such cases, it is best to immediately convert the data upon collection, and a \section{Proof of concept: Maneage} With the longevity problems of existing tools outlined above, a proof-of-concept tool is presented here via an implementation that has been tested in published papers \cite{akhlaghi19, infante20}. -\new{Since the initial submission of this paper, it has also been used in \href{https://doi.org/10.5281/zenodo.3951151}{zenodo.3951151} (on the COVID-19 pandemic) and \href{https://doi.org/10.5281/zenodo.4062460}{zenodo.4062460} (which \href{https://archive.softwareheritage.org/swh:1:cnt:4217e24e4a474ba43a4d30abfb0a42b823ef4640}{illustrates statistical reproducibility}, e.g., for parallelised code).} +\new{Since the initial submission of this paper, it has also been used in \href{https://doi.org/10.5281/zenodo.3951151}{zenodo.3951151} (on the COVID-19 pandemic) and \href{https://doi.org/10.5281/zenodo.4062460}{zenodo.4062460}.} It was also awarded a Research Data Alliance (RDA) adoption grant for implementing the recommendations of the joint RDA and World Data System (WDS) working group on Publishing Data Workflows \cite{austin17}, from the researchers' perspective. The tool is called Maneage, for \emph{Man}aging data Lin\emph{eage} (the ending is pronounced as in ``lineage''), hosted at \url{https://maneage.org}. @@ -408,7 +408,8 @@ Other built files (intermediate analysis steps) cascade down in the lineage to o Just before reaching the ultimate target (\inlinecode{paper.pdf}), the lineage reaches a bottleneck in \inlinecode{verify.mk} to satisfy the verification criteria (this step was not yet available in \cite{akhlaghi19, infante20}). All project deliverables (macro files, plot or table data and other datasets) are verified at this stage, with their checksums, to automatically ensure exact reproducibility. -Where exact reproducibility is not possible \new{(for example, due to parallelization)}, values can be verified by \new{\href{https://archive.softwareheritage.org/swh:1:cnt:4217e24e4a474ba43a4d30abfb0a42b823ef4640}{a statistical method specified}} by the project authors. +Where exact reproducibility is not possible \new{(for example, due to parallelization)}, values can be verified by the project authors. +\new{For example see \new{\href{https://archive.softwareheritage.org/browse/origin/content/?branch=refs/heads/postreferee_corrections&origin_url=https://codeberg.org/boud/elaphrocentre.git&path=reproduce/analysis/bash/verify-parameter-statistically.sh}{verify-parameter-statistically.sh}} of \href{https://doi.org/10.5281/zenodo.4062460}{zenodo.4062460}.} \begin{figure*}[t] \begin{center} \includetikz{figure-branching}{scale=1}\end{center} diff --git a/peer-review/1-answer.txt b/peer-review/1-answer.txt index 76c574e..ae28c5f 100644 --- a/peer-review/1-answer.txt +++ b/peer-review/1-answer.txt @@ -703,15 +703,15 @@ discussed in Section IV, in the part on verification ("Where exact reproducibility is not possible (for example due to parallelization), values can be verified by a statistical method specified by the project authors."). We have linked keywords in the latter sentence to a Software -Heritage URI [swh] with the specific file in the Peper and Roukema -Maneage'd paper that illustrates an example of how statistical -verification of parallelised code can work in practice. +Heritage URI [1] with the specific file in a Maneage'd paper that +illustrates an example of how statistical verification of parallelised code +can work in practice (Peper & Roukema 2020; zenodo.4062460). We would be interested to hear if any other papers already exist that use -automatic statistical verification of parallelised code as has been done -in this Maneage'd paper. +automatic statistical verification of parallelised code as has been done in +this Maneage'd paper. -[swh] https://archive.softwareheritage.org/swh:1:cnt:4217e24e4a474ba43a4d30abfb0a42b823ef4640 +[1] https://archive.softwareheritage.org/browse/origin/content/?branch=refs/heads/postreferee_corrections&origin_url=https://codeberg.org/boud/elaphrocentre.git&path=reproduce/analysis/bash/verify-parameter-statistically.sh ------------------------------ |