diff options
author | Mohammad Akhlaghi <mohammad@akhlaghi.org> | 2020-12-02 15:41:29 +0000 |
---|---|---|
committer | Mohammad Akhlaghi <mohammad@akhlaghi.org> | 2020-12-02 15:41:29 +0000 |
commit | e9dfc5bf433fbf61b97179e3243a20e06a4b01dc (patch) | |
tree | baf31e6434965c841a8f4f47b974d99f1311ea06 | |
parent | 0f3d6ab47f96ef14cd197f2ff3228338b57c937a (diff) |
Minor edits in newly added parts on statistical verification
Thanks a lot Boud for adding that script in your own project and linking it
here. Since the raw file (without context of the whole project) is very
hard to understand for the users, I switched the URL to the navigable URL
the link is actually on the filename. It will always show the most recent
version of this script, not the particular snapshot of now. But infact that
is better, since we can make it better and improve it over time. Maybe even
by the end of this paper's referee review will be able to include it in
Maneage's core branch.
I also removed the link to this discussion at the first paragraph of
Section IV (proof of concept). Since that is just the introduction, and
going into this level of detail there could be confusing for the
readers. Having the name of the script in the proper place is more direct
and understandable for the readers.
Thanks again Boud for the nice work on this ;-).
-rw-r--r-- | paper.tex | 5 | ||||
-rw-r--r-- | peer-review/1-answer.txt | 12 |
2 files changed, 9 insertions, 8 deletions
@@ -287,7 +287,7 @@ In such cases, it is best to immediately convert the data upon collection, and a \section{Proof of concept: Maneage} With the longevity problems of existing tools outlined above, a proof-of-concept tool is presented here via an implementation that has been tested in published papers \cite{akhlaghi19, infante20}. -\new{Since the initial submission of this paper, it has also been used in \href{https://doi.org/10.5281/zenodo.3951151}{zenodo.3951151} (on the COVID-19 pandemic) and \href{https://doi.org/10.5281/zenodo.4062460}{zenodo.4062460} (which \href{https://archive.softwareheritage.org/swh:1:cnt:4217e24e4a474ba43a4d30abfb0a42b823ef4640}{illustrates statistical reproducibility}, e.g., for parallelised code).} +\new{Since the initial submission of this paper, it has also been used in \href{https://doi.org/10.5281/zenodo.3951151}{zenodo.3951151} (on the COVID-19 pandemic) and \href{https://doi.org/10.5281/zenodo.4062460}{zenodo.4062460}.} It was also awarded a Research Data Alliance (RDA) adoption grant for implementing the recommendations of the joint RDA and World Data System (WDS) working group on Publishing Data Workflows \cite{austin17}, from the researchers' perspective. The tool is called Maneage, for \emph{Man}aging data Lin\emph{eage} (the ending is pronounced as in ``lineage''), hosted at \url{https://maneage.org}. @@ -408,7 +408,8 @@ Other built files (intermediate analysis steps) cascade down in the lineage to o Just before reaching the ultimate target (\inlinecode{paper.pdf}), the lineage reaches a bottleneck in \inlinecode{verify.mk} to satisfy the verification criteria (this step was not yet available in \cite{akhlaghi19, infante20}). All project deliverables (macro files, plot or table data and other datasets) are verified at this stage, with their checksums, to automatically ensure exact reproducibility. -Where exact reproducibility is not possible \new{(for example, due to parallelization)}, values can be verified by \new{\href{https://archive.softwareheritage.org/swh:1:cnt:4217e24e4a474ba43a4d30abfb0a42b823ef4640}{a statistical method specified}} by the project authors. +Where exact reproducibility is not possible \new{(for example, due to parallelization)}, values can be verified by the project authors. +\new{For example see \new{\href{https://archive.softwareheritage.org/browse/origin/content/?branch=refs/heads/postreferee_corrections&origin_url=https://codeberg.org/boud/elaphrocentre.git&path=reproduce/analysis/bash/verify-parameter-statistically.sh}{verify-parameter-statistically.sh}} of \href{https://doi.org/10.5281/zenodo.4062460}{zenodo.4062460}.} \begin{figure*}[t] \begin{center} \includetikz{figure-branching}{scale=1}\end{center} diff --git a/peer-review/1-answer.txt b/peer-review/1-answer.txt index 76c574e..ae28c5f 100644 --- a/peer-review/1-answer.txt +++ b/peer-review/1-answer.txt @@ -703,15 +703,15 @@ discussed in Section IV, in the part on verification ("Where exact reproducibility is not possible (for example due to parallelization), values can be verified by a statistical method specified by the project authors."). We have linked keywords in the latter sentence to a Software -Heritage URI [swh] with the specific file in the Peper and Roukema -Maneage'd paper that illustrates an example of how statistical -verification of parallelised code can work in practice. +Heritage URI [1] with the specific file in a Maneage'd paper that +illustrates an example of how statistical verification of parallelised code +can work in practice (Peper & Roukema 2020; zenodo.4062460). We would be interested to hear if any other papers already exist that use -automatic statistical verification of parallelised code as has been done -in this Maneage'd paper. +automatic statistical verification of parallelised code as has been done in +this Maneage'd paper. -[swh] https://archive.softwareheritage.org/swh:1:cnt:4217e24e4a474ba43a4d30abfb0a42b823ef4640 +[1] https://archive.softwareheritage.org/browse/origin/content/?branch=refs/heads/postreferee_corrections&origin_url=https://codeberg.org/boud/elaphrocentre.git&path=reproduce/analysis/bash/verify-parameter-statistically.sh ------------------------------ |