diff options
| -rw-r--r-- | paper.tex | 2 |
1 files changed, 1 insertions, 1 deletions
@@ -115,7 +115,7 @@ The status in other fields, where workflows are not commonly shared, is highly l Nature is already a black box which we are trying hard to unlock. Not being able to experiment on the methods of other researchers is a self-imposed back box over it. -The completeness of a paper's published metadata (usually within the ``Methods'' section) can be measured by the ability to reproduce the result without needing to contact the authors. +The completeness of a paper's published workflow (usually within the ``Methods'' section) can be measured by the ability to reproduce the result without needing to contact the authors. Several studies have attempted to answer this with different levels of detail. For example, \citet{allen18} found that roughly half of the papers in astrophysics do not even mention the names of any analysis software, while \citet{menke20} found this fraction has greatly improved in medical/biological field and is currently above $80\%$. \citet{ioannidis2009} attempted to reproduce 18 published results by two independent groups, but fully succeeded in only 2 of them and partially in 6. \citet{chang15} attempted to reproduce 67 papers in well-regarded journals in Economics with data and code: only 22 could be reproduced without contacting authors, and more than half could not be replicated at all. \tonote{DVG: even after contacting the authors?} |