diff options
Diffstat (limited to 'paper.tex')
| -rw-r--r-- | paper.tex | 13 |
1 files changed, 7 insertions, 6 deletions
@@ -196,7 +196,7 @@ In summary, notebooks can rarely deliver their promised potential \cite{rule18} An exceptional solution we encountered was the Image Processing Online Journal (IPOL, \href{https://www.ipol.im}{ipol.im}). Submitted papers must be accompanied by an ISO C implementation of their algorithm (which is buildable on any widely used OS) with example images/data that can also be executed on their webpage. This is possible owing to the focus on low-level algorithms with no dependencies beyond an ISO C compiler. -However, many data-intensive projects commonly involve dozens of high-level dependencies, with large and complex data formats and analysis, and hence this solution is not scalable. +However, many data-intensive projects commonly involve dozens of high-level dependencies, with large and complex data formats and analysis, so this solution is not scalable. @@ -1225,7 +1225,7 @@ This failure to communicate in the details is a very serious problem, leading to \label{appendix:existingsolutions} As reviewed in the introduction, the problem of reproducibility has received a lot of attention over the last three decades and various solutions have already been proposed. -The core principles that many of the existing solutions (including Maneage) aims to achieve are nicely summarized by the FAIR principles \citeappendix{wilkinson16}. +The core principles that many of the existing solutions (including Maneage) aim to achieve are nicely summarized by the FAIR principles \citeappendix{wilkinson16}. In this appendix, some of the solutions are reviewed. The solutions are based on an evolving software landscape, therefore they are ordered by date: when the project has a webpage, the year of its first release is used for the sorting, otherwise their paper's publication year is used. @@ -1393,15 +1393,16 @@ An IPOL paper is a traditional research paper, but with a focus on implementatio The published narrative description of the algorithm must be detailed to a level that any specialist can implement it in their own programming language (extremely detailed). The author's own implementation of the algorithm is also published with the paper (in C, C++ or MATLAB), the code must be commented well enough and link each part of it with the relevant part of the paper. The authors must also submit several example datasets/scenarios. -The referee actually inspects the code and narrative, confirming that they match with each other, and with the stated conclusions of the published paper. +The referee is expected to inspect the code and narrative, confirming that they match with each other, and with the stated conclusions of the published paper. After publication, each paper also has a ``demo'' button on its webpage, allowing readers to try the algorithm on a web-interface and even provide their own input. The IPOL model is the single most robust model of peer review and publishing computational research methods/implementations that we have seen in this survey. It has grown steadily over the last 10 years, publishing 23 research articles in 2019 alone. We encourage the reader to visit its webpage and see some of its recent papers and their demos. -The reason it can be so thorough and complete is its a very narrow scope (image processing algorithms), where the published algorithms are highly atomic, not needing significant dependencies (beyond input/output), allowing the referees/readers to go deep into each implemented algorithm. -In fact, high-level languages like Perl, Python or Java are not acceptable in IPOL precisely because of the additional complexities/dependencies that they require. -If any referee/reader was inclined to do so, a paper written in Maneage (the proof-of-concept solution presented in this paper) allows for a similar level of scrutiny, but for much more complex research scenarios, involving hundreds of dependencies and complex processing on the data. +The reason it can be so thorough and complete is its very narrow scope (image processing algorithms), where the published algorithms are highly atomic, not needing significant dependencies (beyond input/output), allowing the referees and readers to go deeply into each implemented algorithm. +In fact, high-level languages like Perl, Python or Java are not acceptable in IPOL precisely because of the additional complexities, such as dependencies, that they require. +If any referee or reader were inclined to do so, a paper written in Maneage (the proof-of-concept solution presented in this paper) could be scrutinised at a similar detailed level, but for much more complex research scenarios, involving hundreds of dependencies and complex processing of the data. + |