diff options
author | Boud Roukema <boud@cosmo.torun.pl> | 2020-05-01 02:20:50 +0100 |
---|---|---|
committer | Mohammad Akhlaghi <mohammad@akhlaghi.org> | 2020-05-01 03:25:08 +0100 |
commit | 8f14213f53ce59a7bc13a8018b5b937f53976fda (patch) | |
tree | 03b722b15ae2389e6f30b7b6e12947ba6a18daf6 | |
parent | 842fd2f0f6b17ce040e6c60dff6b9eb4db9e318e (diff) |
Abstract re-organized to be more research-oriented
To make this a research article, we either have to present it as a
theoretical advance, or as an empirical advance.
An empirical research result would be something like doing a survey of
users and getting statistics of their success/failure in using the system,
and of whether their experience is consistent with the claimed properties
and principles of Maneage (e.g. success/failure in creating paper.pdf as
expected? was the user's system POSIX? did the user do the install with
non-root privileges? was this a with-network or without-network ./project
configure ?) This is doable, but would require a bit of extra work that we
are not necessarily motivated to do or have the time to do right now.
I think it's possible to present Maneage as a theoretical advance, but it
has to be worded properly. Maneage is a tool, but it's a tool that
satisfies what we can reasonably present as a unique theoretical proposal.
Here's my proposed rewrite. I've aimed at minimum word length. I've also
included (commented out) keywords for a structured research abstract -
these are just for us, as a guideline to improve the abstract.
I think "criteria" is safer than "standards". Whether a principle is good
or bad tends to lead to debate. Whether a criterion is satisfied or not is
a more objective question, independent of whether you agree with the
criterion or not.
In the rewrite below, we propose a theoretical standard and show that the
new standard can be satisfied. Maneage is *used as a tool* to prove that
the standard is not too difficult to achieve. Maneage is no longer the
subject of the paper. (That won't change the main body of the paper too
much, apart from compression, but the way it's presented will have to
change, under this proposal.)
The title would need to match this. E.g.
TITLE.1: Evidence that a higher standard of reproducibility criteria is
attainable
TITLE.2: Evidence that a rigorous standard of reproducibility criteria is
attainable
TITLE.3: Towards a more rigorous standard of reproducibility criteria
I would probably go for TITLE.3.
-rw-r--r-- | paper.tex | 35 |
1 files changed, 28 insertions, 7 deletions
@@ -52,15 +52,36 @@ %% Abstract {\noindent\mpregular - Over the last 30 years, many reproducible workflow solutions have been proposed, mostly using the common high-level technology of the day. - Thus providing immediate reproducibility, but problematic in the long-term because high-level technologies evolve. - Scientists are accountable to their results decades later, and don't have the resources to re-write their projects. + %%CONTEXT + Many reproducible workflow solutions have been proposed during recent decades. + Most use high-level technology that is popular, providing immediate reproducibility that is not sustainable in the long term. This creates generational gaps between scientists and makes it hard to build upon previous work. - In this paper, we report the result of our research project on a fundamentally new design that is founded on the principles of completeness (e.g., no dependency beyond a POSIX-compatible operating system, no administrator privileges, and no network connection), with modular and straightforward design, temporal provenance, scalability, and free software. - It is called Maneage (managing+lineage) and is stored in machine-actionable, and human-readable plain-text format. - Facilitating version-control, publication, archival, and automatic parsing to extract data provenance. - It can build its environment automatically, possibly, in virtual machines, containers or any future technology as a binary blob for immediate/fast reproduction. + Decades after their results are published, scientists lack the resources to re-write their project software. + %% [This is probably the sentence in this section that could most easily be + %%removed: it more or less repeats the basic issue of reproducibility.] + %%AIM + We aim to introduce a standard of reproducibility criteria that is more rigorous than those previously adopted. + %%METHOD + In this paper, we propose this new standard: completeness (no dependency beyond a POSIX-compatible operating system, no administrator privileges, and no network connection); modular and straightforward design; temporal provenance; scalability; and free software. + %% I would suggest "free-licensed software" or "free-and-open-source + %% software". RMS would scream at us, but the risk is that the editor (or + %% reader) thinks of free-as-in-beer software. Alternatives include "free + %% software (as in free speech)" - but that looks a bit too informal - or + %% long expressions such as "free software (in the sense of the Free + %% Software Definition). If we have enough words available, "software + %% satisfying the Free Software Definition" would be clear and formal (but + %% probably too specific, since there's also the Open Source Software + %% Definition of the OSI, and Debian's DFSG). + % + %%RESULTS + We demonstrate that these criteria are achievable by presenting a concrete example that satisfies these criteria. + "Maneage" (managing+lineage) is stored in machine-actionable, human-readable plain-text format, with version-control, archival, automatic parsing to extract data provenance, and peer-reviewable paper verification. + %% It can build its environment automatically or can be placed in a container as a binary blob for immediate/fast reproduction. + %% [This sentence is probably sort of true for many systems, and is less critical to the "research question"; I suggest dropping it.] Maneage has already been used in several scientific publications including the present one, with snapshot \projectversion. + %%CONCLUSION + Thus, it is realistic to require that reproducibility solutions satisfy our newly proposed standard. + \horizontalline \noindent |