aboutsummaryrefslogtreecommitdiff
path: root/paper.tex
AgeCommit message (Collapse)AuthorLines
2020-11-30Summarized Roberto's CV, further summarized Raul's and Mohammad'sMohammad Akhlaghi-9/+6
Roberto sent me his summarized CV which is now being included and I also removed the extra statements about non-degree things from Raul and my own biography (like mentioning Gnuastro, and scientific interests). To be short, we are only mentioning degrees and positions. For Raul, I added his M.Sc institute.
2020-11-30Imported improved definition that is made better after discussionMohammad Akhlaghi-5/+5
After Mohammad-reza sent me his commit on an improved definition for longevity, we had an indepth discussion (through a video-conference) to avoid complexities in the terminology, while staying on point and word-count. In this commit/merge, I am including the improved version of the definition of longevity, and the newly added term "functionality" (instead of "usability" that Mohammad-reza was originally complaining to).
2020-11-30Minor edit in paragraph on execution timeMohammad Akhlaghi-6/+6
The paragraph was slightly shortened, while keeping the main points.
2020-11-30Rephrased longetivity definitionMohammadreza Khellat-3/+4
Before this commit, Longetivity was defined on the basis of the term usability. Although the scope and context of the term has been mentioned right after its use, this could have caused confusion with the keyword "usability" in the field of software engineering. With this commit, Longetivity definition has been rephrased in a way that it would not require "usability". Furthermore, since longetivity would logically require the availability of the machines and platforms during the time of re-use, this has been explicitly mentioned in the definition.
2020-11-28Shorter biography for RaulRaul Infante-Sainz-3/+2
Following Boud's great suggestion, I also summarized my CV to be less than 40 words.
2020-11-27Shorter biography for MohammadMohammad Akhlaghi-8/+7
Following Boud's great suggestion, I also summarized my CV to be less than 40 words.
2020-11-27Shorter CVs for boud+davidBoud Roukema-10/+4
This commit provides shorter CVs for me (Boud) + David in order to get closer to the 6500 word limit. Our CVs are the least significant part of the paper.
2020-11-27Merged with Boud's corrected answers (generally very similar)Mohammad Akhlaghi-7/+8
The only issue that still remains is how to address statistical reproducibility, and I am in touch with Boud to do this in the best way possible (it has been highlighted with '#####'s in the answers.
2020-11-26All the referee points have been answeredMohammad Akhlaghi-15/+17
There is an answer for all the referee points now. I also did some minor edits in the paper. But we are still over the limit by around 250 words. The only remaining point that is not yet addressed (and has '####' around it) is the discussion on parallelization and its effect on reproducibility.
2020-11-26All questions have now been responded toBoud Roukema-8/+9
This commit is intended to be submittable quality. Point 56 was removed, and the later points renumbered, because it was a point of Reviewer 5 described what we have done - it was not a criticism to respond do. :) The current word count (without abstract and references) is 6091.
2020-11-25Reviewer points 16 to 32Boud Roukema-6/+7
Copyediting of points 16 to 32 (paper.tex + peer-review/1-answer.txt) is done in this commit. TODO list: 2. paper lacking focus 9. tidy up README-hacking.md for appearance on website App B.G. similar to Figure ?? - ref missing 29. website: README-hacking.md and tutorial "on same page"
2020-11-25Reviewer points 1-15; appendix clickable linksBoud Roukema-50/+57
This commit updates "paper.tex" and "peer-review/1-answer.txt" for the first 15 (out of 59!) reviewer points, excluding points 2 (not yet done) and 9 (README-hacking.md needs tidying). A fix to "reproduce/analysis/make/paper.mk" for the links in the appendices is also done in this commit (the same algorithm as for paper.tex is added). The links in the appendices are not (yet) clickable.
2020-11-25Copyedit; no-abstract word count 6084Boud Roukema-31/+37
This commit tidies up minor aspects of the language in the text marked by "\new", e.g. a "wokflow" would be fine for Chinese cooking, but is a little off-topic for Maneage. :) The word count is reduced by about 7 words. I haven't yet got to the serious part: checked that we've responded to the referees' points, and completing the responses which we haven't yet done.
2020-11-23Minor edits and correctionsMohammad Akhlaghi-6/+6
Raul's added point on the answer to the referee was very good, so I edited it a little to be more clear (and removed his name). Also, after looking in a few parts of the text, I fixed a few typos.
2020-11-23Minor corrections to the final paper documentRaul Infante-Sainz-20/+17
With this commit, I make several minor changes to the text of the final paper. They are not important, but minor modifications like avoiding contractions (don't -> do not, and so on).
2020-11-23First draft of all the points addressed by the refereesMohammad Akhlaghi-83/+249
A new directory has been added at the top of the project's source called 'peer-review'. The raw reviews of the paper by the editors and referees has been added there as '1-review.txt'. All the main points raised by the referees have been listed in a numbered list and addressed (mostly) in '1-answers.txt'. The text of the paper now also includes all the implemented answers to the various points.
2020-11-20Highlighting changes can now be toggled at run-timeMohammad Akhlaghi-7/+0
Until now, the core Maneage 'paper.tex' had a '\highlightchanges' macro that defines two LaTeX macros: '\new' and '\tonote'. When '\highlightchanges' was defined, anything that was written within '\new' became dark green (highlighting new things that have been added). Also, anything that was written in '\tonote' was put within a '[]' and became dark red (to show that there is a note here that should be addressed later). When '\highlightchanges' wasn't defined, anything within the '\new' element would be black (like the rest of the text), and the things in '\tonote' would not be shown at all. Commenting the '\newcommand{\highlightchanges}{}' line within 'paper.tex' (to toggle the modes above) would create a different Git hash and has to be committed. But this different commit hash could create a false sense in the reader that other things have also been changed and the only way they could confirm was to actually go and look into the project history (which they will not usually have time to do, and thus won't be able to trust the two modes of the text). Also, the added highlights and the note highlights were bundeled together into one macro, so you couldn't only have one of them. With this commit, the choice of highlighting either one of the two is now done as two new run-time options to the './project' script (which are passed to the Makefiles, and written into the 'project.tex' file which is loaded into 'paper.tex'). In this way, we can generate two PDFs with the same Git commit (project's state): one with the selected highlights and another one without it. This issue actually came up for me while implementing the changes here: we need to submit one PDF to the journal/referees with highlights on the added features. But we also need to submit another PDF to arXiv and Zenodo without any highlights. If the PDFs have different commit hashes, the referees may associate it with other changes in any part of the work. For example https://oadoi.org/10.22541/au.159724632.29528907 that mentions "Another version of the manuscript was published on arXiv: 2006.03018", while the only difference was a few words in the abstract after the journal complained on the abstract word-count of our first submission (where the commit hashes matched with arXiv/Zenodo).
2020-11-15First edits on the newly added appendices in new formMohammad Akhlaghi-306/+336
With the optional appendices added recently to the paper, it was important to go through them and make them more fitting into the paper.
2020-11-04Appendix of long paper added, optionally we can disable itMohammad Akhlaghi-3/+997
Given the referee reports, after discussing with the editors of CiSE, we decided that it is important to include the complete appendix we had before that included a thorough review of existing tools and methods. However, the appendix will not be published in the paper (due to the strict word-count limit). It will only be used in the arXiv/Zenodo versions of the paper. This actually created a technical problem: we want the commit hash of the project source to remain the same when the paper is built with an appendix or without it. To fix this problem the choice of including an appendix has gone into the 'project' script as a run-time option called '--no-appendix'. So by default (when someone just runs './project make'), the PDF will have an appendix, but when we want to submit to the journal, or when the appendix isn't needed for a certain reason, we can use this new option. The appendix also has its own separate bibliography. Some other corrections made in this commit: 1. Some new references were added that had an '_' in their source, they were corrected in 'references.tex'. 2. I noticed that 'preamble-style.tex' is not actually used in this paper, so it has been deleted.
2020-09-14Add machine class related argument and fix small typosMohammadreza Khellat-14/+12
Before this commit, there were no arguments regarding machine related specifications in the manuscript. This was needed as Mohammad Akhlaghi came across a review of the artcile by Dylan Aïssi in which Dylan mentioned the need for discussing CPU architecture dependence in pursuing a long-trem archivable workflow. With this commit, the required argument has been added in Sec.IV POC: Maneage in the paragraph in which it is explained how 'macro files build the core skeleton of Maneage'. Furthermore, few typos in different places have been fixed and the 'pre-make-build.sh' has been updated with the latest fix in Maneage core project.
2020-09-04Minor correction in first sentence of abstractMohammad Akhlaghi-2/+1
This paper is generally about data analysis pipelines, so the abstract now starts with "Analysis pipelines" instead of "Reproducible workflows". I also noticed that the sentence was mistakenly broken into multiple lines.
2020-09-03Imported recent work in Maneage, minor conflicts fixedMohammad Akhlaghi-1/+0
Only two small conflicts came up: * The addition of the hardware architecture macro in 'paper.tex' (which was removed for now, but will be added as the referee has requested within the text). * The usage of "" around directory variables in 'paper.mk'.
2020-09-03Added example of DockerHub deleting unused Docker imagesMohammad Akhlaghi-1/+3
I saw this link today in the news (to be implemented from November 1st, 2020), and because it is directly related to this work, I added it. Many people assume that simply pushing a Docker image to DockerHub is enough to preserve it, but ignore how much it costs to maintain the storage and network capacity.
2020-08-27Machine architecture and byte-order available as LaTeX macroMohammadreza Khellat-13/+14
Until now, no machine-related specifications were being documented in the workflow. This information can become helpful when observing differences in the outcome of both software and analysis segments of the workflow by others (some software may behave differently based on host machine). With this commit, the host machine's 'hardware class' and 'byte-order' are collected and now available as LaTeX macros for the authors to use in the paper. Currently it is placed in the acknowledgments, right after mentioning the Maneage commit. Furthermore, the project and configuration scripts are now capable of dealing with input directory names that have SPACE (and other special characters) by putting them inside double-quotes. However, having spaces and metacharacters in the address of the build directory could cause build/install failure for some software source files which are beyond the control of Maneage. So we now check the user's given build directory string, and if the string has any '@', '#', '$', '%', '^', '&', '*', '(', ')', '+', ';', and ' ' (SPACE), it will ask the user to provide a different directory.
2020-08-20Data lineage and replicated plot in one rowMohammad Akhlaghi-9/+15
Until now, the replicated plot had the width of the full page and the data lineage graph was under it. Together they were covering more than half of the height of the page! But the plot showing the number of papers with tools really doesn't have too much detail, and all the space was being wasted. With this commit, the plot is now much much thinner and the data lineage graph has been fitted to the right of it.
2020-07-04Commit hash of Maneage branch used to build project as LaTeX macroMohammad Akhlaghi-12/+13
To help in the documentation, the Git hash of the Maneage branch commit that the project has most recently merged with (or branched from) is now also provided as a LaTeX macro ('\maneageversion'). It is calculated in 'reproduce/analysis/make/initialize.mk' (in the recipe to 'initialize.tex').
2020-07-04Citing Maneage paper in acknowledgmentsMohammad Akhlaghi-9/+9
In the previous commit, the modified abstract of the acknowledgments only included the URL of Maneage, but its more formal to cite the Maneage paper, the URL is already present in the paper.
2020-07-04Corrected Acknowledgments section in default paperMohammad Akhlaghi-14/+14
Until now, the acknowledgment section didn't contain the new name of Maneage and it also included an acknowledgment of Gnuastro (which is not appropriate for a general project which may not use Gnuastro). With this commit this is fixed.
2020-07-01Corrected small typo: ny --> anyMohammad Akhlaghi-1/+1
This was pointed out by Mervyn O'Luing.
2020-06-30Implemented comments by Mervyn O'LuingMohammad Akhlaghi-19/+17
Mervyn had read the paper and provided some interesting thoughts that I tried to implement. Mervyn's comments are shown below. I just haven't addressed the last point yet, because I am affraid it may make the text too long (we are already on the boundary of the word-limit). We have already discussed that it is a good research topic, and have hopefully triggered the curiosity of the readers to test it ;-). ------------------- Page 2: Regarding Criterion 1: Completeness. A project must be self contained? So this includes not requiring root or administrator privileges. This suggests that the project is only made open after the development has been completed? Regarding Criterion 5: 'a clerk can do it' -- in the pc world that we live in could this be taken as a disparaging comment? Page 5: 'The C library is linked with all programs, and this dependence can hypothetically hinder exact reproducibility of results, but we have not encountered this so far.' - what do you think might happen if this does affect reproducibility? Do you have a plan to deal with this? Or are you going to wait until you hear of such cases as the number will probably be small? Have you done probability analysis to show that the rates are likely to be very small? Or should you have a disclaimer with maneage?
2020-06-28Zenodo identifier is extracted automatically from metadata.confMohammad Akhlaghi-2/+2
Until now, the Zenodo identifier was manually written in the paper. But now we have the Zenodo DOI in 'metadata.conf', so its much more robust to get it from there (in case updated versions of the paper is published).
2020-06-22Acknowledged very usful discussions at the AMIGA group of IAAMohammad Akhlaghi-1/+2
I visited the AMIGA group in January this year and we had some very useful discussion on Maneage.
2020-06-21Minor edits to clarify some pointsMohammad Akhlaghi-241/+116
After going through Terry's corrections, some things were clarified more. Technically, I realized that many new-lines were introduced and corrected them. Also, in Roberto's biography, I noticed that compared to the others it has too much non-reproducibility details, so I removed the redundant parts for this paper.
2020-06-21Edits by Terry MahoneyTerry Mahoney-147/+285
Terry is an astronomer at IAC's Scientific Editorial Service and kindly agreed to review this paper for us and actually pushed this commit. I am just adding a commit message here.
2020-06-16Acknowledged contributions of Marios KarouzosMohammad Akhlaghi-3/+7
Marios had read the first draft of the paper (Commit f990bba) and provided valuable feedback (shown below) that ultimately helped in the current version. But because of all the work that was necessary in those days, I forgot to actually thank him in the acknowledgment, while I had implemented most of his thoughts. Following Marios' thoughts on the Git branching figure, with this commit, I am also adding a few sentences at the end of the caption with a very rough summary of Git. I also changed the branch commit-colors to shades of brown (incrementally becoming lighter as higher-level branches are shown) to avoid the confusion with the blue and green signs within the schematic papers shown in the figure. Marios' comments (April 28th, 2020, on Commit f990bba) ------------------------------------------------------ I think the structure of the paper is more or less fine. There are two places that I thought could be improved: 1) Section 3 (Principles) was somewhat confusing to me in the way that it was structured. I think the main source of confusion is the mixing of what Maeage is about and what other programs have done. I would suggest to separate the two. I would have short intro for the section, similar to what you have now. However, I would suggest to highlight the underlying goals motivating the principles that follow: reproducibility, open science, something else? Then I would go into the details of the seven principles. Some of the principles are less clear to me than others. For example, why is simplicity a guiding principle? Then some other principles appear to be related, for example modularity, minimal complexity and scalability to my eyes are not necessarily separate. Finally, I would separate the comparison with other software and either dedicate a section to that somewhere toward the end of the paper (perhaps a subsection for section 5) or at least condense it and put it as a closing paragraph for Section 3. As it is now I think it draws focus from Maneage and also includes some repetitions. 2) Section 4 (Maneage) was at times confusing because it is written, I think in part as a demonstration of Maneage (i.e., including examples that showed how Maneage was used to write this or other papers) and a manual/description of the software. I wonder whether these two aspects can be more cleanly separated. Perhaps it would be possible to first have a section 4 where each of the modules/units of Maneage are listed and explained and then have the following section discuss a working example of Maneage using this or another paper. 3) I found Figure 7 [the git branching figure] and its explanation not very intuitive. This probably has to do with my zero knowledge of github and how versioning there works, but perhaps the description can be a bit more "user friendly" even for those who are not familiar with the tool. 4) I find Section 6 to be rather inconsequential. It does not add anything and it more or less is just a summary of what was discussed. I would personally remove it and include a very short summary of the ideals/principles/goals of Maneage at the beginning of Section 5, before the discussion.
2020-06-14Better comments for the top macros of paper.texMohammad Akhlaghi-26/+50
The default 'paper.tex' starts by defining some macros and comments describing them. Until now, the text was not too clear and could be confusing for someone that is not at all familiar with Maneage. With this commit, the comments have been edited to be more clear for a first-time reader. For example they all start with FULL CAPS summaries. Two other small things were corrected in 'tex/src/preamble-necessary.tex': - Until now 'project.tex' was included in this preamble. However, because of its importance in Maneage, and prominent place in the demonstration plot of the paper introducing Maneage, it is now included directly in 'paper.tex'. This also allows users to safely ignore/delete this preamble file if their LaTeX style is different. - I noticed that some macros for some astronomical software names from the very first commits in Maneage were still present here! They are no longer used, so they have been removed.
2020-06-14Corrected the relation of POSIX and IEEEMohammad Akhlaghi-2/+3
Until now, we were saying "POSIX is defined by the IEEE", but in issue #12, Michael Crusoe pointed out that this is not accurate. It is actually jointly developed and operated by the IEEE, The Open Group and ISO/IEC JTC 1/SC 22, which together form the Austin Group. So the sentence was modified to say tha the IEEE (potential publisher of this paper) is part of the Austin Group that develops the POSIX standard. Thanks a lot for bringing this up Michael.
2020-06-13Custom-built EPS icons in branching figureMarjan Akbari-1/+1
Until now, we were using three EPS (created from SVG) that were downloaded from https://www.flaticon.com. Therefore it was necessary to acknowledge the creators and put a link to the webpage. This consumed space in the caption and decreased the originality of the plot. Another problem was that the "collaboration" icon (with three people in it) had arrows, and some of those arrows pointed downwards, make ambiguity in relation to the top-ward arrows under the commits. With this commit, three alternative icons are added that I made from scratch, using Inkscape. The collaboration icon now is two figures and two speech-bubbles, without any arrows.
2020-06-13Two small edits in demo listing and paragraph after itMohammad Akhlaghi-3/+3
Recently, by default, Maneage will not take the title directly in the PDF, the title should be given in the 'metdata.conf' file and it is passed onto LaTeX as a variable. So the comment to "add project title" in the listing could be confusing. To avoid confusing, I edited it to "Set your name as author". The comments above the '\title' part is very complete and users will clearly be able to modify the title if they want. Also, we had an extra ')' in the line just under it which is now corrected.
2020-06-10Updated text of default paper.tex, putting more recent examplesMohammad Akhlaghi-78/+87
The text of the default paper hadn't been changed for a very long time! In this time, three papers using Maneage have been published (which can be very good as an example), Maneage also now has a webpage! With these commit these examples and the webpage have been added and generally it was also polished a little to hopefully be more useful.
2020-06-09Two minor typos correctedMohammad Akhlaghi-2/+2
Two words were corrected in the text that made the sentences grammatically wrong (they were actually typos! historically they were correct, but we later changed the later part of the sentence without fixing the first part).
2020-06-07Added SoftwareHeritage link, minor typo corrections and clarificationsMohammad Akhlaghi-22/+25
The git history of the project is now archived on SoftwareHeritage and a link to it as was added in the "Reproducible supplement" tag just under the abstract. Also, some corrections were also made in the text. In particular, the part explaining the separation of software and data reproducibility was slightly clarified to be more clear
2020-06-06IMPORTANT: Added publication checklist, improved relevant infrastructureMohammad Akhlaghi-19/+26
Possible semantic conflicts (that may not show up as Git conflicts but may cause a crash in your project after the merge): 1) The project title (and other basic metadata) should be set in 'reproduce/analysis/conf/metadata.conf'. Please include this file in your merge (if it is ignored because of '.gitattributes'!). 2) Consider importing the changes in 'initialize.mk' and 'verify.mk' (if you have added all analysis Makefiles to the '.gitattributes' file (thus not merging any change in them with your branch). For example with this command: git diff master...maneage -- reproduce/analysis/make/initialize.mk 3) The old 'verify-txt-no-comments-leading-space' function has been replaced by 'verify-txt-no-comments-no-space'. The new function will also remove all white-space characters between the columns (not just white space characters at the start of the line). Thus the resulting check won't involve spacing between columns. A common set of steps are always necessary to prepare a project for publication. Until now, we would simply look at previous submissions and try to follow them, but that was prone to errors and could cause confusion. The internal infrastructure also didn't have some useful features to make good publication possible. Now that the submission of a paper fully devoted to the founding criteria of Maneage is complete (arXiv:2006.03018), it was time to formalize the necessary steps for easier submission of a project using Maneage and implement some low-level features that can make things easier. With this commit a first draft of the publication checklist has been added to 'README-hacking.md', it was tested in the submission of arXiv:2006.03018 and zenodo.3872248. To help guide users on implementing the good practices for output datasets, the outputs of the default project shown in the paper now use the new features). After reading the checklist, please inspect these. Some other relevant changes in this commit: - The publication involves a copy of the necessary software tarballs. Hence a new target ('dist-software') was also added to package all the project's software tarballs in one tarball for easy distribution. - A new 'dist-lzip' target has been defined for those who want to distribute an Lzip-compressed tarball. - The '\includetikz' LaTeX macro now has a second argument to allow configuring the '\includegraphics' call when the plot should not be built, but just imported.
2020-06-06Summarized abstract to be less than 150 wordsMohammad Akhlaghi-16/+15
Upon submission to CiSE we were informed that the abstract has to be less than 150 words to be processed. So with this commit, I am shrinking the abstract slightly, trying to remove some points that are less important and trying to shrink some of the sentences. Also, to avoid confusion and be more clear, the term "temporal provenance" has been replaced by "Recorded history".
2020-06-04Scale element in includegraphics for roughly similar-sized figuresMohammad Akhlaghi-3/+3
Until now, when the figures were built directly from EPS ('\newcommand{\makepdf}{}' was commented), they would take the full line-width becoming a little too large! I noticed this after letting arXiv build the PDF. With this commit, the 'includetikz' tool takes a second argument to be a parameter given to 'includegraphics' (which is scale in this case).
2020-06-04Final full reading, and minor edits to submit to Zenodo and arXivMohammad Akhlaghi-58/+57
Everything else regarding the submission to arXiv and Zenodo has been complete, so I done a final read, making some minor edits to hopefully make the text easier to read.
2020-06-04Verification activated, README added, Proper metadata in plot dataMohammad Akhlaghi-2/+8
All the steps following the to-be-added (in 'README-hacking.md') publication checklist prior to the final check from new clone have been added: - 'README.md' file has been set. - "Reproducible supplement" was added just above the keywords, pointing to Zenodo. - A link to the to-be-uploaded data underlying the plot was added in the caption of the tools-per-year plot. - A new meta-data configuration file was added to store basic project metadata to be used throughout the project. This will later be taken into Maneage. For examle the project title is now stored here and written into the paper's LaTeX source and output datasets automatically. - Verification was activated and plot's data and LaTeX macro files are now automatically verified. - A complete metadata was added for the data underlying the plot. - A generic function was added in 'initialize.mk' that will automatically write project info and copyright in all plain-text outputs.
2020-06-03Adding point on small-ness of final product, some summarizationMohammad Akhlaghi-83/+65
I noticed that we hadn't include the publication of the workflow and the advantage that Maneage provides in this regard. So it was added at the end of the proof-of-concept section. However, it was necessary to summarize some other parts to not increase the wordcount.
2020-06-01Edits by DavidDavid Valls-Gabaud-100/+110
These are some corrections that David sent to me by email and I am committing here.
2020-06-01Implemented Antonio's suggestion and thanked himMohammad Akhlaghi-1/+2
Antonio Diaz Diaz (author of the Lzip program/library), has had a very supportive role in what became Maneage in the last 4 years. For example I really started to appreciate the value of simplicity and archivability while reading Lzip's documentation. Fortunately he also read a recent version of the paper that was again very supportive. Some of the minor points he raised had already been fixed, but using 'supplier' instead of 'server' (in the Free Software) criterion was new so I implemented it here with this commit. With this, I am also thanking him for all his wonderful support and encouragement in the last 4 years.