aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)AuthorLines
2018-11-14Configuration stops if a dependency cannot be builtMohammad Akhlaghi-49/+103
Until now, we used semicolons in Make's Call function definitions to build the programs with GNU build system or CMake. Therefore, if any step of the process failed, the rest would be ignorant to it and pass. Now, we use `&&' to separate the different processing steps. In this way, we can be sure that if any of them fails (during configuration, or building for example), the pipeline will also stop and not continue to the next command (in the same recipe). Since the two Make Call functions were identical in the two `dependencies-basic.mk' and `dependencies.mk', they are now in one file to be imported in both. This bug was found by Raul Infante Sainz.
2018-11-14All used software with versions, acknowledgedMohammad Akhlaghi-3/+35
All the used software are now acknowledged in the template paper along with their versions. This section is also mentioned in the check list, so users don't delete it by mistake.
2018-11-14./configure and building of Bash and Make with more basic toolsMohammad Akhlaghi-262/+410
After a test by Raúl Infante Sainz, we found out that the configure script and the Make script for Bash and Make are making too many assumptions on more recent versions of both. As a result, it couldn't be built. Therefore, the `configure' script was modified to not use more recent tools like `readlink' (to find the absolute address of a relative one). It was also re-organized to not have to read the configuration parameters from a text file. The parameters are directly read from the command-line and are written into the proper file afterwards. This removes the need to opening a text editor by the user (which also caused problems on Raúl's system). To fix the Make version issue, the building of Bash and Make are now done in a new Makefile (`reproduce/src/make/dependencies-basic.mk'). This file doesn't make many of the assumptions that were made in `dependencies.mk'. So it should hopefully work on any version of Make. To help in debugging, for now, the Makefile of configure, are asked to work on one thread (the `-j' option is commented in the `configure'). But after checks, we'll fix this.
2018-11-13Most library versions are now also checkedMohammad Akhlaghi-52/+89
All the libraries that define their version string as a macro in their headers are now also checked in `reproduce/src/make/initialize.mk'. Also, the CFITSIO tarball now follows the same versioning style as the rest of the tarballs: a script is added to convert the version string into what is included in the tarball.
2018-11-13Version of programs checked on each run of pipelineMohammad Akhlaghi-26/+44
The version of all programs is now checked in `reproduce/make/src/initialize.mk' and the pipeline won't complete if any of the program versions change from those listed in `reproduce/config/pipeline/dependency-versions.mk'. Since the pipeline is systematically checking all program versions, we don't need Gnuastro's `--onlyversion' option any more. So it (and all references to it) have been removed.
2018-11-12System's environment (including PATH) only available for LaTeXMohammad Akhlaghi-5/+12
The system's environment is now removed from the internal processing of the pipeline, except for LaTeX.
2018-11-12Corrected CFITSIO building recipeMohammad Akhlaghi-1/+1
We were mistakenly using GSL's name for the unpacked tarball.
2018-11-12Libcurl, Git, CMake, TIFF, Zlib also built at configure timeMohammad Akhlaghi-126/+188
During the configuration step several new programs that were necessary for a more complete controlled environment are now also downloaded and built statically.
2018-11-12Added tarball host webp addresses for downloadMohammad Akhlaghi-12/+15
The host web address of most of the necessary packages was blank (filled with `WWWWWWWWWWWWWWWW' as a place holder). They now point to the correct webpages.
2018-11-12Dependencies built at the start of the pipelineMohammad Akhlaghi-486/+671
To enable easy/proper reproduction of results, all the high-level dependencies are now built within the pipeline and installed in a fixed directory that is added to the PATH of the Makefile. This includes GNU Bash and GNU Make, which are then used to run the pipeline. The `./configure' script will first build Bash and Make within itself, then it will build All the dependencies are also built to be static. So after they are built, changing of the system's low-level libraries (like C library) won't change the tarballs. Currently the C library and C compiler aren't built within the pipeline, but we'll hopefully add them to the build process also. With this change, we now have full control of the shell and Make that will be used in the pipeline, so we can safely remove some of the generalities we had before.
2018-08-24Minor editsMohammad Akhlaghi-10/+13
After a full trial of the checklist, some further minor edits were made to make it more clear.
2018-08-24Minor corrections in README.mdMohammad Akhlaghi-7/+7
Some minor changes were made to be more clear.
2018-08-24Minor edits in README.mdMohammad Akhlaghi-11/+18
A few minor points were corrected in README.md.
2018-08-24Step to check pipeline added in README.md checklistMohammad Akhlaghi-4/+21
A step was added close to the top of the checklist to remind people to check the pipeline before making any changes. Also, the `--origin' option was removed from the `git clone' command into a separate command to rename the origin branch. This helps in readability.
2018-08-24Checklist modified to keep pipeline historyMohammad Akhlaghi-34/+55
Until now, in the check list of `README.md', we were recommending to delete the history of the pipeline and start your own history from that. But this disables users of the pipeline to keep it up to date with new features that are added to it. With this commit, the main branch is now called `pipeline' (to allow users to use `master' for their own research) and in the clone command, the pipeline's remote is now called `pipeline-origin' (to allow the user to use `origin' for their own remote).
2018-08-11Emphasized correcting name in copyright notice in checklistMohammad Akhlaghi-4/+5
While trying the checklist, I noticed that I had forgot to add my name after the copyright year and that `reproduce/src/make/paper.mk' still had my own name on it, the copyright notice also said `script' instead of `Makefile' which is now corrected.
2018-08-11Minor edits in README.md after some testsMohammad Akhlaghi-21/+7
While testing the reproduction pipeline on a small project, I noticed some parts of the checklist that were either repetative or needed to be corrected. This is done with this commit.
2018-08-11Shell to run configure script set to /bin/shMohammad Akhlaghi-1/+1
We had previously started the `configure' script with `/bin/bash'. But this script is meant to check for Bash inside of it. So to be run-able (on a system which may not have Bash), the `configure' script has to be run by `/bin/sh'.
2018-08-11Single quote corrected in README.mdMohammad Akhlaghi-1/+1
In the previous commit, ` was mistakenly written as '. This was only noticed after pushing and rendering the Markup on the webpage.
2018-08-11Minor edit in README.mdMohammad Akhlaghi-3/+4
A minor edit was made in the copyright correction part of the checklist.
2018-08-11Corrected copyright notice correction description in checklistMohammad Akhlaghi-5/+12
To be more generic (and avoid listing file names in `README.md'), a command is now used to find the files where the name has to be added to the copyright notice. Also, I noticed that `reproduce/src/make/paper.mk' lacked a placeholder name/email, so one was added.
2018-08-11Minor edits in comments of .dir-locals.elMohammad Akhlaghi-5/+8
To be more easy to read/understand, the comments in `.dir-locals.el' were edited and made a little more elaborate.
2018-08-11Problem with tabs in .mk file corrected in dir-locals.elMohammad Akhlaghi-11/+9
Previously we had tried to corrected the problem that `.mk' files use space instead of TAB by setting them to be read into makefile-mode at the start. However, for some reason, this caused a problem in reading the maximum line length (the value of 75 wasn't read by Emacs). So after some playing around, I found out that the problem is that the `nil' setting (for all other files) must be defined in the end, not the start of the basic settings and this solved this problem.
2018-08-11Not using random distribution in demonstration plotMohammad Akhlaghi-12/+13
Different implementations of AWK may use different random number generators, so even setting the seed will not ensure a reproducible result. Because of this, the random plot may be different when the pipeline runs on different systems and this can confuse early users (its contrary to the exact reproducibility that is the whole purpose of this pipeline). The plot is just a simple X^2 plot, showing the squared value of the X axis on the Y axis. It is very simple, but atleast it will be identical on all systems. Also, there may be too many complicated things in the pipeline already for an early user, and its just a demonstration, so the easier/simpler, the better.
2018-08-11Added bash as a dependency to be checked at configure timeMohammad Akhlaghi-5/+5
While we had set the default Makefile SHELL to be bash, we weren't actually checking if `bash' is available on the system. With this commit, it is also checked at configure time.
2018-07-27SHELL has been explicitly set to /bin/bashMohammad Akhlaghi-0/+13
On some systems, the default shell `/bin/sh' doesn't point to Bash and this can cause problems and failures when the designer uses its features. Bash (and its extra features) make things very easy and it is very ubiquitous, so it is safe to assume users will have it. This problem was reported by Alejandro Serrano Borlaff.
2018-07-08Files with .mk suffix as Makefiles in EmacsMohammad Akhlaghi-0/+4
Since version 26.1, apparently Emacs doesn't tream `.mk' files in Makefile-mode. So thanks to Mosè Giordano, a line was added in the `.dir-locals.el' file so we can be sure they are always treated as Makefiles and make things more convenient for Emacs users.
2018-03-14Some further minor edits to new parts of README.mdMohammad Akhlaghi-14/+16
Going through the text, some further minor edits were made.
2018-03-14Minor edits in README.mdMohammad Akhlaghi-6/+9
Some minor edits to the newly added parts of `README.md'.
2018-03-14Added future plans (and Nix/Guix usage) in README.mdMohammad Akhlaghi-4/+51
A section was added to `README.md' for describing possible future steps that we can take to make this pipeline even more robust. In it, I added a first interesting thing that I think would be really exciting to add, but I don't have time to do now.
2018-03-13Updated link to slidesMohammad Akhlaghi-1/+1
The link to the slides describing the context was the old address (which still works, but is just a symbolic link to the new address). It was thus corrected to point to the proper filename.
2018-03-09Corrected name to new repository addressMohammad Akhlaghi-3/+3
The main online repository name has been changed to `reproducible-paper' from `reproduction-pipeline-template'. So the cloning command in `README.md' had to be corrected.
2018-03-09Minor edits in README.mdMohammad Akhlaghi-5/+6
Some minor corrections were made regarding the output repository.
2018-03-09Minor typo corrected in link in README.mdMohammad Akhlaghi-2/+2
The markup for the link to the final PDF in the `README.md' file was mistakenly written and is now corrected.
2018-03-09Added description of output templateMohammad Akhlaghi-10/+15
A template was made to keep the output of this pipeline as a demonstration and it is now added to the description in `README.md'.
2018-03-09Added tip for pipeline outputsMohammad Akhlaghi-3/+22
While doing my own project (which has grown to a processing time of about half an hour), I felt that it would be very convenient to a record of the outputs at major points also. But we don't want to bloat the pipeline by commiting PDF files or large datasets that get fully changed and are just by-products. So it occurred to me to have a separate pipeline only for outputs and after trying it out, it indeed seemds to be a good solution.
2018-03-04Edits in README.mdMohammad Akhlaghi-10/+9
Some futher edits were made to the paragraph describing the contents of `README.md' for a smoother reading.
2018-03-04Minor correction in README.mdMohammad Akhlaghi-7/+8
After the last commit, another minor correction is implemeneted to further simplify the reading.
2018-03-04Minor edits in README.mdMohammad Akhlaghi-51/+70
I reviewed the first few sections of `README.md' and made some small corrections to make it easier to understand/read.
2018-03-02Small edit in README.mdMohammad Akhlaghi-2/+1
A small edit was made at the start of `README.md' to make it easier to read.
2018-02-28Edited commends on \makepdfMohammad Akhlaghi-6/+4
The old comments could be a little confusing, so they are now more clear and describe where to look for how this variable is used.
2018-02-28Header for pages, new font and author managementMohammad Akhlaghi-30/+125
The computer modern font that was designed by Donald Knuth and is the default of LaTeX is indeed a very good, elegant and nice font in print. However, most journals choose the roman fonts and thus the computer modern font doesn't (subjectively) fit into the journal format nicely. So the default font of this pipeline's paper now uses LaTeX's `newtx' package for a roman style font. Also, a set of preamble settings were added to allow headers in the pages of the paper to make the result resemble more like a journal paper (familiar to the eye), while also adding important information. A new header was made for this job. This new header now also contains the title and author settings (after all, these are also a type of header). Finally, the LaTeX `authblk' package was used to organize authors and their affiliations.
2018-02-27Abstract added to final PDF along with better explanationMohammad Akhlaghi-25/+93
An abstract is also something most research reports will need, so a simple macro was defined to make it easy (not too many code lines within the text of the main body) to implement an abstract. The title was also moved up a little to better use the extra white space at the top of the page. Finally, the `\highlightchanges' along with its explanation (both as comments and within the text with examples) was added in `paper.tex' to demonstrate how useful the `\new' and `\tonote' macros are.
2018-02-27Multicol package no longer used for two column PDFMohammad Akhlaghi-23/+1
Until now, we were using the `multicol' package which is mainly designed for more than two columns. Instead, we are just passing a `twocolumn' option to the article document class.
2018-02-27Minor edits in README.mdMohammad Akhlaghi-4/+4
A few minor corrections were made in `README.md'.
2018-02-27More clear comments in paper.mkMohammad Akhlaghi-3/+7
The comments needed to be corrected to fit and explain the new logic (LaTeX being run within another directory).
2018-02-27Better comments on TeX preamblesMohammad Akhlaghi-13/+44
The comments in the preambles were made more clear and elaborate.
2018-02-27Cleaner TeX comments in paper.texMohammad Akhlaghi-8/+5
Having a look at the TeX source, some minor edits were made to the comments so it is more clear.
2018-02-27Default PDF now uses PGFPlots and BibLaTeXMohammad Akhlaghi-34/+374
Making plots and including references are integral parts of a scientific paper. Therefore to demonstrate how cleanly they can be used within the pipeline, they are now used to produce the final PDF. To use PGFPlots a random dataset is made (using AWK's random function) and is plotted using PGFPlots. The minimum and maximum values of the dataset are also included in the text to further show how such calculations can go into the macros and text. For the references, the NoiseChisel paper was added as a reference to cite when using this pipeline along with the MUSE UDF paper I, which uses this pipeline for two sections. Following this discussion, citation is also discussed in `README.md` and the NoiseChisel paper is also added as a published work with a reproduction pipeline.
2018-02-27Copyrights and TeX management made more clearMohammad Akhlaghi-33/+156
Until now, the copyright statement was left empty for the users of the pipeline to fill. However, the files have already been created and have an author (or contributing authors) before the user starts using the pipeline. So the original authors of the files are added along with the year. The user can add their own name to the existing files under the "Contributing author" when they start and they will be the "Original author" of the new files they create. Several changes were also made to the TeX management: - LaTeX is run within a `reproduce/build/tex/build' directory now. Not in the top reproduction pipeline directory. This helps keep all the auxiliary TeX files and directories in that directory and keep the top reproduction pipeline directory clean. After the final PDF is built, a copy is put in the top reproduction pipeline directory for easy viewing. - The PGFPlots preamble was also made more useful, allowing the name of the `.tex' file to also be the name of the final plot that is produced. This is a GREAT feature, because without it, the TiKZ externalization would be based on order of the plots within the paper. But now, order is irrelevant and we can even delete the TiKZ files within the processing workhorse-Makefiles so the plots are definitly rebuilt on the next run. - The paper is now in a two-column format to be more similar to published papers. A tip on debugging Make was added to `README.md'.