aboutsummaryrefslogtreecommitdiff
path: root/reproduce/src/make/download.mk
AgeCommit message (Collapse)AuthorLines
2018-12-03Added rpath in basic dependencies, remove input if download failsMohammad Akhlaghi-1/+2
Until now, we weren't including the `rpath' linking options to the basic dependencies. They are now added. Also, when the download of an input file fails for any reason, an empty file won't be replaced there any more.
2018-12-03Trusted CA certificates also downloaded for Wget usageMohammad Akhlaghi-1/+1
To enable easy downloading of HTTPS links with Wget (this pipeline's defaut downloader), we need a set of trusted CA certificates. Until the time that we can generate one ourselves, one generic set of trusted CA certificates is now downloaded like a tarball and placed in the OpenSSL configuration directory. With these CA certificates, within the pipeline we can now safely use the pipeline's own installed Wget.
2018-11-29File is made before calling flock (file locking)Mohammad Akhlaghi-0/+1
While testing on another computer, I noticed that to operate properly, the file given to `flock' must be created before it is called. This is a low-level difference (how the system treats files), so it wasn't apparent on my system. To fix it, we have added a `touch' command before it.
2018-11-29Remove extra lockdir target in download.mkMohammad Akhlaghi-1/+1
There was an extra `$(lockdir)' target in `download.mk'. This has been corrected.
2018-11-26Making lock file directoryMohammad Akhlaghi-1/+1
We had forgot to add the rule to build the lock file directory for downloading data. This has been corrected.
2018-11-25More generic naming conventionMohammad Akhlaghi-13/+21
Until now, we were keeping the input file within the reproduction pipeline's directories using the same name as the database/server. Now, we are using a short/summarized filename convention for the input dataset.
2018-11-25Pipeline now downloads and uses an input datasetMohammad Akhlaghi-13/+44
In most analysis situations (except for simulations), an input dataset is necessary, but that part of the pipeline was just left out and a general `SURVEY' variable was set and never used. So with this commit, we actually use a sample FITS file from the FITS standard webpage, show it (as well as its histogram) and do some basic calculations on it. This preparation of the input datasets is done in a generic way to enable easy addition of more datasets if necessary.
2018-02-27Copyrights and TeX management made more clearMohammad Akhlaghi-2/+3
Until now, the copyright statement was left empty for the users of the pipeline to fill. However, the files have already been created and have an author (or contributing authors) before the user starts using the pipeline. So the original authors of the files are added along with the year. The user can add their own name to the existing files under the "Contributing author" when they start and they will be the "Original author" of the new files they create. Several changes were also made to the TeX management: - LaTeX is run within a `reproduce/build/tex/build' directory now. Not in the top reproduction pipeline directory. This helps keep all the auxiliary TeX files and directories in that directory and keep the top reproduction pipeline directory clean. After the final PDF is built, a copy is put in the top reproduction pipeline directory for easy viewing. - The PGFPlots preamble was also made more useful, allowing the name of the `.tex' file to also be the name of the final plot that is produced. This is a GREAT feature, because without it, the TiKZ externalization would be based on order of the plots within the paper. But now, order is irrelevant and we can even delete the TiKZ files within the processing workhorse-Makefiles so the plots are definitly rebuilt on the next run. - The paper is now in a two-column format to be more similar to published papers. A tip on debugging Make was added to `README.md'.
2018-02-20Necessary programs checked at configure timeMohammad Akhlaghi-17/+1
The mandatory and optional (for example downloader) dependencies are now checked at configure time so users can know what they may be missing before the processing starts. Since its recommended to be run in parallel, it can be hard to find what you are missing after running the pipeline. As part of these checks, the program to use for downloading is now also set at configure time, it is only used as a pre-defined (in `LOCAL.mk') variable during Make's processing. A small title was also added to discus the pipeline architecture that will be filled in the next commit.
2018-02-07First commit to the reproduction pipeline templateMohammad Akhlaghi-0/+68
Let's start working on this pipeline independently with this first commit. It is based on my previous experiences, but I had never made a skeleton of a pipeline before, it was always within a working analysis. But now that the pipeline has a separate repository for its self, we will be able to work on it and use it as a base for future work and modify it to make it even better. Hopefully in time (and with the help of others), it will grow and become much more robust and useful.