diff options
| author | Mohammad Akhlaghi <mohammad@akhlaghi.org> | 2018-02-20 14:27:27 +0100 |
|---|---|---|
| committer | Mohammad Akhlaghi <mohammad@akhlaghi.org> | 2018-02-20 14:38:19 +0100 |
| commit | 8ba0292cd9299e415bc9c2c2a3307d61177f0cf5 (patch) | |
| tree | 5eade2f5d05269e767eecf8d9a1fd601c502d7c4 /reproduce | |
| parent | f21104afffe7a5cb28f7824211f90dc778d45b57 (diff) | |
Necessary programs checked at configure time
The mandatory and optional (for example downloader) dependencies are now
checked at configure time so users can know what they may be missing before
the processing starts. Since its recommended to be run in parallel, it can
be hard to find what you are missing after running the pipeline. As part of
these checks, the program to use for downloading is now also set at
configure time, it is only used as a pre-defined (in `LOCAL.mk') variable
during Make's processing.
A small title was also added to discus the pipeline architecture that will
be filled in the next commit.
Diffstat (limited to 'reproduce')
| -rw-r--r-- | reproduce/config/pipeline/LOCAL.mk.in | 14 | ||||
| -rw-r--r-- | reproduce/src/make/download.mk | 18 |
2 files changed, 15 insertions, 17 deletions
diff --git a/reproduce/config/pipeline/LOCAL.mk.in b/reproduce/config/pipeline/LOCAL.mk.in index ac8e10e..7a29344 100644 --- a/reproduce/config/pipeline/LOCAL.mk.in +++ b/reproduce/config/pipeline/LOCAL.mk.in @@ -68,3 +68,17 @@ BDIR = reproduce/BDIR # be defined here. This value will be used in the programs that support # this feature. MINMAPSIZE = 1000000000 + + + + + +# Downloader program +# ------------------ +# +# The downloder program (and its output option name) that will be used if +# any of the necessary datasets aren't already available on the +# system. This is usually set at an early stage of the configuration system +# automatically before the file is opened for editing by the user. It is +# thus recommended to not modify it manually. +DOWNLOADER = @downloader@ diff --git a/reproduce/src/make/download.mk b/reproduce/src/make/download.mk index 244bd04..378e5eb 100644 --- a/reproduce/src/make/download.mk +++ b/reproduce/src/make/download.mk @@ -24,22 +24,6 @@ -# Identify the downloader tool -# ---------------------------- -# -# If cURL is already present, that will be used, otherwise, we'll use -# Wget. Since the options specifying the output filename are different -# between the two, we'll also specify the output option within the -# `downloader' variable. So it is important to first give the output -# filename after calling `downloader', then the web address. -downloader := $(shell if type curl > /dev/null; then downloader="curl -o"; \ - else downloader="wget -O"; \ - fi; echo "$$downloader"; ) - - - - - # Download SURVEY data # -------------------- # @@ -52,7 +36,7 @@ all-survey = $(foreach f, $(filters-survey), \ $(SURVEY)/a-possibly-additional-$(f)-format.fits ) $(SURVEY):; mkdir $@ $(all-survey): $(SURVEY)/%: | $(SURVEY) $(lockdir) - flock $(lockdir)/download -c "$(downloader) $@ $(web-survey)/$*" + flock $(lockdir)/download -c "$(DOWNLOADER) $@ $(web-survey)/$*" |