First commit to the reproduction pipeline template

Let's start working on this pipeline independently with this first
commit. It is based on my previous experiences, but I had never made a
skeleton of a pipeline before, it was always within a working analysis.

But now that the pipeline has a separate repository for its self, we will
be able to work on it and use it as a base for future work and modify it to
make it even better. Hopefully in time (and with the help of others), it
will grow and become much more robust and useful.

4 files changed, 95 insertions, 0 deletions

diff --git a/reproduce/config/pipeline/DIRECTORIES.mk.in b/reproduce/config/pipeline/DIRECTORIES.mk.in
new file mode 100644
index 0000000..9ebd67b
--- /dev/null
+++ b/reproduce/config/pipeline/DIRECTORIES.mk.in
@@ -0,0 +1,50 @@
+# Top-level user specific directories. Note the points below:
+#
+# - The VALUES to these directories are initially JUST PLACE-HOLDERS!
+#   Please correct them based on your system.
+#
+# - The directories don't need to necessarily exist. If they do not exist,
+#   they will be created and the necessary data will be downloaded into
+#   them. Ofcourse provided that you have write permissions and an internet
+#   connection.
+#
+# - Do not use the tilde expansion `~' or variables for your home
+#   directory. Please use the full address, for example
+#   `/home/your-user-name'.
+#
+# - An ending forward-slash `/' is NOT necessary. In the pipeline, all
+#   these variables will be followed by a `/', so if you put a `/' at the
+#   end of the value here, you will see a `//' in the printed outputs
+#   during the processing. This has no technical problem, but can make
+#   reading the outputs harder and is thus not recommended.
+
+
+
+
+
+# Input data directories
+# ----------------------
+#
+# This is where the input data (with the same file-name standard as the
+# online webpage) are stored. If this directory doesn't exist, or it
+# doesn't contain the files (with the correct file-name formats), it will
+# be created and the images will be downloaded. See
+# `reproduce/config/pipeline/web.mk', for the URLs containing the expected
+# inputs for each survey.
+SURVEY = reproduce/SURVEY
+
+
+
+
+
+# Build directory
+# ---------------
+#
+# This is where the intermediate outputs of each step are kept.
+#
+# Why a separate build directory? So the source and configuration files for
+# this reproduction pipeline do not get crowded by all the
+# intermediate/derivative files. Also to make synchronization and backups
+# more easy: the contents of the build directory do not need to be backed
+# up since they can be reproduced and they can be large.
+BDIR = reproduce/BDIR
diff --git a/reproduce/config/pipeline/filters.mk b/reproduce/config/pipeline/filters.mk
new file mode 100644
index 0000000..6fa785d
--- /dev/null
+++ b/reproduce/config/pipeline/filters.mk
@@ -0,0 +1,25 @@
+# `filters' are the possible different parts of the survey, for
+# example filters in broad or narrow-band astronomical imaging
+# datasets. Since a generic term for them (to apply other types of
+# surveys/datasets) hasn't been considered yet, we'll stick with the
+# `filters' name. But feel free to correct it (or propose a
+# suggestion).
+#
+# If your dataset only has a single filter, or this concept is not
+# defined for your type of input dataset, you can ignore this
+# variable.
+#
+# The values can be any string to identify different parts of a survey
+# separated by white space characters (for example `f125w f160w' or `J
+# H' if you want to specify two filters).
+#
+# To be clean and also help in readability of the pipeline, it is good
+# practice to define a separate `filter-XXXX' variable for each
+# survey/dataset, even if they have overlapping filters.
+#
+# These `filters' are used in the initial downloading of the data and
+# it is good practice (for avoiding bugs) to keep the same filter (and
+# survey) names in the filenames of the intermediate/output files
+# also. This will make sure that the raw input and intermediate/final
+# output are exactly related.
+filters-survey = a b c d e f g h i
diff --git a/reproduce/config/pipeline/pdf.mk b/reproduce/config/pipeline/pdf.mk
new file mode 100644
index 0000000..51ab933
--- /dev/null
+++ b/reproduce/config/pipeline/pdf.mk
@@ -0,0 +1,14 @@
+# Make the final PDF?
+# -------------------
+#
+# During the testing a pipeline, it is usually not necessary to build
+# the PDF file (which makes a lot of output lines on the command-line
+# and can make it hard to find the commands and possible errors (and
+# their outputs). Also, in some cases, only the produced results may
+# be of interest and not the final PDF, so LaTeX (and its necessary
+# packages) may not be installed.
+#
+# If this variable is given any string, a PDF will be made with
+# LaTeX. Otherwise, a notice will just printed that for now, no PDF
+# will be created.
+pdf-compile = yes
diff --git a/reproduce/config/pipeline/web.mk b/reproduce/config/pipeline/web.mk
new file mode 100644
index 0000000..f80b886
--- /dev/null
+++ b/reproduce/config/pipeline/web.mk
@@ -0,0 +1,6 @@
+# Web server(s) hosting the input data for this pipeline.
+#
+# This is the web page containing the files that must be located in the
+# `SURVEY' directory of `reproduce/config/pipeline/DIRECTORIES.mk' on the
+# local system.
+web-survey = https://some.webpage.com/example/server