aboutsummaryrefslogtreecommitdiff
path: root/README.md
diff options
context:
space:
mode:
authorMohammad Akhlaghi <mohammad@akhlaghi.org>2018-11-19 11:58:53 +0000
committerMohammad Akhlaghi <mohammad@akhlaghi.org>2018-11-19 12:27:38 +0000
commit5ae1fdc475d4476c6371fc5b27880634782549d9 (patch)
treed2ab279fdba416e65b6d44b4dad0225ab1ebec63 /README.md
parentfc7fd31a178a6094a3289c1e49ea7e7ac409e452 (diff)
Gzip's tarball in tar.gz instead of tar.lz
Until now, we were using a customized `tar.lz' tarball for Gzip. But on systems that don't have GNU Tar, this will cause a problem (non-GNU Tar doesn't recognize `.tar.lz'). So to keep things simple, we are using the customized gzip in `tar.gz' format. After the internal build of GNU Tar and Lzip, the default method of unpacking (`tar xf XXXXX.tar.XX') will work nicely on all the standard compression algorithms and we don't have to modify our commands based on the algorithm (nice feature of GNU Tar).
Diffstat (limited to 'README.md')
-rw-r--r--README.md63
1 files changed, 38 insertions, 25 deletions
diff --git a/README.md b/README.md
index 19da1df..985ad3b 100644
--- a/README.md
+++ b/README.md
@@ -2,7 +2,8 @@ Introduction
============
This description is for *creators* of the reproduction pipeline. See
-`README` for instructions on running it.
+`README` for instructions on running it (in short, just download/clone it,
+then run `./configure` and `./.local/bin/make -j8`).
This project contains a **fully working template** for a high-level
research reproduction pipeline, or reproducible paper, as defined in the
@@ -238,7 +239,8 @@ In order to adopt this pipeline to your research, it is important to first
understand its architecture so you can navigate your way in the directories
and understand how to implement your research project within its
framework. But before reading this theoretical discussion, please run the
-pipeline without any change, just to see how it works.
+pipeline (described in `README`: first run `./configure`, then
+`./.local/bin/make -j8`) without any change, just to see how it works.
In order to obtain a reproducible result it is important to have an
identical environment (for example same versions the programs that it will
@@ -246,17 +248,25 @@ use). This also has the added advantage that in your separate research
projects, you can use different versions of a single software and they
won't interfere. Therefore, the pipeline builds its own dependencies during
the `./configure` step. Building of the dependencies is managed by
-`reproduce/src/make/dependencies.mk`. So later, if you add a new
-program/library for your processing, don't forget to include a rule on how
-to build it, in this file.
-
-When you run `.local/bin/make` to start the processing, the first file that
-is read is the top-level `Makefile`. Therefore, we'll start our
-navigation/discussion with this file. This file is relatively short and
-heavily commented so hopefully the descriptions in each comment will be
-enough to understand the general details. As you read this section, please
-also look at the contents of the mentioned files and directories to fully
-understand what is being said.
+`reproduce/src/make/dependencies-basic.mk` and
+`reproduce/src/make/dependencies.mk`. These Makefiles are called by the
+`./configure` script. The first is intended for downloading and building
+the most basic tools like GNU Bash, GNU Make, and GNU Tar. Therefore it
+must only contain very basic and portable Make and shell features. The
+second is called after the first, thus enabling usage of the modern and
+advanced features of GNU Bash and GNU Make, similar to the rest of the
+pipeline. Later, if you add a new program/library for your research, you
+will need to include a rule on how to download and build it (in
+`reproduce/src/make/dependencies.mk`).
+
+After configuring, the `.local/bin/make` command will start the processing
+with the custom version of Make that was locally installed during
+configuration. The first file that is read is the top-level
+`Makefile`. Therefore, we'll start our navigation/discussion with this
+file. This file is relatively short and heavily commented so hopefully the
+descriptions in each comment will be enough to understand the general
+details. As you read this section, please also look at the contents of the
+mentioned files and directories to fully understand what is going on.
Before starting to look into the top `Makefile`, it is important to recall
that Make defines dependencies by files. Therefore, the input and output of
@@ -283,18 +293,21 @@ completely (to define dependencies and etc) and starts its execution after
that. So it is fine to define the rule to build `paper.pdf` at a later
stage (this is one beauty of Make!).
-Having defined the top target, we will include all the necessary
-Makefiles. First we include all `reproduce/config/pipeline/*.mk`. The
-configuration of each logical step of the pipeline is placed here as a
-separate file. These Makefiles must only contain raw Make variables
-(pipeline configurations). By raw we mean that the Make variables in these
-files must not depend on any other variables because we don't want to
-assume any order in reading them. It is very important to *not* define any
-rule or other Make construct in any of these _configuration-Makefiles_ (see
-the next paragraph for Makefiles with rules). This will enable you to set
-the respective files in this directory as a prerequisite to any target that
-depends on their variable values. Therefore, if you change any of the
-values, all targets that depend on those values will be re-built.
+Having defined the top target, our next step is to include all the other
+necessary Makefiles. First we include all Makefiles that satisfy this
+wildcard: `reproduce/config/pipeline/*.mk`. These Makefiles don't actually
+have any rules, they just have values for various free parameters
+throughout the pipeline. Open a few of them to see for your self. These
+Makefiles must only contain raw Make variables (pipeline
+configurations). By raw we mean that the Make variables in these files must
+not depend on variables in any other Makefile. This is because we don't
+want to assume any order in reading them. It is very important to *not*
+define any rule or other Make construct in any of these
+_configuration-Makefiles_ (see the next paragraph for Makefiles with
+rules). This will enable you to set the respective Makefiles in this
+directory as a prerequisite to any target that depends on their variable
+values. Therefore, if you change any of their values, all targets that
+depend on those values will be re-built.
Once all the raw variables have been imported into the top Makefile, we are
ready to import the Makefiles containing the details of the processing