aboutsummaryrefslogtreecommitdiff
path: root/README-hacking.md
diff options
context:
space:
mode:
authorMohammad Akhlaghi <mohammad@akhlaghi.org>2020-01-23 18:54:29 +0000
committerMohammad Akhlaghi <mohammad@akhlaghi.org>2020-01-23 19:16:16 +0000
commit2264c0fea5668cd8f827cdbd89004fd390b0b14f (patch)
tree99af64b55557bfb69ea630a653bb8bf07586c63a /README-hacking.md
parentcf0427009decbcad33a269898a428341e9c0ceff (diff)
IMPORTANT: Project preparation is now also done with project make
Until now, the main commands to run the project were these: `./project configure' (to build the software), `./project prepare' (to possibly arrange input datasets and build special configuration Makefiles) and finally `./project make' to run the project. The main logic behind the "prepare" phase `top-prepare.mk' is to build configuration files that can be fed into the "make" step and optimize its operation. For example when the total number of necessary inputs for the majority of the analysis is not as large as the total number of inputs. With "prepare" (when necessary), you go through the raw inputs, select the ones that are necessary for the rest of the project. The output of `top-prepare.mk' is a configuration file (a Make variable) that keeps the IDs (numbers, names, etc). That configuration file would then be used in the `top-make.mk' to identify the lower level targets and allow optimal project organization and management. But the last two are both part of the analysis, and while they indeed need different calls to Make to be executed, many projects don't actually need a preparation phase: ultimately, its an implementation choice by the project developers and doesn't concern the project users (or the developers when they are running it). To avoid confusing the users, or simply annoying them when a projet doesn't need it, with this commit, the top-level `top-prepare.mk' and `top-make.mk' Makefiles are called with the single `./project make' command and `./project prepare' has been dropped. I noticed this while writing the paper on this system.
Diffstat (limited to 'README-hacking.md')
-rw-r--r--README-hacking.md33
1 files changed, 12 insertions, 21 deletions
diff --git a/README-hacking.md b/README-hacking.md
index 83c341b..324ada4 100644
--- a/README-hacking.md
+++ b/README-hacking.md
@@ -279,28 +279,20 @@ example if you run `.local/bin/ls` you will be using the `ls` of the
template, which is problably different from your system's `ls` (run them
both with `--version` to check).
-Once the project is configured for your system, `./project prepare` and
-`./project make` will do the basic preparations and run the project's
-analysis with the custom version of software. The `project` script is just
-a wrapper, and with the commands above, it will call `top-prepare.mk` and
-`top-make.mk` (both are in the `reproduce/analysis/make` directory).
-
-In the template, no particular preparation is necessary, so it will
-immediately finish and instruct you to run `./project make`. But in some
-projects, it can be very useful to do some very basic preparatory steps on
-the input data that can greatly optimize running of `./project make`. For
-example, you may need to query a server, to find how many input files there
-are. Once that number is known in the preparation phase, `./project make`
-can parallelize the analysis much more effectively.
+Once the project is configured for your system, `./project make` will do
+the basic preparations and run the project's analysis with the custom
+version of software. The `project` script is just a wrapper, and with the
+`make` argument, it will first call `top-prepare.mk` and `top-make.mk`
+(both are in the `reproduce/analysis/make` directory).
In terms of organization, `top-prepare.mk` and `top-make.mk` have an
-identical design, only a minor difference. So, let's continue the
-template's architecture with `top-make.mk`. Once you understand that,
-you'll clearly understand `top-prepare.mk` also. These very high-level
-files are relatively short and heavily commented so hopefully the
-descriptions in each comment will be enough to understand the general
-details. As you read this section, please also look at the contents of the
-mentioned files and directories to fully understand what is going on.
+identical design, only minor differences. So, let's continue the template's
+architecture with `top-make.mk`. Once you understand that, you'll clearly
+understand `top-prepare.mk` also. These very high-level files are
+relatively short and heavily commented so hopefully the descriptions in
+each comment will be enough to understand the general details. As you read
+this section, please also look at the contents of the mentioned files and
+directories to fully understand what is going on.
Before starting to look into the top `top-make.mk`, it is important to
recall that Make defines dependencies by files. Therefore, the
@@ -587,7 +579,6 @@ First custom commit
```shell
$ ./project configure # Build the project's software environment (can take an hour or so).
- $ ./project prepare # Pre-processing preparations (doing nothing in the raw template).
$ ./project make # Do the processing and build paper (just a simple demo in the template).
# Open 'paper.pdf' and see if everything is ok.