diff options
author | Mohammad Akhlaghi <mohammad@akhlaghi.org> | 2020-01-23 18:54:29 +0000 |
---|---|---|
committer | Mohammad Akhlaghi <mohammad@akhlaghi.org> | 2020-01-23 19:16:16 +0000 |
commit | 2264c0fea5668cd8f827cdbd89004fd390b0b14f (patch) | |
tree | 99af64b55557bfb69ea630a653bb8bf07586c63a /README-hacking.md | |
parent | cf0427009decbcad33a269898a428341e9c0ceff (diff) |
IMPORTANT: Project preparation is now also done with project make
Until now, the main commands to run the project were these: `./project
configure' (to build the software), `./project prepare' (to possibly
arrange input datasets and build special configuration Makefiles) and
finally `./project make' to run the project.
The main logic behind the "prepare" phase `top-prepare.mk' is to build
configuration files that can be fed into the "make" step and optimize its
operation. For example when the total number of necessary inputs for the
majority of the analysis is not as large as the total number of
inputs. With "prepare" (when necessary), you go through the raw inputs,
select the ones that are necessary for the rest of the project. The output
of `top-prepare.mk' is a configuration file (a Make variable) that keeps
the IDs (numbers, names, etc). That configuration file would then be used
in the `top-make.mk' to identify the lower level targets and allow optimal
project organization and management.
But the last two are both part of the analysis, and while they indeed need
different calls to Make to be executed, many projects don't actually need a
preparation phase: ultimately, its an implementation choice by the project
developers and doesn't concern the project users (or the developers when
they are running it).
To avoid confusing the users, or simply annoying them when a projet doesn't
need it, with this commit, the top-level `top-prepare.mk' and `top-make.mk'
Makefiles are called with the single `./project make' command and
`./project prepare' has been dropped. I noticed this while writing the
paper on this system.
Diffstat (limited to 'README-hacking.md')
-rw-r--r-- | README-hacking.md | 33 |
1 files changed, 12 insertions, 21 deletions
diff --git a/README-hacking.md b/README-hacking.md index 83c341b..324ada4 100644 --- a/README-hacking.md +++ b/README-hacking.md @@ -279,28 +279,20 @@ example if you run `.local/bin/ls` you will be using the `ls` of the template, which is problably different from your system's `ls` (run them both with `--version` to check). -Once the project is configured for your system, `./project prepare` and -`./project make` will do the basic preparations and run the project's -analysis with the custom version of software. The `project` script is just -a wrapper, and with the commands above, it will call `top-prepare.mk` and -`top-make.mk` (both are in the `reproduce/analysis/make` directory). - -In the template, no particular preparation is necessary, so it will -immediately finish and instruct you to run `./project make`. But in some -projects, it can be very useful to do some very basic preparatory steps on -the input data that can greatly optimize running of `./project make`. For -example, you may need to query a server, to find how many input files there -are. Once that number is known in the preparation phase, `./project make` -can parallelize the analysis much more effectively. +Once the project is configured for your system, `./project make` will do +the basic preparations and run the project's analysis with the custom +version of software. The `project` script is just a wrapper, and with the +`make` argument, it will first call `top-prepare.mk` and `top-make.mk` +(both are in the `reproduce/analysis/make` directory). In terms of organization, `top-prepare.mk` and `top-make.mk` have an -identical design, only a minor difference. So, let's continue the -template's architecture with `top-make.mk`. Once you understand that, -you'll clearly understand `top-prepare.mk` also. These very high-level -files are relatively short and heavily commented so hopefully the -descriptions in each comment will be enough to understand the general -details. As you read this section, please also look at the contents of the -mentioned files and directories to fully understand what is going on. +identical design, only minor differences. So, let's continue the template's +architecture with `top-make.mk`. Once you understand that, you'll clearly +understand `top-prepare.mk` also. These very high-level files are +relatively short and heavily commented so hopefully the descriptions in +each comment will be enough to understand the general details. As you read +this section, please also look at the contents of the mentioned files and +directories to fully understand what is going on. Before starting to look into the top `top-make.mk`, it is important to recall that Make defines dependencies by files. Therefore, the @@ -587,7 +579,6 @@ First custom commit ```shell $ ./project configure # Build the project's software environment (can take an hour or so). - $ ./project prepare # Pre-processing preparations (doing nothing in the raw template). $ ./project make # Do the processing and build paper (just a simple demo in the template). # Open 'paper.pdf' and see if everything is ok. |