Until this commit, once Libidn was installed, insted of its own name and
version, the name and version of Libjpeg were saved (in the target if
Libidn). This robably come from a copy/paste of the rule.

With this commit, this minor bug has been corrected. I also added my name
as an author of `reproduce/software/make/xorg.mk' Makefile since I added
some code there.

Until now, in order to build Ghostscript, the project used the host's Xorg
libraries. This was because we hadn't yet added the necessary build rules
for them.

With this commit, the instructions to build the necessary Xorg libraries
for Ghostscript have also been added. Also, the shared Ghostscript library
has been built with this commit and two sets of standard fonts are also
included, setting us on the path to build TeXLive from source later.

This task was done with the help and support of Raul Infante-Sainz.

Until this commit, there was a problem when building Bison in parallel in
macOS systems. With this commit, this problem has been fixed by updating
Bison to its most recent version (3.6).

POSSIBLE EFFECT ON YOUR PROJECT: The changes in this commit may only cause
conflicts to your project if you have changed the software building
Makefiles in your project's branch (e.g., 'basic.mk', 'high-level.mk' and
'python.mk'). If your project has only added analysis, it shouldn't be
affected.

This is a large commit, involving a long series of corrections in a
differnt branch which is now finally being merged into the core Maneage
branch. All changes were related and came up naturally as the low-level
infrastructure was improved. So separating them in the end for the final
merge would have been very time consuming and we are merging them as one
commit.

In general, the software building Makefiles are now much more easier to
read, modify and use, along with several new features that have been
added. See below for the full list.

 - Until now, Maneage needed the host to have a 'make' implementation
   because Make was necessary to build Lzip. Lzip is then used to
   uncompress the source of our own GNU Make. However, in the
   minimalist/slim versions of operating systems (for example used to build
   Docker images) Make isn't included by default. Since Lzip was the only
   program before our own GNU Make was installed, we consulting Antonio
   Diaz Diaz (creator of Lzip) and he kindly added the necessary
   functionality to a new version of Lzip, which we are using now. Hence we
   don't need to assume a Make implementation on the host any more. With
   this commit, Lzip and GNU Make are built without Make, allowing
   everything else to be safely built with our own custom version of GNU
   Make and not using the host's 'make' at all.

 - Until recently (Commit 3d8aa5953c4) GNU Make was built in
   'basic.mk'. Therefore 'basic.mk' was written in a way that it can be
   used with other 'make' implementations also (i.e., important shell
   commands starting with '&&' and ending in '\' without any comments
   between them!). Furthermore, to help in style uniformity, the rules in
   'high-level.mk' and 'python.mk' also followed a similar structure. But
   due to the point above, we can now guarantee that GNU Make is used from
   the very first Makefile, so this hard-to-read structure has been removed
   in the software build recipes and they are much more readable and
   edit-friendly now.

 - Until now, the default backup servers where at some fixed URLs, on our
   own pages or on Gitlab. But recently we uploaded all the necessary
   software to Zenodo (https://doi.org/10.5281/zenodo.3883409) which is
   more suitable for this task (it promises longevity, has a fixed DOI,
   while allowing us to add new content, or new software tarball
   versions). With this commit, a small script has been written to extract
   the most recent Zenodo upload link from the Zenodo DOI and use it for
   downloading the software source codes.

 - Until now, we primarily used the webpage of each software for
   downloading its tarball. But this caused many problems: 1) Some of them
   needed Javascript before the download, 2) Some URLs had a complex
   dependency on the version number, 3) some servers would be randomly down
   for maintenance and etc. So thanks to the point above, we now use the
   Zenodo server as the primary download location. However, if a user wants
   to use a custom software that is not (yet!) in Zenodo, the download
   script gives priority to a custom URL that the users can give as Make
   variables. If that variable is defined, then the script will use that
   URL before going onto Zenodo. We now have a special place for such URLs:
   'reproduce/software/config/urls.conf'. The old URLs (which are a good
   documentation themselves) are preserved here, but are commented by
   default.

 - The software source code downloading and checksum verification step has
   been moved into a Make function called 'import-source' (defined in the
   'build-rules.mk' and loaded in all software Makefiles). Having taken all
   the low-level steps there, I noticed that there is no more need for
   having the tarball as a separate target! So with this commit, a single
   rule is the only place that needs to be edited/added (greatly
   simplifying the software building Makefiles).

 - Following task #15272, A new option has been added to the './project'
   script called '--all-highlevel'. When this option is given, the contents
   of 'TARGETS.conf' are ignored and all the software in Maneage are built
   (selected by parsing the 'versions.conf' file). This new option was
   added to confirm the extensive changes made in all the software building
   recipes and is great for development/testing purposes.

 - Many of the software hadn't been tested for a long time! So after using
   the newly added '--all-highlevel', we noticed that some need to be
   updated. In general, with this commit, 'libpaper' and 'pcre' were added
   as new software, and the versions of the following software was updated:
   'boost', 'flex', 'libtirpc', 'openblas' and 'lzip'. A 'run-parts.in'
   shell script was added in 'reproduce/software/shell/' which is installed
   with 'libpaper'.

 - Even though we intentionally add the necessary flags to add RPATH inside
   the built executable at compilation time, some software don't do it
   (different software on different operating systems!). Until now, for
   historical reasons this check was done in different ways for different
   software on GNU/Linux sytems. But now it is unified: if 'patchelf' is
   present we apply it. Because of this, 'patchelf' has been put as a
   top-level prerequisite, right after Tar and is installed before anything
   else.

 - In 'versions.conf', GNU Libtool is recognized as 'libtool', but in
   'basic.mk', it was 'glibtool'! This caused many confusions and is
   corrected with this commit (in 'basic.mk', it is also 'libtool').

 - A new argument is added to the './project' script to allow easy loading
   of the project's shell and environment for fast/temporary testing of
   things in the same environment as the project. Before activating the
   project's shell, we completely remove all host environment variables to
   simulate the project's environment. It can be called with this command:
   './project shell'. A simple prompt has also been added to highlight that
   the user is using the Maneage shell!

Until this commit, the file `BDIR/software/preparation-done.mk' were not
removed when cleaning the project with `./project make clean'. This file
is generated in the preparation of the data during the analysis step.
However, the cleaning is expected to remove anything generated in the
analysis process! Step by step, with the commands:

./project make        --->  Will make the preparation and analysis
./project make clean  --->  Will remove all analysis outputs (but
                            not `preparation-done.mk')
./project make        --->  Won't do the preparation, only analysis!

However, in the last step it should do the preparation again, because
the input data could have change for any reason. With this commit, the
file `BDIR/software/preparation-done.mk' is removed when cleaning the
project, and consequently, in the analysis step the input data is
prepared.

In Commit 105467fe6402 (Software tarballs are downloaded even if not
built), we introduced tests to download the tarballs of software even if
they don't need to be built on the respective host. However some small
typos in the checks existed that could cause a crash on macOS. In
particular in the building of PatchELF and libbsd we had forgot to add the
necessary 'x' before the 'yes' in the conditional to check if a we are on
macOS or not.

With this commit these two checks have been corrected. Also, in the
building of 'isl' and 'mpc', we now check for 'host_cc' (signifying that
the user wants to use their host C compiler for the high-level step)
instead of 'on_mac_os'. The reason is that even on non-macOS systems, a
user may not want to build the C compiler from scratch and use the
'--host-cc' option. In such cases, they don't need to compile 'isl' and
'mpc'.

With this commit, Maneage now includes instructions to build the memory
tracing tool Valgrind and the program 'patch' (to apply corrections/patches
in text files and in particular the sources of programs).

For this version of Valgrind, some patches were necessary for an interface
with OpenMPI 2.x (which is the case now). Also note that this version of
Valgrind's checks can fail with GCC 10.1.x (when using '--host-cc'), and
the failures aren't due to internal problems but due to how the tests are
designed (https://bugs.gentoo.org/707598). So currently if any of
Valgrind's checks fail, Maneage still assumes that Valgrind was built and
installed successfully.

While testing on macOS, we noticed that it needs the macOS-specific 'mig'
program which we can't build in Maneage. DESCRIPTION: The mig command
invokes the Mach Interface Generator to generate Remote Procedure Call
(RPC) code for client-server style Mach IPC from specification files. So a
symbolic link to the system's 'mig' is now added to the project's programs
on macOS systems.

This commit's build of Patch and Valgrind has been tested on two GNU/Linux
distributions (Debian and ArchLinux) as well as macOS.

Work on this commit started by Boud Roukema, but also involved tests and
corrections by Mohammad Akhlaghi and Raul Infante-Sainz.

Until now Maneage used the host's GNU Gettext if it was present. Gettext is
a relatively low-level software that enables programs to print messages in
different languages based on the host environment. Even though it has not
direct effect on the running of the software for Maneage and the lanugage
environment in Maneage is pre-determined, it is necessary to have it
because if the basic programs see it in the host they will link with it and
will have problems if/when the host's Gettext is updated.

With this commit (which is actually a squashed rebase of 9 commits by Raul
and Mohammad), Gettext and its two extra dependencies (libxml2 and
libunistring) are now installed within Maneage as a basic software and
built before GNU Bash. As a result, all programs built afterwards will
successfully link with our own internal version of Gettext and
libraries. To get this working, some of the basic software dependencies had
to updated and re-ordered and it has been tested in both GNU/Linux and
macoS.

Some other minor issues that are fixed with this commit

 - Until this commit, when TeX was not installed, the warning message
   saying how to run the configure step in order to re-configure the
   project was not showing the option `-e'. However, the use of this option
   is more convenient than entering the top-build directory and etc every
   time. So with this commit, the warning message has been changed in order
   use the option `-e' in the re-configure of the project.

 - Until now, on macOS systems, Bash was not linking with our internally
   built `libncurses'. With this commit, this has been fixed by setting
   `--withcurses=yes' for Bash's configure script.

Until now, Astropy was instructed to build its own internal copy of the
Expat library. However, with the recent commits before, Maneage now
includes an installation of Expat and Astropy can't keep the two (its
internal version and the project's version) separate, so they conflict and
don't let Astropy get built.

With this commit, the problem is fixed by setting the Expat library as an
explicit dependency of Astropy and asking Astropy to ignore its internal
copy.

While doing this, I recognized that it is much easier and elegant to add
steps in various stages of the `pybuild' function through hooks instead of
variables. So the fifth argument of the `pybuild' function was removed and
now it actually checks if hooks are defined as functions and if so, they
will be called.

The `pyhook_after' function was also implemented in the installation of
`pybind11' (which needed it, given that the 5th argument of `pybuild' was
removed) and after doing a test-build, I noticed that two lines were not
ending with a `\' in `boost' (a dependency of `pybind11').

Commit written originally by Mohammad Akghlaghi

Until now we would simply return the version numbers as they were written
into the separate files and situations can happen where the version numbers
contain an underscore (`_'). However, this character is a methematical
character in LaTeX, causing LaTeX to complain and abort.

With this commit, a step has been added at the end of the configure script
to convert any possible `_' to `\_'. Once it is commented (a backslash is
put behind it), the underscore will be printed as it is in the final PDF.

This commit was originally written by Mohammad Akhlaghi