Imported recent updates in Maneage, conflicts fixed

Until now, Maneage had undergone some updates.

With this commit, those updates have been imported and the conflicts that
resulted were fixed. They were all cosmetic and had no effect on the
analysis. The most significant one was about the change in the format of
'INPUTS.conf'.

In the process, I also noticed that the IEEEtran LaTeX package is now
called 'ieeetran' (the 'tlmgr' of TeXLive 2022 was failing).

1 files changed, 274 insertions, 84 deletions

diff --git a/README.md b/README.md
index f361baa..7ef3e08 100644
--- a/README.md
+++ b/README.md
@@ -1,7 +1,7 @@
 Reproducible source for Akhlaghi et al. (2021, arXiv:2006.03018)
 ----------------------------------------------------------------
 
-Copyright (C) 2018-2021 Mohammad Akhlaghi <mohammad@akhlaghi.org>\
+Copyright (C) 2018-2022 Mohammad Akhlaghi <mohammad@akhlaghi.org>\
 See the end of the file for license conditions.
 
 This is the reproducible project source for the paper titled "**Toward
@@ -195,6 +195,55 @@ finally create the final paper).
 
 
 
+### Building on ARM
+
+As of 2021-10-13, very little testing of Maneage has been done on arm64
+(tested in [aarch64](https://en.wikipedia.org/wiki/AArch64)). However,
+_some_ testing has been done on [the
+PinePhone](https://en.wikipedia.org/wiki/PinePhone), running
+[Debian/Mobian](https://wiki.mobian-project.org/doku.php?id=pinephone). In
+principle default Maneage branch (not all high-level software have been
+tested) should run fully (configure + make) from the raw source to the
+final verified pdf. Some issues that you might need to be aware of are
+listed below.
+
+#### Older packages
+
+In old packages that may be still needed and that have an old
+`config.guess` file (e.g. from 2002, such as fftw2-2.1.5-4.2, that are not
+in the base Maneage branch) may crash during the build. A workaround is to
+provide an updated (e.g. 2018) 'config.guess' file (automake --add-missing
+--force-missing --copy) in 'reproduce/software/patches/' and copy it over
+the old file during the build of the package.
+
+#### An un-killable running job
+
+Vampires may be a problem on the pinephone/aarch64. A "vampire" is defined
+here as a job that is in the "R" (running) state, using nearly 95-100% of a
+cpu, for an extremely long time (hours), without producing any output to
+its log file, and is immune to being killed by the user or root with 'kill
+-9'. A reboot and relaunch of the './project configure --existing-conf'
+command is the only solution currently known (as of 2021-10-13) for
+vampires. These are known to have occurred with linux-image-5.13-sunxi64.
+
+
+#### RAM/swap space
+
+Adding atleast 3 Gb of swap space (man swapon, man mkswap, man dd) on the
+eMMC may help to reduce the chance of having errors due to the lack of RAM.
+
+
+#### Time scale
+
+On the PinePhone v1.2b, apart from the time wasted by vampires, expect
+roughly 24 hours' wall time in total for the full 'configure' phase. The
+default 'maneage' example calculations, diagrams and pdf production are
+light and should be very fast.
+
+
+
+
+
 ### Building in Docker containers
 
 Docker containers are a common way to build projects in an independent
@@ -258,8 +307,9 @@ MB), not the full TeXLive collection!
 
     ```shell
     FROM debian:stable-slim
-    RUN apt-get update && apt-get install -y gcc g++ wget
+    RUN apt update && apt install -y gcc g++ wget
     RUN useradd -ms /bin/sh maneager
+    RUN printf '123\n123' | passwd root
     USER maneager
     WORKDIR /home/maneager
     RUN mkdir build
@@ -292,22 +342,22 @@ MB), not the full TeXLive collection!
 
     ```shell
     # C and C++ compiler.
-    RUN apt-get update && apt-get install -y gcc g++
+    RUN apt update && apt install -y gcc g++
 
     # Uncomment this if you don't have 'software-XXXX.tar.gz' (below).
-    #RUN apt-get install -y wget
+    #RUN apt install -y wget
     ```
 
  3. **Define a user:** Some core software packages will complain if you try
     to install them as the default (root) user. Generally, it is also good
-    practice to avoid being the root user. After building the Docker image,
-    you can always run it as root with this command: `docker run -u 0 -it
-    XXXXXXX` (where `XXXXXXX` is the image identifier). Hence with the
-    commands below we define a `maneager` user and activate it for the next
-    steps.
+    practice to avoid being the root user. Hence with the commands below we
+    define a `maneager` user and activate it for the next steps. But just
+    in case root access is necessary temporarily, with the `passwd`
+    command, we are setting the root password to `123`.
 
     ```shell
     RUN useradd -ms /bin/sh maneager
+    RUN printf '123\n123' | passwd root
     USER maneager
     WORKDIR /home/maneager
     ```
@@ -466,6 +516,8 @@ docker cp CONTAINER:/file/path/within/container /host/path/target
 
 
 
+
+
 #### Only software environment in the Docker image
 
 You can set the docker image to only contain the software environment and
@@ -475,96 +527,234 @@ image to a minimum (only containing the built software environment) to
 easily move it from one computer to another. Below we'll summarize the
 steps.
 
-1. Get your user ID with this command: `id -u`.
+ 1.  Get your user ID with this command: `id -u`.
 
-2. Put the following lines into a `Dockerfile` of an otherwise empty
-directory. Just replacing `UID` with your user ID (found in the step
-above). This will build the basic directory structure. for the next steps.
+ 2.  Make a new (empty) directory called `docker` temporarily (will be
+     deleted later).
 
-```shell
-FROM debian:stable-slim
-RUN apt-get update && apt-get install -y gcc g++ wget
-RUN useradd -ms /bin/sh --uid UID maneager
-USER maneager
-WORKDIR /home/maneager
-RUN mkdir build
-```
+     ```shell
+     mkdir docker-tmp
+     cd docker-tmp
+     ```
 
-3. Create an image based on the `Dockerfile` above. Just replace `PROJECT`
-with your desired name.
+ 3.  Make a `Dockerfile` (within the new/empty directory) with the
+     following contents. Just replace `UID` with your user ID (found in
+     step 1 above). Note that we are manually setting the `maneager` (user)
+     password to `123` and the root password to '456' (both should be
+     repeated because they must be confirmed by `passwd`).
 
-```shell
-docker build -t PROJECT ./
-```
+     ```
+     FROM debian:stable-slim
+     RUN useradd -ms /bin/sh --uid UID maneager; \
+         printf '123\n123' | passwd maneager; \
+         printf '456\n456' | passwd root
+     USER maneager
+     WORKDIR /home/maneager
+     RUN mkdir build; mkdir build/analysis
+     ```
 
-4. Run the command below to create a container based on the image and mount
-the desired directories on your host into the special directories of your
-container. Just don't forget to replace `PROJECT` and set the `/PATH`s to
-the respective paths in your host operating system.
+ 4.  Create a Docker image based on the `Dockerfile` above. Just replace
+     `MANEAGEBASE` with your desired name (this won't be your final image,
+     so you can safely use a name like `maneage-base`). Note that you need
+     to have root/administrator previlages when running it, so
 
-```shell
-docker run -v /PATH/TO/PROJECT/SOURCE:/home/maneager/source \
-           -v /PATH/TO/PROJECT/ANALYSIS/OUTPUTS:/home/maneager/build/analysis \
-           -v /PATH/TO/SOFTWARE/SOURCE/CODE/DIR:/home/maneager/software \
-           -v /PATH/TO/RAW/INPUT/DATA:/home/maneager/data \
-           -it PROJECT
-```
+     ```shell
+     sudo docker build -t MANEAGEBASE ./
+     ```
 
-5. After running the command above, you are within the container. Go into
-the project source directory and run these commands to build the software
-environment.
+ 5.  You don't need the temporary directory any more (the docker image is
+     saved in Docker's own location, and accessible from anywhere).
 
-```shell
-cd /home/maneager/source
-./project configure --build-dir=/home/maneager/build \
-                    --software-dir=/home/maneager/software \
-                    --input-dir=/home/maneager/data
-```
+     ```shell
+     cd ..
+     rm -rf docker-tmp
+     ```
 
-6. After the configuration finishes successfully, it will say so and ask
-you to run `./project make`. But don't do that yet. Keep this Docker
-container open and don't exit the container or terminal. Open a new
-terminal, and follow the steps described in the sub-section above to
-preserve the built container as a Docker image. Let's assume you call it
-`PROJECT-ENV`. After the new image is made, you should be able to see the
-new image in the list of images with this command (in the same terminal
-that you created the image):
+ 6.  Put the following contents into a newly created plain-text file called
+     `docker-run`, while setting the mandatory variables based on your
+     system. The name `docker-run` is already inside Maneage's `.gitignore`
+     file, so you don't have to worry about mistakenly commiting this file
+     (which contains private information: directories in this computer).
 
-```shell
-docker image list      # In the other terminal.
-```
+     ```
+     #!/bin/sh
+     #
+     # Create a Docker container from an existing image of the built
+     # software environment, but with the source, data and build (analysis)
+     # directories directly within the host file system. This script should
+     # be run in the top project source directory (that has 'README.md' and
+     # 'paper.tex'). If not, replace the '$(pwd)' part with the project
+     # source directory.
+
+     # MANDATORY: Name of Docker container
+     docker_name=MANEAGEBASE
+
+     # MANDATORY: Location of "build" directory on this system (to host the
+     # 'analysis' sub-directory for output data products and possibly others).
+     build_dir=/PATH/TO/THIS/PROJECT/S/BUILD/DIR
+
+     # OPTIONAL: Location of project's input data in this system. If not
+     # present, a 'data' directory under the build directory will be created.
+     data_dir=/PATH/TO/THIS/PROJECT/S/DATA/DIR
+
+     # OPTIONAL: Location of software tarballs to use in building Maneage's
+     # internal software environment.
+     software_dir=/PATH/TO/SOFTWARE/TARBALL/DIR
+
+
+
+
+
+     # Internal proceessing
+     # --------------------
+     #
+     # Sanity check: Make sure that the build directory actually exists.
+     if ! [ -d $build_dir ]; then
+         echo "ERROR: '$build_dir' doesn't exist"; exit 1;
+     fi
+
+     # If the host operating system has '/dev/shm', then give Docker access
+     # to it also for improved speed in some scenarios (like configuration).
+     if [ -d /dev/shm ]; then shmopt="-v /dev/shm:/dev/shm";
+     else                     shmopt=""; fi
+
+     # If the 'analysis' and 'data' directories (that are mounted), don't exist,
+     # then create them (otherwise Docker will create them as 'root' before
+     # creating the container, and we won't have permission to write in them.
+     analysis_dir="$build_dir"/analysis
+     if ! [ -d $analysis_dir ]; then mkdir $analysis_dir; fi
+
+     # If the data or software directories don't exist, put them in the build
+     # directory (they will remain empty, but this helps in simplifiying the
+     # mounting command!).
+     if ! [ -d $data_dir ]; then
+         data_dir="$build_dir"/data
+         if ! [ -d $data_dir ]; then mkdir $data_dir; fi
+     fi
+     if ! [ -d $software_dir ]; then
+         software_dir="$build_dir"/tarballs-software
+         if ! [ -d $software_dir ]; then mkdir $software_dir; fi
+     fi
+
+     # Run the Docker image while setting up the directories.
+     sudo docker run -v "$software_dir":/home/maneager/tarballs-software \
+                     -v "$analysis_dir":/home/maneager/build/analysis \
+                     -v "$data_dir":/home/maneager/data \
+                     -v "$(pwd)":/home/maneager/source \
+                     $shmopt -it $docker_name
+     ```
 
-7. Now you can run `./project make` in the initial container. You will see
-that all the built products (temporary or final datasets or PDFs), will be
-written in the `/PATH/TO/PROJECT/ANALYSIS/OUTPUTS` directory of your
-host. You can even change the source of your project on your host operating
-system an re-run Make to see the effect on the outputs and add/commit the
-changes to your Git history within your host. You can also exit the
-container any time. You can later load the `PROJECT-ENV` environment image
-into a new container with the same `docker run -v ...` command above, just
-use `PROJECT-ENV` instead of `PROJECT`.
+ 7.  Make the `docker-run` script executable.
 
-8. In case you want to store the image as a single file as backup or to
-move to another computer, you can run the commands below. They will produce
-a single `project-env.tar.gz` file.
+     ```shell
+     chmod +x docker-run
+     ```
 
-```shell
-docker save -o project-env.tar PROJECT-ENV
-gzip --best project-env.tar
-```
+ 8.  You can now start the Docker image by executing your newly added
+     script like below (it will ask for your root password). You will
+     notice that you are in the Docker container with the changed prompt.
+
+     ```shell
+     ./docker-run
+     ```
+
+ 9.  You are now within the container. First, we'll add the GNU C and C++
+     compilers (which are necessary to build our own programs in Maneage)
+     and the GNU WGet downloader (which may be necessary if you don't have
+     a core software's tarball already). Maneage will build pre-defined
+     versions of both and will use them. But for the very first packages,
+     they are necessary. In the process, by setting the `PS1` environment
+     variable, we'll define a color-coding for the interactive shell prompt
+     (red for root and purple for the user).
+
+     ```shell
+     su
+     echo 'export PS1="[\[\033[01;31m\]\u@\h \W\[\033[32m\]\[\033[00m\]]# "' >> ~/.bashrc
+     source ~/.bashrc
+     apt update
+     apt install -y gcc g++ wget
+     exit
+     echo 'export PS1="[\[\033[01;35m\]\u@\h \W\[\033[32m\]\[\033[00m\]]$ "' >> ~/.bashrc
+     source ~/.bashrc
+     ```
+
+ 10. Now that the compiler is ready, we can start Maneage's
+     configuration. So let's go into the project source directory and run
+     these commands to build the software environment.
+
+     ```shell
+     cd source
+     ./project configure --input-dir=/home/maneager/data \
+                         --build-dir=/home/maneager/build \
+                         --software-dir=/home/maneager/tarballs-software
+     ```
+
+ 11. After the configuration finishes successfully, it will say so. It will
+     then ask you to run `./project make`. **But don't do that
+     yet**. Keep this Docker container open and don't exit the container or
+     terminal. Open a new terminal, and follow the steps described in the
+     sub-section above to preserve (or "commit") the built container as a
+     Docker image. Let's assume you call it `MY-PROJECT-ENV`. After the new
+     image is made, you should be able to see the new image in the list of
+     images with this command (in yet another terminal):
+
+     ```shell
+     docker image list      # In the other terminal.
+     ```
+
+ 12. Now that you have safely "committed" your current Docker container
+     into a separate Docker image, you can **exit the container** safely
+     with the `exit` command. Don't worry, you won't loose the built
+     software environment: it is all now saved separately within the Docker
+     image.
+
+ 13. Re-open your `docker-run` script and change `MANEAGEBASE` to
+     `MY-PROJECT-ENV` (or any other name you set for the environment you
+     committed above).
+
+     ```shell
+     emacs docker-run
+     ```
+
+ 14. That is it! You can now always easily enter your container (only for
+     the software environemnt) with the command below. Within the
+     container, any file you save/edit in the `source` directory of the
+     docker container is the same file on your host OS and any file you
+     build in your `build/analysis` directory (within the Maneage'd
+     project) will be on your host OS. You can even use your container's
+     Git to store the history of your project in your host OS. See the next
+     step in case you want to move your built software environment to
+     another computer.
+
+     ```shell
+     ./docker-run
+     ```
+
+ 15. In case you want to store the image as a single file as backup or to
+     move to another computer, you can run the commands below. They will
+     produce a single `project-env.tar.gz` file.
+
+     ```shell
+     docker save -o my-project-env.tar MY-PROJECT-ENV
+     gzip --best project-env.tar
+     ```
+
+ 16. To load the tarball above into a clean docker environment (for example
+     on another system) copy the `my-project-env.tar.gz` file there and run
+     the command below. You can then create the `docker-run` script for
+     that system and run it to enter. Just don't forget that if your
+     `analysis_dir` directory is empty on the new/clean system. So you
+     should first run the same `./project configure ...` command above in
+     the docker image so it connects the environment to your source. Don't
+     worry, it won't build any software and should finish in a second or
+     two. Afterwards, you can safely run `./project make` and continue
+     working like you did on the old system.
+
+     ```shell
+     docker load --input my-project-env.tar.gz
+     ```
 
-9. To load the tarball above into a clean docker environment (either on the
-same system or in another system), and create a new container from the
-image like above (the `docker run -v ...` command). Just don't forget that
-if your `/PATH/TO/PROJECT/ANALYSIS/OUTPUTS` directory is empty on the
-new/clean system, you should first run `./project configure -e` in the
-docker image so it builds the core file structure there. Don't worry, it
-won't build any software and should finish in a second or two. Afterwards,
-you can safely run `./project make`.
 
-```shell
-docker load --input project-env.tar.gz
-```