From 3c63f0febef7c3c8b41ffdd6b7033b1f77f6e21d Mon Sep 17 00:00:00 2001
From: Mohammad Akhlaghi <mohammad@akhlaghi.org>
Date: Sun, 14 Jun 2020 01:55:33 +0100
Subject: Better explanation in the start of project configuration

When './project configure' is run, after the basic checks of the compiler,
a small statement is printed telling the user that some configuration
questions will now be asked to start building Maneage on the system. Until
now this description was confusing: it lead the reader to think that the
local configuration (which was recommended to read before continuing) is in
another file.

With this commit, the text has been edited to explictly mention that the
description of the steps following this notice should be read
carefully. Thus avoiding that confusion.

This issue was mentioned by Michael R. Crusoe.
---
 reproduce/software/shell/configure.sh | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

(limited to 'reproduce/software')

diff --git a/reproduce/software/shell/configure.sh b/reproduce/software/shell/configure.sh
index 789ddd5..e4885d6 100755
--- a/reproduce/software/shell/configure.sh
+++ b/reproduce/software/shell/configure.sh
@@ -569,9 +569,13 @@ cat <<EOF
 Project's local configuration
 -----------------------------
 
-Local configuration includes things like top-level directories, or
-processing steps. It is STRONGLY recommended to read the comments, and set
-the best values for your system (where necessary).
+Below, some basic local settings will be requested to start building
+Maneage on this system (if they haven't been specified on the
+command-line). This includes the top-level directories that Maneage will
+use on your system. Most are only optional and you can simply press ENTER,
+without giving any value (in this case, Maneage will download the necessary
+components from pre-defined webpages). It is STRONGLY recommended to read
+the description above each question before answering it.
 
 EOF
 
-- 
cgit v1.2.1


From f4a44b575d58fd6ddc5ed731ca3c8bd308dac6e9 Mon Sep 17 00:00:00 2001
From: Mohammad Akhlaghi <mohammad@akhlaghi.org>
Date: Sun, 14 Jun 2020 22:09:36 +0100
Subject: Better description for input data directory, pointing to INPUTS.conf

Until now, the description of the input-data directory at configure time
included a description of the input data (created by reading the values of
'INPUTS.conf'). Maintaining this is easy for a single dataset, but it
becomes hard for a general project which may need many input datasets.

To avoid extra complexity (for maintaining this list), the description now
points a user of the project to the 'INPUTS.conf' file and asks them to
look inside of it for seeing the necessary data. This infact helps with the
users becoming familiar with the internal structure of Maneage and will
allow the authors to focus on not having to worry about updating the
low-level 'configure.sh' script.
---
 reproduce/software/shell/configure.sh | 32 +++++++++++++-------------------
 1 file changed, 13 insertions(+), 19 deletions(-)

(limited to 'reproduce/software')

diff --git a/reproduce/software/shell/configure.sh b/reproduce/software/shell/configure.sh
index e4885d6..d9509ca 100755
--- a/reproduce/software/shell/configure.sh
+++ b/reproduce/software/shell/configure.sh
@@ -782,10 +782,6 @@ if [ x"$input_dir" = x ]; then
 else
     indir=$input_dir
 fi
-wfpc2name=$(awk '!/^#/ && $1=="WFPC2IMAGE" {print $3}' $adir/INPUTS.conf)
-wfpc2md5=$(awk  '!/^#/ && $1=="WFPC2MD5"   {print $3}' $adir/INPUTS.conf)
-wfpc2size=$(awk '!/^#/ && $1=="WFPC2SIZE"  {print $3}' $adir/INPUTS.conf)
-wfpc2url=$(awk  '!/^#/ && $1=="WFPC2URL"   {print $3}' $adir/INPUTS.conf)
 if [ $rewritepconfig = yes ] && [ x"$input_dir" = x ]; then
     cat <<EOF
 
@@ -793,26 +789,24 @@ if [ $rewritepconfig = yes ] && [ x"$input_dir" = x ]; then
 (OPTIONAL) Input dataset directory
 ----------------------------------
 
-This project needs the dataset(s) listed below. If you already have them,
-please specify the directory hosting them on this system. If you don't,
-they will be downloaded automatically. Each file is shown with its total
-volume and its 128-bit MD5 checksum in parenthesis.
+This project needs the dataset(s) listed in the following file:
 
-  $wfpc2name ($wfpc2size, $wfpc2md5):
-    A 100x100 Hubble Space Telescope WFPC II image used in the FITS
-    standard webpage as a demonstration of this file format.
-    URL: $wfpc2url/$wfpc2name
+      reproduce/analysis/config/INPUTS.conf
 
-NOTE I: This directory, or the datasets above, are optional. If it doesn't
-exist, the files will be downloaded in the build directory and used.
+If you already have a copy of them on this system, please specify the
+directory hosting them on this system. If they aren't present, they will be
+downloaded automatically when necessary.
 
-NOTE II: This directory (if given) will only be read, nothing will be
-written into it, so no writing permissions are necessary.
+NOTE I: This directory is optional. If not given, or if the files can't be
+found inside it, any necessary file will be downloaded directly in the
+build directory and used.
+
+NOTE II: If a directory is given, it will be used as read-only. Nothing
+will be written into it, so no writing permissions are necessary.
 
 TIP: If you have these files in multiple directories on your system and
-don't want to download them or make duplicates, you can create symbolic
-links to them and put those symbolic links in the given top-level
-directory.
+don't want to make duplicates, you can create symbolic links to them and
+put those symbolic links in the given top-level directory.
 
 EOF
     read -p"(OPTIONAL) Input datasets directory ($indir): " inindir
-- 
cgit v1.2.1


From e763790464563755d176ba798d40cb7c74f0d094 Mon Sep 17 00:00:00 2001
From: Mohammad Akhlaghi <mohammad@akhlaghi.org>
Date: Mon, 15 Jun 2020 01:42:10 +0100
Subject: Configure script now accounts for non-interactive shells

The project configuration requires a build-directory at configuration time,
two other directories can optionally be given to avoid downloading the
project's necessary data and software. It is possible to give these three
directories as command-line options, or by interactively giving them after
running the configure script.

Until now, when these directories weren't given as command-line options,
and the running shell was non-interactive, the configure script would crash
on the line trying to interactively read the user's given directories (the
'read' command).

With this commit, all the 'read' commands for these three directories are
now put within an 'if' statement. Therefore, when 'read' fails (the shell
is non-interactive), instead of a quiet crash, a descriptive message is
printed, telling the user that cause of the problem, and suggesting a fix.

This bug was found by Michael R. Crusoe.
---
 reproduce/software/shell/configure.sh | 39 +++++++++++++++++++++++++++++------
 1 file changed, 33 insertions(+), 6 deletions(-)

(limited to 'reproduce/software')

diff --git a/reproduce/software/shell/configure.sh b/reproduce/software/shell/configure.sh
index d9509ca..b71ea96 100755
--- a/reproduce/software/shell/configure.sh
+++ b/reproduce/software/shell/configure.sh
@@ -712,7 +712,14 @@ EOF
     do
         # Ask the user (if not already set on the command-line).
         if [ x"$build_dir" = x ]; then
-            read -p"Please enter the top build directory: " build_dir
+            if read -p"Please enter the top build directory: " build_dir; then
+                just_a_place_holder_to_avoid_not_equal_test=1;
+            else
+                echo "ERROR: shell is in non-interactive-mode and no build directory specified."
+                echo "The build directory (described above) is mandatory, configuration can't continue."
+                echo "Please use '--build-dir' to specify a build directory non-interactively."
+                exit 1
+            fi
         fi
 
         # If it exists, see if we can write in it. If not, try making it.
@@ -782,6 +789,7 @@ if [ x"$input_dir" = x ]; then
 else
     indir=$input_dir
 fi
+noninteractive_sleep=2
 if [ $rewritepconfig = yes ] && [ x"$input_dir" = x ]; then
     cat <<EOF
 
@@ -809,7 +817,17 @@ don't want to make duplicates, you can create symbolic links to them and
 put those symbolic links in the given top-level directory.
 
 EOF
-    read -p"(OPTIONAL) Input datasets directory ($indir): " inindir
+    # Read the input directory if interactive mode is enabled.
+    if read -p"(OPTIONAL) Input datasets directory ($indir): " inindir; then
+        just_a_place_holder_to_avoid_not_equal_test=1;
+    else
+        echo "WARNING: interactive-mode seems to be disabled!"
+        echo "If you have a local copy of the inputs, use '--input-dir'."
+        echo "... project configuration will continue in $noninteractive_sleep sec ..."
+        sleep $noninteractive_sleep
+    fi
+
+    # In case an input-directory is given, write it in 'indir'.
     if [ x$inindir != x ]; then
         indir=$inindir
         echo " -- Using '$indir'"
@@ -836,15 +854,24 @@ if [ $rewritepconfig = yes ] && [ x"$software_dir" = x ]; then
 
 To ensure an identical build environment, the project will use its own
 build of the programs it needs. Therefore the tarball of the relevant
-programs are necessary. If a tarball isn't present in the specified
-directory, *IT WILL BE DOWNLOADED* automatically.
+programs are necessary.
 
 If you don't specify any directory here, or it doesn't contain the tarball
-of a dependency, it is necessary to have an internet connection. The
+of a dependency, it is necessary to have an internet connection because the
 project will download the tarballs it needs automatically.
 
 EOF
-    read -p"(OPTIONAL) Directory of dependency tarballs ($ddir): " tmpddir
+    # Read the software directory if interactive mode is enabled.
+    if read -p"(OPTIONAL) Directory of dependency tarballs ($ddir): " tmpddir; then
+        just_a_place_holder_to_avoid_not_equal_test=1;
+    else
+        echo "WARNING: interactive-mode seems to be disabled!"
+        echo "If you have a local copy of the software source, use '--software-dir'."
+        echo "... project configuration will continue in $noninteractive_sleep sec ..."
+        sleep $noninteractive_sleep
+    fi
+
+    # If given, write the software directory.
     if [ x"$tmpddir" != x ]; then
         ddir=$tmpddir
         echo " -- Using '$ddir'"
-- 
cgit v1.2.1


From 0b49fb07ec3363495868cb5012b4ab74c2e7f2b7 Mon Sep 17 00:00:00 2001
From: Mohammad Akhlaghi <mohammad@akhlaghi.org>
Date: Mon, 15 Jun 2020 03:26:29 +0100
Subject: OpenSSL now built after Perl

After trying a clean build of Maneage in a Docker image (with a minimal
debian:stable-20200607-slim OS), I noticed that the building of OpenSSL is
failing because it doesn't find the proper Perl functionality. To fix it,
with this commit, Perl is set as a prerequisite of OpenSSL and this fixed
the problem.
---
 reproduce/software/make/basic.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

(limited to 'reproduce/software')

diff --git a/reproduce/software/make/basic.mk b/reproduce/software/make/basic.mk
index b4745e2..82bc42d 100644
--- a/reproduce/software/make/basic.mk
+++ b/reproduce/software/make/basic.mk
@@ -805,7 +805,7 @@ $(idir)/etc:; mkdir $@
 # Note: cert.pm has to be AFTER the tarball, otherwise the build script
 # will try to unpack cert.pm and crash (it unpacks the first dependency
 # under `tdir').
-$(ibidir)/openssl: $(ibidir)/tar \
+$(ibidir)/openssl: $(ibidir)/perl \
                    $(tdir)/openssl-$(openssl-version).tar.gz \
                    $(tdir)/cert.pem \
                    | $(idir)/etc
-- 
cgit v1.2.1


From 6a52c4ee9d2ee8b5723fc9fac4ebc97b77357613 Mon Sep 17 00:00:00 2001
From: Mohammad Akhlaghi <mohammad@akhlaghi.org>
Date: Tue, 16 Jun 2020 21:45:04 +0100
Subject: XLSX I/O properly accounts for local build

Until now, when adding the necessary library flags to the build of XLSX
I/O, we were effectively over-writing the 'LDFLAGS' variables. So the
compiler was effectively not being told where to look for the necessary
libraries.

With this commit, to fix the problem, we now append the new linking flags
to LDFLAGS in XLSX I/O's build, not over-write it.
---
 reproduce/software/make/high-level.mk | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

(limited to 'reproduce/software')

diff --git a/reproduce/software/make/high-level.mk b/reproduce/software/make/high-level.mk
index 7cc2d51..75fa8ac 100644
--- a/reproduce/software/make/high-level.mk
+++ b/reproduce/software/make/high-level.mk
@@ -1224,9 +1224,9 @@ $(ibidir)/xlsxio: $(ibidir)/cmake \
 	if [ x$(on_mac_os) = xyes ]; then \
 	  export CC=clang; \
 	  export CXX=clang++; \
-	  export LDFLAGS="-lbz2"; \
+	  export LDFLAGS="$$LDFLAGS -lbz2"; \
 	else \
-	  export LDFLAGS="-lbz2 -lbsd"; \
+	  export LDFLAGS="$$LDFLAGS -lbz2 -lbsd"; \
 	fi; \
 	$(call cbuild, xlsxio-$(xlsxio-version), static, \
 	       -DMINIZIP_DIR:PATH=$(idir) \
-- 
cgit v1.2.1