Housekeeping: some portability issues fixed; four software updates

Until now, there were several portability issues in Maneage:

 1. Maneage would crash on older operating systems (checked on Debian 6),
    where Wget didn't have the '--no-use-server-timestamps'.

 2. On a Linux kernel 2.6.32 (of the same Debian 6 above) some features in
    'util-linux' (like 'swapon' or 'libmount') wouldn't build and wouldn't
    let 'util-linux' complete. These features need root permissions to be
    useful, so the wouldn't be used in Maneage any way! But they wouldn't
    let Maneage get built

 3. The './project shell' command would still read the host's '~/.bashrc',
    letting the host environment leak-in to Maneage's interactive shell.

 4. The building of Flex 2.64 wouldn't complete due to a segmentation
    fault an Ubuntu, but NetPBM (which depends on Flex) would crash with a
    wrong usage of 'yyunput'. This had actually caused a non-update to
    Flex in a previous Maneage software update.

 5. The update Astrometry.net would assume SExtractor's executable name is
    'source-extractor'; causing a crash in usage. This forced the users to
    manually create a 'source-extractor' symbolic link in the '.local/bin'
    directory.

 6. The 'reproduce/software/shell/tarball-prepare.sh' script (that is used
    for making Maneage-standard tarballs) wouldn't accept option values
    with an '=' between the option name and value! It also didnt' print
    sufficiently informative messages and errors (for example it would say
    "skipping ..." (making the user think there is a problem!), but it was
    actually that the file already existed!

 7. The 'reproduce/analysis/make/prepare.mk' and
    'reproduce/analysis/make/verify.mk' Makefiles that needed to reject
    some of the 'makesrc' sub-Makefiles would simply substitute their names
    with nothing. But this would cause problems when the name is part of
    the name of another sub-Makefile.

 8. On the Debian 6 system mentioned above the raw 'df' command's output
    wasn't in the expected format; so Maneage would fail to properly detect
    the free space in the disk.

With these commit, all the issues above have been solved: for 1, A check
has been added to avoid using that option. For 2, those 'util-linux'
features have been disabled. For 3, the '--norc' and '--noprofile' options
have beed added to the call to Bash. For 4, see below. For 5, the symbolic
link is now automatically made with SExtractor. For 6, the option reading
components of that script have been fully re-written and more robust sanity
checks are also added, with more informative warnings. For 7, the 'subst'
function of Make was replaced with 'filter-out' and this fixed the
problem. For 8, 'df' is called with the '-P' option so it has a unified
format in all versions.

For 4, the versions of 'flex' and 'netpbm' have been updated. Since they
were the dependency of 'astrometrynet', that has also been updated. In the
process, we discovered that 'lzip' has a new version which claims to be
faster, so that is also updated.

       lzip            1.22     -->   1.23
       astrometrynet   0.85     -->   0.89
       flex            2.6.4    -->   2.6.4-410-74a89fd
       netpbm          10.73.39 -->   10.73.39

NetPBM needed some manual manipulation in its source (to remove the extra
line), so the necessary steps have been added to its build recipe in
'reproduce/software/make/high-level.mk'.

2 files changed, 92 insertions, 41 deletions

diff --git a/reproduce/software/shell/configure.sh b/reproduce/software/shell/configure.sh
index 07381e6..7f5f9c4 100755
--- a/reproduce/software/shell/configure.sh
+++ b/reproduce/software/shell/configure.sh
@@ -191,7 +191,7 @@ free_space_warning()
 {
     fs_threshold=$1
     fs_destpath="$2"
-    return $(df "$fs_destpath" \
+    return $(df -P "$fs_destpath" \
                 | awk 'FNR==2 {if($4>'$fs_threshold') print 1; \
                                else                   print 0; }')
 }
@@ -814,9 +814,22 @@ if [ $rewritepconfig = yes ]; then
         # 'which' isn't in POSIX, so we are using 'command -v' instead.
         name=$(command -v wget)
 
+        # See if the host wget has the '--no-use-server-timestamps' option
+        # (for example wget 1.12 doesn't have it). If not, we'll have to
+        # remove it. This won't affect the analysis of Maneage in anyway,
+        # its just to avoid re-downloading if the server timestamps are
+        # bad; at the worst case, it will just cause a re-download of an
+        # input software source code (for data inputs, we will use our own
+        # wget that has this option).
+        tsname="no-use-server-timestamps"
+        tscheck=$(wget --help | grep $tsname || true)
+        if [ x"$tscheck" = x ]; then wgetts=""
+        else                         wgetts="--$tsname";
+        fi
+
         # By default Wget keeps the remote file's timestamp, so we'll have
         # to disable it manually.
-        downloader="$name --no-use-server-timestamps -O";
+        downloader="$name $wgetts -O";
     elif type curl > /dev/null 2>/dev/null; then
         name=$(command -v curl)
 
diff --git a/reproduce/software/shell/tarball-prepare.sh b/reproduce/software/shell/tarball-prepare.sh
index ccc9318..69e6afa 100755
--- a/reproduce/software/shell/tarball-prepare.sh
+++ b/reproduce/software/shell/tarball-prepare.sh
@@ -31,25 +31,21 @@ odir=
 idir=
 quiet=
 basedir=$PWD
+scriptname=$0
 
 
 # The --help output
 print_help() {
     cat <<EOF
-Usage: $0 [OPTIONS]
+Usage: $scriptname [OPTIONS]
 
 Low-level script to create maneage-standard tarballs.
-
   -o, --output-dir         Target directory to write the packed tarballs.
                            Current: $odir
-
-
   -i, --input-dir          Directory containing original tarballs.
                            Current: $idir
-
   -q, --quiet              Suppress logging information. Only print the
                            final packed file and its sha512sum.
-
 Maneage URL: https://maneage.org
 
 Report bugs: https://savannah.nongnu.org/bugs/?group=reproduce
@@ -59,45 +55,87 @@ EOF
 
 
 
+
+# Functions to check option values and complain if necessary.
+on_off_option_error() {
+    if [ x"$2" = x ]; then
+        echo "$scriptname: '$1' doesn't take any values"
+    else
+        echo "$scriptname: '$1' (or '$2') doesn't take any values"
+    fi
+    exit 1
+}
+
+check_v() {
+    if [ x"$2" = x ]; then
+        cat <<EOF
+$scriptname: option '$1' requires an argument. Try '$scriptname --help' for more information
+EOF
+        exit 1;
+    fi
+}
+
+option_given_and_valid() {
+    dirname="$1"
+    optionlong="$2"
+    optionshort="$3"
+    if [ x"$dirname" = x ]; then
+	cat <<EOF
+$scriptname: no '--$optionlong' (or '-$optionshort') given: use this for identifying the directory containing the input tarballs
+EOF
+	exit 1
+    else
+	dirname=$(echo "$dirname" | sed 's|/$||'); # Remove possible trailing slash
+	if [ ! -d "$dirname" ]; then
+	    cat <<EOF
+$scriptname: '$dirname' that is given to '--$optionlong' (or '-$optionshort') couldn't be opened
+EOF
+	    exit 1
+	else
+	    outdir=$(realpath $dirname)
+	fi
+    fi
+    ogvout=$outdir
+}
+
+
+
+
+
 # Parse the arguments
 while [ $# -gt 0 ]
 do
-  case $1 in
-      -q|--quiet)      quiet=1; shift;;
-      -h|--help|-'?')  print_help; exit 0;;
-      -i|--input-dir)
-          # Remove the trailing '/' introduced by autocomplete
-          idir=$(echo "$2" | sed 's|/$||');
-          shift;  # past argument
-          shift;; # past value
-      -o|--output-dir)
-          # Remove the trailing '/' introduced by autocomplete
-          odir=$(echo "$2" | sed 's|/$||');
-          shift;  # past argument
-          shift;; # past value
-      *)  echo "$0: unknown option '$1'"; exit 1;;
+    case $1 in
+	# Input and Output directories
+        -i|--input-dir)      idir="$2";                           check_v "$1" "$idir"; shift;shift;;
+        -i=*|--input-dir=*)  idir="${1#*=}";                      check_v "$1" "$idir"; shift;;
+        -i*)                 idir=$(echo "$1" | sed -e's/-i//');  check_v "$1" "$idir"; shift;;
+        -o|--output-dir)     odir="$2";                           check_v "$1" "$odir"; shift;shift;;
+        -o=*|--output-dir=*) odir="${1#*=}";                      check_v "$1" "$odir"; shift;;
+        -o*)                 odir=$(echo "$1" | sed -e's/-o//');  check_v "$1" "$odir"; shift;;
+
+	# Operating mode options
+        -?|--help)        print_help; exit 0;;
+        -'?'*|--help=*)   on_off_option_error --help -?;;
+        -q|--quiet)       quiet=1; shift;;
+        -q*|--quiet=*)    on_off_option_error --quiet -q;;
+	*)  echo "$scriptname: unknown option '$1'"; exit 1;;
   esac
 done
 
 
 
 
-# Extract the 'absolute path' to input and output directories. Working with
-# relative path is a great source of confusion and unwanted side-effects
-# like moving/removing files by accident.
-if [ ! -d "$idir" ]; then
-    echo "$0: please pass the input directory (option --input-dir or -i)."
-    exit 1
-else
-    idir=$(realpath $idir)
-fi
 
-if [ ! -d "$odir" ]; then
-    echo "$0: please pass the output directory (option --output-dir or -o)."
-    exit 1
-else
-    odir=$(realpath $odir)
-fi
+# Basic sanity checks
+#
+# Make sure the input and output directories are given. Also extract
+# the absolute path to input and output directories and remove any
+# possible trailing '/'. Working with a relative path is a great
+# source of confusion and unwanted side-effects like moving/removing
+# files by accident.
+option_given_and_valid "$idir" "input-dir"  "i" && idir=$ogvout
+option_given_and_valid "$odir" "output-dir" "o" && odir=$ogvout
 
 
 
@@ -117,15 +155,15 @@ fi
 for f in $allfiles; do
 
     # Seperate name and version number
-    name=$(echo $f | sed -e 's/.tar.*//' | \
-                     awk 'BEGIN { FS = "[-_ ]" } {print $1 "-" $2}')
+    name=$(echo $f | sed -e 's/.tar.*//' \
+               | awk 'BEGIN { FS = "[-_ ]" } {print $1 "-" $2}')
 
     # Skip previously packed files
     if [ -f $odir/$name.tar.lz ]; then
 
         # Print the info message if not in quiet mode
         if [ -z $quiet ]; then
-            echo "$0: skipping '$odir/$name.tar.lz'"
+            echo "$scriptname: $odir/$name.tar.lz: already present in output directory"
         fi
 
         # skip this file
@@ -134,7 +172,7 @@ for f in $allfiles; do
 
         # Print the info message if not in quiet mode
         if [ -z $quiet ]; then
-            echo "$0: processing '$idir/$f'"
+            echo "$scriptname: processing '$idir/$f'"
         fi
     fi