IMPORTANT: Python and some basic software updated, improved portabilityHEADmaneage

Summary: this commit has no effect on your project's analysis steps, just
check the list of changed Python software versions (if you used them).

Until now, the Python packages of Maneage had not been maintained for some
time, there were several portability issues: updates to the GNU C Library
needed an update to some of the basic software, and the latest macOS
features caused some crashes.

With this commit, along with the software version updates of [1], the
removed software of [2] and newly added software of [3], the following
issues in Maneage have been addressed:

 - README-hacking:
   - Published paper list: Added links to two published papers.
   - Publication checklist: added tips on how to squash the full LaTeX
     source of a Maneage'd projet (which includes many LaTeX files) into a
     single LaTeX file for some journals.
   - A new "Maneage Development Workflow" section has been added; see it
     for details.

 - paper.mk: empty macro files are not included in the built 'project.tex'
   since they have no effect and because they can cause problems with
   uploading to arXiv (which removes empty files).

 - verify.mk: when verification is activated:
   - When a macro file does not exist (can happen by mistake), an error is
     printed to remind the author.
   - Better dealing with empty macro files (no need to add their hash).

 - versions.conf: 26 Python packages that cannot currently be built with
   the new Python build system of Maneage have been separated and commented
   to cause an early crash if they are required; and encourage volunteers
   to help update them.

This commit is a git "squash" of contributions from the following people
 - Boud Roukema: updated/added Python packages and fixed many portability
   issues.
 - Faezeh Bidjarchian: fixed some portability issues on macOS.
 - Giacomo Lorenzetti: tested and fixed issues in newly added software.
 - Khalid Masum: fixed issue with order of LD_LIBRARY_PATH in 'basic.mk'.
 - Mohammad Akhlaghi: fixed basic software issues and reviewed edits for
   the merge.
 - Raul Infante-Sainz: fixed portability issues on macOS.
 - Sepideh Eskandarlou: helped in finding the verification fixes.

[1] Updated software

   Name              Group         Version (old -> new)
   ----              -----         --------------------
   GNU Coreutils     basic         9.9     ->  9.11
   GNU Gettext       basic         0.26    ->  1.0
   GNU libunistring  basic         1.4.1   ->  1.4.2
   GNU M4            basic         1.4.20  ->  1.4.21
   GNU Nano          basic         8.7     ->  9.0
   Ninja             high-level    1.12.1  ->  1.13.2
   OpenMPI           high-level    4.1.1   ->  5.0.10
   SWIG              high-level    4.0.2   ->  4.3.1
   Zip               high-level    3.0     ->  3.0-15
   Astropy           python        7.0.1   ->  7.1.0
   mpi4py            python        3.1.3   ->  4.0.3
   setuptools        python        75.8.0  ->  80.8.0

[2] Removed software: 'podlators' has been removed from basic software
because the necessary scripts that it provided are included in Perl's
default installation.

[3] New software:

   Name                Group         Version
   ----                -----         -------
   framel              high-level    8.47.3
   GNU Parallel        high-level    20260322
   Healpix             high-level    3.83-2024Nov13
   Metaio              high-level    8.5.0-a5da19d
   PCRE2               high-level    10.45
   SQLite              high-level    3.53.0
   TCL                 high-level    9.1a1
   TK                  high-level    9.1a1
   tzdb                high-level    2026a
   lalsuite            high-level    v7.26-15-g1e4c5e5080
   astropy-healpix     python        1.1.2
   colorama            python        0.4.6
   cosmodesi-mpytools  python        1.0.0-04369fc9
   exceptiongroup      python        1.3.0
   filt-scm            python        1.7.0
   hatchling           python        1.27.0
   hatch-vcs           python        0.5.0
   healpy              python        1.18.1
   joblib              python        1.5.0
   igwn-ligolw         python        2.1.0
   igwn-segments       python        2.1.0
   iniconfig           python        2.1.0
   ligo-skymap         python        2.4.0
   lscsoft-glue        python        4.1.0
   networkx            python        3.5
   pandas              python        2.2.3
   pathspec            python        0.12.1
   pluggy              python        1.6.0
   pyfftw              python        0.15.1
   pygments            python        2.19.2
   pytest              python        8.4.1
   python3-fitsio      python        1.3.0
   pytz                python        2025.2
   scikit-base         python        0.12.2
   scikit-learn        python        1.6.1
   threadpoolctl       python        3.6.0
   tqdm                python        4.67.1
   trove-classifiers   python        2025.5.9.12
   versioneer          python        0.29

5 files changed, 106 insertions, 73 deletions

diff --git a/reproduce/software/shell/apptainer.sh b/reproduce/software/shell/apptainer.sh
index 1785977..8760db6 100755
--- a/reproduce/software/shell/apptainer.sh
+++ b/reproduce/software/shell/apptainer.sh
@@ -195,13 +195,14 @@ done
 #
 # Make sure that the build directory is given and that it exists.
 if [ x$build_dir = x ]; then
-    printf "$scriptname: '--build-dir' not provided, this is the location "
-    printf "that all built analysis files will be kept on the host OS\n"
+    printf "$scriptname: '--build-dir' not provided, this is the "
+    printf "location that all built analysis files will be kept on "
+    printf "the host OS\n"
     exit 1;
 else
     if ! [ -d $build_dir ]; then
-        printf "$scriptname: '$build_dir' (value to '--build-dir') doesn't "
-        printf "exist\n"
+        printf "$scriptname: '$build_dir' (value to '--build-dir') "
+        printf "doesn't exist\n"
         exit 1;
     fi
 fi
@@ -364,7 +365,7 @@ From: $base_sif
      else \\
         printf "$scriptname: '\$maneage_apptainer_stat' (value "; \\
         printf "to 'maneage_apptainer_stat' environment variable) "; \\
-        printf "is not recognized: should be either 'shell' or 'run'"; \\
+        printf "is not recognized: should be either 'shell' or 'run'\n"; \\
         exit 1; \\
      fi; \\
   else \\
@@ -399,7 +400,7 @@ fi
 if ! [ x"$build_only" = x ]; then
     if [ $quiet = 0 ]; then
         printf "$scriptname: info: Maneaged project has been configured "
-        printf "successfully in the '$sif' image"
+        printf "successfully in the '$sif' image\n"
     fi
     exit 0
 fi
diff --git a/reproduce/software/shell/configure.sh b/reproduce/software/shell/configure.sh
index ca3bf6e..3e0163e 100755
--- a/reproduce/software/shell/configure.sh
+++ b/reproduce/software/shell/configure.sh
@@ -228,7 +228,7 @@ empty_build_tmp() {
     # accidentally delete the whole root partition (or a least the '/home'
     # partition of the user).
     if ! [ x"$( ls -A $tmpblddir )" = x ]; then
-        rm -r "$tmpblddir"/*
+        rm -rf "$tmpblddir"/*
     fi
     rm -r "$tmpblddir"
 }
@@ -378,7 +378,11 @@ if [ $built_container = 0 ]; then
         # between Intel or Apple M1 CPUs. Here we disinguish between Apple
         # M1 or others.
         maccputype=$(sysctl -n machdep.cpu.brand_string)
-        if [ x"$maccputype" = x"Apple M1" ]; then
+        if    [ x"$maccputype" = x"Apple M1" ] \
+           || [ x"$maccputype" = x"Apple M2" ] \
+           || [ x"$maccputype" = x"Apple M3" ] \
+           || [ x"$maccputype" = x"Apple M4" ] \
+           || [ x"$maccputype" = x"Apple M5" ] ; then
             address_size_physical=$(sysctl -n machdep.cpu.thread_count)
             address_size_virtual=$(sysctl -n machdep.cpu.logical_per_package)
         else
@@ -781,6 +785,25 @@ fi
 
 
 
+# Older C standard versions
+# -------------------------
+#
+# Some basic packages require an old standard of C compilation; for their
+# list, see the 'Not working with C23' list in '../config/versions.conf'.
+# Here, we first try 'gnu17', but if that fails on the host compiler, we
+# fall back to 'gnu99'.
+if [ $built_container = 0 ]; then
+   printf "int main(void){; return 0;}\n" > $testsource
+   if gcc -std=gnu17 $testsource -o $testprog 2> /dev/null; then
+       std_c_old="gnu17"
+   else std_c_old="gnu99"
+   fi
+   rm $testsource $testprog
+fi
+
+
+
+
 # Fortran compiler
 # ----------------
 #
@@ -856,33 +879,6 @@ fi
 
 
 
-# See if the linker accepts -Wl,-rpath-link
-# -----------------------------------------
-#
-# '-rpath-link' is used to write the information of the linked shared
-# library into the shared object (library or program). But some versions of
-# LLVM's linker don't accept it an can cause problems.
-#
-# IMPORTANT NOTE: This test has to be done **AFTER** the definition of
-# 'instdir', otherwise, it is going to be used as an empty string.
-if [ $built_container = 0 ]; then
-   cat > $testsource <<EOF
-#include <stdio.h>
-#include <stdlib.h>
-int main(void) {return EXIT_SUCCESS;}
-EOF
-   if $CC $testsource -o$testprog -Wl,-rpath-link 2>/dev/null \
-          > /dev/null; then
-       export rpath_command="-Wl,-rpath-link=$instdir/lib"
-   else
-       export rpath_command=""
-   fi
-
-   # Delete the temporary directory for compiler checking.
-   rm -f $testprog $testsource
-   rm -r $compilertestdir
-   elapsed_time_from_prev_step compiler-rpath
-fi
 
 
 
@@ -1426,9 +1422,9 @@ elapsed_time_from_prev_step downloader
 # by the algorithm in 'configure.sh' when evaluating the variable
 # 'sys_library_sh_path'. This leads to a fatal syntax error in (at least)
 # 'ncurses', because the option using 'sys_library_sh_path' contains an
-# unquoted RAM address in parentheses.  Even if the address were quoted, it
-# would still be incorrect. This 'grep command excludes candidate host path
-# strings that look like RAM addresses to address the problem.
+# unquoted RAM address in parentheses. Even if the address were quoted, it
+# would still be incorrect. This 'grep' command excludes candidate host
+# path strings that look like RAM addresses to address the problem.
 #
 # [1] https://savannah.nongnu.org/bugs/index.php?66847
 # [2] https://stackoverflow.com/questions/34428037/how-to-interpret-the-output-of-the-ldd-program
@@ -1550,6 +1546,41 @@ fi
 
 
 
+# See if the linker accepts -Wl,-rpath-link
+# -----------------------------------------
+#
+# '-rpath-link' is used to write the information of the linked shared
+# library into the shared object (library or program). But some versions of
+# LLVM's linker don't accept it an can cause problems.
+#
+# IMPORTANT NOTE: This test has to be done **AFTER** the definition of
+# 'instdir'. Otherwise, the rpath-link value set within rpath_command in
+# the case of a successful test compile (if $CC ...) will be "/lib",
+# i.e. the host system root /lib directory, instead of the maneage library
+# directory.
+if [ $built_container = 0 ]; then
+   cat > $testsource <<EOF
+#include <stdio.h>
+#include <stdlib.h>
+int main(void) {return EXIT_SUCCESS;}
+EOF
+   if $CC $testsource -o$testprog -Wl,-rpath-link 2>/dev/null \
+          > /dev/null; then
+       export rpath_command="-Wl,-rpath-link=$instdir/lib"
+   else
+       export rpath_command=""
+   fi
+
+   # Delete the temporary directory for compiler checking.
+   rm -f $testprog $testsource
+   rm -r $compilertestdir
+   elapsed_time_from_prev_step compiler-rpath
+fi
+
+
+
+
+
 # Software building directory (possibly in RAM)
 # ---------------------------------------------
 #
@@ -1833,6 +1864,7 @@ if [ $quiet = 0 ]; then echo "Building/validating software: basic"; fi
      numthreads=$numthreads \
      needs_ldl=$needs_ldl \
      on_mac_os=$on_mac_os \
+     std_c_old=$std_c_old \
      host_cc=$host_cc \
      -j$numthreads
 elapsed_time_from_prev_step make-software-basic
diff --git a/reproduce/software/shell/docker-README.md b/reproduce/software/shell/docker-README.md
index 9e09282..dd46f4f 100644
--- a/reproduce/software/shell/docker-README.md
+++ b/reproduce/software/shell/docker-README.md
@@ -35,28 +35,25 @@ software environment) to easily move it from one computer to another.
     systemctl start docker
     ```
 
- 2. Using your favorite text editor, create a `run.sh` in your top Maneage
-    directory (as described in the comments at the start of the `docker.sh`
-    script in this directory). Just activate `--build-only` on the first
-    run so it doesn't go onto doing the analysis and just sets up the
-    software environment. Set the respective directory(s) based on your
-    filesystem (the software directory is optional). The `run.sh` file name
-    is already in `.gitignore` (because it contains local directories), so
-    Git will ignore it and it won't be committed by mistake.
+ 2. At the start of the `docker.sh` script in this directory, you can see
+    the necessary command to run. Just activate `--build-only` on the first
+    run so it doesn't go onto doing the analysis (that can be done at any
+    later time). Set the respective directory(s) based on your filesystem
+    (the software directory is optional). You can put the command in a
+    `run.sh` at the top of the project (it is already in `.gitignore`:
+    because it contains local directories).
 
  3. After the setup is complete, remove the `--build-only` and run the
     command below to confirm that `maneage-base` (the OS of the container)
     and `maneaged` (your project's full Maneage'd environment) images are
     available. If you want different names for these images, add the
-    `--project-name` and `--base-name` options to the `docker.sh` call.
+    `--project-name` and `--base-name` options to the `docker.sh` call. You
+    can see the list of available images with this command:
 
     ```shell
     docker image list
     ```
 
- 4. You are now ready to do your analysis by removing the `--build-only`
-    option.
-
 
 
 
diff --git a/reproduce/software/shell/docker.sh b/reproduce/software/shell/docker.sh
index ab454aa..5cd8f11 100755
--- a/reproduce/software/shell/docker.sh
+++ b/reproduce/software/shell/docker.sh
@@ -1,4 +1,4 @@
-#!/bin/sh
+#!/usr/bin/env bash
 #
 # Create a Docker container from an existing image of the built software
 # environment, but with the source, data and build (analysis) directories
@@ -409,22 +409,25 @@ else
         # that the user gave there.
         mkdir $tmp_dir/$tsdir
         cp -r "$software_dir"/* $tmp_dir/$tsdir/
-        printf "    --mount=type=bind,source=$tsdir,target=$dts \x5C\n" >> $df
+        printf "    --mount=type=bind,source=$tsdir,target=$dts \x5C\n" \
+               >> $df
     fi
 
     # Construct the rest of the 'RUN' command.
     printf "    cp -r $dsr /home/maneager/source; \x5C\n"          >> $df
     printf "    cd /home/maneager/source; \x5C\n"                  >> $df
-    printf "    ./project configure --jobs=$jobs \x5C\n"           >> $df
+    printf "    ./project configure \x5C\n"                        >> $df
+    printf "              --no-pause \x5C\n"                       >> $df
+    printf "              --jobs=$jobs \x5C\n"                     >> $df
     printf "              --build-dir=$intbuild \x5C\n"            >> $df
     printf "              --input-dir=/home/maneager/input \x5C\n" >> $df
-    printf "              --software-dir=$dts; \x5C\n"             >> $df
+    printf "              --software-dir=$dts; \x5C\n"  >> $df
 
-    # We are deleting the '.build/software/tarballs' directory because this
-    # directory is not relevant for the analysis of the project. But in
-    # case any tarball was downloaded, it will consume space within the
-    # container.
-    printf "    rm -rf .build/software/tarballs; \x5C\n" >> $df
+    # We are deleting the contents of the '.build/software/tarballs'
+    # directory (tarballs of the sources of the software) because they are
+    # not relevant to the execution of the project and will consume space
+    # within the container.
+    printf "    rm -rf .build/software/tarballs/*; \x5C\n" >> $df
 
     # We are deleting the source directory becaues later (at 'docker run'
     # time), the 'source' will be mounted directly from the host operating
@@ -432,12 +435,9 @@ else
     printf "    cd /home/maneager; \x5C\n" >> $df
     printf "    rm -rf source\n" >> $df
 
-    # Build the Maneage container and delete the temporary directory. The
-    # '--progress plain' option is for Docker to print all the outputs
-    # (otherwise, it will only print a very small part!).
+    # Build the Maneage container and delete the temporary directory.
     cd $tmp_dir
     docker build ./ -t $project_name \
-           --progress=plain \
            --shm-size=$shm_size \
            --no-cache \
            2>&1 | tee build.log
@@ -453,14 +453,15 @@ if ! [ x"$image_file" = x ] && ! [ -f "$image_file" ]; then
     # Save the image into a tarball
     tarname=$(echo $image_file | sed -e's|.gz$||')
     if [ $quiet = 0 ]; then
-        printf "$scriptname: info: saving docker image to '$tarname'"
+        printf "$scriptname: info: saving docker image to '$tarname'\n"
     fi
     docker save -o $tarname $project_name
 
     # Compress the saved image
     if [ $quiet = 0 ]; then
         printf "$scriptname: info: compressing to '$image_file' (can "
-        printf "take +10 minutes, but volume decreases by more than half!)"
+        printf "take +10 minutes, but volume decreases by more than "
+        printf "half!)\n"
     fi
     gzip --best $tarname
 fi
@@ -470,7 +471,7 @@ fi
 if ! [ x"$build_only" = x ]; then
     if [ $quiet = 0 ]; then
         printf "$scriptname: info: Maneaged project has been configured "
-        printf "successfully in the '$project_name' image"
+        printf "successfully in the '$project_name' image\n"
     fi
     exit 0
 fi
diff --git a/reproduce/software/shell/prep-source.sh b/reproduce/software/shell/prep-source.sh
index 861f3bf..2959255 100755
--- a/reproduce/software/shell/prep-source.sh
+++ b/reproduce/software/shell/prep-source.sh
@@ -47,17 +47,18 @@ fi
 
 
 
-# Find all the files that contain the '/bin/sh' string and correct them to
-# Maneage's own Bash. We are using 'while read' to read the file names line
-# by line. This is necessary to account file names that include the 'SPACE'
-# character (happens in CMake for example!).
+# Find all the files that contain the '/bin/sh' or '/bin/bash' strings and
+# correct them to Maneage's own shell (Dash or Bash). We are using 'while
+# read' to read the file names line by line. This is necessary to account
+# file names that include the 'SPACE' character (happens in CMake for
+# example!).
 #
 # Note that dates are important in the source directory (files depend on
 # each other), so we should read the original date and after making. We are
 # also not using GNU SED's '-i' ('--in-place') option because the host OS
 # may not have GNU SED.
 #
-# Actual situation which prompted the addition of this step: a Maneage'd
+# The situation which prompted the addition of this step: a Maneage'd
 # project (with GNU Bash 5.1.8 and Readline 8.1.1) was being built on a
 # system where '/bin/sh' was GNU Bash 5.2.26 and had Readline 8.2.010. The
 # newer version of Bash needed the newer Readline library function(s) that
@@ -74,7 +75,8 @@ fi
 # hard-coded in the source code of almost all programs (their build
 # scripts); and in special programs like GNU Make, GNU M4 or CMake it is
 # actually hardcoded in the source code (not just build scripts).
-if [ -f "$bindir/bash" ]; then shpath="$bindir"/bash
+bashpath="$bindir"/bash
+if [ -f "$bindir/bash" ]; then shpath="$bashpath"
 else                           shpath="$bindir"/dash
 fi
 
@@ -91,7 +93,7 @@ fi
 # precision up to the seconds. We then use 'sed' to remove the information
 # regarding the timezone, as the format is not accepted by 'touch'.
 # LCTYPE and LANG are also required on macos systems by sed.
-grep -I -r -e'/bin/sh' $(pwd)/* \
+grep -I -r -e'/bin/sh' -e'/bin/bash' $(pwd)/* \
     | sed -e's|:|\t|' \
     | awk 'BEGIN{FS="\t"}{print $1}' \
     | sort \