diff options
| author | Mohammad Akhlaghi <mohammad@akhlaghi.org> | 2024-05-03 13:07:41 +0200 |
|---|---|---|
| committer | Mohammad Akhlaghi <mohammad@akhlaghi.org> | 2025-02-10 19:34:26 +0100 |
| commit | 3c9bf3aff30f02c7d31bd86f36c4db2520f8ffa4 (patch) | |
| tree | c4b6ee74518ae5256626f6888c2ceccfa66ce2a2 | |
| parent | 8161194d68665648b93389adb333e741ba230497 (diff) | |
Configuration: no dependency on /bin/sh and useful run-time options
SUMMARY: no change necessary in your project, this commit only involves
changes in how already-existing software are built. Some handy options have
also been added to the top-level project script and the copyright years
have been updated.
Until now, if the host's '/bin/sh' had conflicts with the Maneage
environment, the configuration of Maneage would crash as soon as we entered
the building of high-level software. The full scenario is described in the
comments of the newly added 'reproduce/software/shell/prep-source.sh'. This
is most relevant when building older Maneage'd project in newer
environments.
With this commit, the following changes were made to avoid the problem
above:
- Maneage edits the source code of all installed software to replace
'/bin/sh' with Maneage's own shell before the programs are
built. Through this, we were able to solve the problem described
above.
- The portable '#!/usr/bin/env sh' shebangs are now used at the start of
the scripts that run during configure time so it uses the first
available shell that it finds in its PATH (the system's before Dash is
built), then Dash, and after Dash is built, Bash.
- For TeXLive, since we don't install it from source, it was necessary to
add the libraries necessary for the local '/bin/sh' in LD_LIBRARY_PATH.
Some high-level options have been added to the './project' script to
simplify certain operations:
--keep-going: do not stop upon the first crash, but keep going on to
build targets until all build-able targets have been built. This is
very useful for debugging large pipelines and allows you to isolate the
problematic part of your project.
--highlight-all: equivalent to calling both '--highlight-new' &
'--highlight-notes'.
74 files changed, 533 insertions, 216 deletions
diff --git a/README-hacking.md b/README-hacking.md index 8897333..4c9ff05 100644 --- a/README-hacking.md +++ b/README-hacking.md @@ -1,8 +1,8 @@ Maneage: managing data lineage ============================== -Copyright (C) 2018-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org>\ -Copyright (C) 2020-2023 Raul Infante-Sainz <infantesainz@gmail.com>\ +Copyright (C) 2018-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org>\ +Copyright (C) 2020-2025 Raul Infante-Sainz <infantesainz@gmail.com>\ See the end of the file for license conditions. Maneage is a **fully working template** for doing reproducible research (or @@ -762,8 +762,8 @@ First custom commit and generally, all the files you modified in the previous step. ``` - Copyright (C) 2018-2023 Existing Name <existing@email.address> - Copyright (C) 2023 YOUR NAME <YOUR@EMAIL.ADDRESS> + Copyright (C) 2018-2025 Existing Name <existing@email.address> + Copyright (C) 2024-2025 YOUR NAME <YOUR@EMAIL.ADDRESS> ``` 9. **Configure Git for fist time**: If this is the first time you are @@ -1,7 +1,7 @@ Reproducible source for XXXXXXXXXXXXXXXXX ------------------------------------------------------------------------- -Copyright (C) 2018-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org>\ +Copyright (C) 2018-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org>\ See the end of the file for license conditions. This is the reproducible project source for the paper titled "**XXX XXXXX @@ -1,4 +1,4 @@ -%% Copyright (C) 2018-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> +%% Copyright (C) 2018-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> %% See the end of the file for license conditions. \documentclass[10pt, twocolumn]{article} @@ -200,7 +200,7 @@ The IAC project P/300724, financed by the MCIU, through the Canary Islands Depar \appendix %% Mention all used software in an appendix. -\section{Software acknowledgement} +\section{Software acknowledgment} \label{appendix:software} \input{tex/build/macros/dependencies.tex} @@ -3,8 +3,8 @@ # High-level script to manage the project. # Run './project --help' for a description of how to use it. # -# Copyright (C) 2019-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> -# Copyright (C) 2021-2023 Raul Infante-Sainz <infantesainz@gmail.com> +# Copyright (C) 2019-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> +# Copyright (C) 2021-2025 Raul Infante-Sainz <infantesainz@gmail.com> # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -37,6 +37,7 @@ host_cc=0 operation= build_dir= input_dir= +keep_going= check_config= make_targets= software_dir= @@ -118,6 +119,7 @@ Configure options: Configure and Make options: -d, --debug[=FLAGS] In configure: use -j1, no -k, and no Zenodo check. In make: 'FLAGS' will be directly passed to 'make'. + -k, --keep-going Keep going when some targets can't be made. -g, --group=STR Build and run with write permissions for a group. -j, --jobs=INT Number of threads to build/run the software. -?, --help Print this help list. @@ -127,6 +129,7 @@ Make (analysis) options: Make (final PDF) options: --refresh-bib Force refresh the bibliography. + --highlight-all Activate '--highlight-new' & '--highlight-notes'. --highlight-new Highlight '\new' parts of text as green. --highlight-notes Show '\tonote' regions as red text in PDF. @@ -201,15 +204,20 @@ do -j|--jobs) jobs="$2"; check_v jobs "$jobs"; shift;shift;; -j=*|--jobs=*) jobs="${1#*=}"; check_v jobs "$jobs"; shift;; -j*) jobs=$(echo "$1" | sed -e's/-j//'); check_v jobs "$jobs"; shift;; + -k|--keep-going) keep_going="--keep-going"; shift;; + -k=*|--keep-going=*) on_off_option_error --keep-going -k;; + -k*) on_off_option_error --keep-going -k;; -'?'|--help) print_help; exit 0;; -'?'*|--help=*) on_off_option_error --help -?;; - # Make options - # ------------ - # - # Note that Make's 'debug' can take values, but when called without any - # value, it is like giving it a value of 'a'): + # Make options (analysis): + -p|--prepare-redo) prepare_redo=1; shift;; + -p=*|--prepare-redo=*) on_off_option_error --prepare-redo; shift;; + + # Make options (final PDF): --refresh-bib) [ -f tex/src/references.tex ] && touch tex/src/references.tex; shift;; + --highlight-all) highlightnew=1; highlightnotes=1; shift;; + --highlight-all=*) on_off_option_error --highlight-new;; --highlight-new) highlightnew=1; shift;; --highlight-new=*) on_off_option_error --highlight-new;; --highlight-notes) highlightnotes=1; shift;; @@ -218,15 +226,14 @@ do echo "Please set the operation before calling '--debug'"; exit 1 elif [ x$operation = xconfigure ]; then debug=a; shift; elif [ x$operation = xmake ]; then - if [ x"$2" = x ]; then echo "In make-mode, '--debug' needs a value"; exit 1 + if [ x"$2" = x ]; then + echo "In make-mode, '--debug' needs a value; see GNU Make manual"; exit 1 else debug="$2"; check_v debug "$debug"; shift;shift; fi else echo "Operation '$operation' not recognized, please use 'configure' or 'make'" fi;; -d=*|--debug=*) debug="${1#*=}"; check_v debug "$debug"; shift;; -d*) debug=$(echo "$1" | sed -e's/-d//'); check_v debug "$debug"; shift;; - -p|--prepare-redo) prepare_redo=1; shift;; - -p=*|--prepare-redo=*) on_off_option_error --prepare-redo; shift;; # Unrecognized option: -*) echo "$scriptname: unknown option '$1'"; exit 1;; @@ -390,6 +397,7 @@ controlled_env() { envmake="$envmake highlightnew=$highlightnew" envmake="$envmake highlightnotes=$highlightnotes .local/bin/make" envmake="$envmake --no-builtin-rules --no-builtin-variables -f $1" + envmake="$envmake $keep_going" if ! [ x"$debug" = x ]; then envmake="$envmake --debug=$debug"; fi # Set the number of jobs. Note that for the 'configure.sh' script the diff --git a/reproduce/analysis/bash/download-multi-try b/reproduce/analysis/bash/download-multi-try.sh index d7e9be2..bea88d5 100755 --- a/reproduce/analysis/bash/download-multi-try +++ b/reproduce/analysis/bash/download-multi-try.sh @@ -1,14 +1,19 @@ -#!/bin/sh +#!/usr/bin/env sh # # Attempt downloading multiple times before crashing whole project. From # the top project directory (for the shebang above), this script must be # run like this: # -# $ /path/to/download-multi-try downloader lockfile input-url downloaded-name +# $ $SHELL /path/to/download-multi-try.sh downloader lockfile \ +# input-url downloaded-name # -# NOTE: The 'downloader' must contain the option to specify the output name -# in its end. For example "wget -O". Any other option can also be placed in -# the middle. +# NOTE: +# - This script doesn't have a Shebang because in different stages it +# should be built with different shells ('/bin/sh' before Maneage +# installs its own shell and afterwards with Maneage's own shell). +# - The 'downloader' must contain the option to specify the output name +# in its end. For example "wget -O". Any other option can also be placed in +# the middle. # # Due to temporary network problems, a download may fail suddenly, but # succeed in a second try a few seconds later. Without this script that @@ -26,7 +31,7 @@ # reason, you don't want to use a lock file, set the 'lockfile' name to # 'nolock'. # -# Copyright (C) 2019-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> +# Copyright (C) 2019-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by diff --git a/reproduce/analysis/config/INPUTS.conf b/reproduce/analysis/config/INPUTS.conf index 1090e44..5860806 100644 --- a/reproduce/analysis/config/INPUTS.conf +++ b/reproduce/analysis/config/INPUTS.conf @@ -70,9 +70,10 @@ # file). Don't use this if you give the 'fitsdatasum' # keyvalue. # -# INPUT-%-fitsdatasum: The FITS standard DATASUM value for HDU number 1 -# of the FITS file (counting from 0). Don't use this -# if you give the 'sha256' keyword. +# INPUT-%-fitsdatasum: The FITS standard DATASUM value for the HDU given +# to '-fitshdu' (below) of the FITS file (counting +# from 0). Don't use this if you give the 'sha256' +# keyword. # # INPUT-%-fitshdu: The HDU identifier (counter from 0, or name) to use # for the verification. This is only relevant in the @@ -104,7 +105,7 @@ # also called '%' (if your local copy of the input dataset and the only # repository names are the same, be sure to set '%' accordingly). # -# Copyright (C) 2018-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> +# Copyright (C) 2018-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> # # Copying and distribution of this file, with or without modification, are # permitted in any medium without royalty provided the copyright notice and diff --git a/reproduce/analysis/config/delete-me-squared-num.conf b/reproduce/analysis/config/delete-me-squared-num.conf index c2fa79c..ba8c960 100644 --- a/reproduce/analysis/config/delete-me-squared-num.conf +++ b/reproduce/analysis/config/delete-me-squared-num.conf @@ -1,6 +1,6 @@ # Number of samples in the demonstration analysis (to be deleted). # -# Copyright (C) 2019-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> +# Copyright (C) 2019-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> # # Copying and distribution of this file, with or without modification, are # permitted in any medium without royalty provided the copyright notice and diff --git a/reproduce/analysis/config/metadata.conf b/reproduce/analysis/config/metadata.conf index 7bafc4b..1ab7de1 100644 --- a/reproduce/analysis/config/metadata.conf +++ b/reproduce/analysis/config/metadata.conf @@ -15,7 +15,7 @@ # and the copyright license name and standard link to the fully copyright # license. # -# Copyright (C) 2020-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> +# Copyright (C) 2020-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> # # Copying and distribution of this file, with or without modification, are # permitted in any medium without royalty provided the copyright notice and diff --git a/reproduce/analysis/config/pdf-build.conf b/reproduce/analysis/config/pdf-build.conf index 2aa2e9a..7821306 100644 --- a/reproduce/analysis/config/pdf-build.conf +++ b/reproduce/analysis/config/pdf-build.conf @@ -12,7 +12,7 @@ # LaTeX. Otherwise, a notice will just printed that, no PDF will be # created. # -# Copyright (C) 2018-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> +# Copyright (C) 2018-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> # # Copying and distribution of this file, with or without modification, are # permitted in any medium without royalty provided the copyright notice and diff --git a/reproduce/analysis/config/verify-outputs.conf b/reproduce/analysis/config/verify-outputs.conf index db7751d..031085d 100644 --- a/reproduce/analysis/config/verify-outputs.conf +++ b/reproduce/analysis/config/verify-outputs.conf @@ -1,6 +1,6 @@ # To enable verification of output datasets set this variable to 'yes'. # -# Copyright (C) 2019-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> +# Copyright (C) 2019-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> # # Copying and distribution of this file, with or without modification, are # permitted in any medium without royalty provided the copyright notice and diff --git a/reproduce/analysis/make/delete-me.mk b/reproduce/analysis/make/delete-me.mk index 325280d..a20abc6 100644 --- a/reproduce/analysis/make/delete-me.mk +++ b/reproduce/analysis/make/delete-me.mk @@ -1,6 +1,6 @@ # Dummy Makefile to create a random dataset for plotting. # -# Copyright (C) 2018-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> +# Copyright (C) 2018-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> # # This Makefile is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by diff --git a/reproduce/analysis/make/initialize.mk b/reproduce/analysis/make/initialize.mk index 9e8db4a..92e5eff 100644 --- a/reproduce/analysis/make/initialize.mk +++ b/reproduce/analysis/make/initialize.mk @@ -1,6 +1,6 @@ # Project initialization. # -# Copyright (C) 2018-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> +# Copyright (C) 2018-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> # # This Makefile is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -469,7 +469,7 @@ dist-software: # process with a file and make sure that only one downloading event is in # progress at every moment. $(indir):; mkdir $@ -downloadwrapper = $(bashdir)/download-multi-try +downloadwrapper = $(bashdir)/download-multi-try.sh inputdatasets := $(foreach i, \ $(patsubst INPUT-%-sha256,%, \ $(filter INPUT-%-sha256,$(.VARIABLES))) \ @@ -672,7 +672,7 @@ print-general-metadata = \ # for the final PDF. Since these are not version controlled, it must be # calculated everytime the project is run. So even though this file # actually exists, it is also aded as a '.PHONY' target above. -$(mtexdir)/initialize.tex: | $(mtexdir) +$(mtexdir)/initialize.tex: # Version and title of project. About the starting '@': since these # commands are run every time with './project make', it is annoying diff --git a/reproduce/analysis/make/paper.mk b/reproduce/analysis/make/paper.mk index 791108b..66c6859 100644 --- a/reproduce/analysis/make/paper.mk +++ b/reproduce/analysis/make/paper.mk @@ -1,6 +1,6 @@ # Build the final PDF paper/report. # -# Copyright (C) 2018-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> +# Copyright (C) 2018-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> # # This Makefile is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -128,6 +128,11 @@ $(texbdir)/paper.bbl: tex/src/references.tex $(mtexdir)/dependencies-bib.tex \ # use PGFPlots, then you should remove the '-shell-escape' option # for better security. See https://savannah.nongnu.org/task/?15694 # for details. +# +# We need the modification to 'LD_LIBRARY_PATH' because we do not +# build LaTeX from source and it uses '/bin/sh' (among other +# possible system-wide things). + export LD_LIBRARY_PATH="$(sys_library_sh_path):$$LD_LIBRARY_PATH" pdflatex -shell-escape -halt-on-error "$$p"/paper.tex biber paper @@ -158,6 +163,11 @@ paper.pdf: $(mtexdir)/project.tex paper.tex $(texbdir)/paper.bbl # See above for a warning and brief discussion on the the pdflatex # option '-shell-escape'. +# +# We need the modification to 'LD_LIBRARY_PATH' because we do not +# build LaTeX from source and it uses '/bin/sh' (among other +# possible system-wide things). + export LD_LIBRARY_PATH="$(sys_library_sh_path):$$LD_LIBRARY_PATH" pdflatex -shell-escape -halt-on-error "$$p"/paper.tex # Come back to the top project directory and copy the built PDF diff --git a/reproduce/analysis/make/prepare.mk b/reproduce/analysis/make/prepare.mk index 36f5294..2cc1187 100644 --- a/reproduce/analysis/make/prepare.mk +++ b/reproduce/analysis/make/prepare.mk @@ -1,6 +1,6 @@ # Basic preparations, called by './project make'. # -# Copyright (C) 2019-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> +# Copyright (C) 2019-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> # # This Makefile is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by diff --git a/reproduce/analysis/make/top-make.mk b/reproduce/analysis/make/top-make.mk index 460433b..2689e64 100644 --- a/reproduce/analysis/make/top-make.mk +++ b/reproduce/analysis/make/top-make.mk @@ -1,6 +1,6 @@ # Top-level Makefile (first to be loaded). # -# Copyright (C) 2018-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> +# Copyright (C) 2018-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> # # This Makefile is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by diff --git a/reproduce/analysis/make/top-prepare.mk b/reproduce/analysis/make/top-prepare.mk index 930c2a9..7d92d72 100644 --- a/reproduce/analysis/make/top-prepare.mk +++ b/reproduce/analysis/make/top-prepare.mk @@ -4,7 +4,7 @@ # are not included here. Please see that file for thorough comments on each # step. # -# Copyright (C) 2019-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> +# Copyright (C) 2019-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> # # This Makefile is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by diff --git a/reproduce/analysis/make/verify.mk b/reproduce/analysis/make/verify.mk index aa026b5..c74f8ca 100644 --- a/reproduce/analysis/make/verify.mk +++ b/reproduce/analysis/make/verify.mk @@ -1,6 +1,6 @@ # Verify the project outputs before building the paper. # -# Copyright (C) 2020-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> +# Copyright (C) 2020-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> # # This Makefile is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by diff --git a/reproduce/software/bibtex/astrometrynet.tex b/reproduce/software/bibtex/astrometrynet.tex index 10b487e..5697885 100644 --- a/reproduce/software/bibtex/astrometrynet.tex +++ b/reproduce/software/bibtex/astrometrynet.tex @@ -1,4 +1,4 @@ -%% Copyright (C) 2019-2023 Raul Infante-Sainz <infantesainz@gmail.com> +%% Copyright (C) 2019-2025 Raul Infante-Sainz <infantesainz@gmail.com> %% %% Copying and distribution of this file, with or without modification, %% are permitted in any medium without royalty provided the copyright diff --git a/reproduce/software/bibtex/astropy.tex b/reproduce/software/bibtex/astropy.tex index f0b8349..3df1aa1 100644 --- a/reproduce/software/bibtex/astropy.tex +++ b/reproduce/software/bibtex/astropy.tex @@ -1,4 +1,4 @@ -%% Copyright (C) 2019-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> +%% Copyright (C) 2019-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> %% %% Copying and distribution of this file, with or without modification, %% are permitted in any medium without royalty provided the copyright diff --git a/reproduce/software/bibtex/corner.tex b/reproduce/software/bibtex/corner.tex index 5eda23d..fd7ea76 100644 --- a/reproduce/software/bibtex/corner.tex +++ b/reproduce/software/bibtex/corner.tex @@ -1,4 +1,4 @@ -%% Copyright (C) 2019-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> +%% Copyright (C) 2019-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> %% %% Copying and distribution of this file, with or without modification, %% are permitted in any medium without royalty provided the copyright diff --git a/reproduce/software/bibtex/cython.tex b/reproduce/software/bibtex/cython.tex index 6103028..311072d 100644 --- a/reproduce/software/bibtex/cython.tex +++ b/reproduce/software/bibtex/cython.tex @@ -1,4 +1,4 @@ -%% Copyright (C) 2019-2023 Raul Infante-Sainz <infantesainz@gmail.com> +%% Copyright (C) 2019-2025 Raul Infante-Sainz <infantesainz@gmail.com> %% %% Copying and distribution of this file, with or without modification, %% are permitted in any medium without royalty provided the copyright diff --git a/reproduce/software/bibtex/fftw.tex b/reproduce/software/bibtex/fftw.tex index e1e9bf8..15c5c62 100644 --- a/reproduce/software/bibtex/fftw.tex +++ b/reproduce/software/bibtex/fftw.tex @@ -1,4 +1,4 @@ -%% Copyright (C) 2019-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> +%% Copyright (C) 2019-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> %% %% Copying and distribution of this file, with or without modification, %% are permitted in any medium without royalty provided the copyright diff --git a/reproduce/software/bibtex/galsim.tex b/reproduce/software/bibtex/galsim.tex index 6ca9b45..bbfdec1 100644 --- a/reproduce/software/bibtex/galsim.tex +++ b/reproduce/software/bibtex/galsim.tex @@ -1,4 +1,4 @@ -%% Copyright (C) 2019-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> +%% Copyright (C) 2019-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> %% %% Copying and distribution of this file, with or without modification, %% are permitted in any medium without royalty provided the copyright diff --git a/reproduce/software/bibtex/gnuastro.tex b/reproduce/software/bibtex/gnuastro.tex index 5fd87e4..7d56e3e 100644 --- a/reproduce/software/bibtex/gnuastro.tex +++ b/reproduce/software/bibtex/gnuastro.tex @@ -1,4 +1,4 @@ -%% Copyright (C) 2019-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> +%% Copyright (C) 2019-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> %% %% Copying and distribution of this file, with or without modification, %% are permitted in any medium without royalty provided the copyright diff --git a/reproduce/software/bibtex/healpix.tex b/reproduce/software/bibtex/healpix.tex index cfb96c0..afdd6a6 100644 --- a/reproduce/software/bibtex/healpix.tex +++ b/reproduce/software/bibtex/healpix.tex @@ -1,4 +1,4 @@ -%% Copyright (C) 2019-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> +%% Copyright (C) 2019-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> %% %% Copying and distribution of this file, with or without modification, %% are permitted in any medium without royalty provided the copyright diff --git a/reproduce/software/bibtex/imfit.tex b/reproduce/software/bibtex/imfit.tex index f35ee69..1d5270b 100644 --- a/reproduce/software/bibtex/imfit.tex +++ b/reproduce/software/bibtex/imfit.tex @@ -1,4 +1,4 @@ -%% Copyright (C) 2019-2023 Raul Infante-Sainz <infantesainz@gmail.com> +%% Copyright (C) 2019-2025 Raul Infante-Sainz <infantesainz@gmail.com> %% %% Copying and distribution of this file, with or without modification, %% are permitted in any medium without royalty provided the copyright diff --git a/reproduce/software/bibtex/matplotlib.tex b/reproduce/software/bibtex/matplotlib.tex index 9873c57..ec7bab4 100644 --- a/reproduce/software/bibtex/matplotlib.tex +++ b/reproduce/software/bibtex/matplotlib.tex @@ -1,4 +1,4 @@ -%% Copyright (C) 2019-2023 Raul Infante-Sainz <infantesainz@gmail.com> +%% Copyright (C) 2019-2025 Raul Infante-Sainz <infantesainz@gmail.com> %% %% Copying and distribution of this file, with or without modification, %% are permitted in any medium without royalty provided the copyright diff --git a/reproduce/software/bibtex/missfits.tex b/reproduce/software/bibtex/missfits.tex index 64704e6..83510dd 100644 --- a/reproduce/software/bibtex/missfits.tex +++ b/reproduce/software/bibtex/missfits.tex @@ -1,4 +1,4 @@ -%% Copyright (C) 2020-2023 Surena Fatemi <surena.fatemi@ipm.ir> +%% Copyright (C) 2020-2025 Surena Fatemi <surena.fatemi@ipm.ir> %% %% Copying and distribution of this file, with or without modification, %% are permitted in any medium without royalty provided the copyright diff --git a/reproduce/software/bibtex/mpi4py.tex b/reproduce/software/bibtex/mpi4py.tex index cc6bcb9..10e0e74 100644 --- a/reproduce/software/bibtex/mpi4py.tex +++ b/reproduce/software/bibtex/mpi4py.tex @@ -1,4 +1,4 @@ -%% Copyright (C) 2019-2023 Raul Infante-Sainz <infantesainz@gmail.com> +%% Copyright (C) 2019-2025 Raul Infante-Sainz <infantesainz@gmail.com> %% %% Copying and distribution of this file, with or without modification, %% are permitted in any medium without royalty provided the copyright diff --git a/reproduce/software/bibtex/numpy.tex b/reproduce/software/bibtex/numpy.tex index da54330..fb017c4 100644 --- a/reproduce/software/bibtex/numpy.tex +++ b/reproduce/software/bibtex/numpy.tex @@ -1,4 +1,4 @@ -%% Copyright (C) 2019-2023 Raul Infante-Sainz <infantesainz@gmail.com> +%% Copyright (C) 2019-2025 Raul Infante-Sainz <infantesainz@gmail.com> %% %% Copying and distribution of this file, with or without modification, %% are permitted in any medium without royalty provided the copyright diff --git a/reproduce/software/bibtex/r-cran.tex b/reproduce/software/bibtex/r-cran.tex index 8d691c1..c58eeeb 100644 --- a/reproduce/software/bibtex/r-cran.tex +++ b/reproduce/software/bibtex/r-cran.tex @@ -1,5 +1,5 @@ -%% Copyright (C) 2022-2023 Boud Roukema <boud@cosmo.torun.pl> -%% Copyright (C) 2022-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> +%% Copyright (C) 2022-2025 Boud Roukema <boud@cosmo.torun.pl> +%% Copyright (C) 2022-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> %% %% Copying and distribution of this file, with or without modification, %% are permitted in any medium without royalty provided the copyright diff --git a/reproduce/software/bibtex/scamp.tex b/reproduce/software/bibtex/scamp.tex index 5e5be91..b5c4da9 100644 --- a/reproduce/software/bibtex/scamp.tex +++ b/reproduce/software/bibtex/scamp.tex @@ -1,4 +1,4 @@ -%% Copyright (C) 2019-2023 Raul Infante-Sainz <infantesainz@gmail.com> +%% Copyright (C) 2019-2025 Raul Infante-Sainz <infantesainz@gmail.com> %% %% Copying and distribution of this file, with or without modification, %% are permitted in any medium without royalty provided the copyright diff --git a/reproduce/software/bibtex/scipy.tex b/reproduce/software/bibtex/scipy.tex index c92bbec..1296393 100644 --- a/reproduce/software/bibtex/scipy.tex +++ b/reproduce/software/bibtex/scipy.tex @@ -1,4 +1,4 @@ -%% Copyright (C) 2019-2023 Raul Infante-Sainz <infantesainz@gmail.com> +%% Copyright (C) 2019-2025 Raul Infante-Sainz <infantesainz@gmail.com> %% %% Copying and distribution of this file, with or without modification, %% are permitted in any medium without royalty provided the copyright diff --git a/reproduce/software/bibtex/sextractor.tex b/reproduce/software/bibtex/sextractor.tex index cdffe74..818f04a 100644 --- a/reproduce/software/bibtex/sextractor.tex +++ b/reproduce/software/bibtex/sextractor.tex @@ -1,5 +1,5 @@ -%% Copyright (C) 2019-2023 Raul Infante-Sainz <infantesainz@gmail.com> -%% Copyright (C) 2019-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> +%% Copyright (C) 2019-2025 Raul Infante-Sainz <infantesainz@gmail.com> +%% Copyright (C) 2019-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> %% %% Copying and distribution of this file, with or without modification, %% are permitted in any medium without royalty provided the copyright diff --git a/reproduce/software/bibtex/sip_tpv.tex b/reproduce/software/bibtex/sip_tpv.tex index d13d7a5..0d52e33 100644 --- a/reproduce/software/bibtex/sip_tpv.tex +++ b/reproduce/software/bibtex/sip_tpv.tex @@ -1,4 +1,4 @@ -%% Copyright (C) 2019-2023 Raul Infante-Sainz <infantesainz@gmail.com> +%% Copyright (C) 2019-2025 Raul Infante-Sainz <infantesainz@gmail.com> %% %% Copying and distribution of this file, with or without modification, %% are permitted in any medium without royalty provided the copyright diff --git a/reproduce/software/bibtex/swarp.tex b/reproduce/software/bibtex/swarp.tex index ecac1a7..e0989f9 100644 --- a/reproduce/software/bibtex/swarp.tex +++ b/reproduce/software/bibtex/swarp.tex @@ -1,4 +1,4 @@ -%% Copyright (C) 2019-2023 Raul Infante-Sainz <infantesainz@gmail.com> +%% Copyright (C) 2019-2025 Raul Infante-Sainz <infantesainz@gmail.com> %% %% Copying and distribution of this file, with or without modification, %% are permitted in any medium without royalty provided the copyright diff --git a/reproduce/software/bibtex/sympy.tex b/reproduce/software/bibtex/sympy.tex index 27ea9ac..01e481b 100644 --- a/reproduce/software/bibtex/sympy.tex +++ b/reproduce/software/bibtex/sympy.tex @@ -1,4 +1,4 @@ -%% Copyright (C) 2019-2023 Raul Infante-Sainz <infantesainz@gmail.com> +%% Copyright (C) 2019-2025 Raul Infante-Sainz <infantesainz@gmail.com> %% %% Copying and distribution of this file, with or without modification, %% are permitted in any medium without royalty provided the copyright diff --git a/reproduce/software/bibtex/tides.tex b/reproduce/software/bibtex/tides.tex index dfa8eb8..caf1500 100644 --- a/reproduce/software/bibtex/tides.tex +++ b/reproduce/software/bibtex/tides.tex @@ -1,4 +1,4 @@ -%% Copyright (C) 2019-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> +%% Copyright (C) 2019-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> %% %% Copying and distribution of this file, with or without modification, %% are permitted in any medium without royalty provided the copyright diff --git a/reproduce/software/config/LOCAL.conf.in b/reproduce/software/config/LOCAL.conf.in index 5ee1ff7..341a78e 100644 --- a/reproduce/software/config/LOCAL.conf.in +++ b/reproduce/software/config/LOCAL.conf.in @@ -3,7 +3,7 @@ # This is just a template for the './project configure' script to fill # in. Please don't make any change to this file. # -# Copyright (C) 2018-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> +# Copyright (C) 2018-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> # # Copying and distribution of this file, with or without modification, are # permitted in any medium without royalty provided the copyright notice and diff --git a/reproduce/software/config/TARGETS.conf b/reproduce/software/config/TARGETS.conf index 3d35eed..a8b396e 100644 --- a/reproduce/software/config/TARGETS.conf +++ b/reproduce/software/config/TARGETS.conf @@ -1,7 +1,7 @@ # Necessary high-level software to build in this project. # -# Copyright (C) 2018-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> -# Copyright (C) 2019-2023 Raul Infante-Sainz <infantesainz@gmail.com> +# Copyright (C) 2018-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> +# Copyright (C) 2019-2025 Raul Infante-Sainz <infantesainz@gmail.com> # # Copying and distribution of this file, with or without modification, are # permitted in any medium without royalty provided the copyright notice and diff --git a/reproduce/software/config/checksums.conf b/reproduce/software/config/checksums.conf index e34d9b8..1a81ef1 100644 --- a/reproduce/software/config/checksums.conf +++ b/reproduce/software/config/checksums.conf @@ -1,8 +1,8 @@ # sha512 checksums of all the necessary software tarballs. # -# Copyright (C) 2018-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> -# Copyright (C) 2019-2023 Raul Infante-Sainz <infantesainz@gmail.com> -# Copyright (C) 2022-2023 Pedram Ashofteh Ardakani <pedramardakani@pm.me> +# Copyright (C) 2018-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> +# Copyright (C) 2019-2025 Raul Infante-Sainz <infantesainz@gmail.com> +# Copyright (C) 2022-2025 Pedram Ashofteh Ardakani <pedramardakani@pm.me> # # Copying and distribution of this file, with or without modification, are # permitted in any medium without royalty provided the copyright notice and diff --git a/reproduce/software/config/numpy-scipy.cfg b/reproduce/software/config/numpy-scipy.cfg index 5ad5112..c3cea11 100644 --- a/reproduce/software/config/numpy-scipy.cfg +++ b/reproduce/software/config/numpy-scipy.cfg @@ -7,8 +7,8 @@ # appropriate sections. Not all packages will use all sections so you # should leave out sections that your package does not use. # -# Copyright (C) 2018-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> -# Copyright (C) 2019-2023 Raul Infante-Sainz <infantesainz@gmail.com> +# Copyright (C) 2018-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> +# Copyright (C) 2019-2025 Raul Infante-Sainz <infantesainz@gmail.com> # # Copying and distribution of this file, with or without modification, are # permitted in any medium without royalty provided the copyright notice and diff --git a/reproduce/software/config/servers-backup.conf b/reproduce/software/config/servers-backup.conf index 67eff44..d4aa25e 100644 --- a/reproduce/software/config/servers-backup.conf +++ b/reproduce/software/config/servers-backup.conf @@ -3,7 +3,7 @@ # is irrelevant). Note that this is not a to be read as a variable but will # be parsed as a list. # -# Copyright (C) 2020-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> +# Copyright (C) 2020-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> # # Copying and distribution of this file, with or without modification, are # permitted in any medium without royalty provided the copyright notice and diff --git a/reproduce/software/config/software_acknowledge_context.sh b/reproduce/software/config/software_acknowledge_context.sh index 043d9aa..deae01b 100755 --- a/reproduce/software/config/software_acknowledge_context.sh +++ b/reproduce/software/config/software_acknowledge_context.sh @@ -10,8 +10,8 @@ # your project to make a smoothly readable English text. Afterwards, please # feel free to modify them as you wish. # -# Copyright (C) 2021-2023 Boud Roukema <boud@cosmo.torun.pl> -# Copyright (C) 2021-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> +# Copyright (C) 2021-2025 Boud Roukema <boud@cosmo.torun.pl> +# Copyright (C) 2021-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> # # This script is free software: you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by the diff --git a/reproduce/software/config/texlive-packages.conf b/reproduce/software/config/texlive-packages.conf index 250a4e6..bb38d5a 100644 --- a/reproduce/software/config/texlive-packages.conf +++ b/reproduce/software/config/texlive-packages.conf @@ -4,23 +4,56 @@ # name to this variable (you can check in 'ctan.org' to find the official # name). # -# Copyright (C) 2018-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> +# Copyright (C) 2018-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> # # Copying and distribution of this file, with or without modification, are # permitted in any medium without royalty provided the copyright notice and # this notice are preserved. This file is offered as-is, without any # warranty. - -# Note on 'tex' and 'fancyhdr': These two packages are installed along with -# the basic installation scheme that we used to install tlmgr, they will be -# ignored in the 'tlmgr install' command, but will be used later when we -# want their versions. +# Notes +# ----- +# +# - tex and fancyhdr: These two packages are installed along with the basic +# installation scheme that we used to install tlmgr, they will be ignored +# in the 'tlmgr install' command, but will be used later when we want +# their versions. # -# fancyvrb: Needed by R. -texlive-packages = tex fancyhdr ec newtx fontaxes xkeyval etoolbox xstring \ - xcolor setspace caption footmisc datetime fmtcount \ - titlesec preprint ulem biblatex biber logreq pgf pgfplots \ - fp courier tex-gyre txfonts times csquotes kastrup \ - trimspaces pdftexcmds pdfescape letltxmacro bitset \ - mweights fancyvrb +# - fancyvrb: needed by R. +texlive-packages = biber \ + biblatex \ + bitset \ + caption \ + courier \ + csquotes \ + datetime \ + ec \ + etoolbox \ + fancyhdr \ + fancyvrb \ + fmtcount \ + fontaxes \ + footmisc \ + fp \ + kastrup \ + letltxmacro \ + logreq \ + mweights \ + newtx \ + pdfescape \ + pdftexcmds \ + pgf \ + pgfplots \ + preprint \ + setspace \ + tex \ + tex-gyre \ + times \ + titlesec \ + trimspaces \ + txfonts \ + ulem \ + xcolor \ + xkeyval \ + xpatch \ + xstring diff --git a/reproduce/software/config/texlive.conf b/reproduce/software/config/texlive.conf index 71878e5..0babd69 100644 --- a/reproduce/software/config/texlive.conf +++ b/reproduce/software/config/texlive.conf @@ -2,7 +2,7 @@ # # installdir: Install directory # -# Copyright (C) 2018-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> +# Copyright (C) 2018-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> # # Copying and distribution of this file, with or without modification, are # permitted in any medium without royalty provided the copyright notice and diff --git a/reproduce/software/config/urls.conf b/reproduce/software/config/urls.conf index 3a86d2d..a5998d5 100644 --- a/reproduce/software/config/urls.conf +++ b/reproduce/software/config/urls.conf @@ -3,8 +3,8 @@ # If un-commented the URLs of this file will be used. Otherwise default # servers (primarily on 'zenodo.org') will be checked for the tarball. # -# Copyright (C) 2020-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> -# Copyright (C) 2020-2023 Raul Infante-Sainz <infantesainz@gmail.com> +# Copyright (C) 2020-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> +# Copyright (C) 2020-2025 Raul Infante-Sainz <infantesainz@gmail.com> # # Copying and distribution of this file, with or without modification, are # permitted in any medium without royalty provided the copyright notice and diff --git a/reproduce/software/config/versions.conf b/reproduce/software/config/versions.conf index afb0535..e8b8542 100644 --- a/reproduce/software/config/versions.conf +++ b/reproduce/software/config/versions.conf @@ -1,8 +1,8 @@ # Versions of the various dependencies # -# Copyright (C) 2018-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> -# Copyright (C) 2019-2023 Raul Infante-Sainz <infantesainz@gmail.com> -# Copyright (C) 2022-2023 Pedram Ashofteh Ardakani <pedramardakani@pm.me> +# Copyright (C) 2018-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> +# Copyright (C) 2019-2025 Raul Infante-Sainz <infantesainz@gmail.com> +# Copyright (C) 2022-2025 Pedram Ashofteh Ardakani <pedramardakani@pm.me> # # Copying and distribution of this file, with or without modification, are # permitted in any medium without royalty provided the copyright notice and diff --git a/reproduce/software/make/README.md b/reproduce/software/make/README.md index d2b6952..00afef4 100644 --- a/reproduce/software/make/README.md +++ b/reproduce/software/make/README.md @@ -1,7 +1,7 @@ Software building instructions ------------------------------ -Copyright (C) 2019-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org>\ +Copyright (C) 2019-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org>\ See the end of the file for license conditions. This directory contains Makefiles that are called by the high-level diff --git a/reproduce/software/make/atlas-multiple.mk b/reproduce/software/make/atlas-multiple.mk index 5c8f14c..6e7d415 100644 --- a/reproduce/software/make/atlas-multiple.mk +++ b/reproduce/software/make/atlas-multiple.mk @@ -8,7 +8,7 @@ # # ------------------------------------------------------------------------ # -# Copyright (C) 2019-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> +# Copyright (C) 2019-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> # # This Makefile is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by diff --git a/reproduce/software/make/atlas-single.mk b/reproduce/software/make/atlas-single.mk index 4e2b04a..2b68677 100644 --- a/reproduce/software/make/atlas-single.mk +++ b/reproduce/software/make/atlas-single.mk @@ -8,7 +8,7 @@ # # ------------------------------------------------------------------------ # -# Copyright (C) 2019-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> +# Copyright (C) 2019-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> # # This Makefile is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by diff --git a/reproduce/software/make/basic.mk b/reproduce/software/make/basic.mk index 99e81d2..68c40b3 100644 --- a/reproduce/software/make/basic.mk +++ b/reproduce/software/make/basic.mk @@ -21,9 +21,9 @@ # # ------------------------------------------------------------------------ # -# Copyright (C) 2018-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> -# Copyright (C) 2019-2023 Raul Infante-Sainz <infantesainz@gmail.com> -# Copyright (C) 2022-2023 Pedram Ashofteh Ardakani <pedramardakani@pm.me> +# Copyright (C) 2018-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> +# Copyright (C) 2019-2025 Raul Infante-Sainz <infantesainz@gmail.com> +# Copyright (C) 2022-2025 Pedram Ashofteh Ardakani <pedramardakani@pm.me> # # This Makefile is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -49,14 +49,15 @@ include reproduce/software/config/checksums.conf include reproduce/software/config/urls.conf # Basic directories -lockdir = $(BDIR)/software/locks -tdir = $(BDIR)/software/tarballs -ddir = $(BDIR)/software/build-tmp -idir = $(BDIR)/software/installed -ibdir = $(BDIR)/software/installed/bin -ildir = $(BDIR)/software/installed/lib -iidir = $(BDIR)/software/installed/include -ibidir = $(BDIR)/software/installed/version-info/proglib +lockdir = $(BDIR)/software/locks +tdir = $(BDIR)/software/tarballs +ddir = $(BDIR)/software/build-tmp +idir = $(BDIR)/software/installed +ibdir = $(BDIR)/software/installed/bin +ildir = $(BDIR)/software/installed/lib +iidir = $(BDIR)/software/installed/include +shsrcdir = "$(shell pwd)"/reproduce/software/shell +ibidir = $(BDIR)/software/installed/version-info/proglib # Ultimate Makefile target. GNU Nano (a simple and very light-weight text # editor) is installed by default, it is recommended to have it in the @@ -107,6 +108,15 @@ export CPPFLAGS := -I$(idir)/include $(CPPFLAGS) $(noccwarnings) # 'LD_LIBRARY_PATH', then we'll add our own newly installed libraries. We # will also make sure that there is no "current directory" in it (by # removing a starting or trailing ':' and any occurance of '::'. +# +# But first: in case LD_LIBRARY_PATH is empty, give it the default value of +# $(sys_library_sh_path) (which was the location of the libraries needed by +# the host's shell). This is because after we add the Maneage's library +# path, on some systems, no other libraries will be checked except those +# that are in 'LD_LIBRARY_PATH'. +ifeq ($(strip $(LD_LIBRARY_PATH)),) +export LD_LIBRARY_PATH=$(sys_library_sh_path) +endif export LD_LIBRARY_PATH := $(shell echo $(LD_LIBRARY_PATH):$(ildir) \ | sed -e's/::/:/g' -e's/^://' -e's/:$$//') @@ -127,7 +137,7 @@ export DYLD_LIBRARY_PATH := # Afer putting everything together, we use the first server as the # reference for all software if their '-url' variable isn't defined (in # 'reproduce/software/config/urls.conf'). -downloadwrapper = ./reproduce/analysis/bash/download-multi-try +downloadwrapper = ./reproduce/analysis/bash/download-multi-try.sh maneage_backup_urls := $(shell awk '!/^#/{printf "%s ", $$1}' \ reproduce/software/config/servers-backup.conf) backupservers_all = $(user_backup_urls) $(maneage_backup_urls) @@ -336,6 +346,7 @@ $(ibidir)/bzip2-$(bzip2-version): $(ibidir)/gzip-$(gzip-version) rm -rf $$tdir tar -xf $(tdir)/$$tarball cd $$tdir + $(shsrcdir)/prep-source.sh $(ibdir) sed -e 's@\(ln -s -f \)$$(PREFIX)/bin/@\1@' Makefile \ > Makefile.sed mv Makefile.sed Makefile @@ -634,6 +645,7 @@ $(ibidir)/perl-$(perl-version): $(ibidir)/patchelf-$(patchelf-version) rm -rf perl-$(perl-version) tar -xf $(tdir)/$$tarball cd perl-$(perl-version) + $(shsrcdir)/prep-source.sh $(ibdir) ./Configure -des \ -Dusethreads \ -Duseshrplib \ @@ -691,21 +703,16 @@ $(ibidir)/coreutils-$(coreutils-version): \ $(ibidir)/perl-$(perl-version) \ $(ibidir)/openssl-$(openssl-version) -# Import, unpack and enter the source directory. +# Import the source tarball. tarball=coreutils-$(coreutils-version).tar.lz $(call import-source, $(coreutils-url), $(coreutils-checksum)) + +# Unpack and enter the source. cd $(ddir) rm -rf coreutils-$(coreutils-version) tar -xf $(tdir)/$$tarball cd coreutils-$(coreutils-version) - -# Set the configure script to use our shell, note that we can't -# assume GNU SED here yet (it installs after Coreutils). - sed -e's|\#\! /bin/sh|\#\! $(ibdir)/bash|' \ - -e's|\#\!/bin/sh|\#\! $(ibdir)/bash|' \ - configure > configure-tmp - mv configure-tmp configure - chmod +x configure + $(shsrcdir)/prep-source.sh $(ibdir) # Configure, build and install Coreutils. ./configure --prefix=$(idir) SHELL=$(ibdir)/bash \ @@ -744,6 +751,7 @@ $(ibidir)/podlators-$(podlators-version): $(ibidir)/perl-$(perl-version) rm -rf podlators-$(podlators-version) tar -xf $(tdir)/$$tarball cd podlators-$(podlators-version) + $(shsrcdir)/prep-source.sh $(ibdir) perl Makefile.PL make make install @@ -1030,7 +1038,7 @@ $(ibidir)/gmp-$(gmp-version): \ $(call import-source, $(gmp-url), $(gmp-checksum)) $(call gbuild, gmp-$(gmp-version), static, \ --enable-cxx --enable-fat, \ - -j$(numthreads) ,make check) + -j$(numthreads)) echo "GNU Multiple Precision Arithmetic Library $(gmp-version)" > $@ # Less is useful with Git (to view the diffs within a minimal container) @@ -1070,16 +1078,27 @@ $(ibidir)/libtool-$(libtool-version): $(ibidir)/m4-$(m4-version) $(ibidir)/grep-$(grep-version): $(ibidir)/coreutils-$(coreutils-version) tarball=grep-$(grep-version).tar.lz $(call import-source, $(grep-url), $(grep-checksum)) - $(call gbuild, grep-$(grep-version), static,,V=1) + $(call gbuild, grep-$(grep-version), static,, \ + -j$(numthreads) V=1) echo "GNU Grep $(grep-version)" > $@ # M4 doesn't depend on PatchELF, but just to be consistent with the # levels/phases introduced here (where the compressors are level 1, # PatchELF is level 2, and ...), we'll set it as a dependency. +# +# The '--with-syscmd-shell' is used as the default shell and if not given, +# 'm4' will use '/bin/sh' (which is not under Maneage control and can cause +# problems in 'high-level.mk' because it closes off the system's +# LD_LIBRARY_PATH and if the system's '/bin/sh' needs a special system +# library, the high-level programs will not be built). We are setting this +# default shell to Dash because M4 is built before our own Bash. Recall +# that Dash is built before we enter this Makefile. $(ibidir)/m4-$(m4-version): $(ibidir)/patchelf-$(patchelf-version) tarball=m4-$(m4-version).tar.lz $(call import-source, $(m4-url), $(m4-checksum)) - $(call gbuild, m4-$(m4-version), static,,V=1) + $(call gbuild, m4-$(m4-version), static, \ + --with-syscmd-shell=$(ibdir)/dash, \ + -j$(numthreads) V=1) echo "GNU M4 $(m4-version)" > $@ $(ibidir)/mpfr-$(mpfr-version): $(ibidir)/gmp-$(gmp-version) @@ -1169,7 +1188,7 @@ $(ibidir)/mpc-$(mpc-version): $(ibidir)/mpfr-$(mpfr-version) echo "" > $@ else $(call gbuild, mpc-$(mpc-version), static, , \ - -j$(numthreads), make check) + -j$(numthreads)) echo "GNU Multiple Precision Complex library" > $@ fi @@ -1399,6 +1418,7 @@ $(ibidir)/gcc-$(gcc-version): $(ibidir)/binutils-$(binutils-version) ln -s $$odir/gcc-$(gcc-version) $(ddir)/gcc-$(gcc-version) fi cd gcc-$(gcc-version) + $(shsrcdir)/prep-source.sh $(ibdir) # Unfortunately binutils installs headers like 'ansidecl.h' that # have been seen to conflict with GCC's internal versions of those @@ -1506,7 +1526,18 @@ $(ibidir)/gcc-$(gcc-version): $(ibidir)/binutils-$(binutils-version) -# Software that need re-compilation (to use our own libraries) +# Level 6: need re-compilation +# ---------------------------- +# +# The initial build of these was done with the host's settings, which will +# cause problems later when we completely close-off the host environment. +$(ibidir)/make-$(make-version): $(ibidir)/gcc-$(gcc-version) + tarball=make-$(make-version).tar.lz + $(call import-source, $(make-url), $(make-checksum)) + $(call gbuild, make-$(make-version), static, \ + --disable-dependency-tracking --without-guile) + echo "GNU Make $(make-version)" > $@ + $(ibidir)/lzip-$(lzip-version): $(ibidir)/gcc-$(gcc-version) tarball=lzip-$(lzip-version).tar unpackdir=lzip-$(lzip-version) @@ -1514,6 +1545,7 @@ $(ibidir)/lzip-$(lzip-version): $(ibidir)/gcc-$(gcc-version) rm -rf $$unpackdir tar -xf $(tdir)/$$tarball cd $$unpackdir + $(shsrcdir)/prep-source.sh $(ibdir) ./configure --build --check --installdir="$(ibdir)" if [ -f $(ibdir)/patchelf ]; then $(ibdir)/patchelf --set-rpath $(ildir) $(ibdir)/lzip; @@ -1526,11 +1558,7 @@ $(ibidir)/lzip-$(lzip-version): $(ibidir)/gcc-$(gcc-version) - - - - -# Level 6: Basic text editor +# Level 7: Basic text editor # -------------------------- # # If the project is built in a minimal environment, there is no text @@ -1547,7 +1575,8 @@ $(ibidir)/lzip-$(lzip-version): $(ibidir)/gcc-$(gcc-version) # nano (and use their own optional high-level text editor). To do this, you # can just have to manually remove 'nano' from 'targets-proglib' above and # add their optional text editor in 'TARGETS.conf'. -$(ibidir)/nano-$(nano-version): $(ibidir)/lzip-$(lzip-version) +$(ibidir)/nano-$(nano-version): $(ibidir)/lzip-$(lzip-version) \ + $(ibidir)/make-$(make-version) tarball=nano-$(nano-version).tar.lz $(call import-source, $(nano-url), $(nano-checksum)) $(call gbuild, nano-$(nano-version), static) diff --git a/reproduce/software/make/build-rules.mk b/reproduce/software/make/build-rules.mk index c160d33..62cb6d5 100644 --- a/reproduce/software/make/build-rules.mk +++ b/reproduce/software/make/build-rules.mk @@ -3,7 +3,7 @@ # imported into 'basic.mk' and 'high-level.mk'. They should be activated # with Make's 'Call' function. # -# Copyright (C) 2018-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> +# Copyright (C) 2018-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> # # This Makefile is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -207,17 +207,10 @@ gbuild = if [ x$(static_build) = xyes ] && [ "x$(2)" = xstatic ]; then \ else confscript="$(strip $(6))"; \ fi; \ \ - if [ -f $(ibdir)/bash ]; then \ - if [ -f "$$confscript" ]; then \ - sed -e's|\#\! /bin/sh|\#\! $(ibdir)/bash|' \ - -e's|\#\!/bin/sh|\#\! $(ibdir)/bash|' \ - $$confscript > $$confscript-tmp; \ - mv $$confscript-tmp $$confscript; \ - chmod +x $$confscript; \ - fi; \ + $(shsrcdir)/prep-source.sh $(ibdir); \ + if [ -f $(ibdir)/bash ]; then \ shellop="SHELL=$(ibdir)/bash"; \ - elif [ -f /bin/bash ]; then shellop="SHELL=/bin/bash"; \ - else shellop="SHELL=/bin/sh"; \ + else shellop="SHELL=$(ibdir)/dash"; \ fi; \ \ if [ x$$gbuild_prefix = x ]; then prefixdir="$(idir)"; \ @@ -255,10 +248,7 @@ gbuild = if [ x$(static_build) = xyes ] && [ "x$(2)" = xstatic ]; then \ # CMake # ----- # -# According to the link below, in CMake '/bin/sh' is hardcoded, so there is -# no way to change it unfortunately! -# -# https://stackoverflow.com/questions/21167014/how-to-set-shell-variable-in-makefiles-generated-by-cmake +# Used by packages that are built with CMake. cbuild = if [ x$(static_build) = xyes ] && [ $(2)x = staticx ]; then \ export LDFLAGS="$$LDFLAGS -static"; \ opts="-DBUILD_SHARED_LIBS=OFF"; \ @@ -268,13 +258,18 @@ cbuild = if [ x$(static_build) = xyes ] && [ $(2)x = staticx ]; then \ utarball=$(tdir)/$$tarball; \ $(call uncompress); \ cd $(1); \ - rm -rf project-build; \ - mkdir project-build; \ - cd project-build; \ + $(shsrcdir)/prep-source.sh $(ibdir); \ + if [ -f $(ibdir)/bash ]; then \ + shellop="SHELL=$(ibdir)/bash"; \ + else shellop="SHELL=$(ibdir)/dash"; \ + fi; \ + rm -rf maneage-build; \ + mkdir maneage-build; \ + cd maneage-build; \ cmake .. -DCMAKE_LIBRARY_PATH=$(ildir) \ -DCMAKE_INSTALL_PREFIX=$(idir) \ -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON $$opts $(3); \ - make; \ - make install; \ + make $$shellop; \ + make $$shellop install; \ cd ../..; \ rm -rf $(1) diff --git a/reproduce/software/make/high-level.mk b/reproduce/software/make/high-level.mk index 3f9c40c..2274c45 100644 --- a/reproduce/software/make/high-level.mk +++ b/reproduce/software/make/high-level.mk @@ -12,8 +12,8 @@ # # ------------------------------------------------------------------------ # -# Copyright (C) 2018-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> -# Copyright (C) 2019-2023 Raul Infante-Sainz <infantesainz@gmail.com> +# Copyright (C) 2018-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> +# Copyright (C) 2019-2025 Raul Infante-Sainz <infantesainz@gmail.com> # # This Makefile is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -64,6 +64,12 @@ ipydir = $(BDIR)/software/installed/version-info/python ircrandir = $(BDIR)/software/installed/version-info/r-cran ilibrcrandir = $(BDIR)/software/installed/lib/R/library +# Special files. +makewshell = $(ibdir)/make-with-shell + + + + # Targets to build. ifeq ($(strip $(all_highlevel)),1) @@ -232,7 +238,7 @@ $(idircustom):; mkdir $@ # Afer putting everything together, we use the first server as the # reference for all software if their '-url' variable isn't defined (in # 'reproduce/software/config/urls.conf'). -downloadwrapper = ./reproduce/analysis/bash/download-multi-try +downloadwrapper = ./reproduce/analysis/bash/download-multi-try.sh maneage_backup_urls := $(shell awk '!/^#/{printf "%s ", $$1}' \ reproduce/software/config/servers-backup.conf) backupservers_all = $(user_backup_urls) $(maneage_backup_urls) @@ -339,6 +345,7 @@ $(ibidir)/atlas-$(atlas-version): cd $(ddir) tar -xf $(tdir)/atlas-$(atlas-version).tar.lz cd ATLAS + $(shsrcdir)/prep-source.sh $(ibdir) rm -rf build mkdir build cd build @@ -395,6 +402,7 @@ $(ibidir)/boost-$(boost-version): \ cd $(ddir) tar -xf $(tdir)/$$tarball cd $$unpackdir + $(shsrcdir)/prep-source.sh $(ibdir) ./bootstrap.sh --prefix=$(idir) --with-libraries=all \ --with-python=python3 echo "using mpi ;" > project-config.jam @@ -417,9 +425,7 @@ $(ibidir)/cfitsio-$(cfitsio-version): topdir=$(pwd); cd $(ddir); tar -xf $(tdir)/$$tarball customtar=cfitsio-$(cfitsio-version)-custom.tar.gz cd cfitsio-$(cfitsio-version) - sed configure -e's|@rpath|$(ildir)|g' > configure_tmp - mv configure_tmp configure - chmod +x configure + sed -i -e's|@rpath|$(ildir)|g' configure cd .. tar cf $$customtar cfitsio-$(cfitsio-version) cd $$topdir @@ -518,6 +524,12 @@ $(ibidir)/fftw-$(fftw-version): echo "FFTW $(fftw-version) \citep{fftw}" > $@ $(ibidir)/freetype-$(freetype-version): $(ibidir)/libpng-$(libpng-version) +# As of version 2.13.2, FreeType doesn't account for the 'SHELL' +# environment variable. The issue has been reported to the +# developers. But until future versions, the work-around was +# discoverd to be setting the 'GNUMAKE' environment variable so it +# includes 'SHELL'. + export GNUMAKE="$(makewshell)" tarball=freetype-$(freetype-version).tar.lz $(call import-source, $(freetype-url), $(freetype-checksum)) $(call gbuild, freetype-$(freetype-version), static) @@ -578,7 +590,9 @@ $(ibidir)/healpix-$(healpix-version): $(healpix-python-dep) \ rm -rf $(ddir)/Healpix_$(healpix-version) topdir=$(pwd); cd $(ddir); tar -xf $(tdir)/$$tarball - cd Healpix_$(healpix-version)/src/C/autotools/ + cd Healpix_$(healpix-version) + $(shsrcdir)/prep-source.sh $(ibdir) + cd src/C/autotools autoreconf --install ./configure --prefix=$(idir) make V=1 -j$(numthreads) SHELL=$(ibdir)/bash @@ -670,6 +684,7 @@ $(ibidir)/libpaper-$(libpaper-version): \ tar -xf $(tdir)/$$tarball unpackdir=libpaper-$(libpaper-version) cd $$unpackdir + $(shsrcdir)/prep-source.sh $(ibdir) autoreconf -fi ./configure --prefix=$(idir) --sysconfdir=$(idir)/etc \ --disable-static @@ -799,6 +814,7 @@ $(ibidir)/openblas-$(openblas-version): cd $(ddir) tar -xf $(tdir)/$$tarball cd OpenBLAS-$(openblas-version) + $(shsrcdir)/prep-source.sh $(ibdir) make -j$(numthreads) make PREFIX=$(idir) install cd .. @@ -1007,6 +1023,7 @@ $(ibidir)/astrometrynet-$(astrometrynet-version): \ rm -rf astrometry.net-$(astrometrynet-version) tar -xf $(tdir)/$$tarball cd astrometry.net-$(astrometrynet-version) + $(shsrcdir)/prep-source.sh $(ibdir) sed -e 's|cat /proc/cpuinfo|echo "Ignoring CPU info"|' \ -e 's|-free|echo "Ignoring RAM info"|' Makefile > Makefile.tmp mv Makefile.tmp Makefile @@ -1045,6 +1062,7 @@ $(ibidir)/cdsclient-$(cdsclient-version): cd $(ddir) tar -xf $(tdir)/$$tarball cd cdsclient-$(cdsclient-version) + $(shsrcdir)/prep-source.sh $(ibdir) touch * ./configure --prefix=$(idir) make @@ -1061,25 +1079,29 @@ $(ibidir)/cmake-$(cmake-version): tarball=cmake-$(cmake-version).tar.lz $(call import-source, $(cmake-url), $(cmake-checksum)) -# After searching in 'bootstrap', I couldn't find 'LIBS', only -# 'LDFLAGS'. So the extra libraries are being added to 'LDFLAGS', not -# 'LIBS'. -# # On Mac systems, the build complains about 'clang' specific # features, so we can't use our own GCC build here. if [ x$(on_mac_os) = xyes ]; then export CC=clang export CXX=clang++ fi + +# CMake wants a single executable for 'MAKE', so we can't use 'make +# SHELL=$(SHELL) and we have defined this script. + export MAKE="$(makewshell)" + +# Go into the unpacked directory and build CMake. cd $(ddir) rm -rf cmake-$(cmake-version) tar -xf $(tdir)/$$tarball cd cmake-$(cmake-version) + $(shsrcdir)/prep-source.sh $(ibdir) ./bootstrap --prefix=$(idir) --system-curl --system-zlib \ --system-bzip2 --system-liblzma --no-qt-gui \ --parallel=$(numthreads) - make -j$(numthreads) LIBS="$$LIBS -lssl -lcrypto -lz" VERBOSE=1 - make install + $(makewshell) VERBOSE=1 LIBS="$$LIBS -lssl -lcrypto -lz" \ + -j$(numthreads) + $(makewshell) install cd .. rm -rf cmake-$(cmake-version) echo "CMake $(cmake-version)" > $@ @@ -1129,6 +1151,7 @@ $(ibidir)/ghostscript-$(ghostscript-version): \ cd $(ddir) tar -xf $(tdir)/$$tarball cd ghostscript-$(ghostscript-version) + $(shsrcdir)/prep-source.sh $(ibdir) ./configure --prefix=$(idir) \ --disable-cups \ --enable-dynamic \ @@ -1170,7 +1193,7 @@ $(ibidir)/gnuastro-$(gnuastro-version): \ $(call gbuild, gnuastro-$(gnuastro-version), static, , \ -j$(numthreads)) cp $(dtexdir)/gnuastro.tex $(ictdir)/ - echo "GNU Astronomy Utilities $(gnuastro-version) \citep{gnuastro,akhlaghi19}" > $@ + echo "GNU Astronomy Utilities $(gnuastro-version) \citep{gnuastro}" > $@ $(ibidir)/icu-$(icu-version): $(ibidir)/python-$(python-version) @@ -1188,7 +1211,9 @@ $(ibidir)/icu-$(icu-version): $(ibidir)/python-$(python-version) cd $(ddir) tar -xf $(tdir)/$$tarball unpackdir=icu-$(icu-version) - cd $$unpackdir/icu4c/source + cd $$unpackdir + $(shsrcdir)/prep-source.sh $(ibdir) + cd icu4c/source ./configure --enable-static --prefix=$(idir) make -j$(numthreads) V=1 make install @@ -1255,6 +1280,7 @@ $(ibidir)/imfit-$(imfit-version): \ rm -rf $$unpackdir tar -xf $(tdir)/$$tarball cd $$unpackdir + $(shsrcdir)/prep-source.sh $(ibdir) sed -i 's|/usr/local|$(idir)|g' SConstruct sed -i 's|/usr/include|$(idir)/include|g' SConstruct sed -i 's|.append(|.insert(0,|g' SConstruct @@ -1302,6 +1328,7 @@ $(ibidir)/minizip-$(minizip-version): $(ibidir)/automake-$(automake-version) mkdir $$unpackdir tar -xf $(tdir)/$$tarball -C$$unpackdir --strip-components=1 cd $$unpackdir + $(shsrcdir)/prep-source.sh $(ibdir) ./configure --prefix=$(idir) make cd contrib/minizip @@ -1363,6 +1390,7 @@ $(ibidir)/netpbm-$(netpbm-version): \ rm -rf $$unpackdir tar -xf $(tdir)/$$tarball cd $$unpackdir + $(shsrcdir)/prep-source.sh $(ibdir) # As of NetPBM 10.73.39 and Flex 2.6.4-410-74a89fd (commit 74a89fd in # Flex's Git that is 410 commits after version 2.6.4), there is the @@ -1482,6 +1510,7 @@ $(ibidir)/scons-$(scons-version): $(ibidir)/python-$(python-version) rm -rf $$unpackdir tar -xf $(tdir)/$$tarball cd $$unpackdir + $(shsrcdir)/prep-source.sh $(ibdir) # Unfortuantely SCons hard-codes its search PATH in its source (to # use POSIX operating system defaults)! So the only way to modify it @@ -1625,8 +1654,9 @@ $(ibidir)/util-linux-$(util-linux-version): | $(idircustom) cd $(ddir) tar -xf $(tdir)/$$tarball cd util-linux-$(util-linux-version) + $(shsrcdir)/prep-source.sh $(ibdir) -# If a patch exists for the current version, apply it. +# If a patch is necessary, apply it. if [ -f $(patchdir)/util-linux-$(util-linux-version)-macos.patch ]; then cp $(patchdir)/util-linux-$(util-linux-version)-macos.patch \ util-linux-$(util-linux-version)-macos.patch @@ -1727,6 +1757,7 @@ $(ibidir)/vim-$(vim-version): tar -xf $(tdir)/$$tarball unpackdir=vim-$(vim-version) cd $(ddir)/$$unpackdir + $(shsrcdir)/prep-source.sh $(ibdir) ./configure --prefix=$(idir) \ --disable-canberra \ --enable-multibyte \ @@ -1795,10 +1826,16 @@ $(itidir)/texlive-ready-tlmgr: reproduce/software/config/texlive.conf rm -rf install-tl-* tar -xf $(tdir)/install-tl-unx.tar.gz cd install-tl-* + $(shsrcdir)/prep-source.sh $(ibdir) sed -e's|@installdir[@]|$(idir)|g' \ "$$topdir"/reproduce/software/config/texlive.conf \ > texlive.conf +# We do not build TeXLive from source and for its installation it +# downloads components from the web internally; and those components +# can use '/bin/sh' (which will need '$(sys_library_sh_path)'). + export LD_LIBRARY_PATH="$(sys_library_sh_path):$$LD_LIBRARY_PATH" + # TeX Live's installation may fail due to any reason. But TeX Live is # optional (only necessary for building the final PDF). So we don't # want the configure script to fail if it can't run. Possible error @@ -1867,13 +1904,16 @@ $(itidir)/texlive-ready-tlmgr: reproduce/software/config/texlive.conf tarballurl=$$url/install-tl-unx.tar.gz touch $(lockdir)/download downloader="wget --no-use-server-timestamps -O" - if $(downloadwrapper) "$$downloader" $(lockdir)/download \ - $$tarballurl "$(tdir)/install-tl-unx.tar.gz" \ - "$(backupservers)"; then + if $(downloadwrapper) "$$downloader" \ + $(lockdir)/download \ + $$tarballurl \ + "$(tdir)/install-tl-unx.tar.gz" \ + "$(backupservers)"; then cd $(ddir) rm -rf install-tl-* tar -xf $(tdir)/install-tl-unx.tar.gz cd install-tl-* + $(shsrcdir)/prep-source.sh $(ibdir) sed -e's|@installdir[@]|$(idir)|g' \ $$topdir/reproduce/software/config/texlive.conf \ > texlive.conf @@ -1917,6 +1957,11 @@ $(itidir)/texlive: reproduce/software/config/texlive-packages.conf \ echo "" > $@ else +# We do not build TeXLive from source and for its installation it +# downloads components from the web internally; and those +# components can use '/bin/sh' (which needs 'sys_library_sh_path'). + export LD_LIBRARY_PATH="$(sys_library_sh_path):$$LD_LIBRARY_PATH" + # To update itself, tlmgr needs a backup directory. backupdir=$(idir)/texlive/backups mkdir -p $$backupdir @@ -1940,6 +1985,13 @@ $(itidir)/texlive: reproduce/software/config/texlive-packages.conf \ # directory so we don't have to modify 'PATH'. ln -fs $(idir)/texlive/maneage/bin/*/* $(ibdir)/ +# Correct any reference to '/bin/sh' within the installed LaTeX +# files (this is because we do no yet install LaTeX from source): + cdir=$$(pwd) + cd $(idir)/texlive + $(shsrcdir)/prep-source.sh $(ibdir) + cd $$cdir + # Get all the necessary versions. texlive=$$(pdflatex --version \ | awk 'NR==1' \ diff --git a/reproduce/software/make/python.mk b/reproduce/software/make/python.mk index 936fab1..deeccf2 100644 --- a/reproduce/software/make/python.mk +++ b/reproduce/software/make/python.mk @@ -12,8 +12,8 @@ # # ------------------------------------------------------------------------ # -# Copyright (C) 2019-2023 Raul Infante-Sainz <infantesainz@gmail.com> -# Copyright (C) 2019-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> +# Copyright (C) 2019-2025 Raul Infante-Sainz <infantesainz@gmail.com> +# Copyright (C) 2019-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> # # This Makefile is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -93,13 +93,44 @@ $(ibidir)/python-$(python-version): $(ibidir)/libffi-$(libffi-version) export CC=clang export CXX=clang++ fi - $(call gbuild, python-$(python-version),, \ - --without-ensurepip \ - --with-system-ffi \ - --enable-shared, -j$(numthreads)) + +# Unpack the tarball (see below for the necessary modification). + cd $(ddir) + unpackdir=python-$(python-version) + tar -xf $(tdir)/$$unpackdir.tar.lz + cd $$unpackdir + $(shsrcdir)/prep-source.sh $(ibdir) + +# Python's 'setup.py' uses 'os.system' to run shell scripts. On the +# other hand 'os.system' only runs '/bin/sh' (which has its own +# libraries to link to and those are blocked at this level). So we +# need to add an extra line on top of the 'os.system' funciton and +# put '/usr/lib' in 'LD_LIBRARY_PATH' within Python's environment for +# system calls (with 'os.putenv'). + awk '{if(/os.system\(/) \ + { print " os.putenv(\"LD_LIBRARY_PATH\", \"$$LD_LIBRARY_PATH:$(sys_library_sh_path)\");"; \ + print $$0;} \ + else print $$0}' \ + setup.py > setup-tmp.py + mv setup-tmp.py setup.py + +# Do the basic installation and delete the temporary directory. + ./configure SHELL=$(ibdir)/bash \ + --enable-optimizations \ + --without-ensurepip \ + --prefix="$(idir)" \ + --with-system-ffi \ + --enable-shared + $(makewshell) -j$(numthreads) + $(makewshell) install -j$(numthreads) + cd .. + rm -rf $$unpackdir + +# Set the necessary environment variables and finish the build. ln -sf $(ildir)/python$(python-major-version) $(ildir)/python ln -sf $(ibdir)/python$(python-major-version) $(ibdir)/python - ln -sf $(iidir)/python$(python-major-version)m $(iidir)/python$(python-major-version) + ln -sf $(iidir)/python$(python-major-version)m \ + $(iidir)/python$(python-major-version) rm -rf $(ipydir) mkdir $(ipydir) echo "Python $(python-version)" > $@ diff --git a/reproduce/software/make/r-cran.mk b/reproduce/software/make/r-cran.mk index 484fe74..7c86c23 100644 --- a/reproduce/software/make/r-cran.mk +++ b/reproduce/software/make/r-cran.mk @@ -12,8 +12,8 @@ # # ------------------------------------------------------------------------ # -# Copyright (C) 2022-2023 Boud Roukema <boud@cosmo.torun.pl> -# Copyright (C) 2022-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> +# Copyright (C) 2022-2025 Boud Roukema <boud@cosmo.torun.pl> +# Copyright (C) 2022-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> # # This Makefile is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -204,6 +204,7 @@ $(ibidir)/r-cran-$(r-cran-version): \ tar -xf $(tdir)/$$tarball unpackdir=R-$(r-cran-version) cd $$unpackdir + $(shsrcdir)/prep-source.sh $(ibdir) # We need to manually remove the lines with '~autodetect~', they # cause the configure script to crash in version 4.0.2. They are used diff --git a/reproduce/software/make/xorg.mk b/reproduce/software/make/xorg.mk index 1cc87a5..3f99fab 100644 --- a/reproduce/software/make/xorg.mk +++ b/reproduce/software/make/xorg.mk @@ -14,8 +14,8 @@ # # ------------------------------------------------------------------------ # -# Copyright (C) 2021-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> -# Copyright (C) 2021-2023 Raul Infante-Sainz <infantesainz@gmail.com> +# Copyright (C) 2021-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> +# Copyright (C) 2021-2025 Raul Infante-Sainz <infantesainz@gmail.com> # # This Makefile is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by diff --git a/reproduce/software/shell/bashrc.sh b/reproduce/software/shell/bashrc.sh index 814de2c..6bbd774 100755 --- a/reproduce/software/shell/bashrc.sh +++ b/reproduce/software/shell/bashrc.sh @@ -34,7 +34,7 @@ # shell'). # # -# Copyright (C) 2019-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> +# Copyright (C) 2019-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> # # This script is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by diff --git a/reproduce/software/shell/configure.sh b/reproduce/software/shell/configure.sh index 4f71bee..857f3b5 100755 --- a/reproduce/software/shell/configure.sh +++ b/reproduce/software/shell/configure.sh @@ -2,9 +2,9 @@ # # Necessary preparations/configurations for the reproducible project. # -# Copyright (C) 2018-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> -# Copyright (C) 2021-2023 Raul Infante-Sainz <infantesainz@gmail.com> -# Copyright (C) 2022-2023 Pedram Ashofteh Ardakani <pedramardakani@pm.me> +# Copyright (C) 2018-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> +# Copyright (C) 2021-2025 Raul Infante-Sainz <infantesainz@gmail.com> +# Copyright (C) 2022-2025 Pedram Ashofteh Ardakani <pedramardakani@pm.me> # # This script is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -1252,6 +1252,14 @@ instlibdir="$instdir"/lib if ! [ -d "$instlibdir" ]; then mkdir "$instlibdir"; fi ln -fs "$instlibdir" "$instdir"/lib64 +# Wrapper over Make as a single command so it does not default to '/bin/sh' +# during installation (needed by some programs like CMake). +instbindir=$instdir/bin +if ! [ -d $instbindir ]; then mkdir $instbindir; fi +makewshell="$instbindir/make-with-shell" +echo "$instbindir/make SHELL=$instbindir/bash \$@" > $makewshell +chmod +x $makewshell + @@ -1558,6 +1566,35 @@ fi +# Libraries necessary for the system's shell +# ------------------------------------------ +# +# In some cases (mostly the programs that Maneage doesn't yet build by +# itself), the programs may call the system's shell, not Maneage's +# shell. After we close-off the system environment from Maneage, this will +# cause a crash! To avoid such cases, we need to find the locations of the +# libraries that the shell needs and temporarily add them to the library +# search path. +if [ x"$$on_mac_os" != xyes ]; then + sys_library_sh_path=$(otool -L /bin/sh \ + | awk '/\/lib/{print $1}' \ + | sed 's#/[^/]*$##' \ + | sort \ + | uniq \ + | awk '{printf "%s:", $1}END{printf "\b"}') +else + sys_library_sh_path=$(ldd /bin/sh \ + | awk '{if($3!="") print $3}' \ + | sed 's#/[^/]*$##' \ + | sort \ + | uniq \ + | awk '{printf "%s:", $1}END{printf "\b"}') +fi + + + + + # Find Zenodo URL for software downloading # ---------------------------------------- # @@ -1646,6 +1683,7 @@ fi # tools, but we have to be very portable (and use minimal features in all). echo; echo "Building necessary software (if necessary)..." .local/bin/make $keepgoing -f reproduce/software/make/basic.mk \ + sys_library_sh_path=$sys_library_sh_path \ user_backup_urls="$user_backup_urls" \ sys_library_path=$sys_library_path \ rpath_command=$rpath_command \ @@ -1673,17 +1711,29 @@ else numthreads=$jobs fi .local/bin/env -i HOME=$bdir \ - .local/bin/make $keepgoing -f reproduce/software/make/high-level.mk \ - user_backup_urls="$user_backup_urls" \ - sys_library_path=$sys_library_path \ - rpath_command=$rpath_command \ - all_highlevel=$all_highlevel \ - static_build=$static_build \ - numthreads=$numthreads \ - on_mac_os=$on_mac_os \ - sys_cpath=$sys_cpath \ - host_cc=$host_cc \ - -j$numthreads + .local/bin/make $keepgoing \ + -f reproduce/software/make/high-level.mk \ + sys_library_sh_path=$sys_library_sh_path \ + user_backup_urls="$user_backup_urls" \ + sys_library_path=$sys_library_path \ + rpath_command=$rpath_command \ + all_highlevel=$all_highlevel \ + static_build=$static_build \ + numthreads=$numthreads \ + on_mac_os=$on_mac_os \ + sys_cpath=$sys_cpath \ + host_cc=$host_cc \ + -j$numthreads + + + + + +# Delete the temporary Make wrapper +# --------------------------------- +# +# See above for its description. +rm $makewshell diff --git a/reproduce/software/shell/git-post-checkout b/reproduce/software/shell/git-post-checkout index a1340d7..7b521a3 100755 --- a/reproduce/software/shell/git-post-checkout +++ b/reproduce/software/shell/git-post-checkout @@ -4,7 +4,7 @@ # controlled files (with each commit) using the 'metastore' program. # # Copyright (C) 2016 Przemyslaw Pawelczyk <przemoc@gmail.com> -# Copyright (C) 2018-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> +# Copyright (C) 2018-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> # # This script is taken from the 'examples/hooks/pre-commit' file of the # 'metastore' package (installed within the project, with an MIT license diff --git a/reproduce/software/shell/git-pre-commit b/reproduce/software/shell/git-pre-commit index 02ba488..7b98ad0 100755 --- a/reproduce/software/shell/git-pre-commit +++ b/reproduce/software/shell/git-pre-commit @@ -4,7 +4,7 @@ # controlled files (with each commit) using the 'metastore' program. # # Copyright (C) 2016 Przemyslaw Pawelczyk <przemoc@gmail.com> -# Copyright (C) 2018-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> +# Copyright (C) 2018-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> # # WARNING: # diff --git a/reproduce/software/shell/pre-make-build.sh b/reproduce/software/shell/pre-make-build.sh index 808429b..93d3266 100755 --- a/reproduce/software/shell/pre-make-build.sh +++ b/reproduce/software/shell/pre-make-build.sh @@ -2,7 +2,7 @@ # # Very basic tools necessary to start Maneage's default building. # -# Copyright (C) 2020-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> +# Copyright (C) 2020-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> # # This script is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -48,7 +48,7 @@ instdir="$sdir"/installed tmpblddir="$sdir"/build-tmp confdir=reproduce/software/config ibidir="$instdir"/version-info/proglib -downloadwrapper=reproduce/analysis/bash/download-multi-try +downloadwrapper=reproduce/analysis/bash/download-multi-try.sh # Derived directories bindir="$instdir"/bin @@ -123,7 +123,8 @@ download_tarball() { # Make sure this is the correct tarball. if type sha512sum > /dev/null 2> /dev/null; then checksum=$(sha512sum "$ucname" | awk '{print $1}') - expectedchecksum=$(awk '/^'$progname'-checksum/{print $3}' "$checksumsfile") + expectedchecksum=$(awk '/^'$progname'-checksum/{print $3}' \ + "$checksumsfile") if [ x$checksum = x$expectedchecksum ]; then mv "$ucname" "$maneagetar" else echo "ERROR: Non-matching checksum: $tarball" @@ -135,8 +136,10 @@ download_tarball() { fi fi - # If the tarball is newer than the (possibly existing) program (the version - # has changed), then delete the program. + # If the tarball is newer than the (possibly existing) program (the + # version has changed), then delete the program. When the LaTeX name is + # not given here, the software is re-built later (close to the end of + # 'basic.mk') and the name is properly placed there. if [ -f "$ibidir/$progname" ]; then if [ "$maneagetar" -nt "$ibidir/$progname" ]; then rm "$ibidir/$progname" @@ -210,10 +213,15 @@ build_program() { fi fi - # Clean up the source directory + # Clean up the source directory and write the LaTeX name of the + # software (if necessary: otherwise, just make an empty file). cd "$topdir" rm -rf "$tmpblddir/$unpackdir" - echo "$progname_tex $version" > "$ibidir/$progname" + if [ x"$progname_tex" = x ]; then + echo "" > "$ibidir/$progname" + else + echo "$progname_tex $version" > "$ibidir/$progname" + fi fi } @@ -256,7 +264,7 @@ build_program # Make that has a different executable name (using the '--program-prefix=' # configure option) from the "default" make (which is this one!). progname="make" -progname_tex="GNU Make" +progname_tex="" # Make re-built after GCC (empty string to avoid repetition) url=$(awk '/^'$progname'-url/{print $3}' $urlfile) version=$(awk '/^'$progname'-version/{print $3}' $versionsfile) tarball=$progname-$version.tar.lz diff --git a/reproduce/software/shell/prep-source.sh b/reproduce/software/shell/prep-source.sh new file mode 100755 index 0000000..dcdc472 --- /dev/null +++ b/reproduce/software/shell/prep-source.sh @@ -0,0 +1,94 @@ +#!/usr/bin/env sh +# +# Necessary corrections in the un-packed source of programs to make them +# portable (for example to not use '/bin/sh'). +# +# Usage: Run in top source directory (will work on all files within the +# directory that it is run in ): +# ./prep-source.sh /FULL/ADDRESS/TO/DESIRED/BIN +# +# Copyright (C) 2024-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> +# +# This script is free software: you can redistribute it and/or modify it +# under the terms of the GNU General Public License as published by the +# Free Software Foundation, either version 3 of the License, or (at your +# option) any later version. +# +# This script is distributed in the hope that it will be useful, but +# WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General +# Public License for more details. +# +# You should have received a copy of the GNU General Public License along +# with this script. If not, see <http://www.gnu.org/licenses/>. + + + + + +# Abort the script in case of an error +set -e + + + + +# Read the first argument. +bindir="$1" +if [ x"$bindir" = x ]; then + printf "$0: no argument (location of the 'bin/' directory " + printf "containing the 'bash' executable)\n" + exit 1 +elif ! [ -d "$bindir" ]; then + printf "$0: the directory given as the first argument ('$bindir')" + printf "does not exist" +fi + + + + + +# Find all the files that contain the '/bin/sh' string and correct them to +# Maneage's own Bash. We are using 'while read' to read the file names line +# by line. This is necessary to account file names that include the 'SPACE' +# character (happens in CMake for example!). +# +# Note that dates are important in the source directory (files depend on +# each other), so we should read the original date and after making. We are +# also not using GNU SED's '-i' ('--in-place') option because the host OS +# may not have GNU SED. +# +# Actual situation which prompted the addition of this step: a Maneage'd +# project (with GNU Bash 5.1.8 and Readline 8.1.1) was being built on a +# system where '/bin/sh' was GNU Bash 5.2.26 and had Readline 8.2.010. The +# newer version of Bash needed the newer Readline library function(s) that +# were not available in Maneage's Readline library. Therefore, as soon as +# the basic software were built and Maneage entered the creation of +# high-level software (where we completely close-off the host environment), +# Maneage crashed with the following '/bin/sh' error: +# +# /bin/sh: symbol lookup error: /bin/sh: undefined symbol: rl_trim_arg_from_keyseq +# +# This lead to the discovery that through '/bin/sh' the host operating +# system was leaking into our closed Maneage environment which needs to be +# closed. This needs a source-level correction because '/bin/sh' is +# hard-coded in the source code of almost all programs (their build +# scripts); and in special programs like GNU Make, GNU M4 or CMake it is +# actually hardcoded in the source code (not just build scripts). +if [ -f "$bindir/bash" ]; then shpath="$bindir"/bash +else shpath="$bindir"/dash +fi +grep -I -r -e'/bin/sh' $(pwd)/* \ + | sed -e's|:|\t|' \ + | awk 'BEGIN{FS="\t"}{print $1}' \ + | sort \ + | uniq \ + | while read filename; do \ + tmp="$filename".tmp; \ + origtime="$(date -R -r "$filename")"; \ + origperm=$(stat -c '%a' "$filename"); \ + sed -e's|/bin/sh|'"$shpath"'|g' "$filename" > "$tmp"; \ + mv "$tmp" "$filename"; \ + chmod $origperm "$filename"; \ + touch -d"$origtime" "$filename"; \ + echo "Corrected /bin/sh in $filename"; \ + done diff --git a/reproduce/software/shell/run-parts.in b/reproduce/software/shell/run-parts.in index a7b7705..053f5f3 100755 --- a/reproduce/software/shell/run-parts.in +++ b/reproduce/software/shell/run-parts.in @@ -10,8 +10,8 @@ # However, it didn't have a copyright statement. So one is being added # here. # -# Copyright (C) 2023 Authors mentioned above. -# Copyright (C) 2020-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> +# Copyright (C) 2025 Authors mentioned above. +# Copyright (C) 2025-2024 Mohammad Akhlaghi <mohammad@akhlaghi.org> # # This script is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by diff --git a/reproduce/software/shell/tarball-prepare.sh b/reproduce/software/shell/tarball-prepare.sh index 472b4c0..e12ff98 100755 --- a/reproduce/software/shell/tarball-prepare.sh +++ b/reproduce/software/shell/tarball-prepare.sh @@ -15,8 +15,8 @@ # # Discussion: https://savannah.nongnu.org/task/?15699 # -# Copyright (C) 2022-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> -# Copyright (C) 2022-2023 Pedram Ashofteh Ardakani <pedramardakani@pm.me> +# Copyright (C) 2022-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> +# Copyright (C) 2022-2025 Pedram Ashofteh Ardakani <pedramardakani@pm.me> # Released under GNU GPLv3+ # Abort the script in case of an error. diff --git a/tex/README.md b/tex/README.md index 4963bf5..f100c76 100644 --- a/tex/README.md +++ b/tex/README.md @@ -1,7 +1,7 @@ Directory containing LaTeX-related files ---------------------------------------- -Copyright (C) 2018-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org>\ +Copyright (C) 2018-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org>\ See the end of the file for license conditions. This directory contains directories to various components the LaTeX part of diff --git a/tex/src/delete-me-image-histogram.tex b/tex/src/delete-me-image-histogram.tex index 37618ec..6201c31 100644 --- a/tex/src/delete-me-image-histogram.tex +++ b/tex/src/delete-me-image-histogram.tex @@ -1,6 +1,6 @@ %% Plot the demonstration image and its histogram. % -%% Copyright (C) 2019-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> +%% Copyright (C) 2019-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> % %% This file is free software: you can redistribute it and/or modify it %% under the terms of the GNU General Public License as published by the diff --git a/tex/src/delete-me-squared.tex b/tex/src/delete-me-squared.tex index 968b4d7..51e5f5b 100644 --- a/tex/src/delete-me-squared.tex +++ b/tex/src/delete-me-squared.tex @@ -1,6 +1,6 @@ %% PGFPlots code to plot a random set of numbers as demo %% -%% Copyright (C) 2019-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> +%% Copyright (C) 2019-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> % %% This file is free software: you can redistribute it and/or modify it %% under the terms of the GNU General Public License as published by the diff --git a/tex/src/preamble-biblatex.tex b/tex/src/preamble-biblatex.tex index fd4bf9d..e80d7d8 100644 --- a/tex/src/preamble-biblatex.tex +++ b/tex/src/preamble-biblatex.tex @@ -12,7 +12,7 @@ %% helps in clearly identifying the files and avoiding places that %% complain about '.bib' files. % -%% Copyright (C) 2018-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> +%% Copyright (C) 2018-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> % %% This file is free software: you can redistribute it and/or modify it %% under the terms of the GNU General Public License as published by the diff --git a/tex/src/preamble-maneage-default-style.tex b/tex/src/preamble-maneage-default-style.tex index f811743..b7aa811 100644 --- a/tex/src/preamble-maneage-default-style.tex +++ b/tex/src/preamble-maneage-default-style.tex @@ -6,7 +6,7 @@ %% style defined here as a preamble. To ignore it, uncomment or delete the %% respective line in 'paper.tex'. % -%% Copyright (C) 2019-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> +%% Copyright (C) 2019-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> % %% This file is free software: you can redistribute it and/or modify it %% under the terms of the GNU General Public License as published by the diff --git a/tex/src/preamble-maneage.tex b/tex/src/preamble-maneage.tex index 5dc89fe..9e6cc5a 100644 --- a/tex/src/preamble-maneage.tex +++ b/tex/src/preamble-maneage.tex @@ -1,6 +1,6 @@ %% Preamble for Maneage-related features. % -%% Copyright (C) 2020-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> +%% Copyright (C) 2020-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> % %% This LaTeX file is part of Maneage. Maneage is free software: you can %% redistribute it and/or modify it under the terms of the GNU General diff --git a/tex/src/preamble-pgfplots.tex b/tex/src/preamble-pgfplots.tex index 86f649c..74f5290 100644 --- a/tex/src/preamble-pgfplots.tex +++ b/tex/src/preamble-pgfplots.tex @@ -40,7 +40,7 @@ %% hasn't changed since the last build. TiKZ/PGFPlots won't rebuild the %% plot. % -%% Copyright (C) 2018-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> +%% Copyright (C) 2018-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> % %% This file is part of Maneage (https://maneage.org). % diff --git a/tex/src/preamble-project.tex b/tex/src/preamble-project.tex index 76603f2..adb99b7 100644 --- a/tex/src/preamble-project.tex +++ b/tex/src/preamble-project.tex @@ -4,7 +4,7 @@ %% LaTeX usages. However, if any are not needed in your work, please feel %% free to remove them. % -%% Copyright (C) 2018-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> +%% Copyright (C) 2018-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> %% Copyright (C) YYYY Your Name <your@email.address> % %% This file is free software: you can redistribute it and/or modify it diff --git a/tex/src/references.tex b/tex/src/references.tex index e4117ae..ae3e5d1 100644 --- a/tex/src/references.tex +++ b/tex/src/references.tex @@ -1,7 +1,7 @@ %% Non-software BibTeX entries. The software-specific BibTeX entries are %% stored in a '*.tex' file under the 'tex/dependencies' directory. % -%% Copyright (C) 2018-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org> +%% Copyright (C) 2018-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org> % %% Copying and distribution of this file, with or without modification, %% are permitted in any medium without royalty provided the copyright |