aboutsummaryrefslogtreecommitdiff
path: root/reproduce/software/make
diff options
context:
space:
mode:
authorMohammad Akhlaghi <mohammad@akhlaghi.org>2024-05-03 13:07:41 +0200
committerMohammad Akhlaghi <mohammad@akhlaghi.org>2025-02-10 19:34:26 +0100
commit3c9bf3aff30f02c7d31bd86f36c4db2520f8ffa4 (patch)
treec4b6ee74518ae5256626f6888c2ceccfa66ce2a2 /reproduce/software/make
parent8161194d68665648b93389adb333e741ba230497 (diff)
Configuration: no dependency on /bin/sh and useful run-time options
SUMMARY: no change necessary in your project, this commit only involves changes in how already-existing software are built. Some handy options have also been added to the top-level project script and the copyright years have been updated. Until now, if the host's '/bin/sh' had conflicts with the Maneage environment, the configuration of Maneage would crash as soon as we entered the building of high-level software. The full scenario is described in the comments of the newly added 'reproduce/software/shell/prep-source.sh'. This is most relevant when building older Maneage'd project in newer environments. With this commit, the following changes were made to avoid the problem above: - Maneage edits the source code of all installed software to replace '/bin/sh' with Maneage's own shell before the programs are built. Through this, we were able to solve the problem described above. - The portable '#!/usr/bin/env sh' shebangs are now used at the start of the scripts that run during configure time so it uses the first available shell that it finds in its PATH (the system's before Dash is built), then Dash, and after Dash is built, Bash. - For TeXLive, since we don't install it from source, it was necessary to add the libraries necessary for the local '/bin/sh' in LD_LIBRARY_PATH. Some high-level options have been added to the './project' script to simplify certain operations: --keep-going: do not stop upon the first crash, but keep going on to build targets until all build-able targets have been built. This is very useful for debugging large pipelines and allows you to isolate the problematic part of your project. --highlight-all: equivalent to calling both '--highlight-new' & '--highlight-notes'.
Diffstat (limited to 'reproduce/software/make')
-rw-r--r--reproduce/software/make/README.md2
-rw-r--r--reproduce/software/make/atlas-multiple.mk2
-rw-r--r--reproduce/software/make/atlas-single.mk2
-rw-r--r--reproduce/software/make/basic.mk93
-rw-r--r--reproduce/software/make/build-rules.mk35
-rw-r--r--reproduce/software/make/high-level.mk90
-rw-r--r--reproduce/software/make/python.mk45
-rw-r--r--reproduce/software/make/r-cran.mk5
-rw-r--r--reproduce/software/make/xorg.mk4
9 files changed, 193 insertions, 85 deletions
diff --git a/reproduce/software/make/README.md b/reproduce/software/make/README.md
index d2b6952..00afef4 100644
--- a/reproduce/software/make/README.md
+++ b/reproduce/software/make/README.md
@@ -1,7 +1,7 @@
Software building instructions
------------------------------
-Copyright (C) 2019-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org>\
+Copyright (C) 2019-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org>\
See the end of the file for license conditions.
This directory contains Makefiles that are called by the high-level
diff --git a/reproduce/software/make/atlas-multiple.mk b/reproduce/software/make/atlas-multiple.mk
index 5c8f14c..6e7d415 100644
--- a/reproduce/software/make/atlas-multiple.mk
+++ b/reproduce/software/make/atlas-multiple.mk
@@ -8,7 +8,7 @@
#
# ------------------------------------------------------------------------
#
-# Copyright (C) 2019-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org>
+# Copyright (C) 2019-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org>
#
# This Makefile is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
diff --git a/reproduce/software/make/atlas-single.mk b/reproduce/software/make/atlas-single.mk
index 4e2b04a..2b68677 100644
--- a/reproduce/software/make/atlas-single.mk
+++ b/reproduce/software/make/atlas-single.mk
@@ -8,7 +8,7 @@
#
# ------------------------------------------------------------------------
#
-# Copyright (C) 2019-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org>
+# Copyright (C) 2019-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org>
#
# This Makefile is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
diff --git a/reproduce/software/make/basic.mk b/reproduce/software/make/basic.mk
index 99e81d2..68c40b3 100644
--- a/reproduce/software/make/basic.mk
+++ b/reproduce/software/make/basic.mk
@@ -21,9 +21,9 @@
#
# ------------------------------------------------------------------------
#
-# Copyright (C) 2018-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org>
-# Copyright (C) 2019-2023 Raul Infante-Sainz <infantesainz@gmail.com>
-# Copyright (C) 2022-2023 Pedram Ashofteh Ardakani <pedramardakani@pm.me>
+# Copyright (C) 2018-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org>
+# Copyright (C) 2019-2025 Raul Infante-Sainz <infantesainz@gmail.com>
+# Copyright (C) 2022-2025 Pedram Ashofteh Ardakani <pedramardakani@pm.me>
#
# This Makefile is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -49,14 +49,15 @@ include reproduce/software/config/checksums.conf
include reproduce/software/config/urls.conf
# Basic directories
-lockdir = $(BDIR)/software/locks
-tdir = $(BDIR)/software/tarballs
-ddir = $(BDIR)/software/build-tmp
-idir = $(BDIR)/software/installed
-ibdir = $(BDIR)/software/installed/bin
-ildir = $(BDIR)/software/installed/lib
-iidir = $(BDIR)/software/installed/include
-ibidir = $(BDIR)/software/installed/version-info/proglib
+lockdir = $(BDIR)/software/locks
+tdir = $(BDIR)/software/tarballs
+ddir = $(BDIR)/software/build-tmp
+idir = $(BDIR)/software/installed
+ibdir = $(BDIR)/software/installed/bin
+ildir = $(BDIR)/software/installed/lib
+iidir = $(BDIR)/software/installed/include
+shsrcdir = "$(shell pwd)"/reproduce/software/shell
+ibidir = $(BDIR)/software/installed/version-info/proglib
# Ultimate Makefile target. GNU Nano (a simple and very light-weight text
# editor) is installed by default, it is recommended to have it in the
@@ -107,6 +108,15 @@ export CPPFLAGS := -I$(idir)/include $(CPPFLAGS) $(noccwarnings)
# 'LD_LIBRARY_PATH', then we'll add our own newly installed libraries. We
# will also make sure that there is no "current directory" in it (by
# removing a starting or trailing ':' and any occurance of '::'.
+#
+# But first: in case LD_LIBRARY_PATH is empty, give it the default value of
+# $(sys_library_sh_path) (which was the location of the libraries needed by
+# the host's shell). This is because after we add the Maneage's library
+# path, on some systems, no other libraries will be checked except those
+# that are in 'LD_LIBRARY_PATH'.
+ifeq ($(strip $(LD_LIBRARY_PATH)),)
+export LD_LIBRARY_PATH=$(sys_library_sh_path)
+endif
export LD_LIBRARY_PATH := $(shell echo $(LD_LIBRARY_PATH):$(ildir) \
| sed -e's/::/:/g' -e's/^://' -e's/:$$//')
@@ -127,7 +137,7 @@ export DYLD_LIBRARY_PATH :=
# Afer putting everything together, we use the first server as the
# reference for all software if their '-url' variable isn't defined (in
# 'reproduce/software/config/urls.conf').
-downloadwrapper = ./reproduce/analysis/bash/download-multi-try
+downloadwrapper = ./reproduce/analysis/bash/download-multi-try.sh
maneage_backup_urls := $(shell awk '!/^#/{printf "%s ", $$1}' \
reproduce/software/config/servers-backup.conf)
backupservers_all = $(user_backup_urls) $(maneage_backup_urls)
@@ -336,6 +346,7 @@ $(ibidir)/bzip2-$(bzip2-version): $(ibidir)/gzip-$(gzip-version)
rm -rf $$tdir
tar -xf $(tdir)/$$tarball
cd $$tdir
+ $(shsrcdir)/prep-source.sh $(ibdir)
sed -e 's@\(ln -s -f \)$$(PREFIX)/bin/@\1@' Makefile \
> Makefile.sed
mv Makefile.sed Makefile
@@ -634,6 +645,7 @@ $(ibidir)/perl-$(perl-version): $(ibidir)/patchelf-$(patchelf-version)
rm -rf perl-$(perl-version)
tar -xf $(tdir)/$$tarball
cd perl-$(perl-version)
+ $(shsrcdir)/prep-source.sh $(ibdir)
./Configure -des \
-Dusethreads \
-Duseshrplib \
@@ -691,21 +703,16 @@ $(ibidir)/coreutils-$(coreutils-version): \
$(ibidir)/perl-$(perl-version) \
$(ibidir)/openssl-$(openssl-version)
-# Import, unpack and enter the source directory.
+# Import the source tarball.
tarball=coreutils-$(coreutils-version).tar.lz
$(call import-source, $(coreutils-url), $(coreutils-checksum))
+
+# Unpack and enter the source.
cd $(ddir)
rm -rf coreutils-$(coreutils-version)
tar -xf $(tdir)/$$tarball
cd coreutils-$(coreutils-version)
-
-# Set the configure script to use our shell, note that we can't
-# assume GNU SED here yet (it installs after Coreutils).
- sed -e's|\#\! /bin/sh|\#\! $(ibdir)/bash|' \
- -e's|\#\!/bin/sh|\#\! $(ibdir)/bash|' \
- configure > configure-tmp
- mv configure-tmp configure
- chmod +x configure
+ $(shsrcdir)/prep-source.sh $(ibdir)
# Configure, build and install Coreutils.
./configure --prefix=$(idir) SHELL=$(ibdir)/bash \
@@ -744,6 +751,7 @@ $(ibidir)/podlators-$(podlators-version): $(ibidir)/perl-$(perl-version)
rm -rf podlators-$(podlators-version)
tar -xf $(tdir)/$$tarball
cd podlators-$(podlators-version)
+ $(shsrcdir)/prep-source.sh $(ibdir)
perl Makefile.PL
make
make install
@@ -1030,7 +1038,7 @@ $(ibidir)/gmp-$(gmp-version): \
$(call import-source, $(gmp-url), $(gmp-checksum))
$(call gbuild, gmp-$(gmp-version), static, \
--enable-cxx --enable-fat, \
- -j$(numthreads) ,make check)
+ -j$(numthreads))
echo "GNU Multiple Precision Arithmetic Library $(gmp-version)" > $@
# Less is useful with Git (to view the diffs within a minimal container)
@@ -1070,16 +1078,27 @@ $(ibidir)/libtool-$(libtool-version): $(ibidir)/m4-$(m4-version)
$(ibidir)/grep-$(grep-version): $(ibidir)/coreutils-$(coreutils-version)
tarball=grep-$(grep-version).tar.lz
$(call import-source, $(grep-url), $(grep-checksum))
- $(call gbuild, grep-$(grep-version), static,,V=1)
+ $(call gbuild, grep-$(grep-version), static,, \
+ -j$(numthreads) V=1)
echo "GNU Grep $(grep-version)" > $@
# M4 doesn't depend on PatchELF, but just to be consistent with the
# levels/phases introduced here (where the compressors are level 1,
# PatchELF is level 2, and ...), we'll set it as a dependency.
+#
+# The '--with-syscmd-shell' is used as the default shell and if not given,
+# 'm4' will use '/bin/sh' (which is not under Maneage control and can cause
+# problems in 'high-level.mk' because it closes off the system's
+# LD_LIBRARY_PATH and if the system's '/bin/sh' needs a special system
+# library, the high-level programs will not be built). We are setting this
+# default shell to Dash because M4 is built before our own Bash. Recall
+# that Dash is built before we enter this Makefile.
$(ibidir)/m4-$(m4-version): $(ibidir)/patchelf-$(patchelf-version)
tarball=m4-$(m4-version).tar.lz
$(call import-source, $(m4-url), $(m4-checksum))
- $(call gbuild, m4-$(m4-version), static,,V=1)
+ $(call gbuild, m4-$(m4-version), static, \
+ --with-syscmd-shell=$(ibdir)/dash, \
+ -j$(numthreads) V=1)
echo "GNU M4 $(m4-version)" > $@
$(ibidir)/mpfr-$(mpfr-version): $(ibidir)/gmp-$(gmp-version)
@@ -1169,7 +1188,7 @@ $(ibidir)/mpc-$(mpc-version): $(ibidir)/mpfr-$(mpfr-version)
echo "" > $@
else
$(call gbuild, mpc-$(mpc-version), static, , \
- -j$(numthreads), make check)
+ -j$(numthreads))
echo "GNU Multiple Precision Complex library" > $@
fi
@@ -1399,6 +1418,7 @@ $(ibidir)/gcc-$(gcc-version): $(ibidir)/binutils-$(binutils-version)
ln -s $$odir/gcc-$(gcc-version) $(ddir)/gcc-$(gcc-version)
fi
cd gcc-$(gcc-version)
+ $(shsrcdir)/prep-source.sh $(ibdir)
# Unfortunately binutils installs headers like 'ansidecl.h' that
# have been seen to conflict with GCC's internal versions of those
@@ -1506,7 +1526,18 @@ $(ibidir)/gcc-$(gcc-version): $(ibidir)/binutils-$(binutils-version)
-# Software that need re-compilation (to use our own libraries)
+# Level 6: need re-compilation
+# ----------------------------
+#
+# The initial build of these was done with the host's settings, which will
+# cause problems later when we completely close-off the host environment.
+$(ibidir)/make-$(make-version): $(ibidir)/gcc-$(gcc-version)
+ tarball=make-$(make-version).tar.lz
+ $(call import-source, $(make-url), $(make-checksum))
+ $(call gbuild, make-$(make-version), static, \
+ --disable-dependency-tracking --without-guile)
+ echo "GNU Make $(make-version)" > $@
+
$(ibidir)/lzip-$(lzip-version): $(ibidir)/gcc-$(gcc-version)
tarball=lzip-$(lzip-version).tar
unpackdir=lzip-$(lzip-version)
@@ -1514,6 +1545,7 @@ $(ibidir)/lzip-$(lzip-version): $(ibidir)/gcc-$(gcc-version)
rm -rf $$unpackdir
tar -xf $(tdir)/$$tarball
cd $$unpackdir
+ $(shsrcdir)/prep-source.sh $(ibdir)
./configure --build --check --installdir="$(ibdir)"
if [ -f $(ibdir)/patchelf ]; then
$(ibdir)/patchelf --set-rpath $(ildir) $(ibdir)/lzip;
@@ -1526,11 +1558,7 @@ $(ibidir)/lzip-$(lzip-version): $(ibidir)/gcc-$(gcc-version)
-
-
-
-
-# Level 6: Basic text editor
+# Level 7: Basic text editor
# --------------------------
#
# If the project is built in a minimal environment, there is no text
@@ -1547,7 +1575,8 @@ $(ibidir)/lzip-$(lzip-version): $(ibidir)/gcc-$(gcc-version)
# nano (and use their own optional high-level text editor). To do this, you
# can just have to manually remove 'nano' from 'targets-proglib' above and
# add their optional text editor in 'TARGETS.conf'.
-$(ibidir)/nano-$(nano-version): $(ibidir)/lzip-$(lzip-version)
+$(ibidir)/nano-$(nano-version): $(ibidir)/lzip-$(lzip-version) \
+ $(ibidir)/make-$(make-version)
tarball=nano-$(nano-version).tar.lz
$(call import-source, $(nano-url), $(nano-checksum))
$(call gbuild, nano-$(nano-version), static)
diff --git a/reproduce/software/make/build-rules.mk b/reproduce/software/make/build-rules.mk
index c160d33..62cb6d5 100644
--- a/reproduce/software/make/build-rules.mk
+++ b/reproduce/software/make/build-rules.mk
@@ -3,7 +3,7 @@
# imported into 'basic.mk' and 'high-level.mk'. They should be activated
# with Make's 'Call' function.
#
-# Copyright (C) 2018-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org>
+# Copyright (C) 2018-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org>
#
# This Makefile is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -207,17 +207,10 @@ gbuild = if [ x$(static_build) = xyes ] && [ "x$(2)" = xstatic ]; then \
else confscript="$(strip $(6))"; \
fi; \
\
- if [ -f $(ibdir)/bash ]; then \
- if [ -f "$$confscript" ]; then \
- sed -e's|\#\! /bin/sh|\#\! $(ibdir)/bash|' \
- -e's|\#\!/bin/sh|\#\! $(ibdir)/bash|' \
- $$confscript > $$confscript-tmp; \
- mv $$confscript-tmp $$confscript; \
- chmod +x $$confscript; \
- fi; \
+ $(shsrcdir)/prep-source.sh $(ibdir); \
+ if [ -f $(ibdir)/bash ]; then \
shellop="SHELL=$(ibdir)/bash"; \
- elif [ -f /bin/bash ]; then shellop="SHELL=/bin/bash"; \
- else shellop="SHELL=/bin/sh"; \
+ else shellop="SHELL=$(ibdir)/dash"; \
fi; \
\
if [ x$$gbuild_prefix = x ]; then prefixdir="$(idir)"; \
@@ -255,10 +248,7 @@ gbuild = if [ x$(static_build) = xyes ] && [ "x$(2)" = xstatic ]; then \
# CMake
# -----
#
-# According to the link below, in CMake '/bin/sh' is hardcoded, so there is
-# no way to change it unfortunately!
-#
-# https://stackoverflow.com/questions/21167014/how-to-set-shell-variable-in-makefiles-generated-by-cmake
+# Used by packages that are built with CMake.
cbuild = if [ x$(static_build) = xyes ] && [ $(2)x = staticx ]; then \
export LDFLAGS="$$LDFLAGS -static"; \
opts="-DBUILD_SHARED_LIBS=OFF"; \
@@ -268,13 +258,18 @@ cbuild = if [ x$(static_build) = xyes ] && [ $(2)x = staticx ]; then \
utarball=$(tdir)/$$tarball; \
$(call uncompress); \
cd $(1); \
- rm -rf project-build; \
- mkdir project-build; \
- cd project-build; \
+ $(shsrcdir)/prep-source.sh $(ibdir); \
+ if [ -f $(ibdir)/bash ]; then \
+ shellop="SHELL=$(ibdir)/bash"; \
+ else shellop="SHELL=$(ibdir)/dash"; \
+ fi; \
+ rm -rf maneage-build; \
+ mkdir maneage-build; \
+ cd maneage-build; \
cmake .. -DCMAKE_LIBRARY_PATH=$(ildir) \
-DCMAKE_INSTALL_PREFIX=$(idir) \
-DCMAKE_VERBOSE_MAKEFILE:BOOL=ON $$opts $(3); \
- make; \
- make install; \
+ make $$shellop; \
+ make $$shellop install; \
cd ../..; \
rm -rf $(1)
diff --git a/reproduce/software/make/high-level.mk b/reproduce/software/make/high-level.mk
index 3f9c40c..2274c45 100644
--- a/reproduce/software/make/high-level.mk
+++ b/reproduce/software/make/high-level.mk
@@ -12,8 +12,8 @@
#
# ------------------------------------------------------------------------
#
-# Copyright (C) 2018-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org>
-# Copyright (C) 2019-2023 Raul Infante-Sainz <infantesainz@gmail.com>
+# Copyright (C) 2018-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org>
+# Copyright (C) 2019-2025 Raul Infante-Sainz <infantesainz@gmail.com>
#
# This Makefile is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -64,6 +64,12 @@ ipydir = $(BDIR)/software/installed/version-info/python
ircrandir = $(BDIR)/software/installed/version-info/r-cran
ilibrcrandir = $(BDIR)/software/installed/lib/R/library
+# Special files.
+makewshell = $(ibdir)/make-with-shell
+
+
+
+
# Targets to build.
ifeq ($(strip $(all_highlevel)),1)
@@ -232,7 +238,7 @@ $(idircustom):; mkdir $@
# Afer putting everything together, we use the first server as the
# reference for all software if their '-url' variable isn't defined (in
# 'reproduce/software/config/urls.conf').
-downloadwrapper = ./reproduce/analysis/bash/download-multi-try
+downloadwrapper = ./reproduce/analysis/bash/download-multi-try.sh
maneage_backup_urls := $(shell awk '!/^#/{printf "%s ", $$1}' \
reproduce/software/config/servers-backup.conf)
backupservers_all = $(user_backup_urls) $(maneage_backup_urls)
@@ -339,6 +345,7 @@ $(ibidir)/atlas-$(atlas-version):
cd $(ddir)
tar -xf $(tdir)/atlas-$(atlas-version).tar.lz
cd ATLAS
+ $(shsrcdir)/prep-source.sh $(ibdir)
rm -rf build
mkdir build
cd build
@@ -395,6 +402,7 @@ $(ibidir)/boost-$(boost-version): \
cd $(ddir)
tar -xf $(tdir)/$$tarball
cd $$unpackdir
+ $(shsrcdir)/prep-source.sh $(ibdir)
./bootstrap.sh --prefix=$(idir) --with-libraries=all \
--with-python=python3
echo "using mpi ;" > project-config.jam
@@ -417,9 +425,7 @@ $(ibidir)/cfitsio-$(cfitsio-version):
topdir=$(pwd); cd $(ddir); tar -xf $(tdir)/$$tarball
customtar=cfitsio-$(cfitsio-version)-custom.tar.gz
cd cfitsio-$(cfitsio-version)
- sed configure -e's|@rpath|$(ildir)|g' > configure_tmp
- mv configure_tmp configure
- chmod +x configure
+ sed -i -e's|@rpath|$(ildir)|g' configure
cd ..
tar cf $$customtar cfitsio-$(cfitsio-version)
cd $$topdir
@@ -518,6 +524,12 @@ $(ibidir)/fftw-$(fftw-version):
echo "FFTW $(fftw-version) \citep{fftw}" > $@
$(ibidir)/freetype-$(freetype-version): $(ibidir)/libpng-$(libpng-version)
+# As of version 2.13.2, FreeType doesn't account for the 'SHELL'
+# environment variable. The issue has been reported to the
+# developers. But until future versions, the work-around was
+# discoverd to be setting the 'GNUMAKE' environment variable so it
+# includes 'SHELL'.
+ export GNUMAKE="$(makewshell)"
tarball=freetype-$(freetype-version).tar.lz
$(call import-source, $(freetype-url), $(freetype-checksum))
$(call gbuild, freetype-$(freetype-version), static)
@@ -578,7 +590,9 @@ $(ibidir)/healpix-$(healpix-version): $(healpix-python-dep) \
rm -rf $(ddir)/Healpix_$(healpix-version)
topdir=$(pwd); cd $(ddir);
tar -xf $(tdir)/$$tarball
- cd Healpix_$(healpix-version)/src/C/autotools/
+ cd Healpix_$(healpix-version)
+ $(shsrcdir)/prep-source.sh $(ibdir)
+ cd src/C/autotools
autoreconf --install
./configure --prefix=$(idir)
make V=1 -j$(numthreads) SHELL=$(ibdir)/bash
@@ -670,6 +684,7 @@ $(ibidir)/libpaper-$(libpaper-version): \
tar -xf $(tdir)/$$tarball
unpackdir=libpaper-$(libpaper-version)
cd $$unpackdir
+ $(shsrcdir)/prep-source.sh $(ibdir)
autoreconf -fi
./configure --prefix=$(idir) --sysconfdir=$(idir)/etc \
--disable-static
@@ -799,6 +814,7 @@ $(ibidir)/openblas-$(openblas-version):
cd $(ddir)
tar -xf $(tdir)/$$tarball
cd OpenBLAS-$(openblas-version)
+ $(shsrcdir)/prep-source.sh $(ibdir)
make -j$(numthreads)
make PREFIX=$(idir) install
cd ..
@@ -1007,6 +1023,7 @@ $(ibidir)/astrometrynet-$(astrometrynet-version): \
rm -rf astrometry.net-$(astrometrynet-version)
tar -xf $(tdir)/$$tarball
cd astrometry.net-$(astrometrynet-version)
+ $(shsrcdir)/prep-source.sh $(ibdir)
sed -e 's|cat /proc/cpuinfo|echo "Ignoring CPU info"|' \
-e 's|-free|echo "Ignoring RAM info"|' Makefile > Makefile.tmp
mv Makefile.tmp Makefile
@@ -1045,6 +1062,7 @@ $(ibidir)/cdsclient-$(cdsclient-version):
cd $(ddir)
tar -xf $(tdir)/$$tarball
cd cdsclient-$(cdsclient-version)
+ $(shsrcdir)/prep-source.sh $(ibdir)
touch *
./configure --prefix=$(idir)
make
@@ -1061,25 +1079,29 @@ $(ibidir)/cmake-$(cmake-version):
tarball=cmake-$(cmake-version).tar.lz
$(call import-source, $(cmake-url), $(cmake-checksum))
-# After searching in 'bootstrap', I couldn't find 'LIBS', only
-# 'LDFLAGS'. So the extra libraries are being added to 'LDFLAGS', not
-# 'LIBS'.
-#
# On Mac systems, the build complains about 'clang' specific
# features, so we can't use our own GCC build here.
if [ x$(on_mac_os) = xyes ]; then
export CC=clang
export CXX=clang++
fi
+
+# CMake wants a single executable for 'MAKE', so we can't use 'make
+# SHELL=$(SHELL) and we have defined this script.
+ export MAKE="$(makewshell)"
+
+# Go into the unpacked directory and build CMake.
cd $(ddir)
rm -rf cmake-$(cmake-version)
tar -xf $(tdir)/$$tarball
cd cmake-$(cmake-version)
+ $(shsrcdir)/prep-source.sh $(ibdir)
./bootstrap --prefix=$(idir) --system-curl --system-zlib \
--system-bzip2 --system-liblzma --no-qt-gui \
--parallel=$(numthreads)
- make -j$(numthreads) LIBS="$$LIBS -lssl -lcrypto -lz" VERBOSE=1
- make install
+ $(makewshell) VERBOSE=1 LIBS="$$LIBS -lssl -lcrypto -lz" \
+ -j$(numthreads)
+ $(makewshell) install
cd ..
rm -rf cmake-$(cmake-version)
echo "CMake $(cmake-version)" > $@
@@ -1129,6 +1151,7 @@ $(ibidir)/ghostscript-$(ghostscript-version): \
cd $(ddir)
tar -xf $(tdir)/$$tarball
cd ghostscript-$(ghostscript-version)
+ $(shsrcdir)/prep-source.sh $(ibdir)
./configure --prefix=$(idir) \
--disable-cups \
--enable-dynamic \
@@ -1170,7 +1193,7 @@ $(ibidir)/gnuastro-$(gnuastro-version): \
$(call gbuild, gnuastro-$(gnuastro-version), static, , \
-j$(numthreads))
cp $(dtexdir)/gnuastro.tex $(ictdir)/
- echo "GNU Astronomy Utilities $(gnuastro-version) \citep{gnuastro,akhlaghi19}" > $@
+ echo "GNU Astronomy Utilities $(gnuastro-version) \citep{gnuastro}" > $@
$(ibidir)/icu-$(icu-version): $(ibidir)/python-$(python-version)
@@ -1188,7 +1211,9 @@ $(ibidir)/icu-$(icu-version): $(ibidir)/python-$(python-version)
cd $(ddir)
tar -xf $(tdir)/$$tarball
unpackdir=icu-$(icu-version)
- cd $$unpackdir/icu4c/source
+ cd $$unpackdir
+ $(shsrcdir)/prep-source.sh $(ibdir)
+ cd icu4c/source
./configure --enable-static --prefix=$(idir)
make -j$(numthreads) V=1
make install
@@ -1255,6 +1280,7 @@ $(ibidir)/imfit-$(imfit-version): \
rm -rf $$unpackdir
tar -xf $(tdir)/$$tarball
cd $$unpackdir
+ $(shsrcdir)/prep-source.sh $(ibdir)
sed -i 's|/usr/local|$(idir)|g' SConstruct
sed -i 's|/usr/include|$(idir)/include|g' SConstruct
sed -i 's|.append(|.insert(0,|g' SConstruct
@@ -1302,6 +1328,7 @@ $(ibidir)/minizip-$(minizip-version): $(ibidir)/automake-$(automake-version)
mkdir $$unpackdir
tar -xf $(tdir)/$$tarball -C$$unpackdir --strip-components=1
cd $$unpackdir
+ $(shsrcdir)/prep-source.sh $(ibdir)
./configure --prefix=$(idir)
make
cd contrib/minizip
@@ -1363,6 +1390,7 @@ $(ibidir)/netpbm-$(netpbm-version): \
rm -rf $$unpackdir
tar -xf $(tdir)/$$tarball
cd $$unpackdir
+ $(shsrcdir)/prep-source.sh $(ibdir)
# As of NetPBM 10.73.39 and Flex 2.6.4-410-74a89fd (commit 74a89fd in
# Flex's Git that is 410 commits after version 2.6.4), there is the
@@ -1482,6 +1510,7 @@ $(ibidir)/scons-$(scons-version): $(ibidir)/python-$(python-version)
rm -rf $$unpackdir
tar -xf $(tdir)/$$tarball
cd $$unpackdir
+ $(shsrcdir)/prep-source.sh $(ibdir)
# Unfortuantely SCons hard-codes its search PATH in its source (to
# use POSIX operating system defaults)! So the only way to modify it
@@ -1625,8 +1654,9 @@ $(ibidir)/util-linux-$(util-linux-version): | $(idircustom)
cd $(ddir)
tar -xf $(tdir)/$$tarball
cd util-linux-$(util-linux-version)
+ $(shsrcdir)/prep-source.sh $(ibdir)
-# If a patch exists for the current version, apply it.
+# If a patch is necessary, apply it.
if [ -f $(patchdir)/util-linux-$(util-linux-version)-macos.patch ]; then
cp $(patchdir)/util-linux-$(util-linux-version)-macos.patch \
util-linux-$(util-linux-version)-macos.patch
@@ -1727,6 +1757,7 @@ $(ibidir)/vim-$(vim-version):
tar -xf $(tdir)/$$tarball
unpackdir=vim-$(vim-version)
cd $(ddir)/$$unpackdir
+ $(shsrcdir)/prep-source.sh $(ibdir)
./configure --prefix=$(idir) \
--disable-canberra \
--enable-multibyte \
@@ -1795,10 +1826,16 @@ $(itidir)/texlive-ready-tlmgr: reproduce/software/config/texlive.conf
rm -rf install-tl-*
tar -xf $(tdir)/install-tl-unx.tar.gz
cd install-tl-*
+ $(shsrcdir)/prep-source.sh $(ibdir)
sed -e's|@installdir[@]|$(idir)|g' \
"$$topdir"/reproduce/software/config/texlive.conf \
> texlive.conf
+# We do not build TeXLive from source and for its installation it
+# downloads components from the web internally; and those components
+# can use '/bin/sh' (which will need '$(sys_library_sh_path)').
+ export LD_LIBRARY_PATH="$(sys_library_sh_path):$$LD_LIBRARY_PATH"
+
# TeX Live's installation may fail due to any reason. But TeX Live is
# optional (only necessary for building the final PDF). So we don't
# want the configure script to fail if it can't run. Possible error
@@ -1867,13 +1904,16 @@ $(itidir)/texlive-ready-tlmgr: reproduce/software/config/texlive.conf
tarballurl=$$url/install-tl-unx.tar.gz
touch $(lockdir)/download
downloader="wget --no-use-server-timestamps -O"
- if $(downloadwrapper) "$$downloader" $(lockdir)/download \
- $$tarballurl "$(tdir)/install-tl-unx.tar.gz" \
- "$(backupservers)"; then
+ if $(downloadwrapper) "$$downloader" \
+ $(lockdir)/download \
+ $$tarballurl \
+ "$(tdir)/install-tl-unx.tar.gz" \
+ "$(backupservers)"; then
cd $(ddir)
rm -rf install-tl-*
tar -xf $(tdir)/install-tl-unx.tar.gz
cd install-tl-*
+ $(shsrcdir)/prep-source.sh $(ibdir)
sed -e's|@installdir[@]|$(idir)|g' \
$$topdir/reproduce/software/config/texlive.conf \
> texlive.conf
@@ -1917,6 +1957,11 @@ $(itidir)/texlive: reproduce/software/config/texlive-packages.conf \
echo "" > $@
else
+# We do not build TeXLive from source and for its installation it
+# downloads components from the web internally; and those
+# components can use '/bin/sh' (which needs 'sys_library_sh_path').
+ export LD_LIBRARY_PATH="$(sys_library_sh_path):$$LD_LIBRARY_PATH"
+
# To update itself, tlmgr needs a backup directory.
backupdir=$(idir)/texlive/backups
mkdir -p $$backupdir
@@ -1940,6 +1985,13 @@ $(itidir)/texlive: reproduce/software/config/texlive-packages.conf \
# directory so we don't have to modify 'PATH'.
ln -fs $(idir)/texlive/maneage/bin/*/* $(ibdir)/
+# Correct any reference to '/bin/sh' within the installed LaTeX
+# files (this is because we do no yet install LaTeX from source):
+ cdir=$$(pwd)
+ cd $(idir)/texlive
+ $(shsrcdir)/prep-source.sh $(ibdir)
+ cd $$cdir
+
# Get all the necessary versions.
texlive=$$(pdflatex --version \
| awk 'NR==1' \
diff --git a/reproduce/software/make/python.mk b/reproduce/software/make/python.mk
index 936fab1..deeccf2 100644
--- a/reproduce/software/make/python.mk
+++ b/reproduce/software/make/python.mk
@@ -12,8 +12,8 @@
#
# ------------------------------------------------------------------------
#
-# Copyright (C) 2019-2023 Raul Infante-Sainz <infantesainz@gmail.com>
-# Copyright (C) 2019-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org>
+# Copyright (C) 2019-2025 Raul Infante-Sainz <infantesainz@gmail.com>
+# Copyright (C) 2019-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org>
#
# This Makefile is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -93,13 +93,44 @@ $(ibidir)/python-$(python-version): $(ibidir)/libffi-$(libffi-version)
export CC=clang
export CXX=clang++
fi
- $(call gbuild, python-$(python-version),, \
- --without-ensurepip \
- --with-system-ffi \
- --enable-shared, -j$(numthreads))
+
+# Unpack the tarball (see below for the necessary modification).
+ cd $(ddir)
+ unpackdir=python-$(python-version)
+ tar -xf $(tdir)/$$unpackdir.tar.lz
+ cd $$unpackdir
+ $(shsrcdir)/prep-source.sh $(ibdir)
+
+# Python's 'setup.py' uses 'os.system' to run shell scripts. On the
+# other hand 'os.system' only runs '/bin/sh' (which has its own
+# libraries to link to and those are blocked at this level). So we
+# need to add an extra line on top of the 'os.system' funciton and
+# put '/usr/lib' in 'LD_LIBRARY_PATH' within Python's environment for
+# system calls (with 'os.putenv').
+ awk '{if(/os.system\(/) \
+ { print " os.putenv(\"LD_LIBRARY_PATH\", \"$$LD_LIBRARY_PATH:$(sys_library_sh_path)\");"; \
+ print $$0;} \
+ else print $$0}' \
+ setup.py > setup-tmp.py
+ mv setup-tmp.py setup.py
+
+# Do the basic installation and delete the temporary directory.
+ ./configure SHELL=$(ibdir)/bash \
+ --enable-optimizations \
+ --without-ensurepip \
+ --prefix="$(idir)" \
+ --with-system-ffi \
+ --enable-shared
+ $(makewshell) -j$(numthreads)
+ $(makewshell) install -j$(numthreads)
+ cd ..
+ rm -rf $$unpackdir
+
+# Set the necessary environment variables and finish the build.
ln -sf $(ildir)/python$(python-major-version) $(ildir)/python
ln -sf $(ibdir)/python$(python-major-version) $(ibdir)/python
- ln -sf $(iidir)/python$(python-major-version)m $(iidir)/python$(python-major-version)
+ ln -sf $(iidir)/python$(python-major-version)m \
+ $(iidir)/python$(python-major-version)
rm -rf $(ipydir)
mkdir $(ipydir)
echo "Python $(python-version)" > $@
diff --git a/reproduce/software/make/r-cran.mk b/reproduce/software/make/r-cran.mk
index 484fe74..7c86c23 100644
--- a/reproduce/software/make/r-cran.mk
+++ b/reproduce/software/make/r-cran.mk
@@ -12,8 +12,8 @@
#
# ------------------------------------------------------------------------
#
-# Copyright (C) 2022-2023 Boud Roukema <boud@cosmo.torun.pl>
-# Copyright (C) 2022-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org>
+# Copyright (C) 2022-2025 Boud Roukema <boud@cosmo.torun.pl>
+# Copyright (C) 2022-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org>
#
# This Makefile is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -204,6 +204,7 @@ $(ibidir)/r-cran-$(r-cran-version): \
tar -xf $(tdir)/$$tarball
unpackdir=R-$(r-cran-version)
cd $$unpackdir
+ $(shsrcdir)/prep-source.sh $(ibdir)
# We need to manually remove the lines with '~autodetect~', they
# cause the configure script to crash in version 4.0.2. They are used
diff --git a/reproduce/software/make/xorg.mk b/reproduce/software/make/xorg.mk
index 1cc87a5..3f99fab 100644
--- a/reproduce/software/make/xorg.mk
+++ b/reproduce/software/make/xorg.mk
@@ -14,8 +14,8 @@
#
# ------------------------------------------------------------------------
#
-# Copyright (C) 2021-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org>
-# Copyright (C) 2021-2023 Raul Infante-Sainz <infantesainz@gmail.com>
+# Copyright (C) 2021-2025 Mohammad Akhlaghi <mohammad@akhlaghi.org>
+# Copyright (C) 2021-2025 Raul Infante-Sainz <infantesainz@gmail.com>
#
# This Makefile is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by