From b1e1522a5a5b6d0800ea207f2d93a4f36bffa68d Mon Sep 17 00:00:00 2001 From: Raul Infante-Sainz Date: Fri, 1 May 2020 12:09:18 +0100 Subject: GNU Gettext built as a dependency of Bash Until now Maneage used the host's GNU Gettext if it was present. Gettext is a relatively low-level software that enables programs to print messages in different languages based on the host environment. Even though it has not direct effect on the running of the software for Maneage and the lanugage environment in Maneage is pre-determined, it is necessary to have it because if the basic programs see it in the host they will link with it and will have problems if/when the host's Gettext is updated. With this commit (which is actually a squashed rebase of 9 commits by Raul and Mohammad), Gettext and its two extra dependencies (libxml2 and libunistring) are now installed within Maneage as a basic software and built before GNU Bash. As a result, all programs built afterwards will successfully link with our own internal version of Gettext and libraries. To get this working, some of the basic software dependencies had to updated and re-ordered and it has been tested in both GNU/Linux and macoS. Some other minor issues that are fixed with this commit - Until this commit, when TeX was not installed, the warning message saying how to run the configure step in order to re-configure the project was not showing the option `-e'. However, the use of this option is more convenient than entering the top-build directory and etc every time. So with this commit, the warning message has been changed in order use the option `-e' in the re-configure of the project. - Until now, on macOS systems, Bash was not linking with our internally built `libncurses'. With this commit, this has been fixed by setting `--withcurses=yes' for Bash's configure script. --- reproduce/software/config/versions.conf | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) (limited to 'reproduce/software/config/versions.conf') diff --git a/reproduce/software/config/versions.conf b/reproduce/software/config/versions.conf index d19f095..6caa449 100644 --- a/reproduce/software/config/versions.conf +++ b/reproduce/software/config/versions.conf @@ -28,7 +28,8 @@ file-version = 5.36 findutils-version = 4.7.0 gawk-version = 5.0.1 gcc-version = 9.2.0 -git-version = 2.23.0 +gettext-version = 0.20.2 +git-version = 2.26.2 gmp-version = 6.1.2 grep-version = 3.3 gzip-version = 1.10 @@ -36,6 +37,8 @@ isl-version = 0.18 libbsd-version = 0.9.1 libiconv-version = 1.16 libtool-version = 2.4.6 +libunistring-version = 0.9.10 +libxml2-version = 2.9.9 lzip-version= 1.20 m4-version = 1.4.18 make-version = 4.3 @@ -102,7 +105,6 @@ libnsl-version = 1.2.0-4a062cf libpng-version = 1.6.37 libtiff-version = 4.0.10 libtirpc-version = 1.1.4 -libxml2-version = 2.9.9 missfits-version = 2.8.0 openblas-version = 0.3.5 openmpi-version = 4.0.1 -- cgit v1.2.1 From c0a512e0631a024f18bec847125a9a03e6aa23a1 Mon Sep 17 00:00:00 2001 From: Mohammad Akhlaghi Date: Fri, 22 May 2020 23:35:21 +0100 Subject: Corrected copyright notices to fit GPL suggested format In time, some of the copyright license description had been mistakenly shortened to two paragraphs instead of the original three that is recommended in the GPL. With this commit, they are corrected to be exactly in the same three paragraph format suggested by GPL. The following files also didn't have a copyright notice, so one was added for them: reproduce/software/make/README.md reproduce/software/bibtex/healpix.tex reproduce/analysis/config/delete-me-num.conf reproduce/analysis/config/verify-outputs.conf --- reproduce/software/config/versions.conf | 13 ++++--------- 1 file changed, 4 insertions(+), 9 deletions(-) (limited to 'reproduce/software/config/versions.conf') diff --git a/reproduce/software/config/versions.conf b/reproduce/software/config/versions.conf index 6caa449..7df4539 100644 --- a/reproduce/software/config/versions.conf +++ b/reproduce/software/config/versions.conf @@ -3,15 +3,10 @@ # Copyright (C) 2018-2020 Mohammad Akhlaghi # Copyright (C) 2019-2020 Raul Infante-Sainz # -# This Makefile is part of Maneage. Maneage is free software: you can -# redistribute it and/or modify it under the terms of the GNU General -# Public License as published by the Free Software Foundation, either -# version 3 of the License, or (at your option) any later version. -# -# Maneage is distributed in the hope that it will be useful, but WITHOUT -# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or -# FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for -# more details. See . +# Copying and distribution of this file, with or without modification, are +# permitted in any medium without royalty provided the copyright notice and +# this notice are preserved. This file is offered as-is, without any +# warranty. -- cgit v1.2.1 From 4493acc390a92ba53c7d76766024e7d64c7b31ce Mon Sep 17 00:00:00 2001 From: Boud Roukema Date: Fri, 15 May 2020 04:42:58 +0200 Subject: New software: Valgrind and Patch With this commit, Maneage now includes instructions to build the memory tracing tool Valgrind and the program 'patch' (to apply corrections/patches in text files and in particular the sources of programs). For this version of Valgrind, some patches were necessary for an interface with OpenMPI 2.x (which is the case now). Also note that this version of Valgrind's checks can fail with GCC 10.1.x (when using '--host-cc'), and the failures aren't due to internal problems but due to how the tests are designed (https://bugs.gentoo.org/707598). So currently if any of Valgrind's checks fail, Maneage still assumes that Valgrind was built and installed successfully. While testing on macOS, we noticed that it needs the macOS-specific 'mig' program which we can't build in Maneage. DESCRIPTION: The mig command invokes the Mach Interface Generator to generate Remote Procedure Call (RPC) code for client-server style Mach IPC from specification files. So a symbolic link to the system's 'mig' is now added to the project's programs on macOS systems. This commit's build of Patch and Valgrind has been tested on two GNU/Linux distributions (Debian and ArchLinux) as well as macOS. Work on this commit started by Boud Roukema, but also involved tests and corrections by Mohammad Akhlaghi and Raul Infante-Sainz. --- reproduce/software/config/versions.conf | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) (limited to 'reproduce/software/config/versions.conf') diff --git a/reproduce/software/config/versions.conf b/reproduce/software/config/versions.conf index 7df4539..e9aad2e 100644 --- a/reproduce/software/config/versions.conf +++ b/reproduce/software/config/versions.conf @@ -104,6 +104,7 @@ missfits-version = 2.8.0 openblas-version = 0.3.5 openmpi-version = 4.0.1 openssh-version = 8.0p1 +patch-version = 2.7.6 pixman-version = 0.38.0 python-version = 3.7.4 R-version = 3.6.2 @@ -216,4 +217,8 @@ minizip-version = $(zlib-version) # From version 1.2 OpenSSL may not need a manual addition, as described in # its comments and `https://savannah.nongnu.org/bugs/?58263'. If it doesn't # cause problems, put it back in the list of "Basic/low-level" tools. -openssl-version = 1.1.1a \ No newline at end of file +openssl-version = 1.1.1a + +# Version 3.15.0 needs two patches, please check if they are necessary on +# any future release. +valgrind-version = 3.15.0 -- cgit v1.2.1 From 3d8aa5953c4e0b79278ab2e27ec4e1051310d04f Mon Sep 17 00:00:00 2001 From: Mohammad Akhlaghi Date: Mon, 1 Jun 2020 03:05:37 +0100 Subject: Core software build before using Make to build other software Until now, Maneage would only build Flock before building everything else using Make (calling 'basic.mk') in parallel. Flock was necessary to avoid parallel downloads during the building of software (which could cause network problems). But after recently trying Maneage on FreeBSD (which is not yet complete, see bug #58465), we noticed that the BSD implemenation of Make couldn't parse 'basic.mk' (in particular, complaining with the 'ifeq' parts) and its shell also had some peculiarities. It was thus decided to also install our own minimalist shell, Make and compressor program before calling 'basic.mk'. In this way, 'basic.mk' can now assume the same GNU Make features that high-level.mk and python.mk assume. The pre-make building of software is now organized in 'reproduce/software/shell/pre-make-build.sh'. Another nice feature of this commit is for macOS users: until now the default macOS Make had problems for parallel building of software, so 'basic.mk' was built in one thread. But now that we can build the core tools with GNU Make on macOS too, it uses all threads. Furthermore, since we now run 'basic.mk' with GNU Make, we can use '.ONESHELL' and don't have to finish every line of a long rule with a backslash to keep variables and such. Generally, the pre-make software are now organized like this: first we build Lzip before anything else: it is downloaded as a simple '.tar' file that is not compressed (only ~400kb). Once Lzip is built, the pre-make phase continues with building GNU Make, Dash (a minimalist shell) and Flock. All of their tarballs are in '.tar.lz'. Maneage then enters 'basic.mk' and the first program it builds is GNU Gzip (itself packaged as '.tar.lz'). Once Gzip is built, we build all the other compression software (all downloaded as '.tar.gz'). Afterwards, any compression standard for other software is fine because we have it. In the process, a bug related to using backup servers was found in 'reproduce/analysis/bash/download-multi-try' for calling outside of 'basic.mk' and removed Bash-specific features. As a result of that bug-fix, because we now have multiple servers for software tarballs, the backup servers now have their own configuration file in 'reproduce/software/config/servers-backup.conf'. This makes it much easier to maintain the backup server list across the multiple places that we need it. Some other minor fixes: - In building Bzip2, we need to specify 'CC' so it doesn't use 'gcc'. - In building Zip, the 'generic_gcc' Make option caused a crash on FreeBSD (which doesn't have GCC). - We are now using 'uname -s' to specify if we are on a Linux kernel or not, if not, we are still using the old 'on_mac_os' variable. - While I was trying to build on FreeBSD, I noticed some further corrections that could help. For example the 'makelink' Make-function now takes a third argument which can be a different name compared to the actual program (used for examle to make a link to '/usr/bin/cc' from 'gcc'. - Until now we didn't know if the host's Make implementation supports placing a '@' at the start of the recipe (to avoid printing the actual commands to standard output). Especially in the tarball download phase, there are many lines that are printed for each download which was really annoying. We already used '@' in 'high-level.mk' and 'python.mk' before, but now that we also know that 'basic.mk' is called with our custom GNU Make, we can use it at the start for a cleaner stdout. - Until now, WCSLIB assumed a Fortran compiler, but when the user is on a system where we can't install GCC (or has activated the '--host-cc' option), it may not be present and the project shouldn't break because of this. So with this commit, when a Fortran compiler isn't present, WCSLIB will be built with the '--disable-fortran' configuration option. This commit (task #15667) was completed with help/checks by Raul Infante-Sainz and Boud Roukema. --- reproduce/software/config/versions.conf | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) (limited to 'reproduce/software/config/versions.conf') diff --git a/reproduce/software/config/versions.conf b/reproduce/software/config/versions.conf index e9aad2e..1a4172a 100644 --- a/reproduce/software/config/versions.conf +++ b/reproduce/software/config/versions.conf @@ -18,6 +18,7 @@ bash-version = 5.0.11 binutils-version = 2.32 coreutils-version = 8.31 curl-version = 7.65.3 +dash-version = 0.5.10.2 diffutils-version = 3.7 file-version = 5.36 findutils-version = 4.7.0 @@ -34,7 +35,7 @@ libiconv-version = 1.16 libtool-version = 2.4.6 libunistring-version = 0.9.10 libxml2-version = 2.9.9 -lzip-version= 1.20 +lzip-version = 1.20 m4-version = 1.4.18 make-version = 4.3 metastore-version = 1.1.2-23-fa9170b -- cgit v1.2.1