aboutsummaryrefslogtreecommitdiff
path: root/reproduce/analysis/make/paper.mk
diff options
context:
space:
mode:
authorMohammad Akhlaghi <mohammad@akhlaghi.org>2021-10-04 02:51:45 +0200
committerMohammad Akhlaghi <mohammad@akhlaghi.org>2022-01-21 01:15:24 +0100
commit8463df97c6f26ec4d22cd5828bb0574fd5e450d2 (patch)
treedbaa2e7c2fc44856eb98555b79c6814f210a6c17 /reproduce/analysis/make/paper.mk
parent775fc036e0091f05ff56e41b855bc416b9ed36c8 (diff)
IMPORTANT: Updates to almost all software
This commit primarily affects the configuration step of Maneage'd projects, and in particular, updated versions of the many of the software (see P.S.). So it shouldn't affect your high-level analysis other than the version bumps of the software you use (and the software's possibly improve/changed behavior). The following software (and thus their dependencies) couldn't be updated as described below: - Cryptography: isn't building because it depends on a new setuptools-rust package that has problems (https://savannah.nongnu.org/bugs/index.php?61731), so it has been commented in 'versions.conf'. - SecretStorage: because it depends on Cryptography. - Keyring: because it depends on SecretStorage. - Astroquery: because it depends on Keyring. This is a "squashed" commit after rebasing a development branch of 60 commits corresponding to a roughly two-month time interval. The following people contributed to this branch. - Boudewijn Roukema added all the R software infrastructure and the R packages, as well as greatly helping in fixing many bugs during the update. - Raul Infante-Sainz helped in testing and debugging the build. - Pedram Ashofteh Ardakani found and fixed a bug. - Zahra Sharbaf helped in testing and found several bugs. Below a description of the most noteworthy points is given. - Software tarballs: all updated software now have a unified format tarball (ustar; if not possible, pax) and unified compression (Lzip) in Maneage's software repository in Zenodo (https://doi.org/10.5281/zenodo.3883409). For more on this See https://savannah.nongnu.org/task/?15699 . This won't affect any extra software you would like to add; you can use any format recognized by GNU Tar, and all common compression algorithms. This new requirement is only for software that get merged to the core Maneage branch. - Metastore (and thus libbsd and libmd) moved to highlevel: Metastore (and the packages it depends on) is a high-level product that is only relevant during the project development (like Emacs!): when the user wants the file meta data (like dates) to be unchanged after checking out branches. So it should be considered a high-level software, not basic. Metastore also usually causes many more headaches and error messages, so personally, I have stopped using it! Instead I simply merge my branches in a separate clone, then pull the merge commit: in this way, the files of my project aren't re-written during the checkout phase and therefore their dates are untouched (which can conflict with Make's dates on configuration files). - The un-official cloned version of Flex (2.6.4-91 until this commit) was causing problems in the building of Netpbm, so with this commit, it has been moved back to version 2.6.4. - Netpbm's official page had version 10.73.38 as the latest stable tarball that was just released in late 2021. But I couldn't find our previously-used version 10.86.99 anywhere (to see when it was released and why we used it! Its at last more than one year old!). So the official stable version is being used now. - Improved instructions in 'README.md' for building software environment in a Docker container (while having project source and output data products on the local system; including the usage of the host's '/dev/shm' to speed up temporary operations). - Until now, the convention in Maneage was to put eight SPACE characters before the comment lines within recipes. This was done because by default GNU Emacs (also many other editors) show a TAB as eight characters. However, in other text editors, online browsers, or even the Git diff, a TAB can correspond to a different number of characters. In such cases, the Maneage recipes wouldn't look too interesting (the comments and the recipe commands would show a different indentation!). With this commit, all the comment lines in the Makefiles within the core Maneage branch have a hash ('#') as their first character and a TAB as the second. This allows the comment lines in recipes to have the same indentation as code; making the code much more easier to read in a general scenario including a 'git diff' (editor agnostic!). P.S. List of updated software with their old and new versions - Software with no version update are not mentioned. - The old version of newly added software are shown with '--'. Name (Basic) Old version New version ------------ ----------- ----------- Bzip2 1.0.6 1.0.8 CURL 7.71.1 7.79.1 Dash 0.5.10.2 0.5.11.5 File 5.39 5.41 Flock 0.2.3 0.4.0 GNU Bash 5.0.18 5.1.8 GNU Binutils 2.35 2.37 GNU Coreutils 8.32 9.0 GNU GCC 10.2.0 11.2.0 GNU M4 1.4.18 1.4.19 GNU Readline 8.0 8.1.1 GNU Tar 1.32 1.34 GNU Texinfo 6.7 6.8 GNU diffutils 3.7 3.8 GNU findutils 4.7.0 4.8.0 GNU gmp 6.2.0 6.2.1 GNU grep 3.4 3.7 GNU gzip 1.10 1.11 GNU libunistring 0.9.10 1.0 GNU mpc 1.1.0 1.2.1 GNU mpfr 4.0.2 4.1.0 GNU nano 5.2 6.0 GNU ncurses 6.2 6.3 GNU wget 1.20.3 1.21.2 Git 2.28.0 2.34.0 Less 563 590 Libxml2 2.9.9 2.9.12 Lzip 1.22-rc2 1.22 OpenSLL 1.1.1a 3.0.0 Patchelf 0.10 0.13 Perl 5.32.0 5.34.0 Podlators -- 4.14 Name (Highlevel) Old version New version ---------------- ----------- ----------- Apachelog4cxx 0.10.0-603 0.12.1 Astrometry.net 0.80 0.85 Boost 1.73.0 1.77.0 CFITSIO 3.48 4.0.0 Cmake 3.18.1 3.21.4 Eigen 3.3.7 3.4.0 Expat 2.2.9 2.4.1 FFTW 3.3.8 3.3.10 Flex 2.6.4-91 2.6.4 Fontconfig 2.13.1 2.13.94 Freetype 2.10.2 2.11.0 GNU Astronomy Utilities 0.12 0.16.1-e0f1 GNU Autoconf 2.69.200-babc 2.71 GNU Automake 1.16.2 1.16.5 GNU Bison 3.7 3.8.2 GNU Emacs 27.1 27.2 GNU GDB 9.2 11.1 GNU GSL 2.6 2.7 GNU Help2man 1.47.11 1.48.5 Ghostscript 9.52 9.55.0 ICU -- 70.1 ImageMagick 7.0.8-67 7.1.0-13 Libbsd 0.10.0 0.11.3 Libffi 3.2.1 3.4.2 Libgit2 1.0.1 1.3.0 Libidn 1.36 1.38 Libjpeg 9b 9d Libmd -- 1.0.4 Libtiff 4.0.10 4.3.0 Libx11 1.6.9 1.7.2 Libxt 1.2.0 1.2.1 Netpbm 10.86.99 10.73.38 OpenBLAS 0.3.10 0.3.18 OpenMPI 4.0.4 4.1.1 Pixman 0.38.0 0.40.0 Python 3.8.5 3.10.0 R 4.0.2 4.1.2 SWIG 3.0.12 4.0.2 Util-linux 2.35 2.37.2 Util-macros 1.19.2 1.19.3 Valgrind 3.15.0 3.18.1 WCSLIB 7.3 7.7 Xcb-proto 1.14 1.14.1 Xorgproto 2020.1 2021.5 Name (Python) Old version New version ------------- ----------- ----------- Astropy 4.0 5.0 Beautifulsoup4 4.7.1 4.10.0 Beniget -- 0.4.1 Cffi 1.12.2 1.15.0 Cryptography 2.6.1 36.0.1 Cycler 0.10.0 0.11.0+} Cython 0.29.21 0.29.24 Esutil 0.6.4 0.6.9 Extension-helpers -- 0.1 Galsim 2.2.1 2.3.3 Gast -- 0.5.3 Jinja2 -- 3.0.3 MPI4py 3.0.3 3.1.3 Markupsafe -- 2.0.1 Numpy 1.19.1 1.21.3 Packaging -- 21.3 Pillow -- 8.4.0 Ply -- 3.11 Pyerfa -- 2.0.0.1 Pyparsing 2.3.1 3.0.4 Pythran -- 0.11.0 Scipy 1.5.2 1.7.3 Setuptools 41.6.0 58.3.0 Six 1.12.0 1.16.0 Uncertainties 3.1.2 3.1.6 Wheel -- 0.37.0 Name (R) Old version New version -------- ----------- ----------- Cli -- 2.5.0 Colorspace -- 2.0-1 Cowplot -- 1.1.1 Crayon -- 1.4.1 Digest -- 0.6.27 Ellipsis -- 0.3.2 Fansi -- 0.5.0 Farver -- 2.1.0 Ggplot2 -- 3.3.4 Glue -- 1.4.2 GridExtra -- 2.3 Gtable -- 0.3.0 Isoband -- 0.2.4 Labeling -- 0.4.2 Lifecycle -- 1.0.0 Magrittr -- 2.0.1 MASS -- 7.3-54 Mgcv -- 1.8-36 Munsell -- 0.5.0 Pillar -- 1.6.1 R-Pkgconfig -- 2.0.3 R6 -- 2.5.0 RColorBrewer -- 1.1-2 Rlang -- 0.4.11 Scales -- 1.1.1 Tibble -- 3.1.2 Utf8 -- 1.2.1 Vctrs -- 0.3.8 ViridisLite -- 0.4.0 Withr -- 2.4.2
Diffstat (limited to 'reproduce/analysis/make/paper.mk')
-rw-r--r--reproduce/analysis/make/paper.mk83
1 files changed, 42 insertions, 41 deletions
diff --git a/reproduce/analysis/make/paper.mk b/reproduce/analysis/make/paper.mk
index b5b5b29..740dc7d 100644
--- a/reproduce/analysis/make/paper.mk
+++ b/reproduce/analysis/make/paper.mk
@@ -1,6 +1,6 @@
# Build the final PDF paper/report.
#
-# Copyright (C) 2018-2021 Mohammad Akhlaghi <mohammad@akhlaghi.org>
+# Copyright (C) 2018-2022 Mohammad Akhlaghi <mohammad@akhlaghi.org>
#
# This Makefile is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -23,14 +23,14 @@
#
# To report the input settings and results, the final report's PDF (final
# target of this project) uses macros generated from various steps of the
-# project. All these macros are defined through `$(mtexdir)/project.tex'.
+# project. All these macros are defined through '$(mtexdir)/project.tex'.
#
-# `$(mtexdir)/project.tex' is actually just a combination of separate files
+# '$(mtexdir)/project.tex' is actually just a combination of separate files
# that keep the LaTeX macros related to each workhorse Makefile (in
-# `reproduce/src/make/*.mk'). Those individual macros are pre-requisites to
-# `$(mtexdir)/verify.tex' which will check them before starting to build
+# 'reproduce/src/make/*.mk'). Those individual macros are pre-requisites to
+# '$(mtexdir)/verify.tex' which will check them before starting to build
# the paper. The only workhorse Makefile that doesn't need to produce LaTeX
-# macros is this Makefile (`reproduce/src/make/paper.mk').
+# macros is this Makefile ('reproduce/src/make/paper.mk').
#
# This file is thus the interface between the analysis/processing steps and
# the final PDF: when we get to this point, all the processing has been
@@ -38,33 +38,33 @@
#
# Note that if you don't want the final PDF and just want the processing
# and file outputs, you can give any value other than 'yes' to
-# 'pdf-build-final' in `reproduce/analysis/config/pdf-build.conf'.
+# 'pdf-build-final' in 'reproduce/analysis/config/pdf-build.conf'.
$(mtexdir)/project.tex: $(mtexdir)/verify.tex
- # If no PDF is requested, or if LaTeX isn't available, don't
- # continue to building the final PDF. Otherwise, merge all the TeX
- # macros into one for building the PDF.
+# If no PDF is requested, or if LaTeX isn't available, don't continue
+# to building the final PDF. Otherwise, merge all the TeX macros into
+# one for building the PDF.
@if [ -f .local/bin/pdflatex ] && [ x"$(pdf-build-final)" = xyes ]; then
- # Put a LaTeX input command for all the necessary macro files.
- # 'hardware-parameters.tex' is created in 'configure.sh'.
+# Put a LaTeX input command for all the necessary macro files.
+# 'hardware-parameters.tex' is created in 'configure.sh'.
projecttex=$(mtexdir)/project.tex
rm -f $$projecttex
for t in $(subst paper,,$(makesrc)) hardware-parameters; do
echo "\input{tex/build/macros/$$t.tex}" >> $$projecttex
done
- # Possibly highlight the '\new' parts of the text.
+# Possibly highlight the '\new' parts of the text.
if [ x"$(highlightnew)" = x1 ]; then
echo "\newcommand{\highlightnew}{}" >> $$projecttex
fi
- # Possibly show the text within '\tonote'.
+# Possibly show the text within '\tonote'.
if [ x"$(highlightnotes)" = x1 ]; then
echo "\newcommand{\highlightnotes}{}" >> $$projecttex
fi
- # The paper shouldn't be built.
+# The paper shouldn't be built.
else
echo
echo "-----"
@@ -95,39 +95,39 @@ $(mtexdir)/project.tex: $(mtexdir)/verify.tex
# The bibliography
# ----------------
#
-# We need to run the `biber' program on the output of LaTeX to generate the
+# We need to run the 'biber' program on the output of LaTeX to generate the
# necessary bibliography before making the final paper. So we'll first have
-# one run of LaTeX (similar to the `paper.pdf' recipe), then `biber'.
+# one run of LaTeX (similar to the 'paper.pdf' recipe), then 'biber'.
#
-# NOTE: `$(mtexdir)/project.tex' is an order-only-prerequisite for
-# `paper.bbl'. This is because we need to run LaTeX in both the `paper.bbl'
-# recipe and the `paper.pdf' recipe. But if `tex/src/references.tex' hasn't
+# NOTE: '$(mtexdir)/project.tex' is an order-only-prerequisite for
+# 'paper.bbl'. This is because we need to run LaTeX in both the 'paper.bbl'
+# recipe and the 'paper.pdf' recipe. But if 'tex/src/references.tex' hasn't
# been modified, we don't want to re-build the bibliography, only the final
# PDF.
$(texbdir)/paper.bbl: tex/src/references.tex $(mtexdir)/dependencies-bib.tex \
| $(mtexdir)/project.tex
- # If `$(mtexdir)/project.tex' is empty, don't build PDF.
+# If '$(mtexdir)/project.tex' is empty, don't build PDF.
@macros=$$(cat $(mtexdir)/project.tex)
if [ x"$$macros" != x ]; then
- # We'll run LaTeX first to generate the `.bcf' file (necessary
- # for `biber') and then run `biber' to generate the `.bbl' file.
+# We'll run LaTeX first to generate the '.bcf' file (necessary for
+# 'biber') and then run 'biber' to generate the '.bbl' file.
p=$$(pwd)
export TEXINPUTS=$$p:
cd $(texbdir);
- # Delete any possibly existing target (a '.bbl' file) to avoid
- # complications with LaTeX being run before the command that
- # generates it. Otherwise users will have to manually delete
- # it. It will be built anyway once this rule is done.
+# Delete any possibly existing target (a '.bbl' file) to avoid
+# complications with LaTeX being run before the command that
+# generates it. Otherwise users will have to manually delete it. It
+# will be built anyway once this rule is done.
rm -f $@
- # The pdflatex option '-shell-escape' is "normally disallowed for
- # security reasons" according to the `info pdflatex' manual, but
- # is enabled here in order to allow the use of PGFPlots. If you
- # do not use PGFPlots, then you should remove the `-shell-escape'
- # option for better security. See
- # https://savannah.nongnu.org/task/?15694 for details.
+# The pdflatex option '-shell-escape' is "normally disallowed for
+# security reasons" according to the 'info pdflatex' manual, but is
+# enabled here in order to allow the use of PGFPlots. If you do not
+# use PGFPlots, then you should remove the '-shell-escape' option
+# for better security. See https://savannah.nongnu.org/task/?15694
+# for details.
pdflatex -shell-escape -halt-on-error "$$p"/paper.tex
biber paper
@@ -140,27 +140,28 @@ $(texbdir)/paper.bbl: tex/src/references.tex $(mtexdir)/dependencies-bib.tex \
# The final paper
# ---------------
#
-# Run LaTeX in the `$(texbdir)' directory so all the intermediate and
+# Run LaTeX in the '$(texbdir)' directory so all the intermediate and
# auxiliary files stay there and keep the top directory clean. To be able
# to run everything cleanly from there, it is necessary to add the current
-# directory (top project directory) to the `TEXINPUTS' environment
+# directory (top project directory) to the 'TEXINPUTS' environment
# variable.
paper.pdf: $(mtexdir)/project.tex paper.tex $(texbdir)/paper.bbl
- # If `$(mtexdir)/project.tex' is empty, don't build the PDF.
+# If '$(mtexdir)/project.tex' is empty, don't build the PDF.
@macros=$$(cat $(mtexdir)/project.tex)
if [ x"$$macros" != x ]; then
- # Go into the top TeX build directory and make the paper.
+# Go into the top TeX build directory and make the paper.
p=$$(pwd)
export TEXINPUTS=$$p:
cd $(texbdir)
- # See above for a warning and brief discussion on the the
- # pdflatex option `-shell-escape'.
+
+# See above for a warning and brief discussion on the the pdflatex
+# option '-shell-escape'.
pdflatex -shell-escape -halt-on-error "$$p"/paper.tex
- # Come back to the top project directory and copy the built PDF
- # file here.
+# Come back to the top project directory and copy the built PDF
+# file here.
cd "$$p"
cp $(texbdir)/$@ $(final-paper)