<feed xmlns='http://www.w3.org/2005/Atom'>
<title>project.git/project, branch maneage</title>
<subtitle>Core Maneage branch (where all projects derive from)</subtitle>
<link rel='alternate' type='text/html' href='http://git.maneage.org/project.git/'/>
<entry>
<title>README-hacking.md corrections and new backup server</title>
<updated>2026-07-29T16:04:09+00:00</updated>
<author>
<name>Mohammad Akhlaghi</name>
<email>mohammad@akhlaghi.org</email>
</author>
<published>2026-07-29T16:04:09+00:00</published>
<link rel='alternate' type='text/html' href='http://git.maneage.org/project.git/commit/?id=6f0ecbe1971ee462facf6b0561633b3cdcfcaad9'/>
<id>6f0ecbe1971ee462facf6b0561633b3cdcfcaad9</id>
<content type='text'>
Summary: this commit will not affect any part of your analysis.

Until now, the following problems existed (their solution that is
implemented with this commit is explained within each):

  - The newly re-organized 'README-hacking.md' file contained the
    "./harvester" term instead of './project'. This happened because this
    restructing was first done in the Arrakihs data reduction pipeline
    (known as the Harvester) and brought upstream (here) after that
    experiment. In the Harvester, the './project' script is renamed to
    './harvester' for more clarity for its users. But I had forgot to
    correct this when doing the merge with Maneage in the previous
    commit. With this commit, the names have been corrected.

  - The 'xlsxio' program depends on 'libbsd', but we had not explicitly set
    this relation through the Make prerequisites. This happened because in
    the past 'libbsd' was built within the "basic" set of software (as a
    dependency of Metastore), so it was present when 'xlsxio' was
    built. But Metastore and libbsd later came into the high-level programs
    and we had forgot to explicitly define this dependency. This bug was
    found by Diana Korotun.

  - The indentations of the './project shell' startup file were too little,
    making it hard to visually distinguish between the text and
    locations.

With this commit, besides the changes mentioned above, the following parts
have also been added:

  - A new backup server (under 'https://archive.org') has been added for
    software tarball download.

  - A new Make tip has been added to 'README-hacking.md' with a minimal
    working example to show how to use pattern rules when there is more
    than one component in the pattern (becomes necessary in large
    projects).
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Summary: this commit will not affect any part of your analysis.

Until now, the following problems existed (their solution that is
implemented with this commit is explained within each):

  - The newly re-organized 'README-hacking.md' file contained the
    "./harvester" term instead of './project'. This happened because this
    restructing was first done in the Arrakihs data reduction pipeline
    (known as the Harvester) and brought upstream (here) after that
    experiment. In the Harvester, the './project' script is renamed to
    './harvester' for more clarity for its users. But I had forgot to
    correct this when doing the merge with Maneage in the previous
    commit. With this commit, the names have been corrected.

  - The 'xlsxio' program depends on 'libbsd', but we had not explicitly set
    this relation through the Make prerequisites. This happened because in
    the past 'libbsd' was built within the "basic" set of software (as a
    dependency of Metastore), so it was present when 'xlsxio' was
    built. But Metastore and libbsd later came into the high-level programs
    and we had forgot to explicitly define this dependency. This bug was
    found by Diana Korotun.

  - The indentations of the './project shell' startup file were too little,
    making it hard to visually distinguish between the text and
    locations.

With this commit, besides the changes mentioned above, the following parts
have also been added:

  - A new backup server (under 'https://archive.org') has been added for
    software tarball download.

  - A new Make tip has been added to 'README-hacking.md' with a minimal
    working example to show how to use pattern rules when there is more
    than one component in the pattern (becomes necessary in large
    projects).
</pre>
</div>
</content>
</entry>
<entry>
<title>IMPORTANT: better names for to-publish dirs and portability fixes</title>
<updated>2026-07-20T20:07:00+00:00</updated>
<author>
<name>Mohammad Akhlaghi</name>
<email>mohammad@akhlaghi.org</email>
</author>
<published>2026-07-20T18:20:56+00:00</published>
<link rel='alternate' type='text/html' href='http://git.maneage.org/project.git/commit/?id=ab5e0aa9fa8a6b94b1306fa120736b6fbb1a2f70'/>
<id>ab5e0aa9fa8a6b94b1306fa120736b6fbb1a2f70</id>
<content type='text'>
Summary: this commit only affects the analysis of your project in case you
used the 'tex-publish-dir' or 'data-publish-dir' directories. They need to
be renamed to 'figdir' and 'drdir' respectively.

Until now, the directories containing the to-be-published data were
described deep in the middle of 'initialize.mk' (away from the other shared
directories), they also had hard-to-use (long) names. Furthermore,
'README-hacking.md' that is the main documentation of Maneage had grown
organically, such that the checklists were scattered across the file and
some things were repeated or not classified clearly. Finally, some
portability issues were reported by Stergios Amarantidis and Diana Korotun.

With this commit, the to-be-published directories have been renamed to the
more simple names of 'figdir' and 'drdir' respectively and their definition
in 'initialize.mk' has come right under the rest of the top-level
directories (to be easily found). Furthermore, the 'README-hacking.md' has
been fully restructured to be easier to navigate and follow for a new
reader. Finally, the portability issues mentioned above have been
addressed:

 - Within the '--help' output of './project', the '--keep-going' option was
   listed under "Configure and Make options", while it is only relevant as
   a Make option. It was therefore moved under Make options.

 - We had not explicitly set the locale of Maneage; causing a crash on one
   of the newly tested computers (a macOS with a non-C locale) during the
   execution of 'prep-source.sh'. With this commit, all the top-level
   Makefiles and scripts of Maneage set 'LC_ALL' to 'C' to ensure the same
   locale on any system.

 - The 'download-multi-try.sh' script would crash when building basic
   software in the following scenario: the downloader was still from the
   host, but one of its linked libraries had already been installed in
   Maneage and conflicted with the host. A step has been added to catch
   such cases and remove the Maneage'd library when this was the problem.

 - On macOS, we discovered that 'pkg-config' and 'wget' need a Python
   installation, so the necessary conditions have been added.

 - The 'apptainer.sh' script did not allow passing the '--debug' or
   '--host-cc' options to the execution of './project' within the
   container.

 - Within 'configure.sh', we wrote the absolute path of the downloader to
   use in 'LOCAL.conf'. However, we install both downloaders and it is
   safer to use Maneage's own downloader when it has been installed. So the
   downloader name in 'LOCAL.conf' now only has the name (without its
   absolute path).
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Summary: this commit only affects the analysis of your project in case you
used the 'tex-publish-dir' or 'data-publish-dir' directories. They need to
be renamed to 'figdir' and 'drdir' respectively.

Until now, the directories containing the to-be-published data were
described deep in the middle of 'initialize.mk' (away from the other shared
directories), they also had hard-to-use (long) names. Furthermore,
'README-hacking.md' that is the main documentation of Maneage had grown
organically, such that the checklists were scattered across the file and
some things were repeated or not classified clearly. Finally, some
portability issues were reported by Stergios Amarantidis and Diana Korotun.

With this commit, the to-be-published directories have been renamed to the
more simple names of 'figdir' and 'drdir' respectively and their definition
in 'initialize.mk' has come right under the rest of the top-level
directories (to be easily found). Furthermore, the 'README-hacking.md' has
been fully restructured to be easier to navigate and follow for a new
reader. Finally, the portability issues mentioned above have been
addressed:

 - Within the '--help' output of './project', the '--keep-going' option was
   listed under "Configure and Make options", while it is only relevant as
   a Make option. It was therefore moved under Make options.

 - We had not explicitly set the locale of Maneage; causing a crash on one
   of the newly tested computers (a macOS with a non-C locale) during the
   execution of 'prep-source.sh'. With this commit, all the top-level
   Makefiles and scripts of Maneage set 'LC_ALL' to 'C' to ensure the same
   locale on any system.

 - The 'download-multi-try.sh' script would crash when building basic
   software in the following scenario: the downloader was still from the
   host, but one of its linked libraries had already been installed in
   Maneage and conflicted with the host. A step has been added to catch
   such cases and remove the Maneage'd library when this was the problem.

 - On macOS, we discovered that 'pkg-config' and 'wget' need a Python
   installation, so the necessary conditions have been added.

 - The 'apptainer.sh' script did not allow passing the '--debug' or
   '--host-cc' options to the execution of './project' within the
   container.

 - Within 'configure.sh', we wrote the absolute path of the downloader to
   use in 'LOCAL.conf'. However, we install both downloaders and it is
   safer to use Maneage's own downloader when it has been installed. So the
   downloader name in 'LOCAL.conf' now only has the name (without its
   absolute path).
</pre>
</div>
</content>
</entry>
<entry>
<title>All: copyright years updated to 2026</title>
<updated>2026-01-01T18:05:42+00:00</updated>
<author>
<name>Mohammad Akhlaghi</name>
<email>mohammad@akhlaghi.org</email>
</author>
<published>2026-01-01T18:05:42+00:00</published>
<link rel='alternate' type='text/html' href='http://git.maneage.org/project.git/commit/?id=d33f7ce35dd3eec4fcb7ee5c30f27b95521e20c7'/>
<id>d33f7ce35dd3eec4fcb7ee5c30f27b95521e20c7</id>
<content type='text'>
Summary: this commit will not affect your project's software or analysis.

Until now, the ending copyright years of all files in Maneage were
2025. But we have already entered 2026 so it is important to update them.

With this commit, the ending copyright year of all files is changed to
2026.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Summary: this commit will not affect your project's software or analysis.

Until now, the ending copyright years of all files in Maneage were
2025. But we have already entered 2026 so it is important to update them.

With this commit, the ending copyright year of all files is changed to
2026.
</pre>
</div>
</content>
</entry>
<entry>
<title>IMPORTANT: software configuration optimized and better modularized</title>
<updated>2025-05-12T08:59:26+00:00</updated>
<author>
<name>Giacomo Lorenzetti</name>
<email>glorenzetti@cefca.es</email>
</author>
<published>2025-04-03T13:21:16+00:00</published>
<link rel='alternate' type='text/html' href='http://git.maneage.org/project.git/commit/?id=df9e291826fbc7e717b40d2d07f1d7607a2f2455'/>
<id>df9e291826fbc7e717b40d2d07f1d7607a2f2455</id>
<content type='text'>
Summary: after merging this commit into your project, it should be
re-configured since the location of software installation files like
'LOCAL.conf' or the LaTeX macros of the software environment have
changed. But it should not affect the analysis phase of your project.

Until this commit, it was not possible to run a pre-built Maneage'd project
(in a container) on a newly cloned Maneage'd project source. This was
because the containers should be read-only, but during the various checks
of the configuration (to verify that we are using the same software
environment in the container and the source), we were writing/testing many
things in the build directory, and 'LOCAL.conf' which was actually in the
source directory!

Furthermore, the '.local' and '.build' were built at configure time, making
it hard to run the same container from a newly cloned Maneage'd project. To
make things harder for the scenario above, the 'configure.sh' script would
pause on every message and didn't have a quiet mode (making it practically
impossible to run './project configure' before './project make' on every
container run).

With this commit, all these issues have been addressed and it is now
possible to simply get a built container, clone a Maneage'd project and run
the analysis (using the built environment of the container that is verified
on every run). The respective changes/additions are described below:

 - The high-level container scripts ('apptainer.sh' and 'docker.sh', along
   with their READMEs) have been moved to the 'reproduce/software/shell'
   directory and the old 'reproduce/software/containers' directory has been
   deleted. This is because we have classified the software files by their
   language/format and the container scripts are scripts in the end.

 - The './project' script:

    - Now has two extra options: '--quiet' and '--no-pause'. Both are
      directly passed to the 'configure.sh' script. They will respectively
      disable any informative printed message or any pause after that
      message (if it is printed).

    - The '--build-dir' option is now also relevant for './project make':
      when it is given, it will re-create the two '.build' and '.local'
      symbolic links at the top source directory in all scenarios
      ('configure', 'make' or 'shell'). This will allow both the
      configuration, analysis and shell phases to safely assume they exist
      and match the user's desire at run-time.

    - The build/analysis directory's sub-directories that need to be built
      before 'top-make.mk' are now built in a separate function to help in
      readability.

 - The 'configure.sh' script:

    - For developers: a new 'check_elapsed' variable has been defined that
      will enable the newly added 'elapsed_time_from_prev_step'
      function. This function should be used from now on at the end of
      every major step to help find bottlenecks.

 - The targets of the software in 'pre-make-build.sh' now also have the
   version of the software in their file name. Until now, they didn't have
   the version, so there was no way to detect if the software has been
   updated or not in the source. For Lzip and Make (that also get built
   after GCC), the ones in this script have a '-pre-make' suffix also.

 - 'Local.conf.in' now has descriptions for every variable.

 - The '-std=gnu17' option is now used instead of '-std=c17' for basic
   software that cannot be built without specifying the C standard in GCC
   15.1 (described in previous commit: 2881fc0a6205). See [1] for more
   details; in summary: '-std=gnu17' is also supported on macOS's Clang and
   has some features that 'pkg-config' needs

 - Generally: some longer code lines have been broken or indentation
   decreased to fit the 75 character line length. This has not reduced
   readability however. For example the long 'echo' commands are now
   replaced by multiple 'printf's, or the indentation is still clearly
   visible.

The seeds of the work on this commit started by a branch containing three
commits by Giacomo Lorenzetti (133 insertions, 100 deletions). Upon merging
with the main 'maneage' branch, they were generalized and re-organized to
become this commit.

The following issues have also been addressed with this commit:

 - The LaTeX calls (during the building of 'paper.pdf') do not contain
   Maneage'd dynamic libraries. This is because we don't build the LaTeX
   binaries from source, an TeXLive manager uses the host environment.

 - The 'docker.sh' script:

    - Adds the '--project-name' option: its internal variable existed, but
      the option for the user to define it at run-time was not.

    - Ported to macOS: it does not check being a member of the 'docker'
      group, and finds the number of threads using macOS-specific tools.

 - The 'apptainer.sh' script:

    - Now installs 'wget' in the base container also (necessary when the
      user doesn't have the tarballs).

[1] https://savannah.nongnu.org/bugs/?67068#comment2
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Summary: after merging this commit into your project, it should be
re-configured since the location of software installation files like
'LOCAL.conf' or the LaTeX macros of the software environment have
changed. But it should not affect the analysis phase of your project.

Until this commit, it was not possible to run a pre-built Maneage'd project
(in a container) on a newly cloned Maneage'd project source. This was
because the containers should be read-only, but during the various checks
of the configuration (to verify that we are using the same software
environment in the container and the source), we were writing/testing many
things in the build directory, and 'LOCAL.conf' which was actually in the
source directory!

Furthermore, the '.local' and '.build' were built at configure time, making
it hard to run the same container from a newly cloned Maneage'd project. To
make things harder for the scenario above, the 'configure.sh' script would
pause on every message and didn't have a quiet mode (making it practically
impossible to run './project configure' before './project make' on every
container run).

With this commit, all these issues have been addressed and it is now
possible to simply get a built container, clone a Maneage'd project and run
the analysis (using the built environment of the container that is verified
on every run). The respective changes/additions are described below:

 - The high-level container scripts ('apptainer.sh' and 'docker.sh', along
   with their READMEs) have been moved to the 'reproduce/software/shell'
   directory and the old 'reproduce/software/containers' directory has been
   deleted. This is because we have classified the software files by their
   language/format and the container scripts are scripts in the end.

 - The './project' script:

    - Now has two extra options: '--quiet' and '--no-pause'. Both are
      directly passed to the 'configure.sh' script. They will respectively
      disable any informative printed message or any pause after that
      message (if it is printed).

    - The '--build-dir' option is now also relevant for './project make':
      when it is given, it will re-create the two '.build' and '.local'
      symbolic links at the top source directory in all scenarios
      ('configure', 'make' or 'shell'). This will allow both the
      configuration, analysis and shell phases to safely assume they exist
      and match the user's desire at run-time.

    - The build/analysis directory's sub-directories that need to be built
      before 'top-make.mk' are now built in a separate function to help in
      readability.

 - The 'configure.sh' script:

    - For developers: a new 'check_elapsed' variable has been defined that
      will enable the newly added 'elapsed_time_from_prev_step'
      function. This function should be used from now on at the end of
      every major step to help find bottlenecks.

 - The targets of the software in 'pre-make-build.sh' now also have the
   version of the software in their file name. Until now, they didn't have
   the version, so there was no way to detect if the software has been
   updated or not in the source. For Lzip and Make (that also get built
   after GCC), the ones in this script have a '-pre-make' suffix also.

 - 'Local.conf.in' now has descriptions for every variable.

 - The '-std=gnu17' option is now used instead of '-std=c17' for basic
   software that cannot be built without specifying the C standard in GCC
   15.1 (described in previous commit: 2881fc0a6205). See [1] for more
   details; in summary: '-std=gnu17' is also supported on macOS's Clang and
   has some features that 'pkg-config' needs

 - Generally: some longer code lines have been broken or indentation
   decreased to fit the 75 character line length. This has not reduced
   readability however. For example the long 'echo' commands are now
   replaced by multiple 'printf's, or the indentation is still clearly
   visible.

The seeds of the work on this commit started by a branch containing three
commits by Giacomo Lorenzetti (133 insertions, 100 deletions). Upon merging
with the main 'maneage' branch, they were generalized and re-organized to
become this commit.

The following issues have also been addressed with this commit:

 - The LaTeX calls (during the building of 'paper.pdf') do not contain
   Maneage'd dynamic libraries. This is because we don't build the LaTeX
   binaries from source, an TeXLive manager uses the host environment.

 - The 'docker.sh' script:

    - Adds the '--project-name' option: its internal variable existed, but
      the option for the user to define it at run-time was not.

    - Ported to macOS: it does not check being a member of the 'docker'
      group, and finds the number of threads using macOS-specific tools.

 - The 'apptainer.sh' script:

    - Now installs 'wget' in the base container also (necessary when the
      user doesn't have the tarballs).

[1] https://savannah.nongnu.org/bugs/?67068#comment2
</pre>
</div>
</content>
</entry>
<entry>
<title>IMPORTANT: Apptainer and Docker containers, minor restructuring</title>
<updated>2025-04-23T13:38:19+00:00</updated>
<author>
<name>Giacomo Lorenzetti</name>
<email>glorenzetti@cefca.es</email>
</author>
<published>2025-02-27T16:49:28+00:00</published>
<link rel='alternate' type='text/html' href='http://git.maneage.org/project.git/commit/?id=a1f8947ab7784af4b7e66c617ce19a8bdd9c99ed'/>
<id>a1f8947ab7784af4b7e66c617ce19a8bdd9c99ed</id>
<content type='text'>
Summary: it is necessary to re-configure your project (just running
'./project configure -e', not deleting 'build/software' to re-build
software) after this commit, see "Affected files" item below).

Until now, we only had a relatively long set of manual instructions for
building Maneage within Docker in the top-level README. This was hard to
automate, focing Maneage users to write custom commands based on the
instructions and maintain those scripts outside of Maneage. As a result,
experience could not be shared between projects (or at most in the README
file!).

With this commit, a new 'reproduce/software/containers' directory has been
created within Maneage that contains two scripts (with a unified interface)
greatly simplifying the building of the project's software environment
within a container (one script for Apptainer and one for Docker). Two
READMEs have been added for each container to help in their first time
usage. Also, the old checklist within the main README has been replaced
with a short introduction on containers and points the interested readers
to the custom README of each container technology.

Since we wanted the containers to be read-only after build, we needed to
fully decouple the 'build/software' and 'build/analysis', such that
'./project configure' only writes to the former and './project make' only
writes the latter. The file and directories mentioned in the affected files
are cases that both project phases was writing to the 'build/software' and
'build/analysis' directories.

Affected files: 'preparation-done.mk' and 'lockdir' which were previously
in the 'build/software' directory are now made during the 'make' phase and
the 'configure' phase no longer builds the 'build/analysis' or anything
within it. Also, the software version LaTeX macros (which were previously
written during the 'configure' phase in the 'analysis' directory) are now
written in the software directory and copied into the analysis for usage in
LaTeX while building the paper.

Other minor additions in this commit:

  - The './project' script has a new '--timing' option to write the
    starting and ending times of the project in a file. It also builds the
    high-level analysis directories when './project make' is called (but
    before calling 'top-make.mk'.

  - The 'tar' calls in the custom build commands of the software building
    Makefiles now have the '--no-same-owner --no-same-permissions' options
    like the 'tar' call within the 'uncompress' function of
    'build-rules.mk'.

This commit was originally written by Giacomo Lorenzetti only for Apptainer
on the registered commit date. It was later re-implemented from scratch by
Mohammad Akhlaghi to have a unified interface for both Apptainer and Docker
and merged into Maneage on 2025-04-23.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Summary: it is necessary to re-configure your project (just running
'./project configure -e', not deleting 'build/software' to re-build
software) after this commit, see "Affected files" item below).

Until now, we only had a relatively long set of manual instructions for
building Maneage within Docker in the top-level README. This was hard to
automate, focing Maneage users to write custom commands based on the
instructions and maintain those scripts outside of Maneage. As a result,
experience could not be shared between projects (or at most in the README
file!).

With this commit, a new 'reproduce/software/containers' directory has been
created within Maneage that contains two scripts (with a unified interface)
greatly simplifying the building of the project's software environment
within a container (one script for Apptainer and one for Docker). Two
READMEs have been added for each container to help in their first time
usage. Also, the old checklist within the main README has been replaced
with a short introduction on containers and points the interested readers
to the custom README of each container technology.

Since we wanted the containers to be read-only after build, we needed to
fully decouple the 'build/software' and 'build/analysis', such that
'./project configure' only writes to the former and './project make' only
writes the latter. The file and directories mentioned in the affected files
are cases that both project phases was writing to the 'build/software' and
'build/analysis' directories.

Affected files: 'preparation-done.mk' and 'lockdir' which were previously
in the 'build/software' directory are now made during the 'make' phase and
the 'configure' phase no longer builds the 'build/analysis' or anything
within it. Also, the software version LaTeX macros (which were previously
written during the 'configure' phase in the 'analysis' directory) are now
written in the software directory and copied into the analysis for usage in
LaTeX while building the paper.

Other minor additions in this commit:

  - The './project' script has a new '--timing' option to write the
    starting and ending times of the project in a file. It also builds the
    high-level analysis directories when './project make' is called (but
    before calling 'top-make.mk'.

  - The 'tar' calls in the custom build commands of the software building
    Makefiles now have the '--no-same-owner --no-same-permissions' options
    like the 'tar' call within the 'uncompress' function of
    'build-rules.mk'.

This commit was originally written by Giacomo Lorenzetti only for Apptainer
on the registered commit date. It was later re-implemented from scratch by
Mohammad Akhlaghi to have a unified interface for both Apptainer and Docker
and merged into Maneage on 2025-04-23.
</pre>
</div>
</content>
</entry>
<entry>
<title>IMPORTANT: 73 software upgrades and added offline mode</title>
<updated>2025-02-11T19:47:53+00:00</updated>
<author>
<name>Giacomo Lorenzetti</name>
<email>glorenzetti@cefca.es</email>
</author>
<published>2025-01-21T17:57:10+00:00</published>
<link rel='alternate' type='text/html' href='http://git.maneage.org/project.git/commit/?id=890858795fd6e0c8c1f3050adec7f4bc78e9e47c'/>
<id>890858795fd6e0c8c1f3050adec7f4bc78e9e47c</id>
<content type='text'>
Summary: This is a software update to make Maneage more portable and up to
date. It does not involve any Maneage infrastructure changes. You should
just re-build your project to make sure the updated software haven't
removed/changed any of their features that you were using.

Until now, the software in Maneage had not been updated for almost 2.5
years (since August 2022) and were starting to get dated! Also, when no
internet is available (for example in a HPC), the Zenodo check would always
cause a crash and needed to be commented manually.

With this commit, all the basic software that had been updated as well as
the dependencies of Gnuastro (which we commit to maintaining) have been
updated after a check of their webpage (see the full list below). To fix
the offline issue, a new '--offline' option has been added to the project
script. The other following changes were made:

 - tarball-prepare: has become more resilient (for dealing with file name
   with spaces or Lzip, which should not be compressed).

 - Readme: improve documentation when used in Docker.

Below you can see the list of basic software that have been updated. The
only basic software that have not been updated upstream since the previous
Maneage software update are 'bison', 'flock', 'm4', 'pkgconfig', 'unzip'
and 'zip'.

    bash                5.2-rc2            5.2.37
    binutils            2.39               2.43.1
    coreutils           9.1                9.6
    curl                7.84.0             8.11.1
    dash                0.5.11-057cd65     0.5.12
    diffutils           3.8                3.10
    file                5.42               5.46
    findutils           4.9.0              4.10.0
    gawk                5.1.1              5.3.1
    gcc                 12.1.0             14.2.0
    gettext             0.21               0.23.1
    git                 2.37.1             2.48.1
    gmp                 6.2.1              6.3.0
    grep                3.7                3.11
    gzip                1.12               1.13
    help2man            1.49.2             1.49.3
    isl                 0.24               0.27
    less                590                668
    libiconv            1.17               1.18
    libtool             2.4.7              2.5.4
    libunistring        1.0                1.3
    libxml2             2.9.12             2.13.5
    lzip                1.23               1.25
    make                4.3                4.4.1
    mpc                 1.2.1              1.3.1
    mpfr                4.1.0              4.2.1
    nano                6.4                8.3
    ncurses             6.3                6.5
    openssl             3.0.5              3.4.0
    perl                5.36.0             5.40.1
    podlators           4.14               6.0.2
    readline            8.2-rc2            8.2.13
    sed                 4.8                4.9
    tar                 1.34               1.35
    texinfo             6.8                7.2
    wget                1.21.2             1.25.0
    which               2.21               2.23
    xz                  5.2.5              5.6.3
    zlib                1.2.11             1.3.1

The high-level software that have been updated.

    autoconf            2.71               2.72
    automake            1.16.5             1.17
    cfitsio             4.1.0              4.5.0
    cmake               3.24.0             3.31.5
    expat               2.4.1              2.6.4
    fontconfig          2.14.0             2.16.0
    freetype            2.11.0             2.13.3
    ghostscript         9.56.1             10.04.0
    gnuastro            0.18               0.23
    gsl                 2.7                2.8
    libffi              3.4.2              3.4.7
    libgit2             1.3.0              1.9.0
    libice              1.0.10             1.1.2
    libidn              1.38               1.42
    libjpeg             9e                 9f
    libpaper            1.1.28             1.1.29
    libpng              1.6.37             1.6.46
    libpthread-stubs    0.4                0.5
    libsm               1.2.3              1.2.5
    libtiff             4.4.0              4.7.0
    libxau              1.0.9              1.0.12
    libxcb              1.15               1.17
    libxdmcp            1.1.3              1.1.5
    libxext             1.3.4              1.3.6
    libxt               1.2.1              1.3.1
    python              3.10.6             3.13.12
    util-macros         1.19.3             1.20.2
    wcslib              7.11               8.4
    xcb-proto           1.15               1.17
    xorgproto           2022.1             2024.1
    xtrans              1.4.0              1.5.2
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Summary: This is a software update to make Maneage more portable and up to
date. It does not involve any Maneage infrastructure changes. You should
just re-build your project to make sure the updated software haven't
removed/changed any of their features that you were using.

Until now, the software in Maneage had not been updated for almost 2.5
years (since August 2022) and were starting to get dated! Also, when no
internet is available (for example in a HPC), the Zenodo check would always
cause a crash and needed to be commented manually.

With this commit, all the basic software that had been updated as well as
the dependencies of Gnuastro (which we commit to maintaining) have been
updated after a check of their webpage (see the full list below). To fix
the offline issue, a new '--offline' option has been added to the project
script. The other following changes were made:

 - tarball-prepare: has become more resilient (for dealing with file name
   with spaces or Lzip, which should not be compressed).

 - Readme: improve documentation when used in Docker.

Below you can see the list of basic software that have been updated. The
only basic software that have not been updated upstream since the previous
Maneage software update are 'bison', 'flock', 'm4', 'pkgconfig', 'unzip'
and 'zip'.

    bash                5.2-rc2            5.2.37
    binutils            2.39               2.43.1
    coreutils           9.1                9.6
    curl                7.84.0             8.11.1
    dash                0.5.11-057cd65     0.5.12
    diffutils           3.8                3.10
    file                5.42               5.46
    findutils           4.9.0              4.10.0
    gawk                5.1.1              5.3.1
    gcc                 12.1.0             14.2.0
    gettext             0.21               0.23.1
    git                 2.37.1             2.48.1
    gmp                 6.2.1              6.3.0
    grep                3.7                3.11
    gzip                1.12               1.13
    help2man            1.49.2             1.49.3
    isl                 0.24               0.27
    less                590                668
    libiconv            1.17               1.18
    libtool             2.4.7              2.5.4
    libunistring        1.0                1.3
    libxml2             2.9.12             2.13.5
    lzip                1.23               1.25
    make                4.3                4.4.1
    mpc                 1.2.1              1.3.1
    mpfr                4.1.0              4.2.1
    nano                6.4                8.3
    ncurses             6.3                6.5
    openssl             3.0.5              3.4.0
    perl                5.36.0             5.40.1
    podlators           4.14               6.0.2
    readline            8.2-rc2            8.2.13
    sed                 4.8                4.9
    tar                 1.34               1.35
    texinfo             6.8                7.2
    wget                1.21.2             1.25.0
    which               2.21               2.23
    xz                  5.2.5              5.6.3
    zlib                1.2.11             1.3.1

The high-level software that have been updated.

    autoconf            2.71               2.72
    automake            1.16.5             1.17
    cfitsio             4.1.0              4.5.0
    cmake               3.24.0             3.31.5
    expat               2.4.1              2.6.4
    fontconfig          2.14.0             2.16.0
    freetype            2.11.0             2.13.3
    ghostscript         9.56.1             10.04.0
    gnuastro            0.18               0.23
    gsl                 2.7                2.8
    libffi              3.4.2              3.4.7
    libgit2             1.3.0              1.9.0
    libice              1.0.10             1.1.2
    libidn              1.38               1.42
    libjpeg             9e                 9f
    libpaper            1.1.28             1.1.29
    libpng              1.6.37             1.6.46
    libpthread-stubs    0.4                0.5
    libsm               1.2.3              1.2.5
    libtiff             4.4.0              4.7.0
    libxau              1.0.9              1.0.12
    libxcb              1.15               1.17
    libxdmcp            1.1.3              1.1.5
    libxext             1.3.4              1.3.6
    libxt               1.2.1              1.3.1
    python              3.10.6             3.13.12
    util-macros         1.19.3             1.20.2
    wcslib              7.11               8.4
    xcb-proto           1.15               1.17
    xorgproto           2022.1             2024.1
    xtrans              1.4.0              1.5.2
</pre>
</div>
</content>
</entry>
<entry>
<title>Configuration: no dependency on /bin/sh and useful run-time options</title>
<updated>2025-02-10T18:34:26+00:00</updated>
<author>
<name>Mohammad Akhlaghi</name>
<email>mohammad@akhlaghi.org</email>
</author>
<published>2024-05-03T11:07:41+00:00</published>
<link rel='alternate' type='text/html' href='http://git.maneage.org/project.git/commit/?id=3c9bf3aff30f02c7d31bd86f36c4db2520f8ffa4'/>
<id>3c9bf3aff30f02c7d31bd86f36c4db2520f8ffa4</id>
<content type='text'>
SUMMARY: no change necessary in your project, this commit only involves
changes in how already-existing software are built. Some handy options have
also been added to the top-level project script and the copyright years
have been updated.

Until now, if the host's '/bin/sh' had conflicts with the Maneage
environment, the configuration of Maneage would crash as soon as we entered
the building of high-level software. The full scenario is described in the
comments of the newly added 'reproduce/software/shell/prep-source.sh'. This
is most relevant when building older Maneage'd project in newer
environments.

With this commit, the following changes were made to avoid the problem
above:

  - Maneage edits the source code of all installed software to replace
    '/bin/sh' with Maneage's own shell before the programs are
    built. Through this, we were able to solve the problem described
    above.

  - The portable '#!/usr/bin/env sh' shebangs are now used at the start of
    the scripts that run during configure time so it uses the first
    available shell that it finds in its PATH (the system's before Dash is
    built), then Dash, and after Dash is built, Bash.

  - For TeXLive, since we don't install it from source, it was necessary to
    add the libraries necessary for the local '/bin/sh' in LD_LIBRARY_PATH.

Some high-level options have been added to the './project' script to
simplify certain operations:

  --keep-going: do not stop upon the first crash, but keep going on to
    build targets until all build-able targets have been built. This is
    very useful for debugging large pipelines and allows you to isolate the
    problematic part of your project.

  --highlight-all: equivalent to calling both '--highlight-new' &amp;
    '--highlight-notes'.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
SUMMARY: no change necessary in your project, this commit only involves
changes in how already-existing software are built. Some handy options have
also been added to the top-level project script and the copyright years
have been updated.

Until now, if the host's '/bin/sh' had conflicts with the Maneage
environment, the configuration of Maneage would crash as soon as we entered
the building of high-level software. The full scenario is described in the
comments of the newly added 'reproduce/software/shell/prep-source.sh'. This
is most relevant when building older Maneage'd project in newer
environments.

With this commit, the following changes were made to avoid the problem
above:

  - Maneage edits the source code of all installed software to replace
    '/bin/sh' with Maneage's own shell before the programs are
    built. Through this, we were able to solve the problem described
    above.

  - The portable '#!/usr/bin/env sh' shebangs are now used at the start of
    the scripts that run during configure time so it uses the first
    available shell that it finds in its PATH (the system's before Dash is
    built), then Dash, and after Dash is built, Bash.

  - For TeXLive, since we don't install it from source, it was necessary to
    add the libraries necessary for the local '/bin/sh' in LD_LIBRARY_PATH.

Some high-level options have been added to the './project' script to
simplify certain operations:

  --keep-going: do not stop upon the first crash, but keep going on to
    build targets until all build-able targets have been built. This is
    very useful for debugging large pipelines and allows you to isolate the
    problematic part of your project.

  --highlight-all: equivalent to calling both '--highlight-new' &amp;
    '--highlight-notes'.
</pre>
</div>
</content>
</entry>
<entry>
<title>Copyright years: updated to 2023, accompanied by some minor fixes</title>
<updated>2023-05-07T09:48:14+00:00</updated>
<author>
<name>Mohammad Akhlaghi</name>
<email>mohammad@akhlaghi.org</email>
</author>
<published>2023-05-07T09:31:06+00:00</published>
<link rel='alternate' type='text/html' href='http://git.maneage.org/project.git/commit/?id=644a236b70fd64df6f1526c7e3ca3d148ad08044'/>
<id>644a236b70fd64df6f1526c7e3ca3d148ad08044</id>
<content type='text'>
SUMMARY: just house-cleaning, no need to do anything major in your
branch. Just update the copyright years in files that you have added.

Until now, the latest copyright years of the whole Maneage source code was
2022! As of this commit, we have already moved to 2023 for 5 months!
Furthermore, there were a few other minor issues that needed correction:

 - The URL to download input datasets wasn't quoted in 'initialize.mk' or
   the download script! As a result, when the input URL had characters that
   are meaningful to the shell (like '&amp;'), the download command would not
   work.

 - The only program that had 'make check' in the 'basic.mk' programs was
   MPFR. At that stage, we still haven't built our own compiler at this
   stage, this is not accurate.

 - The 'pyerfa' and 'extension-helpers' packages in Python need
   'setuptools_scm' on some systems. But until now, it was not in the list
   of their prerequisites.

With this commit, all the issues above have been corrected.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
SUMMARY: just house-cleaning, no need to do anything major in your
branch. Just update the copyright years in files that you have added.

Until now, the latest copyright years of the whole Maneage source code was
2022! As of this commit, we have already moved to 2023 for 5 months!
Furthermore, there were a few other minor issues that needed correction:

 - The URL to download input datasets wasn't quoted in 'initialize.mk' or
   the download script! As a result, when the input URL had characters that
   are meaningful to the shell (like '&amp;'), the download command would not
   work.

 - The only program that had 'make check' in the 'basic.mk' programs was
   MPFR. At that stage, we still haven't built our own compiler at this
   stage, this is not accurate.

 - The 'pyerfa' and 'extension-helpers' packages in Python need
   'setuptools_scm' on some systems. But until now, it was not in the list
   of their prerequisites.

With this commit, all the issues above have been corrected.
</pre>
</div>
</content>
</entry>
<entry>
<title>IMPORTANT: Software update</title>
<updated>2022-08-13T20:36:13+00:00</updated>
<author>
<name>Mohammad Akhlaghi</name>
<email>mohammad@akhlaghi.org</email>
</author>
<published>2022-08-05T18:27:56+00:00</published>
<link rel='alternate' type='text/html' href='http://git.maneage.org/project.git/commit/?id=79ee52ed1e136fbd2592a5c2a058f73024d8b5b0'/>
<id>79ee52ed1e136fbd2592a5c2a058f73024d8b5b0</id>
<content type='text'>
SUMMARY: This is a software update to make Maneage more portable and up to
date. It does not involve any Maneage infrastructure changes. You should
just re-build your project to make sure the updated software haven't
removed/changed any of their features that you were using. In particular,
for Astrometry.net users, please see the respective note in P.S.2 below.

Until now, there have been many updates in the software that are built
within Maneage. The last software upadte was almost one year ago.

With this commit, the software in the P.S.1 have been updated. A
description of notable changes in the software environment is given in
P.S.2. This software environment has been tested on an Arch GNU/Linux,
Debian, CentOS-7 and macOS.

This commit is the merging of 24 individual commits by Raul Infante-Sainz
(who put a lot of energy on porting the software below for macOS, and
updating citations), Boudewijn Roukema (who helped with memory checking for
GCC, and testing on Debian and CentOS), Sepideh Eskandarlou (who tested the
environment) and myself.

Besides the updates in the core software, the followimg improvements have
also been implemented in this commit:

 - When you run './project shell':

   - A welcome message is printed that will remind the caller that they
     have entered a new environment, it will print the location of 'HOME'
     and the location of the shell startup file.

   - The 'reproduce/software/shell/bashrc.sh' is loaded as a startup
     file. This allows you to customize your interactive Maneage shell. A
     default step has already been placed there that will put the git
     branch name (in green) within the shell prompt (which was
     purple). This greatly helps when dealing with directories under Git
     version control. These settings won't bother with Maneage's default
     operations: through environment variables we make sure that these
     './project shell' features will not slow-down the calls to the shell
     within the non-interactive Make calls.

   - The host's 'COLORTERM' is passed to the Maneage environment. It is
     used by some programs that can have color outputs on the terminal.

 - Updates to citations:
   - Numpy and Scipy (as requested on their pages):
     https://numpy.org/citing-numpy and https://scipy.org/citing-scipy
   - Gnuastro: Added https://arxiv.org/abs/1909.11230 which describes major
     updates to Gnuastro after 10 releases.
   - When a software's paper is indexed in the SAO/NASA Astrophysics Data
     System (ADS), Maneage now use the BibTeX entries provided by ADS. This
     helps to give a unified format to most software, and more information
     (like ADS+arXiv hyperlinks in the BibLaTeX compilation of the default
     bibliography).

 - We were able to build this version of Maneage on a Debian system from
   2010 (+12 years ago!). Only three downgrades were necessary in the
   "basic" software (not affecting the high-level science software!). A
   description of the necessary downgrades for such old systems has been
   added in 'README.md'.

P.S.1 List of updated software:

 Basic software:
   cURL                                7.79.1    --&gt;  7.84.0
   Dash                                0.5.11.5  --&gt;  0.5.11-057cd65
   File                                5.41      --&gt;  5.42
   GNU AWK                             5.1.0     --&gt;  5.1.1
   GNU Bash                            5.1.8     --&gt;  5.2-rc2
   GNU Binutils                        2.37      --&gt;  2.39
   GNU Compiler Collection (GCC)       11.2.1    --&gt;  12.1.0
   GNU Findutils                       4.8.0     --&gt;  4.9.0
   GNU Gzip                            1.11      --&gt;  1.12
   GNU Help2man                        1.48.5    --&gt;  1.49.2
   GNU Integer Set Library (ISL)       0.18      --&gt;  0.24
   GNU Libtool                         2.4.6     --&gt;  2.4.7
   GNU Nano                            6.0       --&gt;  6.4
   GNU Readline                        8.1.1     --&gt;  8.2-rc2
   GNU libiconv                        0.16      --&gt;  0.17
   Git                                 2.36.0    --&gt;  2.37.1
   OpenSSL                             3.0.0     --&gt;  3.0.5
   PatchELF                            0.13      --&gt;  0.15.0
   Perl                                5.34.0    --&gt;  5.36.0

 High-level software:
   Astrometry.net                      0.89      --&gt;  0.91
   CFITSIO                             4.0.0     --&gt;  4.1.0
   CMake                               3.21.4    --&gt;  3.24.0
   GNU Astronomy Utilities (Gnuastro)  0.16.1    --&gt;  0.18
   GPL Ghostscript                     9.55.0    --&gt;  9.56.1
   HDF5                                1.10.5    --&gt;  1.13.1
   Libjpeg                             9d        --&gt;  9e
   Libtiff                             4.3.0     --&gt;  4.4.0
   OpenBLAS                            0.3.18    --&gt;  0.3.21
   PLplot                              n/a       --&gt;  5.15.0
   Python                              3.10.0    --&gt;  3.10.6
   SCAMP                               2.6.7     --&gt;  2.10.0
   SWarp                               2.38.0    --&gt;  2.41.5
   Util-Linux                          2.37.2    --&gt;  2.38.1
   Vim                                 8.2       --&gt;  9.0
   WCSLIB                              7.7       --&gt;  7.11

 X.org packages (used by graphical software like Ghostscript and LaTeX):
   Fontconfig                          2.13.94   --&gt;  2.14.0
   LibX11                              1.7.2     --&gt;  1.8
   LibXCB                              1.14      --&gt;  1.15
   XCB-proto                           1.14.1    --&gt;  1.15
   Xorg-proto                          2021.5    --&gt;  2022.1

 Python modules:
   Astropy                             5.0       --&gt;  5.1
   GalSim                              2.3.3     --&gt;  2.3.5

P.S.2: Notable points regarding the software environment:

 - Two new links from the host's low-level tools are now included in
   Maneage's build environment:
   - On GNU/Linux systems, the host's 'ldd' is linked inside the custom
     environment. This belongs to the GNU C Library (which is not yet
     installed in Maneage). But helps in checking the linking status of the
     binaries on GNU/Linux systems.
   - On macOS: the 'codesign' binary is included, which is used by GNU
     Emacs on macOS to sign the built executable.

 - GNU Bison has been moved in basic software (necessary for GNU Binutils).

 - The Zip and Unzip programs have been moved as high-level software that
   have to be manually requested when necessary. This is because they are
   not used by any of the basic software anymore. They were just installed
   as dependencies of GNU Tar to be close the other compression
   programs. Also, in the past we would use the original tarballs, and some
   (for example Numpy) were distributed in Zip format. However, by default,
   we now use a custom Lzip tarball and don't need Zip or Unzip. This was
   suggested by Zahra Sharbaf and Raul Infante-Sainz.

 - Some minor edits in 'reproduce/software/shell/tarball-prepare.sh'. In
   particular the 'awk' command was effectively just replacing a '_' with
   '-', so it just uses a simple SED expression instead.

 - Fixed bug 62700 (https://savannah.nongnu.org/bugs/index.php?62700) by
   compiling 'xz' with a patched version of the xz source file
   'src/liblzma/liblzma.map'.

 - Astrometry.net doesn't depend on NetPBM any more. NetPBM (and its
   dependencies) were causing many crashes on macOS and it also a very
   strange build system that is hard to maintain. Astrometry.net uses it to
   take images as input. However, it isn't necessary when you provide
   Astrometry.net with a catalog. Therefore, Raul added some instructions
   on how to run astrometry from your own custom X-Y catalog. These
   instruction can be seen on top of the build rule of Astrometry.net in
   'reproduce/software/make/high-level.mk'.

 - h5py has been removed as a dependency of Astropy. It is an optional
   dependency to write tables into HDF5 format. But since we couldn't get
   it to build on macOS it has been removed. None of the current Maneage
   users/developers also use this feature of Astropy!

 - PLplot is added a new software, but not a default pre-requisite of SCAMP
   (which can use it to generate figures), because there were many build
   problems on macOS. Instructions have been added on top of SCAMP on how
   to add PLplot as a dependency.

 - With the aim of being able to install Plplot on macOS, we have wrote
   several lines to fix header problems. However, we didn't succeed. In any
   case we are leaving these lines in case they are useful in the future.

 - The '-Wno-nullability-completeness' compiler flag (which is primarily
   necessary for macOS) is now only added for macOS systems. It was causing
   many warnings of un-recognized option in GNU/Linux systems.

 - The 'mkswap' program of Util-Linux has been disabled because it caused
   crashes on older kernels. Generally, its not necessary for a Maneage
   project because it needs root permissions to run!

 - LibXT (of the x.org software) has been added as a dependency of Cairo.

 - ImageMagick and Lzip were using the host's C++ standard library! But on
   GNU/Linux we build our own C++ Standard Library with GCC, so with this
   commit, they properly link with Maneage's C++ standard library.

 - ImageMagick on macOS couldn't properly link with Maneage's Ghostscript
   library! This has been fixed using macOS's install_name_tool.

 - Necessary RAM to build GCC on GNU/Linux systems changed to ~8GB, see
   https://savannah.nongnu.org/task/?16244#comment12

 - Pythran is no longer as prerequisite of Scipy. Until now, Pythran was a
   prerequisite of Scipy. But we noticed that it is optional and was
   causing problems on macOS.

 - The URLs of some of the software have been updated in
   'reproduce/software/config/urls.conf'. By default, these are all
   commented, but they can be useful when searching for new versions or
   when a project needs custom software that is not (yet) in Maneage.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
SUMMARY: This is a software update to make Maneage more portable and up to
date. It does not involve any Maneage infrastructure changes. You should
just re-build your project to make sure the updated software haven't
removed/changed any of their features that you were using. In particular,
for Astrometry.net users, please see the respective note in P.S.2 below.

Until now, there have been many updates in the software that are built
within Maneage. The last software upadte was almost one year ago.

With this commit, the software in the P.S.1 have been updated. A
description of notable changes in the software environment is given in
P.S.2. This software environment has been tested on an Arch GNU/Linux,
Debian, CentOS-7 and macOS.

This commit is the merging of 24 individual commits by Raul Infante-Sainz
(who put a lot of energy on porting the software below for macOS, and
updating citations), Boudewijn Roukema (who helped with memory checking for
GCC, and testing on Debian and CentOS), Sepideh Eskandarlou (who tested the
environment) and myself.

Besides the updates in the core software, the followimg improvements have
also been implemented in this commit:

 - When you run './project shell':

   - A welcome message is printed that will remind the caller that they
     have entered a new environment, it will print the location of 'HOME'
     and the location of the shell startup file.

   - The 'reproduce/software/shell/bashrc.sh' is loaded as a startup
     file. This allows you to customize your interactive Maneage shell. A
     default step has already been placed there that will put the git
     branch name (in green) within the shell prompt (which was
     purple). This greatly helps when dealing with directories under Git
     version control. These settings won't bother with Maneage's default
     operations: through environment variables we make sure that these
     './project shell' features will not slow-down the calls to the shell
     within the non-interactive Make calls.

   - The host's 'COLORTERM' is passed to the Maneage environment. It is
     used by some programs that can have color outputs on the terminal.

 - Updates to citations:
   - Numpy and Scipy (as requested on their pages):
     https://numpy.org/citing-numpy and https://scipy.org/citing-scipy
   - Gnuastro: Added https://arxiv.org/abs/1909.11230 which describes major
     updates to Gnuastro after 10 releases.
   - When a software's paper is indexed in the SAO/NASA Astrophysics Data
     System (ADS), Maneage now use the BibTeX entries provided by ADS. This
     helps to give a unified format to most software, and more information
     (like ADS+arXiv hyperlinks in the BibLaTeX compilation of the default
     bibliography).

 - We were able to build this version of Maneage on a Debian system from
   2010 (+12 years ago!). Only three downgrades were necessary in the
   "basic" software (not affecting the high-level science software!). A
   description of the necessary downgrades for such old systems has been
   added in 'README.md'.

P.S.1 List of updated software:

 Basic software:
   cURL                                7.79.1    --&gt;  7.84.0
   Dash                                0.5.11.5  --&gt;  0.5.11-057cd65
   File                                5.41      --&gt;  5.42
   GNU AWK                             5.1.0     --&gt;  5.1.1
   GNU Bash                            5.1.8     --&gt;  5.2-rc2
   GNU Binutils                        2.37      --&gt;  2.39
   GNU Compiler Collection (GCC)       11.2.1    --&gt;  12.1.0
   GNU Findutils                       4.8.0     --&gt;  4.9.0
   GNU Gzip                            1.11      --&gt;  1.12
   GNU Help2man                        1.48.5    --&gt;  1.49.2
   GNU Integer Set Library (ISL)       0.18      --&gt;  0.24
   GNU Libtool                         2.4.6     --&gt;  2.4.7
   GNU Nano                            6.0       --&gt;  6.4
   GNU Readline                        8.1.1     --&gt;  8.2-rc2
   GNU libiconv                        0.16      --&gt;  0.17
   Git                                 2.36.0    --&gt;  2.37.1
   OpenSSL                             3.0.0     --&gt;  3.0.5
   PatchELF                            0.13      --&gt;  0.15.0
   Perl                                5.34.0    --&gt;  5.36.0

 High-level software:
   Astrometry.net                      0.89      --&gt;  0.91
   CFITSIO                             4.0.0     --&gt;  4.1.0
   CMake                               3.21.4    --&gt;  3.24.0
   GNU Astronomy Utilities (Gnuastro)  0.16.1    --&gt;  0.18
   GPL Ghostscript                     9.55.0    --&gt;  9.56.1
   HDF5                                1.10.5    --&gt;  1.13.1
   Libjpeg                             9d        --&gt;  9e
   Libtiff                             4.3.0     --&gt;  4.4.0
   OpenBLAS                            0.3.18    --&gt;  0.3.21
   PLplot                              n/a       --&gt;  5.15.0
   Python                              3.10.0    --&gt;  3.10.6
   SCAMP                               2.6.7     --&gt;  2.10.0
   SWarp                               2.38.0    --&gt;  2.41.5
   Util-Linux                          2.37.2    --&gt;  2.38.1
   Vim                                 8.2       --&gt;  9.0
   WCSLIB                              7.7       --&gt;  7.11

 X.org packages (used by graphical software like Ghostscript and LaTeX):
   Fontconfig                          2.13.94   --&gt;  2.14.0
   LibX11                              1.7.2     --&gt;  1.8
   LibXCB                              1.14      --&gt;  1.15
   XCB-proto                           1.14.1    --&gt;  1.15
   Xorg-proto                          2021.5    --&gt;  2022.1

 Python modules:
   Astropy                             5.0       --&gt;  5.1
   GalSim                              2.3.3     --&gt;  2.3.5

P.S.2: Notable points regarding the software environment:

 - Two new links from the host's low-level tools are now included in
   Maneage's build environment:
   - On GNU/Linux systems, the host's 'ldd' is linked inside the custom
     environment. This belongs to the GNU C Library (which is not yet
     installed in Maneage). But helps in checking the linking status of the
     binaries on GNU/Linux systems.
   - On macOS: the 'codesign' binary is included, which is used by GNU
     Emacs on macOS to sign the built executable.

 - GNU Bison has been moved in basic software (necessary for GNU Binutils).

 - The Zip and Unzip programs have been moved as high-level software that
   have to be manually requested when necessary. This is because they are
   not used by any of the basic software anymore. They were just installed
   as dependencies of GNU Tar to be close the other compression
   programs. Also, in the past we would use the original tarballs, and some
   (for example Numpy) were distributed in Zip format. However, by default,
   we now use a custom Lzip tarball and don't need Zip or Unzip. This was
   suggested by Zahra Sharbaf and Raul Infante-Sainz.

 - Some minor edits in 'reproduce/software/shell/tarball-prepare.sh'. In
   particular the 'awk' command was effectively just replacing a '_' with
   '-', so it just uses a simple SED expression instead.

 - Fixed bug 62700 (https://savannah.nongnu.org/bugs/index.php?62700) by
   compiling 'xz' with a patched version of the xz source file
   'src/liblzma/liblzma.map'.

 - Astrometry.net doesn't depend on NetPBM any more. NetPBM (and its
   dependencies) were causing many crashes on macOS and it also a very
   strange build system that is hard to maintain. Astrometry.net uses it to
   take images as input. However, it isn't necessary when you provide
   Astrometry.net with a catalog. Therefore, Raul added some instructions
   on how to run astrometry from your own custom X-Y catalog. These
   instruction can be seen on top of the build rule of Astrometry.net in
   'reproduce/software/make/high-level.mk'.

 - h5py has been removed as a dependency of Astropy. It is an optional
   dependency to write tables into HDF5 format. But since we couldn't get
   it to build on macOS it has been removed. None of the current Maneage
   users/developers also use this feature of Astropy!

 - PLplot is added a new software, but not a default pre-requisite of SCAMP
   (which can use it to generate figures), because there were many build
   problems on macOS. Instructions have been added on top of SCAMP on how
   to add PLplot as a dependency.

 - With the aim of being able to install Plplot on macOS, we have wrote
   several lines to fix header problems. However, we didn't succeed. In any
   case we are leaving these lines in case they are useful in the future.

 - The '-Wno-nullability-completeness' compiler flag (which is primarily
   necessary for macOS) is now only added for macOS systems. It was causing
   many warnings of un-recognized option in GNU/Linux systems.

 - The 'mkswap' program of Util-Linux has been disabled because it caused
   crashes on older kernels. Generally, its not necessary for a Maneage
   project because it needs root permissions to run!

 - LibXT (of the x.org software) has been added as a dependency of Cairo.

 - ImageMagick and Lzip were using the host's C++ standard library! But on
   GNU/Linux we build our own C++ Standard Library with GCC, so with this
   commit, they properly link with Maneage's C++ standard library.

 - ImageMagick on macOS couldn't properly link with Maneage's Ghostscript
   library! This has been fixed using macOS's install_name_tool.

 - Necessary RAM to build GCC on GNU/Linux systems changed to ~8GB, see
   https://savannah.nongnu.org/task/?16244#comment12

 - Pythran is no longer as prerequisite of Scipy. Until now, Pythran was a
   prerequisite of Scipy. But we noticed that it is optional and was
   causing problems on macOS.

 - The URLs of some of the software have been updated in
   'reproduce/software/config/urls.conf'. By default, these are all
   commented, but they can be useful when searching for new versions or
   when a project needs custom software that is not (yet) in Maneage.
</pre>
</div>
</content>
</entry>
<entry>
<title>Housekeeping: some portability issues fixed; four software updates</title>
<updated>2022-06-10T00:21:14+00:00</updated>
<author>
<name>Mohammad Akhlaghi</name>
<email>mohammad@akhlaghi.org</email>
</author>
<published>2022-06-09T23:43:30+00:00</published>
<link rel='alternate' type='text/html' href='http://git.maneage.org/project.git/commit/?id=ab811d0952ac93ce608c81ab2cc44d67c7b02dbe'/>
<id>ab811d0952ac93ce608c81ab2cc44d67c7b02dbe</id>
<content type='text'>
Until now, there were several portability issues in Maneage:

 1. Maneage would crash on older operating systems (checked on Debian 6),
    where Wget didn't have the '--no-use-server-timestamps'.

 2. On a Linux kernel 2.6.32 (of the same Debian 6 above) some features in
    'util-linux' (like 'swapon' or 'libmount') wouldn't build and wouldn't
    let 'util-linux' complete. These features need root permissions to be
    useful, so the wouldn't be used in Maneage any way! But they wouldn't
    let Maneage get built

 3. The './project shell' command would still read the host's '~/.bashrc',
    letting the host environment leak-in to Maneage's interactive shell.

 4. The building of Flex 2.64 wouldn't complete due to a segmentation
    fault an Ubuntu, but NetPBM (which depends on Flex) would crash with a
    wrong usage of 'yyunput'. This had actually caused a non-update to
    Flex in a previous Maneage software update.

 5. The update Astrometry.net would assume SExtractor's executable name is
    'source-extractor'; causing a crash in usage. This forced the users to
    manually create a 'source-extractor' symbolic link in the '.local/bin'
    directory.

 6. The 'reproduce/software/shell/tarball-prepare.sh' script (that is used
    for making Maneage-standard tarballs) wouldn't accept option values
    with an '=' between the option name and value! It also didnt' print
    sufficiently informative messages and errors (for example it would say
    "skipping ..." (making the user think there is a problem!), but it was
    actually that the file already existed!

 7. The 'reproduce/analysis/make/prepare.mk' and
    'reproduce/analysis/make/verify.mk' Makefiles that needed to reject
    some of the 'makesrc' sub-Makefiles would simply substitute their names
    with nothing. But this would cause problems when the name is part of
    the name of another sub-Makefile.

 8. On the Debian 6 system mentioned above the raw 'df' command's output
    wasn't in the expected format; so Maneage would fail to properly detect
    the free space in the disk.

With these commit, all the issues above have been solved: for 1, A check
has been added to avoid using that option. For 2, those 'util-linux'
features have been disabled. For 3, the '--norc' and '--noprofile' options
have beed added to the call to Bash. For 4, see below. For 5, the symbolic
link is now automatically made with SExtractor. For 6, the option reading
components of that script have been fully re-written and more robust sanity
checks are also added, with more informative warnings. For 7, the 'subst'
function of Make was replaced with 'filter-out' and this fixed the
problem. For 8, 'df' is called with the '-P' option so it has a unified
format in all versions.

For 4, the versions of 'flex' and 'netpbm' have been updated. Since they
were the dependency of 'astrometrynet', that has also been updated. In the
process, we discovered that 'lzip' has a new version which claims to be
faster, so that is also updated.

       lzip            1.22     --&gt;   1.23
       astrometrynet   0.85     --&gt;   0.89
       flex            2.6.4    --&gt;   2.6.4-410-74a89fd
       netpbm          10.73.39 --&gt;   10.73.39

NetPBM needed some manual manipulation in its source (to remove the extra
line), so the necessary steps have been added to its build recipe in
'reproduce/software/make/high-level.mk'.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Until now, there were several portability issues in Maneage:

 1. Maneage would crash on older operating systems (checked on Debian 6),
    where Wget didn't have the '--no-use-server-timestamps'.

 2. On a Linux kernel 2.6.32 (of the same Debian 6 above) some features in
    'util-linux' (like 'swapon' or 'libmount') wouldn't build and wouldn't
    let 'util-linux' complete. These features need root permissions to be
    useful, so the wouldn't be used in Maneage any way! But they wouldn't
    let Maneage get built

 3. The './project shell' command would still read the host's '~/.bashrc',
    letting the host environment leak-in to Maneage's interactive shell.

 4. The building of Flex 2.64 wouldn't complete due to a segmentation
    fault an Ubuntu, but NetPBM (which depends on Flex) would crash with a
    wrong usage of 'yyunput'. This had actually caused a non-update to
    Flex in a previous Maneage software update.

 5. The update Astrometry.net would assume SExtractor's executable name is
    'source-extractor'; causing a crash in usage. This forced the users to
    manually create a 'source-extractor' symbolic link in the '.local/bin'
    directory.

 6. The 'reproduce/software/shell/tarball-prepare.sh' script (that is used
    for making Maneage-standard tarballs) wouldn't accept option values
    with an '=' between the option name and value! It also didnt' print
    sufficiently informative messages and errors (for example it would say
    "skipping ..." (making the user think there is a problem!), but it was
    actually that the file already existed!

 7. The 'reproduce/analysis/make/prepare.mk' and
    'reproduce/analysis/make/verify.mk' Makefiles that needed to reject
    some of the 'makesrc' sub-Makefiles would simply substitute their names
    with nothing. But this would cause problems when the name is part of
    the name of another sub-Makefile.

 8. On the Debian 6 system mentioned above the raw 'df' command's output
    wasn't in the expected format; so Maneage would fail to properly detect
    the free space in the disk.

With these commit, all the issues above have been solved: for 1, A check
has been added to avoid using that option. For 2, those 'util-linux'
features have been disabled. For 3, the '--norc' and '--noprofile' options
have beed added to the call to Bash. For 4, see below. For 5, the symbolic
link is now automatically made with SExtractor. For 6, the option reading
components of that script have been fully re-written and more robust sanity
checks are also added, with more informative warnings. For 7, the 'subst'
function of Make was replaced with 'filter-out' and this fixed the
problem. For 8, 'df' is called with the '-P' option so it has a unified
format in all versions.

For 4, the versions of 'flex' and 'netpbm' have been updated. Since they
were the dependency of 'astrometrynet', that has also been updated. In the
process, we discovered that 'lzip' has a new version which claims to be
faster, so that is also updated.

       lzip            1.22     --&gt;   1.23
       astrometrynet   0.85     --&gt;   0.89
       flex            2.6.4    --&gt;   2.6.4-410-74a89fd
       netpbm          10.73.39 --&gt;   10.73.39

NetPBM needed some manual manipulation in its source (to remove the extra
line), so the necessary steps have been added to its build recipe in
'reproduce/software/make/high-level.mk'.
</pre>
</div>
</content>
</entry>
</feed>
