| Age | Commit message (Collapse) | Author | Lines |
|---|
|
One of the main reasons to building Maneage is to properly
acknowledge/attribute the authors of software in research. So we have
adopted a standard of never referring to the GNU-based operating systems
running the Linux kernel simply as "Linux", we avoid terms like "Open
Sourse" and use Free Software instead (in the same spirit).
With this commit, a few instances of the cases above have been corrected,
they had slipped through our fingers when we initially imported them into
the project. In the special case of the "Journal for Open Source Software",
we simply replaced it with its abbreviation (JOSS). This was done because
in effect we were generally using journal name abbreviations in almost all
the citations already. To avoid any inconsistancies, the names of the three
other journals that weren't abbreviated are also abbreviated.
|
|
With this commit, Maneage now includes instructions to build the memory
tracing tool Valgrind and the program 'patch' (to apply corrections/patches
in text files and in particular the sources of programs).
For this version of Valgrind, some patches were necessary for an interface
with OpenMPI 2.x (which is the case now). Also note that this version of
Valgrind's checks can fail with GCC 10.1.x (when using '--host-cc'), and
the failures aren't due to internal problems but due to how the tests are
designed (https://bugs.gentoo.org/707598). So currently if any of
Valgrind's checks fail, Maneage still assumes that Valgrind was built and
installed successfully.
While testing on macOS, we noticed that it needs the macOS-specific 'mig'
program which we can't build in Maneage. DESCRIPTION: The mig command
invokes the Mach Interface Generator to generate Remote Procedure Call
(RPC) code for client-server style Mach IPC from specification files. So a
symbolic link to the system's 'mig' is now added to the project's programs
on macOS systems.
This commit's build of Patch and Valgrind has been tested on two GNU/Linux
distributions (Debian and ArchLinux) as well as macOS.
Work on this commit started by Boud Roukema, but also involved tests and
corrections by Mohammad Akhlaghi and Raul Infante-Sainz.
|
|
Until now, two of the software BibTeX sources (Matplolib and Sympy) had an
"abstract" entry that was long, not similar to the rest, and not relevant
in this context, so they are removed with this commit.
|
|
In time, some of the copyright license description had been mistakenly
shortened to two paragraphs instead of the original three that is
recommended in the GPL. With this commit, they are corrected to be exactly
in the same three paragraph format suggested by GPL.
The following files also didn't have a copyright notice, so one was added
for them:
reproduce/software/make/README.md
reproduce/software/bibtex/healpix.tex
reproduce/analysis/config/delete-me-num.conf
reproduce/analysis/config/verify-outputs.conf
|
|
Until this commit, when the version of Gnuastro doesn't match with the
version that the project was designed to use, the warning message saying
how to run the configure step was not showing the option `-e'. This
situation is normal when updating the version of Gnuastro to the most
recent one (with the project already configured). However, the use of this
option is more convenient than giving the top-build directory, etc, every
time. With this commit, the warning message has been changed in order show
also the option `-e' in the re-configure of the project.
|
|
Until this commit, Scamp was installed with the option
`--enable-plplot=yes' (the default). However, Maneage does not have PLplot
included. As it is possible to install Scamp without PLplot (in that case
it won't generate plots), with this commit this option has been set to
`no'. As a consequence, Scamp will be installed even if the host system
does not have PLplot without crashing (but it won't make any plot).
|
|
Until now Maneage used the host's GNU Gettext if it was present. Gettext is
a relatively low-level software that enables programs to print messages in
different languages based on the host environment. Even though it has not
direct effect on the running of the software for Maneage and the lanugage
environment in Maneage is pre-determined, it is necessary to have it
because if the basic programs see it in the host they will link with it and
will have problems if/when the host's Gettext is updated.
With this commit (which is actually a squashed rebase of 9 commits by Raul
and Mohammad), Gettext and its two extra dependencies (libxml2 and
libunistring) are now installed within Maneage as a basic software and
built before GNU Bash. As a result, all programs built afterwards will
successfully link with our own internal version of Gettext and
libraries. To get this working, some of the basic software dependencies had
to updated and re-ordered and it has been tested in both GNU/Linux and
macoS.
Some other minor issues that are fixed with this commit
- Until this commit, when TeX was not installed, the warning message
saying how to run the configure step in order to re-configure the
project was not showing the option `-e'. However, the use of this option
is more convenient than entering the top-build directory and etc every
time. So with this commit, the warning message has been changed in order
use the option `-e' in the re-configure of the project.
- Until now, on macOS systems, Bash was not linking with our internally
built `libncurses'. With this commit, this has been fixed by setting
`--withcurses=yes' for Bash's configure script.
|
|
Until now there we had manually inserted a `\' before the `_' of sip_tpv
program. However, we also recently added a step in the configure script to
add a `\' before every `_' when writing the final LaTeX macro. This was
because some C compilers (when the host's is used) have an `_' in their
version that we had no control over.
With this commit, the `\' is removed from `sip_tpv' in its build-rule and
we let the backslash be inserted automatically.
|
|
Until now, when you changed the version of a software in an already-built
system, its tarball would be downloaded, but it wouldn't actually
build. The only way would be to force the build by deleting the main target
of that file (under `.local/version-info/TYPE/PROGRAM'). This was because
the tarballs were an order-only prerequisite which was implemented some
time ago based on some theoretical argument that if the tarball dates
changes, the software should not be rebuilt (because we check the
checksum).
However, the problems this causes are more than those it solves: Users may
forget to delete the main target of the program and mistakenly think that
they are using the new version. The fact that all the numbers going into
the paper also contain this number further hides this.
With this commit, tarballs are no longer order-only and any time a version
of a software is updated, it will be automatically built and not cause
confusion and manual intervention by the users. As a result of this change,
I also had to correct the way we find the tarball from the list of
prerequisites.
|
|
Until this commit, the configure step would fail with an error when
compiling libgit2 on a test system. The origin of this bug, on the OS that
was tested, appears to be that in OpenSSL Version 1.1.1a, openssl/ec.h
fails to include openssl/openconf.h. The bug is described in more detail at
https://savannah.nongnu.org/bugs/index.php?58263
With this commit, this is fixed by manually inserting a necessary
components. In particular, `sed` is used to insert a preprocessor
instruction into `openssl/openconf.h`, defining `DEPRECATED_1_2_0(f)`, for
an arbitrary section of code `f`, to include that code rather than exclude
it or warn about it.
This commit is valid provided that openssl remains at a version earlier
than 1.2.0. Starting at version 1.2.0, deprecation warnings should be run
normally. We have thus moved the version of OpenSSL in `versions.conf' to
the section for programs that need to be manually checked for version
updates with a note to remind the user when reaching that version.
Other packages that use OpenSSL may benefit from this commit, not just
libgit2.
|
|
Until now, if GCC couldn't be built for any reason, Maneage would crash and
the user had no way forward. Since GCC is complicated, it may happen and is
frustrating to wait until the bug is fixed. Also, while debugging Maneage,
when we know GCC has no problem, because it takes so long, it discourages
testing.
With this commit, we have re-activated the `--host-cc' option. It was
already defined in the options of `./project', but its affect was nullified
by hard-coding it to zero in the configure script on GNU/Linux systems. So
with this commit that has been removed and the user can use their own C
compiler on a GNU/Linux operating system also.
Furthermore, to inform the user about this option and its usefulness, when
GCC fails to build, a clear warning message is printed, instructing the
user to post the problem as a bug and telling them how to continue building
the project with the `--host-cc' option.
|
|
Until now, at the end of the configuration step, we would tell the user
this: "To change the configuration later, please re-run './project
configure', DO NOT manually edit the relevant files". However, as Boud
suggested in Bug #58243, this is against our principle to encourage users
to modify Maneage.
With this commit, that explanation has been expanded by a few sentences to
tell the users what to change and warn them in case they decide to change
the build-directory.
|
|
Until now Gnuastro and Astropy where installed by default in any clean
build of Maneage. Gnuastro is used to do the demonstration analysis that is
reported in the paper and Astropy was just there to help in testing the
building of the MANY tools it depends on! It (and its dependencies) also
had several papers that helped show software citation.
However, as Boud suggested in task #15619, the burden of installing them
for a new user may be too much and any future changes will cause merge
conflicts. It may also give the impression that Maneage is only/mainly
written for astronomers.
So with this commit, I am removing Astropy as a default target. But we can
only remove Gnuastro after we include an alternative analysis in the
demonstration `delete-me' files. Following Boud's suggestion in that task,
`TARGETS.conf' was also added to the files to be ignored in any future
merge (in the checklist of `README-hacking.mk'). The solution was already
described there, but mainly focused on the deleted `delete-me' files. So
with this commit, I brought out this item as a more prominent item in the
list. Maybe we can later add the analysis done in the Maneage paper (not
yet published).
In terms of testing the software builds, we already have task #15272
(Single target to build all high-level software, for testing) that aims to
have a single configure option to install ALL high-level software and we
can ask people to try if they like and report errors.
|
|
Similar to the previous commit (e43e3291483699), following a change made
yesterday in the identification of software names from their tarballs, a
few other problematic names are corrected with this commit: `apr-util',
HDF5, TeX Live's installation tarball and `rpcsvc-proto'.
Even though we have visually checked the list of software, other
unidentified similar cases may remain and will be fixed when found in
practice.
|
|
Until Commit 3409a54 (from yesterday), pkg-config was found correctly in
`reproduce/software/make/basic.mk` by searching for `pkg`. However, commit
a21ea20 made an improvement in the regular expression for relating package
names and download filenames, and the string `pkg-config` with the new
regex no longer simplifies to `pkg`. The result of this was that the
basic.mk could not find `pkg-config` in the list of packages, since it was
still listed as `pkg`. This blocked downloading for a system without
pkg-config preloaded.
With this commit (of just a few bytes), the bug is fixed.
|
|
Until now, we wouldn't explicity check for GNU gettext. If it was present
on the system, we would just add a link to it in Maneage's installation
directory. However, in bug #58248, Boud noticed that Git (a basic software)
actually needs it to complete its installation. Unfortunately we haven't
had the tiem to include a build of Gettext in Maneage. Because it is mostly
available on many systems, it hasn't been reported too commonly, it also
has many dependencies which make it a little time consuming to install.
So with this commit, we actually check for GNU gettext right after checking
the compiler and if its not available an informative error message is
written to inform the user of the problem, along with suggestions on fixing
it (how to install GNU gettext from their package manager).
|
|
Until now, the sed script for determining URL download rules in the three
software building Makefiles (`basic.mk', `high-level.mk' and `python.mk')
considered package names such as `fftw-3...` and `fftw2-2.1...` to be
identical. As the example above shows, this would make it hard to include
some software that may hav conflicting non-number names.
With this commit, the SED script that is used to separate the version from
the tarball name only matches numbers that are after a dash
(`-'). Therefore considers `fftw-3...` and `fftw-2...` to be identical, but
`fftw-3-...` and `fftw2-2.1...` to be different. As a result of this
change, the `elif' check for some of the other programs like `m4', or
`help2man' was also corrected in all three Makefiles.
While doing this check on all the software, we noticed that `zlib-version'
is being repeated two times in `version.conf' so it was removed. It caused
no complications, because both were the same number, but could lead to bugs
later.
|
|
Recently (since Commit 7d0c5ef77), the preparation is not run automatically
every time. It is only run automatically the first time and needs to be
manually called with the `--prepare-redo' option. But this wasn't explained
in `README-hacking.md' (currently the main documentation of Maneage).
With this commit, a description about invoking the preparation process
after the first attempt of the running project has been added to
`README-hacking.md'.
|
|
Recently (in Commit 8eb0892e) the Gnuastro configuration files moved under
"reproduce/analysis/config/gnuastro" directory (before that they were in
`reproduce/software/config/gnuastro)'. But this hadn't been reflected in it
the variable that defines this directory in `initialize.mk'.
With this commit, the address of the Gnuastro configuration files directory
is corrected, allowing Gnuastro programs to operate properly when it is
used.
|
|
Until now, the comment in the file said that setting the `verify-outputs`
variable to `yes` disables the verification. Looking at
`reproduce/analysis/make/verify.mk` shows that the opposite is true.
With this commit, the word `disable` is replaced with `enable` so that the
user is not confused by the conflict between the source code in the other
file and this comment.
|
|
Until now we only checked for the existance and write-ability of the build
directory. But we recently discovered that if the specified build-directory
is in a non-POSIX compatible partition (for example NTFS), permissions
can't be modified and this can cause crashs in some programs (in
particular, while building Perl, see [1]). The thing that makes this
problem hard to identify is that on such partitions, `chmod' will still
return 0 (so it was hard to find).
With this commit, a check has been added after the user specifies the
build-directory. If the proposed build directory is not able to handle
permissions as expected, the configure script will not continue and will
let the user know and will ask them for another directory.
Also, the two printed characters at the start of error messages were
changed to `**' (instead of `--'). When everything is good, we'll use `--'
to tell the user that their given directory will be used as the build
directory. And since there are multiple checks now, the final message to
specify a new build directory is now moved to the end and not repeated in
every check.
[1] https://savannah.nongnu.org/support/?110220
|
|
Until now, the primary Maneage URLs were under GitLab, but since we now
have a dedicated URL and Git repository, its better to transfer to this as
soon as possible. Therefore with this commit, throughout Maneage, any place
that Maneage was referenced through GitLab has been corrected.
Please correct your project's remote to point to the new repository at
`git.maneage.org/project.git', and please make sure it follows the
`maneage' branch. There is no more `master' branch on Maneage.
|
|
Tags are not a fixed piece of history (they can easily be moved and not
imported in a different repository), so they are only confusing in the
context of Maneage (where people should branch-off the main project). the
raw commit hashes are a much more robust way to store a precise moment in
history.
Before this commit, I removed all Tags from the main Git repositories of
Maneage and thus removed any mention of Tags with
`README-hacking.md'. Ofcourse, if a project decides to use tags is upto
them, but we won't implement it in the main branch.
|
|
Roberto Baena recently tried building a new project with Maneage and
provided the following suggestions to make it more clear for a new user:
1) In the part where we talk about creating a Git repository, we should
highlight that it must be empty. This is because some (for example Gitlab)
propose to include a `README' file. But if the project is not empty, Git
will not allow pushing to it.
2) The `(can be done later)' comment was removed from the "Delete dummy
parts") to avoid confusion about applying some of them, but not others: if
only some are done, it may cause problems in the build.
|
|
Until now, the message that we printed just before starting to build
software didn't actually print the current directory, but only `pwd'. With
this commit, this is fixed (it uses the `currentdir' variable that is
already found before).
|
|
We recently fixed the problem of TeXLive that hard-codes the year of its
build in its installation directory. But the note on this problem was still
kept in `README-hacking.md'. That part is now removed.
Also, to help in following the checklist, it is now an ordered list.
|
|
Until now, throughout Maneage we were using the old name of "Reproducible
Paper Template". But we have finally decided to use Maneage, so to avoid
confusion, the name has been corrected in `README-hacking.md' and also in
the copyright notices.
Note also that in `README-hacking.md', the main Maneage branch is now
called `maneage', and the main Git remote has been changed to
`https://gitlab.com/maneage/project' (this is a new GitLab Group that I
have setup for all Maneage-related projects). In this repository there is
only one `maneage' branch to avoid complications with the `master' branch
of the projects using Maneage later.
|
|
Of the GCC dynamically linked libraries we need to manually add RPATH to
all and for `libstdc++' we also need to tell it to link with
`libiconv'. Until now, the conditional to check for libstdc++ was not
working and thus libiconv wasn't been added to it.
With this commit the conditional has been corrected and is now
working. Also, to help in reading the logs, an echo statement was added
after every call to PatchELF.
|
|
Until now, when a the raw tarball of some software wasn't usable, I would
put it under my own webpage, or `akhlaghi.org/reproduce-software'. That
same address was also used as a backup server. However, now the project has
a proper name: Maneage. So I changed the directory on my own server to
`akhlaghi.org/maneage-software'.
With this commit, this new address has replaced the old one. But to avoid
crashes in projects that haven't yet merged with the main Maneage branch,
the old `reproduce-software' still works (its actually a symbolic link to
the new directory now).
|
|
In the previous commit, we remove the `-static' flag from building PatchELF
because it wasn't necessary any more. Howver, the comment for the check
still included it and could be confusing. This is corrected with this
commit. Also, we don't need the `good_static_libc' variable (that was only
defined to pass onto PatchELF). This has also been corrected.
|
|
Until a few commits ago, PatchELF was built statically because it was used
to patch `libstdc++' at the end of the GCC building phase, but PatchELF
also depends on `libstdc++', so it would crash. However, recently when
patching the GCC libraries, we don't directly apply Patchelf to the
library, first we copy it to a temporary place, do the patching, then put
it in its proper place. So the problem above won't happen any more.
With this commit, I am thus removing the static flag from patchelf and
letting it built dynamically all the time. The main problem was that some
systems don't have a static C++ library, so PatchELF couldn't be built
statically. Instead of adding more checks, we just fixed the core
foundation of the problem.
|
|
Until now the software configuration parameters were defined under the
`reproduce/software/config/installation/' directory. This was because the
configuration parameters of analysis software (for example Gnuastro's
configurations) were placed under there too. But this was terribly
confusing, because the run-time options of programs falls under the
"analysis" phase of the project.
With this commit, the Gnuastro configuration files have been moved under
the new `reproduce/analysis/config/gnuastro' directory and the software
configuration files are directly under `reproduce/software/config'. A clean
build was done with this change and it didn't crash, but it may cause
crashes in derived projects, so after merging with Maneage, please
re-configure your project to see if anything has been missed. Please let us
know if there is a problem.
|
|
In the previous commit, I removed the year from the basic installation of
TeXLive packages, but I forgot to correct this in the high-level TeXLive
packages! This is corrected with this commit.
|
|
It is this time of year again: TeXLive has transitioned to its 2020 release
and the year is imprinted into the installation directory of TeXLive. Until
now, we have had to manually change this year and it caused complications
and was very annoying.
With this commit, the explicit year has been removed from TeXLive's
installation and we now simply put a `maneage' instead of the year. I tried
this on another system and it worked nicely. Until the time that we can
fully install LaTeX packages from source tarballs, this is the best thing
we could do for now.
|
|
Elham Saremi recently reported the following errors when building Numpy in
numpy/core/src/npysort/radixsort.c.src: "error: 'for' loop initial
declarations are only allowed in C99 or C11 mode". After some searching, I
found Issue 14147[1] on Numpy's main repository about the same problem. As
described there, apparently Numpy needs C99 compiler, but doesn't check for
it or set it manually (for some strange reason, leaving it to the packagers
to check if they want!!!).
Any way, after a check with Elham, we were able to fix it by adding the
`--std=c99' to CFLAGS of Numpy's build and with this commit, it is now
being implemented in the core Maneage to not cause a problem in any other
project.
[1] https://github.com/numpy/numpy/issues/14147
|
|
Until now, Astropy was instructed to build its own internal copy of the
Expat library. However, with the recent commits before, Maneage now
includes an installation of Expat and Astropy can't keep the two (its
internal version and the project's version) separate, so they conflict and
don't let Astropy get built.
With this commit, the problem is fixed by setting the Expat library as an
explicit dependency of Astropy and asking Astropy to ignore its internal
copy.
While doing this, I recognized that it is much easier and elegant to add
steps in various stages of the `pybuild' function through hooks instead of
variables. So the fifth argument of the `pybuild' function was removed and
now it actually checks if hooks are defined as functions and if so, they
will be called.
The `pyhook_after' function was also implemented in the installation of
`pybind11' (which needed it, given that the 5th argument of `pybuild' was
removed) and after doing a test-build, I noticed that two lines were not
ending with a `\' in `boost' (a dependency of `pybind11').
Commit written originally by Mohammad Akghlaghi
|
|
Until now we would simply return the version numbers as they were written
into the separate files and situations can happen where the version numbers
contain an underscore (`_'). However, this character is a methematical
character in LaTeX, causing LaTeX to complain and abort.
With this commit, a step has been added at the end of the configure script
to convert any possible `_' to `\_'. Once it is commented (a backslash is
put behind it), the underscore will be printed as it is in the final PDF.
This commit was originally written by Mohammad Akhlaghi
|
|
Until now, the M4 that was built on macOS had internal problems (as
discussed in #1): it would simply print `Abort trap: 6' in the output and
abort. After looking at the build of Homebrew, I noticed that they apply a
patch (correct one line) to fix this problem. To be able to apply that
patch on macOS systems, I had fully open up the build recipe of M4 and
atleast on the testing system, it was built successfully.
Also, after successfully building M4, and thus Autoconf and thus Minizip,
we were able to build XLSX I/O on a macOS and found out that the internal
library's full address wasn't being put in the libraries and
executables. With this commit, we now use macOS's `install_name_tool' to
correct the positions of the two `libxlsxio_*' libraries in all its
executables.
This commit was originally written by Mohammad Akhlaghi
|
|
Until this commit, only the word `Minizip' was written into the Minizip
installation target (without the version number of Minizip). With this
commit, this minor bug has been fixed by using the appropiate Make
variable: `$(minizip-version)'.
|
|
Minizip is a dependency of XLSX I/O and until now, I was just using the
most recent version I found (2.9.2), but XLSX I/O is written for the
Minizip 1.x series, not 2.x. Somehow it didn't cause a crash on my
computer!!! I think XLSX I/O's CMake is instructed to look into system
directories by default when it doesn't find the directories in the given
places. And because I had installed Minizip on my operating system, it
did't complain.
Upon trying the build on their systems, Yahya, Raul and Zahra reported a
failure in the build of XLSX I/O which was due the to the problem above (we
were installing the wrong version of Minizip!).
With this commit, this has been fixed by installing the 1.x series of
Minizip (whish is actually installed within zlib!).
This commit was original done by Mohammad Akhlaghi.
|
|
With this commit, CMake has been updated to its most recent version.
This upgrade has been done because in the installation of XLSX I/O on
macOS laptop, it crashes complaining about C compiler "not able to
compile a simple test program". After a fast search, I found it could be
possible to just use the most recent version of CMake to solve the
problem. But it didn't work. In any case, it is good to have the most
recent version of CMake included.
|
|
There weren't any conflicts in this merge.
|
|
The subdirectories here (and the fact that they may be symbolic links) may
be confusing for some early project users, so a `README.md' file was added
there describing them and when they are links, when directories and when
some may not yet exist.
|
|
Until this commit, the year of this paper was 2019 and the linking url
was the temporal one. However, the final official publication year is
2020. With this commit, the year and the url have been changed to the
final ones.
|
|
With this commit, multiples tabs in the definition of MissFITS tarball
have been removed. Now they are white spaces.
|
|
Until this commit, PyYAML was not set as prerequisite of Astropy. This
package is an optional dependency of Astropy for some particular
functions. However, we have already included PyYAML into this project so
it is available. With this commit, PyYAML has been set as a prerequisite
of Astropy.
In addition to this, Html5lib and Beautifulsoup4 have been also added as
prerequsites of Astropy (and removed from Astroquery prerequisites). I
noticed that both of them are optional dependencies of Astropy.
|
|
In the last update of Astropy to version 4.0 they removed some things
that the previous version of Astroquery needs. As a consequence, it is
also necessary to update the Astroquery version to be a ble to run with
the Astropy 4.0. With this commit, the update of Astroquery to it most
recent version (0.4) has been done.
|
|
Until now, the preparation phase was always executed before the final build
phase when running `./project make'. But when it becomes necessary, project
preparation can be slow and will un-necessarily slow down the project while
the project is growing (focus is on the analysis that is done after
preparation).
With this commit, preparation will be done automatically the first time
that the project is run (`.build/software/preparation-done.mk' doesn't
exist). However, after preperation is complete once, future runs of
`./project make' won't do preparation any more (by calling
`top-prepare.mk'). They will directly call `top-make.mk' for the analysis.
To manually invoke preparation after the first attempt, the `./project
make' script should be run with the new `--prepare-redo' option.
Also, since the preparation phase is now automatically done before the
analysis phase, the long notice that describes running `./project make' at
the end of the preparation phase has been removed in `top-prepare.mk'. It
now just prints a short line, saying the preparation has been complete.
Finally, when the project has not been run with the proper group
configuration, it ends with an `exit 1' so the main `./project' script
doesn't proceed any further.
|
|
MissFITS is package for manipulating FITS files.
I added it as my first commit to the project for educational
purposes.
|
|
Until now, the final preparation target of the preparation phase depended
on all the `$(makesrc)' files. This caused a problem because we were
telling it to also depend on `prepare.tex' (which is the same file that is
being built).
With this commit, we are applying the same solution we have already done in
`paper.mk' (for `paper.tex'): we are removing `prepare' from the list of
prerequisites.
This bug was found by Zahra Sharbaf.
|
