| Age | Commit message (Collapse) | Author | Lines |
|---|
|
Until now, OpenMPI would complain about not having `ssh' or `rsh' as a
remote shell feature. However, such features should not be necessary in a
reproducible scenario and they also have major security issues.
With this commit, we are now using OpenMPI's `OMPI_MCA_plm_rsh_agent'
environment variable to disable any remote shell dependency for it (as
suggested by Boud). Therefore, any dependency for OpenSSH has been
removed. But I thought to keep the build instructions incase it may be
useful under some un-foreseen scenario. However, to discourage people from
building it, a notice was added ontop of the build instructions.
This bug was found, tested and solved thanks to Roberto Baena Gallé and
Boud Roukema.
This fixes bug #56724.
|
|
Until now, when you needed to completely clean a project (with `./project
make distclean') the Git hooks that are installed during configure time
would cause problems when committing (the `pre-commit' hook in particular
won't allow you to commit anything!).
With this commit, before deleting the software, the template first removes
these Git hooks.
|
|
Until now the only way to define the environment of the Make recipes was
through the exported Make variables (mostly in `initialize.mk' for the
analysis steps for example). However, there is only so much you can do with
environment variables! In some situations you want slightly more
complicated environment control, like setting an alias or running of
scripts (things that are commonly done in the `~/.bashrc' file of users to
configure their interactive, non-login shells).
With this commit, a `reproduce/software/bash/bashrc.sh' has been defined
for this job (which is currently empty!). Every major Make step of the
project adds this file as the `BASH_ENV' environment variable, so the shell
that is created to execute a recipe first executes this file, then the
recipe. Each top-level Makefile also defines a `PROJECT_STATUS' environment
variable that enables users to limit their envirnoment setup based on the
condition it is being setup (in particular in the early phase of
`basic.mk', where the user can't make any assumption about the programs and
has to write a portable shell script).
|
|
We recently moved the system's `rm' program absolute address to a shell
variable that is found during the `./project' script. But I had forgot to
account for the difference between the Make and Bash variable naming
differences. I had also forgot to add a value to the HOME variable.
With this commit both are corrected: the system's `rm' path is now called
`sys_rm' and the HOME variable is set.
|
|
Until now, to work on a project, it was necessary to `./configure' it and
build the software. Then we had to run `.local/bin/make' to run the project
and do the analysis every time. If the project was a shared project between
many users on a large server, it was necessary to call the `./for-group'
script.
This way of managing the project had a major problem: since the user
directly called the lower-level `./configure' or `.local/bin/make' it was
not possible to provide high-level control (for example limiting the
environment variables). This was especially noticed recently with a bug
that was related to environment variables (bug #56682).
With this commit, this problem is solved using a single script called
`project' in the top directory. To configure and build the project, users
can now run these commands:
$ ./project configure
$ ./project make
To work on the project with other users in a group these commands can be
used:
$ ./project configure --group=GROUPNAME
$ ./project make --group=GROUPNAME
The old options to both configure and make the project are still valid. Run
`./project --help' to see a list. For example:
$ ./project configure -e --host-cc
$ ./project make -j8
The old `configure' script has been moved to
`reproduce/software/bash/configure.sh' and is called by the new `./project'
script. The `./project' script now just manages the options, then passes
control to the `configure.sh' script. For the "make" step, it also reads
the options, then calls Make. So in the lower-level nothing has
changed. Only the `./project' script is now the single/direct user
interface of the project.
On a parallel note: as part of bug #56682, we also found out that on some
macOS systems, the `DYLD_LIBRARY_PATH' environment variable has to be set
to blank. This is no problem because RPATH is automatically set in macOS
and the executables and libraries contain the absolute address of the
libraries they should link with. But having `DYLD_LIBRARY_PATH' can
conflict with some low-level system libraries and cause very hard to debug
linking errors (like that reported in the bug report).
This fixes bug #56682.
|
|
Until now we were only setting the `LD_LIBRARY_PATH' environment variable
for GNU/Linux systems. But macOS systems use the `DYLD_LIBRARY_PATH'.
With this commit, for better control over the environment, we are also
fixing `DYLD_LIBRARY_PATH' in all the places that we are setting the
general environment variables.
|
|
While reviewing Prasenjit's commits, I noticed that we had forgot to add
the citation for TIDES, also to make things clear, the program/library
build rules are now sorted alphabetically.
Finally, I noticed that after building the TiKZ PDF figures, it is crashing
(like on Prasenjit's computer). After looking around, I noticed its because
we were setting the of the `TEXINPUTS' environment variable to be the
installed TeX Live directory (which was ultimately redundant because by
default TeX will look into where it was installed). The important thing is
just that we remove any possible value the host system has, not to set new
directories.
|
|
TIDES is an ODE integrator with multiple-precision arithmetic.
|
|
Several corrections were necessary in the basic build: 1) the
version of GCC on some systems includes an `_' which would cause
a crash when building the PDF. 2) libcharset had to be manually
added to the Git build.
|
|
Until now, for some reason, the permission flags of `top.mk' were 755 (good
for an executable), not 644 (which is what they should be for a plain text
file that is run by another program).
This is corrected with this commit.
|
|
There weren't any conflicts in this merge.
|
|
With this commit, ImageMagick software has been added into the project.
This software is useful to deal and treat images from the command line.
Since it is widely used and a lot of other programs rely on it, it is
worth to have it into the project.
|
|
Until now, the `tex/build' symbolic link was put in the clone/source tree
when the build-directory's `tex' directory was being built. Thanks to
Roberto Baena, we just found a bug because of this behavior: when a second
group member is trying to build the pipeline, since the build directory's
`tex' directory already exists, no `tex/build' will be put in their
clone/source directory. As a result, the PDF building will crash.
To fix this (and keep things organized), the two `tex/build' and `tex/tikz'
links (to the build directory) are now built in the configure step while it
is building all the top-level directories. They are no longer built within
the Makefiles.
Also, a comment was added on top of every directory built during the
configuration phase to be clear.
This fixes bug #56362.
|
|
Until now, the `download-multi-try' script assumed GNU Bash features (when
comparing the number of attempts at downloading), but it didn't explicitly
ask the operating system to be run with Bash. As a result, when weaker
shells were used (like the default Debian minimalist `dash' shell), the `>'
("larger than" operator in a math context) is interpreted a redirection and
two extra files are created: `1' and `maxcount'!
With this commit, we now start this script with `/bin/bash'. Ofcourse, this
will assume that the host has GNU Bash installed, but we are also making
this assumption in the configure script. So atleast for now, Bash (any
version) is a critical dependency of this template anyway.
|
|
When we need to quote the new-line character we end the line with a
backslash (`\'). Until now, our convention has been to put all such
backslashes under each other to help in visual inspection.
But this causes a lot of confusion in version control: if only one line's
length is larger, the whole block will be marked as changed and thus makes
it hard to visually see the actual change. It also makes debuging the code
(adding some temporary lines) hard.
With this commit, I went through all the files and tried to fix all such
cases so only a single white space character is between the last command
character and the backslash. Where there was an empty line (ending with a
backslash, to help in visually separating the code into blocks), I put the
backslash right under the previous line's.
This completes task #15259.
|
|
In a few cases, `reproduce/analysis/make/initialize.mk' still assumed the
old architecture. With this commit, they have been corrected.
|
|
Until now, there were erros in the citation of Astrometry-net and Scamp
papers.
With this commit, we fix these problems. The Swarp bibtex has also been
modify to follow the stetic of the citation style we have right now in
the project.
We also added the `dependency-bib.tex' as a prerequisite of `paper.bbl'.
|
|
Until now, the software building and analysis steps of the pipeline were
intertwined. However, these steps (of how to build a software, and how to
use it) are logically completely independent.
Therefore with this commit, the pipeline now has a new architecture
(particularly in the `reproduce' directory) to emphasize this distinction:
The `reproduce' directory now has the two `software' and `analysis'
subdirectories and the respective parts of the previous architecture have
been broken up between these two based on their function. There is also no
more `src' directory. The `config' directory for software and analysis is
now mixed with the language-specific directories.
Also, some of the software versions were also updated after some checks
with their webpages.
This new architecture will allow much more focused work on each part of the
pipeline (to install the software and to run them for an analysis).
|
