<feed xmlns='http://www.w3.org/2005/Atom'>
<title>paper-concept.git/configure, branch maneage</title>
<subtitle>Paper (Towards Long-term and Archivable Reproducibility)</subtitle>
<link rel='alternate' type='text/html' href='https://git.maneage.org/paper-concept.git/'/>
<entry>
<title>Single wrapper instead of old ./configure, Makefile and ./for-group</title>
<updated>2019-07-28T01:53:34+00:00</updated>
<author>
<name>Mohammad Akhlaghi</name>
<email>mohammad@akhlaghi.org</email>
</author>
<published>2019-07-28T01:32:20+00:00</published>
<link rel='alternate' type='text/html' href='https://git.maneage.org/paper-concept.git/commit/?id=6ef4cc854d1df46b719de5d66b45537b0aa11f92'/>
<id>6ef4cc854d1df46b719de5d66b45537b0aa11f92</id>
<content type='text'>
Until now, to work on a project, it was necessary to `./configure' it and
build the software. Then we had to run `.local/bin/make' to run the project
and do the analysis every time. If the project was a shared project between
many users on a large server, it was necessary to call the `./for-group'
script.

This way of managing the project had a major problem: since the user
directly called the lower-level `./configure' or `.local/bin/make' it was
not possible to provide high-level control (for example limiting the
environment variables). This was especially noticed recently with a bug
that was related to environment variables (bug #56682).

With this commit, this problem is solved using a single script called
`project' in the top directory. To configure and build the project, users
can now run these commands:

   $ ./project configure
   $ ./project make

To work on the project with other users in a group these commands can be
used:

  $ ./project configure --group=GROUPNAME
  $ ./project make      --group=GROUPNAME

The old options to both configure and make the project are still valid. Run
`./project --help' to see a list. For example:

  $ ./project configure -e --host-cc
  $ ./project make -j8

The old `configure' script has been moved to
`reproduce/software/bash/configure.sh' and is called by the new `./project'
script. The `./project' script now just manages the options, then passes
control to the `configure.sh' script. For the "make" step, it also reads
the options, then calls Make. So in the lower-level nothing has
changed. Only the `./project' script is now the single/direct user
interface of the project.

On a parallel note: as part of bug #56682, we also found out that on some
macOS systems, the `DYLD_LIBRARY_PATH' environment variable has to be set
to blank. This is no problem because RPATH is automatically set in macOS
and the executables and libraries contain the absolute address of the
libraries they should link with. But having `DYLD_LIBRARY_PATH' can
conflict with some low-level system libraries and cause very hard to debug
linking errors (like that reported in the bug report).

This fixes bug #56682.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Until now, to work on a project, it was necessary to `./configure' it and
build the software. Then we had to run `.local/bin/make' to run the project
and do the analysis every time. If the project was a shared project between
many users on a large server, it was necessary to call the `./for-group'
script.

This way of managing the project had a major problem: since the user
directly called the lower-level `./configure' or `.local/bin/make' it was
not possible to provide high-level control (for example limiting the
environment variables). This was especially noticed recently with a bug
that was related to environment variables (bug #56682).

With this commit, this problem is solved using a single script called
`project' in the top directory. To configure and build the project, users
can now run these commands:

   $ ./project configure
   $ ./project make

To work on the project with other users in a group these commands can be
used:

  $ ./project configure --group=GROUPNAME
  $ ./project make      --group=GROUPNAME

The old options to both configure and make the project are still valid. Run
`./project --help' to see a list. For example:

  $ ./project configure -e --host-cc
  $ ./project make -j8

The old `configure' script has been moved to
`reproduce/software/bash/configure.sh' and is called by the new `./project'
script. The `./project' script now just manages the options, then passes
control to the `configure.sh' script. For the "make" step, it also reads
the options, then calls Make. So in the lower-level nothing has
changed. Only the `./project' script is now the single/direct user
interface of the project.

On a parallel note: as part of bug #56682, we also found out that on some
macOS systems, the `DYLD_LIBRARY_PATH' environment variable has to be set
to blank. This is no problem because RPATH is automatically set in macOS
and the executables and libraries contain the absolute address of the
libraries they should link with. But having `DYLD_LIBRARY_PATH' can
conflict with some low-level system libraries and cause very hard to debug
linking errors (like that reported in the bug report).

This fixes bug #56682.
</pre>
</div>
</content>
</entry>
<entry>
<title>Corrected preparation to build software in shared memory</title>
<updated>2019-07-24T23:13:15+00:00</updated>
<author>
<name>Mohammad Akhlaghi</name>
<email>mohammad@akhlaghi.org</email>
</author>
<published>2019-07-24T23:13:15+00:00</published>
<link rel='alternate' type='text/html' href='https://git.maneage.org/paper-concept.git/commit/?id=c58835bc4196ac3abad608b67455df8fd5f42683'/>
<id>c58835bc4196ac3abad608b67455df8fd5f42683</id>
<content type='text'>
The previous implementation of using shared memory to build software was
edited to be more clear and less prone to errors.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
The previous implementation of using shared memory to build software was
edited to be more clear and less prone to errors.
</pre>
</div>
</content>
</entry>
<entry>
<title>Temporary software building done in shared memory if available</title>
<updated>2019-07-24T13:38:29+00:00</updated>
<author>
<name>Mohammad Akhlaghi</name>
<email>mohammad@akhlaghi.org</email>
</author>
<published>2019-07-24T13:38:29+00:00</published>
<link rel='alternate' type='text/html' href='https://git.maneage.org/paper-concept.git/commit/?id=dbf525a90a096931551274f3e2fce49e00d0bf72'/>
<id>dbf525a90a096931551274f3e2fce49e00d0bf72</id>
<content type='text'>
Until now, the template would unpack the software and build them directly
in the build directory of each project. After installation, the whole
unpacked directory is deleted. However, building the software involves the
reading and writing of millions of files, so on the long run, it can be bad
for the non-volatile memory (HDDs or SSDs), it can also be slightly slow.

To fix this, if the system has a shared-memory directory (commonly named
`/dev/shm'), we can do the temporary building of the software there. The
great thing about this unique directory is that it is actually in the RAM,
not on the HDD/SSD. This can slightly improve the speed (not much
probably), but more importantly it will not do any long-term harm to the
host's HDDs/SSDs.

With this commit, when there is a shared memory directory mounted in
`/dev/shm', and it has enough space (currently set to 2GB), the
`./configure' script will make `.build/software/build-tmp' as a symbolic
link to a fixed directory there. Otherwise, it will just build it as a
directory in the project's shared directory.

The structure has been defined in such a way that we can later easily add
different standard shared-memory locations (for different operating
systems).

This completes task #15336.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Until now, the template would unpack the software and build them directly
in the build directory of each project. After installation, the whole
unpacked directory is deleted. However, building the software involves the
reading and writing of millions of files, so on the long run, it can be bad
for the non-volatile memory (HDDs or SSDs), it can also be slightly slow.

To fix this, if the system has a shared-memory directory (commonly named
`/dev/shm'), we can do the temporary building of the software there. The
great thing about this unique directory is that it is actually in the RAM,
not on the HDD/SSD. This can slightly improve the speed (not much
probably), but more importantly it will not do any long-term harm to the
host's HDDs/SSDs.

With this commit, when there is a shared memory directory mounted in
`/dev/shm', and it has enough space (currently set to 2GB), the
`./configure' script will make `.build/software/build-tmp' as a symbolic
link to a fixed directory there. Otherwise, it will just build it as a
directory in the project's shared directory.

The structure has been defined in such a way that we can later easily add
different standard shared-memory locations (for different operating
systems).

This completes task #15336.
</pre>
</div>
</content>
</entry>
<entry>
<title>Minor corrections in configure and high-level</title>
<updated>2019-07-15T00:16:31+00:00</updated>
<author>
<name>Mohammad Akhlaghi</name>
<email>mohammad@akhlaghi.org</email>
</author>
<published>2019-07-15T00:16:31+00:00</published>
<link rel='alternate' type='text/html' href='https://git.maneage.org/paper-concept.git/commit/?id=9901c3b2db0a3f85105489faa860fc27227ad4ac'/>
<id>9901c3b2db0a3f85105489faa860fc27227ad4ac</id>
<content type='text'>
Configure script: when `texlive-ready-tlmgr' is not created, it is similar
to not having installed TeXLive. A check was added so in this scenario the
`./configure' script doesn't crash.

high-level.mk: `cairo' and `pixman' are now installed in parallel and with
`V=1' (so the full compilation and linking command is printed).
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Configure script: when `texlive-ready-tlmgr' is not created, it is similar
to not having installed TeXLive. A check was added so in this scenario the
`./configure' script doesn't crash.

high-level.mk: `cairo' and `pixman' are now installed in parallel and with
`V=1' (so the full compilation and linking command is printed).
</pre>
</div>
</content>
</entry>
<entry>
<title>Rpath in libstdc++ and passing host_cc to high-level</title>
<updated>2019-07-07T23:03:49+00:00</updated>
<author>
<name>Mohammad Akhlaghi</name>
<email>mohammad@akhlaghi.org</email>
</author>
<published>2019-07-07T23:03:49+00:00</published>
<link rel='alternate' type='text/html' href='https://git.maneage.org/paper-concept.git/commit/?id=bc49a299fb2c64d9c26dd7bf3682bc79800cb6a3'/>
<id>bc49a299fb2c64d9c26dd7bf3682bc79800cb6a3</id>
<content type='text'>
Until this commit, the addion of `-liconv' to `CXXFLAGS' in `high-level.mk'
dependend on `host_cc', but I had forgot that `host_cc' isn't defined for
`high-level.mk'. It is now defined for this Makefile also in the configure
script.

Also, the Standard C++ library depends on `libgcc_s.so.1', so after
building GCC, it was necessary to add Rpath to it.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Until this commit, the addion of `-liconv' to `CXXFLAGS' in `high-level.mk'
dependend on `host_cc', but I had forgot that `host_cc' isn't defined for
`high-level.mk'. It is now defined for this Makefile also in the configure
script.

Also, the Standard C++ library depends on `libgcc_s.so.1', so after
building GCC, it was necessary to add Rpath to it.
</pre>
</div>
</content>
</entry>
<entry>
<title>Compiler checks added to the configure script</title>
<updated>2019-07-07T21:27:20+00:00</updated>
<author>
<name>Mohammad Akhlaghi</name>
<email>mohammad@akhlaghi.org</email>
</author>
<published>2019-07-07T21:27:20+00:00</published>
<link rel='alternate' type='text/html' href='https://git.maneage.org/paper-concept.git/commit/?id=db322d5bfef46f3b7ad11d3c8b26f45f5ec1dc1a'/>
<id>db322d5bfef46f3b7ad11d3c8b26f45f5ec1dc1a</id>
<content type='text'>
To build the software, we need a C/C++ compiler. But until now at configure
time there was no check on this. As a result, when the system didn't have
them, the user would only learn in a crash of the programs.

With this commit, some checks have been added at configure time to do basic
checks and inform the user if necessary.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
To build the software, we need a C/C++ compiler. But until now at configure
time there was no check on this. As a result, when the system didn't have
them, the user would only learn in a crash of the programs.

With this commit, some checks have been added at configure time to do basic
checks and inform the user if necessary.
</pre>
</div>
</content>
</entry>
<entry>
<title>Corrections to basic build</title>
<updated>2019-06-28T09:16:46+00:00</updated>
<author>
<name>Prasenjit Saha</name>
<email>psaha@physik.uzh.ch</email>
</author>
<published>2019-06-28T09:15:48+00:00</published>
<link rel='alternate' type='text/html' href='https://git.maneage.org/paper-concept.git/commit/?id=73cd532901865fe0b3a53c3e526ab5f7bd7d3122'/>
<id>73cd532901865fe0b3a53c3e526ab5f7bd7d3122</id>
<content type='text'>
Several corrections were necessary in the basic build: 1) the
version of GCC on some systems includes an `_' which would cause
a crash when building the PDF. 2) libcharset had to be manually
added to the Git build.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Several corrections were necessary in the basic build: 1) the
version of GCC on some systems includes an `_' which would cause
a crash when building the PDF. 2) libcharset had to be manually
added to the Git build.
</pre>
</div>
</content>
</entry>
<entry>
<title>Removing extra spaces in variable definition in configure script</title>
<updated>2019-05-22T09:04:49+00:00</updated>
<author>
<name>Raul Infante-Sainz</name>
<email>infantesainz@gmail.com</email>
</author>
<published>2019-05-22T09:04:49+00:00</published>
<link rel='alternate' type='text/html' href='https://git.maneage.org/paper-concept.git/commit/?id=51bae08d52d3e6013b3b0fbfb4e5193e9f7b06f4'/>
<id>51bae08d52d3e6013b3b0fbfb4e5193e9f7b06f4</id>
<content type='text'>
With this commit, extra spaces in the definition of two variables in the
`configure' script have been removed. Since they are defined in a
`shell' context (not in `make'), variable assignation is not allow to
have any space at both sides of the `=' character. I detected these
typos when trying to run `./configure' in a particular project.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
With this commit, extra spaces in the definition of two variables in the
`configure' script have been removed. Since they are defined in a
`shell' context (not in `make'), variable assignation is not allow to
have any space at both sides of the `=' character. I detected these
typos when trying to run `./configure' in a particular project.
</pre>
</div>
</content>
</entry>
<entry>
<title>Source directory links to build directory all managed in configure</title>
<updated>2019-05-21T13:43:09+00:00</updated>
<author>
<name>Mohammad Akhlaghi</name>
<email>mohammad@akhlaghi.org</email>
</author>
<published>2019-05-21T13:43:09+00:00</published>
<link rel='alternate' type='text/html' href='https://git.maneage.org/paper-concept.git/commit/?id=ba3546617998735301f1e72fa98872e27520829f'/>
<id>ba3546617998735301f1e72fa98872e27520829f</id>
<content type='text'>
Until now, the `tex/build' symbolic link was put in the clone/source tree
when the build-directory's `tex' directory was being built. Thanks to
Roberto Baena, we just found a bug because of this behavior: when a second
group member is trying to build the pipeline, since the build directory's
`tex' directory already exists, no `tex/build' will be put in their
clone/source directory. As a result, the PDF building will crash.

To fix this (and keep things organized), the two `tex/build' and `tex/tikz'
links (to the build directory) are now built in the configure step while it
is building all the top-level directories. They are no longer built within
the Makefiles.

Also, a comment was added on top of every directory built during the
configuration phase to be clear.

This fixes bug #56362.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Until now, the `tex/build' symbolic link was put in the clone/source tree
when the build-directory's `tex' directory was being built. Thanks to
Roberto Baena, we just found a bug because of this behavior: when a second
group member is trying to build the pipeline, since the build directory's
`tex' directory already exists, no `tex/build' will be put in their
clone/source directory. As a result, the PDF building will crash.

To fix this (and keep things organized), the two `tex/build' and `tex/tikz'
links (to the build directory) are now built in the configure step while it
is building all the top-level directories. They are no longer built within
the Makefiles.

Also, a comment was added on top of every directory built during the
configuration phase to be clear.

This fixes bug #56362.
</pre>
</div>
</content>
</entry>
<entry>
<title>Imported some recent/parallel work, conflicts fixed</title>
<updated>2019-04-30T18:12:56+00:00</updated>
<author>
<name>Mohammad Akhlaghi</name>
<email>mohammad@akhlaghi.org</email>
</author>
<published>2019-04-30T18:12:56+00:00</published>
<link rel='alternate' type='text/html' href='https://git.maneage.org/paper-concept.git/commit/?id=78737b78ac45d1213395819f3b85b4226088bd1a'/>
<id>78737b78ac45d1213395819f3b85b4226088bd1a</id>
<content type='text'>
Especially because of the new convention regarding backslashes, there were
many conflicts that are now fixed. But none were substantial.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Especially because of the new convention regarding backslashes, there were
many conflicts that are now fixed. But none were substantial.
</pre>
</div>
</content>
</entry>
</feed>
