| Age | Commit message (Collapse) | Author | Lines |
|---|
|
It was recently announced by both RedHat[1] and CentOS[2] that CentOS 8
(which was meant to end LTS at 2030) will be terminated 8 years early (by
the end of 2021). This is a perfect example of the longevity issues when
relying on third-party providers.
With this commit, I added this as a parenthesis after mentioning Ubuntu's
LTS web address. Some minor edits were also done in other parts of this
paragraph.
[1] https://www.redhat.com/en/blog/centos-stream-building-innovative-future-enterprise-linux
[2] https://blog.centos.org/2020/12/future-is-centos-stream
|
|
Until now, there was no warning when the 'maneage' branch didn't exist in
the Git history. This can happen when you forget to push the 'maneage'
branch to a remote for your project, and you later clone your project from
that remote (for example on another computer). We use the 'maneage' branch
to report the latest commit hash and date in the final paper (which can
greatly help future readers). Since we check the 'maneage' branch on every
run of './project make' (in 'initialize.mk') this would result in a printed
statement like this:
fatal: Not a valid object name maneage
Also until now, the description of what to do when TeXLive wasn't installed
properly wasn't complete: it didn't mention that it is necessary to delete
the TeXLive target files. This could confuse users (they would re-run
'./project configure -e', but with no effect).
With this commit, for the 'maneage' branch issue a complete warning will be
printed. Telling the user what to do to get the 'maneage' branch (and thus
fix this warning). Also, the LaTeX macros that go in the paper are now red
when the 'maneage' branch doesn't exist, telling the user to see the
printed warning (thus encouraging the user to get the branch). For the
TeXLive issue, the necessary commands to run are now also printed in the
warning.
|
|
Until now, when building the high-level (optional) software, we would give
both 'CPPFLAGS' and 'C_INCLUDE_PATH' the same value/directory in
'high-level.mk'. But we recently found that on macOS's C compiler
('clang'), if a directory is included in both 'CPPFLAGS' and
'C_INCLUDE_PATH', then that directory is ignored in 'CPPFLAGS' (which has
higher priority). This caused linking problems when the version of a
software on the host was different from the Maneage version.
With this commit, 'C_INCLUDE_PATH' is not set on macOS any more and this
fixed the problem on the reported systems.
This bug was fixed with the help of Mohammad Akhlaghi and Mahdieh Navabi.
|
|
Today, Richard Stallman sent a mail in 'info-gnu@gnu.org' (GNU's public
announcements mailing list) about proprietary obsolescence (or planned
obsolescence) [1]. After looking into it, I saw there is actually a
Wikipedia page for this concept. Since it direclty relates to our Free
software criteria, I thought its good to use this technical term there.
[1] https://www.gnu.org/proprietary/proprietary-obsolescence.html
[2] https://en.wikipedia.org/wiki/Planned_obsolescence
|
|
I just remembered that in the paragraph we compare with Jupyter, another
important point is that with based on the modularity principle, people can
choose their favorite text editor and aren't limited to one. I also tried
to remove redundant parts to avoid adding too many extra words.
|
|
Thanks a lot Boud for adding that script in your own project and linking it
here. Since the raw file (without context of the whole project) is very
hard to understand for the users, I switched the URL to the navigable URL
the link is actually on the filename. It will always show the most recent
version of this script, not the particular snapshot of now. But infact that
is better, since we can make it better and improve it over time. Maybe even
by the end of this paper's referee review will be able to include it in
Maneage's core branch.
I also removed the link to this discussion at the first paragraph of
Section IV (proof of concept). Since that is just the introduction, and
going into this level of detail there could be confusing for the
readers. Having the name of the script in the proper place is more direct
and understandable for the readers.
Thanks again Boud for the nice work on this ;-).
|
|
This commit adds the SWH URL of the statistical verification
script to the paper and tidies up the corresponding answer in
'1-answer.txt'. The script file includes more extensive
documentation than the earlier 'make' version of the method.
|
|
While going through Mohammad-reza's recent two commits, I noticed that we
had missed an importnat discussion on modularity in this version of the
paper (discussing how file management should also be modular resulting in
cheaper archival, and thus better longevity), so a few sentences were added
under criteria 2 (Modularity).
Mohammad-reza's edits were also generally very good and helped clarify many
points. I only reset the part that we discuss the problems with POSIX, and
not being able to produce bitwise reproducible software (which systems like
Guix work very hard at, and thus need root permissions). I felt the edit
missed the main point here (that while bitwise reproducibility of the
software is good, it is not always necessary).
|
|
Before this commit, there were discussions in different sections related to
POSIX compliance and features. Since the relevant Cmpleteness criterion has
been changed to execution within a Unix-like OS, such dicussions had to be
modifies as well.
With this commit, the parts that were related to condition (1) of the
Completeness criterion have been modified to be relevant to new Unix-like
OS requirement. Also, few spelling problems were fixed.
|
|
Before this commit, condition (1) for the Completeness criterion was
referring to POSIX compliance. POSIX is a very detailed dynamic standard
which goes under revision continuously and not a lot of operating systems,
GNU/Linux included are completely/officially POSIX-compliant. Furthermore,
not all sections of the huge 4000 pages standard are really important
specifically to the current Maneage functionality.
With this commit, condition (1) has been replaced by a looser condition of
execution within a Unix-like OS. Also since the term environment might have
been mistaken with the term "Operating Environment", it was replaced by the
unmistakable term "environment variables" in conditions (3) and (5). Last
but not least, condition (2) was made more restrict by adding ASCII
encoding as the condition for storing the plain text files.
TO-DO:
POSIX could contain valuable ideas regarding portability of programming
practices. These can be taken advantage of later in providing necessary and
sufficient conditions for project completeness. Another idea could be to
make LFS construct or something else as a sharp definition for what we mean
by minimal Unix-like OS.
|
|
Less is rarely used in non-interactive mode and is primarily intended for
interactively viewing large files. So its need within Maneage (for batch
processing) wasn't often felt until now. However, when running './project
shell' (which completely closes-off the outside environment), or building a
Maneage'd project within a minimal container that doesn't have less, it
becomes hard to use Git (and in particular its 'diff' output which depends
on 'less').
With this commit, Less has been added as a dependency of Git in
'basic.mk'. In total its built product is roughly 800KB and builds within a
second or two. So it isn't a burden on any project. But it can be very
useful when the projects are being developed within the Maneage environment
itself.
|
|
In a recent build on a macOS, we recognized that Texinfo needs the
'libintl.h' headers of Gettext. However, Gettext depends on M4, and until
now we had set M4 to depend on Texinfo. Therefore adding Gettext as a
dependency of Texinfo would cause a circular dependency.
On the macOS, we temporarily disabled M4's Texinfo dependency, and the
build went through. I also checked on my GNU/Linux system: temporarily
renamed all Texinfo built files from my system and done a clean build of M4
and it succeeded. To be further safe, I built Maneage from this commit
(where M4 doesn't depend on Texinfo) in a Docker container, and it went
through with no problems. So the current M4 version indeed doesn't need
Texinfo. I think adding Texinfo as a dependency of M4 was a historic issue
from the early days.
In the process, I also cleaned 'basic.mk' a little:
- A "# Level N" comment was added on top of each group of software that
can be built in parallel (generally).
- GNU Nano was moved to the end of the file (to be "Level 6").
- Some comments were edited in some places.
|
|
Until now at the end of the updating process, we hadn't explicity talked
about pushing the branches. So people would usually only push their
'master' branch to their remote. While the merged 'master' branch does
contain the commits from the core Maneage branch, having a no-updated
'maneage' branch reference on their remote can be confusing.
With this commit, at the end of the process to merge with the 'maneage'
branch we explicitly recommend to push both the 'master' and 'maneage'
branches.
|
|
Some minor conflicts that came up during the merge were fixed.
|
|
Until now, Maneage only provided the commit hashes (of the project and
Maneage) as LaTeX macros to use in your paper. However, they are too
cryptic and not really human friendly (unless you have access to the Git
history on a computer).
With this commit, to make things easier for the readers, the date of both
commits are also available as LaTeX macros for use in the paper. The date
of the Maneage commit is also included in the acknowledgements.
Also, the paragraph above the acknowledgements has been updated with better
explanation on why adding this acknowledgement in the science papers is
good/necessary.
|
|
This only concerns the TeX sources in the default branch. In case you don't
use them, there should only be a clean conflict in 'paper.tex' (that is
obvious and easy to fix). Conflicts may only happen in some of the
'tex/src/preamble-*.tex' files if you have actually changed them for your
project. But generally any conflict that does arise by this commit with
your project branch should be very clear and easy to fix and test.
In short, from now on things will even be easier: any LaTeX configuration
that you want to do for your project can be done in
'tex/src/preamble-project.tex', so you don't have to worry about any other
LaTeX preamble file. They are either templates (like the ones for PGFPlots
and BibLaTeX) or low-level things directly related to Maneage. Until now,
this distinction wasn't too clear.
Here is a summary of the improvements:
- Two new options to './project make': with '--highlight-new' and
'--highlight-notes' it is now possible to activate highlighting on the
command-line. Until now, there was a LaTeX macro for this at the start
of 'paper.tex' (\highlightchanges). But changing that line would change
the Git commit hash, making it hard for the readers to trust that this
is the same PDF. With these two new run-time options, the printed commit
hash will not changed.
- paper.tex: the sentences are formatted as one sentence per line (and one
line per sentence). This helps in version controlling narrative and
following the changes per sentence. A description of this format (and
its advantages) is also included in the default text.
- The internal Maneage preambles have been modified:
- 'tex/src/preamble-header.tex' and 'tex/src/preamble-style.tex' have
been merged into one preamble file called
'tex/src/preamble-maneage-default-style.tex'. This helps a lot in
simply removing it when you use a journal style file for example.
- Things like the options to highlight parts of the text are now put in
a special 'tex/src/preamble-maneage.tex'. This helps highlight that
these are Maneage-specific features that are independent of the style
used in the paper.
- There is a new 'tex/src/preamble-project.tex' that is the place you
can add your project-specific customizations.
|
|
These can help a first-time reader of 'paper.tex'.
|
|
Until now, the Maneage-only features of LaTeX where mixed with
'tex/src/preamble-project.tex' (which is reserved for project-specific
things). But we want to move the highlighting features (that have started
here) into the core Maneage branch, so its best for these Maneage-specific
features to be in a Maneage-specific preamble file.
With this commit, a hew 'tex/src/preamble-maneage.tex' has been created for
this purpose and the highlighting modes have been put in there. In the
process, I noticed that 'tex/src/preamble-project.tex' doesn't have a
copyright! This has been corrected.
|
|
Roberto sent me his summarized CV which is now being included and I also
removed the extra statements about non-degree things from Raul and my own
biography (like mentioning Gnuastro, and scientific interests). To be
short, we are only mentioning degrees and positions. For Raul, I added his
M.Sc institute.
|
|
After Mohammad-reza sent me his commit on an improved definition for
longevity, we had an indepth discussion (through a video-conference) to
avoid complexities in the terminology, while staying on point and
word-count.
In this commit/merge, I am including the improved version of the definition
of longevity, and the newly added term "functionality" (instead of
"usability" that Mohammad-reza was originally complaining to).
|
|
The paragraph was slightly shortened, while keeping the main points.
|
|
Before this commit, Longetivity was defined on the basis of the term
usability. Although the scope and context of the term has been mentioned
right after its use, this could have caused confusion with the keyword
"usability" in the field of software engineering.
With this commit, Longetivity definition has been rephrased in a way that
it would not require "usability". Furthermore, since longetivity would
logically require the availability of the machines and platforms during the
time of re-use, this has been explicitly mentioned in the definition.
|
|
Until now, we were asking the users of Maneage to cite the first paper that
used its primoridal version (arXiv:1505:01664). But there is now a paper
that fully describes the concept (arXiv:2006.03018).
With this commit, in the 'citation' section of 'README-hacking.md' we now
ask to cite the new paper.
|
|
Following Boud's great suggestion, I also summarized my CV to be less
than 40 words.
|
|
Following Boud's great suggestion, I also summarized my CV to be less than
40 words.
|
|
This commit provides shorter CVs for me (Boud) + David
in order to get closer to the 6500 word limit. Our CVs
are the least significant part of the paper.
|
|
This commit makes the numbered links to references such as [13]
[14] [15] in the appendices clickable in the pdf. The solution
was to call the "\newcites" command from the "multilibs" package
*after* loading "hyperref".
First do "rm -fv .build/tex/build/*.bbl .build/tex/build/*.aux"
and then "./project make" a few times.
|
|
This commit fixes the error of trying to run bibtex on
appendix.tex when the --no-appendix option is selected.
A hardwired hack, appropriate only for this specific paper,
replaces the more-than-three-author parts of two long author
lists by "et al." To test this without having to redownload
the menke file, first do
"rm -fv .build/tex/build/*.aux .build/tex/build/*.bbl"
and then "./project make --no-appendix" a few times.
This commit should reduce the word length by about 70 words.
|
|
The only issue that still remains is how to address statistical
reproducibility, and I am in touch with Boud to do this in the best way
possible (it has been highlighted with '#####'s in the answers.
|
|
There is an answer for all the referee points now. I also did some minor
edits in the paper. But we are still over the limit by around 250 words.
The only remaining point that is not yet addressed (and has '####' around
it) is the discussion on parallelization and its effect on reproducibility.
|
|
This commit is intended to be submittable quality.
Point 56 was removed, and the later points renumbered,
because it was a point of Reviewer 5 described what we
have done - it was not a criticism to respond do. :)
The current word count (without abstract and references)
is 6091.
|
|
This commit only modifies "peer-review/1-answer.txt", giving
answers to Reviewer 4; these mostly take into account David's
email list of proposed answers. No changes are done to
"paper.tex".
|
|
Copyediting of points 16 to 32 (paper.tex +
peer-review/1-answer.txt) is done in this commit.
TODO list:
2. paper lacking focus
9. tidy up README-hacking.md for appearance on website
App B.G. similar to Figure ?? - ref missing
29. website: README-hacking.md and tutorial "on same page"
|
|
This commit updates "paper.tex" and "peer-review/1-answer.txt"
for the first 15 (out of 59!) reviewer points, excluding
points 2 (not yet done) and 9 (README-hacking.md needs
tidying).
A fix to "reproduce/analysis/make/paper.mk" for the
links in the appendices is also done in this commit (the same
algorithm as for paper.tex is added). The links in the appendices
are not (yet) clickable.
|
|
This commit tidies up minor aspects of the language in the text
marked by "\new", e.g. a "wokflow" would be fine for Chinese
cooking, but is a little off-topic for Maneage. :) The word count
is reduced by about 7 words.
I haven't yet got to the serious part: checked that we've responded
to the referees' points, and completing the responses which we
haven't yet done.
|
|
This commit does a minor copyedit of "peer-review/1-answer.txt",
mostly just at the top, plus some hashes to highlight an
unanswered concern; and removes the @ symbols (and full stops)
from email addresses in the peer review email in order to reduce
our feeding of email harvesters (spiders that collect email addresses
for spammers).
|
|
Raul's added point on the answer to the referee was very good, so I edited
it a little to be more clear (and removed his name).
Also, after looking in a few parts of the text, I fixed a few typos.
|
|
With this commit, I make several minor changes to the text of the final
paper. They are not important, but minor modifications like avoiding
contractions (don't -> do not, and so on).
|
|
With this commit, I am just adding several minor corrections to the
answer to the referees. They are very minor typos. I would only
emphasize the fact that in Maneage there is the "Minimal complexity"
criteria, and because of that, even if the project is not able to be
executed in the future, the interested reader could have a look at the
analysis steps (because it is in plain text). Note that I put "Raul" at
the beginning of the line, so my name should have to be removed in the
final document to be sent to the referees.
|
|
A new directory has been added at the top of the project's source called
'peer-review'. The raw reviews of the paper by the editors and referees has
been added there as '1-review.txt'. All the main points raised by the
referees have been listed in a numbered list and addressed (mostly) in
'1-answers.txt'. The text of the paper now also includes all the
implemented answers to the various points.
|
|
Until now, the core Maneage 'paper.tex' had a '\highlightchanges' macro
that defines two LaTeX macros: '\new' and '\tonote'.
When '\highlightchanges' was defined, anything that was written within
'\new' became dark green (highlighting new things that have been
added). Also, anything that was written in '\tonote' was put within a '[]'
and became dark red (to show that there is a note here that should be
addressed later).
When '\highlightchanges' wasn't defined, anything within the '\new' element
would be black (like the rest of the text), and the things in '\tonote'
would not be shown at all.
Commenting the '\newcommand{\highlightchanges}{}' line within 'paper.tex'
(to toggle the modes above) would create a different Git hash and has to be
committed.
But this different commit hash could create a false sense in the reader
that other things have also been changed and the only way they could
confirm was to actually go and look into the project history (which they
will not usually have time to do, and thus won't be able to trust the two
modes of the text).
Also, the added highlights and the note highlights were bundeled together
into one macro, so you couldn't only have one of them.
With this commit, the choice of highlighting either one of the two is now
done as two new run-time options to the './project' script (which are
passed to the Makefiles, and written into the 'project.tex' file which is
loaded into 'paper.tex'). In this way, we can generate two PDFs with the
same Git commit (project's state): one with the selected highlights and
another one without it.
This issue actually came up for me while implementing the changes here: we
need to submit one PDF to the journal/referees with highlights on the added
features. But we also need to submit another PDF to arXiv and Zenodo
without any highlights. If the PDFs have different commit hashes, the
referees may associate it with other changes in any part of the work. For
example https://oadoi.org/10.22541/au.159724632.29528907 that mentions
"Another version of the manuscript was published on arXiv: 2006.03018",
while the only difference was a few words in the abstract after the journal
complained on the abstract word-count of our first submission (where the
commit hashes matched with arXiv/Zenodo).
|
|
With the optional appendices added recently to the paper, it was important
to go through them and make them more fitting into the paper.
|
|
Until now, when the 'pdf-build-final' configuration variable (defined in
'reproduce/analysis/config/pdf-build.conf') was given any string a PDF
would be built. This was very confusing, because people could put a 'no'
and the PDF would still be built!
With this commit, only when this variable has a value of 'yes' will the PDF
be built. If given any other string (or no string at all), it will not
produce a PDF.
This issue was reported by Zahra Sharbaf.
|
|
Until now we had described the basic commands on how to create and use
Docker images, but we hadn't mentioned how you can delete them.
With this commit the commands necessary for deleting Docker images have
also been added at the bottom of the section on Docker.
|
|
Given the referee reports, after discussing with the editors of CiSE, we
decided that it is important to include the complete appendix we had before
that included a thorough review of existing tools and methods. However, the
appendix will not be published in the paper (due to the strict word-count
limit). It will only be used in the arXiv/Zenodo versions of the paper.
This actually created a technical problem: we want the commit hash of the
project source to remain the same when the paper is built with an appendix
or without it.
To fix this problem the choice of including an appendix has gone into the
'project' script as a run-time option called '--no-appendix'. So by default
(when someone just runs './project make'), the PDF will have an appendix,
but when we want to submit to the journal, or when the appendix isn't
needed for a certain reason, we can use this new option. The appendix also
has its own separate bibliography.
Some other corrections made in this commit:
1. Some new references were added that had an '_' in their source, they
were corrected in 'references.tex'.
2. I noticed that 'preamble-style.tex' is not actually used in this paper,
so it has been deleted.
|
|
The LaTeX macro files for these two subMakefiles are created on every run
of './project make'. So their commands are also printed every time and
hardly ever will a normal user want to modify or change these.
So to avoid populating the standard output of a Maneaged project with all
these extra lines every time (possibly getting mixed with the important
analysis or LaTeX outputs), an '@' has been placed at the start of the
recipes. With an '@' at the start of the recipe, Make is instructed to not
print the commands it wants to run in the standard output.
|
|
This commit updates README-hacking.md with the URIs for the 'elaphrocentre'
galaxy formation pipeline paper arXiv:2010.03742. This makes three papers
currently in the peer review pipeline: arXiv:2006.03018, arXiv:2007.11779,
and arXiv:2010.03742, each chronologically corresponding to various stages
of the review process.
|
|
After a fresh build of Maneage with a newly downloaded TeXLive, I noticed
that it is complaining about not finding 'xstring.sty', apparently some
package that depeneded on it is no longer including it itself!
It is thus now added to the packages that are built by Maneage's TeXLive.
|
|
Until now, the core Maneage branch included some configuration files for
Gnuastro's programs. This was actually a remnant of the distant past when
Maneage didn't actually build its own software and we had to rely on the
host's software versions. This file contained the configuration files
specific to Gnuastro for this project and also had a feature to avoid
checking the host's own configuration files.
However, we now build all our software ourselves with fixed configuration
files (for the version that is being installed and its version is
stored). So those extra configuration files were just extra and caused
confusion and problems in some scenarios. With this commit, those extra
files are now removed.
Also, two small issues are also addressed in parallel with this commit:
- When running './project make clean', the 'hardware-parameters.tex' macro
file (which is created by './project configure' is not deleted.
- The project title is now written into the default output's PDF's
properties (through 'hypersetup' in 'tex/src/preamble-header.tex')
through the LaTeX macro.
All these issues were found and fixed with the help of Samane Raji.
|
|
Until now, during the configure step it was checked if the host Operative
System were GNU/Linux, and if not, we assumed it is macOS. However, it can
be any other different OS! With this commit, now we explicity check if the
system is GNU/Linux or Darwin (macOS). If it is not any of them, a warning
message says to the user that the host system is different from which we
have checked so far (and invite to contact us if there is any problem).
In addition to this, if the system is macOS, now it checks if Xcode is
already installed in the host system. If it is not installed, a warning
message informs the user to do that in case a problem/crash in the
configure step occurs. We have found that it is convenient to have Xcode
installed in order to avoid some problems.
|
