| Age | Commit message (Collapse) | Author | Lines |
|---|
|
Boud's suggestions in the previous commit were great and really helped in
improving the tone of the abstract (and thus the whole paper shortly!),
better putting it in the big picture. I had forgot to give the exact word
limit (which was 250), so Boud had set it to a very conservative value of
190, I added around 22 words to better highlight the points we want to
make, while still being below the limit.
|
|
To make this a research article, we either have to present it as a
theoretical advance, or as an empirical advance.
An empirical research result would be something like doing a survey of
users and getting statistics of their success/failure in using the system,
and of whether their experience is consistent with the claimed properties
and principles of Maneage (e.g. success/failure in creating paper.pdf as
expected? was the user's system POSIX? did the user do the install with
non-root privileges? was this a with-network or without-network ./project
configure ?) This is doable, but would require a bit of extra work that we
are not necessarily motivated to do or have the time to do right now.
I think it's possible to present Maneage as a theoretical advance, but it
has to be worded properly. Maneage is a tool, but it's a tool that
satisfies what we can reasonably present as a unique theoretical proposal.
Here's my proposed rewrite. I've aimed at minimum word length. I've also
included (commented out) keywords for a structured research abstract -
these are just for us, as a guideline to improve the abstract.
I think "criteria" is safer than "standards". Whether a principle is good
or bad tends to lead to debate. Whether a criterion is satisfied or not is
a more objective question, independent of whether you agree with the
criterion or not.
In the rewrite below, we propose a theoretical standard and show that the
new standard can be satisfied. Maneage is *used as a tool* to prove that
the standard is not too difficult to achieve. Maneage is no longer the
subject of the paper. (That won't change the main body of the paper too
much, apart from compression, but the way it's presented will have to
change, under this proposal.)
The title would need to match this. E.g.
TITLE.1: Evidence that a higher standard of reproducibility criteria is
attainable
TITLE.2: Evidence that a rigorous standard of reproducibility criteria is
attainable
TITLE.3: Towards a more rigorous standard of reproducibility criteria
I would probably go for TITLE.3.
|
|
This abstract is a first step in order to put more focus on the research
aspects of Maneage.
|
|
Given the very strict limits of journals, we needed to remove these
sections and images. The removed images are: the
`figure-file-architecture', `figure-src-topmake' and
`figure-src-inputconf'. In total, with `wc' we now have 9019 words.
This will be futher reduced when we remove all the technical parts of the
Maneage section, in short, we will only describe the generalities, not any
specific details.
|
|
David suggested some interesting references in particular about the
problems with Juypyter notebooks that are now added to the long version of
the paper. We'll later decide if/how they can be used.
|
|
Until now, if GCC couldn't be built for any reason, Maneage would crash and
the user had no way forward. Since GCC is complicated, it may happen and is
frustrating to wait until the bug is fixed. Also, while debugging Maneage,
when we know GCC has no problem, because it takes so long, it discourages
testing.
With this commit, we have re-activated the `--host-cc' option. It was
already defined in the options of `./project', but its affect was nullified
by hard-coding it to zero in the configure script on GNU/Linux systems. So
with this commit that has been removed and the user can use their own C
compiler on a GNU/Linux operating system also.
Furthermore, to inform the user about this option and its usefulness, when
GCC fails to build, a clear warning message is printed, instructing the
user to post the problem as a bug and telling them how to continue building
the project with the `--host-cc' option.
|
|
Until now, at the end of the configuration step, we would tell the user
this: "To change the configuration later, please re-run './project
configure', DO NOT manually edit the relevant files". However, as Boud
suggested in Bug #58243, this is against our principle to encourage users
to modify Maneage.
With this commit, that explanation has been expanded by a few sentences to
tell the users what to change and warn them in case they decide to change
the build-directory.
|
|
Until now Gnuastro and Astropy where installed by default in any clean
build of Maneage. Gnuastro is used to do the demonstration analysis that is
reported in the paper and Astropy was just there to help in testing the
building of the MANY tools it depends on! It (and its dependencies) also
had several papers that helped show software citation.
However, as Boud suggested in task #15619, the burden of installing them
for a new user may be too much and any future changes will cause merge
conflicts. It may also give the impression that Maneage is only/mainly
written for astronomers.
So with this commit, I am removing Astropy as a default target. But we can
only remove Gnuastro after we include an alternative analysis in the
demonstration `delete-me' files. Following Boud's suggestion in that task,
`TARGETS.conf' was also added to the files to be ignored in any future
merge (in the checklist of `README-hacking.mk'). The solution was already
described there, but mainly focused on the deleted `delete-me' files. So
with this commit, I brought out this item as a more prominent item in the
list. Maybe we can later add the analysis done in the Maneage paper (not
yet published).
In terms of testing the software builds, we already have task #15272
(Single target to build all high-level software, for testing) that aims to
have a single configure option to install ALL high-level software and we
can ask people to try if they like and report errors.
|
|
Similar to the previous commit (e43e3291483699), following a change made
yesterday in the identification of software names from their tarballs, a
few other problematic names are corrected with this commit: `apr-util',
HDF5, TeX Live's installation tarball and `rpcsvc-proto'.
Even though we have visually checked the list of software, other
unidentified similar cases may remain and will be fixed when found in
practice.
|
|
Until Commit 3409a54 (from yesterday), pkg-config was found correctly in
`reproduce/software/make/basic.mk` by searching for `pkg`. However, commit
a21ea20 made an improvement in the regular expression for relating package
names and download filenames, and the string `pkg-config` with the new
regex no longer simplifies to `pkg`. The result of this was that the
basic.mk could not find `pkg-config` in the list of packages, since it was
still listed as `pkg`. This blocked downloading for a system without
pkg-config preloaded.
With this commit (of just a few bytes), the bug is fixed.
|
|
Until now, we wouldn't explicity check for GNU gettext. If it was present
on the system, we would just add a link to it in Maneage's installation
directory. However, in bug #58248, Boud noticed that Git (a basic software)
actually needs it to complete its installation. Unfortunately we haven't
had the tiem to include a build of Gettext in Maneage. Because it is mostly
available on many systems, it hasn't been reported too commonly, it also
has many dependencies which make it a little time consuming to install.
So with this commit, we actually check for GNU gettext right after checking
the compiler and if its not available an informative error message is
written to inform the user of the problem, along with suggestions on fixing
it (how to install GNU gettext from their package manager).
|
|
They supported my visit and talk on Maneage at the Barcelona Super
Computing center. They have also offerred to read the paper and are
providing comments.
Also, I noticed that in the author list, we had forgot to put an `,' after
Boud's name. That is also corrected here.
|
|
Until now, the sed script for determining URL download rules in the three
software building Makefiles (`basic.mk', `high-level.mk' and `python.mk')
considered package names such as `fftw-3...` and `fftw2-2.1...` to be
identical. As the example above shows, this would make it hard to include
some software that may hav conflicting non-number names.
With this commit, the SED script that is used to separate the version from
the tarball name only matches numbers that are after a dash
(`-'). Therefore considers `fftw-3...` and `fftw-2...` to be identical, but
`fftw-3-...` and `fftw2-2.1...` to be different. As a result of this
change, the `elif' check for some of the other programs like `m4', or
`help2man' was also corrected in all three Makefiles.
While doing this check on all the software, we noticed that `zlib-version'
is being repeated two times in `version.conf' so it was removed. It caused
no complications, because both were the same number, but could lead to bugs
later.
|
|
Recently (since Commit 7d0c5ef77), the preparation is not run automatically
every time. It is only run automatically the first time and needs to be
manually called with the `--prepare-redo' option. But this wasn't explained
in `README-hacking.md' (currently the main documentation of Maneage).
With this commit, a description about invoking the preparation process
after the first attempt of the running project has been added to
`README-hacking.md'.
|
|
Recently (in Commit 8eb0892e) the Gnuastro configuration files moved under
"reproduce/analysis/config/gnuastro" directory (before that they were in
`reproduce/software/config/gnuastro)'. But this hadn't been reflected in it
the variable that defines this directory in `initialize.mk'.
With this commit, the address of the Gnuastro configuration files directory
is corrected, allowing Gnuastro programs to operate properly when it is
used.
|
|
Until now, the comment in the file said that setting the `verify-outputs`
variable to `yes` disables the verification. Looking at
`reproduce/analysis/make/verify.mk` shows that the opposite is true.
With this commit, the word `disable` is replaced with `enable` so that the
user is not confused by the conflict between the source code in the other
file and this comment.
|
|
Until now we only checked for the existance and write-ability of the build
directory. But we recently discovered that if the specified build-directory
is in a non-POSIX compatible partition (for example NTFS), permissions
can't be modified and this can cause crashs in some programs (in
particular, while building Perl, see [1]). The thing that makes this
problem hard to identify is that on such partitions, `chmod' will still
return 0 (so it was hard to find).
With this commit, a check has been added after the user specifies the
build-directory. If the proposed build directory is not able to handle
permissions as expected, the configure script will not continue and will
let the user know and will ask them for another directory.
Also, the two printed characters at the start of error messages were
changed to `**' (instead of `--'). When everything is good, we'll use `--'
to tell the user that their given directory will be used as the build
directory. And since there are multiple checks now, the final message to
specify a new build directory is now moved to the end and not repeated in
every check.
[1] https://savannah.nongnu.org/support/?110220
|
|
Until now, we were using GitLab as the main Git repository of Maneage. But
today I finally setup our own Git repository under `git.maneage.org' and
enabled a CGit web interface for a simple and fast viewing of the commits
and changes.
Since this URL is under our own control, we can always ensure that it will
point to somewhere meaningful, on any server so in the long-run its much
better than publishing the paper an explicit reliance of `gitlab.com'.
|
|
Until now, the primary Maneage URLs were under GitLab, but since we now
have a dedicated URL and Git repository, its better to transfer to this as
soon as possible. Therefore with this commit, throughout Maneage, any place
that Maneage was referenced through GitLab has been corrected.
Please correct your project's remote to point to the new repository at
`git.maneage.org/project.git', and please make sure it follows the
`maneage' branch. There is no more `master' branch on Maneage.
|
|
|
|
Reading over Boud's edits, I noticed a few other parts that I could
summarize more and corrected one or two other parts to fit the original
purpose of the sentence better.
|
|
Reduction by about 5 words.
Although it's true that the low-level tools - make, bash, gcc -
are still being actively developed, only expert users will tend
to notice the differences, and in this context, it's probably
more useful to point out that these are actively *maintained*.
(Comment: I felt that the first sentence in the Conclusion is
missing one of the obvious criteria for handling big data -
citizen control so that big data could hopefully become less
Orwellian than it is right now, with GAFAM having the main big
data databases that are used by AI researchers and will tend to
affect people's lives more than traditional "scientific"
databases. But there's no point adding this here, since the
criteria that tend to satisfy the scientific requirements
("principles") and citizens' rights tend to overlap to a fair
degree...)
|
|
Reduction of about 50 words.
There were a couple of expressions that look a bit like
some sort of software/research analysis jargon, such as
`Research Objects`, `Software Heritage`, `Machine actionable`.
Unless these are defined, capitalising them makes the reader
assume that there is some well-known formal meaning and that
s/he has to search for that him/herself. As lower case
expressions, the reader can guess some reasonable meanings
of these.
The word "embargo" was introduced for proposal 2) to handle
the third caveat.
|
|
[Compared to first submission to DSJ last week with 11436 words in raw PDF,
we have decreased the paper by ~1000 words to 10493 :-)]
As with the previous commits, the moment Boud changed the structure of
sentences, I was able to find the redundancies and remove them! This is a
fascinating feature of collaboration I had never felt before: it is so hard
to find redundancies in my own raw text, but even a minor correction by
someone else suddeny breaks my mental memories/barrier on the sentence,
allowing me to be more critical to it!
Anyway, besides such corrections, I fixed a few other things: 1) In the
DSJ's recently published papers, ther is no `~' between "Figure" and its
number. 2) I noticed that in `tex/src/figure-src-inputconf.tex' I was
actually using manually input strings for the filename, checksum and size!
This was contrary to the whole philosophy of Maneage(!), I must have rushed
and forgot! So LaTeX variables are now defined and used.
|
|
About 20 words less. The ArXiv URL is added - this adds no extra
length in words, and some readers will not be familiar with ArXiv
(although the COVID-19 pandemic has attracted attention to BiorXiv).
|
|
Increase by 5 words. We don't need to give a big warning here,
but "Permissions management" is meant to be a brief way of saying
that whether or not different users can really read/write/execute
in subdirectories will firstly depend on whether the user who
cloned Maneage has handled these permissions correctly and whether
s/he is able to allow others to edit in his/her subdirectories.
Comment: Users would have to check who else is logged in at the time,
who else is running jobs, and so on. On a supercomputer this might
make sense, to avoid unnecessary recompiles. Anyway, this edit
summary is not the place to discuss this...
|
|
Reduction by 15 words. "Branch" is fine as a verb, and "off"
is fine as a preposition; there's no need for a second preposition.
"We branched off the main forest path onto a smaller path".
|
|
Length reduction by about 15 words.
A semantically significant change is from `leading to more robust
scientific results` to `evolves in the case of exploratory
research papers, and better self-consistency in hypothesis
testing papers`.
I said this in a previous commit, but it can't hurt repeating:
In the covidian epoch (though not only), it is especially
important to distinguish bayesian type exploratory research
(typical in astronomy or searching for a good COVID-19 treatment
or vaccine) from hypothesis testing (clinical testing in
double-blind random access trials with clinical trials methods
published on a public registry prior to the trials taking
place). In the latter case, you want your results to be analysed
consistently with the plan published before the trials even
begin, and ideally you want them to be published (or at least
posted on the trial registry website) even if your results are
insignificant, to avoid a publication bias in favour of
significant results. Test homeopathy against placebos in 1000
independent experiments, analyse them all the same way, and 2-3
experiments will be significant at the 3 sigma level...
|
|
Reduction by about 7 words.
I added "internet security" as an extra reason for having all the
downloads in a single file. Modularity and minimal complexity in
themselves generally contribute to internet security, but in this
case, it's obvious that having all the communication with the
outside world managed through a single file makes internet security
management much simpler.
I replaced the "fake URL" by the real one, because at least in the
present format, the URL fits in nicely. So both `paper.tex` and
`tex/src/figure-src-inputconf.tex` are modified in this commit.
|
|
Reduction by about 20 words - minor rewording.
|
|
Idafen has helped in testing Maneage a lot during the last year and has
provided very useful feedback and suggestions.
|
|
Regarding Docker Konrad pointed out that "Linux has an excellent track
record for stability. It's more likely that the Docker itself becomes
incompatible with older containers. Docker isn't developed for
reproducibility after all".
So I tried to modify that paragraph to include this important point too. In
the process, I also shrank it a little more (without loosing anything
substantial), so it doesn't add to the paper's length.
|
|
After going through Boud's corrections, I thought it can be further
summarized without loosing any major point.
|
|
Reduction by 4 words. Minor rewording; removal of "Note that"
and "simply" (the opposite of "complicatedly"). If a checksum
is simple for a given user, then s/he already knows that; if s/he
doesn't yet know what a checksum is, then stating that it's simple
doesn't help very much. :)
|
|
Reduction of about 15 words.
The phrase "which does not need it" is removed. On its own, this is
a claim, not an explanation. If the reader is wondering why `paper.tex`
is not a produced file, then stating that the file is not needed will
not help very much. Looking at the diagram will show that `paper.tex`
is the overall article template; and the diagram strongly suggests
that values from initialize.tex, ..., are passed into verify.tex,
and from there into project.tex, which goes into paper.tex.
The phrase "files, possibly in another subMakefile" should really
be something like "files, possibly created by another subMakefile".
But this would add more words, and given that the user has full control
to modify and adapt the overall scheme (including making a mess of it),
we can safely drop the info that the scheme can be made more complicated. :)
|
|
Only 3 words are reduced in this commit, but I think the
improvements are worth it.
"Note that" and "It is worth mentioning" are phrases still quite
often used by academics (even in astronomy) that can be politely
described as "pontification" or informally as "empty blabla";
these add no meaning except "I am teaching you something and I
expect you to pay attention to what I am saying". :) There are
also less polite descriptions.
|
|
Minor rewording of 4.3 Project analysis - introduction.
Reduction of about 40 words.
4.2 `parallel` quote: s/http:/https:/
|
|
Today Konrad made the following suggestions after reading through the paper
(created from Commit 1ac5c12). Thanks a lot Konrad ;-). I tried to address
them all in this commit. Afterwards, while looking over the corrected
parts, some minor edits came up to me to remove redundant parts and add
extra points where it helps.
In particular to be able to print the International Phonetic Alphabet
(IPA), I had to include the LaTeX `TIPA' package, but it was interesting to
see that it was already available in the project as a dependency of another
package we loaded.
|
|
Tags are not a fixed piece of history (they can easily be moved and not
imported in a different repository), so they are only confusing in the
context of Maneage (where people should branch-off the main project). the
raw commit hashes are a much more robust way to store a precise moment in
history.
Before this commit, I removed all Tags from the main Git repositories of
Maneage and thus removed any mention of Tags with
`README-hacking.md'. Ofcourse, if a project decides to use tags is upto
them, but we won't implement it in the main branch.
|
|
Roberto Baena recently tried building a new project with Maneage and
provided the following suggestions to make it more clear for a new user:
1) In the part where we talk about creating a Git repository, we should
highlight that it must be empty. This is because some (for example Gitlab)
propose to include a `README' file. But if the project is not empty, Git
will not allow pushing to it.
2) The `(can be done later)' comment was removed from the "Delete dummy
parts") to avoid confusion about applying some of them, but not others: if
only some are done, it may cause problems in the build.
|
|
Until now, the message that we printed just before starting to build
software didn't actually print the current directory, but only `pwd'. With
this commit, this is fixed (it uses the `currentdir' variable that is
already found before).
|
|
Until now, the message that we printed just before starting to build
software didn't actually print the current directory, but only `pwd'. With
this commit, this is fixed (it uses the `currentdir' variable that is
already found before).
|
|
We recently fixed the problem of TeXLive that hard-codes the year of its
build in its installation directory. But the note on this problem was still
kept in `README-hacking.md'. That part is now removed.
Also, to help in following the checklist, it is now an ordered list.
|
|
Boud previously pointed out that that he couldn't find a reference to the
citation, so I added it as a link over "its FAQ" (since its described in
its `doc/citation-notice-faq.txt' file). I also removed the first part of
the quote which was not really necessary, the heart of the quote is the
latter part that still remains.
|
|
I tried to make it slightly shorter, but I felt that it is important to
keep the quote from GNU Parallel and in particular the financial aid it
asks for. It will help readers feel the gravity of the sitution for this
software author. The precise citation of the quote was given in the long
version.
|
|
This reduces the length by about 70 words.
The biggest change is to remove what looks like a citation from
`parallel'. I couldn't find the citation in GNU parallel
20161222-1 (Debian/stretch), nor with search engines.
I don't think that the quote is really so useful (even assuming
it's a valid quote from somewhere): citation practices are a mix
between ethics, preparation to convince referees, citing those who
are already cited frequently, and the practicality of searching for
and verifying references against the information for which they
are used. Showing that Maneage makes citation not only easy, but
more or less automatic, bypasses some of the compromises between
practicality and ethics.
|
|
Minor rewording; a reduction of about 12 words.
|
|
Minor edits - reduces about 17 words.
|
|
This commit reduces about 25 words from the 4.1 Maneage
orchestration, aka `make`, section.
|
|
This drops the word count in the introductory part of the Maneage
section by about 15 words.
|
