| Age | Commit message (Collapse) | Author | Lines |
|---|
|
TeXLive recently transitioned from its 2019 version to its 2020 version
thanks to Elham Saremi's trial of the this project. The fact that
traditionally Maneage installs all TeXLive packages in a per-year directory
is very annoying and required an update in the core Maneage system every
year. So I suddently recognized that we can fix this by setting a different
name for the directory holding the release year. This has been implemented
with this commit.
I have also done this change in the main Maneage branch for other projects
to also benefit from this correction.
|
|
Thanks Roberto, they are now implemented.
|
|
I corrected bugs, typos, double words, and punctuations along the whole
text. I do some comments which are always highlighted with \hl{this is my
comment}, so you can identify them easily in the pdf. If you want to
remove, then you can do it easily with Ctrl+R since I think you never used
\hl. Finally, I added my name as coauthor but, please, feel free to remove
it if you want.
Note from Mohammad: since there were two other suggested commits before
this that were already merged, I rembased Roberto's commits and fixed a few
minor conflicts.
|
|
There weren't any conflicts in this merge.
|
|
With this commit, a short line telling that Maneage has tutorials
showing the workflow in a practical way has been added. Because of we
are near to the limit of words, I have added just a very short line. The
sentence does not specify any file name since the tutorial(s) is not
included (it will be in a near future).
|
|
With this commit, I have included a small line to recognize Julia
Aguilar-Cabello as the designer of the Maneage's logo.
|
|
|
|
The contents until two commits ago when I started to summarize the paper
are now in a new and shorter format: previously the discussion started on
page 25, but now it starts on page 17. It is still a little longer than
8000 words, but not as significantly as before. I will add the discussion
and also try to summarize it futher before submission.
|
|
As described in the previous commit, we had to shorten the paper to roughly
8000 words (which is significant decrease!). With this commit, I am
committing the current version of the summarized paper, where the
introduction, defintions and principles have now been summarized. I am now
summarizing the rest (describing Maneage and the discussion).
|
|
We just recently recognied that the final paper should not be longer than
8000 words. The easiest way was just to start a new `paper.tex' and bring
in parts from the original/long version. We can use all the hard work that
went into writing the long paper later (possibly in a manual for
Maneage). So I don't want to suddenly distroy its history at this point.
To let Git know about renaming the original `paper.tex' to
`tex/src/paper-long.tex', I am making this commit. This commit doesn't have
any `paper.tex' and only records the fact that it has been renamed. In the
next commit, I'll re-create `paper.tex' which will host the short/final
version. But thanks to this commit, if we later make any changes to long
version, Git will know that it was originally the main `paper.tex'.
|
|
Thank you very much guys :-).
|
|
I tried to get all the words I knew. Some may be correct in different
conventions. It definitely needs a second or third review for spell
checking.
Suggested some additional formatting, including but not limited to
using the LaTeX \textsuperscript{} command for stating dates.
Also, some unfamiliar rare words that finished with `-able` or
`-ability` may need to be changed. Finding better alternatives to
better simplify and ease the `readabiliy` ;-) of the paper - I see
it's hard not to use them actually. It has got me wondering what
better alternatives are available? We'll find out.
|
|
to shorten some sentences, fix some spelling/typos, and further
simplify some parts.
I can see that there are some spelling errors in the rest of the paper.
They will be taken care of in the next commit.
|
|
There was only a small conflict in the abstract with Zahra's corrections
and that has been fixed.
|
|
A parenthesis was added to the abstract to hightlight the importance of
data lineage for reproducibility. Also, the definitions that Zahra had
given for reproducibility were added as comments above the part on defining
reproducibility. We'll later decide how to blend them in, if possible.
|
|
With this commit, I have corrected several minor typos in Section 2
(Definitions). I have also put a couple of notes for modify or ignore
some phrases.
|
|
With this commit, minor typos have been corrected in the Introduction
section. The majority of them are just small corrections, others are in
order to not use contractions ("did not" instead of "didn't" and so on).
Other modifications have been added with the aim of remove some small
portion of the phrases to make it more focused.
|
|
With this commit, I have tried to make the Abstract a bit shorter. I
think it was too long considering that there are plenty of space in the
paper to describe some of the points that were noticed in the abstract.
The main point is just to try to be atractive to the reader being
focused to the main points. In any case I think there are room for
improving it. The keywords have also been sorted alphabetically.
|
|
With this commit, I have added my name as co-author of the paper. Since
my affiliation is the same as Mohammad's affiliations, I did not have to
add any additional line for that.
|
|
Because one of the most important properties of Manaege is
reproducibility. I think is it better to say something about it in the
abstract, like the thing that you do in your speech.
With this commit, I noted something about it in the abstract.
|
|
Raul noticed this during the build: I had mistakenly put an extra `&&' at
the start of the line where the line before ended with a `;'.
|
|
Until now, Astropy was instructed to build its own internal copy of the
Expat library. However, with the recent commits before, Maneage now
includes an installation of Expat and Astropy can't keep the two (its
internal version and the project's version) separate, so they conflict and
don't let Astropy get built.
When trying to build Manage (the actual project, not this paper) after
applying the commits before there, Raul discovered this problem.
With this commit, the problem is fixed by setting the Expat library as an
explicit dependency of Astropy and asking Astropy to ignore its internal
copy.
While doing this, I recognized that it is much easier and elegant to add
steps in various stages of the `pybuild' function through hooks instead of
variables. So the fifth argument of the `pybuild' function was removed and
now it actually checks if hooks are defined as functions and if so, they
will be called.
The `pyhook_after' function was also implemented in the installation of
`pybind11' (which needed it, given that the 5th argument of `pybuild' was
removed) and after doing a test-build, I noticed that two lines were not
ending with a `\' in `boost' (a dependency of `pybind11').
|
|
Until now we would simply return the version numbers as they were written
into the separate files and situations can happen where the version numbers
contain an underscore (`_'). However, this character is a methematical
character in LaTeX, causing LaTeX to complain and abort.
With this commit, a step has been added at the end of the configure script
to convert any possible `_' to `\_'. Once it is commented (a backslash is
put behind it), the underscore will be printed as it is in the final PDF.
|
|
Until now, the M4 that was built on macOS had internal problems (as
discussed in #1): it would simply print `Abort trap: 6' in the output and
abort. After looking at the build of Homebrew, I noticed that they apply a
patch (correct one line) to fix this problem. To be able to apply that
patch on macOS systems, I had fully open up the build recipe of M4 and
atleast on the testing system, it was built successfully.
Also, after successfully building M4, and thus Autoconf and thus Minizip,
we were able to build XLSX I/O on a macOS and found out that the internal
library's full address wasn't being put in the libraries and
executables. With this commit, we now use macOS's `install_name_tool' to
correct the positions of the two `libxlsxio_*' libraries in all its
executables.
|
|
Minizip is a dependency of XLSX I/O and until now, I was just using the
most recent version I found (2.9.2), but XLSX I/O is written for the
Minizip 1.x series, not 2.x. Somehow it didn't cause a crash on my
computer!!! I think XLSX I/O's CMake is instructed to look into system
directories by default when it doesn't find the directories in the given
places. And because I had installed Minizip on my operating system, it
did't complain.
Upon trying the build on their systems, Yahya, Raul and Zahra reported a
failure in the build of XLSX I/O which was due the to the problem above (we
were installing the wrong version of Minizip!).
With this commit, this has been fixed by installing the 1.x series of
Minizip (whish is actually installed within zlib!).
|
|
A few minor conflicts occurred and were fixed.
|
|
I hadn't updated the abstract since first writing it. With this commit, it
has been updated to be more precise and generically interesting, focusing
more on the principles and usability. I also greatly improved the section
on publishing the workflow.
|
|
There weren't any conflicts in this merge.
|
|
The subdirectories here (and the fact that they may be symbolic links) may
be confusing for some early project users, so a `README.md' file was added
there describing them and when they are links, when directories and when
some may not yet exist.
|
|
Until this commit, the year of this paper was 2019 and the linking url
was the temporal one. However, the final official publication year is
2020. With this commit, the year and the url have been changed to the
final ones.
|
|
With this commit, multiples tabs in the definition of MissFITS tarball
have been removed. Now they are white spaces.
|
|
This section (of sharing a build directory between multiple members of the
project) is also a good features of Maneage.
|
|
The figure was greatly improved, becoming much more clear and descriptive
of some of the main advantages of having version control in a complete
project like Maneage.
|
|
With the main structure of Maneage explained, I have started to explain how
a new project is created, along with a schematic diagram that shows two
scenarios of how Git can help with project management.
|
|
Until now, the introduction had repeated several things and also had a
relatively long list of things to add in its end. Also, it was highly
focused to scientific papers.
With this commit, I effectively re-wrote it, with the starting paragraphs
becoming more industry-friendly, while also focusing on the scientific
cases. Many of the repetative parts were removed and the listed items in
the end were put into the text in a much better context.
Also, now that the name of the system involves "lineage" (and a lot of
focus is put on it in the start) the terms data provenance and lineage were
defined in the definition section.
Some other intersting points that I encountered during the research on
definitions were added to the discussion and final lists, and the DOI of
one reference paper was corrected.
|
|
With this commit a description of these two important parts have been added
to the project, along with several figures showing various parts of the
files that are discussed. I also done some other restructuring of the
figures and files to make things fit better into the the description of the
paper.
|
|
Until this commit, PyYAML was not set as prerequisite of Astropy. This
package is an optional dependency of Astropy for some particular
functions. However, we have already included PyYAML into this project so
it is available. With this commit, PyYAML has been set as a prerequisite
of Astropy.
In addition to this, Html5lib and Beautifulsoup4 have been also added as
prerequsites of Astropy (and removed from Astroquery prerequisites). I
noticed that both of them are optional dependencies of Astropy.
|
|
In the last update of Astropy to version 4.0 they removed some things
that the previous version of Astroquery needs. As a consequence, it is
also necessary to update the Astroquery version to be a ble to run with
the Astropy 4.0. With this commit, the update of Astroquery to it most
recent version (0.4) has been done.
|
|
Until now, I was mistakenly multiplying the fraction of papers in that
journal. This is corrected with this commit.
|
|
In order to make the description more clear and readable, the rules in the
demonstrated Makefile (and their links to the data lineage plot) were made
more clear.
|
|
Until now, there was no explanation on an actual analysis phase, therefore
with this commit an example scenario with a readable Makefile is included.
The Data lineage graph was also simplified to both be more readable, and
also to correspond to this new explanation and subMakefile.
Some random edits/typos were also corrected and some references added for
discussion.
|
|
Until now, the preparation phase was always executed before the final build
phase when running `./project make'. But when it becomes necessary, project
preparation can be slow and will un-necessarily slow down the project while
the project is growing (focus is on the analysis that is done after
preparation).
With this commit, preparation will be done automatically the first time
that the project is run (`.build/software/preparation-done.mk' doesn't
exist). However, after preperation is complete once, future runs of
`./project make' won't do preparation any more (by calling
`top-prepare.mk'). They will directly call `top-make.mk' for the analysis.
To manually invoke preparation after the first attempt, the `./project
make' script should be run with the new `--prepare-redo' option.
Also, since the preparation phase is now automatically done before the
analysis phase, the long notice that describes running `./project make' at
the end of the preparation phase has been removed in `top-prepare.mk'. It
now just prints a short line, saying the preparation has been complete.
Finally, when the project has not been run with the proper group
configuration, it ends with an `exit 1' so the main `./project' script
doesn't proceed any further.
|
|
MissFITS is package for manipulating FITS files.
I added it as my first commit to the project for educational
purposes.
|
|
Until now, the final preparation target of the preparation phase depended
on all the `$(makesrc)' files. This caused a problem because we were
telling it to also depend on `prepare.tex' (which is the same file that is
being built).
With this commit, we are applying the same solution we have already done in
`paper.mk' (for `paper.tex'): we are removing `prepare' from the list of
prerequisites.
This bug was found by Zahra Sharbaf.
|
|
Until now, like all software on GNU/Linux systems GCC would be built in RAM
(to speed up the build slightly and also not put too much stress on the
HDDs/SSDs). But some systems don't have enough RAM for building GCC and
will complain and crash.
With this commit, we have added a check on the amount of free space in the
`build_tmp' directory (which will be `/dev/shm' on GNU/Linux systems). If
the amount of free space isn't more than 10GB, then GCC won't be built
there and a temporary directory will be built under the `$(BDIR)/software'
directory for it.
This bug was found by Zahra Sharbaf.
This fixes bug #57853.
|
|
I had forgot to add a `./' before the call to `project' for the
`--check-config'.
|
|
This was done just to get going with describing the analysis process.
|
|
The main problems with this dataset was the names of the journals (which
sometimes have single quotes or apostrophes in them that is really annoying
for SED)! But ultimately, for the simple study we want to do here, the
journal names are irrelevant, so in the end I just ignored the names. Later
we can set an identifier for the journals if necessary.
But now we have the basic information in a way that is usable in a plot to
show in this paper.
|
|
Until now, there was no easy way to read/write `.xlsx' files (Microsoft
Excel spreadsheets) within the template. But XLSX I/O provides to simple
programs and some libraries to easily convert `.xlsx' files to CSV that can
easily be read by any tool.
This has also been implemented in the core template branch.
|
|
XLSX I/O is a very simple and fast program and library for reading and
writing `.xls' and `.xlsx' files (mainly used by Microsoft Excel) to CSV
files. It has two separate executables that can be called for an Excel file
and will output a CSV plain text file that can then be used within the
pipeline with more standard tools.
|
