aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)AuthorLines
2020-04-13Configure (TeXLive): Year of distribution no longer in directoryMohammad Akhlaghi-13/+11
It is this time of year again: TeXLive has transitioned to its 2020 release and the year is imprinted into the installation directory of TeXLive. Until now, we have had to manually change this year and it caused complications and was very annoying. With this commit, the explicit year has been removed from TeXLive's installation and we now simply put a `maneage' instead of the year. I tried this on another system and it worked nicely. Until the time that we can fully install LaTeX packages from source tarballs, this is the best thing we could do for now.
2020-04-13Implemented Roberto's raised points in the paper's sourceMohammad Akhlaghi-24/+20
Thanks Roberto, they are now implemented.
2020-04-13Minor corrections and thoughtsRoberto Baena Gallé-99/+101
I corrected bugs, typos, double words, and punctuations along the whole text. I do some comments which are always highlighted with \hl{this is my comment}, so you can identify them easily in the pdf. If you want to remove, then you can do it easily with Ctrl+R since I think you never used \hl. Finally, I added my name as coauthor but, please, feel free to remove it if you want. Note from Mohammad: since there were two other suggested commits before this that were already merged, I rembased Roberto's commits and fixed a few minor conflicts.
2020-04-13Imported Pedram's corrections, no conflictsMohammad Akhlaghi-8/+8
There weren't any conflicts in this merge.
2020-04-13Added short line to inform that there are tutorialsRaul Infante-Sainz-0/+1
With this commit, a short line telling that Maneage has tutorials showing the workflow in a practical way has been added. Because of we are near to the limit of words, I have added just a very short line. The sentence does not specify any file name since the tutorial(s) is not included (it will be in a near future).
2020-04-13Adding Julia to acknowledgments for creating the logoRaul Infante-Sainz-0/+1
With this commit, I have included a small line to recognize Julia Aguilar-Cabello as the designer of the Maneage's logo.
2020-04-13Fix typosPedram Ashofteh Ardakani-8/+8
2020-04-13Full existing contents summaried, only discussion to goMohammad Akhlaghi-33/+458
The contents until two commits ago when I started to summarize the paper are now in a new and shorter format: previously the discussion started on page 25, but now it starts on page 17. It is still a little longer than 8000 words, but not as significantly as before. I will add the discussion and also try to summarize it futher before submission.
2020-04-12First draft of first three sections of shortened paper are doneMohammad Akhlaghi-0/+418
As described in the previous commit, we had to shorten the paper to roughly 8000 words (which is significant decrease!). With this commit, I am committing the current version of the summarized paper, where the introduction, defintions and principles have now been summarized. I am now summarizing the rest (describing Maneage and the discussion).
2020-04-12Configure (numpy): added --std=c99 to CFLAGS to fix errorMohammad Akhlaghi-0/+1
Elham Saremi recently reported the following errors when building Numpy in numpy/core/src/npysort/radixsort.c.src: "error: 'for' loop initial declarations are only allowed in C99 or C11 mode". After some searching, I found Issue 14147[1] on Numpy's main repository about the same problem. As described there, apparently Numpy needs C99 compiler, but doesn't check for it or set it manually (for some strange reason, leaving it to the packagers to check if they want!!!). Any way, after a check with Elham, we were able to fix it by adding the `--std=c99' to CFLAGS of Numpy's build and with this commit, it is now being implemented in the core Maneage to not cause a problem in any other project. [1] https://github.com/numpy/numpy/issues/14147
2020-04-10Renamed initial paper to paper-long.tex, to write shorter new versionMohammad Akhlaghi-0/+0
We just recently recognied that the final paper should not be longer than 8000 words. The easiest way was just to start a new `paper.tex' and bring in parts from the original/long version. We can use all the hard work that went into writing the long paper later (possibly in a manual for Maneage). So I don't want to suddenly distroy its history at this point. To let Git know about renaming the original `paper.tex' to `tex/src/paper-long.tex', I am making this commit. This commit doesn't have any `paper.tex' and only records the fact that it has been renamed. In the next commit, I'll re-create `paper.tex' which will host the short/final version. But thanks to this commit, if we later make any changes to long version, Git will know that it was originally the main `paper.tex'.
2020-04-10Acknowledged the help of Pedram, Zahra and SurenaMohammad Akhlaghi-0/+1
Thank you very much guys :-).
2020-04-10Fix spelling errors, suggest alternative wordsPedram Ashofteh Ardakani-135/+135
I tried to get all the words I knew. Some may be correct in different conventions. It definitely needs a second or third review for spell checking. Suggested some additional formatting, including but not limited to using the LaTeX \textsuperscript{} command for stating dates. Also, some unfamiliar rare words that finished with `-able` or `-ability` may need to be changed. Finding better alternatives to better simplify and ease the `readabiliy` ;-) of the paper - I see it's hard not to use them actually. It has got me wondering what better alternatives are available? We'll find out.
2020-04-10Suggest minor changes in the abstractPedram Ashofteh Ardakani-8/+9
to shorten some sentences, fix some spelling/typos, and further simplify some parts. I can see that there are some spelling errors in the rest of the paper. They will be taken care of in the next commit.
2020-04-10Imported Raul's corrections to the paper, minor conflict fixedMohammad Akhlaghi-50/+55
There was only a small conflict in the abstract with Zahra's corrections and that has been fixed.
2020-04-10Minor corrections based on Zahra's suggestionsMohammad Akhlaghi-7/+9
A parenthesis was added to the abstract to hightlight the importance of data lineage for reproducibility. Also, the definitions that Zahra had given for reproducibility were added as comments above the part on defining reproducibility. We'll later decide how to blend them in, if possible.
2020-04-09Minor typos and spelling corrections in Definitions, some notes addedRaul Infante-Sainz-24/+25
With this commit, I have corrected several minor typos in Section 2 (Definitions). I have also put a couple of notes for modify or ignore some phrases.
2020-04-09Minor typos and spelling corrections in IntroductionRaul Infante-Sainz-17/+17
With this commit, minor typos have been corrected in the Introduction section. The majority of them are just small corrections, others are in order to not use contractions ("did not" instead of "didn't" and so on). Other modifications have been added with the aim of remove some small portion of the phrases to make it more focused.
2020-04-09Trying to make the Abstract shorter, keywords sorted alphabeticallyRaul Infante-Sainz-11/+9
With this commit, I have tried to make the Abstract a bit shorter. I think it was too long considering that there are plenty of space in the paper to describe some of the points that were noticed in the abstract. The main point is just to try to be atractive to the reader being focused to the main points. In any case I think there are room for improving it. The keywords have also been sorted alphabetically.
2020-04-09Adding Raul as co-author of the paperRaul Infante-Sainz-1/+2
With this commit, I have added my name as co-author of the paper. Since my affiliation is the same as Mohammad's affiliations, I did not have to add any additional line for that.
2020-04-08Reproducibility in the abstractzahra sharbaf-2/+7
Because one of the most important properties of Manaege is reproducibility. I think is it better to say something about it in the abstract, like the thing that you do in your speech. With this commit, I noted something about it in the abstract.
2020-04-06Astropy now depending on the Expat library to fix internal conflictRaul Infante-Sainz-10/+26
Until now, Astropy was instructed to build its own internal copy of the Expat library. However, with the recent commits before, Maneage now includes an installation of Expat and Astropy can't keep the two (its internal version and the project's version) separate, so they conflict and don't let Astropy get built. With this commit, the problem is fixed by setting the Expat library as an explicit dependency of Astropy and asking Astropy to ignore its internal copy. While doing this, I recognized that it is much easier and elegant to add steps in various stages of the `pybuild' function through hooks instead of variables. So the fifth argument of the `pybuild' function was removed and now it actually checks if hooks are defined as functions and if so, they will be called. The `pyhook_after' function was also implemented in the installation of `pybind11' (which needed it, given that the 5th argument of `pybuild' was removed) and after doing a test-build, I noticed that two lines were not ending with a `\' in `boost' (a dependency of `pybind11'). Commit written originally by Mohammad Akghlaghi
2020-04-06Corrected typo in the definition of pybuildMohammad Akhlaghi-1/+1
Raul noticed this during the build: I had mistakenly put an extra `&&' at the start of the line where the line before ended with a `;'.
2020-04-05Astropy now depending on the Expat library to fix internal conflictMohammad Akhlaghi-9/+25
Until now, Astropy was instructed to build its own internal copy of the Expat library. However, with the recent commits before, Maneage now includes an installation of Expat and Astropy can't keep the two (its internal version and the project's version) separate, so they conflict and don't let Astropy get built. When trying to build Manage (the actual project, not this paper) after applying the commits before there, Raul discovered this problem. With this commit, the problem is fixed by setting the Expat library as an explicit dependency of Astropy and asking Astropy to ignore its internal copy. While doing this, I recognized that it is much easier and elegant to add steps in various stages of the `pybuild' function through hooks instead of variables. So the fifth argument of the `pybuild' function was removed and now it actually checks if hooks are defined as functions and if so, they will be called. The `pyhook_after' function was also implemented in the installation of `pybind11' (which needed it, given that the 5th argument of `pybuild' was removed) and after doing a test-build, I noticed that two lines were not ending with a `\' in `boost' (a dependency of `pybind11').
2020-04-05Commenting version numbers with an underscore for LaTeXRaul Infante-Sainz-1/+3
Until now we would simply return the version numbers as they were written into the separate files and situations can happen where the version numbers contain an underscore (`_'). However, this character is a methematical character in LaTeX, causing LaTeX to complain and abort. With this commit, a step has been added at the end of the configure script to convert any possible `_' to `\_'. Once it is commented (a backslash is put behind it), the underscore will be printed as it is in the final PDF. This commit was originally written by Mohammad Akhlaghi
2020-04-05The build of M4 and XLSX I/O on macOS has been fixedRaul Infante-Sainz-6/+37
Until now, the M4 that was built on macOS had internal problems (as discussed in #1): it would simply print `Abort trap: 6' in the output and abort. After looking at the build of Homebrew, I noticed that they apply a patch (correct one line) to fix this problem. To be able to apply that patch on macOS systems, I had fully open up the build recipe of M4 and atleast on the testing system, it was built successfully. Also, after successfully building M4, and thus Autoconf and thus Minizip, we were able to build XLSX I/O on a macOS and found out that the internal library's full address wasn't being put in the libraries and executables. With this commit, we now use macOS's `install_name_tool' to correct the positions of the two `libxlsxio_*' libraries in all its executables. This commit was originally written by Mohammad Akhlaghi
2020-04-05Including version number of Minizip in its installation targetRaul Infante-Sainz-1/+1
Until this commit, only the word `Minizip' was written into the Minizip installation target (without the version number of Minizip). With this commit, this minor bug has been fixed by using the appropiate Make variable: `$(minizip-version)'.
2020-04-05Building Minizip 1.x instead of Minizip 2.xRaul Infante-Sainz-17/+67
Minizip is a dependency of XLSX I/O and until now, I was just using the most recent version I found (2.9.2), but XLSX I/O is written for the Minizip 1.x series, not 2.x. Somehow it didn't cause a crash on my computer!!! I think XLSX I/O's CMake is instructed to look into system directories by default when it doesn't find the directories in the given places. And because I had installed Minizip on my operating system, it did't complain. Upon trying the build on their systems, Yahya, Raul and Zahra reported a failure in the build of XLSX I/O which was due the to the problem above (we were installing the wrong version of Minizip!). With this commit, this has been fixed by installing the 1.x series of Minizip (whish is actually installed within zlib!). This commit was original done by Mohammad Akhlaghi.
2020-04-04Commenting version numbers with an underscore for LaTeXMohammad Akhlaghi-1/+3
Until now we would simply return the version numbers as they were written into the separate files and situations can happen where the version numbers contain an underscore (`_'). However, this character is a methematical character in LaTeX, causing LaTeX to complain and abort. With this commit, a step has been added at the end of the configure script to convert any possible `_' to `\_'. Once it is commented (a backslash is put behind it), the underscore will be printed as it is in the final PDF.
2020-04-04The build of M4 and XLSX I/O on macOS has been fixedMohammad Akhlaghi-5/+36
Until now, the M4 that was built on macOS had internal problems (as discussed in #1): it would simply print `Abort trap: 6' in the output and abort. After looking at the build of Homebrew, I noticed that they apply a patch (correct one line) to fix this problem. To be able to apply that patch on macOS systems, I had fully open up the build recipe of M4 and atleast on the testing system, it was built successfully. Also, after successfully building M4, and thus Autoconf and thus Minizip, we were able to build XLSX I/O on a macOS and found out that the internal library's full address wasn't being put in the libraries and executables. With this commit, we now use macOS's `install_name_tool' to correct the positions of the two `libxlsxio_*' libraries in all its executables.
2020-04-04Building Minizip 1.x instead of Minizip 2.xMohammad Akhlaghi-17/+68
Minizip is a dependency of XLSX I/O and until now, I was just using the most recent version I found (2.9.2), but XLSX I/O is written for the Minizip 1.x series, not 2.x. Somehow it didn't cause a crash on my computer!!! I think XLSX I/O's CMake is instructed to look into system directories by default when it doesn't find the directories in the given places. And because I had installed Minizip on my operating system, it did't complain. Upon trying the build on their systems, Yahya, Raul and Zahra reported a failure in the build of XLSX I/O which was due the to the problem above (we were installing the wrong version of Minizip!). With this commit, this has been fixed by installing the 1.x series of Minizip (whish is actually installed within zlib!).
2020-04-03CMake updated to version 3.17.0Raul Infante-Sainz-2/+2
With this commit, CMake has been updated to its most recent version. This upgrade has been done because in the installation of XLSX I/O on macOS laptop, it crashes complaining about C compiler "not able to compile a simple test program". After a fast search, I found it could be possible to just use the most recent version of CMake to solve the problem. But it didn't work. In any case, it is good to have the most recent version of CMake included.
2020-04-02Imported recent work on Maneage, minor conflicts fixedMohammad Akhlaghi-783/+937
A few minor conflicts occurred and were fixed.
2020-04-02Rewrote abstract, better organization in publishing sectionMohammad Akhlaghi-26/+175
I hadn't updated the abstract since first writing it. With this commit, it has been updated to be more precise and generically interesting, focusing more on the principles and usability. I also greatly improved the section on publishing the workflow.
2020-04-02Imported two recent minor corrections, no conflictsMohammad Akhlaghi-2/+2
There weren't any conflicts in this merge.
2020-04-01Added a README.md file under the top-level tex/ directoryMohammad Akhlaghi-0/+59
The subdirectories here (and the fact that they may be symbolic links) may be confusing for some early project users, so a `README.md' file was added there describing them and when they are links, when directories and when some may not yet exist.
2020-04-01Corrected reference for Infante-Sainz+2020 in README-hacking.mdRaul Infante-Sainz-1/+1
Until this commit, the year of this paper was 2019 and the linking url was the temporal one. However, the final official publication year is 2020. With this commit, the year and the url have been changed to the final ones.
2020-04-01Removed multiple tabs in MissFITS tarball definitionRaul Infante-Sainz-1/+1
With this commit, multiples tabs in the definition of MissFITS tarball have been removed. Now they are white spaces.
2020-04-01Completed first draft of group working in build directoryMohammad Akhlaghi-4/+21
This section (of sharing a build directory between multiple members of the project) is also a good features of Maneage.
2020-03-31First draft of the version control section and figure completeMohammad Akhlaghi-88/+110
The figure was greatly improved, becoming much more clear and descriptive of some of the main advantages of having version control in a complete project like Maneage.
2020-03-30Section on starting new projects, and publishing project addedMohammad Akhlaghi-19/+242
With the main structure of Maneage explained, I have started to explain how a new project is created, along with a schematic diagram that shows two scenarios of how Git can help with project management.
2020-03-28Cleaned up the introduction, definitions for provenance and lineageMohammad Akhlaghi-118/+219
Until now, the introduction had repeated several things and also had a relatively long list of things to add in its end. Also, it was highly focused to scientific papers. With this commit, I effectively re-wrote it, with the starting paragraphs becoming more industry-friendly, while also focusing on the scientific cases. Many of the repetative parts were removed and the listed items in the end were put into the text in a much better context. Also, now that the name of the system involves "lineage" (and a lot of focus is put on it in the start) the terms data provenance and lineage were defined in the definition section. Some other intersting points that I encountered during the research on definitions were added to the discussion and final lists, and the DOI of one reference paper was corrected.
2020-03-23Analysis and configuration file sections completeMohammad Akhlaghi-89/+348
With this commit a description of these two important parts have been added to the project, along with several figures showing various parts of the files that are discussed. I also done some other restructuring of the figures and files to make things fit better into the the description of the paper.
2020-03-20Adding PyYAML, Html5lib, and Beautifulsoup4 as prerequsites of AstropyRaul Infante-Sainz-4/+4
Until this commit, PyYAML was not set as prerequisite of Astropy. This package is an optional dependency of Astropy for some particular functions. However, we have already included PyYAML into this project so it is available. With this commit, PyYAML has been set as a prerequisite of Astropy. In addition to this, Html5lib and Beautifulsoup4 have been also added as prerequsites of Astropy (and removed from Astroquery prerequisites). I noticed that both of them are optional dependencies of Astropy.
2020-03-17Astroquery updated to version 0.4Raul Infante-Sainz-3/+3
In the last update of Astropy to version 4.0 they removed some things that the previous version of Astroquery needs. As a consequence, it is also necessary to update the Astroquery version to be a ble to run with the Astropy 4.0. With this commit, the update of Astroquery to it most recent version (0.4) has been done.
2020-03-08Menke+20 example: properly count number of papers with softwareMohammad Akhlaghi-5/+17
Until now, I was mistakenly multiplying the fraction of papers in that journal. This is corrected with this commit.
2020-03-08Edited description of example subMakefile for analysis-1Mohammad Akhlaghi-27/+48
In order to make the description more clear and readable, the rules in the demonstrated Makefile (and their links to the data lineage plot) were made more clear.
2020-03-02Described the first analysis phase with a demo subMakefileMohammad Akhlaghi-98/+383
Until now, there was no explanation on an actual analysis phase, therefore with this commit an example scenario with a readable Makefile is included. The Data lineage graph was also simplified to both be more readable, and also to correspond to this new explanation and subMakefile. Some random edits/typos were also corrected and some references added for discussion.
2020-02-29IMPORTANT: re-preparation can only be done with --prepare-redoMohammad Akhlaghi-18/+21
Until now, the preparation phase was always executed before the final build phase when running `./project make'. But when it becomes necessary, project preparation can be slow and will un-necessarily slow down the project while the project is growing (focus is on the analysis that is done after preparation). With this commit, preparation will be done automatically the first time that the project is run (`.build/software/preparation-done.mk' doesn't exist). However, after preperation is complete once, future runs of `./project make' won't do preparation any more (by calling `top-prepare.mk'). They will directly call `top-make.mk' for the analysis. To manually invoke preparation after the first attempt, the `./project make' script should be run with the new `--prepare-redo' option. Also, since the preparation phase is now automatically done before the analysis phase, the long notice that describes running `./project make' at the end of the preparation phase has been removed in `top-prepare.mk'. It now just prints a short line, saying the preparation has been complete. Finally, when the project has not been run with the proper group configuration, it ends with an `exit 1' so the main `./project' script doesn't proceed any further.
2020-02-24MissFITS is now added to the templateSurena Fatemi-1/+28
MissFITS is package for manipulating FITS files. I added it as my first commit to the project for educational purposes.