aboutsummaryrefslogtreecommitdiff
path: root/README.md
AgeCommit message (Collapse)AuthorLines
2022-01-21IMPORTANT: Updates to almost all softwareMohammad Akhlaghi-58/+174
This commit primarily affects the configuration step of Maneage'd projects, and in particular, updated versions of the many of the software (see P.S.). So it shouldn't affect your high-level analysis other than the version bumps of the software you use (and the software's possibly improve/changed behavior). The following software (and thus their dependencies) couldn't be updated as described below: - Cryptography: isn't building because it depends on a new setuptools-rust package that has problems (https://savannah.nongnu.org/bugs/index.php?61731), so it has been commented in 'versions.conf'. - SecretStorage: because it depends on Cryptography. - Keyring: because it depends on SecretStorage. - Astroquery: because it depends on Keyring. This is a "squashed" commit after rebasing a development branch of 60 commits corresponding to a roughly two-month time interval. The following people contributed to this branch. - Boudewijn Roukema added all the R software infrastructure and the R packages, as well as greatly helping in fixing many bugs during the update. - Raul Infante-Sainz helped in testing and debugging the build. - Pedram Ashofteh Ardakani found and fixed a bug. - Zahra Sharbaf helped in testing and found several bugs. Below a description of the most noteworthy points is given. - Software tarballs: all updated software now have a unified format tarball (ustar; if not possible, pax) and unified compression (Lzip) in Maneage's software repository in Zenodo (https://doi.org/10.5281/zenodo.3883409). For more on this See https://savannah.nongnu.org/task/?15699 . This won't affect any extra software you would like to add; you can use any format recognized by GNU Tar, and all common compression algorithms. This new requirement is only for software that get merged to the core Maneage branch. - Metastore (and thus libbsd and libmd) moved to highlevel: Metastore (and the packages it depends on) is a high-level product that is only relevant during the project development (like Emacs!): when the user wants the file meta data (like dates) to be unchanged after checking out branches. So it should be considered a high-level software, not basic. Metastore also usually causes many more headaches and error messages, so personally, I have stopped using it! Instead I simply merge my branches in a separate clone, then pull the merge commit: in this way, the files of my project aren't re-written during the checkout phase and therefore their dates are untouched (which can conflict with Make's dates on configuration files). - The un-official cloned version of Flex (2.6.4-91 until this commit) was causing problems in the building of Netpbm, so with this commit, it has been moved back to version 2.6.4. - Netpbm's official page had version 10.73.38 as the latest stable tarball that was just released in late 2021. But I couldn't find our previously-used version 10.86.99 anywhere (to see when it was released and why we used it! Its at last more than one year old!). So the official stable version is being used now. - Improved instructions in 'README.md' for building software environment in a Docker container (while having project source and output data products on the local system; including the usage of the host's '/dev/shm' to speed up temporary operations). - Until now, the convention in Maneage was to put eight SPACE characters before the comment lines within recipes. This was done because by default GNU Emacs (also many other editors) show a TAB as eight characters. However, in other text editors, online browsers, or even the Git diff, a TAB can correspond to a different number of characters. In such cases, the Maneage recipes wouldn't look too interesting (the comments and the recipe commands would show a different indentation!). With this commit, all the comment lines in the Makefiles within the core Maneage branch have a hash ('#') as their first character and a TAB as the second. This allows the comment lines in recipes to have the same indentation as code; making the code much more easier to read in a general scenario including a 'git diff' (editor agnostic!). P.S. List of updated software with their old and new versions - Software with no version update are not mentioned. - The old version of newly added software are shown with '--'. Name (Basic) Old version New version ------------ ----------- ----------- Bzip2 1.0.6 1.0.8 CURL 7.71.1 7.79.1 Dash 0.5.10.2 0.5.11.5 File 5.39 5.41 Flock 0.2.3 0.4.0 GNU Bash 5.0.18 5.1.8 GNU Binutils 2.35 2.37 GNU Coreutils 8.32 9.0 GNU GCC 10.2.0 11.2.0 GNU M4 1.4.18 1.4.19 GNU Readline 8.0 8.1.1 GNU Tar 1.32 1.34 GNU Texinfo 6.7 6.8 GNU diffutils 3.7 3.8 GNU findutils 4.7.0 4.8.0 GNU gmp 6.2.0 6.2.1 GNU grep 3.4 3.7 GNU gzip 1.10 1.11 GNU libunistring 0.9.10 1.0 GNU mpc 1.1.0 1.2.1 GNU mpfr 4.0.2 4.1.0 GNU nano 5.2 6.0 GNU ncurses 6.2 6.3 GNU wget 1.20.3 1.21.2 Git 2.28.0 2.34.0 Less 563 590 Libxml2 2.9.9 2.9.12 Lzip 1.22-rc2 1.22 OpenSLL 1.1.1a 3.0.0 Patchelf 0.10 0.13 Perl 5.32.0 5.34.0 Podlators -- 4.14 Name (Highlevel) Old version New version ---------------- ----------- ----------- Apachelog4cxx 0.10.0-603 0.12.1 Astrometry.net 0.80 0.85 Boost 1.73.0 1.77.0 CFITSIO 3.48 4.0.0 Cmake 3.18.1 3.21.4 Eigen 3.3.7 3.4.0 Expat 2.2.9 2.4.1 FFTW 3.3.8 3.3.10 Flex 2.6.4-91 2.6.4 Fontconfig 2.13.1 2.13.94 Freetype 2.10.2 2.11.0 GNU Astronomy Utilities 0.12 0.16.1-e0f1 GNU Autoconf 2.69.200-babc 2.71 GNU Automake 1.16.2 1.16.5 GNU Bison 3.7 3.8.2 GNU Emacs 27.1 27.2 GNU GDB 9.2 11.1 GNU GSL 2.6 2.7 GNU Help2man 1.47.11 1.48.5 Ghostscript 9.52 9.55.0 ICU -- 70.1 ImageMagick 7.0.8-67 7.1.0-13 Libbsd 0.10.0 0.11.3 Libffi 3.2.1 3.4.2 Libgit2 1.0.1 1.3.0 Libidn 1.36 1.38 Libjpeg 9b 9d Libmd -- 1.0.4 Libtiff 4.0.10 4.3.0 Libx11 1.6.9 1.7.2 Libxt 1.2.0 1.2.1 Netpbm 10.86.99 10.73.38 OpenBLAS 0.3.10 0.3.18 OpenMPI 4.0.4 4.1.1 Pixman 0.38.0 0.40.0 Python 3.8.5 3.10.0 R 4.0.2 4.1.2 SWIG 3.0.12 4.0.2 Util-linux 2.35 2.37.2 Util-macros 1.19.2 1.19.3 Valgrind 3.15.0 3.18.1 WCSLIB 7.3 7.7 Xcb-proto 1.14 1.14.1 Xorgproto 2020.1 2021.5 Name (Python) Old version New version ------------- ----------- ----------- Astropy 4.0 5.0 Beautifulsoup4 4.7.1 4.10.0 Beniget -- 0.4.1 Cffi 1.12.2 1.15.0 Cryptography 2.6.1 36.0.1 Cycler 0.10.0 0.11.0+} Cython 0.29.21 0.29.24 Esutil 0.6.4 0.6.9 Extension-helpers -- 0.1 Galsim 2.2.1 2.3.3 Gast -- 0.5.3 Jinja2 -- 3.0.3 MPI4py 3.0.3 3.1.3 Markupsafe -- 2.0.1 Numpy 1.19.1 1.21.3 Packaging -- 21.3 Pillow -- 8.4.0 Ply -- 3.11 Pyerfa -- 2.0.0.1 Pyparsing 2.3.1 3.0.4 Pythran -- 0.11.0 Scipy 1.5.2 1.7.3 Setuptools 41.6.0 58.3.0 Six 1.12.0 1.16.0 Uncertainties 3.1.2 3.1.6 Wheel -- 0.37.0 Name (R) Old version New version -------- ----------- ----------- Cli -- 2.5.0 Colorspace -- 2.0-1 Cowplot -- 1.1.1 Crayon -- 1.4.1 Digest -- 0.6.27 Ellipsis -- 0.3.2 Fansi -- 0.5.0 Farver -- 2.1.0 Ggplot2 -- 3.3.4 Glue -- 1.4.2 GridExtra -- 2.3 Gtable -- 0.3.0 Isoband -- 0.2.4 Labeling -- 0.4.2 Lifecycle -- 1.0.0 Magrittr -- 2.0.1 MASS -- 7.3-54 Mgcv -- 1.8-36 Munsell -- 0.5.0 Pillar -- 1.6.1 R-Pkgconfig -- 2.0.3 R6 -- 2.5.0 RColorBrewer -- 1.1-2 Rlang -- 0.4.11 Scales -- 1.1.1 Tibble -- 3.1.2 Utf8 -- 1.2.1 Vctrs -- 0.3.8 ViridisLite -- 0.4.0 Withr -- 2.4.2
2021-04-25README.md: edited steps to only build software env in Docker imageMohammad Akhlaghi-75/+149
Until now, while the series of steps mentioned in 'README.md' were complete, they had some implicit thing in them that made it a little hard to run as a checklist (the commands to do some basic things weren't included). Also, it was recommending to run a long 'docker run ...' command, which wasn't too user friendly. With this commit, the series of steps is now a complete checklist, containing every step. Also, the checklist now recommends putting the long 'docker run' command inside a script called 'docker-run' that will also do a 'sudo' internally (thus making things very easy for a first-time user). Also, since the 'docker-run' script contains host OS-specific directory names, it should not be under control, so it has been added to the '.gitignore' file in case users decide to keep this same name (which is recommended).
2021-01-09IMPORTANT: analysis outputs written in BDIR/analysisMohammad Akhlaghi-12/+133
Until now, the build directory contained a 'software/' directory (that hosted all the built software), a 'tex/' subdirectory for the final building of the paper, and many other directories containing intermediate/final data of the specific project. But this mixing of built software and data is against our modularity and minimal complexity principles: built software and built data are separate things and keeping them separate will enable many optimizations. With this commit, the build directory of the core Maneage branch will only contain two sub-directories: 'software/' and 'analysis/'. The 'software/' directory has the same contents as before and is not touched in this commit. However, the 'analysis/' directory is new and everything created in the './project make' phase of the project will be created inside of this directory. To facilitate easy access to these top-level built directories, two new variables are defined at the top of 'initialize.mk': 'badir', which is short for "built-analysis directory" and 'bsdir', which is short for "built-software directory". HOW TO IMPLEMENT THIS CHANGE IN YOUR PROJECT. It is easy: simply replace all occurances of '$(BDIR)' in your project's subMakefiles (except the ones below) to '$(badir)'. To confirm if everything is fine before building your project from scratch after merging, you can run the following command to see where 'BDIR' is used and confirm the only remaning cases. $ grep -r BDIR reproduce/analysis/* --> make/verify.mk: innobdir=$$(echo $$infile | sed -e's|$(BDIR)/||g'); \ --> make/initialize.mk:badir=$(BDIR)/analysis --> make/initialize.mk:bsdir=$(BDIR)/software --> make/initialize.mk: $$sys_rm -rf $(BDIR) --> make/top-prepare.mk:all: $(BDIR)/software/preparation-done.mk 'BDIR' should only be present in lines of the files above. If you see '$(BDIR)' used anywhere else, simply change it to '$(badir)'. Ofcourse, if your project assumes BDIR in other contexts, feel free to keep it, it will not conflict. If anything un-expected happens, please post a comment on the link below (you need to be registered on Savannah to post a comment): https://savannah.nongnu.org/task/?15855 One consequence of this change is that the 'analysis/' subdirectory can be optionally mounted on a separate partition. The need for this actually came up for some new users of Maneage in a Docker image. Docker can fix portability problems on systems that we haven't yet supported (even Windows!), or had a chance to fix low-level issues on. However, Docker doesn't have a GUI interface. So to see the built PDF or intermediate data, it was necessary to copy the built data to the host system after every change, which is annoying during working on a project. It would also need two copies of the source: one in the host, one in the container. All these frustrations can be fixed with this new feature. To describe this scenario, README.md now has a new section titled "Only software environment in the Docker image". It explains step-by-step how you can make a Docker image to only host the built software environment. While your project's source, software tarballs and 'BDIR/analysis' directories are on your host operating system. It has been tested before this commit and works very nicely.
2021-01-04README.md: summary Dockerfile with all necessary lines in one stepMohammad Akhlaghi-0/+26
Until now, the description in 'README.md' to build the Dockerfile in 'README.md' had one item per line, thoroughly describing the reason behind that line. But in many cases, the user is already familiar with Docker (or has already read through the items) and just wants to have the Dockerfile ready fast. In these cases, all those extra explanations are annoying. With this commit, an item '0' has been added at the start of the item list for summary. It only contains the necessary Dockerfile contents with no extra explanation.
2021-01-02Copyright year updated in all source filesMohammad Akhlaghi-1/+1
Having entered 2021, it was necessary to update the copyright years at the top of the source files. We recommend that you do this for all your project-specific source files also.
2020-11-13README.md: added commands to delete all Docker imagesMohammad Akhlaghi-0/+12
Until now we had described the basic commands on how to create and use Docker images, but we hadn't mentioned how you can delete them. With this commit the commands necessary for deleting Docker images have also been added at the bottom of the section on Docker.
2020-08-28Edited README.md to remove installation of a text editorMohammad Akhlaghi-10/+7
With the previous commit, we now build Nano by default within Maneage, and project authors can ask to install Emacs and Vim within 'TARGETS.conf'. So in the instructions to build within a Docker image have been removed.
2020-08-25README.md: added explanation on copying files from Docker imageMohammad Akhlaghi-2/+17
When building Maneage inside a Docker container, in the end the users want to extract the final outputs from the container into their host operating system to inspect more comfortably. So with this commit, a short examplanation has been added on how to do this. We also noticed that it is much better if the 'Dockerfile' is stored and run in an empty directory, otherwise, it will start parsing the full directory and its subdirectories as the docker image's environment.
2020-07-17README.md now has description of building project in DockerMohammad Akhlaghi-0/+218
Docker is a "container" technology that allows an almost independent operating system run on the host. It is useful when the host OS doesn't support some features or has internal problems (for example its C library or C compiler have problems). Fortunately a Maneaged project can easily be built within a Docker image and a minimal image operating system. With this commit, a section has been added to 'README.md' to describe this process. Each step of the Dockerfile is explined, to help users that may not be too familiar with Docker, or help Docker user who are not familiar with Maneage.
2020-06-06IMPORTANT: Added publication checklist, improved relevant infrastructureMohammad Akhlaghi-58/+90
Possible semantic conflicts (that may not show up as Git conflicts but may cause a crash in your project after the merge): 1) The project title (and other basic metadata) should be set in 'reproduce/analysis/conf/metadata.conf'. Please include this file in your merge (if it is ignored because of '.gitattributes'!). 2) Consider importing the changes in 'initialize.mk' and 'verify.mk' (if you have added all analysis Makefiles to the '.gitattributes' file (thus not merging any change in them with your branch). For example with this command: git diff master...maneage -- reproduce/analysis/make/initialize.mk 3) The old 'verify-txt-no-comments-leading-space' function has been replaced by 'verify-txt-no-comments-no-space'. The new function will also remove all white-space characters between the columns (not just white space characters at the start of the line). Thus the resulting check won't involve spacing between columns. A common set of steps are always necessary to prepare a project for publication. Until now, we would simply look at previous submissions and try to follow them, but that was prone to errors and could cause confusion. The internal infrastructure also didn't have some useful features to make good publication possible. Now that the submission of a paper fully devoted to the founding criteria of Maneage is complete (arXiv:2006.03018), it was time to formalize the necessary steps for easier submission of a project using Maneage and implement some low-level features that can make things easier. With this commit a first draft of the publication checklist has been added to 'README-hacking.md', it was tested in the submission of arXiv:2006.03018 and zenodo.3872248. To help guide users on implementing the good practices for output datasets, the outputs of the default project shown in the paper now use the new features). After reading the checklist, please inspect these. Some other relevant changes in this commit: - The publication involves a copy of the necessary software tarballs. Hence a new target ('dist-software') was also added to package all the project's software tarballs in one tarball for easy distribution. - A new 'dist-lzip' target has been defined for those who want to distribute an Lzip-compressed tarball. - The '\includetikz' LaTeX macro now has a second argument to allow configuring the '\includegraphics' call when the plot should not be built, but just imported.
2020-05-22Corrected copyright notices to fit GPL suggested formatMohammad Akhlaghi-1/+1
In time, some of the copyright license description had been mistakenly shortened to two paragraphs instead of the original three that is recommended in the GPL. With this commit, they are corrected to be exactly in the same three paragraph format suggested by GPL. The following files also didn't have a copyright notice, so one was added for them: reproduce/software/make/README.md reproduce/software/bibtex/healpix.tex reproduce/analysis/config/delete-me-num.conf reproduce/analysis/config/verify-outputs.conf
2020-04-25IMPORTANT: Primary Maneage repositories are now under maneage.orgMohammad Akhlaghi-2/+2
Until now, the primary Maneage URLs were under GitLab, but since we now have a dedicated URL and Git repository, its better to transfer to this as soon as possible. Therefore with this commit, throughout Maneage, any place that Maneage was referenced through GitLab has been corrected. Please correct your project's remote to point to the new repository at `git.maneage.org/project.git', and please make sure it follows the `maneage' branch. There is no more `master' branch on Maneage.
2020-04-20Maneage instead of Template in README-hacking.md and copyright noticesMohammad Akhlaghi-9/+6
Until now, throughout Maneage we were using the old name of "Reproducible Paper Template". But we have finally decided to use Maneage, so to avoid confusion, the name has been corrected in `README-hacking.md' and also in the copyright notices. Note also that in `README-hacking.md', the main Maneage branch is now called `maneage', and the main Git remote has been changed to `https://gitlab.com/maneage/project' (this is a new GitLab Group that I have setup for all Maneage-related projects). In this repository there is only one `maneage' branch to avoid complications with the `master' branch of the projects using Maneage later.
2020-01-23IMPORTANT: Project preparation is now also done with project makeMohammad Akhlaghi-15/+1
Until now, the main commands to run the project were these: `./project configure' (to build the software), `./project prepare' (to possibly arrange input datasets and build special configuration Makefiles) and finally `./project make' to run the project. The main logic behind the "prepare" phase `top-prepare.mk' is to build configuration files that can be fed into the "make" step and optimize its operation. For example when the total number of necessary inputs for the majority of the analysis is not as large as the total number of inputs. With "prepare" (when necessary), you go through the raw inputs, select the ones that are necessary for the rest of the project. The output of `top-prepare.mk' is a configuration file (a Make variable) that keeps the IDs (numbers, names, etc). That configuration file would then be used in the `top-make.mk' to identify the lower level targets and allow optimal project organization and management. But the last two are both part of the analysis, and while they indeed need different calls to Make to be executed, many projects don't actually need a preparation phase: ultimately, its an implementation choice by the project developers and doesn't concern the project users (or the developers when they are running it). To avoid confusing the users, or simply annoying them when a projet doesn't need it, with this commit, the top-level `top-prepare.mk' and `top-make.mk' Makefiles are called with the single `./project make' command and `./project prepare' has been dropped. I noticed this while writing the paper on this system.
2020-01-01Copyright statements updated to include 2020Mohammad Akhlaghi-1/+1
Now that its 2020, its necessary to include this year in the copyright statements.
2019-10-01Preparation phase added before final buildingMohammad Akhlaghi-5/+18
In many real-world scenarios, `./project make' can really benefit from having some basic information about the data before being run. For example when quering a server. If we know how many datasets were downloaded and their general properties, it can greatly optmize the process when we are designing the solution to be run in `./project make'. Therefore with this commit, a new phase has been added to the template's design: `./project prepare'. In the raw template this is empty, because the simple analysis done in the template doesn't warrant it. But everything is ready for projects using the template to add preparation phases prior to the analysis.
2019-09-28Main project script sets executable flags at configure timeMohammad Akhlaghi-3/+3
Until now, when the project's source was downloaded from something like arXiv, in `README.md', we were instructing them to set the executable flags of all the files that need it. But except for `./project', the reader shouldn't have to worry about the project internals! Once its executable, `./project' can easily fix the executable flags of all the files that need it automatically. With this commit, in `README.md', we just instruct the reader to set the executable flag of `./project' and any other file that needs an executable flag is given one at the start of the set of commands for `./project configure'. In customized projects, if an author needs executable flags on any other files, they can easily add it there without involving the user.
2019-09-27Changing to the cloned directory added to README.mdMohammad Akhlaghi-0/+1
Konrad Hinsen pointed out that this part was missing from the instructions in `README.md' after cloning. So it is added.
2019-09-26Minor edits/clarifications in README.mdMohammad Akhlaghi-13/+7
The two modifications to the LaTeX source of an arXiv-downloaded source weren't rendered properly on Gitlab, so they are corrected to be in the same line and not have a separate code-block.
2019-09-26Working project when downloaded from arXivMohammad Akhlaghi-1/+77
Until now, we were assuming that the users would just clone the project in Git. But after submitting arXiv:1909.11230, and trying to build directly from the arXiv source, I noticed several problems that wouldn't allow users to build it automatically. So I tried the build step by step and was able to find a fix for the several issues that came up. The scripting parts of the fix were primarily related to the fact that the unpacked arXiv tarball isn't under version control, so some checks had to be put there. Also, we wanted to make it easy to remove the extra files, so an extra `--clean-texdit' option was added to `./project'. Finally, some manual corrections were necessary (prior to running `./project', which are now described in `README.md'. Most of the later steps can be automated and we should do it later, I just don't have enough time now.
2019-09-18README.md written to be more generic and easy to customizeMohammad Akhlaghi-37/+38
Until now customizing it was a little more detailed, for example the copyright statement wasn't generic and was about "this template". So the user would have to correct it. With this commit, the copyright statment just says "this project", so it can apply to the raw template and also any customization of it. Also, some minor edits were made in the various parts of the text to make it more clear.
2019-08-28Minor cosmectic markdown corrections in README.mdMohammad Akhlaghi-2/+2
The Copyright year is now on a separate line (by adding a backslash), and the `file-metadata' is now enclosed in two "`" characters to show differently after rendering.
2019-07-28Single wrapper instead of old ./configure, Makefile and ./for-groupMohammad Akhlaghi-6/+6
Until now, to work on a project, it was necessary to `./configure' it and build the software. Then we had to run `.local/bin/make' to run the project and do the analysis every time. If the project was a shared project between many users on a large server, it was necessary to call the `./for-group' script. This way of managing the project had a major problem: since the user directly called the lower-level `./configure' or `.local/bin/make' it was not possible to provide high-level control (for example limiting the environment variables). This was especially noticed recently with a bug that was related to environment variables (bug #56682). With this commit, this problem is solved using a single script called `project' in the top directory. To configure and build the project, users can now run these commands: $ ./project configure $ ./project make To work on the project with other users in a group these commands can be used: $ ./project configure --group=GROUPNAME $ ./project make --group=GROUPNAME The old options to both configure and make the project are still valid. Run `./project --help' to see a list. For example: $ ./project configure -e --host-cc $ ./project make -j8 The old `configure' script has been moved to `reproduce/software/bash/configure.sh' and is called by the new `./project' script. The `./project' script now just manages the options, then passes control to the `configure.sh' script. For the "make" step, it also reads the options, then calls Make. So in the lower-level nothing has changed. Only the `./project' script is now the single/direct user interface of the project. On a parallel note: as part of bug #56682, we also found out that on some macOS systems, the `DYLD_LIBRARY_PATH' environment variable has to be set to blank. This is no problem because RPATH is automatically set in macOS and the executables and libraries contain the absolute address of the libraries they should link with. But having `DYLD_LIBRARY_PATH' can conflict with some low-level system libraries and cause very hard to debug linking errors (like that reported in the bug report). This fixes bug #56682.
2019-04-14Replaced all occurances of pipeline in textMohammad Akhlaghi-11/+11
All occurances of "pipeline" have been chanaged to "project" or "template" withint the text (comments, READMEs, and comments) of the template. The main template branch is now also named `template'. This was all because `pipeline' is too generic and couldn't be distinguished from the base, and customized project.
2019-04-14Corrected copyright information for .file-metadataMohammad Akhlaghi-1/+2
Since `.file-metadata' is a binary file, we can't include a copyright inside of it so we have to use `README.md' to mention its copyright and license notice. However, this was not done clearly and is now corrected.
2019-04-13Corrected copyright notices and info about adding copyright infoMohammad Akhlaghi-4/+2
Until now, the files where the people were meant to change didn't have a proper copyright notice (for example `Copyright (C) YOUR NAME.'). This was wrong because the license does not convey copyright ownership. So the name of the file's original author must always be included and when people modify it (and add their own copyright-able modifications). With this commit, the file's original author (and email) are added to the copyright notice and when more than one person modified a file, both names have their individual copyright notice. Based on this, the description for adding a copyright notice in `README-hacking.md' has also been modified.
2019-04-11.file-metadata also given a copyright in top READMEMohammad Akhlaghi-3/+5
Since `.file-metadata' is a binary file and we couldn't put a copyright notice within it, it has been mentioned in `README.md' to have the same copyright. Also, the copyright modification step in `README-hacking.md' was brought to a later step to be more clear that it should always be done (on new files or files that are changed).
2019-04-07Copyright notice added to all files missing oneMohammad Akhlaghi-3/+6
Until now, for short files, we only had a license notice, not an actual copyright notice. With this commit, a copyright notice has also been added. We use this new command to find these files, suggested by `ineiev@gnu.org'.
2019-03-29Added Copyright to all TeX and README filesMohammad Akhlaghi-0/+25
In order to be more clear, a copyright statement was added to all the LaTeX and README files.
2019-01-23README-pipeline.md is now called README-hacking.mdMohammad Akhlaghi-1/+1
To be more generic and recognizable, the `README-pipeline.md' script was renamed to `README-hacking.md'. In essence, it is just that: to hack the existing pipeline for your own project. We follow a similar naming convention in many GNU software.
2019-01-17README-pipeline.md referenced in README.mdMohammad Akhlaghi-1/+3
Until now, there was no reference to `README-pipeline.md' within the `README.md' file. Since `README.md' is the first file that someone reads and the basic perpose and structure of the pipeline is described in `README-pipeline.md', it was necessary to bring it up there.
2018-12-06Edited README.md to show example dependency repositoryMohammad Akhlaghi-3/+3
To help and be more clear a link to this pipeline's dependency repository has been added to `README.md'.
2018-12-05Updated README.mdMohammad Akhlaghi-15/+15
The README.md file was updated to reflect recent changes in the pipeline (especially regarding the downloader).
2018-11-22Spell check in two READMEsMohammad Akhlaghi-2/+2
A spellcheck was run on the two README files.
2018-11-22Minor edit/correction in README.mdMohammad Akhlaghi-9/+8
The note to the pipeline designers was corrected to display properly on Gitlab.
2018-11-22Placeholder in README.md for pipeline dependenciesMohammad Akhlaghi-2/+8
A placeholder link is now used in `README.md' to encourage the pipeline designers to keep a backup of all the dependencies they use.
2018-11-22Top level READMEs renamed to be similar to actual projectMohammad Akhlaghi-944/+49
Until now, were were advising the users to rename the two README files after cloning the project. This was because online Git browsers usually display the `README.md' file, so we wanted the description of the pipeline to be visible in the pipeline, and later when a project adopts it, they can have their own `README.md'. But the problem is that any change in `REAME.md' will later cause conflicts with a project's `README.md'. So we are now using the same naming convention as the papers that use the pipeline.
2018-11-22Checklist defining remote moved to topMohammad Akhlaghi-19/+26
In the checklist, we are now defining the remote host of the repository at an early stage. This is because we will need it in the `README.md' file (which now has a placeholder `XXXXXXX' instead of a valid URL).
2018-11-22Using .local instead of ./.local in READMEsMohammad Akhlaghi-6/+6
Until now, in the instructions, we were suggesting to run `./.local/bin/make', but the `./' part is extra: this is already a directory and so the shell will be able to find it. So to make things more clear and easy to read/write, we removed the `./' part from the calls to our custom Make installation.
2018-11-21Changing of README files in checklistMohammad Akhlaghi-19/+25
When you point to this project, the `README.md' file is the default file that opens on GitLab and other online git repositories. Since a reproduction pipeline project is different from the actual pipeline, its best for the default text that opens to describe the paper, not the pipeline. The old `README.md' is also kept, but its now called `REAME-pipeline.md'.
2018-11-21Pulling into pipeline branch instead of fetching in README.mdMohammad Akhlaghi-13/+15
In the previous commit, we were recommending to fetch the work from this pipeline. But since we have a separate `pipeline' branch, we can simply checkout to that branch and pull all the recent changes. So with this commit, the steps to get recent updates to the pipeline are updated.
2018-11-21Fetching pipeline updates explained in README.md checklistMohammad Akhlaghi-7/+26
Since working on the pipeline will evolve along with the projects that use it, it can be useful for projects to fetch updates in the pipeline. So the checklist in `README.md' updated to explain how to do this cleanly.
2018-11-21Updated description of Make in README.mdMohammad Akhlaghi-7/+13
Until now, because we didn't build the dependencies internally, it was important for the pipeline to be usable with any version of Make. But because of the new installation of dependencies (including GNU Make), that is no longer the case. So we can safely use GNU Make and this needs to be mentioned in `README.md'.
2018-11-20GNU Coreutils now built in basic dependenciesMohammad Akhlaghi-4/+6
GNU Coreutils are basic programs that can help in the configuration of higher-level programs. Because of that, it was a dependency of almost all software built in `dependencies.mk'. To make things more clear, easier to read and faster (when building in parallel), the building of Coreutils is now moved to the `dependencies-basic.mk' rules. There, it is built along-side Bash. Since `dependenceis-basic.mk' is run and completed before `dependencies.mk', with this, we can be sure that Coreutils is present by the time we want to build the higher-level programs. Also, Zlib is now added as a dependency of Git also (it is necessary for its build).
2018-11-19Minor corrections for easy applying of checklistMohammad Akhlaghi-26/+30
After going through the checklist for starting a new project based on the pipeline, I noticed some parts that could be modified to be more clear. They are now applied.
2018-11-19Gzip's tarball in tar.gz instead of tar.lzMohammad Akhlaghi-25/+38
Until now, we were using a customized `tar.lz' tarball for Gzip. But on systems that don't have GNU Tar, this will cause a problem (non-GNU Tar doesn't recognize `.tar.lz'). So to keep things simple, we are using the customized gzip in `tar.gz' format. After the internal build of GNU Tar and Lzip, the default method of unpacking (`tar xf XXXXX.tar.XX') will work nicely on all the standard compression algorithms and we don't have to modify our commands based on the algorithm (nice feature of GNU Tar).
2018-11-18Updated README and README.md for new dependency building featuresMohammad Akhlaghi-3/+12
The two README files have been updated to explain the new feature of downloading and building dependencies.
2018-11-14All used software with versions, acknowledgedMohammad Akhlaghi-3/+4
All the used software are now acknowledged in the template paper along with their versions. This section is also mentioned in the check list, so users don't delete it by mistake.
2018-11-13Version of programs checked on each run of pipelineMohammad Akhlaghi-9/+3
The version of all programs is now checked in `reproduce/make/src/initialize.mk' and the pipeline won't complete if any of the program versions change from those listed in `reproduce/config/pipeline/dependency-versions.mk'. Since the pipeline is systematically checking all program versions, we don't need Gnuastro's `--onlyversion' option any more. So it (and all references to it) have been removed.
2018-11-12Dependencies built at the start of the pipelineMohammad Akhlaghi-157/+161
To enable easy/proper reproduction of results, all the high-level dependencies are now built within the pipeline and installed in a fixed directory that is added to the PATH of the Makefile. This includes GNU Bash and GNU Make, which are then used to run the pipeline. The `./configure' script will first build Bash and Make within itself, then it will build All the dependencies are also built to be static. So after they are built, changing of the system's low-level libraries (like C library) won't change the tarballs. Currently the C library and C compiler aren't built within the pipeline, but we'll hopefully add them to the build process also. With this change, we now have full control of the shell and Make that will be used in the pipeline, so we can safely remove some of the generalities we had before.
generated by cgit v1.2.1 (git 2.18.0) at 2024-04-28 10:21:21 +0000