diff options
Diffstat (limited to 'README-hacking.md')
| -rw-r--r-- | README-hacking.md | 134 |
1 files changed, 69 insertions, 65 deletions
diff --git a/README-hacking.md b/README-hacking.md index e42bf42..b721d6d 100644 --- a/README-hacking.md +++ b/README-hacking.md @@ -1,8 +1,8 @@ Maneage: managing data lineage ============================== -Copyright (C) 2018-2021 Mohammad Akhlaghi <mohammad@akhlaghi.org>\ -Copyright (C) 2020-2021 Raul Infante-Sainz <infantesainz@gmail.com>\ +Copyright (C) 2018-2022 Mohammad Akhlaghi <mohammad@akhlaghi.org>\ +Copyright (C) 2020-2022 Raul Infante-Sainz <infantesainz@gmail.com>\ See the end of the file for license conditions. Maneage is a **fully working template** for doing reproducible research (or @@ -180,29 +180,44 @@ evolving rapidly, so some details will differ between the different versions. The more recent papers will tend to be the most useful as good working examples. - - Peper & Roukema ([2020](https://arxiv.org/abs/2010.03742), - arXiv:2010.03742): The live version of the controlled source is [at + - Borkowska & Roukema + ([2022](https://ui.adsabs.harvard.edu/abs/2021arXiv211214174B), MNRAS + Submitted, arXiv:2112.14174): The live version of the controlled source + is [at Codeberg](https://codeberg.org/boud/gevcurvtest); the main input + dataset, a software snapshot, the software tarballs, the project outputs + and editing history are available at + [zenodo.5806027](https://doi.org/10.5281/zenodo.5806027); and the + archived git history is available at [swh:1:rev:54398b720ddbac269ede30bf1e27fe27f07567f7](https://archive.softwareheritage.org/browse/revision/54398b720ddbac269ede30bf1e27fe27f07567f7). + + - Peper & Roukema + ([2021](https://ui.adsabs.harvard.edu/abs/2021MNRAS.505.1223P), MNRAS, + 505, 1223, DOI:10.1093/mnras/stab1342, arXiv:2010.03742): The live + version of the controlled source is [at Codeberg](https://codeberg.org/boud/elaphrocentre); the main input - dataset, a software snapshot, the software tarballs, the project + dataset, a software snapshot, the software tarballs, the project outputs + and editing history are available at + [zenodo.4699702](https://zenodo.org/record/4699702); and the archived + git history is available at + [swh:1:rev:a029edd32d5cd41dbdac145189d9b1a08421114e](https://archive.softwareheritage.org/swh:1:rev:a029edd32d5cd41dbdac145189d9b1a08421114e). + + - Roukema ([2021](https://ui.adsabs.harvard.edu/abs/2021PeerJ...911856R), + PeerJ, 9:e11856, arXiv:2007.11779): The live version of the controlled + source is [at Codeberg](https://codeberg.org/boud/subpoisson); the main + input dataset, a software snapshot, the software tarballs, the project outputs and editing history are available at - [zenodo.4062461](https://zenodo.org/record/4062461); and the - archived git history is available at - [swh:1:dir:c4770e81288f340083dd8aa9fe017103c4eaf476](https://archive.softwareheritage.org/swh:1:dir:c4770e81288f340083dd8aa9fe017103c4eaf476). - - - Roukema ([2020](https://arxiv.org/abs/2007.11779), - arXiv:2007.11779): The live version of the controlled source is [at - Codeberg](https://codeberg.org/boud/subpoisson); the main input - dataset, a software snapshot, the software tarballs, the project - outputs and editing history are available at - [zenodo.3951152](https://zenodo.org/record/3951152); and the - archived git history is available at - [swh:1:dir:fcc9d6b111e319e51af88502fe6b233dc78d5166](https://archive.softwareheritage.org/swh:1:dir:fcc9d6b111e319e51af88502fe6b233dc78d5166). - - - Akhlaghi et al. ([2020](https://arxiv.org/abs/2006.03018), - arXiv:2006.03018): The project's version controlled source is [on + [zenodo.4765705](https://zenodo.org/record/4765705); and the archived + git history is available at + [swh:1:rev:72242ca8eade9659031ea00394a30e0cc5cc1c37](https://archive.softwareheritage.org/swh:1:rev:72242ca8eade9659031ea00394a30e0cc5cc1c37). + + - Akhlaghi et + al. ([2021](https://ui.adsabs.harvard.edu/abs/2021CSE....23c..82A), + CiSE, 23(3), 82 DOI:10.1109/MCSE.2021.3072860 arXiv:2006.03018): The + project's version controlled source is [on Gitlab](https://gitlab.com/makhlaghi/maneage-paper), necessary software, - outputs and backup of history is available in - [zenodo.3872248](https://doi.org/10.5281/zenodo.3872248). + outputs and backup of history are available at + [zenodo.3872248](https://doi.org/10.5281/zenodo.3872248); and the + archived git history is available at + [swh:1:dir:45a9e282a86145fe9babef529c8fce52ffe8d717](https://archive.softwareheritage.org/swh:1:dir:45a9e282a86145fe9babef529c8fce52ffe8d717). - Infante-Sainz et al. ([2020](https://ui.adsabs.harvard.edu/abs/2020MNRAS.491.5317I), @@ -212,8 +227,8 @@ working examples. [zenodo.3524937](https://zenodo.org/record/3524937). - Akhlaghi ([2019](https://arxiv.org/abs/1909.11230), IAU Symposium - 355). The version controlled project source is available [on - GitLab](https://gitlab.com/makhlaghi/iau-symposium-355) and is also + 355). The version controlled project source is available + [on GitLab](https://gitlab.com/makhlaghi/iau-symposium-355) and is also archived on Zenodo with all the necessary software tarballs: [zenodo.3408481](https://doi.org/10.5281/zenodo.3408481). @@ -553,7 +568,7 @@ First custom commit the default `origin` remote server to specify that this is Maneage's remote server. This will allow you to use the conventional `origin` name for your own project as shown in the next steps. Second, you will - create and go into the conventional `master` branch to start + create and go into the conventional `main` branch to start committing in your project later. ```shell @@ -561,7 +576,7 @@ First custom commit $ mv project my-project # Change the name to your project's name. $ cd my-project # Go into the cloned directory. $ git remote rename origin origin-maneage # Rename current/only remote to "origin-maneage". - $ git checkout -b master # Create and enter your own "master" branch. + $ git checkout -b main # Create and enter your own "main" branch. $ pwd # Just to confirm where you are. ``` @@ -616,7 +631,7 @@ First custom commit a new project which is bad in this scenario, and will not allow you to push to it). It will give you a URL (usually starting with `git@` and ending in `.git`), put this URL in place of `XXXXXXXXXX` in the first - command below. With the second command, "push" your `master` branch to + command below. With the second command, "push" your `main` branch to your `origin` remote, and (with the `--set-upstream` option) set them to track/follow each other. However, the `maneage` branch is currently tracking/following your `origin-maneage` remote (automatically set @@ -627,7 +642,7 @@ First custom commit ```shell git remote add origin XXXXXXXXXX # Newly created repo is now called 'origin'. - git push --set-upstream origin master # Push 'master' branch to 'origin' (with tracking). + git push --set-upstream origin main # Push 'main' branch to 'origin' (with tracking). git push origin maneage # Push 'maneage' branch to 'origin' (no tracking). ``` @@ -635,7 +650,7 @@ First custom commit your name (with your possible coauthors) and tentative abstract in `paper.tex`. You should see the relevant place in the preamble (prior to `\begin{document}`. Just note that some core project metadata like - the project tile are actually set in + the project title are actually set in `reproduce/analysis/config/metadata.conf`. So set your project title in there. After you are done, run the `./project make` command again to see your changes in the final PDF and make sure that your changes @@ -681,13 +696,14 @@ First custom commit $ rm reproduce/analysis/config/delete-me* ``` - - Disable verification of outputs by removing the `yes` from - `reproduce/analysis/config/verify-outputs.conf`. Later, when you are - ready to submit your paper, or publish the dataset, activate - verification and make the proper corrections in this file (described - under the "Other basic customizations" section below). This is a - critical step and only takes a few minutes when your project is - finished. So DON'T FORGET to activate it in the end. + - `reproduce/analysis/config/verify-outputs.conf`: Disable + verification of outputs by changing the `yes` (the value of + `verify-outputs`) to `no`. Later, when you are ready to submit your + paper, or publish the dataset, activate verification and make the + proper corrections in this file (described under the "Other basic + customizations" section below). This is a critical step and only + takes a few minutes when your project is finished. So DON'T FORGET + to activate it in the end. - Re-make the project (after a cleaning) to see if you haven't introduced any errors. @@ -699,7 +715,7 @@ First custom commit 7. **Ignore changes in some Maneage files**: One of the main advantages of Maneage is that you can later update your infra-structure by merging - your `master` branch with the `maneage` branch. This is good for many + your `main` branch with the `maneage` branch. This is good for many low-level features that you will likely never modify yourself. But it is not desired for some files like `paper.tex` (you don't want changes in Maneage's default `paper.tex` to cause conflicts with all the text @@ -743,12 +759,12 @@ First custom commit add a copyright notice in your name under the existing one(s), like the line with capital letters below. To start with, add this line with your name and email address to `paper.tex`, - `tex/src/preamble-header.tex`, `reproduce/analysis/make/top-make.mk`, + `tex/src/preamble-project.tex`, `reproduce/analysis/make/top-make.mk`, and generally, all the files you modified in the previous step. ``` - Copyright (C) 2018-2021 Existing Name <existing@email.address> - Copyright (C) 2021 YOUR NAME <YOUR@EMAIL.ADDRESS> + Copyright (C) 2018-2022 Existing Name <existing@email.address> + Copyright (C) 2022 YOUR NAME <YOUR@EMAIL.ADDRESS> ``` 9. **Configure Git for fist time**: If this is the first time you are @@ -766,7 +782,7 @@ First custom commit ``` 10. **Your first commit**: You have already made some small and basic - changes in the steps above and you are in your project's `master` + changes in the steps above and you are in your project's `main` branch. So, you can officially make your first commit in your project's history and push it. But before that, you need to make sure that there are no problems in the project. This is a good habit to @@ -823,24 +839,12 @@ Other basic customizations Gnuastro, go through the analysis steps in `reproduce/analysis` and remove all its use cases (clearly marked). - - **Input dataset**: The input datasets are managed through the - `reproduce/analysis/config/INPUTS.conf` file. It is best to gather all - the information regarding all the input datasets into this one central - file. To ensure that the proper dataset is being downloaded and used - by the project, it is also recommended get an [MD5 - checksum](https://en.wikipedia.org/wiki/MD5) of the file and include - that in `INPUTS.conf` so the project can check it automatically. The - preparation/downloading of the input datasets is done in - `reproduce/analysis/make/download.mk`. Have a look there to see how - these values are to be used. This information about the input datasets - is also used in the initial `configure` script (to inform the users), - so also modify that file. You can find all occurrences of the demo - dataset with the command below and replace it with your input's - dataset. - - ```shell - $ grep -ir wfpc2 ./* - ``` + - **Input datasets**: The input datasets are managed through the + `reproduce/analysis/config/INPUTS.conf` file. It is best to gather the + following information regarding all the input datasets into this one + central file: 1) the SHA256 checksum of the file, 2) the URL where the + file can be downloaded online. Please read the comments at the start + of `reproduce/analysis/config/INPUTS.conf` carefully. - **`README.md`**: Correct all the `XXXXX` place holders (name of your project, your own name, address of your project's online/remote @@ -1518,12 +1522,12 @@ for the benefit of others. # Have a look at the commits in the 'maneage' branch in relation # with your project. - $ git log --oneline --graph --decorate --all # General view of branches. + $ git log --oneline --graph --all # General view of branches. - # Go to your 'master' branch and import all the updates into - # 'master', don't worry about the printed outputs (in particular + # Go to your 'main' branch and import all the updates into + # 'main', don't worry about the printed outputs (in particular # the 'CONFLICT's), we'll clean them up in the next step. - $ git checkout master + $ git checkout main $ git merge maneage # Ignore conflicting Maneage files that you had previously deleted @@ -1541,7 +1545,7 @@ for the benefit of others. git status # TIP: If you want the changes in one file to be only from a - # special branch ('maneage' or 'master', completely ignoring + # special branch ('maneage' or 'main', completely ignoring # changes in the other), use this command: # $ git checkout <BRANCH-NAME> -- <FILENAME> @@ -1564,7 +1568,7 @@ for the benefit of others. ./project make # When everything is OK, before continuing with your project's - # work, don't forget to push both your 'master' branch and your + # work, don't forget to push both your 'main' branch and your # updated 'maneage' branch to your remote server. git push git push origin maneage |