1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
|
<!DOCTYPE html>
<!-- Copyright notes are just below the head and before body -->
<html lang="en-US">
<!-- HTML Header -->
<head>
<!-- Title of the page. -->
<title>Maneage -- Managing data lineage</title>
<!-- Enable UTF-8 encoding to easily use non-ASCII charactes -->
<meta charset="UTF-8">
<meta http-equiv="Content-type" content="text/html; charset=UTF-8">
<!-- Put logo beside the address bar -->
<link rel="shortcut icon" href="./img/maneage-logo.svg" />
<!-- The viewport meta tag is placed mainly for mobile browsers
that are pre-configured in different ways (for example setting the
different widths for the page than the actual width of the device,
or zooming to different values. Without this the CSS media
solutions might not work properly on all mobile browsers.-->
<meta name="viewport"
content="width=device-width, initial-scale=1">
<!-- Basic styles -->
<link rel="stylesheet" href="css/base.css" />
</head>
<!--
Webpage of Maneage: a framework for managing data lineage
Copyright (C) 2020-2023 Pedram Ashofteh Ardakani <pedramardakani@pm.me>
Copyright (C) 2020-2023 Mohammad Akhlaghi <mohammad@akhlaghi.org>
This file is part of Maneage. Maneage is free software: you can
redistribute it and/or modify it under the terms of the GNU General
Public License as published by the Free Software Foundation, either
version 3 of the License, or (at your option) any later version.
Maneage is distributed in the hope that it will be useful, but
WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
General Public License for more details. See
<http://www.gnu.org/licenses/>. -->
<!-- Start the main body. -->
<body>
<div id="container">
<header role="banner">
<!-- global navigation -->
<nav role="navigation" id="nav-hamburger-wrapper">
<input type="checkbox" id="nav-hamburger-input"/>
<label for="nav-hamburger-input">|||</label>
<div id="nav-hamburger-items" class="button">
<a href="index.html">Home</a>
<a href="about.html">About</a>
<a href="http://git.maneage.org/project.git/">Git</a>
<a href="tutorial.html">Tutorial</a>
</div>
</nav>
</header>
<section>
<!-- Maneage logo -->
<div class="banner">
<div>
<img src="img/maneage-logo.svg" />
</div>
<div>
<h1>Maneage</h1>
<h2>Managing Data Lineage</h2>
</div>
</div>
<p class="clear">Maneage is a customizable workflow controller program (implemented in GNU Make), providing low-level control over a project's data lineage and the version controlled history of the data lineage (implemented in Git).
Data lineage or data provenance refer to keeping track of how data evolves within a project.
For example the process of converting raw data from external data bases to a final plot in the published paper/report (this usually involves many different analysis steps and depends on many software packages).
Therefore Maneage is used for <a href="http://akhlaghi.org/reproducible-science.html">reproducible research</a> (which is a critical issue for the integrity of modern scientific results).
The name "Maneage" is a <a href="https://en.wikipedia.org/wiki/Portmanteau">portmanteau</a>; created by merging the first three characters of "<i>man</i>age" with the last four characters of "lin<i>eage</i>".</p>
<p>Maneage is <a href="https://www.gnu.org/philosophy/free-sw.en.html">free software</a> and released under <a href="https://www.gnu.org/licenses/gpl-3.0.en.html">GNU GPL v3+</a>.
Maneage is a <i>program</i>, not a <i>service</i>.
This means that you download a copy of Maneage and install a copy on your own machine.
Then your copy does the work. Since Maneage is free/libre software, you can change your copy.
Therefore, you have full control over what it does.
To learn more about its founding criteria and a basic introduction, see Akhlaghi et al. (<a href="https://doi.org/10.1109/MCSE.2021.3072860">CiSE 2021, vol 23, issue 3, pp 82-91</a>), also available as <a href="https://arxiv.org/abs/2006.03018">arXiv:2006.03018</a> (with extended appendix in one PDF: the recommended format).
Maneage is a recipient of the <a href="https://www.rd-alliance.org/node/64603">RDA Europe Adoption grant</a> and was featured in a Nature Astronomy "<a href="https://doi.org/10.1038/s41550-021-01402-3">News and Views</a>" article (Kuttel 2021, <a href="https://rdcu.be/cmYVx">free-to-read link</a>).
You can also watch the short "Video presentation of Maneage" below, or see this published <a href="https://www.rd-alliance.org/sites/default/files/RDA%20adoption%20story_IAC_final.pdf">RDA Adoption story</a> (a short PDF).
</p>
<img class="center" src="img/project-flow.svg" width="85%" />
<h3>Video presentation of Maneage:</h3>
<p class="clean">In the box below, you can watch a video of an invited talk at the "Reproducibility and Open Science in Astronomy" (<a href="https://www.eso.org/sci/meetings/2022/REPRODUCIBILITY2022.html">ROSA2022</a>) workshop that was organized by the European Southern Observatory (ESO) and held in May 2022. It is 32 minutes, hosted <a href="https://peertube.stream/videos/embed/d1373ec6-0fc7-4f3d-a5e6-3a4c763951dd">on PeerTube</a>).
For a more detailed talk (80 minutes), see this <a href="https://peertube.stream/videos/watch/e7fdf27b-61b8-40cd-8306-482d79e4d6c5">invited lecture at CiTIUS</a> (link opens in PeerTube).
The slides are also <a href="pdf/slides-intro.pdf">available as PDF</a>.</p>
<div class="aspect-ratio">
<iframe class="center" sandbox="allow-same-origin allow-scripts allow-popups" title="Maneage: Managing data lineage for long-term and archivable reproducibility (Invited talk at ESO's ROSA2022)" src="https://peertube.stream/videos/embed/d1373ec6-0fc7-4f3d-a5e6-3a4c763951dd" frameborder="0" allowfullscreen></iframe>
</div>
<p class="clean" style="font-size: 0.7em;">If the video <i>doesn't load</i> after clicking on it, you are probably viewing this page through the HTTP protocol. You have three solutions: 1) either revisit this page through HTTPS at <a href="https://maneage.org">https://maneage.org</a>. 2) watch the video directly <a href="https://peertube.stream/w/rQqt2RpyYPmGzAD4uBbxQz">on PeerTube</a>. 3) <a href="https://git.maneage.org/videos.git/plain/maneage-eso-2022-05-12.mp4">Download the video</a>.</p>
</section>
<section>
<h3>Start building your project in Maneage</h3>
<p>To start a new project, simply run these commands to clone it from its <a href="http://git.maneage.org/project.git">Git repository</a>.
<pre><code>git clone https://git.maneage.org/project.git <span class="comment"># Clone Maneage, default branch `maneage'.</span>
mv project my-project && cd my-project <span class="comment"># Set custom name and enter directory.</span>
git remote rename origin origin-maneage <span class="comment"># Rename remote server to use `origin' later.</span>
git checkout -b main <span class="comment"># Make new `main' branch, start customizing.</span></code></pre>
</p>
<p>You are now ready to configure and make the raw template with the commands below.
If they are successful, you can start customizing it.
<pre><code>./project configure <span class="comment"># Build all necessary software from source.</span>
./project make <span class="comment"># Do the analysis (download data, run software on data, build PDF).</span></code></pre>
</p>
<p>See the <a href="https://gitlab.com/maneage/project/-/blob/maneage/README-hacking.md#customization-checklist">Customization Checklist</a> in the cloned <code>README-hacking.md</code> file for the next steps to start customizing Maneage for your project.
</p>
<h3>Messaging rooms for direct contact</h3>
<p>Maneage uses the <a href="https://en.wikipedia.org/wiki/Matrix_(protocol)">Matrix protocol</a> for fast, robust and free communication between the developers and users.
The top-level Maneage "Space" (which collects the Maneage-related rooms) is available at <code>#maneage:matrix.org</code>. The available list of rooms are:</p>
<ul>
<li><a href="https://app.element.io/#/room/#maneage-general:matrix.org"><code>#maneage-general:matrix.org</code></a>: The best place for new "Maneagers".</li>
<li><a href="https://app.element.io/#/room/#maneage-make:matrix.org"><code>#maneage-make:matrix.org</code></a>: Discussions specific to <a href="https://en.wikipedia.org/wiki/Make_(software)">GNU Make</a> within Maneage (Make is the core workflow manager in Maneage).</li>
<li><a href="https://app.element.io/#/room/#maneage-latex:matrix.org"><code>#maneage-latex:matrix.org</code></a>: Discussions specific to <a href="https://en.wikipedia.org/wiki/LaTeX">LaTeX</a> within Maneage (LaTeX is the type-setting system to produce the final report/PDF).</li>
<li><a href="https://app.element.io/#/room/#maneage-dev:matrix.org"><code>#maneage-dev:matrix.org</code></a>: Development discussions in Maneage (on the core infrastructure; maybe too technical for early Maneagers).</li>
</ul>
<i>How can I access Matrix-based messaging rooms?</i> There are <a href="https://matrix.org/clients/">many clients</a> for the Matrix protocol for web-based, smartphone or Desktop (GUI or command-line).
One suggested client can be <a href="https://element.io/get-started">Element</a>, which has both a web-based interface (only requiring a modern browser), a GUI desktop application, or smartphone apps.
<!--
<h3>Submitting bugs and suggesting new features</h3>
<p>Development discussions (like list of existing <a href="https://savannah.nongnu.org/bugs/?group=reproduce">bugs</a> and <a href="https://savannah.nongnu.org/task/?group=reproduce">tasks</a>) are currently maintained in <a href="https://savannah.nongnu.org/projects/reproduce/">GNU Savannah</a>.
You can <a href="https://savannah.nongnu.org/account/register.php">register in GNU Savannah</a> to <a href="https://savannah.nongnu.org/bugs/?func=additem&group=reproduce">submit a bug</a> or <a href="https://savannah.nongnu.org/task/?func=additem&group=reproduce">submit a task</a>, or comment on an existing bug or task. If you want to submit a general issue without registering on Savannah, you can <a href="https://savannah.nongnu.org/support/?func=additem&group=reproduce">submit an item</a>.</p>
-->
<h3>Merge/Pull requests</h3>
<p>As you continue customizing Maneage for your own project, you will notice generic improvements that can be useful for other projects too.
In such cases, please send us those changes to implement in the core Maneage branch and let them propagate to all projects using it.
If you look through the history of the Maneage branch, you'll notice many users have already started doing this, and this is how Maneage is planned to grow.
The recommended process is very similar to
<a href="https://www.gnu.org/software/gnuastro/manual/html_node/Forking-tutorial.html">this forking tutorial</a>.
Here is a summary:
</p>
<ol>
<li>Go to the <code>maneage</code> branch and create a new branch from there like below:
<pre><code>git checkout maneage
git branch -b my-fix</code></pre>
</li>
<li>Commit your fix over this new branch (see these <a href="https://www.gnu.org/software/gnuastro/manual/html_node/Commit-guidelines.html">commit message guidelines</a>).</li>
<li>Build a new project on your favorite Git repository (GitLab, BitBucket, or GitHub for example) and assign it to a new Git remote in your project.
Let's call it <code>my-remote</code>.
You only need to do this once and keep this for future fixes.
</li>
<li>Push your branch to that remote:
<pre><code>git push my-remote my-fix</code></pre>
</li>
<li>Submit a link to your fork and the corresponding branch <a href="http://savannah.nongnu.org/support/?func=additem&group=reproduce">on Savannah</a>.
If you are <a href="https://savannah.nongnu.org/account/register.php">registered on Savannah</a>, you can also submit it as <a href="https://savannah.nongnu.org/bugs/?func=additem&group=reproduce">a bug</a> or <a href="https://savannah.nongnu.org/task/?func=additem&group=reproduce">a task</a>.
</li>
</ol>
</section>
<section>
<h3>Supporting organizations:</h3>
<div class="support">
<div id="logo-rda"><a href="https://cordis.europa.eu/project/id/777388"><img src="./img/rda-europe.png"></a></div>
<div><a href="https://www.cefca.es/"><img src="./img/cefca.png"></a></div>
<div><a href="https://www.iac.es/"><img src="./img/iac-compressed.svg"></a></div>
<div><a href="https://cordis.europa.eu/project/id/339659"><img src="./img/erc-compressed.svg"></a></div>
<div id="logo-mext"><a href="https://www.mext.go.jp/en"><img src="./img/mext-compressed.svg"></a></div>
</div>
</section>
<footer role="contentinfo" id="page-footer">
<ul>
<li><p>Maneage is currently based in the Centro de Estudios de Física del Cosmos de Aragón (CEFCA).</p></li>
<li><p>Address: CEFCA, Plaza San Juan 1, Planta 2, Teruel, Spain, 44001.</p></li>
<li><p>Contact: with <a href="https://savannah.nongnu.org/support/?func=additem&group=reproduce">this form</a>, or <a href="https://app.element.io/#/room/#maneage-general:matrix.org">#maneage-general:matrix.org</a>, or project PI (<a href="http://akhlaghi.org">Mohammad Akhlaghi</a>).</p></li>
<li><p>Copyright © 2020-2023 Maneage volunteers</p></li>
<li>This page is distributed under GNU General Public License (<a href="https://www.gnu.org/licenses/gpl-3.0.en.html">GPL</a>).</li>
<li><p>All logos are copyrighted by the respective institutions</p></li>
</ul>
</footer>
</div>
</body>
</html>
|
