aboutsummaryrefslogtreecommitdiff
path: root/about.html
blob: 248142aef6b46ad26072357dc76007c6b33ef8ee (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113
1114
1115
1116
1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207
1208
1209
1210
1211
1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
1230
1231
1232
1233
1234
1235
1236
1237
1238
1239
1240
1241
1242
1243
1244
1245
1246
1247
1248
1249
1250
1251
1252
1253
1254
1255
1256
1257
1258
1259
1260
1261
1262
1263
1264
1265
1266
1267
1268
1269
<!DOCTYPE html>
<!--
    Webpage of Maneage: a framework for managing data lineage

    Copyright (C) 2020, Mohammad Akhlaghi <mohammad@akhlaghi.org>

    This file is part of Maneage. Maneage is free software: you can
    redistribute it and/or modify it under the terms of the GNU General
    Public License as published by the Free Software Foundation, either
    version 3 of the License, or (at your option) any later version.

    Maneage is distributed in the hope that it will be useful, but
    WITHOUT ANY WARRANTY; without even the implied warranty of
    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
    General Public License for more details. See
    <http://www.gnu.org/licenses/>.  -->

    <html lang="en-US">

        <!-- HTML Header -->
        <head>
            <!-- Title of the page. -->
            <title>Maneage -- Managing data lineage</title>

            <!-- Enable UTF-8 encoding to easily use non-ASCII charactes -->
            <meta charset="UTF-8">
            <meta http-equiv="Content-type" content="text/html; charset=UTF-8">

            <!-- Put logo beside the address bar -->
            <link rel="shortcut icon" href="./img/favicon.svg" />

            <!-- The viewport meta tag is placed mainly for mobile browsers
                that are pre-configured in different ways (for example setting the
                different widths for the page than the actual width of the device,
                or zooming to different values. Without this the CSS media
                solutions might not work properly on all mobile browsers.-->
                <meta name="viewport"
                      content="width=device-width, initial-scale=1">

                <!-- Basic styles -->
                <link rel="stylesheet" href="css/base.css" />
        </head>




        <!-- Start the main body. -->
        <body>
            <div id="container">
                <header role="banner">
                    <!-- global navigation -->
                    <nav role="navigation" id="hamnav">
                        <label for="hamburger">&#9776;</label>
                        <input type="checkbox" id="hamburger"/>
                        <div id="hamitems" class="button">
                            <a href="index.html">Home</a>
                            <a href="about.html">About</a>
                            <a href="http://git.maneage.org/project.git/">&#10515; Git Repository</a>
                            <a href="tutorial.html">Tutorial</a>
                        </div>
                    </nav>
                </header>
                <div class="banner">
                    <div>
                        <a href="index.html"><img src="img/maneage-logo.svg" /></a>
                    </div>
                    <div>
                        <h1>Maneage</h1><h2>Managing data lineage</h2>
                        <p>Copyright (C) 2018-2020 Mohammad Akhlaghi <a href="&#109;&#x61;&#x69;&#x6C;&#x74;&#x6F;:&#x6D;&#111;&#104;&#97;&#x6D;&#109;a&#x64;&#64;&#x61;&#107;&#x68;&#x6C;&#x61;&#x67;&#104;&#x69;.&#x6F;&#x72;&#103;">&#x6D;&#111;&#104;&#97;&#x6D;&#109;a&#x64;&#64;&#x61;&#107;&#x68;&#x6C;&#x61;&#x67;&#104;&#x69;.&#x6F;&#x72;&#103;</a><br />
                        Copyright (C) 2020 Raul Infante-Sainz <a href="m&#x61;&#105;&#108;t&#111;:&#x69;&#x6E;&#x66;&#x61;&#x6E;&#116;&#101;&#115;&#97;&#x69;n&#122;&#64;&#103;&#x6D;&#x61;&#x69;&#x6C;&#x2E;&#x63;&#111;&#x6D;">&#x69;&#x6E;&#x66;&#x61;&#x6E;&#116;&#101;&#115;&#97;&#x69;n&#122;&#64;&#103;&#x6D;&#x61;&#x69;&#x6C;&#x2E;&#x63;&#111;&#x6D;</a><br />
                        See the end of the file for license conditions.</p>
                    </div>
                </div>

                <p>Maneage is a <strong>fully working template</strong> for doing reproducible research (or
                writing a reproducible paper) as defined in the link below. If the link
                below is not accessible at the time of reading, please see the appendix at
                the end of this file for a portion of its introduction. Some
                <a href="http://akhlaghi.org/pdf/reproducible-paper.pdf">slides</a> are also available
                to help demonstrate the concept implemented here.</p>

                <ul>
                    <li><a href="http://akhlaghi.org/reproducible-science.html">http://akhlaghi.org/reproducible-science.html</a></li>
                </ul>

                <p>Maneage is created with the aim of supporting reproducible research by
                making it easy to start a project in this framework. As shown below, it is
                very easy to customize Maneage for any particular (research) project and
                expand it as it starts and evolves. It can be run with no modification (as
                described in <code>README.md</code>) as a demonstration and customized for use in any
                project as fully described below.</p>

                <p>A project designed using Maneage will download and build all the necessary
                libraries and programs for working in a closed environment (highly
                independent of the host operating system) with fixed versions of the
                necessary dependencies. The tarballs for building the local environment are
                also collected in a <a href="http://git.maneage.org/tarballs-software.git/tree/">separate
                    repository</a>. The final
                output of the project is <a href="http://git.maneage.org/output-raw.git/plain/paper.pdf">a
                    paper</a>.  Notice the
                last paragraph of the Acknowledgments where all the necessary software are
                mentioned with their versions.</p>

                <p>Below, we start with a discussion of why Make was chosen as the high-level
                language/framework for project management and how to learn and master Make
                easily (and freely). The general architecture and design of the project is
                then discussed to help you navigate the files and their contents. This is
                followed by a checklist for the easy/fast customization of Maneage to your
                exciting research. We continue with some tips and guidelines on how to
                manage or extend your project as it grows based on our experiences with it
                so far. The main body concludes with a description of possible future
                improvements that are planned for Maneage (but not yet implemented). As
                discussed above, we end with a short introduction on the necessity of
                reproducible science in the appendix.</p>

                <p>Please don't forget to share your thoughts, suggestions and
                criticisms. Maintaining and designing Maneage is itself a separate project,
                so please join us if you are interested. Once it is mature enough, we will
                describe it in a paper (written by all contributors) for a formal
                introduction to the community.</p>

                <h2>Why Make?</h2>

                <p>When batch processing is necessary (no manual intervention, as in a
                reproducible project), shell scripts are usually the first solution that
                come to mind. However, the inherent complexity and non-linearity of
                progress in a scientific project (where experimentation is key) make it
                hard to manage the script(s) as the project evolves. For example, a script
                will start from the top/start every time it is run. So if you have already
                completed 90% of a research project and want to run the remaining 10% that
                you have newly added, you have to run the whole script from the start
                again. Only then will you see the effects of the last new steps (to find
                possible errors, or better solutions and etc).</p>

                <p>It is possible to manually ignore/comment parts of a script to only do a
                special part. However, such checks/comments will only add to the complexity
                of the script and will discourage you to play-with/change an already
                completed part of the project when an idea suddenly comes up. It is also
                prone to very serious bugs in the end (when trying to reproduce from
                scratch). Such bugs are very hard to notice during the work and frustrating
                to find in the end.</p>

                <p>The Make paradigm, on the other hand, starts from the end: the final
                <em>target</em>. It builds a dependency tree internally, and finds where it should
                start each time the project is run. Therefore, in the scenario above, a
                researcher that has just added the final 10% of steps of her research to
                her Makefile, will only have to run those extra steps. With Make, it is
                also trivial to change the processing of any intermediate (already written)
                <em>rule</em> (or step) in the middle of an already written analysis: the next
                time Make is run, only rules that are affected by the changes/additions
                will be re-run, not the whole analysis/project.</p>

                <p>This greatly speeds up the processing (enabling creative changes), while
                keeping all the dependencies clearly documented (as part of the Make
                language), and most importantly, enabling full reproducibility from scratch
                with no changes in the project code that was working during the
                research. This will allow robust results and let the scientists get to what
                they do best: experiment and be critical to the methods/analysis without
                having to waste energy and time on technical problems that come up as a
                result of that experimentation in scripts.</p>

                <p>Since the dependencies are clearly demarcated in Make, it can identify
                independent steps and run them in parallel. This further speeds up the
                processing. Make was designed for this purpose. It is how huge projects
                like all Unix-like operating systems (including GNU/Linux or Mac OS
                operating systems) and their core components are built. Therefore, Make is
                a highly mature paradigm/system with robust and highly efficient
                implementations in various operating systems perfectly suited for a complex
                non-linear research project.</p>

                <p>Make is a small language with the aim of defining <em>rules</em> containing
                <em>targets</em>, <em>prerequisites</em> and <em>recipes</em>. It comes with some nice features
                like functions or automatic-variables to greatly facilitate the management
                of text (filenames for example) or any of those constructs. For a more
                detailed (yet still general) introduction see the article on Wikipedia:</p>

                <ul>
                    <li><a href="https://en.wikipedia.org/wiki/Make_(software)">https://en.wikipedia.org/wiki/Make_(software)</a></li>
                </ul>

                <p>Make is a +40 year old software that is still evolving, therefore many
                implementations of Make exist. The only difference in them is some extra
                features over the <a href="https://pubs.opengroup.org/onlinepubs/009695399/utilities/make.html">standard
                    definition</a>
                (which is shared in all of them). Maneage is primarily written in GNU Make
                (which it installs itself, you don't have to have it on your system). GNU
                Make is the most common, most actively developed, and most advanced
                implementation. Just note that Maneage downloads, builds, internally
                installs, and uses its own dependencies (including GNU Make), so you don't
                have to have it installed before you try it out.</p>

                <h2>How can I learn Make?</h2>

                <p>The GNU Make book/manual (links below) is arguably the best place to learn
                Make. It is an excellent and non-technical book to help get started (it is
                only non-technical in its first few chapters to get you started easily). It
                is freely available and always up to date with the current GNU Make
                release. It also clearly explains which features are specific to GNU Make
                and which are general in all implementations. So the first few chapters
                regarding the generalities are useful for all implementations.</p>

                <p>The first link below points to the GNU Make manual in various formats and
                in the second, you can download it in PDF (which may be easier for a first
                time reading).</p>

                <ul>
                    <li><a href="https://www.gnu.org/software/make/manual/">https://www.gnu.org/software/make/manual/</a></li>
                    <li><a href="https://www.gnu.org/software/make/manual/make.pdf">https://www.gnu.org/software/make/manual/make.pdf</a></li>
                </ul>

                <p>If you use GNU Make, you also have the whole GNU Make manual on the
                command-line with the following command (you can come out of the "Info"
                environment by pressing <code>q</code>).</p>

                <pre><code>info make</code></pre>

                <p>If you aren't familiar with the Info documentation format, we strongly
                recommend running <code>$ info info</code> and reading along. In less than an hour,
                you will become highly proficient in it (it is very simple and has a great
                manual for itself). Info greatly simplifies your access (without taking
                your hands off the keyboard!) to many manuals that are installed on your
                system, allowing you to be much more efficient as you work. If you use the
                GNU Emacs text editor (or any of its variants), you also have access to all
                Info manuals while you are writing your projects (again, without taking
                your hands off the keyboard!).</p>

                <h2>Published works using Maneage</h2>

                <p>The list below shows some of the works that have already been published
                with (earlier versions of) Maneage. Previously it was simply called
                "Reproducible paper template". Note that Maneage is evolving, so some
                details may be different in them. The more recent ones can be used as a
                good working example.</p>

                <ul>
                    <li><p>Infante-Sainz et
                        al. (<a href="https://ui.adsabs.harvard.edu/abs/2020MNRAS.491.5317I">2020</a>,
                        MNRAS, 491, 5317): The version controlled project source is available
                        <a href="https://gitlab.com/infantesainz/sdss-extended-psfs-paper">on GitLab</a>
                        and is also archived on Zenodo with all the necessary software tarballs:
                        <a href="https://zenodo.org/record/3524937">zenodo.3524937</a>.</p></li>
                    <li><p>Akhlaghi (<a href="https://arxiv.org/abs/1909.11230">2019</a>, IAU Symposium
                        355). The version controlled project source is available <a href="https://gitlab.com/makhlaghi/iau-symposium-355">on
                            GitLab</a> and is also
                        archived on Zenodo with all the necessary software tarballs:
                        <a href="https://doi.org/10.5281/zenodo.3408481">zenodo.3408481</a>.</p></li>
                    <li><p>Section 7.3 of Bacon et
                        al. (<a href="http://adsabs.harvard.edu/abs/2017A%26A...608A...1B">2017</a>, A&amp;A
                        608, A1): The version controlled project source is available <a href="https://gitlab.com/makhlaghi/muse-udf-origin-only-hst-magnitudes">on
                            GitLab</a>
                        and a snapshot of the project along with all the necessary input
                        datasets and outputs is available in
                        <a href="https://doi.org/10.5281/zenodo.1164774">zenodo.1164774</a>.</p></li>
                    <li><p>Section 4 of Bacon et
                        al. (<a href="http://adsabs.harvard.edu/abs/2017A%26A...608A...1B">2017</a>, A&amp;A,
                        608, A1): The version controlled project is available <a href="https://gitlab.com/makhlaghi/muse-udf-photometry-astrometry">on
                            GitLab</a> and
                        a snapshot of the project along with all the necessary input datasets is
                        available in <a href="https://doi.org/10.5281/zenodo.1163746">zenodo.1163746</a>.</p></li>
                    <li><p>Akhlaghi &amp; Ichikawa
                        (<a href="http://adsabs.harvard.edu/abs/2015ApJS..220....1A">2015</a>, ApJS, 220,
                        1): The version controlled project is available <a href="https://gitlab.com/makhlaghi/NoiseChisel-paper">on
                            GitLab</a>. This is the
                        very first (and much less mature!) incarnation of Maneage: the history
                        of Maneage started more than two years after this paper was
                        published. It is a very rudimentary/initial implementation, thus it is
                        only included here for historical reasons. However, the project source
                        is complete, accurate and uploaded to arXiv along with the paper.</p></li>
                </ul>

                <h2>Citation</h2>

                <p>A paper to fully describe Maneage has been submitted. Until then, if you
                used it in your work, please cite the paper that implemented its first
                version: Akhlaghi &amp; Ichikawa
                (<a href="http://adsabs.harvard.edu/abs/2015ApJS..220....1A">2015</a>, ApJS, 220, 1).</p>

                <p>Also, when your paper is published, don't forget to add a notice in your
                own paper (in coordination with the publishing editor) that the paper is
                fully reproducible and possibly add a sentence or paragraph in the end of
                the paper shortly describing the concept. This will help spread the word
                and encourage other scientists to also manage and publish their projects in
                a reproducible manner.</p>

                <h1>Project architecture</h1>

                <p>In order to customize Maneage to your research, it is important to first
                understand its architecture so you can navigate your way in the directories
                and understand how to implement your research project within its framework:
                where to add new files and which existing files to modify for what
                purpose. But if this the first time you are using Maneage, before reading
                this theoretical discussion, please run Maneage once from scratch without
                any changes (described in <code>README.md</code>). You will see how it works (note that
                the configure step builds all necessary software, so it can take long, but
                you can continue reading while its working).</p>

                <p>The project has two top-level directories: <code>reproduce</code> and
                <code>tex</code>. <code>reproduce</code> hosts all the software building and analysis
                steps. <code>tex</code> contains all the final paper's components to be compiled into
                a PDF using LaTeX.</p>

                <p>The <code>reproduce</code> directory has two sub-directories: <code>software</code> and
                <code>analysis</code>. As the name says, the former contains all the instructions to
                download, build and install (independent of the host operating system) the
                necessary software (these are called by the <code>./project configure</code>
                command). The latter contains instructions on how to use those software to
                do your project's analysis.</p>

                <p>After it finishes, <code>./project configure</code> will create the following symbolic
                links in the project's top source directory: <code>.build</code> which points to the
                top build directory and <code>.local</code> for easy access to the custom built
                software installation directory. With these you can easily access the build
                directory and project-specific software from your top source directory. For
                example if you run <code>.local/bin/ls</code> you will be using the <code>ls</code> of Maneage,
                which is probably different from your system's <code>ls</code> (run them both with
                <code>--version</code> to check).</p>

                <p>Once the project is configured for your system, <code>./project make</code> will do
                the basic preparations and run the project's analysis with the custom
                version of software. The <code>project</code> script is just a wrapper, and with the
                <code>make</code> argument, it will first call <code>top-prepare.mk</code> and <code>top-make.mk</code>
                (both are in the <code>reproduce/analysis/make</code> directory).</p>

                <p>In terms of organization, <code>top-prepare.mk</code> and <code>top-make.mk</code> have an
                identical design, only minor differences. So, let's continue Maneage's
                architecture with <code>top-make.mk</code>. Once you understand that, you'll clearly
                understand <code>top-prepare.mk</code> also. These very high-level files are
                relatively short and heavily commented so hopefully the descriptions in
                each comment will be enough to understand the general details. As you read
                this section, please also look at the contents of the mentioned files and
                directories to fully understand what is going on.</p>

                <p>Before starting to look into the top <code>top-make.mk</code>, it is important to
                recall that Make defines dependencies by files. Therefore, the
                input/prerequisite and output of every step/rule must be a file. Also
                recall that Make will use the modification date of the prerequisite(s) and
                target files to see if the target must be re-built or not. Therefore during
                the processing, <em>many</em> intermediate files will be created (see the tips
                section below on a good strategy to deal with large/huge files).</p>

                <p>To keep the source and (intermediate) built files separate, the user <em>must</em>
                define a top-level build directory variable (or <code>$(BDIR)</code>) to host all the
                intermediate files (you defined it during <code>./project configure</code>). This
                directory doesn't need to be version controlled or even synchronized, or
                backed-up in other servers: its contents are all products, and can be
                easily re-created any time. As you define targets for your new rules, it is
                thus important to place them all under sub-directories of <code>$(BDIR)</code>. As
                mentioned above, you always have fast access to this "build"-directory with
                the <code>.build</code> symbolic link. Also, beware to <em>never</em> make any manual change
                in the files of the build-directory, just delete them (so they are
                re-built).</p>

                <p>In this architecture, we have two types of Makefiles that are loaded into
                the top <code>Makefile</code>: <em>configuration-Makefiles</em> (only independent
                variables/configurations) and <em>workhorse-Makefiles</em> (Makefiles that
                actually contain analysis/processing rules).</p>

                <p>The configuration-Makefiles are those that satisfy these two wildcards:
                <code>reproduce/software/config/*.conf</code> (for building the necessary software
                when you run <code>./project configure</code>) and <code>reproduce/analysis/config/*.conf</code>
                (for the high-level analysis, when you run <code>./project make</code>). These
                Makefiles don't actually have any rules, they just have values for various
                free parameters throughout the configuration or analysis. Open a few of
                them to see for yourself. These Makefiles must only contain raw Make
                variables (project configurations). By "raw" we mean that the Make
                variables in these files must not depend on variables in any other
                configuration-Makefile. This is because we don't want to assume any order
                in reading them. It is also very important to <em>not</em> define any rule, or
                other Make construct, in these configuration-Makefiles.</p>

                <p>Following this rule-of-thumb enables you to set these configure-Makefiles
                as a prerequisite to any target that depends on their variable
                values. Therefore, if you change any of their values, all targets that
                depend on those values will be re-built. This is very convenient as your
                project scales up and gets more complex.</p>

                <p>The workhorse-Makefiles are those satisfying this wildcard
                <code>reproduce/software/make/*.mk</code> and <code>reproduce/analysis/make/*.mk</code>. They
                contain the details of the processing steps (Makefiles containing
                rules). Therefore, in this phase <em>order is important</em>, because the
                prerequisites of most rules will be the targets of other rules that will be
                defined prior to them (not a fixed name like <code>paper.pdf</code>). The lower-level
                rules must be imported into Make before the higher-level ones.</p>

                <p>All processing steps are assumed to ultimately (usually after many rules)
                end up in some number, image, figure, or table that will be included in the
                paper. The writing of these results into the final report/paper is managed
                through separate LaTeX files that only contain macros (a name given to a
                number/string to be used in the LaTeX source, which will be replaced when
                compiling it to the final PDF). So the last target in a workhorse-Makefile
                is a <code>.tex</code> file (with the same base-name as the Makefile, but in
                <code>$(BDIR)/tex/macros</code>). As a result, if the targets in a workhorse-Makefile
                aren't directly a prerequisite of other workhorse-Makefile targets, they
                can be a prerequisite of that intermediate LaTeX macro file and thus be
                called when necessary. Otherwise, they will be ignored by Make.</p>

                <p>Maneage also has a mode to share the build directory between several
                users of a Unix group (when working on large computer clusters). In this
                scenario, each user can have their own cloned project source, but share the
                large built files between each other. To do this, it is necessary for all
                built files to give full permission to group members while not allowing any
                other users access to the contents. Therefore the <code>./project configure</code> and
                <code>./project make</code> steps must be called with special conditions which are
                managed in the <code>--group</code> option.</p>

                <p>Let's see how this design is implemented. Please open and inspect
                <code>top-make.mk</code> it as we go along here. The first step (un-commented line) is
                to import the local configuration (your answers to the questions of
                <code>./project configure</code>). They are defined in the configuration-Makefile
                <code>reproduce/software/config/LOCAL.conf</code> which was also built by <code>./project
                    configure</code> (based on the <code>LOCAL.conf.in</code> template of the same directory).</p>

                <p>The next non-commented set of the top <code>Makefile</code> defines the ultimate
                target of the whole project (<code>paper.pdf</code>). But to avoid mistakes, a sanity
                check is necessary to see if Make is being run with the same group settings
                as the configure script (for example when the project is configured for
                group access using the <code>./for-group</code> script, but Make isn't). Therefore we
                use a Make conditional to define the <code>all</code> target based on the group
                permissions.</p>

                <p>Having defined the top/ultimate target, our next step is to include all the
                other necessary Makefiles. However, order matters in the importing of
                workhorse-Makefiles and each must also have a TeX macro file with the same
                base name (without a suffix). Therefore, the next step in the top-level
                Makefile is to define the <code>makesrc</code> variable to keep the base names
                (without a <code>.mk</code> suffix) of the workhorse-Makefiles that must be imported,
                in the proper order.</p>

                <p>Finally, we import all the necessary remaining Makefiles: 1) All the
                analysis configuration-Makefiles with a wildcard. 2) The software
                configuration-Makefile that contains their version (just in case its
                necessary). 3) All workhorse-Makefiles in the proper order using a Make
                <code>foreach</code> loop.</p>

                <p>In short, to keep things modular, readable and manageable, follow these
                recommendations: 1) Set clear-to-understand names for the
                configuration-Makefiles, and workhorse-Makefiles, 2) Only import other
                Makefiles from top Makefile. These will let you know/remember generally
                which step you are taking before or after another. Projects will scale up
                very fast. Thus if you don't start and continue with a clean and robust
                convention like this, in the end it will become very dirty and hard to
                manage/understand (even for yourself). As a general rule of thumb, break
                your rules into as many logically-similar but independent steps as
                possible.</p>

                <p>The <code>reproduce/analysis/make/paper.mk</code> Makefile must be the final Makefile
                that is included. This workhorse Makefile ends with the rule to build
                <code>paper.pdf</code> (final target of the whole project). If you look in it, you
                will notice that this Makefile starts with a rule to create
                <code>$(mtexdir)/project.tex</code> (<code>mtexdir</code> is just a shorthand name for
                <code>$(BDIR)/tex/macros</code> mentioned before). As you see, the only dependency of
                <code>$(mtexdir)/project.tex</code> is <code>$(mtexdir)/verify.tex</code> (which is the last
                analysis step: it verifies all the generated results).  Therefore,
                <code>$(mtexdir)/project.tex</code> is <em>the connection</em> between the
                processing/analysis steps of the project, and the steps to build the final
                PDF.</p>

                <p>During the research, it often happens that you want to test a step that is
                not a prerequisite of any higher-level operation. In such cases, you can
                (temporarily) define that processing as a rule in the most relevant
                workhorse-Makefile and set its target as a prerequisite of its TeX
                macro. If your test gives a promising result and you want to include it in
                your research, set it as prerequisites to other rules and remove it from
                the list of prerequisites for TeX macro file. In fact, this is how a
                project is designed to grow in this framework.</p>

                <h2>File modification dates (meta data)</h2>

                <p>While Git does an excellent job at keeping a history of the contents of
                files, it makes no effort in keeping the file meta data, and in particular
                the dates of files. Therefore when you checkout to a different branch,
                files that are re-written by Git will have a newer date than the other
                project files. However, file dates are important in the current design of
                Maneage: Make checks the dates of the prerequisite files and target files
                to see if the target should be re-built.</p>

                <p>To fix this problem, for Maneage we use a forked version of
                <a href="https://github.com/mohammad-akhlaghi/metastore">Metastore</a>. Metastore use
                a binary database file (which is called <code>.file-metadata</code>) to keep the
                modification dates of all the files under version control. This file is
                also under version control, but is hidden (because it shouldn't be modified
                by hand). During the project's configuration, Maneage installs to Git hooks
                to run Metastore 1) before making a commit to update its database with the
                file dates in a branch, and 2) after doing a checkout, to reset the
                file-dates after the checkout is complete and re-set the file dates back to
                what they were.</p>

                <p>In practice, Metastore should work almost fully invisibly within your
                project. The only place you might notice its presence is that you'll see
                <code>.file-metadata</code> in the list of modified/staged files (commonly after
                merging your branches). Since its a binary file, Git also won't show you
                the changed contents. In a merge, you can simply accept any changes with
                <code>git add -u</code>. But if Git is telling you that it has changed without a merge
                (for example if you started a commit, but canceled it in the middle), you
                can just do <code>git checkout .file-metadata</code> and set it back to its original
                state.</p>

                <h2>Summary</h2>

                <p>Based on the explanation above, some major design points you should have in
                mind are listed below.</p>

                <ul>
                    <li><p>Define new <code>reproduce/analysis/make/XXXXXX.mk</code> workhorse-Makefile(s)
                        with good and human-friendly name(s) replacing <code>XXXXXX</code>.</p></li>
                    <li><p>Add <code>XXXXXX</code>, as a new line, to the values in <code>makesrc</code> of the top-level
                        <code>Makefile</code>.</p></li>
                    <li><p>Do not use any constant numbers (or important names like filter names)
                        in the workhorse-Makefiles or paper's LaTeX source. Define such
                        constants as logically-grouped, separate configuration-Makefiles in
                        <code>reproduce/analysis/config/XXXXX.conf</code>. Then set this
                        configuration-Makefiles file as a prerequisite to any rule that uses
                        the variable defined in it.</p></li>
                    <li><p>Through any number of intermediate prerequisites, all processing steps
                        should end in (be a prerequisite of) <code>$(mtexdir)/verify.tex</code> (defined in
                        <code>reproduce/analysis/make/verify.mk</code>). <code>$(mtexdir)/verify.tex</code> is the sole
                        dependency of <code>$(mtexdir)/project.tex</code>, which is the bridge between the
                        processing steps and PDF-building steps of the project.</p></li>
                </ul>

                <h1>Customization checklist</h1>

                <p>Take the following steps to fully customize Maneage for your research
                project. After finishing the list, be sure to run <code>./project configure</code> and
                <code>project make</code> to see if everything works correctly. If you notice anything
                missing or any in-correct part (probably a change that has not been
                explained here), please let us know to correct it.</p>

                <p>As described above, the concept of reproducibility (during a project)
                heavily relies on <a href="https://en.wikipedia.org/wiki/Version_control">version
                    control</a>. Currently Maneage
                uses Git as its main version control system. If you are not already
                familiar with Git, please read the first three chapters of the <a href="https://git-scm.com/book/en/v2">ProGit
                    book</a> which provides a wonderful practical
                understanding of the basics. You can read later chapters as you get more
                advanced in later stages of your work.</p>

                <h2>First custom commit</h2>

                <ol>
                    <li><p><strong>Get this repository and its history</strong> (if you don't already have it):
                        Arguably the easiest way to start is to clone Maneage and prepare for
                        your customizations as shown below. After the cloning first you rename
                        the default <code>origin</code> remote server to specify that this is Maneage's
                        remote server. This will allow you to use the conventional <code>origin</code>
                        name for your own project as shown in the next steps. Second, you will
                        create and go into the conventional <code>master</code> branch to start
                        committing in your project later.</p>

                        <pre><code>git clone https://git.maneage.org/project.git    <span class="comment"># Clone/copy the project and its history.</span>
mv project my-project                            <span class="comment"># Change the name to your project's name.</span>
cd my-project                                    <span class="comment"># Go into the cloned directory.</span>
git remote rename origin origin-maneage          <span class="comment"># Rename current/only remote to "origin-maneage".</span>
git checkout -b master                           <span class="comment"># Create and enter your own "master" branch.</span>
pwd                                              <span class="comment"># Just to confirm where you are.</span></code></pre></li>
                    <li><p><strong>Prepare to build project</strong>: The <code>./project configure</code> command of the
                        next step will build the different software packages within the
                        "build" directory (that you will specify). Nothing else on your system
                        will be touched. However, since it takes long, it is useful to see
                        what it is being built at every instant (its almost impossible to tell
                        from the torrent of commands that are produced!). So open another
                        terminal on your desktop and navigate to the same project directory
                        that you cloned (output of last command above). Then run the following
                        command. Once every second, this command will just print the date
                        (possibly followed by a non-existent directory notice). But as soon as
                        the next step starts building software, you'll see the names of
                        software get printed as they are being built. Once any software is
                        installed in the project build directory it will be removed. Again,
                        don't worry, nothing will be installed outside the build directory.</p>

                        <pre><code><span class="comment"># On another terminal (go to top project source directory, last command above)</span>
./project --check-config</code></pre></li>
                    <li><p><strong>Test Maneage</strong>: Before making any changes, it is important to test it
                        and see if everything works properly with the commands below. If there
                        is any problem in the <code>./project configure</code> or <code>./project make</code> steps,
                        please contact us to fix the problem before continuing. Since the
                        building of dependencies in configuration can take long, you can take
                        the next few steps (editing the files) while its working (they don't
                        affect the configuration). After <code>./project make</code> is finished, open
                        <code>paper.pdf</code>. If it looks fine, you are ready to start customizing the
                        Maneage for your project. But before that, clean all the extra Maneage
                        outputs with <code>make clean</code> as shown below.</p>

                        <pre><code>./project configure     <span class="comment"># Build the project's software environment (can take an hour or so).</span>
./project make          <span class="comment"># Do the processing and build paper (just a simple demo).</span>
<span class="comment"># Open 'paper.pdf' and see if everything is ok.</code></pre></li>
                    <li><p><strong>Setup the remote</strong>: You can use any <a href="https://en.wikipedia.org/wiki/Comparison_of_source_code_hosting_facilities">hosting
                            facility</a>
                        that supports Git to keep an online copy of your project's version
                        controlled history. We recommend <a href="https://gitlab.com">GitLab</a> because
                        it is <a href="https://www.gnu.org/software/repo-criteria-evaluation.html">more ethical (although not
                            perfect)</a>,
                        and later you can also host GitLab on your own server. Anyway, create
                        an account in your favorite hosting facility (if you don't already
                        have one), and define a new project there. Please make sure <em>the newly
                            created project is empty</em> (some services ask to include a <code>README</code> in
                        a new project which is bad in this scenario, and will not allow you to
                        push to it). It will give you a URL (usually starting with <code>git@</code> and
                        ending in <code>.git</code>), put this URL in place of <code>XXXXXXXXXX</code> in the first
                        command below. With the second command, "push" your <code>master</code> branch to
                        your <code>origin</code> remote, and (with the <code>--set-upstream</code> option) set them
                        to track/follow each other. However, the <code>maneage</code> branch is currently
                        tracking/following your <code>origin-maneage</code> remote (automatically set
                        when you cloned Maneage). So when pushing the <code>maneage</code> branch to your
                        <code>origin</code> remote, you <em>shouldn't</em> use <code>--set-upstream</code>. With the last
                        command, you can actually check this (which local and remote branches
                        are tracking each other).</p>

                        <pre><code>git remote add origin XXXXXXXXXX        <span class="comment"># Newly created repo is now called 'origin'.</span>
git push --set-upstream origin master   <span class="comment"># Push 'master' branch to 'origin' (with tracking).</span>
git push origin maneage                 <span class="comment"># Push 'maneage' branch to 'origin' (no tracking).</span></code></pre></li>
                    <li><p><strong>Title</strong>, <strong>short description</strong> and <strong>author</strong>: The title and basic
                        information of your project's output PDF paper should be added in
                        <code>paper.tex</code>. You should see the relevant place in the preamble (prior
                        to <code>\begin{document}</code>. After you are done, run the <code>./project make</code>
                        command again to see your changes in the final PDF, and make sure that
                        your changes don't cause a crash in LaTeX. Of course, if you use a
                        different LaTeX package/style for managing the title and authors (in
                        particular a specific journal's style), please feel free to use it
                        your own methods after finishing this checklist and doing your first
                        commit.</p></li>
                    <li><p><strong>Delete dummy parts</strong>: Maneage contains some parts that are only for
                        the initial/test run, mainly as a demonstration of important steps,
                        which you can use as a reference to use in your own project. But they
                        not for any real analysis, so you should remove these parts as
                        described below:</p>

                        <ul>
                            <li><p><code>paper.tex</code>: 1) Delete the text of the abstract (from
                                <code>\includeabstract{</code> to <code>\vspace{0.25cm}</code>) and write your own (a
                                single sentence can be enough now, you can complete it later). 2)
                                Add some keywords under it in the keywords part. 3) Delete
                                everything between <code>%% Start of main body.</code> and <code>%% End of main
                                    body.</code>. 4) Remove the notice in the "Acknowledgments" section (in
                                <code>\new{}</code>) and Acknowledge your funding sources (this can also be
                                done later). Just don't delete the existing acknowledgment
                                statement: Maneage is possible thanks to funding from several
                                grants. Since Maneage is being used in your work, it is necessary to
                                acknowledge them in your work also.</p></li>
                            <li><p><code>reproduce/analysis/make/top-make.mk</code>: Delete the <code>delete-me</code> line
                                in the <code>makesrc</code> definition. Just make sure there is no empty line
                                between the <code>download \</code> and <code>verify \</code> lines (they should be
                                directly under each other).</p></li>
                            <li><p><code>reproduce/analysis/make/verify.mk</code>: In the final recipe, under the
                                commented line <code>Verify TeX macros</code>, remove the full line that
                                contains <code>delete-me</code>, and set the value of <code>s</code> in the line for
                                <code>download</code> to <code>XXXXX</code> (any temporary string, you'll fix it in the
                                end of your project, when its complete).</p></li>
                            <li><p>Delete all <code>delete-me*</code> files in the following directories:</p>
                                <pre><code>rm tex/src/delete-me*
rm reproduce/analysis/make/delete-me*
rm reproduce/analysis/config/delete-me*</code></pre></li>
                            <li><p>Disable verification of outputs by removing the <code>yes</code> from
                                <code>reproduce/analysis/config/verify-outputs.conf</code>. Later, when you are
                                ready to submit your paper, or publish the dataset, activate
                                verification and make the proper corrections in this file (described
                                under the "Other basic customizations" section below). This is a
                                critical step and only takes a few minutes when your project is
                                finished. So DON'T FORGET to activate it in the end.</p></li>
                            <li><p>Re-make the project (after a cleaning) to see if you haven't
                                introduced any errors.</p>

                                <pre><code>./project make clean
./project make</code></pre></li>
                        </ul></li>
                        <li><p><strong>Don't merge some files in future updates</strong>: As described below, you
                            can later update your infra-structure (for example to fix bugs) by
                            merging your <code>master</code> branch with <code>maneage</code>. For files that you have
                            created in your own branch, there will be no problem. However if you
                            modify an existing Maneage file for your project, next time its
                            updated on <code>maneage</code> you'll have an annoying conflict. The commands
                            below show how to fix this future problem. With them, you can
                            configure Git to ignore the changes in <code>maneage</code> for some of the files
                            you have already edited and deleted above (and will edit below). Note
                            that only the first <code>echo</code> command has a <code>&gt;</code> (to write over the file),
                            the rest are <code>&gt;&gt;</code> (to append to it). If you want to avoid any other
                            set of files to be imported from Maneage into your project's branch,
                            you can follow a similar strategy. We recommend only doing it when you
                            encounter the same conflict in more than one merge and there is no
                            other change in that file. Also, don't add core Maneage Makefiles,
                            otherwise Maneage can break on the next run.</p>

                            <pre><code>echo "paper.tex merge=ours" &gt; .gitattributes
echo "tex/src/delete-me.mk merge=ours" &gt;&gt; .gitattributes
echo "tex/src/delete-me-demo.mk merge=ours" &gt;&gt; .gitattributes
echo "reproduce/analysis/make/delete-me.mk merge=ours" &gt;&gt; .gitattributes
echo "reproduce/software/config/TARGETS.conf merge=ours" &gt;&gt; .gitattributes
echo "reproduce/analysis/config/delete-me-num.conf merge=ours" &gt;&gt; .gitattributes
git add .gitattributes</code></pre></li>
                        <li><p><strong>Copyright and License notice</strong>: It is necessary that <em>all</em> the
                            "copyright-able" files in your project (those larger than 10 lines)
                            have a copyright and license notice. Please take a moment to look at
                            several existing files to see a few examples. The copyright notice is
                            usually close to the start of the file, it is the line starting with
                            <code>Copyright (C)</code> and containing a year and the author's name (like the
                            examples below). The License notice is a short description of the
                            copyright license, usually one or two paragraphs with a URL to the
                            full license. Don't forget to add these <em>two</em> notices to <em>any new
                                file</em> you add in your project (you can just copy-and-paste). When you
                            modify an existing Maneage file (which already has the notices), just
                            add a copyright notice in your name under the existing one(s), like
                            the line with capital letters below. To start with, add this line with
                            your name and email address to <code>paper.tex</code>,
                            <code>tex/src/preamble-header.tex</code>, <code>reproduce/analysis/make/top-make.mk</code>,
                            and generally, all the files you modified in the previous step.</p>

                            <pre><code>Copyright (C) 2018-2020 Existing Name &lt;existing@email.address&gt;
Copyright (C) 2020 YOUR NAME &lt;YOUR@EMAIL.ADDRESS&gt;</code></pre></li>
                        <li><p><strong>Configure Git for fist time</strong>: If this is the first time you are
                            running Git on this system, then you have to configure it with some
                            basic information in order to have essential information in the commit
                            messages (ignore this step if you have already done it). Git will
                            include your name and e-mail address information in each commit. You
                            can also specify your favorite text editor for making the commit
                            (<code>emacs</code>, <code>vim</code>, <code>nano</code>, and etc.).</p>

                            <pre><code>git config --global user.name "YourName YourSurname"
git config --global user.email your-email@example.com
git config --global core.editor nano</code></pre></li>
                        <li><p><strong>Your first commit</strong>: You have already made some small and basic
                            changes in the steps above and you are in your project's <code>master</code>
                            branch. So, you can officially make your first commit in your
                            project's history and push it. But before that, you need to make sure
                            that there are no problems in the project. This is a good habit to
                            always re-build the system before a commit to be sure it works as
                            expected.</p>

                            <pre><code>git status                 <span class="comment"># See which files you have changed.</span>
git diff                   <span class="comment"># Check the lines you have added/changed.</span>
./project make             <span class="comment"># Make sure everything builds successfully.</span>
git add -u                 <span class="comment"># Put all tracked changes in staging area.</span>
git status                 <span class="comment"># Make sure everything is fine.</span>
git diff --cached          <span class="comment"># Confirm all the changes that will be committed.</span>
git commit                 <span class="comment"># Your first commit: put a good description!</span>
git push                   <span class="comment"># Push your commit to your remote.</span></code></pre></li>
                        <li><p><strong>Start your exciting research</strong>: You are now ready to add flesh and
                            blood to this raw skeleton by further modifying and adding your
                            exciting research steps. You can use the "published works" section in
                            the introduction (above) as some fully working models to learn
                            from. Also, don't hesitate to contact us if you have any
                            questions.</p></li>
                </ol>

                <h2>Other basic customizations</h2>

                <ul>
                    <li><p><strong>High-level software</strong>: Maneage installs all the software that your
                        project needs. You can specify which software your project needs in
                        <code>reproduce/software/config/TARGETS.conf</code>. The necessary software are
                        classified into two classes: 1) programs or libraries (usually written
                        in C/C++) which are run directly by the operating system. 2) Python
                        modules/libraries that are run within Python. By default
                        <code>TARGETS.conf</code> only has GNU Astronomy Utilities (Gnuastro) as one
                        scientific program and Astropy as one scientific Python module. Both
                        have many dependencies which will be installed into your project
                        during the configuration step. To see a list of software that are
                        currently ready to be built in Maneage, see
                        <code>reproduce/software/config/versions.conf</code> (which has their versions
                        also), the comments in <code>TARGETS.conf</code> describe how to use the software
                        name from <code>versions.conf</code>. Currently the raw pipeline just uses
                        Gnuastro to make the demonstration plots. Therefore if you don't need
                        Gnuastro, go through the analysis steps in <code>reproduce/analysis</code> and
                        remove all its use cases (clearly marked).</p></li>
                    <li><p><strong>Input dataset</strong>: The input datasets are managed through the
                        <code>reproduce/analysis/config/INPUTS.conf</code> file. It is best to gather all
                        the information regarding all the input datasets into this one central
                        file. To ensure that the proper dataset is being downloaded and used
                        by the project, it is also recommended get an <a href="https://en.wikipedia.org/wiki/MD5">MD5
                            checksum</a> of the file and include
                        that in <code>INPUTS.conf</code> so the project can check it automatically. The
                        preparation/downloading of the input datasets is done in
                        <code>reproduce/analysis/make/download.mk</code>. Have a look there to see how
                        these values are to be used. This information about the input datasets
                        is also used in the initial <code>configure</code> script (to inform the users),
                        so also modify that file. You can find all occurrences of the demo
                        dataset with the command below and replace it with your input's
                        dataset.</p>

                        <pre><code>grep -ir wfpc2 ./*</code></pre></li>
                    <li><p><strong><code>README.md</code></strong>: Correct all the <code>XXXXX</code> place holders (name of your
                        project, your own name, address of your project's online/remote
                        repository, link to download dependencies and etc). Generally, read
                        over the text and update it where necessary to fit your project. Don't
                        forget that this is the first file that is displayed on your online
                        repository and also your colleagues will first be drawn to read this
                        file. Therefore, make it as easy as possible for them to start
                        with. Also check and update this file one last time when you are ready
                        to publish your project's paper/source.</p></li>
                    <li><p><strong>Verify outputs</strong>: During the initial customization checklist, you
                        disabled verification. This is natural because during the project you
                        need to make changes all the time and its a waste of time to enable
                        verification every time. But at significant moments of the project
                        (for example before submission to a journal, or publication) it is
                        necessary. When you activate verification, before building the paper,
                        all the specified datasets will be compared with their respective
                        checksum and if any file's checksum is different from the one recorded
                        in the project, it will stop and print the problematic file and its
                        expected and calculated checksums. First set the value of
                        <code>verify-outputs</code> variable in
                        <code>reproduce/analysis/config/verify-outputs.conf</code> to <code>yes</code>. Then go to
                        <code>reproduce/analysis/make/verify.mk</code>. The verification of all the files
                        is only done in one recipe. First the files that go into the
                        plots/figures are checked, then the LaTeX macros. Validation of the
                        former (inputs to plots/figures) should be done manually. If its the
                        first time you are doing this, you can see two examples of the dummy
                        steps (with <code>delete-me</code>, you can use them if you like). These two
                        examples should be removed before you can run the project. For the
                        latter, you just have to update the checksums. The important thing to
                        consider is that a simple checksum can be problematic because some
                        file generators print their run-time date in the file (for example as
                        commented lines in a text table). When checking text files, this
                        Makefile already has this function:
                        <code>verify-txt-no-comments-leading-space</code>. As the name suggests, it will
                        remove comment lines and empty lines before calculating the MD5
                        checksum. For FITS formats (common in astronomy, fortunately there is
                        a <code>DATASUM</code> definition which will return the checksum independent of
                        the headers. You can use the provided function(s), or define one for
                        your special formats.</p></li>
                    <li><p><strong>Feedback</strong>: As you use Maneage you will notice many things that if
                        implemented from the start would have been very useful for your
                        work. This can be in the actual scripting and architecture of Maneage,
                        or useful implementation and usage tips, like those below. In any
                        case, please share your thoughts and suggestions with us, so we can
                        add them here for everyone's benefit.</p></li>
                    <li><p><strong>Re-preparation</strong>: Automatic preparation is only run in the first run
                        of the project on a system, to re-do the preparation you have to use
                        the option below. Here is the reason for this: when its necessary, the
                        preparation process can be slow and will unnecessarily slow down the
                        whole project while the project is under development (focus is on the
                        analysis that is done after preparation). Because of this, preparation
                        will be done automatically for the first time that the project is run
                        (when <code>.build/software/preparation-done.mk</code> doesn't exist). After the
                        preparation process completes once, future runs of <code>./project make</code>
                        will not do the preparation process anymore (will not call
                        <code>top-prepare.mk</code>). They will only call <code>top-make.mk</code> for the
                        analysis. To manually invoke the preparation process after the first
                        attempt, the <code>./project make</code> script should be run with the
                        <code>--prepare-redo</code> option, or you can delete the special file above.</p>

                        <pre><code>./project make --prepare-redo</code></pre></li>
                    <li><p><strong>Pre-publication</strong>: add notice on reproducibility**: Add a notice
                        somewhere prominent in the first page within your paper, informing the
                        reader that your research is fully reproducible. For example in the
                        end of the abstract, or under the keywords with a title like
                        "reproducible paper". This will encourage them to publish their own
                        works in this manner also and also will help spread the word.</p></li>
                </ul>

                <h1>Tips for designing your project</h1>

                <p>The following is a list of design points, tips, or recommendations that
                have been learned after some experience with this type of project
                management. Please don't hesitate to share any experience you gain after
                using it with us. In this way, we can add it here (with full giving credit)
                for the benefit of others.</p>

                <ul>
                    <li><p><strong>Modularity</strong>: Modularity is the key to easy and clean growth of a
                        project. So it is always best to break up a job into as many
                        sub-components as reasonable. Here are some tips to stay modular.</p>

                        <ul>
                            <li><p><em>Short recipes</em>: if you see the recipe of a rule becoming more than a
                                handful of lines which involve significant processing, it is probably
                                a good sign that you should break up the rule into its main
                                components. Try to only have one major processing step per rule.</p></li>
                            <li><p><em>Context-based (many) Makefiles</em>: For maximum modularity, this design
                                allows easy inclusion of many Makefiles: in
                                <code>reproduce/analysis/make/*.mk</code> for analysis steps, and
                                <code>reproduce/software/make/*.mk</code> for building software. So keep the
                                rules for closely related parts of the processing in separate
                                Makefiles.</p></li>
                            <li><p><em>Descriptive names</em>: Be very clear and descriptive with the naming of
                                the files and the variables because a few months after the
                                processing, it will be very hard to remember what each one was
                                for. Also this helps others (your collaborators or other people
                                reading the project source after it is published) to more easily
                                understand your work and find their way around.</p></li>
                            <li><p><em>Naming convention</em>: As the project grows, following a single standard
                                or convention in naming the files is very useful. Try best to use
                                multiple word filenames for anything that is non-trivial (separating
                                the words with a <code>-</code>). For example if you have a Makefile for
                                creating a catalog and another two for processing it under models A
                                and B, you can name them like this: <code>catalog-create.mk</code>,
                                <code>catalog-model-a.mk</code> and <code>catalog-model-b.mk</code>. In this way, when
                                listing the contents of <code>reproduce/analysis/make</code> to see all the
                                Makefiles, those related to the catalog will all be close to each
                                other and thus easily found. This also helps in auto-completions by
                                the shell or text editors like Emacs.</p></li>
                            <li><p><em>Source directories</em>: If you need to add files in other languages for
                                example in shell, Python, AWK or C, keep the files in the same
                                language in a separate directory under <code>reproduce/analysis</code>, with the
                                appropriate name.</p></li>
                            <li><p><em>Configuration files</em>: If your research uses special programs as part
                                of the processing, put all their configuration files in a devoted
                                directory (with the program's name) within
                                <code>reproduce/software/config</code>. Similar to the
                                <code>reproduce/software/config/gnuastro</code> directory (which is put in
                                Maneage as a demo in case you use GNU Astronomy Utilities). It is
                                much cleaner and readable (thus less buggy) to avoid mixing the
                                configuration files, even if there is no technical necessity.</p></li>
                        </ul></li>
                        <li><p><strong>Contents</strong>: It is good practice to follow the following
                            recommendations on the contents of your files, whether they are source
                            code for a program, Makefiles, scripts or configuration files
                            (copyrights aren't necessary for the latter).</p>

                            <ul>
                                <li><p><em>Copyright</em>: Always start a file containing programming constructs
                                    with a copyright statement like the ones that Maneage starts with
                                    (for example in the top level <code>Makefile</code>).</p></li>
                                <li><p><em>Comments</em>: Comments are vital for readability (by yourself in two
                                    months, or others). Describe everything you can about why you are
                                    doing something, how you are doing it, and what you expect the result
                                    to be. Write the comments as if it was what you would say to describe
                                    the variable, recipe or rule to a friend sitting beside you. When
                                    writing the project it is very tempting to just steam ahead with
                                    commands and codes, but be patient and write comments before the
                                    rules or recipes. This will also allow you to think more about what
                                    you should be doing. Also, in several months when you come back to
                                    the code, you will appreciate the effort of writing them. Just don't
                                    forget to also read and update the comment first if you later want to
                                    make changes to the code (variable, recipe or rule). As a general
                                    rule of thumb: first the comments, then the code.</p></li>
                                <li><p><em>File title</em>: In general, it is good practice to start all files with
                                    a single line description of what that particular file does. If
                                    further information about the totality of the file is necessary, add
                                    it after a blank line. This will help a fast inspection where you
                                    don't care about the details, but just want to remember/see what that
                                    file is (generally) for. This information must of course be commented
                                    (its for a human), but this is kept separate from the general
                                    recommendation on comments, because this is a comment for the whole
                                    file, not each step within it.</p></li>
                            </ul></li>
                            <li><p><strong>Make programming</strong>: Here are some experiences that we have come to
                                learn over the years in using Make and are useful/handy in research
                                contexts.</p>

                                <ul>
                                    <li><p><em>Environment of each recipe</em>: If you need to define a special
                                        environment (or aliases, or scripts to run) for all the recipes in
                                        your Makefiles, you can use a Bash startup file
                                        <code>reproduce/software/shell/bashrc.sh</code>. This file is loaded before every
                                        Make recipe is run, just like the <code>.bashrc</code> in your home directory is
                                        loaded every time you start a new interactive, non-login terminal. See
                                        the comments in that file for more.</p></li>
                                    <li><p><em>Automatic variables</em>: These are wonderful and very useful Make
                                        constructs that greatly shrink the text, while helping in
                                        read-ability, robustness (less bugs in typos for example) and
                                        generalization. For example even when a rule only has one target or
                                        one prerequisite, always use <code>$@</code> instead of the target's name, <code>$&lt;</code>
                                        instead of the first prerequisite, <code>$^</code> instead of the full list of
                                        prerequisites and etc. You can see the full list of automatic
                                        variables
                                        <a href="https://www.gnu.org/software/make/manual/html_node/Automatic-Variables.html">here</a>. If
                                        you use GNU Make, you can also see this page on your command-line:</p>

                                        <pre><code>info make "automatic variables"</code></pre></li>
                                    <li><p><em>Debug</em>: Since Make doesn't follow the common top-down paradigm, it
                                        can be a little hard to get accustomed to why you get an error or
                                        un-expected behavior. In such cases, run Make with the <code>-d</code>
                                        option. With this option, Make prints a full list of exactly which
                                        prerequisites are being checked for which targets. Looking
                                        (patiently) through this output and searching for the faulty
                                        file/step will clearly show you any mistake you might have made in
                                        defining the targets or prerequisites.</p></li>
                                    <li><p><em>Large files</em>: If you are dealing with very large files (thus having
                                        multiple copies of them for intermediate steps is not possible), one
                                        solution is the following strategy (Also see the next item on "Fast
                                        access to temporary files"). Set a small plain text file as the
                                        actual target and delete the large file when it is no longer needed
                                        by the project (in the last rule that needs it). Below is a simple
                                        demonstration of doing this. In it, we use Gnuastro's Arithmetic
                                        program to add all pixels of the input image with 2 and create
                                        <code>large1.fits</code>. We then subtract 2 from <code>large1.fits</code> to create
                                        <code>large2.fits</code> and delete <code>large1.fits</code> in the same rule (when its no
                                        longer needed). We can later do the same with <code>large2.fits</code> when it
                                        is no longer needed and so on.
                                        <pre><code>large1.fits.txt: input.fits
astarithmetic $&lt; 2 + --output=$(subst .txt,,$@)
echo "done" &gt; $@
large2.fits.txt: large1.fits.txt
astarithmetic $(subst .txt,,$&lt;) 2 - --output=$(subst .txt,,$@)
rm $(subst .txt,,$&lt;)
echo "done" &gt; $@</code></pre>
                                        A more advanced Make programmer will use Make's <a href="https://www.gnu.org/software/make/manual/html_node/Call-Function.html">call function</a>
                                        to define a wrapper in <code>reproduce/analysis/make/initialize.mk</code>. This
                                        wrapper will replace <code>$(subst .txt,,XXXXX)</code>. Therefore, it will be
                                        possible to greatly simplify this repetitive statement and make the
                                        code even more readable throughout the whole project.</p></li>
                                    <li><p><em>Fast access to temporary files</em>: Most Unix-like operating systems
                                        will give you a special shared-memory device (directory): on systems
                                        using the GNU C Library (all GNU/Linux system), it is <code>/dev/shm</code>. The
                                        contents of this directory are actually in your RAM, not in your
                                        persistence storage like the HDD or SSD. Reading and writing from/to
                                        the RAM is much faster than persistent storage, so if you have enough
                                        RAM available, it can be very beneficial for large temporary files to
                                        be put there. You can use the <code>mktemp</code> program to give the temporary
                                        files a randomly-set name, and use text files as targets to keep that
                                        name (as described in the item above under "Large files") for later
                                        deletion. For example, see the minimal working example Makefile below
                                        (which you can actually put in a <code>Makefile</code> and run if you have an
                                        <code>input.fits</code> in the same directory, and Gnuastro is installed).
                                        <pre><code>.ONESHELL:
.SHELLFLAGS = -ec
all: mean-std.txt
shm-maneage := /dev/shm/$(shell whoami)-maneage-XXXXXXXXXX
large1.txt: input.fits
out=$$(mktemp $(shm-maneage))
astarithmetic $&lt; 2 + --output=$$out.fits
echo "$$out" &gt; $@
large2.txt: large1.txt
input=$$(cat $&lt;)
out=$$(mktemp $(shm-maneage))
astarithmetic $$input.fits 2 - --output=$$out.fits
rm $$input.fits $$input
echo "$$out" &gt; $@
mean-std.txt: large2.txt
input=$$(cat $&lt;)
aststatistics $$input.fits --mean --std &gt; $@
rm $$input.fits $$input</code></pre>
                                        The important point here is that the temporary name template
                                        (<code>shm-maneage</code>) has no suffix. So you can add the suffix
                                        corresponding to your desired format afterwards (for example
                                        <code>$$out.fits</code>, or <code>$$out.txt</code>). But more importantly, when <code>mktemp</code>
                                        sets the random name, it also checks if no file exists with that name
                                        and creates a file with that exact name at that moment. So at the end
                                        of each recipe above, you'll have two files in your <code>/dev/shm</code>, one
                                        empty file with no suffix one with a suffix. The role of the file
                                        without a suffix is just to ensure that the randomly set name will
                                        not be used by other calls to <code>mktemp</code> (when running in parallel) and
                                        it should be deleted with the file containing a suffix. This is the
                                        reason behind the <code>rm $$input.fits $$input</code> command above: to make
                                        sure that first the file with a suffix is deleted, then the core
                                        random file (note that when working in parallel on powerful systems,
                                        in the time between deleting two files of a single <code>rm</code> command, many
                                        things can happen!). When using Maneage, you can put the definition
                                        of <code>shm-maneage</code> in <code>reproduce/analysis/make/initialize.mk</code> to be
                                        usable in all the different Makefiles of your analysis, and you won't
                                        need the three lines above it. <strong>Finally, BE RESPONSIBLE:</strong> after you
                                        are finished, be sure to clean up any possibly remaining files (due
                                        to crashes in the processing while you are working), otherwise your
                                        RAM may fill up very fast. You can do it easily with a command like
                                        this on your command-line: <code>rm -f /dev/shm/$(whoami)-*</code>.</p></li>
                                </ul></li>
                                <li><p><strong>Software tarballs and raw inputs</strong>: It is critically important to
                                    document the raw inputs to your project (software tarballs and raw
                                    input data):</p>

                                    <ul>
                                        <li><p><em>Keep the source tarball of dependencies</em>: After configuration
                                            finishes, the <code>.build/software/tarballs</code> directory will contain all
                                            the software tarballs that were necessary for your project. You can
                                            mirror the contents of this directory to keep a backup of all the
                                            software tarballs used in your project (possibly as another version
                                            controlled repository) that is also published with your project. Note
                                            that software web-pages are not written in stone and can suddenly go
                                            offline or not be accessible in some conditions. This backup is thus
                                            very important. If you intend to release your project in a place like
                                            Zenodo, you can upload/keep all the necessary tarballs (and data)
                                            there with your
                                            project. <a href="https://doi.org/10.5281/zenodo.1163746">zenodo.1163746</a> is
                                            one example of how the data, Gnuastro (main software used) and all
                                            major Gnuastro's dependencies have been uploaded with the project's
                                            source. Just note that this is only possible for free and open-source
                                            software.</p></li>
                                        <li><p><em>Keep your input data</em>: The input data is also critical to the
                                            project's reproducibility, so like the above for software, make sure
                                            you have a backup of them, or their persistent identifiers (PIDs).</p></li>
                                    </ul></li>
                                    <li><p><strong>Version control</strong>: Version control is a critical component of
                                        Maneage. Here are some tips to help in effectively using it.</p>

                                        <ul>
                                            <li><p><em>Regular commits</em>: It is important (and extremely useful) to have the
                                                history of your project under version control. So try to make commits
                                                regularly (after any meaningful change/step/result).</p></li>
                                            <li><p><em>Keep Maneage up-to-date</em>: In time, Maneage is going to become more
                                                and more mature and robust (thanks to your feedback and the feedback
                                                of other users). Bugs will be fixed and new/improved features will be
                                                added. So every once and a while, you can run the commands below to
                                                pull new work that is done in Maneage. If the changes are useful for
                                                your work, you can merge them with your project to benefit from
                                                them. Just pay <strong>very close attention</strong> to resolving possible
                                                <strong>conflicts</strong> which might happen in the merge (updated settings that
                                                you have customized in Maneage).</p>

                                                <pre><code>git checkout maneage
git pull                            <span class="comment"># Get recent work in Maneage</span>
git log XXXXXX..XXXXXX --reverse    <span class="comment"># Inspect new work (replace XXXXXXs with hashs mentioned in output of previous command).</span>
git log --oneline --graph --decorate --all <span class="comment"># General view of branches.</span>
git checkout master                 <span class="comment"># Go to your top working branch.</span>
git merge maneage                   <span class="comment"># Import all the work into master.</span></code></pre></li>
                                            <li><p><em>Adding Maneage to a fork of your project</em>: As you and your colleagues
                                                continue your project, it will be necessary to have separate
                                                forks/clones of it. But when you clone your own project on a
                                                different system, or a colleague clones it to collaborate with you,
                                                the clone won't have the <code>origin-maneage</code> remote that you started the
                                                project with. As shown in the previous item above, you need this
                                                remote to be able to pull recent updates from Maneage. The steps
                                                below will setup the <code>origin-maneage</code> remote, and a local <code>maneage</code>
                                                branch to track it, on the new clone.</p>

                                                <pre><code>git remote add origin-maneage https://git.maneage.org/project.git
git fetch origin-maneage
git checkout -b maneage --track origin-maneage/maneage</code></pre></li>
                                            <li><p><em>Commit message</em>: The commit message is a very important and useful
                                                aspect of version control. To make the commit message useful for
                                                others (or yourself, one year later), it is good to follow a
                                                consistent style. Maneage already has a consistent formatting
                                                (described below), which you can also follow in your project if you
                                                like. You can see many examples by running <code>git log</code> in the <code>maneage</code>
                                                branch. If you intend to push commits to Maneage, for the consistency
                                                of Maneage, it is necessary to follow these guidelines. 1) No line
                                                should be more than 75 characters (to enable easy reading of the
                                                message when you run <code>git log</code> on the standard 80-character
                                                terminal). 2) The first line is the title of the commit and should
                                                summarize it (so <code>git log --oneline</code> can be useful). The title should
                                                also not end with a point (<code>.</code>, because its a short single sentence,
                                                so a point is not necessary and only wastes space). 3) After the
                                                title, leave an empty line and start the body of your message
                                                (possibly containing many paragraphs). 4) Describe the context of
                                                your commit (the problem it is trying to solve) as much as possible,
                                                then go onto how you solved it. One suggestion is to start the main
                                                body of your commit with "Until now ...", and continue describing the
                                                problem in the first paragraph(s). Afterwards, start the next
                                                paragraph with "With this commit ...".</p></li>
                                            <li><p><em>Project outputs</em>: During your research, it is possible to checkout a
                                                specific commit and reproduce its results. However, the processing
                                                can be time consuming. Therefore, it is useful to also keep track of
                                                the final outputs of your project (at minimum, the paper's PDF) in
                                                important points of history.  However, keeping a snapshot of these
                                                (most probably large volume) outputs in the main history of the
                                                project can unreasonably bloat it. It is thus recommended to make a
                                                separate Git repo to keep those files and keep your project's source
                                                as small as possible. For example if your project is called
                                                <code>my-exciting-project</code>, the name of the outputs repository can be
                                                <code>my-exciting-project-output</code>. This enables easy sharing of the output
                                                files with your co-authors (with necessary permissions) and not
                                                having to bloat your email archive with extra attachments also (you
                                                can just share the link to the online repo in your
                                                communications). After the research is published, you can also
                                                release the outputs repository, or you can just delete it if it is
                                                too large or un-necessary (it was just for convenience, and fully
                                                reproducible after all). For example Maneage's output is available
                                                for demonstration in <a href="http://git.maneage.org/output-raw.git/">a
                                                    separate</a> repository.</p></li>
                                            <li><p><em>Full Git history in one file</em>: When you are publishing your project
                                                (for example to Zenodo for long term preservation), it is more
                                                convenient to have the whole project's Git history into one file to
                                                save with your datasets. After all, you can't be sure that your
                                                current Git server (for example GitLab, Github, or Bitbucket) will be
                                                active forever. While they are good for the immediate future, you
                                                can't rely on them for archival purposes. Fortunately keeping your
                                                whole history in one file is easy with Git using the following
                                                commands. To learn more about it, run <code>git help bundle</code>.</p>

                                                <ul>
                                                    <li>"bundle" your project's history into one file (just don't forget to
                                                        change <code>my-project-git.bundle</code> to a descriptive name of your
                                                        project):</li>
                                                </ul>

                                                <pre><code>git bundle create my-project-git.bundle --all</code></pre>

                                                <ul>
                                                    <li>You can easily upload <code>my-project-git.bundle</code> anywhere. Later, if
                                                        you need to un-bundle it, you can use the following command.</li>
                                                </ul>

                                                <p><p><pre><code>git clone my-project-git.bundle</code></pre></li>
                                        </ul></p></li>
                </ul></p>

                <h1>Future improvements</h1>

                <p>This is an evolving project and as time goes on, it will evolve and become
                more robust. Some of the most prominent issues we plan to implement in the
                future are listed below, please join us if you are interested.</p>

                <h2>Package management</h2>

                <p>It is important to have control of the environment of the project. Maneage
                currently builds the higher-level programs (for example GNU Bash, GNU Make,
                GNU AWK and domain-specific software) it needs, then sets <code>PATH</code> so the
                analysis is done only with the project's built software. But currently the
                configuration of each program is in the Makefile rules that build it. This
                is not good because a change in the build configuration does not
                automatically cause a re-build. Also, each separate project on a system
                needs to have its own built tools (that can waste a lot of space).</p>

                <p>A good solution is based on the <a href="https://nixos.org/nix/about.html">Nix package manager</a>: a separate file is present for
                each software, containing all the necessary info to build it (including its
                URL, its tarball MD5 hash, dependencies, configuration parameters, build
                steps and etc). Using this file, a script can automatically generate the
                Make rules to download, build and install program and its dependencies
                (along with the dependencies of those dependencies and etc).</p>

                <p>All the software are installed in a "store". Each installed file (library
                or executable) is prefixed by a hash of this configuration (and the OS
                architecture) and the standard program name. For example (from the Nix
                webpage):</p>

                <pre><code>/nix/store/b6gvzjyb2pg0kjfwrjmg1vfhh54ad73z-firefox-33.1/</code></pre>

                <p>The important thing is that the "store" is <em>not</em> in the project's search
                path. After the complete installation of the software, symbolic links are
                made to populate each project's program and library search paths without a
                hash. This hash will be unique to that particular software and its
                particular configuration. So simply by searching for this hash in the
                installed directory, we can find the installed files of that software to
                generate the links.</p>

                <p>This scenario has several advantages: 1) a change in a software's build
                configuration triggers a rebuild. 2) a single "store" can be used in many
                projects, thus saving space and configuration time for new projects (that
                commonly have large overlaps in lower-level programs).</p>

                <h1>Appendix: Necessity of exact reproduction in scientific research</h1>

                <p>In case <a href="http://akhlaghi.org/reproducible-science.html">the link above</a> is
                not accessible at the time of reading, here is a copy of the introduction
                of that link, describing the necessity for a reproducible project like this
                (copied on February 7th, 2018):</p>

                <p>The most important element of a "scientific" statement/result is the fact
                that others should be able to falsify it. The Tsunami of data that has
                engulfed astronomers in the last two decades, combined with faster
                processors and faster internet connections has made it much more easier to
                obtain a result. However, these factors have also increased the complexity
                of a scientific analysis, such that it is no longer possible to describe
                all the steps of an analysis in the published paper. Citing this
                difficulty, many authors suffice to describing the generalities of their
                analysis in their papers.</p>

                <p>However, It is impossible to falsify (or even study) a result if you can't
                exactly reproduce it. The complexity of modern science makes it vitally
                important to exactly reproduce the final result. Because even a small
                deviation can be due to many different parts of an analysis. Nature is
                already a black box which we are trying so hard to comprehend. Not letting
                other scientists see the exact steps taken to reach a result, or not
                allowing them to modify it (do experiments on it) is a self-imposed black
                box, which only exacerbates our ignorance.</p>

                <p>Other scientists should be able to reproduce, check and experiment on the
                results of anything that is to carry the "scientific" label. Any result
                that is not reproducible (due to incomplete information by the author) is
                not scientific: the readers have to have faith in the subjective experience
                of the authors in the very important choice of configuration values and
                order of operations: this is contrary to the scientific spirit.</p>

                <h2>Copyright information</h2>

                <p>This file is part of Maneage's core: <a href="https://git.maneage.org/project.git">https://git.maneage.org/project.git</a></p>

                <p>Maneage is free software: you can redistribute it and/or modify it under
                the terms of the GNU General Public License as published by the Free
                Software Foundation, either version 3 of the License, or (at your option)
                any later version.</p>

                <p>Maneage is distributed in the hope that it will be useful, but WITHOUT ANY
                WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
                FOR A PARTICULAR PURPOSE. See the GNU General Public License for more
                details.</p>

                <p>You should have received a copy of the GNU General Public License along
                with Maneage.  If not, see <a href="https://www.gnu.org/licenses/">https://www.gnu.org/licenses/</a>.</p>
            </div>
        </body>