IMPORTANT: Added publication checklist, improved relevant infrastructure

Possible semantic conflicts (that may not show up as Git conflicts but may
cause a crash in your project after the merge):

   1) The project title (and other basic metadata) should be set in
      'reproduce/analysis/conf/metadata.conf'. Please include this file in
      your merge (if it is ignored because of '.gitattributes'!).

   2) Consider importing the changes in 'initialize.mk' and 'verify.mk' (if
      you have added all analysis Makefiles to the '.gitattributes' file
      (thus not merging any change in them with your branch). For example
      with this command:
        git diff master...maneage -- reproduce/analysis/make/initialize.mk

   3) The old 'verify-txt-no-comments-leading-space' function has been
      replaced by 'verify-txt-no-comments-no-space'. The new function will
      also remove all white-space characters between the columns (not just
      white space characters at the start of the line). Thus the resulting
      check won't involve spacing between columns.

A common set of steps are always necessary to prepare a project for
publication. Until now, we would simply look at previous submissions and
try to follow them, but that was prone to errors and could cause
confusion. The internal infrastructure also didn't have some useful
features to make good publication possible. Now that the submission of a
paper fully devoted to the founding criteria of Maneage is complete
(arXiv:2006.03018), it was time to formalize the necessary steps for easier
submission of a project using Maneage and implement some low-level features
that can make things easier.

With this commit a first draft of the publication checklist has been added
to 'README-hacking.md', it was tested in the submission of arXiv:2006.03018
and zenodo.3872248. To help guide users on implementing the good practices
for output datasets, the outputs of the default project shown in the paper
now use the new features). After reading the checklist, please inspect
these.

Some other relevant changes in this commit:

  - The publication involves a copy of the necessary software
    tarballs. Hence a new target ('dist-software') was also added to
    package all the project's software tarballs in one tarball for easy
    distribution.

  - A new 'dist-lzip' target has been defined for those who want to
    distribute an Lzip-compressed tarball.

  - The '\includetikz' LaTeX macro now has a second argument to allow
    configuring the '\includegraphics' call when the plot should not be
    built, but just imported.

1 files changed, 69 insertions, 29 deletions

diff --git a/reproduce/analysis/make/delete-me.mk b/reproduce/analysis/make/delete-me.mk
index fa16102..f45f9ea 100644
--- a/reproduce/analysis/make/delete-me.mk
+++ b/reproduce/analysis/make/delete-me.mk
@@ -22,18 +22,40 @@
 # Dummy dataset
 # -------------
 #
-# We will use AWK to generate a table showing X and X^2 and draw its plot.
-delete-numdir = $(texdir)/delete-me-num
-delete-num    = $(delete-numdir)/data.txt
-$(delete-numdir): | $(texdir); mkdir $@
-$(delete-num): $(pconfdir)/delete-me-num.conf | $(delete-numdir)
+# Just as a demonstration(!): we will use AWK to generate a table showing X
+# and X^2 and draw its plot.
+#
+# Note that this dataset is directly read by LaTeX to generate a plot, so
+# we need to put it in the $(tex-publish-dir) directory.
+dm-squared = $(tex-publish-dir)/squared.txt
+$(dm-squared): $(pconfdir)/delete-me-squared-num.conf | $(tex-publish-dir)
 
         # When the plotted values are re-made, it is necessary to also
-        # delete the TiKZ externalized files so the plot is also re-made.
-	rm -f $(tikzdir)/delete-me.pdf
+        # delete the TiKZ externalized files so the plot is also re-made by
+        # PGFPlots.
+	rm -f $(tikzdir)/delete-me-squared.pdf
+
+        # Write the column metadata in a temporary file name (appending
+        # '.tmp' to the actual target name). Once all steps are done, it is
+        # renamed to the final target. We do this because if there is an
+        # error in the middle, Make will not consider the job to be
+        # complete and will stop here.
+	echo "# Data for demonstration plot of default Maneage (MANaging data linEAGE)." > $@.tmp
+	echo "# It is a simple plot, showing the power of two: y=x^2! " >> $@.tmp
+	echo "# " >> $@.tmp
+	echo "# Column 1: X       [arbitrary, f32] The horizontal axis numbers." \
+	     >> $@.tmp
+	echo "# Column 2: X_POW2  [arbitrary, f32] The horizontal axis to the power of two." \
+	     >> $@.tmp
+	echo "# " >> $@.tmp
+	$(call print-copyright, $@.tmp)
 
         # Generate the table of random values.
-	awk 'BEGIN {for(i=1;i<=$(delete-me-num);i+=0.5) print i, i*i; }' > $@
+	awk 'BEGIN {for(i=1;i<=$(delete-me-squared-num);i+=0.5) \
+	              printf("%-8.1f%.2f\n", i, i*i); }' >> $@.tmp
+
+        # Write it into the final target
+	mv $@.tmp $@
 
 
 
@@ -44,14 +66,14 @@ $(delete-num): $(pconfdir)/delete-me-num.conf | $(delete-numdir)
 #
 # For an example image, we'll make a PDF copy of the WFPC II image to
 # display in the paper.
-delete-demodir = $(texdir)/delete-me-demo
-$(delete-demodir): | $(texdir); mkdir $@
-delete-pdf = $(delete-demodir)/wfpc2.pdf
-$(delete-pdf): $(delete-demodir)/%.pdf: $(indir)/%.fits | $(delete-demodir)
+dm-histdir = $(texdir)/image-histogram
+$(dm-histdir): | $(texdir); mkdir $@
+dm-img-pdf = $(dm-histdir)/wfpc2.pdf
+$(dm-img-pdf): $(dm-histdir)/%.pdf: $(indir)/%.fits | $(dm-histdir)
 
         # When the plotted values are re-made, it is necessary to also
         # delete the TiKZ externalized files so the plot is also re-made.
-	rm -f $(tikzdir)/delete-me-wfpc2.pdf
+	rm -f $(tikzdir)/delete-me-image-histogram.pdf
 
         # Convert the dataset to a PDF.
 	astconvertt --colormap=gray --fluxhigh=4 $< -h0 -o$@
@@ -63,17 +85,35 @@ $(delete-pdf): $(delete-demodir)/%.pdf: $(indir)/%.fits | $(delete-demodir)
 # Histogram of WFPC2 image
 # ------------------------
 #
-# For an example plot, we'll show the pixel value histogram also.
-delete-histogram = $(delete-demodir)/wfpc2-hist.txt
-$(delete-histogram): $(delete-demodir)/%-hist.txt: $(indir)/%.fits \
-                     | $(delete-demodir)
+# For an example plot, we'll show the pixel value histogram also. IMPORTANT
+# NOTE: because this histogram contains data that is included in a plot, we
+# should publish it, so it will go into the $(tex-publish-dir).
+dm-img-histogram = $(tex-publish-dir)/wfpc2-histogram.txt
+$(dm-img-histogram): $(tex-publish-dir)/%-histogram.txt: $(indir)/%.fits \
+                     | $(tex-publish-dir)
 
         # When the plotted values are re-made, it is necessary to also
         # delete the TiKZ externalized files so the plot is also re-made.
-	rm -f $(tikzdir)/delete-me-wfpc2.pdf
+	rm -f $(tikzdir)/delete-me-image-histogram.pdf
+
+        # Generate the pixel value histogram.
+	aststatistics --lessthan=5 $< -h0 --histogram -o$@.data
+
+        # Put a two-line description of the dataset, copy the column
+        # metadata from '$@.data', and add copyright.
+	echo "# Histogram of example image to demonstrate Maneage (MANaging data linEAGE)." \
+	     > $@.tmp
+	echo "# Example image URL: $(WFPC2URL)/$(WFPC2IMAGE)" >> $@.tmp
+	echo "# " >> $@.tmp
+	awk '/^# Column .:/' $@.data >> $@.tmp
+	echo "# " >> $@.tmp
+	$(call print-copyright, $@.tmp)
 
-        # Generate the pixel value distribution
-	aststatistics --lessthan=5 $< -h0 --histogram -o$@
+        # Add the column numbers in a formatted manner, rename it to the
+        # output and clean up.
+	awk '!/^#/{printf("%-15.4f%d\n", $$1, $$2)}' $@.data >> $@.tmp
+	mv $@.tmp $@
+	rm $@.data
 
 
 
@@ -84,9 +124,9 @@ $(delete-histogram): $(delete-demodir)/%-hist.txt: $(indir)/%.fits \
 #
 # This is just as a demonstration on how to get analysic configuration
 # parameters from variables defined in `reproduce/analysis/config/'.
-delete-stats = $(delete-demodir)/wfpc2-stats.txt
-$(delete-stats): $(delete-demodir)/%-stats.txt: $(indir)/%.fits \
-                 | $(delete-demodir)
+dm-img-stats = $(dm-histdir)/wfpc2-stats.txt
+$(dm-img-stats): $(dm-histdir)/%-stats.txt: $(indir)/%.fits \
+                 | $(dm-histdir)
 	aststatistics $< -h0 --mean --median > $@
 
 
@@ -100,11 +140,11 @@ $(delete-stats): $(delete-demodir)/%-stats.txt: $(indir)/%.fits \
 #
 # NOTE: In LaTeX you cannot use any non-alphabetic character in a variable
 # name.
-$(mtexdir)/delete-me.tex: $(delete-num) $(delete-pdf) $(delete-histogram) \
-                          $(delete-stats)
+$(mtexdir)/delete-me.tex: $(dm-squared) $(dm-img-pdf) $(dm-img-histogram) \
+                          $(dm-img-stats)
 
         # Write the number of random values used.
-	echo "\newcommand{\deletemenum}{$(delete-me-num)}" > $@
+	echo "\newcommand{\deletemenum}{$(delete-me-squared-num)}" > $@
 
         # Note that since Make variables start with a `$(', if you want to
         # use `$' within the shell (not Make), you have to quote any
@@ -116,14 +156,14 @@ $(mtexdir)/delete-me.tex: $(delete-num) $(delete-pdf) $(delete-histogram) \
         # macro definition.
 	mm=$$(awk 'BEGIN{min=99999; max=-min}
 	           !/^#/{if($$2>max) max=$$2; if($$2<min) min=$$2;}
-	           END{print min, max}' $(delete-num));
+	           END{print min, max}' $(dm-squared));
 	v=$$(echo "$$mm" | awk '{printf "%.3f", $$1}');
 	echo "\newcommand{\deletememin}{$$v}"             >> $@
 	v=$$(echo "$$mm" | awk '{printf "%.3f", $$2}');
 	echo "\newcommand{\deletememax}{$$v}"             >> $@
 
         # Write the statistics of the WFPC2 image as a macro.
-	mean=$$(awk     '{printf("%.2f", $$1)}' $(delete-stats))
+	mean=$$(awk     '{printf("%.2f", $$1)}' $(dm-img-stats))
 	echo "\newcommand{\deletemewfpctwomean}{$$mean}"          >> $@
-	median=$$(awk   '{printf("%.2f", $$2)}' $(delete-stats))
+	median=$$(awk   '{printf("%.2f", $$2)}' $(dm-img-stats))
 	echo "\newcommand{\deletemewfpctwomedian}{$$median}"      >> $@