Data lineage and replicated plot in one row

Until now, the replicated plot had the width of the full page and the data
lineage graph was under it. Together they were covering more than half of
the height of the page! But the plot showing the number of papers with
tools really doesn't have too much detail, and all the space was being
wasted.

With this commit, the plot is now much much thinner and the data lineage
graph has been fitted to the right of it.

3 files changed, 276 insertions, 40 deletions

diff --git a/paper.tex b/paper.tex
index dd805f2..80c8376 100644
--- a/paper.tex
+++ b/paper.tex
@@ -293,19 +293,19 @@ Figure \ref{fig:datalineage} (bottom) is the data lineage graph that produced it
 \begin{figure*}[t]
   \begin{center}
     \includetikz{figure-tools-per-year}{width=0.95\linewidth}
-    \includetikz{figure-data-lineage}{width=0.85\linewidth}
+%    \includetikz{figure-data-lineage}{width=0.85\linewidth}
   \end{center}
   \vspace{-3mm}
   \caption{\label{fig:datalineage}
-    Top: an enhanced replica of Figure 1C in \cite{menke20}, shown here for demonstrating Maneage.
+    Left: an enhanced replica of Figure 1C in \cite{menke20}, shown here for demonstrating Maneage.
     It shows the ratio of the number of papers mentioning software tools (green line, left vertical axis) to the total number of papers studied in that year (light red bars, right vertical axis on a log scale).
-    Bottom: Schematic representation of the data lineage, or workflow, to generate the plot above.
+    Right: Schematic representation of the data lineage, or workflow, to generate the plot above.
     Each colored box is a file in the project and the arrows show the dependencies between them.
     Green files/boxes are plain-text files that are under version control and in the project source directory.
     Blue files/boxes are output files in the build directory, shown within the Makefile (\inlinecode{*.mk}) where they are defined as a \emph{target}.
     For example, \inlinecode{paper.pdf} depends on \inlinecode{project.tex} (in the build directory; generated automatically) and \inlinecode{paper.tex} (in the source directory; written manually).
     The solid arrows and full-opacity built boxes correspond to this paper.
-    The dashed arrows and low-opacity built boxes show the scalability by adding hypothetical steps to the project.
+    The dotted arrows and built boxes show the scalability by adding hypothetical steps to the project.
     The underlying data of the top plot is available at
     \href{https://zenodo.org/record/\projectzenodoid/files/tools-per-year.txt}{zenodo.\projectzenodoid/tools-per-year.txt}.
   }
@@ -380,6 +380,13 @@ Finally, to satisfy the recorded history criterion, version control (currently i
 Maneage is a Git branch that contains the shared components (infrastructure) of all projects (e.g., software tarball URLs, build recipes, common subMakefiles, and interface script).
 Derived projects start by branching off and customizing it (e.g., adding a title, data links, narrative, and subMakefiles for its particular analysis, see Listing \ref{code:branching}, there is customization checklist in \inlinecode{README-hacking.md}).
 
+The branch-based design of Figure \ref{fig:branching} allows projects to re-import Maneage at a later time (technically: \emph{merge}), thus improving its low-level infrastructure: in (a) authors do the merge during an ongoing project;
+in (b) readers do it after publication; e.g., the project remains reproducible but the infrastructure is outdated, or a bug is fixed in Maneage.
+Low-level improvements in Maneage can thus propagate to all projects, greatly reducing the cost of curation and maintenance of each individual project, before \emph{and} after publication.
+
+Finally, the complete project source is usually $\sim100$ kilo-bytes.
+It can thus easily be published or archived in many servers, for example it can be uploaded to arXiv (with the \LaTeX{} source, see the arXiv source in \cite{akhlaghi19, infante20, akhlaghi15}), published on Zenodo and archived in SoftwareHeritage.
+
 \begin{lstlisting}[
     label=code:branching,
     caption={Starting a new project with Maneage, and building it},
@@ -400,12 +407,11 @@ $ ./project make            # Re-build to see effect.
 $ git add -u && git commit  # Commit changes.
 \end{lstlisting}
 
-The branch-based design of Figure \ref{fig:branching} allows projects to re-import Maneage at a later time (technically: \emph{merge}), thus improving its low-level infrastructure: in (a) authors do the merge during an ongoing project;
-in (b) readers do it after publication; e.g., the project remains reproducible but the infrastructure is outdated, or a bug is fixed in Maneage.
-Low-level improvements in Maneage can thus propagate to all projects, greatly reducing the cost of curation and maintenance of each individual project, before \emph{and} after publication.
 
-Finally, the complete project source is usually $\sim100$ kilo-bytes.
-It can thus easily be published or archived in many servers, for example it can be uploaded to arXiv (with the \LaTeX{} source, see the arXiv source in \cite{akhlaghi19, infante20, akhlaghi15}), published on Zenodo and archived in SoftwareHeritage.
+
+
+
+
 
 
 
diff --git a/tex/src/figure-tools-per-year.tex b/tex/src/figure-tools-per-year.tex
index f6dc239..738cdad 100644
--- a/tex/src/figure-tools-per-year.tex
+++ b/tex/src/figure-tools-per-year.tex
@@ -1,34 +1,258 @@
-\begin{tikzpicture}[scale=0.9]
-  %% Add the right-side Y axis.
+% All macros commented
+\newcommand{\paperpdf}{}              %  1
+\newcommand{\papertex}{}              %  2
+\newcommand{\projecttex}{}            %  3
+\newcommand{\verifytex}{}             %  4
+\newcommand{\demoplottex}{}           %  5
+\newcommand{\toolsperyear}{}          %  6
+\newcommand{\tablethree}{}            %  7
+\newcommand{\menkexlsx}{}             %  8
+\newcommand{\inputsconf}{}            %  9
+\newcommand{\downloadtex}{}           % 10
+\newcommand{\formattex}{}             % 11
+\newcommand{\demoyearconf}{}          % 12
+\newcommand{\initializetex}{}         % 13
+\newcommand{\expandingproject}{}      % 14
+
+
+
+\begin{tikzpicture}
+
+  %% These white lines are only relevant when we want to add boxes in
+  %% multiple figures (for example to build slides). They are used to fix
+  %% the vertical position of the boxs in the figure so it doesn't change
+  %% as we add more boxes.
+  \draw [white] (-9cm,0) -- (9cm,0);
+  \draw [white] (0,-5cm) -- (0,4cm);
+
+  %% Right-side Y axis (red, logarithmic histogram). This should be created
+  %% under the line, because the line doesn't interfere with interpretting
+  %% the histogram, but the inverse is not true.
   \begin{axis}[
       ymode=log,
-      width=\linewidth,
-      height=0.3\linewidth,
+      width=5cm,
+      height=5cm,
+      axis on top,
       axis x line=none,
       axis y line*=right,
+      at={(-7.4cm,-1.8cm)},
       enlarge x limits = false,
-      ylabel=Num. papers (log-scale),
+      ylabel={Num. papers (red, log-scale)},
       max space between ticks=20,
     ]
-    \addplot+ [ybar, mark=none, fill=red!50!white, red, opacity=0.25]
+    \addplot+ [ybar, mark=none, fill=red!40!white, red!40!white]
     table [x index=0, y index=2] {tex/build/to-publish/tools-per-year.txt};
   \end{axis}
 
+  %% Left-side Y axis (green, linear/percent line).
   \begin{axis}[
       ymin=0,
       ymax=100,
-      width=\linewidth,
-      height=0.3\linewidth,
+      width=5cm,
+      height=5cm,
       xlabel={Year},
-      ylabel={Frac. papers with tools},
       axis y line*=left,
+      at={(-7.4cm,-1.8cm)},
       enlarge x limits = false,
+      ylabel={Frac. papers with tools (green)},
       yticklabel=\pgfmathprintnumber{\tick}\,\%,
       x tick label style={/pgf/number format/1000 sep=},
     ]
 
     %% Linear plot, showing the number of papers mentioning tools.
-    \addplot+ [mark=none, very thick, green!60!black]
+    \addplot+ [mark=none, ultra thick, green!60!black]
     table {tex/build/to-publish/tools-per-year.txt};
   \end{axis}
+
+  %% Use small fonts for the rest.
+  \scriptsize
+
+  %% top-make.mk
+  \node [rectangle,
+         very thick,
+         text centered,
+         font=\ttfamily,
+         text width=2.8cm,
+         anchor=north west,
+         at={(-2.6cm,3.5cm)},
+         minimum width=11.6cm,
+         minimum height=7.25cm,
+         draw=green!50!black!50,
+         fill=black!10!green!2!white,
+         label={[shift={(0,-5mm)}]\texttt{top-make.mk}}] {};
+
+  %% verify.mk
+  \node [at={(-0.7cm,-2.9cm)},
+         thick,
+         rectangle,
+         text centered,
+         font=\ttfamily,
+         text width=2.45cm,
+         minimum width=3.5cm,
+         minimum height=1.3cm,
+         draw=green!50!black!50,
+         fill=black!10!green!12!white,
+         label={[shift={(1cm,-5mm)}]\texttt{verify.mk}}] {};
+
+  %% Paper.mk
+  \node [at={(5.35cm,-2.9cm)},
+         thick,
+         rectangle,
+         text centered,
+         text width=2.8cm,
+         minimum width=7cm,
+         minimum height=1.3cm,
+         draw=green!50!black!50,
+         fill=black!10!green!12!white,
+         font=\ttfamily,
+         label={[shift={(0,-5mm)}]\texttt{paper.mk}}] {};
+
+  %% Work-horse Makefiles, the X axis value of the files is the same. The Y
+  %% axis values range like this:
+  %%      2.1cm
+  %%      1.3cm
+  %%      0.5cm
+  %%     -0.4cm
+  %%     -1.3cm
+  \node [node-makefile, at={(-1.4cm,3cm)},
+        label={[shift={(0,-5mm)}]\texttt{initialize.mk}}] {};
+  \node [node-makefile, at={(0.9cm,3cm)},
+        label={[shift={(0,-5mm)}]\texttt{download.mk}}] {};
+  \node [node-makefile, at={(3.2cm,3cm)},
+        label={[shift={(0,-5mm)}]\texttt{format.mk}}] {};
+  \node [node-makefile, at={(5.5cm,3cm)},
+        label={[shift={(0,-5mm)}]\texttt{demo-plot.mk}}] {};
+
+  %% paper.pdf
+  \ifdefined\paperpdf
+  \node (paperpdf) [node-terminal, at={(7.8cm,-3cm)}] {paper.pdf};
+  \fi
+
+  %% paper.tex and references.tex
+  \ifdefined\papertex
+  \node (reftex) [node-nonterminal, at={(5.5cm,-3.9cm)}] {references.tex};
+  \node (papertex) [node-nonterminal, at={(7.8cm,-3.9cm)}] {paper.tex};
+  \node (papertex-north) [node-point, at={(7.8cm,-3.65cm)}] {};
+  \draw [rounded corners, black!50, line width=1.5pt] (reftex) |- (papertex-north);
+  \draw [->, black!50, line width=1.5pt] (papertex) -- (paperpdf);
+  \fi
+
+  %% project.tex
+  \ifdefined\projecttex
+  \node (projecttex) [node-terminal, at={(3.2cm,-3cm)}] {project.tex};
+  \draw [->, black!50, line width=1.5pt] (projecttex) -- (paperpdf);
+  \fi
+
+  %% verify.tex
+  \ifdefined\verifytex
+  \node (verifytex) [node-terminal, at={(-1.4cm,-3cm)}] {verify.tex};
+  \draw [->, black!50, line width=1.5pt] (verifytex) -- (projecttex);
+  \fi
+
+  %% demo-plot.tex
+  \ifdefined\demoplottex
+  \node (initialize-south) [node-point, at={(-1.4cm,-2cm)}] {};
+  \node (verifytop) [node-point, at={(-1.4cm,-2.75cm)}] {};
+  \node (dptex) [node-terminal, at={(5.5cm,-1.3cm)}] {demo-plot.tex};
+  \draw [rounded corners, ->, black!50, line width=1.5pt]
+        (dptex) |- (initialize-south) |- (verifytop);
+  \fi
+
+  %% tools-per-year.txt
+  \ifdefined\toolsperyear
+  \node (tpyear) [node-terminal, at={(5.5cm,-0.4cm)}] {tools-per-\\year.txt};
+  \draw [->, black!50, line width=1.5pt] (tpyear) -- (dptex);
+  \fi
+
+  %% table-3.txt
+  \ifdefined\tablethree
+  \node (tabthree) [node-terminal, at={(3.2cm,0.5cm)}] {table-3.txt};
+  \draw [rounded corners, ->, black!50, line width=1.5pt] (tabthree) |- (tpyear);
+  \fi
+
+  %% menkexlsx
+  \ifdefined\menkexlsx
+  \node (xlsx) [node-terminal, at={(0.9cm,1.3cm)}] {menke20.xlsx};
+  \draw [->, rounded corners, black!50, line width=1.5pt] (xlsx) |- (tabthree);
+  \fi
+
+  %% INPUTS.conf
+  \ifdefined\inputsconf
+  \node (INPUTS) [node-nonterminal, at={(0.9cm,4cm)}] {INPUTS.conf};
+  \node (xlsx-west) [node-point, at={(-0.25cm,1.37cm)}] {};
+  \draw [->,rounded corners, black!50, line width=1.5pt]
+        (INPUTS.west) -| (xlsx-west) |- (xlsx);
+  \fi
+
+  %% download.tex
+  \ifdefined\downloadtex
+  \node (downloadtex) [node-terminal, at={(0.9cm,-1.3cm)}] {download.tex};
+  \node (downloadtex-west) [node-point, at={(-0.25cm,-1.25cm)}] {};
+  \draw [->,rounded corners, black!50, line width=1.5pt]
+        (INPUTS.west) -| (downloadtex-west) |- (downloadtex);
+  \draw [rounded corners, -, black!50, line width=1.5pt]
+        (downloadtex) |- (initialize-south);
+  \fi
+
+  %% format.tex
+  \ifdefined\formattex
+  \node (fmttex) [node-terminal, at={(3.2cm,-1.3cm)}] {format.tex};
+  \draw [->, black!50, line width=1.5pt] (tabthree) -- (fmttex);
+  \draw [rounded corners, -, black!50, line width=1.5pt]
+        (fmttex) |- (initialize-south);
+  \fi
+
+  %% demo-year.conf
+  \ifdefined\demoyearconf
+  \node (dyearconf) [node-nonterminal, at={(5.5cm,4cm)}] {demo-year.conf};
+  \node (dptex-west) [node-point, at={(4.35cm,-1.25cm)}] {};
+  \draw [->,rounded corners, black!50, line width=1.5pt]
+        (dyearconf.west) -| (dptex-west) |- (dptex);
+  \fi
+
+  %% Initialize.tex
+  \ifdefined\initializetex
+  \node (initializetex) [node-terminal, at={(-1.4cm,-1.3cm)}] {initialize.tex};
+  \draw [->, black!50, line width=1.5pt] (initializetex) -- (verifytex);
+  \node [anchor=west, at={(-2.4cm,1.5cm)}] {Basic project info};
+  \node [anchor=west, at={(-2.4cm,1.2cm)}] {(e.g., Git commit).};
+  \node [anchor=west, at={(-2.4cm,0.5cm)}] {Also defines};
+  \node [anchor=west, at={(-2.4cm,0.2cm)}] {project structure};
+  \node [anchor=west, at={(-2.4cm,-0.1cm)}] {(for \texttt{*.mk} files).};
+  \fi
+
+  %% Expanding project
+  \ifdefined\expandingproject
+
+    %% The Makefile.
+    \node [node-makefile, dotted, at={(7.8cm,3cm)},
+          label={[shift={(0,-5mm)}]\texttt{next-step.mk}}] {};
+
+    %% next-step.tex
+    \node [dotted] (a3tex) [node-terminal, at={(7.8cm,-1.3cm)}] {next-step.tex};
+    \draw [dotted, rounded corners, -, black!50, line width=1.4pt]
+          (a3tex) |- (initialize-south);
+
+    % out-3a.dat and out-3b.dat
+    \node [dotted] (out3a) [node-terminal, at={(7.8cm,2.1cm)}] {out-a.dat};
+    \node [dotted] (out3b) [node-terminal, at={(7.8cm,0.5cm)}] {out-b.dat};
+    \node (a3tex-east) [node-point, at={(8.93cm,-0.8cm)}] {};
+    \draw [dotted, ->, rounded corners, black!50, line width=1.5pt]
+          (out3a.east) -| (a3tex-east) |- (a3tex);
+    \draw [dotted, ->, black!50, line width=1.5pt] (out3b) -- (a3tex);
+
+    %% demo-out.dat
+    \node [dotted] (dout) [node-terminal, at={(5.5cm,1.3cm)}] {demo-out.dat};
+    \draw [dotted, rounded corners, ->, black!50, line width=1.5pt] (dout.south) |- (out3b);
+
+    %% links
+    \node (dout-west) [node-point, at={(5cm,1.3cm)}] {};
+    \draw [dotted, ->, black!50, line width=1.5pt] (xlsx) -- (dout);
+    \node [opacity=0.7] (out3a-west) [node-point, at={(6.65cm,2.1cm)}] {};
+    \draw [dotted, ->, rounded corners, black!50, line width=1.5pt] (xlsx) |- (out3a);
+    \node [dotted] (a3conf1) [node-nonterminal, at={(7.8cm,4cm)}]  {param.conf};
+    \draw [dotted, rounded corners, black!50, line width=1.5pt]
+          (a3conf1.west) -| (out3a-west) |- (out3a);
+  \fi
+
 \end{tikzpicture}
diff --git a/tex/src/preamble-pgfplots.tex b/tex/src/preamble-pgfplots.tex
index 05e73b7..0c0485c 100644
--- a/tex/src/preamble-pgfplots.tex
+++ b/tex/src/preamble-pgfplots.tex
@@ -126,29 +126,45 @@
 
 
 %% Nodes in demo graphs
-\tikzset{node-terminal/.style={
+
+%% sub-Makefiles.
+\tikzset{node-makefile/.style={
+  thick,
+  rectangle,
+  anchor=north,
+  minimum height=4.7cm,
+  minimum width=2.1cm,
+  draw=green!50!black!50,
+  fill=black!10!green!12!white}}
+
+%% Input files (green, sharp-edged boxes).
+\tikzset{node-nonterminal/.style={
   rectangle,
   very thick,
-  draw=blue!50,
+  anchor=north,
   text centered,
   top color=white,
-  minimum size=6mm,
-  text width=2.1cm,
-  rounded corners=3mm,
-  bottom color=blue!20,
+  text width=1.7cm,
+  minimum height=4mm,
+  draw=green!50!black!50,
+  bottom color=green!80!black!50,
   font=\ttfamily}}
 
-\tikzset{node-nonterminal/.style={
+\tikzset{node-terminal/.style={
   rectangle,
   very thick,
+  draw=blue!50,
   text centered,
   top color=white,
-  text width=2.1cm,
-  minimum size=6mm,
-  draw=green!50!black!50,
-  bottom color=green!80!black!50,
+  text width=1.7cm,
+  minimum height=4mm,
+  rounded corners=2mm,
+  bottom color=blue!20,
   font=\ttfamily}}
 
+
+%%%%%%%%%%%%%%%%%%%
+
 \tikzset{node-nonterminal-thin/.style={
   rectangle,
   thick,
@@ -160,16 +176,6 @@
   bottom color=green!80!black!50,
   font=\ttfamily\scriptsize}}
 
-\tikzset{node-makefile/.style={
-  thick,
-  rectangle,
-  anchor=south,
-  minimum width=2.6cm,
-  minimum height=5cm,
-  draw=green!50!black!50,
-  fill=black!10!green!12!white,
-}}
-
 \tikzset{node-point/.style={
   circle,
   black!50,