aboutsummaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorMohammad Akhlaghi <mohammad@akhlaghi.org>2020-04-17 01:06:20 +0100
committerMohammad Akhlaghi <mohammad@akhlaghi.org>2020-04-17 01:06:20 +0100
commit8eb0892e179b4970a9835d29b5f57dd912ca4464 (patch)
treeb0858d1021efc399106e2936aa44c4fc37196edb
parente0a57efde9206c0c7420e03e5fb2a354d568c3ad (diff)
IMPORTANT: software config directly under reproduce/software/config
Until now the software configuration parameters were defined under the `reproduce/software/config/installation/' directory. This was because the configuration parameters of analysis software (for example Gnuastro's configurations) were placed under there too. But this was terribly confusing, because the run-time options of programs falls under the "analysis" phase of the project. With this commit, the Gnuastro configuration files have been moved under the new `reproduce/analysis/config/gnuastro' directory and the software configuration files are directly under `reproduce/software/config'. A clean build was done with this change and it didn't crash, but it may cause crashes in derived projects, so after merging with Maneage, please re-configure your project to see if anything has been missed. Please let us know if there is a problem.
-rw-r--r--README-hacking.md59
-rw-r--r--reproduce/analysis/config/gnuastro/astbuildprog.conf (renamed from reproduce/software/config/gnuastro/astbuildprog.conf)0
-rw-r--r--reproduce/analysis/config/gnuastro/astconvertt.conf (renamed from reproduce/software/config/gnuastro/astconvertt.conf)0
-rw-r--r--reproduce/analysis/config/gnuastro/aststatistics.conf (renamed from reproduce/software/config/gnuastro/aststatistics.conf)0
-rw-r--r--reproduce/analysis/config/gnuastro/gnuastro.conf (renamed from reproduce/software/config/gnuastro/gnuastro.conf)0
-rw-r--r--reproduce/analysis/make/initialize.mk4
-rw-r--r--reproduce/analysis/make/top-make.mk2
-rw-r--r--reproduce/analysis/make/top-prepare.mk2
-rw-r--r--reproduce/software/config/LOCAL.conf.in (renamed from reproduce/software/config/installation/LOCAL.conf.in)0
-rw-r--r--reproduce/software/config/TARGETS.conf (renamed from reproduce/software/config/installation/TARGETS.conf)2
-rw-r--r--reproduce/software/config/checksums.conf (renamed from reproduce/software/config/installation/checksums.conf)6
-rw-r--r--reproduce/software/config/numpy-scipy.cfg (renamed from reproduce/software/config/installation/numpy-scipy.cfg)0
-rw-r--r--reproduce/software/config/texlive-packages.conf (renamed from reproduce/software/config/installation/texlive-packages.conf)0
-rw-r--r--reproduce/software/config/texlive.conf (renamed from reproduce/software/config/installation/texlive.conf)0
-rw-r--r--reproduce/software/config/versions.conf (renamed from reproduce/software/config/installation/versions.conf)2
-rw-r--r--reproduce/software/make/basic.mk10
-rw-r--r--reproduce/software/make/high-level.mk16
-rw-r--r--reproduce/software/make/python.mk4
-rwxr-xr-xreproduce/software/shell/configure.sh14
19 files changed, 60 insertions, 61 deletions
diff --git a/README-hacking.md b/README-hacking.md
index 82b40e3..c1efccc 100644
--- a/README-hacking.md
+++ b/README-hacking.md
@@ -321,17 +321,17 @@ variables/configurations) and _workhorse-Makefiles_ (Makefiles that
actually contain analysis/processing rules).
The configuration-Makefiles are those that satisfy these two wildcards:
-`reproduce/software/config/installation/*.conf` (for building the necessary
-software when you run `./project configure`) and
-`reproduce/analysis/config/*.conf` (for the high-level analysis, when you
-run `./project make`). These Makefiles don't actually have any rules, they
-just have values for various free parameters throughout the configuration
-or analysis. Open a few of them to see for yourself. These Makefiles must
-only contain raw Make variables (project configurations). By "raw" we mean
-that the Make variables in these files must not depend on variables in any
-other configuration-Makefile. This is because we don't want to assume any
-order in reading them. It is also very important to *not* define any rule,
-or other Make construct, in these configuration-Makefiles.
+`reproduce/software/config/*.conf` (for building the necessary software
+when you run `./project configure`) and `reproduce/analysis/config/*.conf`
+(for the high-level analysis, when you run `./project make`). These
+Makefiles don't actually have any rules, they just have values for various
+free parameters throughout the configuration or analysis. Open a few of
+them to see for yourself. These Makefiles must only contain raw Make
+variables (project configurations). By "raw" we mean that the Make
+variables in these files must not depend on variables in any other
+configuration-Makefile. This is because we don't want to assume any order
+in reading them. It is also very important to *not* define any rule, or
+other Make construct, in these configuration-Makefiles.
Following this rule-of-thumb enables you to set these configure-Makefiles
as a prerequisite to any target that depends on their variable
@@ -372,9 +372,8 @@ Let's see how this design is implemented. Please open and inspect
`top-make.mk` it as we go along here. The first step (un-commented line) is
to import the local configuration (your answers to the questions of
`./project configure`). They are defined in the configuration-Makefile
-`reproduce/software/config/installation/LOCAL.conf` which was also built by
-`./project configure` (based on the `LOCAL.conf.in` template of the same
-directory).
+`reproduce/software/config/LOCAL.conf` which was also built by `./project
+configure` (based on the `LOCAL.conf.in` template of the same directory).
The next non-commented set of the top `Makefile` defines the ultimate
target of the whole project (`paper.pdf`). But to avoid mistakes, a sanity
@@ -767,21 +766,21 @@ Other basic customizations
- **High-level software**: The template installs all the software that
your project needs. You can specify which software your project needs
- in `reproduce/software/config/installation/TARGETS.conf`. The
- necessary software are classified into two classes: 1) programs or
- libraries (usually written in C/C++) which are run directly by the
- operating system. 2) Python modules/libraries that are run within
- Python. By default `TARGETS.conf` only has GNU Astronomy Utilities
- (Gnuastro) as one scientific program and Astropy as one scientific
- Python module. Both have many dependencies which will be installed
- into your project during the configuration step. To see a list of
- software that are currently ready to be built in the template, see
- `reproduce/software/config/installation/versions.conf` (which has
- their versions also), the comments in `TARGETS.conf` describe how to use
- the software name from `versions.conf`. Currently the raw pipeline just
- uses Gnuastro to make the demonstration plots. Therefore if you don't
- need Gnuastro, go through the analysis steps in `reproduce/analysis`
- and remove all its use cases (clearly marked).
+ in `reproduce/software/config/TARGETS.conf`. The necessary software
+ are classified into two classes: 1) programs or libraries (usually
+ written in C/C++) which are run directly by the operating system. 2)
+ Python modules/libraries that are run within Python. By default
+ `TARGETS.conf` only has GNU Astronomy Utilities (Gnuastro) as one
+ scientific program and Astropy as one scientific Python module. Both
+ have many dependencies which will be installed into your project
+ during the configuration step. To see a list of software that are
+ currently ready to be built in the template, see
+ `reproduce/software/config/versions.conf` (which has their versions
+ also), the comments in `TARGETS.conf` describe how to use the software
+ name from `versions.conf`. Currently the raw pipeline just uses
+ Gnuastro to make the demonstration plots. Therefore if you don't need
+ Gnuastro, go through the analysis steps in `reproduce/analysis` and
+ remove all its use cases (clearly marked).
- **Input dataset**: The input datasets are managed through the
`reproduce/analysis/config/INPUTS.conf` file. It is best to gather all
@@ -856,7 +855,7 @@ Other basic customizations
TeXLive comes (once every year), if you would like to build the paper,
its necessary to update it in your project (otherwise the configure
script will crash). To do that, just modify the years in
- `reproduce/software/config/installation/texlive.conf`, then delete
+ `reproduce/software/config/texlive.conf`, then delete
`.build/software/tarballs/install-tl-unx.tar.gz`. The next time you
run `./project configure`, the new TeXLive will be installed and used.
diff --git a/reproduce/software/config/gnuastro/astbuildprog.conf b/reproduce/analysis/config/gnuastro/astbuildprog.conf
index 7adb4d1..7adb4d1 100644
--- a/reproduce/software/config/gnuastro/astbuildprog.conf
+++ b/reproduce/analysis/config/gnuastro/astbuildprog.conf
diff --git a/reproduce/software/config/gnuastro/astconvertt.conf b/reproduce/analysis/config/gnuastro/astconvertt.conf
index 770ac39..770ac39 100644
--- a/reproduce/software/config/gnuastro/astconvertt.conf
+++ b/reproduce/analysis/config/gnuastro/astconvertt.conf
diff --git a/reproduce/software/config/gnuastro/aststatistics.conf b/reproduce/analysis/config/gnuastro/aststatistics.conf
index 90f985d..90f985d 100644
--- a/reproduce/software/config/gnuastro/aststatistics.conf
+++ b/reproduce/analysis/config/gnuastro/aststatistics.conf
diff --git a/reproduce/software/config/gnuastro/gnuastro.conf b/reproduce/analysis/config/gnuastro/gnuastro.conf
index 82cb100..82cb100 100644
--- a/reproduce/software/config/gnuastro/gnuastro.conf
+++ b/reproduce/analysis/config/gnuastro/gnuastro.conf
diff --git a/reproduce/analysis/make/initialize.mk b/reproduce/analysis/make/initialize.mk
index 3af0012..d8a8b73 100644
--- a/reproduce/analysis/make/initialize.mk
+++ b/reproduce/analysis/make/initialize.mk
@@ -308,8 +308,8 @@ $(packagecontents): paper.pdf | $(texdir)
# have this machine's local settings and are irrelevant for anyone
# else.
rm -rf $$dir/tex/build/build*
- rm $$dir/reproduce/software/config/installation/LOCAL.conf
- rm $$dir/reproduce/software/config/gnuastro/gnuastro-local.conf
+ rm $$dir/reproduce/software/config/LOCAL.conf
+ rm $$dir/reproduce/analysis/config/gnuastro/gnuastro-local.conf
# When submitting to places like arXiv, they will just run LaTeX
# once and won't run `biber'. So we need to also keep the `.bbl'
diff --git a/reproduce/analysis/make/top-make.mk b/reproduce/analysis/make/top-make.mk
index 6c940b8..43e88b2 100644
--- a/reproduce/analysis/make/top-make.mk
+++ b/reproduce/analysis/make/top-make.mk
@@ -21,7 +21,7 @@
# Load the local configuration (created after running
# `./project configure').
-include reproduce/software/config/installation/LOCAL.conf
+include reproduce/software/config/LOCAL.conf
diff --git a/reproduce/analysis/make/top-prepare.mk b/reproduce/analysis/make/top-prepare.mk
index 1778c6b..be7482c 100644
--- a/reproduce/analysis/make/top-prepare.mk
+++ b/reproduce/analysis/make/top-prepare.mk
@@ -25,7 +25,7 @@
# Load the local configuration (created after running
# `./project configure').
-include reproduce/software/config/installation/LOCAL.conf
+include reproduce/software/config/LOCAL.conf
diff --git a/reproduce/software/config/installation/LOCAL.conf.in b/reproduce/software/config/LOCAL.conf.in
index 0579218..0579218 100644
--- a/reproduce/software/config/installation/LOCAL.conf.in
+++ b/reproduce/software/config/LOCAL.conf.in
diff --git a/reproduce/software/config/installation/TARGETS.conf b/reproduce/software/config/TARGETS.conf
index d2d5e2c..84962f7 100644
--- a/reproduce/software/config/installation/TARGETS.conf
+++ b/reproduce/software/config/TARGETS.conf
@@ -18,7 +18,7 @@
# All software that are currently available for installation can be seen in
# the following file.
#
-# reproduce/software/config/installation/versions.conf
+# reproduce/software/config/versions.conf
#
# Please add any software that you need for your project in the respective
# part below (using its name in `versions.conf', but without the `-version'
diff --git a/reproduce/software/config/installation/checksums.conf b/reproduce/software/config/checksums.conf
index 7531749..37fe64d 100644
--- a/reproduce/software/config/installation/checksums.conf
+++ b/reproduce/software/config/checksums.conf
@@ -71,9 +71,9 @@ zlib-checksum = 73fd3fff4adeccd4894084c15ddac89890cd10ef105dd5e1835e1e9bbb6a49ff
# ------------------------------------------
#
# These are programs and libraries that are optional, The ones in
-# `reproduce/software/config/installation/TARGETS.conf' will be built as
-# part of a project. To specify a software there, just remove the
-# `-checksum' suffix from the list below.
+# `reproduce/software/config/TARGETS.conf' will be built as part of a
+# project. To specify a software there, just remove the `-checksum' suffix
+# from the list below.
apachelog4cxx-checksum = aa59ce549c2c5cbeec031361dfce09cdfc3e62ee3bc9ecbc809507b7ec878c14409b98536b7d13c27690809c8e9d5ebafc3589c9fb5e4aecd5cc064943ae7d6b
apr-checksum = daa140c83c7e2c45c3980d9dc81d34fa662bebd050653562c39572d0ddf2eaedb71767c518a59d77f59db9b32e00221ef48b9f72ec3666c4521dd511969f3706
apr-util-checksum = 84da76e9b64da2de0996d4d6f3ab3f23db3724eb6352d218e0e8196bcc0b0a5d4fe791f41b4cc350ce3d04cce3bb3cf8bfb513d777d0cd030928368e6b55a536
diff --git a/reproduce/software/config/installation/numpy-scipy.cfg b/reproduce/software/config/numpy-scipy.cfg
index 82cc1e1..82cc1e1 100644
--- a/reproduce/software/config/installation/numpy-scipy.cfg
+++ b/reproduce/software/config/numpy-scipy.cfg
diff --git a/reproduce/software/config/installation/texlive-packages.conf b/reproduce/software/config/texlive-packages.conf
index c53e170..c53e170 100644
--- a/reproduce/software/config/installation/texlive-packages.conf
+++ b/reproduce/software/config/texlive-packages.conf
diff --git a/reproduce/software/config/installation/texlive.conf b/reproduce/software/config/texlive.conf
index b5075c6..b5075c6 100644
--- a/reproduce/software/config/installation/texlive.conf
+++ b/reproduce/software/config/texlive.conf
diff --git a/reproduce/software/config/installation/versions.conf b/reproduce/software/config/versions.conf
index 39642d4..0c2cd0b 100644
--- a/reproduce/software/config/installation/versions.conf
+++ b/reproduce/software/config/versions.conf
@@ -69,7 +69,7 @@ zlib-version = 1.2.11
# ------------------------------------------
#
# These are programs and libraries that are optional, The ones in
-# `reproduce/software/config/installation/TARGETS.conf' will be built as
+# `reproduce/software/config/TARGETS.conf' will be built as
# part of a project. To specify a software there, just remove the
# `-version' suffix from the list below.
apachelog4cxx-version = 0.10.0-603-014954db
diff --git a/reproduce/software/make/basic.mk b/reproduce/software/make/basic.mk
index 47a01a3..17bb47a 100644
--- a/reproduce/software/make/basic.mk
+++ b/reproduce/software/make/basic.mk
@@ -33,10 +33,10 @@
# Top level environment
+include reproduce/software/config/LOCAL.conf
include reproduce/software/make/build-rules.mk
-include reproduce/software/config/installation/LOCAL.conf
-include reproduce/software/config/installation/versions.conf
-include reproduce/software/config/installation/checksums.conf
+include reproduce/software/config/versions.conf
+include reproduce/software/config/checksums.conf
lockdir = $(BDIR)/locks
tdir = $(BDIR)/software/tarballs
@@ -1334,12 +1334,14 @@ $(ibidir)/gcc: | $(ibidir)/binutils \
&& cd ../.. \
&& tempname=$$odir/gcc-$(gcc-version)/build/rpath-temp-copy \
&& if [ "x$(on_mac_os)" != xyes ]; then \
- patchelf --add-needed $(ildir)/libiconv.so $(ildir)/libstdc++.so; \
for f in $$(find $(idir)/libexec/gcc) $(ildir)/libstdc++*; do \
isdynamic=$$(file $$f | grep "dynamically linked"); \
if [ x"$$isdynamic" != x ]; then \
cp $$f $$tempname; \
patchelf --set-rpath $(ildir) $$tempname; \
+ if [ "$$f" = $(ildir)/libstdc++.so ]; then \
+ patchelf --add-needed $(ildir)/libiconv.so $$tempname; \
+ fi; \
mv $$tempname $$f; echo "corrected"; \
fi; \
done; \
diff --git a/reproduce/software/make/high-level.mk b/reproduce/software/make/high-level.mk
index 6e01354..e62f02f 100644
--- a/reproduce/software/make/high-level.mk
+++ b/reproduce/software/make/high-level.mk
@@ -27,12 +27,12 @@
# Top level environment
+include reproduce/software/config/LOCAL.conf
include reproduce/software/make/build-rules.mk
-include reproduce/software/config/installation/LOCAL.conf
-include reproduce/software/config/installation/TARGETS.conf
-include reproduce/software/config/installation/versions.conf
-include reproduce/software/config/installation/checksums.conf
-include reproduce/software/config/installation/texlive-packages.conf
+include reproduce/software/config/TARGETS.conf
+include reproduce/software/config/versions.conf
+include reproduce/software/config/checksums.conf
+include reproduce/software/config/texlive-packages.conf
lockdir = $(BDIR)/locks
tdir = $(BDIR)/software/tarballs
@@ -1240,7 +1240,7 @@ $(ibidir)/xlsxio: $(ibidir)/cmake \
tlmirror=http://mirrors.rit.edu/CTAN/systems/texlive/tlnet
# The core TeX Live system.
-$(itidir)/texlive-ready-tlmgr: reproduce/software/config/installation/texlive.conf \
+$(itidir)/texlive-ready-tlmgr: reproduce/software/config/texlive.conf \
| $(tdir)/install-tl-unx.tar.gz
# Unpack, enter the directory, and install based on the given
@@ -1251,7 +1251,7 @@ $(itidir)/texlive-ready-tlmgr: reproduce/software/config/installation/texlive.co
tar xf $(tdir)/install-tl-unx.tar.gz
cd install-tl-*
sed -e's|@installdir[@]|$(idir)|g' \
- $$topdir/reproduce/software/config/installation/texlive.conf \
+ $$topdir/reproduce/software/config/texlive.conf \
> texlive.conf
# TeX Live's installation may fail due to any reason. But TeX Live
@@ -1292,7 +1292,7 @@ $(itidir)/texlive-ready-tlmgr: reproduce/software/config/installation/texlive.co
#else
#forbiber = $(ibidir)/libnsl
#endif
-$(itidir)/texlive: reproduce/software/config/installation/texlive-packages.conf \
+$(itidir)/texlive: reproduce/software/config/texlive-packages.conf \
$(itidir)/texlive-ready-tlmgr \
$(forbiber)
diff --git a/reproduce/software/make/python.mk b/reproduce/software/make/python.mk
index a15841c..1ef1bf2 100644
--- a/reproduce/software/make/python.mk
+++ b/reproduce/software/make/python.mk
@@ -600,7 +600,7 @@ $(ipydir)/numpy: $(ibidir)/unzip \
export LDFLAGS="$(LDFLAGS) -shared"; \
fi; \
export CFLAGS="--std=c99 $$CFLAGS"; \
- conf="$$(pwd)/reproduce/software/config/installation/numpy-scipy.cfg"; \
+ conf="$$(pwd)/reproduce/software/config/numpy-scipy.cfg"; \
$(call pybuild, unzip, numpy-$(numpy-version),$$conf, \
Numpy $(numpy-version)) \
&& cp $(dtexdir)/numpy.tex $(ictdir)/ \
@@ -679,7 +679,7 @@ $(ipydir)/scipy: $(ipydir)/numpy \
else \
export LDFLAGS="$(LDFLAGS) -shared"; \
fi; \
- conf="$$(pwd)/reproduce/software/config/installation/numpy-scipy.cfg"; \
+ conf="$$(pwd)/reproduce/software/config/numpy-scipy.cfg"; \
$(call pybuild, tar xf, scipy-$(scipy-version),$$conf) \
&& cp $(dtexdir)/scipy.tex $(ictdir)/ \
&& echo "Scipy $(scipy-version) \citep{scipy2007,scipy2011}" > $@
diff --git a/reproduce/software/shell/configure.sh b/reproduce/software/shell/configure.sh
index a0fc228..919acde 100755
--- a/reproduce/software/shell/configure.sh
+++ b/reproduce/software/shell/configure.sh
@@ -36,15 +36,13 @@ optionaldir="/optional/path"
adir=reproduce/analysis/config
cdir=reproduce/software/config
-sbdir=$cdir/installation
-
-pconf=$sbdir/LOCAL.conf
-ptconf=$sbdir/LOCAL_tmp.conf
-poconf=$sbdir/LOCAL_old.conf
-depverfile=$cdir/installation/versions.conf
-depshafile=$cdir/installation/checksums.conf
+pconf=$cdir/LOCAL.conf
+ptconf=$cdir/LOCAL_tmp.conf
+poconf=$cdir/LOCAL_old.conf
+depverfile=$cdir/versions.conf
+depshafile=$cdir/checksums.conf
# --------- Delete for no Gnuastro ---------
-glconf=$cdir/gnuastro/gnuastro-local.conf
+glconf=$adir/gnuastro/gnuastro-local.conf
# ------------------------------------------