aboutsummaryrefslogtreecommitdiff
path: root/configure
diff options
context:
space:
mode:
authorMohammad Akhlaghi <mohammad@akhlaghi.org>2019-07-24 14:38:29 +0100
committerMohammad Akhlaghi <mohammad@akhlaghi.org>2019-07-24 14:38:29 +0100
commitdbf525a90a096931551274f3e2fce49e00d0bf72 (patch)
tree89f62d3b1519acdca89f419daa2f697832c5037c /configure
parent9901c3b2db0a3f85105489faa860fc27227ad4ac (diff)
Temporary software building done in shared memory if available
Until now, the template would unpack the software and build them directly in the build directory of each project. After installation, the whole unpacked directory is deleted. However, building the software involves the reading and writing of millions of files, so on the long run, it can be bad for the non-volatile memory (HDDs or SSDs), it can also be slightly slow. To fix this, if the system has a shared-memory directory (commonly named `/dev/shm'), we can do the temporary building of the software there. The great thing about this unique directory is that it is actually in the RAM, not on the HDD/SSD. This can slightly improve the speed (not much probably), but more importantly it will not do any long-term harm to the host's HDDs/SSDs. With this commit, when there is a shared memory directory mounted in `/dev/shm', and it has enough space (currently set to 2GB), the `./configure' script will make `.build/software/build-tmp' as a symbolic link to a fixed directory there. Otherwise, it will just build it as a directory in the project's shared directory. The structure has been defined in such a way that we can later easily add different standard shared-memory locations (for different operating systems). This completes task #15336.
Diffstat (limited to 'configure')
-rwxr-xr-xconfigure52
1 files changed, 46 insertions, 6 deletions
diff --git a/configure b/configure
index 49f7d9e..b376384 100755
--- a/configure
+++ b/configure
@@ -640,10 +640,6 @@ if ! [ -d $tardir ]; then mkdir $tardir; fi
instdir=$sdir/installed
if ! [ -d $instdir ]; then mkdir $instdir; fi
-# Temporary software un-packing.
-tmpblddir=$sdir/build-tmp
-if ! [ -d $tmpblddir ]; then mkdir $tmpblddir; fi
-
# To record software versions and citation.
verdir=$instdir/version-info
if ! [ -d $verdir ]; then mkdir $verdir; fi
@@ -707,6 +703,47 @@ ln -s $topdir/reproduce/software/config/gnuastro .gnuastro
# ------------------------------------------
+# Temporary software un-packing/build directory: if the host has the
+# standard `/dev/shm' mounting-point, we'll do it in shared memory (on the
+# RAM), to avoid harming/over-using the HDDs/SSDs. The RAM of most systems
+# today (>8GB) is large enough for the parallel building of the software.
+#
+# For the name of the directory under `/dev/shm' (for this project), we'll
+# use the names of the two parent directories to the current/running
+# directory, separated by a `-' instead of `/'. We'll then appended that
+# with the user's name (in case multiple users may be working on similar
+# project names). Maybe later, we can use something like `mktemp' to add
+# random characters to this name and make it unique to every run (even for
+# a single user).
+tmpblddir=$sdir/build-tmp
+if [ -d /dev/shm ]; then
+ dirname=$(pwd | sed -e's/\// /g' | awk '{l=NF-1; printf("%s-%s",$l, $NF)}')
+ tbshmdir=/dev/shm/"$dirname"-$(whoami)
+else
+ tbshmdir=""
+fi
+
+# If a shared memory mounted directory exists and there is enough space
+# there (in RAM), build a temporary directory for this project.
+use_shm=0
+needed_space=2000000
+if [ x"$tbshmdir" != x ]; then
+ available_space=$(df $tbshmdir | awk 'NR==2{print $4}')
+ if [ $available_space -gt $needed_space ]; then
+ use_shm=1
+ if ! [ -d $tbshmdir ]; then mkdir $tbshmdir; fi
+ fi
+fi
+
+# If no shared memory directory was created, just build the temporary build
+# directory under the project build directory.
+if [ x$use_shm = x0 ]; then
+ if ! [ -d $tmpblddir ]; then mkdir $tmpblddir; fi
+else
+ if ! [ -d $tmpblddir ]; then ln -s $tbshmdir $tmpblddir; fi
+fi
+
+
@@ -1283,8 +1320,11 @@ fi
# ---------------------------------
#
# By the time the script reaches here the temporary software build
-# directory should be empty, so just delete it.
-rm -rf $tmpblddir
+# directory should be empty, so just delete it. Note `tmpblddir' may be a
+# symbolic link to shared memory. So, to work in any scenario, first delete
+# the contents of the directory (if it has any), then delete `tmpblddir'.
+.local/bin/rm -rf $tmpblddir/*
+.local/bin/rm -rf $tmpblddir