diff options
author | Mohammad Akhlaghi <mohammad@akhlaghi.org> | 2019-04-15 01:47:58 +0100 |
---|---|---|
committer | Mohammad Akhlaghi <mohammad@akhlaghi.org> | 2019-04-15 02:24:09 +0100 |
commit | 313b936b502d22b6a2ff43f560dee0bb51fd01d0 (patch) | |
tree | 70f884b91b393be4d3c6b7cfaeaf3412900bd16f /reproduce/src/bash | |
parent | 4722ea598edd6b630227404c48c1c09ac527e9b8 (diff) |
New architecture to separate software-building and analysis steps
Until now, the software building and analysis steps of the pipeline were
intertwined. However, these steps (of how to build a software, and how to
use it) are logically completely independent.
Therefore with this commit, the pipeline now has a new architecture
(particularly in the `reproduce' directory) to emphasize this distinction:
The `reproduce' directory now has the two `software' and `analysis'
subdirectories and the respective parts of the previous architecture have
been broken up between these two based on their function. There is also no
more `src' directory. The `config' directory for software and analysis is
now mixed with the language-specific directories.
Also, some of the software versions were also updated after some checks
with their webpages.
This new architecture will allow much more focused work on each part of the
pipeline (to install the software and to run them for an analysis).
Diffstat (limited to 'reproduce/src/bash')
-rwxr-xr-x | reproduce/src/bash/download-multi-try | 112 | ||||
-rw-r--r-- | reproduce/src/bash/git-post-checkout | 66 | ||||
-rw-r--r-- | reproduce/src/bash/git-pre-commit | 83 |
3 files changed, 0 insertions, 261 deletions
diff --git a/reproduce/src/bash/download-multi-try b/reproduce/src/bash/download-multi-try deleted file mode 100755 index 1fd7497..0000000 --- a/reproduce/src/bash/download-multi-try +++ /dev/null @@ -1,112 +0,0 @@ -# Attempt downloading multiple times before crashing whole project. From -# the top project directory (for the shebang above), this script must be -# run like this: -# -# $ /path/to/download-multi-try downloader lockfile input-url downloaded-name -# -# NOTE: The `downloader' must contain the option to specify the output name -# in its end. For example "wget -O". Any other option can also be placed in -# the middle. -# -# Due to temporary network problems, a download may fail suddenly, but -# succeed in a second try a few seconds later. Without this script that -# temporary glitch in the network will permanently crash the project and -# it can't continue. The job of this script is to be patient and try the -# download multiple times before crashing the whole project. -# -# LOCK FILE: Since there is ultimately only one network port to the outside -# world, downloading is done much faster in serial, not in parallel. But -# the project's processing may be done in parallel (with multiple threads -# needing to download different files at the same time). Therefore, this -# script uses the `flock' program to only do one download at a time. To -# benefit from it, any call to this script must be given the same lock -# file. -# -# Copyright (C) 2019 Mohammad Akhlaghi <mohammad@akhlaghi.org> -# -# This script is free software: you can redistribute it and/or modify it -# under the terms of the GNU General Public License as published by the -# Free Software Foundation, either version 3 of the License, or (at your -# option) any later version. -# -# This script is distributed in the hope that it will be useful, but -# WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General -# Public License for more details. See <http://www.gnu.org/licenses/>. - - - - - -# Script settings -# --------------- -# Stop the script if there are any errors. -set -e - - - - - -# Input arguments and necessary sanity checks. -inurl="$3" -outname="$4" -lockfile="$2" -downloader="$1" -if [ "x$downloader" = x ]; then - echo "$0: downloader (first argument) not given."; exit 1; -fi -if [ "x$lockfile" = x ]; then - echo "$0: lock file (second argument) not given."; exit 1; -fi -if [ "x$inurl" = x ]; then - echo "$0: full input URL (third argument) not given."; exit 1; -fi -if [ "x$outname" = x ]; then - echo "$0: output name (fourth argument) not given."; exit 1; -fi - - - - - -# Try downloading multiple times before crashing. -counter=0 -maxcounter=10 -while [ ! -f "$outname" ]; do - - # Increment the counter. We need the `counter=' part here because - # without it the evaluation of arithmetic expression will be like and - # error and the script is set to crash on errors. - counter=$((counter+1)) - - # If we have passed a maximum number of trials, just exit with - # a failed code. - if (( counter > maxcounter )); then - echo - echo "Failed $maxcounter download attempts: $outname" - echo - exit 1 - fi - - # If this isn't the first attempt print a notice and wait a little for - # the next trail. - if (( counter > 1 )); then - tstep=$((counter*5)) - echo "Download trial $counter for '$outname' in $tstep seconds." - sleep $tstep - fi - - # Attempt downloading the file (one-at-a-time). Note that the - # `downloader' ends with the respective option to specify the output - # name. For example "wget -O" (so `outname', that comes after it) will - # be the name of the downloaded file. - flock "$lockfile" bash -c \ - "if ! $downloader $outname $inurl; then rm -f $outname; fi" -done - - - - - -# Return successfully -exit 0 diff --git a/reproduce/src/bash/git-post-checkout b/reproduce/src/bash/git-post-checkout deleted file mode 100644 index 9552f01..0000000 --- a/reproduce/src/bash/git-post-checkout +++ /dev/null @@ -1,66 +0,0 @@ -#!@BINDIR@/bash -# -# The example hook script to store the metadata information of version -# controlled files (with each commit) using the `metastore' program. -# -# Copyright (C) 2016 Przemyslaw Pawelczyk <przemoc@gmail.com> -# Copyright (C) 2018-2019 Mohammad Akhlaghi <mohammad@akhlaghi.org> -# -# This script is taken from the `examples/hooks/pre-commit' file of the -# `metastore' package (installed within the project, with an MIT license -# for copyright). We have just changed the name of the `MSFILE' and also -# set special characters for the installation location of meta-store so our -# own installation is found by Git. -# -# Permission is hereby granted, free of charge, to any person obtaining a -# copy of this software and associated documentation files (the -# "Software"), to deal in the Software without restriction, including -# without limitation the rights to use, copy, modify, merge, publish, -# distribute, sublicense, and/or sell copies of the Software, and to permit -# persons to whom the Software is furnished to do so, subject to the -# following conditions: -# -# The above copyright notice and this permission notice shall be included -# in all copies or substantial portions of the Software. -# -# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS -# OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF -# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN -# NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, -# DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR -# OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE -# USE OR OTHER DEALINGS IN THE SOFTWARE. - - -# File containig the metadata and metastore executable. -MSFILE=".file-metadata" -MSBIN=@BINDIR@/metastore - -# If metastore is not installed, then ignore this script (exit with a -# status of 0). -if [ ! -f $MSBIN ]; then exit 0; fi - -# Delete all temporary files -find @TOP_PROJECT_DIR@/ -name "*~" -type f -delete - -# Function to help in reporting a crash. -exit_on_fail() { - "$@" - if [ $? -ne 0 ]; then - echo "Failed to execute: $@" >&2 - exit 1 - fi -} - -# Check if the metadata file exists. -if [ ! -e "$MSFILE" ]; then - echo "\"$MSFILE\" missing" >&2 - exit 1 -fi - -# Run metastore. -exit_on_fail \ - $MSBIN -a -m -e -E -q -O @USER@ -G @GROUP@ -f "$MSFILE" - -# Return with a success code (0). -exit 0 diff --git a/reproduce/src/bash/git-pre-commit b/reproduce/src/bash/git-pre-commit deleted file mode 100644 index dbe0ecc..0000000 --- a/reproduce/src/bash/git-pre-commit +++ /dev/null @@ -1,83 +0,0 @@ -#!@BINDIR@/bash -# -# The example hook script to store the metadata information of version -# controlled files (with each commit) using the `metastore' program. -# -# Copyright (C) 2016 Przemyslaw Pawelczyk <przemoc@gmail.com> -# Copyright (C) 2018-2019 Mohammad Akhlaghi <mohammad@akhlaghi.org> -# -# WARNING: -# -# If the commit is aborted (e.g. by not entering any synopsis), -# then updated metastore file (.metadata by default) is not reverted, -# so its new version remains in the index. -# To undo any changes in metastore file written since HEAD commit, -# you may want to reset and checkout HEAD version of the file: -# -# git reset HEAD -- .metadata -# git checkout HEAD -- .metadata -# -# This script is taken from the `examples/hooks/pre-commit' file of the -# `metastore' package (installed within the project, with an MIT license -# for copyright). Here, the name of the `MSFILE' and also set special -# characters for the installation location of meta-store so our own -# installation is found by Git. -# -# Permission is hereby granted, free of charge, to any person obtaining a -# copy of this software and associated documentation files (the -# "Software"), to deal in the Software without restriction, including -# without limitation the rights to use, copy, modify, merge, publish, -# distribute, sublicense, and/or sell copies of the Software, and to permit -# persons to whom the Software is furnished to do so, subject to the -# following conditions: -# -# The above copyright notice and this permission notice shall be included -# in all copies or substantial portions of the Software. -# -# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS -# OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF -# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN -# NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, -# DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR -# OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE -# USE OR OTHER DEALINGS IN THE SOFTWARE. - -# File containig the metadata and metastore executable. -MSFILE=".file-metadata" -MSBIN=@BINDIR@/metastore - -# If metastore is not installed, then ignore this script (exit with a -# status of 0). -if [ ! -f $MSBIN ]; then exit 0; fi - -# Function to help in reporting a crash. -exit_on_fail() { - "$@" - if [ $? -ne 0 ]; then - echo "Failed to execute: $@" >&2 - exit 1 - fi -} - -# Run metastore. -exit_on_fail \ - $MSBIN -O @USER@ -G @GROUP@ -s -f "$MSFILE" - -# If it's first metastore commit, store again to include $MSFILE in $MSFILE. -if ! git-ls-tree --name-only HEAD 2>/dev/null | grep -Fqx "$MSFILE"; then - exit_on_fail \ - $MSBIN -O @USER@ -G @GROUP@ -s -f "$MSFILE" -fi - -# Check if the metadata file exists. -if [ ! -e "$MSFILE" ]; then - echo "\"$MSFILE\" missing" >&2 - exit 1 -fi - -# Add the metadata file to the Git repository. -exit_on_fail \ - git-add "$MSFILE" - -# Return with a success code (0). -exit 0 |