diff options
| author | Mohammad Akhlaghi <mohammad@akhlaghi.org> | 2019-10-02 12:47:06 +0100 |
|---|---|---|
| committer | Mohammad Akhlaghi <mohammad@akhlaghi.org> | 2019-10-02 12:47:06 +0100 |
| commit | d605dc03d1d52e9b7acac97df0afc7f2c6b9647e (patch) | |
| tree | 2e36ec18caff48260cb6b98fcafd061d72abef43 /reproduce/analysis | |
| parent | 8bbe54dd3db007752cd1145caaaa9fe7d7084762 (diff) | |
Possibile to use download-multi-try script without locks
Until now, this script would always only work with a file-lock. But in some
scenarios, we might want to download in parallel. For example when the
system has multiple ports to the internet.
With this commit, we have added this feature: when the lockfile name is
`nolock', it won't lock and will download in parallel.
Diffstat (limited to 'reproduce/analysis')
| -rwxr-xr-x | reproduce/analysis/bash/download-multi-try | 15 |
1 files changed, 11 insertions, 4 deletions
diff --git a/reproduce/analysis/bash/download-multi-try b/reproduce/analysis/bash/download-multi-try index 03472b3..2ac106c 100755 --- a/reproduce/analysis/bash/download-multi-try +++ b/reproduce/analysis/bash/download-multi-try @@ -16,13 +16,15 @@ # it can't continue. The job of this script is to be patient and try the # download multiple times before crashing the whole project. # -# LOCK FILE: Since there is ultimately only one network port to the outside +# LOCK FILE: Since there is usually only one network port to the outside # world, downloading is done much faster in serial, not in parallel. But # the project's processing may be done in parallel (with multiple threads # needing to download different files at the same time). Therefore, this # script uses the `flock' program to only do one download at a time. To # benefit from it, any call to this script must be given the same lock -# file. +# file. If your system has multiple ports to the internet, or for any +# reason, you don't want to use a lock file, set the `lockfile' name to +# `nolock'. # # Copyright (C) 2019 Mohammad Akhlaghi <mohammad@akhlaghi.org> # @@ -102,8 +104,13 @@ while [ ! -f "$outname" ]; do # `downloader' ends with the respective option to specify the output # name. For example "wget -O" (so `outname', that comes after it) will # be the name of the downloaded file. - flock "$lockfile" bash -c \ - "if ! $downloader $outname $inurl; then rm -f $outname; fi" + if [ x"$lockfile" = xnolock ]; then + if ! $downloader $outname $inurl; then rm -f $outname; fi + else + flock "$lockfile" bash -c \ + "if ! $downloader $outname $inurl; then rm -f $outname; fi" + fi + done |