aboutsummaryrefslogtreecommitdiff
path: root/reproduce/analysis/bash
diff options
context:
space:
mode:
authorMohammad Akhlaghi <mohammad@akhlaghi.org>2022-08-25 17:02:29 +0200
committerMohammad Akhlaghi <mohammad@akhlaghi.org>2022-09-02 19:45:31 +0200
commit43186705c89e99fd4dbf5549ad031d27d77dfe6f (patch)
tree284b34727c4124eb1f77e11dce3a92ad682ad273 /reproduce/analysis/bash
parent79ee52ed1e136fbd2592a5c2a058f73024d8b5b0 (diff)
Added server authentication and FITS DATASUM for verficiation
SUMMARY: Nothing special is necessary for your existing projects. This commit just addds two new features (read the commit description for more): 1. To provide a user and password to servers that need authentication before they allow downloading of proprietary data, 2. To use the FITS Standard's DATASUM for file verification (for cases where the file is not static on the server, and is generated upon receiving your download request). Until now, Maneage didn't have any infrastructure for databases that require authentication (through a user or password, when calling 'wget'). Furthermore, when the downloaded file is automatically generated by the server upon request, the server usually adds metadata (like file date, or query number and etc) in the header. Therefore the simple SHA256 checksum of the file would differ on every download! This made it very hard to verify if the data (not headers) are unchanged. With this commit, both these problems have been addressed: - Server authentication: the 'reproduce/software/config/LOCAL.conf' now contains three new variables for this purpose. With them, you can give your username and password, along with the authentication method of the server. The comments on top of these three variables give a full description of their usage. - Verifying only the data in a file (ignoring the headers): The 'reproduce/analysis/config/INPUTS.conf' now accepts two new optional variables for each input file using the FITS standard's DATASUM convention: 'INPUT-%-fitsdatasum' and 'INPUT-%-fitshdu'. If the SHA256 isn't specified for a file, Maneage will use these to verify the file. With the latter, you specify the HDU of the data you want to verify and with the former you give the DATASUM value for that HDU. As the name suggests, this is only valid for FITS files. If we find other formats that support a similar behavior, we can add this feature for those formats also. This is also thoroughly discussed in the comments of 'reproduce/analysis/config/INPUTS.conf'. This commit was done with the help of Pedram Ashofte Ardakani, Sepideh Eskandarlou and Mohammadreza Khellat.
Diffstat (limited to 'reproduce/analysis/bash')
0 files changed, 0 insertions, 0 deletions