aboutsummaryrefslogtreecommitdiff
path: root/reproduce/src/make/dependencies-python.mk
AgeCommit message (Collapse)AuthorLines
2019-04-12Dependency BibTeX entries included only when necessaryMohammad Akhlaghi-6/+9
Until now, there was a single `tex/src/references.tex' file that housed the BibTex entries for everything (software and non-software). Since we have started to include the BibTeX entry for more software, it will be hard to manage the large (sometime unused) BibTeX entries of the software in the middle of the non-software related citations in the text of the paper. Therefore, with this commit, a `tex/dependencies' directory has been made which has a separate BibTeX entry file for each software that needs one. After the software is built, this file is copied to the new `.local/version-info/cite' directory. At the end, the configure script will concatenate all the files in this directory into one file which will later be used with `tex/src/references.tex' by BibLaTeX. This greatly simplifies managing of citations. Allowing us to focus on the software-building and paper-writing citations separately/cleanly (and thus be more efficient in both).
2019-04-12Fixed some Scipy-related packages citationsRaul Infante-Sainz-1/+1
Until now, the Scipy citation was only one paper and not the correct one (it was the online manual). With this commit, Scipy is properly cited using the two papers. Also some modifications in the `tex/src/references.tex' have been done (remove last page number).
2019-04-12Acknowledged Scipy-related packages: Cython, Matplotlib, Numpy and ScipyRaul Infante-Sainz-11/+12
Until now, name and version of all Python packages were indicated in the final paper, but not the main paper of them (if it exists). With this commit, some Python packages (Cython, Matplotlib, Numpy and Scipy) are now properly acknoledged by citating the source paper. `mpi4py' is also cited although this package is not yet included into the pipeline.
2019-04-11Unzip set as prerequisite of Numpy and SetuptoolsRaul Infante-Sainz-2/+4
Since we mixed the installation of Python packages with all other software, it may occur that some Python packages start to be installed before having installed `unzip'. As a consecuence, they could not be decompressed and the installation will fail. In particular, tarballs of Numpy and Setuptools are .zip files. With this commit, we fix this issue by setting `unzip' as a prerequisit of Numpy and Setuptools.
2019-04-07Corrections in configure script and astroquery, astropy rulesMohammad Akhlaghi-6/+9
The step where we check the possibility of using `sys/cdefs.h' was still using `$$' for shell variables (in Make), not `$' (for the shell). This was corrected. Also, since Astropy needs two citations, the `,' in the citation command would conflict with Make's parsing. So we just used an `echo' command to re-write the version info. In Astroquery, the prerequisite list was just reordered by length to be more clear to the eye.
2019-04-05GCC not building on GNU/Linux system with incomplete complierMohammad Akhlaghi-5/+11
On some GNU/Linux distros, the compiler is separated into `multilib' mode (for 32-bit and 64-bit support) and by default the extra component of the compiler is not installed! In such systems for now, we are just creating symbolic links to the host's compiler (similar to Mac). While testing, I noticed that we weren't passing a "$downloader" option to the downloading script of `dependencies.mk' and `dependencies-python.mk'. Also, I noticed that the Cython and Python-pkg-config packages didn't have setuptools as a dependency! Both have now been fixed. Also, Cython's tarball name is now all small-caps (as in all the other tarballs).
2019-04-05Imported work on adding Cython and Python's pkgconfig moduleMohammad Akhlaghi-3/+17
This work is now merged, I just added the new argument to the `pybuild' function.
2019-04-05Software acknowledgement section is automatically generatedMohammad Akhlaghi-75/+108
Until now, management of the software names and versions in the paper was done manually (a macro had to be defined in `initialize.mk', then used in `paper.tex', so they had to be manually set in two places). Managing this was not easy. To fix this, with this commit, each software building rule's target is a text file that contains its human-readable name and its version. In the end, the configure script sorts them by their name and writes them into a LaTeX input file that we can easily import as a file into the main paper.
2019-04-05Python cython and pkgconfig packages added for h5pyRaul Infante-Sainz-3/+15
After trying to set the pipeline from scratch with no internet conection (but all tarballs already downladed), `h5py' Python package complained about not having access to download `pkgconfig'. After solving this dependency, it also complained about not having `cython'. With this commit, we add `pkgconfig' (Python) and `cython' to the pipeline in order to be able to install `h5py' properly.
2019-04-03OpenMPI, HDF5 libraries added for h5pyMohammad Akhlaghi-2/+22
After trying the build a system with no Python library, I noticed that Python's HDF5 module (`h5py') needs the HDF5 library and OpenMPI (to work in parallel). So they were added. Finally `h5py' uses the `mpi4py' module to communicate with OpenMPI, so it was also added. However, for some reason, mpi4py doesn't work with this version of OpenMPI (as described in the comments above). So for now, h5py doesn't use it and can only work on a single thread, while the HDF5 C library links with OpenMPI with no problem.
2019-04-02Python packages are installed as high level program dependenciesRaul Infante-Sainz-51/+6
Until this commit, the installation of all Python packages were done in a separate Makefile. With this commit, the pipeline install Python packages as part of the hight level software. All Python packages rules them remain in a separate Makefile, but this Makefile is included in the high level dependency `reproduce/src/make/dependencies.mk'.
2019-04-02Numpy and Scipy using OpenBLAS instead of ATLASRaul Infante-Sainz-8/+16
We could not get ATLAS shared libraries on Mac (while the static ATLAS libraries are built and can be used successfully on Mac). So, the pipeline now builds OpenBLAS, which both Numpy and Scipy can use on Mac and GNU/Linux. We also added FFTW as a dependency of Numpy. Altough Numpy is not linking to FFTW for some reason. However, since FFTW is a low level library used by many programs, we have kept it as a dependency of Numpy anyway for now.
2019-03-29Disabling ATLAS shared libraries on Mac OSRaul Infante-Sainz-9/+5
The Makefile that build the shared libraries comes from Arch Linux so it does not work easily on Mac. But the full ATLAS build goes successfully for static libraries. For now we are disabling shared libraries on Mac. Python was built explicity with `clang' on Mac.
2019-03-27GCC is now built on a Mac, not yet ATLASRaul Infante-Sainz-1/+4
Until now, we were simply using the host's GCC for Mac systems. But we found that except for a single step (to fixing `rpath'), it works on Mac!!! So, GCC is now part of the Mac build as well. However, we are still having some problems in building ATLAS on Mac. It works on GNU/Linux, but not in Mac. So for the time being (just temporarily), we are avoiding ATLAS (and thus Scipy) on Mac systems. We just filed an issue on the ATLAS discussion list to hopefully fix the problem soon.
2019-03-22H5py added and setuptools as main Python module dependencyRaul Infante-Sainz-27/+36
In this commit we add `h5py' Python package. We also include `setuptools' as a main dependency of Python because with the previous commit it (as well as `pip') is no longer installed with Python. Numpy version also has been incremented.
2019-03-21ATLAS and Scipy working on GNU/LinuxRaul Infante-Sainz-17/+34
Numpy needs ATLAS as shared libraries. So we also need to build Python with shared libraries. We also need to define site.cfg for numpy and scipy so we define a master template: `reproduce/config/pipeline/dependency-numpy-scipy.cfg' Also `Openssl' did not have rpath so we added with this commit.
2019-03-20Including ATLAS in the pipeline, not yet completeRaul Infante-Sainz-3/+0
An initial installation of atlas is now included in the pipeline, but we are still trying to make it compile and build smoothly. In the process, we found that GCC also needs some modifications (for example rpath issues).
2019-03-07Several new Python packages added for full buildMohammad Akhlaghi-12/+70
To ensure that we have all the necessary Python dependencies, I done an offline build and noticed that several packages were also necessary for the `./configure' step to finish (`libffi', `asn1crypto', `cffi', `jeepney', `pycparser' and `secretstorage'). With this commit they are added.
2019-03-06Reseting Python environment variables also when building PythonMohammad Akhlaghi-1/+20
Until now, we were only resetting the Python environment variables in the actual processing Makefiles, not in the Makefile that build Python and its modules. They are now added there also.
2019-03-06Astroquery, astropy, matplotlib and numpy are now in the pipelineRaul Infante-Sainz-0/+1
Until this commit, we had some of the python packages intalled but they did not work properly because of the `PYTHONPATH' variables. That is, the pipeline's `python' was the `python' of the system instead of the pipeline's `python'. With this commit this issue has been fixed by setting the correct `PYTHONPATH'. In this commit we also modify the installation of `bzip2' because `CMake' was complaining about some libraries built statically.
2019-02-28Python installation and related packages have been splitRaul Infante-Sainz-1/+13
Until now the installation of Python and its packages (numpy, astropy, astroquery, etc.) were done in the same `makefile'. With this commit the installation of Python and its packages have been split and now it is independent of the other programs. The installation of all Python packages needs to be written explicitely because pip is not used anymore.
2019-02-21Matplotlib is now in the pipelineRaul Infante-Sainz-5/+63
As matplotlib is a general package for plotting and it is widely used in science, we have added it to the pipeline. When installing a dependency of matplotlib `python-dateutil', we found a conflict in the download of the tarball. This is because the name has a dash (-) in the middle. In addition, the name starts with 'python', so it is the same as the python itself. Now it is possible to install any package with any name, just adding an elif in before the URL direction.
2019-02-20Installed astroquery in the pipelineRaul Infante-Sainz-0/+294
All dependencies for building astroquery package have been done. Until nowthe Python dependencies were built in the same Makefile as the high level libraries and programs. But, because astroquery has many dependencies we split the Python and Python packages installation in a new Makefile. The installation of differents packages are done using Python and not pip, because we found some problems when doing it with pip. Apparently there are some interferences between the packages installed by the pip of the system and the pip installed as part of Python in the pipeline.