|author||Mohammad Akhlaghi <firstname.lastname@example.org>||2021-04-25 23:52:22 +0100|
|committer||Mohammad Akhlaghi <email@example.com>||2021-04-25 23:59:41 +0100|
Until now, while the series of steps mentioned in 'README.md' were complete, they had some implicit thing in them that made it a little hard to run as a checklist (the commands to do some basic things weren't included). Also, it was recommending to run a long 'docker run ...' command, which wasn't too user friendly. With this commit, the series of steps is now a complete checklist, containing every step. Also, the checklist now recommends putting the long 'docker run' command inside a script called 'docker-run' that will also do a 'sudo' internally (thus making things very easy for a first-time user). Also, since the 'docker-run' script contains host OS-specific directory names, it should not be under control, so it has been added to the '.gitignore' file in case users decide to keep this same name (which is recommended).
2 files changed, 151 insertions, 76 deletions
@@ -39,8 +39,9 @@ tex/tikz
-.compiler_test_dir_please_delete \ No newline at end of file
@@ -459,6 +459,8 @@ docker cp CONTAINER:/file/path/within/container /host/path/target
#### Only software environment in the Docker image
You can set the docker image to only contain the software environment and
@@ -468,96 +470,168 @@ image to a minimum (only containing the built software environment) to
easily move it from one computer to another. Below we'll summarize the
-1. Get your user ID with this command: `id -u`.
+ 1. Get your user ID with this command: `id -u`.
-2. Put the following lines into a `Dockerfile` of an otherwise empty
-directory. Just replacing `UID` with your user ID (found in the step
-above). This will build the basic directory structure. for the next steps.
+ 2. Make a new (empty) directory called `docker` temporarily (will be
+ deleted later).
-RUN apt-get update && apt-get install -y gcc g++ wget
-RUN useradd -ms /bin/sh --uid UID maneager
-RUN mkdir build
+ mkdir docker
+ cd docker
-3. Create an image based on the `Dockerfile` above. Just replace `PROJECT`
-with your desired name.
+ 3. Make a `Dockerfile` (within the new/empty directory) with the
+ following contents. Just replacing `UID` with your user ID (found in
+ step 1 above).
-docker build -t PROJECT ./
+ FROM debian:stable-slim
+ RUN apt-get update && apt-get install -y gcc g++ wget
+ RUN useradd -ms /bin/sh --uid UID maneager
+ USER maneager
+ WORKDIR /home/maneager
+ RUN mkdir build
-4. Run the command below to create a container based on the image and mount
-the desired directories on your host into the special directories of your
-container. Just don't forget to replace `PROJECT` and set the `/PATH`s to
-the respective paths in your host operating system.
+ 4. Create a Docker image based on the `Dockerfile` above. Just replace
+ `MANEAGEBASE` with your desired name (this won't be your final image,
+ so you can safely use a name like `maneage-base`). Note that you need
+ to have root/administrator previlages when running it, so
-docker run -v /PATH/TO/PROJECT/SOURCE:/home/maneager/source \
- -v /PATH/TO/PROJECT/ANALYSIS/OUTPUTS:/home/maneager/build/analysis \
- -v /PATH/TO/SOFTWARE/SOURCE/CODE/DIR:/home/maneager/software \
- -v /PATH/TO/RAW/INPUT/DATA:/home/maneager/data \
- -it PROJECT
+ sudo su
+ docker build -t MANEAGEBASE ./
-5. After running the command above, you are within the container. Go into
-the project source directory and run these commands to build the software
+ 5. You don't need the temporary directory any more (the docker image is
+ saved in Docker's own location, and accessible from anywhere).
-./project configure --build-dir=/home/maneager/build \
- --software-dir=/home/maneager/software \
+ cd ..
+ rm -rf docker
-6. After the configuration finishes successfully, it will say so and ask
-you to run `./project make`. But don't do that yet. Keep this Docker
-container open and don't exit the container or terminal. Open a new
-terminal, and follow the steps described in the sub-section above to
-preserve the built container as a Docker image. Let's assume you call it
-`PROJECT-ENV`. After the new image is made, you should be able to see the
-new image in the list of images with this command (in the same terminal
-that you created the image):
+ 6. Put the following contents into a newly created plain-text file called
+ `docker-run`, while setting the initial variables based on your system
+ (the `software_dir` and `data_dir` can point to empty directories: if
+ you don't already have the necessary software or data, they
+ will/should be downloaded automatically).
-docker image list # In the other terminal.
+ # Create Docker container from existing image. This script be run in the
+ # top project source directory (that has 'README.md' and 'paper.tex'). If
+ # not, replace the '$(pwd)' with the project source directory.
+ sudo docker run -v $(pwd):/home/maneager/source \
+ -v $analysis_dir:/home/maneager/build/analysis \
+ -v $software_dir:/home/maneager/software \
+ -v $data_dir:/home/maneager/data \
+ -it $docker_name
-7. Now you can run `./project make` in the initial container. You will see
-that all the built products (temporary or final datasets or PDFs), will be
-written in the `/PATH/TO/PROJECT/ANALYSIS/OUTPUTS` directory of your
-host. You can even change the source of your project on your host operating
-system an re-run Make to see the effect on the outputs and add/commit the
-changes to your Git history within your host. You can also exit the
-container any time. You can later load the `PROJECT-ENV` environment image
-into a new container with the same `docker run -v ...` command above, just
-use `PROJECT-ENV` instead of `PROJECT`.
+ 7. Make the `docker-run` script executable and **put its name in your
+ `.gitignore`**. It is important that this file doesn't go into your
+ project's history because it contains directory names only for this
+ particular system (you can always recreate it and update the directory
+ values for another system, by looking at this `README.md` and
+ copy-pasting). If you use the standard `docker-run` name for this tiny
+ script, it is already included in Maneage's `.gitignore`, so you don't
+ need to re-insert it there.
-8. In case you want to store the image as a single file as backup or to
-move to another computer, you can run the commands below. They will produce
-a single `project-env.tar.gz` file.
+ chmod docker-run
+ emacs .gitignore
-docker save -o project-env.tar PROJECT-ENV
-gzip --best project-env.tar
+ 8. You can now start the Docker image by executing your newly added
+ script like below (it will ask for your root password). You will
+ notice that you are in the Docker container with the changed prompt.
+ 9. You are now within the container. Go into the project source directory
+ and run these commands to build the software environment.
+ cd source
+ ./project configure --build-dir=/home/maneager/build \
+ --software-dir=/home/maneager/software \
+ 10. After the configuration finishes successfully, it will say so. It will
+ then ask you to run `./project make`. **But don't do that yet**. Keep
+ this Docker container open and don't exit the container or
+ terminal. Open a new terminal, and follow the steps described in the
+ sub-section above to preserve (or "commit") the built container as a
+ Docker image. Let's assume you call it `MY-PROJECT-ENV`. After the new
+ image is made, you should be able to see the new image in the list of
+ images with this command (in yet another terminal):
+ docker image list # In the other terminal.
+ 11. Now that you have safely "committed" your current Docker container
+ into a separate Docker image, you can **exit the container** safely
+ with the `exit` command. Don't worry, you won't loose the built
+ software environment: it is all now saved separately within the Docker
+ 12. Re-open your `docker-run` script and change `MANEAGEBASE` to
+ `MY-PROJECT-ENV` (or any other name you set for the environment you
+ committed above).
+ emacs docker-run
+ 13. That is it! You can now always easily enter your container (only for
+ the software environemnt) with the command below. Within the
+ container, any file you save/edit in the `source` directory of the
+ docker container is the same file on your host OS and any file you
+ build in your `build/analysis` directory (within the Maneage'd
+ project) will be on your host OS. You can even use your container's
+ Git to store the history of your project in your host OS. See the next
+ step in case you want to move your built software environment to
+ another computer.
+ 14. In case you want to store the image as a single file as backup or to
+ move to another computer, you can run the commands below. They will
+ produce a single `project-env.tar.gz` file.
+ docker save -o my-project-env.tar MY-PROJECT-ENV
+ gzip --best project-env.tar
+ 15. To load the tarball above into a clean docker environment (for example
+ on another system) copy the `my-project-env.tar.gz` file there and run
+ the command below. You can then create the `docker-run` script for
+ that system and run it to enter. Just don't forget that if your
+ `analysis_dir` directory is empty on the new/clean system. So you
+ should first run the same `./project configure ...` command above in
+ the docker image so it connects the environment to your source. Don't
+ worry, it won't build any software and should finish in a second or
+ two. Afterwards, you can safely run `./project make` and continue
+ working like you did on the old system.
+ docker load --input my-project-env.tar.gz
-9. To load the tarball above into a clean docker environment (either on the
-same system or in another system), and create a new container from the
-image like above (the `docker run -v ...` command). Just don't forget that
-if your `/PATH/TO/PROJECT/ANALYSIS/OUTPUTS` directory is empty on the
-new/clean system, you should first run `./project configure -e` in the
-docker image so it builds the core file structure there. Don't worry, it
-won't build any software and should finish in a second or two. Afterwards,
-you can safely run `./project make`.
-docker load --input project-env.tar.gz