|
1 |
| -# Using a Custom Production Image |
| 1 | +# Building a Custom Production Image |
2 | 2 |
|
3 | 3 | Using a container has many advantages and one of them is the ability to develop code on one machine (e.g. your personal laptop),
|
4 | 4 | but the deploy the _exact same code_ to run on several other computers (e.g. SLAC batch computing).
|
5 |
| -This page details a process you can follow to generate your own production image that has your developments of ldmx-sw inside of it and has two big parts. |
6 |
| - |
7 |
| -1. Building a custom production image with your developments |
8 |
| -2. Running any production image (with specific focus on using `singularity`). |
9 |
| - |
10 |
| -## 1. Building a Custom Production Image |
| 5 | +This page details a process you can follow to generate your own production image that has your developments of ldmx-sw inside of it. |
| 6 | +Refer to the [Batch Computing](../using/batch.md) for how to use these production images. |
11 | 7 |
|
12 | 8 | Building a docker image is complicated, but hopefully you can get one of the two methods listed below to work for your specific case.
|
13 | 9 | The common denominator in these methods is that you *need* to have a DockerHub repository that you have administrator access to.
|
@@ -61,34 +57,3 @@ _Note: If you haven't yet, you may need to `docker login` on your computer for t
|
61 | 57 | ```
|
62 | 58 | docker push docker-user-name/docker-repo-name:some-tag
|
63 | 59 | ```
|
64 |
| - |
65 |
| -## 2. Running the Production Image on the Batch Computer |
66 |
| -0. Decide where you want to save the production image: `export LDMX_PRODUCTION_IMG=$(pwd -P)/ldmx_my_production_image_some_tag.sif` |
67 |
| -1. Pull the docker container down from Docker Hub and build it into a `.sif` image file. _Note: This step requires your Docker Hub repository to be public._ |
68 |
| -``` |
69 |
| -singularity build ${LDMX_PRODUCTION_IMG} docker://docker-user-name/docker-repo-name:some-tag |
70 |
| -``` |
71 |
| -2. Now you can run a configuration script with your developments in the container using |
72 |
| -``` |
73 |
| -singularity run --no-home ${LDMX_PRODUCTION_IMG} . config.py |
74 |
| -``` |
75 |
| -This is the command you want to be giving to `bsub` or some other submission program. |
76 |
| -The only files it needs access to are the configuration script that you want to run and the `.sif` image file; |
77 |
| -both of which are only used at the start-up of the container. |
78 |
| - |
79 |
| -_Note: On SLAC computers, the default singularity cache directory is $HOME, but SLAC users are not given very much space in $HOME. It may help your singularity build and run commands if you change the cache directory 'SINGULARITY_CACHEDIR' to somewhere with more space._ |
80 |
| - |
81 |
| -## 3. Submission Script |
82 |
| -It is best practice to write a "submission script" that handles the running of this command _and_ any pre- or post- run actions. |
83 |
| -A lot of different submission scripts have been written in `bash` and `python`, but they all have a similar structure: |
84 |
| -1. Setup the batch environment (e.g. Find singularity image file and turn off email notifications) |
85 |
| -2. Configure or write a job script which does all the pre- and post- run actions as well as the `singularity run` command. |
86 |
| - - Go to a scratch or temporary directory to work |
87 |
| - - Pre-Run Actions: copying over input file, inserting parameters into configuration script, etc... |
88 |
| - - Run `singularity run` command |
89 |
| - - Post-Run Actions: copying output files to output directory cleaning up scractch directory |
90 |
| -3. Submit the job script using the submission program (e.g. `bsub` or `condor`) however many times |
91 |
| - |
92 |
| -The `batch` directory in the [LDMX-Software/ldmx-sw-scripts](https://github.com/LDMX-Software/ldmx-sw-scripts) |
93 |
| -repository offers some examples of these submission scripts, although they tend to be a little old |
94 |
| -and will need to be updated. |
0 commit comments