Introduction to containers

[aturner-epcc]

The lesson does not really introduce what containers are. This is a bit of a hangover from the origin of the lesson as a second day of a workshop with the Docker containers course (which does introduce containers) on the first day. We probably need to either:

1. Update the requirements to note that you need pre-existing experience with containers for this lesson
2. Add additional content to the lesson (perhaps from the Docker lesson) introducing containers

[orchid00]

I have a video solution for this!
Australian Research Data Commons, 2021. How can software containers help your research?. [video] Available at: https://www.youtube.com/watch?v=HelrQnm3v4g DOI: http://doi.org/10.5281/zenodo.5091260

<iframe width="560" height="315" src="https://www.youtube.com/embed/HelrQnm3v4g" title="How can software containers help your research?" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>

[jonathancasco]

As this is meant to be an introductory course for Singularity providing information on how containers help with portability and collaboration (as mentioned in the video link) is more likely to be useful for students than the more in depth explanation on how containers work within the OS.

[liz-is]

I'm interested in running this workshop at my institution - I think the primary audience would be people who want to use (and maybe customise/extend) Singularity containers to run on our HPC platform. Therefore I'd probably want to run just the Singularity workshop, supplemented by adding the introduction episode from the Docker workshop.

I was wondering if people who've run this material before have any thoughts about how feasible this would be / what other content we'd need to bring over from the Docker lesson? I can see some of the content about finding containers on Dockerhub and understanding their versions being useful as well, since Singularity can run from a Docker image.

[jcohen02]

I was wondering if people who've run this material before have any thoughts about how feasible this would be / what other content we'd need to bring over from the Docker lesson? I can see some of the content about finding containers on Dockerhub and understanding their versions being useful as well, since Singularity can run from a Docker image.

Great to hear that you're interested in running the course @liz-is. When we've run this course previously, we tend to run alongside the introduction to Docker material so we cover Docker on the first day and then Singularity on the second day. However, there is a fair bit of crossover between the two courses.

One option, along the lines of what you suggest, could be to add the "Introducing containers" section from the docker course to the start of the Singularity course (and then update the "Singularity: Getting Started" section to remove any duplication from the intro material where necessary). Then you could integrate the Docker Hub material - you might want to split the "Singularity: Getting Started" section and put the Docker Hub material in the middle somewhere, just before where it talks about Singularity Hub - this may also not be so relevant if you want to focus on getting containers from Docker Hub. I might also make the Docker Hub material a bit more generic and have a section that says something like "Working with container repositories" where you can highlight the concept of a container repository and then point out that one of the most widely used such repositories is Docker Hub where there's a huge range of existing container images available. Then you could highlight that Singularity/Apptainer can pull and use images directly from Docker Hub. After that you've probably got something that would work as a useful standalone course.

Something similar was done with the Singularity course run at RSECon23 where I believe it was run as a standalone course and the sort of integration you've suggested has been done.

Hope that is of some help. It would be good to hear any other thoughts from the wider community on how this material may be effectively used as a standalone course.

[aturner-epcc]

When I have run this in one day in the past I typically drop all the parts on building container images using Singularity/Apptainer itself and bring in the information from the Docker course on building container images (which should now probably switch to Podman, a different story/issue!) . The reasoning is that in a real workflow with containers you would not expect anyone to ever bother buidling containers using Singularity/Apptainer, it is too much of a pain to setup and the flexibility is so inferior to building with Docker/Podman. What I want is the lesson I teach to reflect what people would do in practice, which is typically:

• Build images using Docker/Podman on their local resource (laptop/workstation)
• Push images to some repo (e.g. Docker Hub, quay.io, Github)
• Download images from the repo onto the HPC system using Singularity/Apptainer and convert to SIF
• Use images on HPC system

[liz-is]

Thanks both! Seeing an example of a standalone course is very helpful.

@aturner-epcc that's a great point, that's the only way I've seen a real workflow work as well. Having looked through the Docker material now, it seems like it would work well to mix and match in this way, with a bit of explanation around the differences between Singularity and Docker to act as a bridge. Have you found that it fits well in one day when you've run this in the past?

[aturner-epcc]

Yes, it fitted into a day fine in this format. I will have a look about and see if I can find a repo for when I taught it in this way.