Is there any way to cache docker images for use in CI?

[alex-statsig]

I've been using testcontainers for tests running in CI for a while, and one thing I hadn't thought about is how it will cache images. The mongodb image we use is ~400MB, and I assume it's pulling that on every run currently. Is there a certain folder I can cache in my CI (ex. ~/.testcontainers or something) to prevent needing to redownload these images every time? Or a recommended way to first pull the image to a file, then use the image locally so it can be cached?

[cristianrgreco]

Hi @alex-statsig,

I assume your mongodb image is being built with Testcontainers? If yes then indeed the built image will be removed once the test suite ends because of the resource reaper (ryuk). If this is undesired you can simply turn off the resource reaper by setting the env variable TESTCONTAINERS_RYUK_DISABLED=true.

[alex-statsig]

Yes that's correct. I ended up switching to use github's builtin action containers if the action is running in CI (and only use testcontainers for tests running locally) so that I could just rely on github's built-in caching. For posterity's sake, where are the images stored if TESTCONTAINERS_RYUK_DISABLED is set? Theoretically that folder could be cached in CI

[cristianrgreco]

I ended up switching to use github's builtin action containers if the action is running in CI (and only use testcontainers for tests running locally) so that I could just rely on github's built-in caching

Could you pls give a bit more info about the GH built-in cache to which you are referring?

For posterity's sake, where are the images stored if TESTCONTAINERS_RYUK_DISABLED is set? Theoretically that folder could be cached in CI

Docker itself maintains its own image cache, I believe in /var/lib/docker/<storage-driver>. There's nothing custom that Testcontainers adds on top.

[alex-statsig]

I updated my github action to include in services:

      mongo:
        image: mongo:6
        ports:
          - 27017:27017

I'm not totally sure, but I assume github has optimized the pulling of images configured in these services (it seems faster, perhaps just from parallelization with other service setup though)

Sounds like caching the directory /var/lib/docker/ and using TESTCONTAINERS_RYUK_DISABLED would be the way to preserve it across runs otherwise though

[vinaysshenoy]

I use https://github.com/ScribeMD/docker-cache which works well for me.