Building a new package does not use intermediate images

[CristianCantoro]

Hi,

I am trying to build a new package, with the idea of submitting a pull request to subuser-security/subuser-default-repository.

However, I am finding this being more difficult than it should be, especially in terms of build time and bandwidth consumption.

I have been following the packaging tutorial and I have already contributed a package in the past.

Here's my package structure:

electrum-base
├── image
│   └── SubuserImagefile
└── permissions.json

Now, I would change SubuserImagefile and then I would try to build it with:

$ subuser subuser add electrum-test-01 electrum-base@./

Let's say that this build fails at some point in the middle because of some error in the SubuserImagefile, in this case I would need to change the SubuserImagefile and try to build again:

$ subuser subuser add electrum-test-02 electrum-base@./

This takes quite a long time because all the command in the SubuserImagefile are executed from scratch, subuser is not reusing intermediate images even if a part of the previous built was successful.

I have tested it with an image with no apt-get update or apt-get upgrade commands because I know that this command produce different inputs and so the layers created would be different.

I tested this by adding the line at the end of the SubuserImagefile:

RUN echo 'All done!'

and the image gets rebuilt from scratch.

In principle subuser should be able to reuse the intermediate images when they are the same, which is something that Docker can do when building, if I recall correctly.

So, am I doing something wrong?

p.s.: furthermore, I would also say that it would be nice to have a way to have sudo in containers (see also #324) so that I could launch a shell in a subuser and try out commands in a partial build. Should I file a separate bug with this request?

[CristianCantoro]

It probably depends on the fact that I am using the following base image

FROM-SUBUSER-IMAGE libx11@default

which uses apt-get update and apt-get upgrade.

[timthelion]

Subuser is configured to rebuild everything when it builds an image. Otherwise, the images that were built would be out of date. For testing when creating a package, it is therefore better to use a dev subuser.

Run this command in the repository:

subuser dev --update electrum-base

The --update flag will tell subuser to use the docker image cache while building the image, and therefore, only steps which have changed or which previously failed will be rebuilt.

It is also possible to pass the --use-cache option to subuser update in order to get the same effect with non-dev subusers.

As for being able to play with images and test build steps, the best way to do this in my opinion would be to run subuser describe subuser <subuser-name> and looking at the Docker image Id: and using docker directly. Another option, if the image doesn't build correctly at all, would be to run a command like this cat ~/.subuser/registry/commit_log | grep -n2 "\-\-\->" | tail -n 15 and look for a line that starts with --->.

Here is some example output:

662502-Step 2/2 : RUN mkdir -p /subuser ; echo bb11cd5f-b55e-4701-b30f-612b9ecaac70 > /subuser/uuid
662503: ---> Running in b64650ea1b10
662504: ---> 614dd19b49b9
662505-Removing intermediate container b64650ea1b10

I can now issue docker run -it 614dd19b49b9 bash in order to enter that layer as root.

Hope this helps,
Tim

[CristianCantoro]

Hi @timthelion, thanks for the info, I will look into them.

I also think that this could be useful information to put in the tutorial on packaging.