diff --git a/.versionbot/CHANGELOG.yml b/.versionbot/CHANGELOG.yml index 01866248..311d82c3 100644 --- a/.versionbot/CHANGELOG.yml +++ b/.versionbot/CHANGELOG.yml @@ -1,3 +1,49 @@ +- commits: + - subject: Update layers/meta-balena to 6898f00e2d312aa468a44a98880882c31e81b26e + hash: 48ad801977c75fc8fa0dddfa559d06e11d4644bd + body: Update layers/meta-balena + footer: + Changelog-entry: Update layers/meta-balena to 6898f00e2d312aa468a44a98880882c31e81b26e + changelog-entry: Update layers/meta-balena to 6898f00e2d312aa468a44a98880882c31e81b26e + author: Self-hosted Renovate Bot + nested: + - commits: + - subject: "hostapp-update-hooks: Soft include balena-config-defaults" + hash: a6cd568a3b3aec127e2ef5b9b480b657a3ea246e + body: > + a203bcdfd567c0cc4b4ed9de493513142cd7463f introduced a dependency + + on /usr/sbin/balena-config-defaults to hostapp-update-hooks, + however + + during HUP the script is not only executed in the "new" OS + container + + but directly in the context of the "old" OS as well, so + + /usr/sbin/balena-config-defaults needs to exist there. + + The file was introduced in balenaOS v2.99.28, so trying to HUP + + from anything before that will fail. + + + This patch changes this to a soft dependency so even if the file + + is missing HUP will continue. + footer: + Change-type: patch + change-type: patch + Signed-off-by: Michal Toman + signed-off-by: Michal Toman + author: Michal Toman + nested: [] + version: meta-balena-5.1.48 + title: "" + date: 2024-03-06T08:25:08.800Z + version: 5.1.48 + title: "" + date: 2024-03-06T12:45:17.263Z - commits: - subject: pass input parameters to common esr workflow hash: 0cbb0b7c03c87ffb8d774e93f7119827a126e30f @@ -181,7 +227,8 @@ by UUID and assumes that only a single device is returned. This - assumption breaks when the root is on a MD RAID1 device as not only + assumption breaks when the root is on a MD RAID1 device as not + only the virtual MD device holds a filesystem with the given UUID, @@ -261,7 +308,10 @@ https://balena.zulipchat.com/#narrow/stream/360838-balena-io.2Fos.2Fdevices/topic/balena-raspberrypi.20jenkins.20build.20failures/near/423970246 - Currently devices with on-board storage fail to build in jenkins, if they don't provide a flasher image. One example is the CM4. Since there are multiple devices using this configuration, let's re-enable builds for all of them. + Currently devices with on-board storage fail to build in + jenkins, if they don't provide a flasher image. One example is + the CM4. Since there are multiple devices using this + configuration, let's re-enable builds for all of them. footer: Change-type: patch change-type: patch @@ -339,9 +389,11 @@ RPI firmware configuration allows repeating overlays to define - configurations on multiple devices. For instance, for configuring + configurations on multiple devices. For instance, for + configuring - multiple `ads` devices, `config.txt` needs to be setup this way + multiple `ads` devices, `config.txt` needs to be setup + this way ``` @@ -353,11 +405,14 @@ ``` - Before this change, the supervisor would interpret both lines as + Before this change, the supervisor would interpret both + lines as - belonging to the same overlay, preventing users from configuring multiple + belonging to the same overlay, preventing users from + configuring multiple - devices, and leading to a loop when trying to apply configurations with + devices, and leading to a loop when trying to apply + configurations with repeated overlays coming from the cloud side. footer: @@ -376,9 +431,11 @@ v14](https://github.com/balena-os/balena-supervisor/commit/460c3ba0aab31d18a02e3f5dda1838691768c494). While considered legacy, - they are still used by a few customers with devices running OS < 2.47.1. + they are still used by a few customers with devices + running OS < 2.47.1. - This should fix v2 delta support for those devices until we can + This should fix v2 delta support for those devices until + we can completely remove rsync deltas from the supervisor footer: @@ -414,13 +471,17 @@ Previously, getBootConfig() of the config.txt backend was omitting - array configurations such as gpio settings, thus resulting in the SV + array configurations such as gpio settings, thus + resulting in the SV - mistakenly assuming that boot config had not been applied, since gpio + mistakenly assuming that boot config had not been + applied, since gpio - would not be in current config.txt config but would be in target config. + would not be in current config.txt config but would be + in target config. - This resulted in SV entering an infinite loop of attempting to apply the + This resulted in SV entering an infinite loop of + attempting to apply the gpio config when it wasn't necessary. footer: @@ -454,26 +515,34 @@ While ordering is important in the RPI firmware configuration file (config.txt), - some dt params are by default considered part of the base dt overlay + some dt params are by default considered part of the + base dt overlay if they are not used by other overlays. - Unfortunately the [list of dtparams](https://github.com/raspberrypi/firmware/blob/master/boot/overlays/README#L133) + Unfortunately the [list of + dtparams](https://github.com/raspberrypi/firmware/blob/master/boot/overlays/README#L133) - is too long to add all of them as exceptions, but we can add the params + is too long to add all of them as exceptions, but we can + add the params - used in the default config.txt provided in OS images, to avoid reboots + used in the default config.txt provided in OS images, to + avoid reboots - when updating to this new supervisor and correctly parsing the + when updating to this new supervisor and correctly + parsing the provisioning config.txt as variables. - While this addition handles most common scenarios, there is still a + While this addition handles most common scenarios, there + is still a - chance a user may have use other base overlay dt params in the initial + chance a user may have use other base overlay dt params + in the initial - config, in which case those will be interpreted according to the + config, in which case those will be interpreted + according to the relative ordering footer: @@ -487,23 +556,29 @@ DT overlays and DT params need to be consumed in the order that they - appear on the file. DT params apply to the last dtoverlay defined on the + appear on the file. DT params apply to the last + dtoverlay defined on the file, or to the base overlay. - This commit updates config.txt parsing to consider this ordering, and it + This commit updates config.txt parsing to consider this + ordering, and it - also ensures global dtparams are written first so they cannot be + also ensures global dtparams are written first so they + cannot be overriden by later overlays. - Because of the more strict parsing method, it is possible that existing + Because of the more strict parsing method, it is + possible that existing - HOST_CONFIG vars do not match the interpretation of the parser. If + HOST_CONFIG vars do not match the interpretation of the + parser. If - that's the case, the supervisor will re-apply the target state which + that's the case, the supervisor will re-apply the target + state which will cause the device to reboot. footer: @@ -773,7 +848,8 @@ Calling `cryptsetup resize` on LUKS2 actually prompts for a password - and it is not needed as the partition will auto-expand on unlock. + and it is not needed as the partition will auto-expand on + unlock. footer: Change-type: patch change-type: patch @@ -870,7 +946,8 @@ This helper file is to be overwritten by device integration layers - to provide hostOS update customizations for secure boot devices that + to provide hostOS update customizations for secure boot devices + that split the boot partition into encrypted and non-encrypted. footer: @@ -887,7 +964,8 @@ The flasher image is now able to self-install when launched from an - external storage. This is useful for use cases where an installation + external storage. This is useful for use cases where an + installation steps that re-partitions/encrypts disk is required for example. footer: @@ -1026,18 +1104,23 @@ During HUP, rollback-health-breadcrumb and rollback-altboot-breadcrumb - are created in the state partition to trigger rollback-health and + are created in the state partition to trigger rollback-health + and - rollback-altboot respectively on the next boot. After these services + rollback-altboot respectively on the next boot. After these + services complete, they will remove these breadcrumbs. - Make the broken init fallback tests wait for these services to become + Make the broken init fallback tests wait for these services to + become - inactive before testing the state of the breadcrumbs. Otherwise, a race + inactive before testing the state of the breadcrumbs. Otherwise, + a race - condition can make these tests fail. Most notably on slower systems, + condition can make these tests fail. Most notably on slower + systems, such as emulated generic-aarch64. footer: @@ -1303,36 +1386,47 @@ The balena bootloader initramfs contains the rootfs module and that - will get the rootfs mounted but not checked first for errors. This is + will get the rootfs mounted but not checked first for errors. + This is - problematic because at first boot with network connectivity available, + problematic because at first boot with network connectivity + available, - time will sync but the rootfs will still have the last mount time in + time will sync but the rootfs will still have the last mount + time in - 1970. If at that point the rootfs gets corrupted then at next boot + 1970. If at that point the rootfs gets corrupted then at next + boot - the rootfs' initramfs module from balena-bootloader will try to mount + the rootfs' initramfs module from balena-bootloader will try to + mount - the rootfs without checking it first and then after that the filesystem + the rootfs without checking it first and then after that the + filesystem - check triggered by the fsck module from the actual kernel initramfs will + check triggered by the fsck module from the actual kernel + initramfs will fail like this: - [init][INFO] Running filesystem checks on partition resin-rootA (/dev/disk/by-state/resin-rootA) + [init][INFO] Running filesystem checks on partition resin-rootA + (/dev/disk/by-state/resin-rootA) resin-rootA contains a file system with errors, check forced. - resin-rootA: Inodes that were part of a corrupted orphan linked list found. + resin-rootA: Inodes that were part of a corrupted orphan linked + list found. resin-rootA: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY. (i.e., without -a or -p options) - This commit will add the fsck module to balena bootloader's initramfs + This commit will add the fsck module to balena bootloader's + initramfs - which will trigger filesystem checks before the rootfs module runs. + which will trigger filesystem checks before the rootfs module + runs. footer: Change-type: patch change-type: patch @@ -1486,7 +1580,8 @@ Also update containerd component from balena-containerd to also use - runc v1.1.12. Also update dependencies as indicated from balena-runc go.mod. + runc v1.1.12. Also update dependencies as indicated from + balena-runc go.mod. footer: Change-type: patch change-type: patch @@ -1857,14 +1952,16 @@ This fixes rollback-health failures for boards like the Jetson Seeeed - J4012, which updates efivars for capsule updates from container hooks + J4012, which updates efivars for capsule updates from container + hooks and not from the current OS hooks. While the actual HUP works because of /sys being bindmounted by - hostapp-update, rollbacks will fail to run the old OS hooks unless + hostapp-update, rollbacks will fail to run the old OS hooks + unless efivarfs is mounted. footer: @@ -2019,14 +2116,18 @@ Moby v25 adds the dynamically generated MAC address to the Config field, - which breaks the state engine, preventing the state from settling as the + which breaks the state engine, preventing the state from + settling as the - current state never matches the target state (empty mac address). This + current state never matches the target state (empty mac + address). This - seems to be a bug in Moby that we reported in moby/moby#47228 + seems to be a bug in Moby that we reported in + moby/moby#47228 - The issue won't affect Balena devices until balenaEngine is updated, and this + The issue won't affect Balena devices until balenaEngine + is updated, and this change fixes testing for now. footer: @@ -2040,7 +2141,8 @@ From docker 25, the engine will validate IPAM config. This would cause - the docker utils test to fail since the network/subnet configuration was + the docker utils test to fail since the network/subnet + configuration was incorrect. footer: @@ -2078,7 +2180,8 @@ The tsserver lsp doesn't seem to work well with .editorconfig. Using - prettierrc will allow formatting to work correctly with editor plugins + prettierrc will allow formatting to work correctly with + editor plugins This also updates editorconfig to work well with json @@ -2113,7 +2216,8 @@ to be used as the balena bootloader. Device types are expected - to inherit it and perform additional device-specific configuration. + to inherit it and perform additional device-specific + configuration. footer: Change-type: patch change-type: patch @@ -2127,9 +2231,11 @@ The kexec initrd script currently removes the maxcpus=0 kernel arg - which we use to put the system into non-SMP mode. This however does + which we use to put the system into non-SMP mode. This however + does - not work on all platforms and nr_cpus=1 seems to be a more robust + not work on all platforms and nr_cpus=1 seems to be a more + robust solution, so with this patch nr_cpus will be removed as well. footer: @@ -2288,20 +2394,26 @@ body: > At this moment there is a race condition between NetworkManager - and the engine when a shared interface is configured. If the interface + and the engine when a shared interface is configured. If the + interface - is configured first and the engine second, the containers are allowed + is configured first and the engine second, the containers are + allowed - to access DHCP hosts behind the shared interface. If the engine comes + to access DHCP hosts behind the shared interface. If the engine + comes up first and the shared interface second, access will be denied. - This patch adds a dispatcher script that always configures the firewall + This patch adds a dispatcher script that always configures the + firewall - rules as if the engine came up last. This does not really address + rules as if the engine came up last. This does not really + address - the underlying issue but it overcomes the race condition and makes + the underlying issue but it overcomes the race condition and + makes the behavior deterministic, which is good enough at this point. footer: @@ -2497,7 +2609,8 @@ UEFI firmware in secure boot needs to authenticate the kernel plus - initramfs in the chain of trust. Other firmware implements secure boot + initramfs in the chain of trust. Other firmware implements + secure boot differently and does not need this. footer: @@ -2778,12 +2891,14 @@ This is meant to allow users to configure their device to - resolve `.local` queries via dnsmasq by modifying config.json, e.g. `dnsServers": + resolve `.local` queries via dnsmasq by modifying + config.json, e.g. `dnsServers": "/bob.local/172.17.0.33`. - This would fail before as MDNS lookups would always come first + This would fail before as MDNS lookups would always come + first footer: Change-type: minor change-type: minor @@ -2825,15 +2940,20 @@ The `updateMetadata` step renames the container to match the target - release when the service doesn't change between releases. We have seen + release when the service doesn't change between + releases. We have seen - this step fail because of an engine bug that seems to relate to the + this step fail because of an engine bug that seems to + relate to the - engine keeping stale references after container restarts. The only way + engine keeping stale references after container + restarts. The only way - around this issue is to remove the old container and create it again. + around this issue is to remove the old container and + create it again. - This implements that workaround during the updateMetadata step to deal + This implements that workaround during the + updateMetadata step to deal with that issue. footer: @@ -3288,7 +3408,8 @@ There have been reports of an empty config vars cache file - probably - because of a race condition when the reading of config.json happens just + because of a race condition when the reading of config.json + happens just as the file is being replaced. @@ -3435,7 +3556,8 @@ If update-balena-supervisor runs and finds the image is already downloaded - it will run the specified supervisor but will not check that supervisor.conf + it will run the specified supervisor but will not check that + supervisor.conf is updated so the version will revert on the next update. footer: @@ -3479,12 +3601,14 @@ This provides an easy switch to enable tracing on HUP hooks that works - both on old and new OS hooks as enabling it depends on a config.json + both on old and new OS hooks as enabling it depends on a + config.json setting. - It is meant to debug field issues with HUP failure where all we see is: + It is meant to debug field issues with HUP failure where all we + see is: ``` @@ -3691,27 +3815,35 @@ (see https://github.com/systemd/systemd/pull/17917). - Properly detecting this is too cumbersome for a bash logging script, + Properly detecting this is too cumbersome for a bash logging + script, - see https://github.com/systemd/systemd/pull/17902, however, falling + see https://github.com/systemd/systemd/pull/17902, however, + falling - back to the last check, that is, seeing if `/.dockerenv` exists is easy + back to the last check, that is, seeing if `/.dockerenv` exists + is easy enough and works for our use case. - This script will only be called from the hostOS, and the only case it is + This script will only be called from the hostOS, and the only + case it is - called from a container is during HUP and the container is always a hostOS + called from a container is during HUP and the container is + always a hostOS - image. So even though the interface chosen by moby, a file under /, + image. So even though the interface chosen by moby, a file under + /, - is a bad interface in general, it works fine for the specific limitations + is a bad interface in general, it works fine for the specific + limitations of balenaOS. - Also, check for `/run/.containerenv` which is the equivalent interface + Also, check for `/run/.containerenv` which is the equivalent + interface for podman for future proofing. footer: @@ -3864,13 +3996,17 @@ Whenever the Supervisor reports current state, it diffs the current state - with its last reported current state. However, when the Supervisor starts + with its last reported current state. However, when the + Supervisor starts - up, there is no last reported state, since that last report is stored in + up, there is no last reported state, since that last + report is stored in - process memory. Caching the last report in a location that survives + process memory. Caching the last report in a location + that survives - Supervisor restarts will reduce the current report bandwidth used on startup. + Supervisor restarts will reduce the current report + bandwidth used on startup. footer: Change-type: patch change-type: patch @@ -3933,11 +4069,14 @@ We previously tried to use a single time limit for the execution of the - healthcheck test on all device types. This was causing occasional false + healthcheck test on all device types. This was causing + occasional false - positives in our Continuous Integration pipeline, though -- especially + positives in our Continuous Integration pipeline, though -- + especially - on slow devices like Pi Zeros and the generic-aarch64, which runs on + on slow devices like Pi Zeros and the generic-aarch64, which + runs on emulated hardware. @@ -3945,7 +4084,8 @@ This commit addresses this issue, this commit: - 1. Limits execution for device types for which we have collected enough + 1. Limits execution for device types for which we have collected + enough data to have a good idea of how long the test should take. 2. Uses time limits specific for each device type. footer: @@ -3987,11 +4127,14 @@ PR #2217 removed the expose configuration but also caused a regresion - where ports set via the `ports` configuration would no longer get + where ports set via the `ports` configuration would no + longer get - exposed to the host, despite portmappings being set. This fixes that + exposed to the host, despite portmappings being set. + This fixes that - issue by exposing only those ports comming from port mappings. + issue by exposing only those ports comming from port + mappings. footer: Change-type: patch change-type: patch @@ -4007,33 +4150,44 @@ The docker EXPOSE directive and corresponding docker-compose `expose` - service configuration serves as documentation/metadata that a container + service configuration serves as documentation/metadata + that a container - listens on a certain port that may be used for service discovery but it doesn't + listens on a certain port that may be used for service + discovery but it doesn't have any real impact on the ability for - other containers on the same network to access the exposed service via + other containers on the same network to access the + exposed service via - the port. In newer engine implementations, this property may conflict + the port. In newer engine implementations, this property + may conflict - with other network configurations, and prevent the container from being + with other network configurations, and prevent the + container from being started by the docker engine (see #2211). - This PR removes code that would manage the expose property and takes the + This PR removes code that would manage the expose + property and takes the - property out of the whitelist. A composition with the `expose` property + property out of the whitelist. A composition with the + `expose` property - will result in the log message `Ignoring unsupported or unknown compose fields: expose`. + will result in the log message `Ignoring unsupported or + unknown compose fields: expose`. - While this change should not have operational impact, it still removes + While this change should not have operational impact, it + still removes - a previously supported configuration and as such there is a chance of it + a previously supported configuration and as such there + is a chance of it - being a breaking change for some applications. For this reason it is + being a breaking change for some applications. For this + reason it is being published as a new major version. footer: @@ -4296,17 +4450,21 @@ The code moved from meta-balena-kirkstone was not really specific to - kirkstone so let's move it here so that future branches for newer yocto + kirkstone so let's move it here so that future branches for + newer yocto - releases which we'll base off kirkstone don't continue to add this + releases which we'll base off kirkstone don't continue to add + this unneeded duplication. There are other meta-balena-* directories that still contain the - duplication we moved from meta-balena-kirkstone but we're not really + duplication we moved from meta-balena-kirkstone but we're not + really - concerned with that because going forward those old directories will + concerned with that because going forward those old directories + will naturally get deprecated. footer: @@ -4489,12 +4647,14 @@ body: > If the target supervisor image is already cached but there is no - container running with it, the update script would just exit without + container running with it, the update script would just exit + without actually running the target supervisor. - This commit checks whether there is a running container using the + This commit checks whether there is a running container using + the target image and restarts the supervisor if there is none. footer: @@ -4680,7 +4840,8 @@ This check is now done in the cryptsetup initramfs hook rather than - during installation, which obviates the need to perform it during setup. + during installation, which obviates the need to perform it + during setup. Remove it. footer: @@ -4696,14 +4857,17 @@ During installation, some firmwares may allow keys to be enrolled but - fail to tip the system into user mode until the system is rebooted. We + fail to tip the system into user mode until the system is + rebooted. We - don't want to mislead users with only full-disk encryption into thinking + don't want to mislead users with only full-disk encryption into + thinking their system also has secure boot enabled when it doesn't. - Disable the hook to unlock encrypted partitions if the firmware fails to + Disable the hook to unlock encrypted partitions if the firmware + fails to boot into user mode. footer: @@ -4719,7 +4883,8 @@ We now have several places where secure boot specific configuration is - checked. Create an os-helpers-secureboot package to consolidate and + checked. Create an os-helpers-secureboot package to consolidate + and reuse this code. footer: @@ -4999,7 +5164,8 @@ This script is used by balenaHup to report provisioning failures to - the cloud. Adding retries, return status code check and error output + the cloud. Adding retries, return status code check and error + output should make it more resilient and easier to debug. footer: @@ -5296,13 +5462,17 @@ This reverts commit 0c7bad779291e15e419166a2c66c2a21dd06aa83, as that - change causes a service restart loop. The supervisor cannot distinguish + change causes a service restart loop. The supervisor + cannot distinguish - between ports exposed via the `EXPOSE` directive and the docker-compose + between ports exposed via the `EXPOSE` directive and the + docker-compose - `expose` property. Because of this, in the case of `network-mode: + `expose` property. Because of this, in the case of + `network-mode: - service:<...>` the current state and target state never match, leading + service:<...>` the current state and target state never + match, leading to a service restart loop. footer: @@ -5344,13 +5514,17 @@ The supervisor exposes ports configured using the `EXPOSE` directive in - the dockerfile when configuring the container for runtime. This can + the dockerfile when configuring the container for + runtime. This can - cause issues if using `network_mode: service:` as the + cause issues if using `network_mode: service:` as the - expose configuration is not compatible with that network mode. This + expose configuration is not compatible with that network + mode. This - fix now skips image exposed ports for that particular network mode. + fix now skips image exposed ports for that particular + network mode. footer: Change-type: patch change-type: patch @@ -5401,7 +5575,8 @@ devDependencies are tree-shaked, while dependencies are stored in the - image. We reserve dependencies just for those that contain binary + image. We reserve dependencies just for those that + contain binary bindings footer: @@ -5679,12 +5854,15 @@ body: > When searching for devices matching the glob list in - get_internal_device(), a glob match breaks from a nested loop rather + get_internal_device(), a glob match breaks from a nested loop + rather - than the parent loop, allowing the function to output multiple matches. + than the parent loop, allowing the function to output multiple + matches. - When running the flasher, this results in the script failing with an + When running the flasher, this results in the script failing + with an incorrect path to the internal disk. @@ -5945,7 +6123,8 @@ Alpine allows the `~=` syntax to match a part of the package version - when installing. In this case we want to use it to specify node and + when installing. In this case we want to use it to + specify node and npm major versions footer: @@ -6027,7 +6206,8 @@ security reasons. - This new balenaOS ESR bot has contents:write and workflows:write permissions + This new balenaOS ESR bot has contents:write and workflows:write + permissions but is only available on balenaOS repositories. footer: @@ -6344,16 +6524,19 @@ This is done by the bootloader (uboot/grub) at this moment but as we - are moving towards the balena 2nd stage bootloader, it needs to be + are moving towards the balena 2nd stage bootloader, it needs to + be moved into the initramfs. - This adds a standalone recipe - by default yocto tries to build all + This adds a standalone recipe - by default yocto tries to build + all modules defined in the initramfs-framework recipe, which breaks - on armv7 when abroot is defined there. This is because it depends + on armv7 when abroot is defined there. This is because it + depends on grub-editenv which is not supported on armv7. footer: @@ -6382,9 +6565,11 @@ body: > The rootfs script uses both os-helpers-fs and os-helpers-logging - though the package depends on neither. This seems to work now because + though the package depends on neither. This seems to work now + because - in most cases something else pulls in the dependencies or the code + in most cases something else pulls in the dependencies or the + code on a particular device does not fall under the branches that use @@ -6882,10 +7067,12 @@ This commit updates balena-containerd to a new version in which we - cherry-picked the change from here: https://github.com/containerd/containerd/pull/8086 + cherry-picked the change from here: + https://github.com/containerd/containerd/pull/8086 - This change avoids enabling AppArmor if the `/sbin/apparmor_parser` + This change avoids enabling AppArmor if the + `/sbin/apparmor_parser` binary is not found in the system. footer: @@ -7065,7 +7252,8 @@ The meta-balena version of modemmanager is no longer compatible with - Yocto Pyro, so stop trying to apply bbappend to it from meta-balena. + Yocto Pyro, so stop trying to apply bbappend to it from + meta-balena. footer: Change-type: patch change-type: patch @@ -7694,7 +7882,8 @@ The node-dbus module is unmaintained and a blocker for the update to - Node 18. Switching to our own node bindings for systemd solves this + Node 18. Switching to our own node bindings for systemd + solves this issue footer: @@ -7747,7 +7936,8 @@ mobile-broadband-provider-info 'master' branch was renamed to 'main', - causing do_fetch() to fail before it was changes in Yocto Kirkstone + causing do_fetch() to fail before it was changes in Yocto + Kirkstone commit e4795393c4882cf38273521539cc255a4ffcb34a. footer: @@ -7913,7 +8103,8 @@ Verify kernel lockdown prohibits loading of unsigned modules, and still - loads modules with a signature that validates against a trusted key. + loads modules with a signature that validates against a trusted + key. footer: Change-type: patch change-type: patch @@ -8087,24 +8278,31 @@ 314047e and b5c5214 made flasher block until the resin-device-register - service exits and made resin-device-register give up after 6 seconds + service exits and made resin-device-register give up after 6 + seconds - not to block infinitely when no network is available. This effectively + not to block infinitely when no network is available. This + effectively - means that if the device fails to register within first 6 seconds, + means that if the device fails to register within first 6 + seconds, - it will never retry, flasher will not report status to the dashboard + it will never retry, flasher will not report status to the + dashboard and the device will only register on first boot. - This patch changes the logic back to resin-device-register trying + This patch changes the logic back to resin-device-register + trying - in the background in an infinite loop and moves the "give the device + in the background in an infinite loop and moves the "give the + device a chance to register" delay to flasher itself. It also extends - the wait to openvpn as flasher already does that and wants VPN to run + the wait to openvpn as flasher already does that and wants VPN + to run to be debuggable - in case flashing fails, it would be possible @@ -8182,32 +8380,40 @@ CONIFG_SECURITY=n - which is mispelled and not being applied. The commit where this was + which is mispelled and not being applied. The commit where this + was - introduced claims it's needed to completely disable the audit logs, and + introduced claims it's needed to completely disable the audit + logs, and also that the security framework is unused. - I disagree in that it's unused - the hostOS is not using any security + I disagree in that it's unused - the hostOS is not using any + security - framework, but applications may, so luckily the security framework was + framework, but applications may, so luckily the security + framework was never disabled. - Removing this mispelled entry should have no functional effect. Whether + Removing this mispelled entry should have no functional effect. + Whether the audit subsystem is disabled will depend on the final kernel - configuration. Definitely we have not seen a need to disable it recently, + configuration. Definitely we have not seen a need to disable it + recently, and we have not seen the kernel log flooded with messages. - I'd argue the disabling of the audit subsystem in meta-balena serves no + I'd argue the disabling of the audit subsystem in meta-balena + serves no - need but I also have no specific reason to remove it at the moment. + need but I also have no specific reason to remove it at the + moment. Fixes #2947 @@ -8241,7 +8447,8 @@ The flasher/installer image can be configured by the user and that - configuration finishes up in the installed image. Add the dispatcher + configuration finishes up in the installed image. Add the + dispatcher scripts to this existing mechanim. footer: @@ -8257,7 +8464,8 @@ On boot, the dispatcher script are copied from the boot partition where - the user has configured them, to the bind mount used by the running + the user has configured them, to the bind mount used by the + running applications. footer: @@ -8303,18 +8511,23 @@ At this moment grub.cfg sources /grub/grub_extraenv which works fine - on MBR systems, however on EFI systems this does not work because GRUB + on MBR systems, however on EFI systems this does not work + because GRUB is installed in /EFI/BOOT/ rather than /grub/. - This patch replaces the hardcoded /grub with ${prefix} which should + This patch replaces the hardcoded /grub with ${prefix} which + should - expand to the appropriate directory regardless of the platform. It also + expand to the appropriate directory regardless of the platform. + It also - removes the loading of grub_extraenv from the secure boot variant + removes the loading of grub_extraenv from the secure boot + variant - of the GRUB config since this would not load without a signature anyway. + of the GRUB config since this would not load without a signature + anyway. footer: Change-type: patch change-type: patch @@ -8343,7 +8556,8 @@ body: > This fix has been ported from the following upstream - change: https://patchwork.yoctoproject.org/project/oe-core/patch/002c31d6add77e1002fb1ccd4050ce826a654170.1659653543.git.bruce.ashfield@gmail.com/ + change: + https://patchwork.yoctoproject.org/project/oe-core/patch/002c31d6add77e1002fb1ccd4050ce826a654170.1659653543.git.bruce.ashfield@gmail.com/ and fixes the following compilation error on generic-aarch64: @@ -8398,7 +8612,8 @@ body: > Repackage iwlwifi-cc-a0 to include all firmware versions shipped - upstream, rather than only an older version (48) that's no longer + upstream, rather than only an older version (48) that's no + longer shipped as of 20230404. footer: @@ -8427,7 +8642,8 @@ body: > Replace older versioned iwlwifi packages with - linux-firmware-iwlwifi-3160 package that includes all versions shipped + linux-firmware-iwlwifi-3160 package that includes all versions + shipped in linux-firmware. footer: @@ -8458,7 +8674,8 @@ body: > Some board BSPs may define UBOOT_MACHINE, others UBOOT_CONFIG, - let's make sure we include the extra_uEnv.txt file in the non-flasher + let's make sure we include the extra_uEnv.txt file in the + non-flasher image for both cases. footer: @@ -8758,7 +8975,8 @@ `libgcc_s.so.1 must be installed for pthread_exit to work` - which panics the kernel and triggers a reboot loop indistinguishable + which panics the kernel and triggers a reboot loop + indistinguishable from a "device has been tampered with" state on regular builds @@ -8910,7 +9128,8 @@ deprecation. This allows to just remove the coffee file from the - device repository when a device is deprecated so there will be no + device repository when a device is deprecated so there will be + no more releases and no need for checks on a discontinued state. footer: @@ -8927,7 +9146,8 @@ deprecation. This allows to just remove the coffee file from the - device repository when a device is deprecated so there will be no + device repository when a device is deprecated so there will be + no more releases and no need for checks on a discontinued state. footer: @@ -8944,7 +9164,8 @@ deprecation. This allows to just remove the coffee file from the - device repository when a device is deprecated so there will be no + device repository when a device is deprecated so there will be + no more releases and no need for checks on a discontinued state. footer: @@ -9004,7 +9225,8 @@ systems with secure boot and full-disk encryption. - If kexec fails, we don't want to continue with the rest of the boot + If kexec fails, we don't want to continue with the rest of the + boot process in the first stage kernel, so bail out on failure. footer: @@ -9173,11 +9395,14 @@ It's not an official status from container inspects, and the Supervisor - doesn't set it internally anywhere. It's better to remove it entirely as the + doesn't set it internally anywhere. It's better to + remove it entirely as the - method by which Supervisor sets internal service statuses is by using a global + method by which Supervisor sets internal service + statuses is by using a global - event emitter (reportNewStatus) which makes things difficult to test. + event emitter (reportNewStatus) which makes things + difficult to test. footer: Change-type: patch change-type: patch @@ -9284,7 +9509,8 @@ Explain that balenaOS does not take control of the TPM and that it - is possible to fill all the key slots with enough provisioning cycles. + is possible to fill all the key slots with enough provisioning + cycles. footer: Change-type: patch change-type: patch @@ -9298,13 +9524,16 @@ We have seen devices that won't change PCR1 hash when a temporary boot - order override was applied or secure boot was disabled via BIOS setup. + order override was applied or secure boot was disabled via BIOS + setup. The implementation of what PCR1 actually measures is very - device-specific, but many of the risks can be mitigated by setting up + device-specific, but many of the risks can be mitigated by + setting up - a BIOS password and disabling F-key shortcuts for interacting with + a BIOS password and disabling F-key shortcuts for interacting + with the firmware. @@ -9396,32 +9625,41 @@ Both `kernel-modules-headers` and `kernel-devsrc` provide kernel headers - since Yocto Thud switched `kernel-devsrc` from full source to just + since Yocto Thud switched `kernel-devsrc` from full source to + just kernel headers. - The only difference between them is that `kernel-modules-headers` builds + The only difference between them is that + `kernel-modules-headers` builds - some target binaries which need to be built with `make modules_prepare` + some target binaries which need to be built with `make + modules_prepare` - when using `kernel-devsrc` headers. These binaries depend on libc version + when using `kernel-devsrc` headers. These binaries depend on + libc version matching though so they have shown to be problematic. - This commit removes the `kernel-modules-headers` recipe and modifies + This commit removes the `kernel-modules-headers` recipe and + modifies - `kernel-devsrc` to replace it. The deployed artifact remains named as + `kernel-devsrc` to replace it. The deployed artifact remains + named as `kernel-modules-headers` as it's a more descriptive name. - This introduces a breaking change in the balenaOS API as customers that + This introduces a breaking change in the balenaOS API as + customers that - are using `kernel-modules-headers` to build external kernel modules will + are using `kernel-modules-headers` to build external kernel + modules will - now need to issue a `make modules_prepare` as part of their build scripts. + now need to issue a `make modules_prepare` as part of their + build scripts. Fixes #1822 @@ -9515,9 +9753,11 @@ This variable accepts the base64 encoded public key of a kernel module - signing keypair and appends it to the list of trusted keys the kernel + signing keypair and appends it to the list of trusted keys the + kernel - will use to validate signed modules. Multiple keys may be appended, + will use to validate signed modules. Multiple keys may be + appended, delimited with a semicolon. @@ -9525,7 +9765,8 @@ A PEM file can be used like so: - SIGN_KMOD_KEY_APPEND="$( sed -e '/-----BEGIN CERTIFICATE-----/d' \ + SIGN_KMOD_KEY_APPEND="$( sed -e '/-----BEGIN CERTIFICATE-----/d' + \ -e 's/-----END CERTIFICATE-----/;/g' \ -e '$d' signing_key.pem \ | tr -d '\n' )" @@ -9572,22 +9813,30 @@ The previous implementation in #2170 of parsing the container status was too general, - because it relied on the mistaken assumption that a container would have a status of + because it relied on the mistaken assumption that a + container would have a status of - `Stopped` if it was manually stopped. This turned out to be untrue, as manually stopped + `Stopped` if it was manually stopped. This turned out to + be untrue, as manually stopped - containers were also getting restarted by the Supervisor due to their inspect status of + containers were also getting restarted by the Supervisor + due to their inspect status of - `exited`. With this, parsing the exit message became unavoidable as there are no other + `exited`. With this, parsing the exit message became + unavoidable as there are no other - clear ways to discern a container that has been manually stopped and shouldn't be started + clear ways to discern a container that has been manually + stopped and shouldn't be started - from a container experiencing the Engine-host race condition issue (again, see #2170). + from a container experiencing the Engine-host race + condition issue (again, see #2170). - Since we're just parsing the exit error message, we don't need to worry about different behaviors + Since we're just parsing the exit error message, we + don't need to worry about different behaviors - amongst restart policies, as any container with the error message on exit should be started. + amongst restart policies, as any container with the + error message on exit should be started. footer: Change-type: patch change-type: patch @@ -9632,7 +9881,8 @@ Previously, `concatReadSeekCloser.Read()` would incorrectly return - an `io.ErrUnexpectedEOF` if the last read from the second concatenated + an `io.ErrUnexpectedEOF` if the last read from the + second concatenated `Reader` didn't completely fill the passed buffer. @@ -9648,60 +9898,79 @@ ``` - In this example, we have a `concatReadSeekCloser` that concatenates two + In this example, we have a `concatReadSeekCloser` that + concatenates two - `Reader`s (`aaa...` and `bbb...`). The last `Read()` used a buffer + `Reader`s (`aaa...` and `bbb...`). The last `Read()` + used a buffer - larger than the yet-to-be-read portion of the `bbb...`. So, it would + larger than the yet-to-be-read portion of the `bbb...`. + So, it would incorrectly return an `io.ErrUnexpectedEOF`. - This commit makes sure that last `Read()` returns all the remaining data + This commit makes sure that last `Read()` returns all + the remaining data without an error. It also adds various test cases for - `concatReadSeekCloser.Read()`, many of which would fail before this + `concatReadSeekCloser.Read()`, many of which would fail + before this correction. - Interestingly, this bug was silently affecting us. Not in a fatal way, + Interestingly, this bug was silently affecting us. Not + in a fatal way, - but causing deltas to be larger than necessary. Indeed, running + but causing deltas to be larger than necessary. Indeed, + running - `TestDeltaSize` after this commit shows that some test cases are + `TestDeltaSize` after this commit shows that some test + cases are - producing deltas smaller than what we expected before. For posterity, + producing deltas smaller than what we expected before. + For posterity, see all the details below. - We use `concatReadSeekCloser`s to concatenate all layers of the basis + We use `concatReadSeekCloser`s to concatenate all layers + of the basis - image when creating the "signature" of the basis image. In this process, + image when creating the "signature" of the basis image. + In this process, - the `concatReadSeekCloser`s are wrapped around by a buffered reader with + the `concatReadSeekCloser`s are wrapped around by a + buffered reader with a buffer of 65kB. - If, in any read, part of this 65kB buffer was beyond the second + If, in any read, part of this 65kB buffer was beyond the + second - concatenated reader, it would result in an `io.ErrUnexpectedEOF`. This + concatenated reader, it would result in an + `io.ErrUnexpectedEOF`. This - would not cause the whole process to fail, but would prematurely end the + would not cause the whole process to fail, but would + prematurely end the - signature generation: some of the final blocks in the basis image would + signature generation: some of the final blocks in the + basis image would - not be added to the signature. Therefore, if those blocks appeared in + not be added to the signature. Therefore, if those + blocks appeared in - the target image, they'd result in (larger) LITERAL, instead of + the target image, they'd result in (larger) LITERAL, + instead of (smaller) COPY operations. - For illustration, here's the delta generated for the `delta-006-008` + For illustration, here's the delta generated for the + `delta-006-008` test case. First before this commit: @@ -9754,7 +10023,8 @@ ``` - That 21kB LITERAL is the difference in size we saw in the test results. + That 21kB LITERAL is the difference in size we saw in + the test results. footer: Signed-off-by: Leandro Motta Barros signed-off-by: Leandro Motta Barros @@ -9768,7 +10038,8 @@ Using `defer` for the sake of being more idiomatic (and maybe slightly - more reliable); plus, using the proper doc comment standards. + more reliable); plus, using the proper doc comment + standards. footer: Signed-off-by: Leandro Motta Barros signed-off-by: Leandro Motta Barros @@ -9957,16 +10228,21 @@ This is necessary since the builder no longer passes the platform flag - to the build. This would lead to dockerfiles that are mixing multi and single + to the build. This would lead to dockerfiles that are + mixing multi and single - arch stages to pull the wrong architecture images, particularly when + arch stages to pull the wrong architecture images, + particularly when - trying to build images in emulated builds (e.g. armv7hf built on aarch64). + trying to build images in emulated builds (e.g. armv7hf + built on aarch64). - Moving the full build to multi-arch solves this as the docker engine is + Moving the full build to multi-arch solves this as the + docker engine is - capable of chosing the right architecture from the manifest. + capable of chosing the right architecture from the + manifest. footer: Relatest-to: balena-io/balena-builder#1010 relatest-to: balena-io/balena-builder#1010 @@ -10140,7 +10416,8 @@ This should be the default but with no explicit argument we still - end up with LUKS1 partitions. This patch adds the parameter to enforce + end up with LUKS1 partitions. This patch adds the parameter to + enforce LUKS2 formatting and adds conversion to LUKS2 to the cryptsetup @@ -10430,38 +10707,52 @@ There exists a race condition between Engine and a host resource that may not - be immediately created. In this race condition, if a container's compose config + be immediately created. In this race condition, if a + container's compose config - depends on the existence of that host resource, such as a network interface, and the + depends on the existence of that host resource, such as + a network interface, and the - Engine tries to create & start the container before the host resource is created, the + Engine tries to create & start the container before the + host resource is created, the - Engine will not reattempt to start the container, regardless of the restart policy. + Engine will not reattempt to start the container, + regardless of the restart policy. - This is undesireable behavior but seems to be the behavior as implemented by Docker. + This is undesireable behavior but seems to be the + behavior as implemented by Docker. - To rectify this, the Supervisor state funnel noops for a grace period of 1 minute + To rectify this, the Supervisor state funnel noops for a + grace period of 1 minute - after starting a container to see that the container's status has become 'running`. + after starting a container to see that the container's + status has become 'running`. - If the container exits because of the race condition, the status becomes 'exited' and the + If the container exits because of the race condition, + the status becomes 'exited' and the - Supervisor will attempt to generate another start step. This noop-wait-start step loop + Supervisor will attempt to generate another start step. + This noop-wait-start step loop will repeat until the container is able to start. - If the container is never able to start, there was a problem in the host in the creation of the + If the container is never able to start, there was a + problem in the host in the creation of the - host resource, and that should be fixed at the host level. + host resource, and that should be fixed at the host + level. - This commit does not handle the case of services with restart policies "no" or "on-failure" + This commit does not handle the case of services with + restart policies "no" or "on-failure" - which encounter this host race, as metadata from container inspects needs to be introduced + which encounter this host race, as metadata from + container inspects needs to be introduced - during step calculation in order to figure out whether services with those restart policies + during step calculation in order to figure out whether + services with those restart policies need to be started. This will be fixed in a future PR. footer: @@ -10487,7 +10778,8 @@ Removing the pull_request_target run for ESR branches fixes this, but - also removes the possibility of external pull requests into ESR branches, + also removes the possibility of external pull requests into ESR + branches, which we don't actually need. @@ -10519,7 +10811,8 @@ This patch extends secure boot documentation with more details about - how the feature works and tries to explain why some decisions were made. + how the feature works and tries to explain why some decisions + were made. footer: Change-type: patch change-type: patch @@ -10571,12 +10864,14 @@ Comply with AWS public AMI quota, taking into account we have two - architectures that publish AMI images and we need free slots for custom + architectures that publish AMI images and we need free slots for + custom version request. - Make the oldest public image back to private before publishing a new image. + Make the oldest public image back to private before publishing a + new image. footer: Change-type: patch change-type: patch @@ -10594,7 +10889,8 @@ When building signed images, add the secureBoot feature flag into the - OS contract. This is needed for other components to identify secureBoot + OS contract. This is needed for other components to identify + secureBoot compatible software releases. footer: @@ -10626,12 +10922,14 @@ When running in the initramfs, the resin-device-progress package is not - installed as we cannot guarantee that the initramfs would be able to + installed as we cannot guarantee that the initramfs would be + able to bring up all types of network interfaces. - This commit adds a check for the script to exists instead of getting a + This commit adds a check for the script to exists instead of + getting a `command not found` when an API endpoint is defined. footer: @@ -10651,10 +10949,12 @@ The purpose of testing the API calls is to detect breaking changes, not - to fail builds because of temporary network or API access problems. + to fail builds because of temporary network or API access + problems. - Printing a warning instead should be enough for developers to detect + Printing a warning instead should be enough for developers to + detect breaking changes. footer: @@ -10798,11 +11098,14 @@ Support for colon characters was added v14.6.0 which enabled - configurations for HDMI port 2 (e.g on the RPi 4). These configurations + configurations for HDMI port 2 (e.g on the RPi 4). These + configurations - are not documented anywhere else so this allows users to be able to + are not documented anywhere else so this allows users to + be able to - better find the relevant information for working with HDMI. + better find the relevant information for working with + HDMI. footer: Change-type: patch change-type: patch @@ -10903,9 +11206,11 @@ the setup mode flag after a new PK is installed. In this case - flasher will reboot in order to ensure the keys are actually saved + flasher will reboot in order to ensure the keys are actually + saved - and the device comes back with secure boot enabled. Since we changed + and the device comes back with secure boot enabled. Since we + changed flasher to be unsigned by default, this reboot causes a security @@ -10914,7 +11219,8 @@ With this patch flasher will add a new boot entry before issuing - the reboot so that signed flasher comes up and the installation process + the reboot so that signed flasher comes up and the installation + process can continue. footer: @@ -10931,14 +11237,17 @@ Currently the db.auth file is signed as "append" in order to make HUP work. - Most UEFI firmwares will accept such file even for "replace", which we do + Most UEFI firmwares will accept such file even for "replace", + which we do - during the initial provisioning, however we have seen devices that will + during the initial provisioning, however we have seen devices + that will only allow appending, which makes flasher fail. - With this patch flasher will use the esl file for initial programming + With this patch flasher will use the esl file for initial + programming of the db variable. @@ -11086,11 +11395,14 @@ for about 17 minutes (for details see commit - 582487f832c59c2f734a780ab0492833f29002c9). This worked fine in most + 582487f832c59c2f734a780ab0492833f29002c9). This worked + fine in most - situations, but we have seen at least one case of a particularly + situations, but we have seen at least one case of a + particularly - unreliable network connection that would not be able to finish a large + unreliable network connection that would not be able to + finish a large pull when operating under this policy. @@ -11098,17 +11410,20 @@ This commit: - * Completely removes timeouts from image pulls. We'll keep retrying + * Completely removes timeouts from image pulls. We'll + keep retrying forever, or until users cancel the pull. (From the perspective of the REST API, "canceling" mean closing the HTTP connection. This is what happens when a user Ctrl+C during a pull in the CLI, or, say, we kill a curl process that was using the REST API to pull image.) - * Still uses exponential back-off, but we now limit the interval between + * Still uses exponential back-off, but we now limit the + interval between retries to 5 minutes. The rationale is that some very unreliable networks may be up only for relatively small time windows. Therefore, using intervals that are too long would increase the risk of missing these windows. - * Tries to avoid flooding the log stream with messages about retries. + * Tries to avoid flooding the log stream with messages + about retries. We'll log every retry attempt up to the 10th. After that, we'll log retries only once about every 2h. This retry count is reset every time we successfully download any amount of data. @@ -11288,11 +11603,14 @@ the flasher image to force an installer migration. - With the current QEMU setting, `installerForceMigration` alone is not + With the current QEMU setting, `installerForceMigration` + alone is not - enough and the QEMU_INTERNAL_STORAGE also needs to be set to false in + enough and the QEMU_INTERNAL_STORAGE also needs to be + set to false in - the worker's environment so only a single external disk is attached to + the worker's environment so only a single external disk + is attached to the emulator. footer: @@ -11377,7 +11695,8 @@ Whether the internal disk is attached or not will be defined by the - environment. This allows to test the migrator that requires booting only + environment. This allows to test the migrator that + requires booting only the external disk. footer: @@ -11422,25 +11741,33 @@ After a recent change enforcing all the partitions to be on the same - block device, encrypted partitions are no longer being detected + block device, encrypted partitions are no longer being + detected - correctly. This is because the assumption that the parent block device + correctly. This is because the assumption that the + parent block device - is a substring of the actually mounted block device does not work + is a substring of the actually mounted block device does + not work - for LUKS devices - the mount will either be /dev/mapper/luks-XXX + for LUKS devices - the mount will either be + /dev/mapper/luks-XXX - or /dev/dm-X while the parent device is still e.g. /dev/sda. + or /dev/dm-X while the parent device is still e.g. + /dev/sda. - The usual balenaOS boot partition is also split in two - boot and efi. + The usual balenaOS boot partition is also split in two - + boot and efi. - The boot partition (mounted under /mnt/boot) is encrypted and the efi + The boot partition (mounted under /mnt/boot) is + encrypted and the efi partition (mounted under /mnt/efi) is not. - This patch generalizes the detection of the parent device so that + This patch generalizes the detection of the parent + device so that it works with both encrypted and unencrypted partitions. footer: @@ -11460,14 +11787,17 @@ The docker compose V2 spec no longer accepts `network_mode: bridge`, - which means we can no longer override the network configuration of + which means we can no longer override the network + configuration of the `balena-supervisor` service for tests. - For this reason we now create a separate service to run the built + For this reason we now create a separate service to run + the built - supervisor `balena-supervisor-sut` and run API tests against this + supervisor `balena-supervisor-sut` and run API tests + against this service instead of the default `balena-supervisor`. footer: @@ -11500,7 +11830,8 @@ Both the migrator and secureboot tests assumed they were creating an - installer config.json section. Modify the code so both settings are + installer config.json section. Modify the code so both settings + are included. footer: @@ -11579,13 +11910,17 @@ This patch adds a wait4file loop to the script that waits - for the /dev/disk/by-state directory. This is not tied to any particular + for the /dev/disk/by-state directory. This is not tied to any + particular - partition or device but since the directory does not exist by default + partition or device but since the directory does not exist by + default - and is only created by a custom balenaOS udev rule, its existence + and is only created by a custom balenaOS udev rule, its + existence - implies that the rule fired and a device with balenaOS partitions + implies that the rule fired and a device with balenaOS + partitions is present in the system. footer: @@ -11651,14 +11986,17 @@ A bug in service comparison would make it that a device already running - a service from a new release with network changes would never stop the + a service from a new release with network changes would + never stop the - running service so remaining services would forever get stuck in + running service so remaining services would forever get + stuck in `Downloaded` state. - This fixes the comparison so the service will get killed in this case, + This fixes the comparison so the service will get killed + in this case, particularly allowing devices to recover from #1576 footer: @@ -11672,11 +12010,14 @@ Devices affected by the bug described in 1576, are also stuck with some - services in the `Downloaded` state, because the state engine does not + services in the `Downloaded` state, because the state + engine does not - detect that the running services should be killed on a network change + detect that the running services should be killed on a + network change - even if they belong to a new release. This is a bug, which can be + even if they belong to a new release. This is a bug, + which can be replicated by the tests in this commit footer: @@ -11690,9 +12031,11 @@ Previous behavior would make it that an `updateMetadata` step would take - precedence over a `kill` step when network changes are present. This + precedence over a `kill` step when network changes are + present. This - would lead to an inconsistent state if an update included a + would lead to an inconsistent state if an update + included a network and a container change. footer: @@ -11716,10 +12059,12 @@ These tests use the supervisor API to check that applying a target state - allows the device to eventually get to the desired target configuration. + allows the device to eventually get to the desired + target configuration. - This are high-level tests that work with real images and containers + This are high-level tests that work with real images and + containers using dind. footer: @@ -11733,11 +12078,14 @@ The supervisor allows the target image to be an image without a - registry (e.g. `alpine:latest`), while this really only happens while in + registry (e.g. `alpine:latest`), while this really only + happens while in - local mode, we don't want to pass credentials to the default registry as + local mode, we don't want to pass credentials to the + default registry as - those credentials are meant for balena registry and will otherwise fail. + those credentials are meant for balena registry and will + otherwise fail. footer: Change-type: patch change-type: patch @@ -11768,7 +12116,8 @@ A safe copy would only work for files that are read by fatrw also and that is - not the case for boot files. Still, some file like `config.json` would + not the case for boot files. Still, some file like `config.json` + would benefit from a safe copy so we still try that first. @@ -11787,14 +12136,17 @@ If a safe copy is preferred but non-critical, the unsafe fatrw command - can be used and if fatrw does not have enough resources to make a safe + can be used and if fatrw does not have enough resources to make + a safe copy it will fallback to a standard cp. - This is useful when performing hostOS updates for example where a safe + This is useful when performing hostOS updates for example where + a safe - copy would only work for files that are read by fatrw also and that is + copy would only work for files that are read by fatrw also and + that is not the case for boot files for example. footer: @@ -11915,7 +12267,8 @@ https://github.com/moby/libnetwork/pull/1805 - This patch is meant to avoid cases in which libnetwork internal state + This patch is meant to avoid cases in which libnetwork + internal state gets inconsistent in case of crashes. footer: @@ -12042,7 +12395,8 @@ Target volatile doesn't make sense now that we can use the - current state as a target. It wasn't actually being used for anything + current state as a target. It wasn't actually being used + for anything anymore apparently footer: @@ -12059,7 +12413,8 @@ from the rest of the code. - The function `applyIntermediateTarget` will now call `pausingApply` + The function `applyIntermediateTarget` will now call + `pausingApply` before applying the target @@ -12079,12 +12434,14 @@ engine. - - doPurge first removes the user app from the target state and passes + - doPurge first removes the user app from the target + state and passes that to the state engine for purging. Since intermediate state doesn't remove images, this will have the effect of basically re-installing the app. - - doRestart modifies the target state by first removing only the + - doRestart modifies the target state by first removing + only the services from the current state but keeping volumes and networks. This has the same effect as before where services were stopped one by one footer: @@ -12098,7 +12455,8 @@ Local mode uses a numeric `appUuid` which was messing up parsing the - network name. This fixes this issue so the current state can be used + network name. This fixes this issue so the current state + can be used as a target state footer: @@ -12112,12 +12470,15 @@ The Service class in `compose/service.ts` cannot get the image name - from the image id when building the object from the container metadata. + from the image id when building the object from the + container metadata. - We query the metadata in the application manager getCurrentApps method + We query the metadata in the application manager + getCurrentApps method - so the current state can be used as target by API methods + so the current state can be used as target by API + methods footer: Change-type: patch change-type: patch @@ -12129,7 +12490,8 @@ Network aliases are now compared checking that the target state is a - subset of the current state. This will prevent service restarts due to + subset of the current state. This will prevent service + restarts due to additional aliases created by docker in the container. footer: @@ -12145,12 +12507,15 @@ When getting the service from the docker container, remove the - containerId from the list of aliases (which gets added by docker). This + containerId from the list of aliases (which gets added + by docker). This - will make it easier to use the current service state as a target. + will make it easier to use the current service state as + a target. - This will help us remove the `safeStateClone` function in the API in a + This will help us remove the `safeStateClone` function + in the API in a future commit footer: @@ -12164,7 +12529,8 @@ This replaces the previous flag `isApplyingIntermediate` on application - manager and simplifies the interface of the state engine to make temporary changes to the + manager and simplifies the interface of the state engine + to make temporary changes to the general app state. footer: @@ -12178,21 +12544,26 @@ There were multiple places in the state engine that skipped some - operations while in local mode. In reality, all it's needed while in + operations while in local mode. In reality, all it's + needed while in local mode is to skip image and volume deletion. - This commit simplifies application-manager and compose app to be more + This commit simplifies application-manager and compose + app to be more - local mode agnostic and instead making the image deletion and volume + local mode agnostic and instead making the image + deletion and volume deletion configurable via function arguments. - This also has the benefit to make the treatment of local mode + This also has the benefit to make the treatment of local + mode - applications more similar to cloud mode applications, allowing for + applications more similar to cloud mode applications, + allowing for API endpoints to function the same way both modes. footer: @@ -12237,13 +12608,16 @@ When parsing additional variables to be passed to the bitbake build, - keys and values are split using equals as a delimiter. However, the + keys and values are split using equals as a delimiter. However, + the - splitting process does not split only on the first occurrence, which + splitting process does not split only on the first occurrence, + which results in removing equals signs from the value as well. This is - problematic with base64 encoded strings, which are padded with equals + problematic with base64 encoded strings, which are padded with + equals signs. @@ -12288,11 +12662,14 @@ The OS since v2.82.6 will monitor changes to config.json and restart - the relevant services to apply the changes. There is no need to trigger + the relevant services to apply the changes. There is no + need to trigger - restart of the services via the supervisor. Users on older OS versions + restart of the services via the supervisor. Users on + older OS versions - will need to update their OS or restart the services manually as OS + will need to update their OS or restart the services + manually as OS loses support after 2y. footer: @@ -12312,7 +12689,8 @@ We don't need this anonymous volume as /data is bind mounted into - the container from host (legacy), and will soon be mounted by the + the container from host (legacy), and will soon be + mounted by the Supervisor itself on startup. footer: @@ -12367,7 +12745,8 @@ In order to use hashes we can not use UEFI time-based authentication - for updates as this would prevent rollbacks. Instead we ship appendable + for updates as this would prevent rollbacks. Instead we ship + appendable updates for both db and dbx that HUP can use. footer: @@ -12383,18 +12762,23 @@ This patch changes the validation of bootable images from certificate - signatures to a list of allowed hashes of binaries. This only applies + signatures to a list of allowed hashes of binaries. This only + applies on db level, PK and KEK are still certificates. - The motivation is that certificates expire and we need to be sure + The motivation is that certificates expire and we need to be + sure - that even devices that have been lying on a shelf for several years + that even devices that have been lying on a shelf for several + years - or whose CMOS battery has died and reset date to 1970-01-01 are still + or whose CMOS battery has died and reset date to 1970-01-01 are + still - bootable. Using hashes is more aligned with this use-case and also + bootable. Using hashes is more aligned with this use-case and + also more similar to the approach that embedded SoCs use. footer: @@ -12426,18 +12810,22 @@ Shipping a single image with signature checks enabled will enforce - the signatures on non-secure-boot systems as well. GRUB does not have + the signatures on non-secure-boot systems as well. GRUB does not + have - a simple method to check whether secure boot is enabled that could + a simple method to check whether secure boot is enabled that + could be embedded with the default built-in config. With this patch we build two separate images - one enforcing - the signatures and the other one not, keeping the original behavior. + the signatures and the other one not, keeping the original + behavior. - HUP and flasher both can detect if secure boot is enabled so they + HUP and flasher both can detect if secure boot is enabled so + they put the correct image in place when installing/updating GRUB. footer: @@ -12483,9 +12871,11 @@ The installer is to copy configuration files into the boot partition on - the installer disk - searching by label needs to be restriced to the + the installer disk - searching by label needs to be restriced to + the - booting disk to avoid clashes if there are other disks with matching + booting disk to avoid clashes if there are other disks with + matching labels present. footer: @@ -12511,7 +12901,8 @@ Search for the installation disk on the same device the system is being - installed on. This avoids problems when there are more than one disk + installed on. This avoids problems when there are more than one + disk with balena/resin labelling. footer: @@ -12527,7 +12918,8 @@ The internal target device to program is not always the device the system - is booting from. Make sure the `flash-boot` partition search is done + is booting from. Make sure the `flash-boot` partition search is + done on the booting device. footer: @@ -12617,14 +13009,18 @@ rejections](https://github.com/nodejs/node/blob/main/doc/changelogs/CHANGELOG_V15.md#throw-on-unhandled-rejections---33021) from a warning to a throw. - For this reason errors like a corrupt migration directory, that happens when trying to + For this reason errors like a corrupt migration + directory, that happens when trying to - roll back to a previous supervisor version were no longer showing a + roll back to a previous supervisor version were no + longer showing a - message but dumping the full minimized code into the journal logs. + message but dumping the full minimized code into the + journal logs. - This PR adds a catchall on app.ts to log the exception and throw an exit + This PR adds a catchall on app.ts to log the exception + and throw an exit code of 1. footer: @@ -12642,9 +13038,11 @@ From: https://github.com/balena-os/balena-supervisor/pull/2153/commits/c0b4fafe842115933b1da9b4d68e601a19c3e4eb - Restart-service checks that both services have restarted in its test assertion, which is + Restart-service checks that both services have restarted + in its test assertion, which is - incorrect as restart-service should only restart one service. + incorrect as restart-service should only restart one + service. footer: Change-type: patch change-type: patch @@ -12871,24 +13269,32 @@ As the Supervisor is a privileged container, it has access to host /dev, and therefore has access - to boot, data, and state balenaOS partitions. This commit sets up the framework for the following: + to boot, data, and state balenaOS partitions. This + commit sets up the framework for the following: - - Finds the /dev partition that corresponds to each partition based on partition label + - Finds the /dev partition that corresponds to each + partition based on partition label - - Mounts the partitions into set mountpoints in the device + - Mounts the partitions into set mountpoints in the + device - - Removes reliance on env vars and mountpoints provided by host's start-balena-supervisor script + - Removes reliance on env vars and mountpoints provided + by host's start-balena-supervisor script - - Simplifies host path querying by centralizing these queries through methods in lib/host-utils.ts + - Simplifies host path querying by centralizing these + queries through methods in lib/host-utils.ts - This particular changes env vars for and mounts the boot partition. + This particular changes env vars for and mounts the boot + partition. - Since the Supervisor would no longer rely on container `run` arguments provided by a host script, + Since the Supervisor would no longer rely on container + `run` arguments provided by a host script, - this change moves Supervisor closer to being able to start itself (Supervisor-as-an-app). + this change moves Supervisor closer to being able to + start itself (Supervisor-as-an-app). footer: Change-type: minor change-type: minor @@ -12931,11 +13337,14 @@ Notable improvements these new versions bring: - * Optimized code path for generating deltas with blocks that are + * Optimized code path for generating deltas with blocks + that are power-of-two-sized. - * Avoid allocating unbounded amounts of memory when the target differs + * Avoid allocating unbounded amounts of memory when the + target differs completely from the source. - * Several bugfixes in edge cases that shall not affect balenaEngine. + * Several bugfixes in edge cases that shall not affect + balenaEngine. footer: Signed-off-by: Leandro Motta Barros signed-off-by: Leandro Motta Barros @@ -13032,14 +13441,17 @@ --tpmstate argument - If the same state directory/files are used for multiple installations, + If the same state directory/files are used for multiple + installations, - the available space can be filled, and the installer will fail with the + the available space can be filled, and the installer + will fail with the error "insufficient space for NV allocation". - Move swtpm state to tmpfs to create new state files every run. + Move swtpm state to tmpfs to create new state files + every run. footer: Change-type: patch change-type: patch @@ -13176,12 +13588,14 @@ Setting `LimitCORE=0` will avoid the creation of core dump files on - containers. This will avoid cases in which a crashlooping user app ends + containers. This will avoid cases in which a crashlooping user + app ends up filling up the entire storage with dump files. - Users can re-enable core dumps in their services by manually setting the + Users can re-enable core dumps in their services by manually + setting the `ulimits.core`. For example: @@ -13217,7 +13631,8 @@ https://github.com/moby/moby/commit/d16737f971092767c1b9d28302a3f5aedbe2f576 - And also is recommended by systemd: https://systemd.io/CGROUP_DELEGATION/ + And also is recommended by systemd: + https://systemd.io/CGROUP_DELEGATION/ footer: Signed-off-by: Leandro Motta Barros signed-off-by: Leandro Motta Barros @@ -13267,16 +13682,20 @@ The original delta code on the `xfer` package used to set `d.err` when - an unexpected EOF was found in the delta tar stream. Turns out `d.err` + an unexpected EOF was found in the delta tar stream. + Turns out `d.err` - would end up being overwritten before it was read, so that initial + would end up being overwritten before it was read, so + that initial assignment was effectively a no-op. - This commit simplifies the code a little bit by removing this bogus + This commit simplifies the code a little bit by removing + this bogus - assignment and also improves the error reporting a notch by adding more + assignment and also improves the error reporting a notch + by adding more context to the error messages. footer: @@ -13292,7 +13711,8 @@ This factors out portions of the `xfer` package, so that we can - reuse this functionality between `balena pull` and `balena load`. There + reuse this functionality between `balena pull` and + `balena load`. There was a good deal of duplication. footer: @@ -13308,31 +13728,39 @@ This factors out portions of the `distribution` package, so that we can - reuse this functionality between `balena pull` and `balena load`. There + reuse this functionality between `balena pull` and + `balena load`. There was a good deal of duplication. - This piece of code is tricky to factor out into a separate function. + This piece of code is tricky to factor out into a + separate function. Basically we had two options: - 1. Create a completely reusable, DRY function that encapsulates all the + 1. Create a completely reusable, DRY function that + encapsulates all the duplicate code. Sounds nice until you noticed that would be a horrendous function with 5 return values and overly obscure semantics. - 2. We create a small set of functions with clearer interfaces and + 2. We create a small set of functions with clearer + interfaces and semantics, but which will still lead to some code duplication between the `pull` and `load` implementations. - I opted for the second alternative because the resulting code is much + I opted for the second alternative because the resulting + code is much - easier to understand and maintain. Also, the remaining duplication is + easier to understand and maintain. Also, the remaining + duplication is - mostly dumb, integration and error handling code that almost writes + mostly dumb, integration and error handling code that + almost writes - itself as we call the new reusable functions -- so, this is sort of a + itself as we call the new reusable functions -- so, this + is sort of a benign duplication. footer: @@ -13378,7 +13806,8 @@ 00e389e5f559dd10e49cfa411784b89498c3c0eb. - Images generated using this dockerfile still don't have the right + Images generated using this dockerfile still don't have + the right architecture. More testing is needed footer: @@ -13469,16 +13898,21 @@ This is necessary since the builder no longer passes the platform flag - to the build. This would lead to dockerfiles that are mixing multi and single + to the build. This would lead to dockerfiles that are + mixing multi and single - arch stages to pull the wrong architecture images, particularly when + arch stages to pull the wrong architecture images, + particularly when - trying to build images in emulated builds (e.g. armv7hf built on aarch64). + trying to build images in emulated builds (e.g. armv7hf + built on aarch64). - Moving the full build to multi-arch solves this as the docker engine is + Moving the full build to multi-arch solves this as the + docker engine is - capable of chosing the right architecture from the manifest. + capable of chosing the right architecture from the + manifest. footer: Relatest-to: balena-io/balena-builder#1010 relatest-to: balena-io/balena-builder#1010 @@ -13537,7 +13971,8 @@ There were various usages of Bash-specific features. As a result, the - script would work correctly only on OSes that have `sh` as an alias to + script would work correctly only on OSes that have `sh` + as an alias to `bash`. It would fail on Ubuntu, for example. footer: @@ -13553,9 +13988,11 @@ A couple of changes here: - * Check for sudo necessity and availability before doing any real work. + * Check for sudo necessity and availability before doing + any real work. Better to warn and exit quick and early! - * Remove the support for using `su`. It was broken for two reasons. + * Remove the support for using `su`. It was broken for + two reasons. First, unlike `sudo`, `su -c` expects the command as a single argument. Second, `su`, unlike `sudo`, reads the password from stdin which in this case "contains" the tarball being downloaded. The second @@ -13576,9 +14013,11 @@ A couple of changes here: - * Check for missing dependencies before doing any real work. Better to + * Check for missing dependencies before doing any real + work. Better to warn and exit quick and early! - * Fix the actual check. We previously used `[ $abort ] && exit 1` which + * Fix the actual check. We previously used `[ $abort ] + && exit 1` which caused the script to always exit (`abort` is never empty). footer: Signed-off-by: Leandro Motta Barros @@ -13669,7 +14108,8 @@ Added an `Asserting` suffix to all functions that internally call - `assert.*()`. This makes clearer what is really going on at the point of + `assert.*()`. This makes clearer what is really going on + at the point of call, without needing to look under the hood. footer: @@ -13695,7 +14135,8 @@ Most notably, on the "delta root" feature, which is important for HUPs, - not very well-known and not documented anywhere else I know. + not very well-known and not documented anywhere else I + know. footer: Signed-off-by: Leandro Motta Barros signed-off-by: Leandro Motta Barros @@ -13723,7 +14164,8 @@ We apparently have broken this during the 20.10 merge. Not setting the - delta image store breaks delta-based balenaOS updates (HUPs). + delta image store breaks delta-based balenaOS updates + (HUPs). footer: Signed-off-by: Leandro Motta Barros signed-off-by: Leandro Motta Barros @@ -13808,9 +14250,11 @@ https://github.com/containerd/console/pull/10/commits/c358734ec94e72903243bd1c9034874a1de09424 - This fix is present in balena engine since v17.13.5, which has been in + This fix is present in balena engine since v17.13.5, which has + been in - use since commit 53ce147. Drop this patch from meta-balena-dunfell and + use since commit 53ce147. Drop this patch from + meta-balena-dunfell and later. footer: @@ -13826,9 +14270,11 @@ Plymouth services are modified in the installation directory with balena - specific customizations using patches. This increases the manual intervention necessary + specific customizations using patches. This increases the manual + intervention necessary - when adding support for a new version of this package or Yocto. Replace + when adding support for a new version of this package or Yocto. + Replace the patches with equivalent drop-in configs. footer: @@ -13844,9 +14290,11 @@ We disable systemd-getty-generator to allow explicit control over when - we setup getty to create consoles. Previously, this was done using a + we setup getty to create consoles. Previously, this was done + using a - patch to systemd, removing this generator. Mask this instead so we can + patch to systemd, removing this generator. Mask this instead so + we can consolidate this configuration in meta-balena-common. footer: @@ -13862,11 +14310,14 @@ Certain services, such as getty@.service, and systemd-logind.service are - disabled when running in a container using a patch to the source files. + disabled when running in a container using a patch to the source + files. - This increases the manual intervention necessary when adding support for + This increases the manual intervention necessary when adding + support for - a new version of systemd. Replace the patch with drop-in configs. + a new version of systemd. Replace the patch with drop-in + configs. footer: Change-type: patch change-type: patch @@ -13939,21 +14390,27 @@ Some incoming tests require QEMU to exit, simulating a device powering - off, before starting QEMU again. This is used to "reflash" a virtualized + off, before starting QEMU again. This is used to + "reflash" a virtualized - device before continuing with testing, for instance after tampering with + device before continuing with testing, for instance + after tampering with - boot files on a secure boot enabled device to verify secure boot checks. + boot files on a secure boot enabled device to verify + secure boot checks. - However, swtpm will exit when QEMU disconnects. The `--exit-code-from` + However, swtpm will exit when QEMU disconnects. The + `--exit-code-from` - compose argument implies `--abort-on-container-exit`, so this results + compose argument implies `--abort-on-container-exit`, so + this results in the test run aborting prematurely. - Adapt the entrypoint and command of the swtpm container to always + Adapt the entrypoint and command of the swtpm container + to always restart the program without exiting the container. footer: @@ -14021,9 +14478,11 @@ Disable ad-hoc unwrapping in the HUP test suite in favor of utilizing - the QEMU worker's new ability to bind a disk image to an emulated + the QEMU worker's new ability to bind a disk image to an + emulated - external USB mass storage device. This runs the flasher in QEMU, and + external USB mass storage device. This runs the flasher in QEMU, + and installs to emulated internal storage. footer: @@ -14057,11 +14516,14 @@ It should be safe to assume that boards now use newer u-boot versions - that all have Kconfig support so we default to that. This allows for + that all have Kconfig support so we default to that. This allows + for - device repos not to specify it and use Kconfig support or if for some + device repos not to specify it and use Kconfig support or if for + some - reason there are boards with old u-boot versions they can overwrite the + reason there are boards with old u-boot versions they can + overwrite the UBOOT_KCONFIG_SUPPORT variable to 0. footer: @@ -14124,7 +14586,8 @@ be overwritten in append files. - This change is an extension of https://github.com/balena-os/meta-balena/commit/a3c276a1058d05e66991871bf167079fc2824843 + This change is an extension of + https://github.com/balena-os/meta-balena/commit/a3c276a1058d05e66991871bf167079fc2824843 footer: Change-type: patch change-type: patch @@ -14227,9 +14690,11 @@ The unsafe-perm config option has been dropped in npm 9, trying to set it - ends with an error and therefore fails the build. With this patch + ends with an error and therefore fails the build. With this + patch - the build script parses the major version from `npm --version` and only + the build script parses the major version from `npm --version` + and only sets unsafe-perm on npm 8 and older. footer: @@ -14261,11 +14726,14 @@ Because we use this patch with various u-boot versions it often happens that this patch - does not apply so we then need to rework it in the device integration layer. Instead it + does not apply so we then need to rework it in the device + integration layer. Instead it - would be better to have some code at configure time parsing the same file and inserting + would be better to have some code at configure time parsing the + same file and inserting - the balena env dynamically, so regardless of u-boot versions we use. + the balena env dynamically, so regardless of u-boot versions we + use. footer: Change-type: patch change-type: patch @@ -14293,18 +14761,23 @@ Preloaded devices can require that the device is pinned to the preloaded - release on provisioning. However if the provisioned release gets + release on provisioning. However if the provisioned + release gets - released in the future, that would lead to the device remaining in "VPN + released in the future, that would lead to the device + remaining in "VPN - only" state forever as the provisioning process could not finish due to + only" state forever as the provisioning process could + not finish due to pinning failure. - This commit changes the behavior so if the release does not exist, the + This commit changes the behavior so if the release does + not exist, the - pinning step is skipped and the device follows the fleet pinning state. + pinning step is skipped and the device follows the fleet + pinning state. footer: Closes: "#2133" closes: "#2133" @@ -14322,16 +14795,21 @@ This is necessary since the builder no longer passes the platform flag - to the build. This would lead to dockerfiles that are mixing multi and single + to the build. This would lead to dockerfiles that are + mixing multi and single - arch stages to pull the wrong architecture images, particularly when + arch stages to pull the wrong architecture images, + particularly when - trying to build images in emulated builds (e.g. armv7hf built on aarch64). + trying to build images in emulated builds (e.g. armv7hf + built on aarch64). - Moving the full build to single-arch solves this as the docker engine is + Moving the full build to single-arch solves this as the + docker engine is - capable of chosing the right architecture from the manifest. Once some + capable of chosing the right architecture from the + manifest. Once some of the builder issues are fixed, we should move to #2141 footer: @@ -14351,13 +14829,17 @@ The issue with the original Supervisor implementation of the firewall is that - on Supervisor start, the Supervisor flushes the INPUT chain of the filter table. + on Supervisor start, the Supervisor flushes the INPUT + chain of the filter table. - This doesn't play well with services that add to the INPUT chain on startup that + This doesn't play well with services that add to the + INPUT chain on startup that - may start up before the Supervisor, such as certain NetworkManager connection + may start up before the Supervisor, such as certain + NetworkManager connection - profiles. This change only replaces the BALENA-FIREWALL rule in the INPUT chain, + profiles. This change only replaces the BALENA-FIREWALL + rule in the INPUT chain, preserving the other rules as well as their order. footer: @@ -14423,9 +14905,11 @@ In the Internet connection sharing test one of the checks may run - into a racing problem. The following command is holding the iptables + into a racing problem. The following command is holding the + iptables - lock for 3 seconds while NetworkManager activates a connection with + lock for 3 seconds while NetworkManager activates a connection + with sharing enabled: @@ -14433,20 +14917,25 @@ `flock /run/xtables.lock sleep 3 & nmcli c up dummy & wait` - NetworkManager waits for 2 seconds for the lock to be released and + NetworkManager waits for 2 seconds for the lock to be released + and - those three seconds should be enough for one of the iptables rules + those three seconds should be enough for one of the iptables + rules - to fail. However there is no guarantee that NetworkManager will start + to fail. However there is no guarantee that NetworkManager will + start - adding the iptables rules that quickly - it may start adding those + adding the iptables rules that quickly - it may start adding + those after one out of those three seconds already passed, which will lead to all iptables rules to be set at the end. - This check is non-essential for the test itself, so it is removed with + This check is non-essential for the test itself, so it is + removed with this commit. footer: @@ -14466,9 +14955,11 @@ At this moment, when module signing is enabled, the peak module is signed - but do_install ignores the signed variant and installs the original + but do_install ignores the signed variant and installs the + original - unsigned file. With this patch do_install installs the signed file + unsigned file. With this patch do_install installs the signed + file if available. footer: @@ -14500,21 +14991,26 @@ Previously, we bailed out of the installer when the system was in user - mode (keys enrolled) but the user had not opted in to secure boot, as it + mode (keys enrolled) but the user had not opted in to secure + boot, as it was ambiguous whether the user actually wanted SB/FDE. - However, some systems come with vendor keys pre-enrolled, and a user may + However, some systems come with vendor keys pre-enrolled, and a + user may - simply turn off secure boot in the firmware setup menu without erasing + simply turn off secure boot in the firmware setup menu without + erasing - the keys. This would result in the installer bailing out even though + the keys. This would result in the installer bailing out even + though secure boot is disabled in the firmware menu. - Check that secure boot is enabled in addition to having keys enrolled + Check that secure boot is enabled in addition to having keys + enrolled before bailing out. footer: @@ -14546,16 +15042,20 @@ https://github.com/balena-os/meta-balena/pull/2963 - A racing condition between balenaEngine and NetworkManager led to some Internet + A racing condition between balenaEngine and NetworkManager led + to some Internet - connection sharing iptables rules not being applied when NM connection profile + connection sharing iptables rules not being applied when NM + connection profile with Internet sharing was activated at boot. - This test checks whether all necessary iptables rules are added by artificially + This test checks whether all necessary iptables rules are added + by artificially - blocking iptables for 1 second and then for 3 seconds while a connection + blocking iptables for 1 second and then for 3 seconds while a + connection profile with Internet sharing is activated. footer: @@ -14608,13 +15108,16 @@ do_resin_boot_dirgen_and_deploy needs all the partial files deployed - when it runs as it will be copying them to the actual boot partition. + when it runs as it will be copying them to the actual boot + partition. There is a race condition between it and grub_conf:do_deploy, - we have seen builds fail when grub_conf:do_deploy does not execute + we have seen builds fail when grub_conf:do_deploy does not + execute - in time. This patch adds an explicit dependency to avoid such situation. + in time. This patch adds an explicit dependency to avoid such + situation. footer: Change-type: patch change-type: patch @@ -14631,17 +15134,22 @@ body: > We have added DER certificates because it is the only format - that QEMU/TianoCore accepts in UEFI setup. Since we made setup mode + that QEMU/TianoCore accepts in UEFI setup. Since we made setup + mode - work properly, there is no need to program the QEMU keys manually + work properly, there is no need to program the QEMU keys + manually - and therefore there is no reason to pollute the boot partition with + and therefore there is no reason to pollute the boot partition + with the DER files. If indeed necessary, they can always be extracted - from the ESL, which is still shipped. Shipping the DER files is also + from the ESL, which is still shipped. Shipping the DER files is + also - not possible when db uses hashes instead of certificates, which we + not possible when db uses hashes instead of certificates, which + we want to move towards. footer: @@ -14661,7 +15169,8 @@ This allows to transition from unmanaged to managed by just writing - a config.json file, like it's the case with AWS cloud configuration. + a config.json file, like it's the case with AWS cloud + configuration. footer: Change-type: patch change-type: patch @@ -14681,7 +15190,8 @@ that this is about balenaEngine makes it simpler to grep for - Engine-related portions of balenaOS. Might also help a tad bit when + Engine-related portions of balenaOS. Might also help a tad bit + when looking at logs. footer: @@ -14726,7 +15236,8 @@ The `du` utility has the same output format for single or several files, - while the `wc` utility doesn't and does not display a total for single + while the `wc` utility doesn't and does not display a total for + single files. footer: @@ -14760,9 +15271,11 @@ finish the flashing with a reboot instead of a shutdown. - This is possible as the initramfs does not currently have a shutdown + This is possible as the initramfs does not currently have a + shutdown - command and avoids the complexity of having to detect whether we are + command and avoids the complexity of having to detect whether we + are running from initramfs or not. footer: @@ -14789,7 +15302,8 @@ This module allows to program the internal storage from initramfs hence - making it possible to migrate an existing OS by booting from the same + making it possible to migrate an existing OS by booting from the + same disk. footer: @@ -14829,7 +15343,8 @@ The migrate module needs to umount the rootfs, so by moving the mountpoint - moving into the migrate module it's only done when not migrating. + moving into the migrate module it's only done when not + migrating. footer: Change-type: patch change-type: patch @@ -14846,12 +15361,14 @@ not boot into the OS but launch adbd. - A connection can then be established via the network (if a DHCP address + A connection can then be established via the network (if a DHCP + address is provided) or using a USB gadget connection. - For a network connection, the client needs to install `adb` and run: + For a network connection, the client needs to install `adb` and + run: ``` @@ -14950,7 +15467,8 @@ characters with a `*`. - [1] https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#filter-pattern-cheat-sheet + [1] + https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#filter-pattern-cheat-sheet [skip ci] @@ -14980,9 +15498,11 @@ There is nothing in `balena-config-vars` itself that needs `fatrw`, so - change it so scripts don't exit if it is not available. For example, + change it so scripts don't exit if it is not available. For + example, - `balena-config-vars` gets used in the initramfs but `fatrw` is not + `balena-config-vars` gets used in the initramfs but `fatrw` is + not required (and it is quite heavy). footer: @@ -15000,7 +15520,8 @@ balena-config scripts and not unit configuration dependencies. - This allows to include only the balena-config scripts in the initramfs. + This allows to include only the balena-config scripts in the + initramfs. footer: Change-type: patch change-type: patch @@ -15017,7 +15538,8 @@ the images size significantly. - This commit introduces a `raid` machine feature that device types need + This commit introduces a `raid` machine feature that device + types need to define if RAID support is required. footer: @@ -15035,7 +15557,8 @@ will also be used from the initramfs, move the dependency to - packagegroup-resin so that resin-device-progress is still included in + packagegroup-resin so that resin-device-progress is still + included in the flasher image. footer: @@ -15055,9 +15578,11 @@ flasher is not running from initramfs. - These dependencies are already part of the corresponding packagegroups, + These dependencies are already part of the corresponding + packagegroups, - so listing them here is redundant and wrong as it increases the size of + so listing them here is redundant and wrong as it increases the + size of the initramfs with no reason. footer: @@ -15108,7 +15633,8 @@ it is only used in the installer script. - Also, make the resin-init-flasher script check for it's existance before using + Also, make the resin-init-flasher script check for it's + existance before using it. footer: @@ -15127,7 +15653,8 @@ in disk encryption). - Adding a loop that waits for the links to be available adds robustness + Adding a loop that waits for the links to be available adds + robustness in case there are device specific delays. footer: @@ -15190,7 +15717,8 @@ - The Radxa CM3 on RPI CM4 IOBoard as well as the Radxa zero use rockchip software tools in order to put the eMMC in mass-storage mode - - The CM4 module comes in two flavors: one with eMMC and the Lite version + - The CM4 module comes in two flavors: one with eMMC and the + Lite version which uses the carrier board sd-card slot to load the image. Both use the same balenaOS image. I switched the storage to internal for this DT because it *may* have @@ -15228,16 +15756,19 @@ saving the entire uboot environment in any device specific - partitions. This because it relies on the default environment being + partitions. This because it relies on the default environment + being stored in the u-boot binary. Let's disable the saveenv command and avoid potential incorrect - usage which may overwrite the partition table, resin-boot filesystem + usage which may overwrite the partition table, resin-boot + filesystem - or other areas of the eMMC that may be pre-configured by the BSP. + or other areas of the eMMC that may be pre-configured by the + BSP. footer: Change-type: patch change-type: patch @@ -15312,9 +15843,11 @@ Block device nodes are sometimes created without attached media. These - devices can neither be read from, nor written to. In this case, the + devices can neither be read from, nor written to. In this case, + the - flasher will attempt to install to the invalid disk and fail. Detect + flasher will attempt to install to the invalid disk and + fail. Detect this case and skip the disk to allow flashing to continue. footer: @@ -15373,7 +15906,8 @@ Secure boot is now opt-in, even in the case where the image is signed, - and it's supported in firmware. Skip the secure boot tests when it's not + and it's supported in firmware. Skip the secure boot tests when + it's not enabled at runtime. footer: @@ -15420,7 +15954,8 @@ exceptions, and makes debugging and log messages worse. - When we don't have a valid way to handle an exception, just throw it. + When we don't have a valid way to handle an exception, + just throw it. The traceback is more useful than the handler. footer: @@ -15626,7 +16161,8 @@ body: > This is used to support falling back into the original OS when - performing a brownfield migration into balenaOS from a flasher image. + performing a brownfield migration into balenaOS from a flasher + image. footer: Change-type: patch change-type: patch @@ -15694,7 +16230,8 @@ It's not clear how the feature to skip tests work, so modify the commit - message to be of type patch to avoid balenaCI errors on type none. + message to be of type patch to avoid balenaCI errors on type + none. footer: Change-type: patch change-type: patch @@ -15782,25 +16319,32 @@ We have seen a few times devices with duplicated network names for some - reason. While we don't know the cause the networks get duplicates, this + reason. While we don't know the cause the networks get + duplicates, this - can be disruptive for updates as trying to create a container referencing a duplicate + can be disruptive for updates as trying to create a + container referencing a duplicate network results in a 400 error from the engine. - This commit finds and removes duplicate networks via the state engine, + This commit finds and removes duplicate networks via the + state engine, - this means that even if somehow a container could be referencing a + this means that even if somehow a container could be + referencing a - network that has been duplicated later somehow, this will remove the + network that has been duplicated later somehow, this + will remove the container first. - While thies doesn't solve the problem of duplicate networks being + While thies doesn't solve the problem of duplicate + networks being - created in the first place, it will fix the state of the system to + created in the first place, it will fix the state of the + system to correct the inconsistency. footer: @@ -15816,29 +16360,38 @@ We have seen a few times devices with duplicated network names for some - reason. While we don't know the cause the networks get duplicates, + reason. While we don't know the cause the networks get + duplicates, - this is disruptive of updates, as the supervisor usually queries + this is disruptive of updates, as the supervisor usually + queries - resource by name, resulting in a 400 error from the engine because of + resource by name, resulting in a 400 error from the + engine because of the ambiguity. - This replaces those queries by name to queries by id. This includes + This replaces those queries by name to queries by id. + This includes - network removal. If a `removeNetwork` step is generated, the supervisor + network removal. If a `removeNetwork` step is generated, + the supervisor - opts to remove all instances of the network with the same name as it + opts to remove all instances of the network with the + same name as it cannot easily resolve the ambiguity. - This doesn't solve the problem of ambiguous networks, because even if + This doesn't solve the problem of ambiguous networks, + because even if - networks are referenced by id when creating a container, the engine will + networks are referenced by id when creating a container, + the engine will - throw an error (see https://github.com/balena-os/balena-supervisor/issues/590#issuecomment-1423557871) + throw an error (see + https://github.com/balena-os/balena-supervisor/issues/590#issuecomment-1423557871) footer: Change-type: patch change-type: patch @@ -16033,12 +16586,14 @@ - references in docs - - references device-state, api-binder, compose modules, API + - references device-state, api-binder, compose modules, + API - references in tests - The commit also adds a migration to remove the 4 dependent device tables from the DB. + The commit also adds a migration to remove the 4 + dependent device tables from the DB. footer: Change-type: minor change-type: minor @@ -16109,7 +16664,8 @@ body: > Drop support for Fedora 34 35. - Fedora 36 will be the last version for armv7 as it is no longer supported. + Fedora 36 will be the last version for armv7 as it is no longer + supported. footer: Change-type: patch change-type: patch @@ -16151,9 +16707,11 @@ executed. When sourced, the shebang should be ignored. - However, we have seen instances where a bash script sourcing a sh + However, we have seen instances where a bash script sourcing a + sh - os-helper scripts triggers POSIX behaviour, specifically glob parsing + os-helper scripts triggers POSIX behaviour, specifically glob + parsing failures. footer: @@ -16438,7 +16996,8 @@ specific flags being misapplied, breaking the build - Set ARCH based on the target architecture, and override OBJCOPY to the + Set ARCH based on the target architecture, and override OBJCOPY + to the binary provided by the target architecture's toolchain. footer: @@ -16505,27 +17064,35 @@ When a user runs the flasher with secure boot enabled in `config.json`, - the public keys used to validate the bootloader are enrolled. If any + the public keys used to validate the bootloader are enrolled. If + any - other bootloader signature fails to validate against this public key, it + other bootloader signature fails to validate against this public + key, it won't be executed. - If the user attempts to run the balenaOS flasher on that system again + If the user attempts to run the balenaOS flasher on that system + again - without first enabling the secure boot option, the flasher won't enroll + without first enabling the secure boot option, the flasher won't + enroll - keys, but the installed system will be signed. This will result in a + keys, but the installed system will be signed. This will result + in a secure boot enabled system without full-disk encryption. - Bail out in this case so the user must choose to explicitly opt-in to + Bail out in this case so the user must choose to explicitly + opt-in to - secure boot for the new installation, and full-disk encryption along + secure boot for the new installation, and full-disk encryption + along - with it. Otherwise, the user must reset the enrolled keys to install + with it. Otherwise, the user must reset the enrolled keys to + install without secure boot. footer: @@ -16541,11 +17108,14 @@ Extended globbing is not enabled by default, which makes the substring - match for trimming leading zeroes not work. This causes SETUPMODEVAR to + match for trimming leading zeroes not work. This causes + SETUPMODEVAR to - evaluate to "01", which fails comparison with the string "1", skipping + evaluate to "01", which fails comparison with the string "1", + skipping - key enrollment when secure boot is enabled. Compare using an integer + key enrollment when secure boot is enabled. Compare using an + integer expression instead. footer: @@ -16561,21 +17131,26 @@ When refactoring secure boot setup, a logic mistake in the purpose and - use of SECUREBOOT_VAR meant that devices booting the flasher with keys already + use of SECUREBOOT_VAR meant that devices booting the flasher + with keys already - enrolled would bail out with an incorrect message about secure boot not + enrolled would bail out with an incorrect message about secure + boot not being supported in firmware. - This variable is `00` on systems with secure boot support in firmware, + This variable is `00` on systems with secure boot support in + firmware, - but not enabled and enforced, `01` on systems where secure boot is + but not enabled and enforced, `01` on systems where secure boot + is enforced, and empty when secure boot is unsupported. - Change this conditional to bail out only when the variable is empty, + Change this conditional to bail out only when the variable is + empty, indicating that secure boot is unsupported. footer: @@ -16673,9 +17248,11 @@ Not all platforms support secure boot, notably aarch64 using tianocore - firmware. Additionally, swtpm may not be available for all platforms. + firmware. Additionally, swtpm may not be available for + all platforms. - Accordingly, move the swtpm service to a separate compose file that is + Accordingly, move the swtpm service to a separate + compose file that is only used when secure boot is enabled. footer: @@ -16737,7 +17314,8 @@ QEMU is capable of using an emulated software TPM exposed via socket. A - TPM is necessary for full disk encryption (FDE), so add a service to + TPM is necessary for full disk encryption (FDE), so add + a service to provide this to the QEMU worker. footer: @@ -16782,7 +17360,8 @@ Some firmwares will not boot balenaOS by default without explicitly - creating a boot entry, so create one on EFI platforms after flashing. + creating a boot entry, so create one on EFI platforms after + flashing. footer: Change-type: patch change-type: patch @@ -16796,19 +17375,23 @@ get_dev_path_from_label() calls lsblk to get the name and label of a - disk, then filters the list using the label and returns a /dev path. + disk, then filters the list using the label and returns a /dev + path. The name returned when using a luks encrypted partition is the - /dev/mapper name, rather than the kernel's device mapper name under + /dev/mapper name, rather than the kernel's device mapper name + under - /dev/dm-*. When assembling a path under /dev using the luks name, the + /dev/dm-*. When assembling a path under /dev using the luks + name, the path is invalid, and the by-state links aren't created. - This leads to the rootfs hook failing to find and mount the resin-rootA + This leads to the rootfs hook failing to find and mount the + resin-rootA partition. @@ -16843,15 +17426,18 @@ The flasher image enrolls the secure boot keys before rebooting into - secured user mode and creating the encrypted luks volumes on disk. + secured user mode and creating the encrypted luks volumes on + disk. - If the image is not signed, the key enrollment will fail, and the + If the image is not signed, the key enrollment will fail, and + the flasher will enter a loop trying to enroll them and rebooting. - Instead, skip the key enrollment if the image is not signed, resulting + Instead, skip the key enrollment if the image is not signed, + resulting in a non secure-boot installation. footer: @@ -16976,7 +17562,8 @@ old hooks from, the EFI partition must be bind-mounted as well - otherwise the /mnt/boot/EFI symlink is invalid and rollback fails + otherwise the /mnt/boot/EFI symlink is invalid and rollback + fails to deploy files into that directory. footer: @@ -17117,7 +17704,8 @@ body: > iptables takes a file lock at /run/xtables.lock. By default, if - the file is locked, iptables will fail with error. When that happens, + the file is locked, iptables will fail with error. When that + happens, the iptables rules won't be configured, and the shared mode @@ -17185,10 +17773,12 @@ This EFI image contains the secure boot certificates and when executed it - is supposed to load the keys into the respective secure boot slots. + is supposed to load the keys into the respective secure boot + slots. - We don't use this binary in our secure boot implementation, but currently + We don't use this binary in our secure boot implementation, but + currently the build breaks as the binary is installed but not packaged. footer: @@ -17551,7 +18141,8 @@ to a new maximum of 2048. - See https://github.com/darkk/redsocks/blob/19b822e345f6a291f6cff6b168f1cfdfeeb2cd7d/base.c#L419 + See + https://github.com/darkk/redsocks/blob/19b822e345f6a291f6cff6b168f1cfdfeeb2cd7d/base.c#L419 footer: Change-type: patch change-type: patch @@ -17827,14 +18418,17 @@ body: > The previous method of disabling NTP by stopping the nameserver - (dnsmasq) relied on the dnsmasq unit not being reactivated before the + (dnsmasq) relied on the dnsmasq unit not being reactivated + before the test completed. - Instead, disable NTP by blocking ntp.org in the local dnsmasq instance + Instead, disable NTP by blocking ntp.org in the local dnsmasq + instance - using a dbus method call. NTP is re-enabled as before, by restarting + using a dbus method call. NTP is re-enabled as before, by + restarting dnsmasq. footer: @@ -17861,14 +18455,17 @@ body: > The previous method of disabling NTP by stopping the nameserver - (dnsmasq) relied on the dnsmasq unit not being reactivated before the + (dnsmasq) relied on the dnsmasq unit not being reactivated + before the test completed. - Instead, disable NTP by blocking ntp.org in the local dnsmasq instance + Instead, disable NTP by blocking ntp.org in the local dnsmasq + instance - using a dbus method call. NTP is re-enabled as before, by restarting + using a dbus method call. NTP is re-enabled as before, by + restarting dnsmasq. footer: @@ -18010,7 +18607,8 @@ Custom actions can only use certain secrets and single-dimension - run matrices. By running an entirely separate job after Flowzone + run matrices. By running an entirely separate job after + Flowzone is successful we have a lot more options. footer: @@ -18281,11 +18879,13 @@ In rare cases (believed to be caused by a non-atomic file creation and - writing operation in containerd), we end up with an empty file at + writing operation in containerd), we end up with an empty file + at `/mnt/data/docker/containerd/daemon/io.containerd.grpc.v1.introspection/uuid`. - This causes `ctr version` (and hence the health check) to fail. See + This causes `ctr version` (and hence the health check) to fail. + See https://github.com/balena-os/balena-engine/issues/322 @@ -18293,13 +18893,16 @@ This commit addresses this issue in two ways: - 1. Before running `ctr version`, we check if the uuid file exists and is + 1. Before running `ctr version`, we check if the uuid file + exists and is empty. If so, we remove it. (The subsequent execution of `ctr version` by the healthcheck will create the file again.) - 2. After running `ctr version`, we check if the uuid file was really + 2. After running `ctr version`, we check if the uuid file was + really created and is not empty. - In both cases, when an empty uuid file is detected, we log the event to + In both cases, when an empty uuid file is detected, we log the + event to help us confirm our hypothesis about the root cause. footer: @@ -18468,16 +19071,19 @@ If the signing server's response is anything other than successful, such - as with an authentication failure or bad request, the HTTP status code + as with an authentication failure or bad request, the HTTP + status code and response are hidden due to the --silent flag passed to cURL. - Drop the stdio redirect to the output file along with the --silent flag, + Drop the stdio redirect to the output file along with the + --silent flag, and instead use the -o parameter to output the response to the - appropriate file on success. This allows the status code and response to + appropriate file on success. This allows the status code and + response to be shown in the logs upon failure. footer: @@ -18511,12 +19117,14 @@ meta-openembedded, so that all improvements are merged now. - Excluded from it are `iwd` and `dhcpcd` daemon configurations that are + Excluded from it are `iwd` and `dhcpcd` daemon configurations + that are not used by us. - Default NM firewall in meta-openembedded is `nftables` where we are still + Default NM firewall in meta-openembedded is `nftables` where we + are still using `iptables`. @@ -18524,11 +19132,14 @@ The new recipe relies on `meson` as a build system now. - The .bbapend file that contains modifications specific to balena is preserved. + The .bbapend file that contains modifications specific to balena + is preserved. - Only `balena-client-id.patch` is removed as it references code that no longer + Only `balena-client-id.patch` is removed as it references code + that no longer - exists. This is because the internal systemd DHCPv4 client code that NM used + exists. This is because the internal systemd DHCPv4 client code + that NM used is now replaced by nettools' n-dhcp4 implementation. @@ -18556,7 +19167,8 @@ so we reuse the fixed version from upstream. - The symptom is that DNS servers provided by DHCP are not being used. + The symptom is that DNS servers provided by DHCP are not being + used. Closes #2907 @@ -18907,7 +19519,8 @@ a unique subnet that is not in use. - The DinD daemon in the core service will also start with a non-default + The DinD daemon in the core service will also start with + a non-default subnet. footer: @@ -18967,11 +19580,14 @@ When unlocking LUKS devices, udev events initializing the DM devices are still - generated in the background even after cryptsetup luksOpen returns. We need to + generated in the background even after cryptsetup luksOpen + returns. We need to - wait for the udev processing to finish before killing udev and cleaning up + wait for the udev processing to finish before killing udev and + cleaning up - the udev database to avoid having to deal with partially initialized devices + the udev database to avoid having to deal with partially + initialized devices or corrupted udev database in the target OS. footer: @@ -19136,9 +19752,11 @@ Our initramfs is built into the kernel, which is always compressed. - Disable redundant initramfs compression, which should save some CPU + Disable redundant initramfs compression, which should save some + CPU - cycles during build and boot, as well as improving compression ratio. + cycles during build and boot, as well as improving compression + ratio. footer: Change-type: patch change-type: patch @@ -19168,16 +19786,21 @@ In the current state the cryptsetup initrd script tries to unlock all - LUKS volumes in the system using the TPM. This includes user-defined LUKS + LUKS volumes in the system using the TPM. This includes + user-defined LUKS - volumes that, if present, fail to unlock and make the system unbootable. + volumes that, if present, fail to unlock and make the system + unbootable. - We should also not touch user-defined volumes in the first place. + We should also not touch user-defined volumes in the first + place. - This patch modifies the cryptsetup script to only unlock LUKS volumes + This patch modifies the cryptsetup script to only unlock LUKS + volumes - that are on the OS drive (same block device as the EFI partition). + that are on the OS drive (same block device as the EFI + partition). footer: Change-type: patch change-type: patch @@ -19271,12 +19894,16 @@ ``` - /dev/sdd2: LABEL="flash-rootA" UUID="5585296a-c183-4b10-89ae-20607e5604be" TYPE="ext4" PARTLABEL="resin-rootA" PARTUUID="582478f2-be4b-4279-9124-536385c9551d" + /dev/sdd2: LABEL="flash-rootA" + UUID="5585296a-c183-4b10-89ae-20607e5604be" TYPE="ext4" + PARTLABEL="resin-rootA" + PARTUUID="582478f2-be4b-4279-9124-536385c9551d" ``` - This commit fixes the inconsistency as the PARTLABEL is used as a fallback + This commit fixes the inconsistency as the PARTLABEL is used as + a fallback method to identify devices. footer: @@ -19461,11 +20088,14 @@ Not all the boards we support have the redsocks uid as 995 in their rootfs so let's - fetch the actual redsocks uid from the DUT before running the proxy tests and + fetch the actual redsocks uid from the DUT before running the + proxy tests and - update that in the docker-compose.yml. We do so because the REDSOCKS_UID value + update that in the docker-compose.yml. We do so because the + REDSOCKS_UID value - isn't substituted in the compose if the variable, even if it is passed trough + isn't substituted in the compose if the variable, even if it is + passed trough the cli. footer: @@ -19497,19 +20127,23 @@ Enabling CONFIG_KERNEL_ZSTD=y improves the compression ratio compared - to gzip while being faster to decompress. With kernel 5.15 in balenaOS + to gzip while being faster to decompress. With kernel 5.15 in + balenaOS v2.105, we see the 24 MB kernel compress to approximately 19 MB. - Zstd support was added in commit 48f7ddf, first introduced in kernel + Zstd support was added in commit 48f7ddf, first introduced in + kernel v5.9. Enable this config unconditionally in supported kernels. - Note that not every architecture and device support this option, but in + Note that not every architecture and device support this option, + but in - those cases, Kconfig will automatically disable it as HAVE_ZSTD is also + those cases, Kconfig will automatically disable it as HAVE_ZSTD + is also missing. footer: @@ -19529,7 +20163,8 @@ Generate a bmap file from the sparse image to allow for punching holes - in the disk image ranges that were unmapped after building. This data is + in the disk image ranges that were unmapped after building. This + data is lost during compression, and the bmapfile allows for recreating, @@ -19599,12 +20234,14 @@ eb69ff445fe0cac4f2060e67fa6994e61c3ca4b9. - Hardcoding the bridge address like this results in conflicts + Hardcoding the bridge address like this results in + conflicts when multiple instances are running on one jenkins node. - A new solution for local workstation testing will have to be + A new solution for local workstation testing will have + to be considered. footer: @@ -19645,7 +20282,8 @@ Instead of retrying to get the DUT IP address 120 times on a 1 seconds interval, - let's reduce it to 30 times because the resolveLocalTarget which we call will + let's reduce it to 30 times because the + resolveLocalTarget which we call will timeout too in 15 seconds: @@ -19653,7 +20291,8 @@ https://github.com/balena-os/leviathan-worker/blob/master/lib/helpers/index.ts#L162 - So reducing the retries number to 30 will effectly bring the total combined timeout to a maximum of 8 minutes. + So reducing the retries number to 30 will effectly bring + the total combined timeout to a maximum of 8 minutes. footer: Change-type: patch change-type: patch @@ -19690,7 +20329,8 @@ The testbot AP is visible and is discovered during a scan. - Let's remove the hidden attribute as it may cause problems + Let's remove the hidden attribute as it may cause + problems for the 243390-rpi wireless tests. footer: @@ -19844,9 +20484,11 @@ There are two GRUB config variants - one for regular devices and one - for devices with FDE enabled. This commit makes flasher include the latter + for devices with FDE enabled. This commit makes flasher include + the latter - in the boot partition when secure boot and FDE is included in the image. + in the boot partition when secure boot and FDE is included in + the image. footer: Change-type: patch change-type: patch @@ -19974,7 +20616,8 @@ get_part_number_by_label expects the block device name without the /dev/ - prefix, flasher uses this correctly in all but one place, this patch fixes it. + prefix, flasher uses this correctly in all but one place, this + patch fixes it. footer: Change-type: patch change-type: patch @@ -19987,7 +20630,8 @@ body: > On most device types rootA and rootB are partitions 2 and 3 - but with LUKS encryption and boot/EFI split they are shifted to 3 and 4 + but with LUKS encryption and boot/EFI split they are shifted to + 3 and 4 footer: Change-type: patch change-type: patch @@ -20013,7 +20657,8 @@ We are using two variants of GRUB configs - one for LUKS-encrypted OS - and the other one for the rest. HUP needs to acknowledge this and use + and the other one for the rest. HUP needs to acknowledge this + and use the correct one based on the system being updated. footer: @@ -20040,7 +20685,8 @@ On full disk encrypted devices the EFI partition is a soft link in the - boot partition. This commit fixes detecting files in the EFI partition + boot partition. This commit fixes detecting files in the EFI + partition from the boot partition. footer: @@ -20056,7 +20702,8 @@ On full disk encrypted devices the EFI partition is a soft link in the - boot partition. This commit fixes detecting files in the EFI partition + boot partition. This commit fixes detecting files in the EFI + partition from the boot partition. footer: @@ -20076,9 +20723,11 @@ |-sda2 8:2 0 42M 0 part | `-luks-a91cd125-9e4c-45e6-b3f4-1e9b4ec9e5b9 250:0 0 40M 0 crypt /mnt/boot - This commit allows extracting the physical device (sdaN) whic is needed + This commit allows extracting the physical device (sdaN) whic is + needed - to extract the partition index using sysfs both for luks or standard + to extract the partition index using sysfs both for luks or + standard devices. footer: @@ -20156,7 +20805,8 @@ container and the DUT does not allow to ssh as a non-root user. - Run ssh from the worker to test local SSH authentication with a cloud + Run ssh from the worker to test local SSH authentication with a + cloud user. footer: @@ -20172,14 +20822,17 @@ Given that testbot devices use a tunnel to specific ports to communicate - with the DUT that is established with the suite-generated keys, using + with the DUT that is established with the suite-generated keys, + using - a different keypair for the ssh-auth test would require to tear down and + a different keypair for the ssh-auth test would require to tear + down and re-establish the tunnel. - It's easier to just use the existing key pair in the ssh-auth test. + It's easier to just use the existing key pair in the ssh-auth + test. footer: Change-type: patch change-type: patch @@ -20193,7 +20846,8 @@ Using two set of keys, the one created by the suite to authenticate by - the proxy and a new custom key, is tricky as when running on testbot the + the proxy and a new custom key, is tricky as when running on + testbot the key is used to establish the tunnel between core and DUT. @@ -20305,12 +20959,14 @@ When adding a kernel configuration conditional in a provided kernel - version, make the check include the provided kernel version as that is + version, make the check include the provided kernel version as + that is the intuitive way to understand it. - The two places that use this function already used it in this way. + The two places that use this function already used it in this + way. footer: Change-type: patch change-type: patch @@ -20339,7 +20995,8 @@ body: > chrony 4.2 introduces security hardening in the - service definition that removes the CAP_SYS_ADMIN permission, affecting + service definition that removes the CAP_SYS_ADMIN permission, + affecting the way healthdog uses execve to become chronyd. @@ -20347,7 +21004,8 @@ commit 83f96efdfd2d (examples: harden systemd services) - This commits works around it by allowing all members of the service's + This commits works around it by allowing all members of the + service's control group to send notification messages. footer: @@ -20439,7 +21097,8 @@ Add the wireguard module by default so it is included in all device - types. This is a frequently requested by customers and will avoid having + types. This is a frequently requested by customers and will + avoid having to patch individual device repositories. footer: @@ -20471,12 +21130,15 @@ There are two sets of keys used in this test, one stored in `/root/id` - which is created by the cloud suite to SSH via the proxy server, and + which is created by the cloud suite to SSH via the proxy server, + and - a custom key stored in `/root/test_id` used in some of the subtests. + a custom key stored in `/root/test_id` used in some of the + subtests. - Fix the test cases using the custom key to use the correct private key. + Fix the test cases using the custom key to use the correct + private key. footer: Change-type: patch change-type: patch @@ -20564,7 +21226,8 @@ configuration and starting the `openvpn` service unit. - As the `openvpn` service units stops `os-config`, it might not get to + As the `openvpn` service units stops `os-config`, it might not + get to restart the supervisor. @@ -20632,12 +21295,15 @@ The sshd daemon is configured to fetch keys from the API for local - user connections. The script that fetches the keys, cloud-public-sshkeys, + user connections. The script that fetches the keys, + cloud-public-sshkeys, - sources balena-config-vars and is run as an exclusive non-root user. + sources balena-config-vars and is run as an exclusive non-root + user. - Let's set the correct permissions for this file to allow not to break + Let's set the correct permissions for this file to allow not to + break the above. @@ -20668,12 +21334,14 @@ database and files modified outside of the pseudo context [0]. - This will occasionally cause builds to fail in the do_deploy step of the + This will occasionally cause builds to fail in the do_deploy + step of the kernel-devsrc recipe. [1] - Fix this by not removing the kernel_source tarball in the do_deploy + Fix this by not removing the kernel_source tarball in the + do_deploy step. @@ -20723,7 +21391,8 @@ The old test no longer matches on full disk paths including /dev, which - can potentially result in the installation disk not being excluded from + can potentially result in the installation disk not being + excluded from the pool of installation targets. @@ -20744,9 +21413,11 @@ Previously, globs such as 'md/balena{,_*}' and 'mmcblk?' weren't being - properly expanded, resulting in the old behavior of explicit lists of + properly expanded, resulting in the old behavior of explicit + lists of - disks continuing to work, but consolidated globs matching multiple disks + disks continuing to work, but consolidated globs matching + multiple disks would not. @@ -20831,7 +21502,8 @@ Since kirkstone tasks have network access disabled by default so we need - to enable it explicitly for tasks that talk to the signing service. + to enable it explicitly for tasks that talk to the signing + service. footer: Change-type: patch change-type: patch @@ -21135,7 +21807,8 @@ Handle ENOENT ErrnoException when attempting to unwrap a non-flasher - image in HUP tests. This mirrors a similar change made in ce2d33ad8. + image in HUP tests. This mirrors a similar change made in + ce2d33ad8. footer: Change-type: patch change-type: patch @@ -21275,7 +21948,8 @@ ``` - ERROR: libical-2.0.0-r0 do_package: QA Issue: libical: Files/directories were installed but not shipped in any package: + ERROR: libical-2.0.0-r0 do_package: QA Issue: libical: + Files/directories were installed but not shipped in any package: /usr/lib/cmake @@ -21318,7 +21992,8 @@ body: > Newer versions fail on the configuration step with: - Requested 'libcrypto >= 1.1.0' but version of OpenSSL-libcrypto is 1.0.2o + Requested 'libcrypto >= 1.1.0' but version of OpenSSL-libcrypto + is 1.0.2o footer: Change-type: patch change-type: patch @@ -21516,14 +22191,17 @@ This config file hasn't been used since commit 2db88c2, which unified - how managed and unmanaged images operate. Since that commit, openvpn + how managed and unmanaged images operate. Since that commit, + openvpn - starts up if the config file at /etc/openvpn/openvpn.conf is found, and + starts up if the config file at /etc/openvpn/openvpn.conf is + found, and otherwise remains inactive. This file is populated by os-config. - Remove the old config to prevent misdirection and cleanup the layer. + Remove the old config to prevent misdirection and cleanup the + layer. footer: Change-type: patch change-type: patch @@ -21611,26 +22289,32 @@ Chronyd checks that the directory specified as `sourcedir` in `chrony.conf` - (in this case `/var/chrony`) is not world accessible if it exists (chrony + (in this case `/var/chrony`) is not world accessible if it + exists (chrony - will create it correctly if it does not exist), and does not start + will create it correctly if it does not exist), and does not + start if that's the case. - The way that the `/var/chrony` is created when it does not exist opens + The way that the `/var/chrony` is created when it does not exist + opens - the possibility of the directory existing with the wrong permissions and + the possibility of the directory existing with the wrong + permissions and hitting this problem. - This commit creates the directory with the correct permissions from the + This commit creates the directory with the correct permissions + from the start to avoid the race condition. - It also changes the permissiong from 750 to 770 to match what chrony + It also changes the permissiong from 750 to 770 to match what + chrony does (see @@ -21796,7 +22480,8 @@ hostOS updates between aufs and overlay2 balenaOS versions. - This commit adds support for 5.15 kernels and improves the branch + This commit adds support for 5.15 kernels and improves the + branch selection logic to cover some corner cases. @@ -21985,7 +22670,8 @@ body: > This fixes the following error when building mkfs-hostapp-native - with Honister for a Variscite iMX8MM which only has Hardknott support: + with Honister for a Variscite iMX8MM which only has Hardknott + support: mkfs-hostapp-native-1.0-r0 do_prepare_recipe_sysroot: @@ -22141,7 +22827,8 @@ Unfortunately the standalone balena-cli package is linked to glibc - and does not work with musl (alpine) so we need to switch to debian. + and does not work with musl (alpine) so we need to + switch to debian. The trade-off seems worth it for build times though. @@ -22158,7 +22845,8 @@ This version can likely be increased now that the balena-cli - is no longer part of the ndoe dependencies, but for now just + is no longer part of the ndoe dependencies, but for now + just publish the current setting. footer: @@ -22205,7 +22893,8 @@ If a block device specified in resin-init-flasher.conf is part of an - array, but that assembled array name wasn't specified, skip it to avoid + array, but that assembled array name wasn't specified, skip it + to avoid data loss. footer: @@ -22221,18 +22910,23 @@ Instead of querying devices w/ `fdisk -l`, glob match patterns specified - in resin-init-flasher.conf with devices present in `/dev`. This allows us to + in resin-init-flasher.conf with devices present in `/dev`. This + allows us to - specify devices like `hd? sd? mmcblk?` instead of individual device + specify devices like `hd? sd? mmcblk?` instead of individual + device numbers, which don't consistently map to any particular disk. - This also allows RAID arrays to be matched with the array name and a + This also allows RAID arrays to be matched with the array name + and a - pattern that glob matches even arrays assembled automatically on a + pattern that glob matches even arrays assembled automatically on + a - non-matching host, such as `md/balena?(_?)` matching an array named + non-matching host, such as `md/balena?(_?)` matching an array + named `balena` and assembled on-device at `/dev/md/balena_0`. footer: @@ -22367,7 +23061,8 @@ Before kirkstone, the way to not include the kernel image was to - override the `RDEPENDS:${KERNEL_PACKAGE_NAME}-base` not to include + override the `RDEPENDS:${KERNEL_PACKAGE_NAME}-base` not to + include `kernel-image`, as was done in the `kernel-resin-noimage` class. @@ -22377,7 +23072,8 @@ Poky's commit f6d963fa6d0e64d53f7ef56fd2c12d67f5811829 - Now excluding the kernel image needs to `PACKAGE_EXCLUDE = "kernel-image-*"` + Now excluding the kernel image needs to `PACKAGE_EXCLUDE = + "kernel-image-*"` footer: Change-type: patch change-type: patch @@ -22477,7 +23173,9 @@ Yocto kirkstone complains with: - ERROR: packagegroup-resin-1.0-r1 do_package_write_ipk: An allarch packagegroup shouldn't depend on packages which are dynamically renamed (libnss-ato to libnss-ato2) + ERROR: packagegroup-resin-1.0-r1 do_package_write_ipk: An + allarch packagegroup shouldn't depend on packages which are + dynamically renamed (libnss-ato to libnss-ato2) For lack of a better place, move to the balena-image recipe. @@ -22832,7 +23530,8 @@ Run the resin-update-state rules that create the by-state links after md - arrays are assembled. This fixes state link creation when running on a + arrays are assembled. This fixes state link creation when + running on a RAID array. footer: @@ -22914,7 +23613,8 @@ The latest meta-balena includes `util-linux-findmnt` as a kexec module - dependency and this package has not yet been split from `util-linux` in + dependency and this package has not yet been split from + `util-linux` in thud. footer: @@ -23009,9 +23709,11 @@ In order to use the same rust toolchain across all supported Yocto - versions this commit updates the cmake version on all integration layers + versions this commit updates the cmake version on all + integration layers - below Zeus to 3.13.4, which is the minimum version to compile the rust + below Zeus to 3.13.4, which is the minimum version to compile + the rust 1.62 toolchain. @@ -23048,7 +23750,8 @@ it was living in meta-rust. - We want to use the balena-rust layer across a wide variety of Yocto + We want to use the balena-rust layer across a wide variety of + Yocto versions so include the fetcher conditionally. footer: @@ -23068,11 +23771,14 @@ systems and is not present in older Yocto releases. - This commit reverts to the previous way of setting the rust architecture. + This commit reverts to the previous way of setting the rust + architecture. - It will not work for ppc64le and if we would need to support such an + It will not work for ppc64le and if we would need to support + such an - architecture the arch_to_rust_arch() function will still be called if it + architecture the arch_to_rust_arch() function will still be + called if it exists in Kirkstone or newer Yocto versions. footer: @@ -23089,14 +23795,17 @@ With Kirkstone and the support of openSSL 3.0 it's not possible to find - a set of dependencies that work for all of our rust applications across + a set of dependencies that work for all of our rust applications + across - the 1.32 to 1.62 toolchain versions that are supported across all the + the 1.32 to 1.62 toolchain versions that are supported across + all the Yocto versions we keep compatibility with. - This layer allows to set a preferred version as a distro setting that can + This layer allows to set a preferred version as a distro setting + that can be used across all Yocto versions. @@ -23357,18 +24066,23 @@ container. This had two downsides: - 1. It was relatively heavyweight. In devices under heavy load, it would + 1. It was relatively heavyweight. In devices under heavy load, + it would sometimes take so long to run that the Engine was killed by the watchdog. - 2. It wrote to the storage media. Creating a container involves writing + 2. It wrote to the storage media. Creating a container involves + writing some data to persistent storage, therefore the healthcheck was wearing the storage media. - This new healthcheck simply pings both `balenad` and `containerd`, which + This new healthcheck simply pings both `balenad` and + `containerd`, which - is much faster than starting a new container and doesn't write to disk. + is much faster than starting a new container and doesn't write + to disk. - The step of pinging `containerd` is important because we have seen at + The step of pinging `containerd` is important because we have + seen at least one case in the past in which `balenad` was working but @@ -23386,11 +24100,13 @@ With `WatchdogSignal=SIGTERM` systemd will send a SIGTERM and give the - Engine 90 seconds to gracefully shutdown before sending a SIGKILL. We + Engine 90 seconds to gracefully shutdown before sending a + SIGKILL. We had cases of Engine metadata on disk getting corrupted after the - watchdog sent it a SIGKILL directly. This change shall minimize this + watchdog sent it a SIGKILL directly. This change shall minimize + this issue. footer: @@ -23464,7 +24180,8 @@ Recent versions of meta-balena include a balena-rust layer used to - specify a distro-set Rust version across all supported Yocto versions + specify a distro-set Rust version across all supported Yocto + versions As such, the syntax of this layer also needs to be converted. @@ -23536,7 +24253,8 @@ This is a temporary way that will allow to update the bblayers.conf of - device type repositories to include meta-balena/meta-balena-rust while + device type repositories to include meta-balena/meta-balena-rust + while still building. @@ -23655,7 +24373,8 @@ Our tests perform two HUPs (into and out of the release under test), and - the code for both of these HUPs were duplicated. This commit factors + the code for both of these HUPs were duplicated. This commit + factors this code out to a common function. footer: @@ -23671,7 +24390,8 @@ This is a small improvement over our previous test: in addition to - checking that the volumes themselves are preserved over HUPs, we now + checking that the volumes themselves are preserved over HUPs, we + now check if the contents of these volumes is preserved. @@ -23877,9 +24597,11 @@ The boot partition is currently a FAT filesystem that does not support - atomic writes. To prevent corruption, this commit introduces a fatrw + atomic writes. To prevent corruption, this commit introduces a + fatrw - application that needs to be used both when reading and writing files + application that needs to be used both when reading and writing + files to the boot partition to provide safe accesses. footer: @@ -23949,7 +24671,8 @@ Including the 2min systemd watchdog timer, plus 60 attempts to sync - the time via chronyc waitsync, it may take longer than expected to + the time via chronyc waitsync, it may take longer than expected + to trigger the healthcheck condition. footer: @@ -24255,7 +24978,8 @@ 87a741fd22a78c190bec59fa6628de921ac2809f. - This change didn't actually help to resolve the original ETIMEDOUT + This change didn't actually help to resolve the original + ETIMEDOUT issues so it can be reverted. footer: @@ -24453,13 +25177,17 @@ The executeCommand family of methods default to retrying on failure. In - some cases, such as in the ssh-auth test in the cloud test suite, we + some cases, such as in the ssh-auth test in the cloud + test suite, we - expect failures to happen, and want them to be raised immediately. Other + expect failures to happen, and want them to be raised + immediately. Other - situations might demand adjusting the number of retries and interval to + situations might demand adjusting the number of retries + and interval to - fit specific tests. Add a retryOptions object to these methods to allow + fit specific tests. Add a retryOptions object to these + methods to allow for this behavior to be configured. footer: @@ -24482,7 +25210,8 @@ This test was broken previously, and would fail with "All configured - authentication methods failed" after a long delay caused by excessive + authentication methods failed" after a long delay caused by + excessive retries. @@ -24515,9 +25244,11 @@ By creating empty configuration units when no configuratin is applied to - a service instead we avoid regenerating them at boot if stored in + a service instead we avoid regenerating them at boot if stored + in - persistent memory helping with boot times in less powerful devices. + persistent memory helping with boot times in less powerful + devices. footer: Change-type: patch change-type: patch @@ -24531,12 +25262,14 @@ This allows for specific devices to override the defaults, which is to - store configuration units on volatile memory re-generating them every + store configuration units on volatile memory re-generating them + every boot and avoiding storage media writes. - Changing this to permanent storage trades boot times with media life + Changing this to permanent storage trades boot times with media + life expectancy. footer: @@ -24590,14 +25323,16 @@ Currently `os-config-json` is also parsing `units-conf.json` at runtime. - This is expensive to do for smaller devices, so this commit performs the + This is expensive to do for smaller devices, so this commit + performs the processing at build time. For this, it also splits the configuration units processing in - balena-config-vars into its own recipe to allow for task ordering. + balena-config-vars into its own recipe to allow for task + ordering. footer: Change-type: patch change-type: patch @@ -24623,7 +25358,8 @@ This makes it more general and allows it to be expanded. - Also, allow its native use so it can be included in recipe tests. + Also, allow its native use so it can be included in recipe + tests. footer: Change-type: patch change-type: patch @@ -24677,9 +25413,11 @@ example a RaspberryPi Zero. - This commit introduces a cached memory file with the configuration + This commit introduces a cached memory file with the + configuration - environment that is recreated when `config.json` changes and will be + environment that is recreated when `config.json` changes and + will be used if present. @@ -24892,12 +25630,14 @@ body: > When parallelizing fingerprint checks with Promise.any(), the - unsuccessful command would continue retrying in the background, causing + unsuccessful command would continue retrying in the background, + causing spurious error messages. - With mDNS resolution memoization, this optimization no longer saves us + With mDNS resolution memoization, this optimization no longer + saves us time, so remove it. footer: @@ -25034,7 +25774,8 @@ BOOT_MOUNTPOINT is used in 5. - This commit replaces BOOT_MOUNTPOINT with BALENA_BOOT_MOUNTPOINT to + This commit replaces BOOT_MOUNTPOINT with BALENA_BOOT_MOUNTPOINT + to remove the duplication. footer: @@ -25069,16 +25810,20 @@ When accessing a test device as part of a fleet, a cloud API key is - required in order to generate an SSH key and access the device through + required in order to generate an SSH key and access the device + through - the VPN. However, when accessing a device locally, such as a QEMU + the VPN. However, when accessing a device locally, such as a + QEMU - instance running on the workstation itself, we have a direct path, and + instance running on the workstation itself, we have a direct + path, and no VPN is necessary. - Make the apiKey optional, and don't login when it's not specified. This + Make the apiKey optional, and don't login when it's not + specified. This allows direct connections to work without it. footer: @@ -25144,7 +25889,8 @@ Reduce the interval between scans as well as the maximum number of scans - for modems, reducing the time spent waiting when no modem is present + for modems, reducing the time spent waiting when no modem is + present from ~50s to ~5s. footer: @@ -25173,11 +25919,14 @@ worker.rebootDut() contains retry logic using utils.waitUntil() wrapping - this.executeCommandInHostOS(). The latter contains its own retry logic, + this.executeCommandInHostOS(). The latter contains its + own retry logic, - which will try to execute a given command for up to five minutes before + which will try to execute a given command for up to five + minutes before - timing out. Remove the retry logic from worker.rebootDut(), as it's + timing out. Remove the retry logic from + worker.rebootDut(), as it's redundant and adds latency. footer: @@ -25193,9 +25942,11 @@ The default interval for retrying this command is 5s, with a total - timeout of 2m30s. Reduce the timeout to 1s to reduce latency for the + timeout of 2m30s. Reduce the timeout to 1s to reduce + latency for the - successful case, while increasing the total timeout to 5m. + successful case, while increasing the total timeout to + 5m. footer: Change-type: patch change-type: patch @@ -25237,9 +25988,11 @@ Certain chrony tests require the ability to block NTP requests. Switch - from blocking these requests using iptables rules to simply stopping the + from blocking these requests using iptables rules to simply + stopping the - local DNS server, which is faster and simpler, and doesn't conflict with + local DNS server, which is faster and simpler, and doesn't + conflict with the supervisor firewall. footer: @@ -25319,7 +26072,8 @@ In conclusion, we rework how the blacklist is constructed - so that users of meta-balena can alter this list as they see fit. + so that users of meta-balena can alter this list as they see + fit. footer: Change-type: patch change-type: patch @@ -25387,24 +26141,32 @@ This changes the condition in the unit file from checking whether - /dev/disk/by-state/balena-efi exists to checking whether /mnt/boot/EFI + /dev/disk/by-state/balena-efi exists to checking whether + /mnt/boot/EFI - is a symlink. The original approach has a race condition populating + is a symlink. The original approach has a race condition + populating - the by-state symlink - it is depending on udev and if the link is + the by-state symlink - it is depending on udev and if the link + is - not present when the service is started (after the boot partition is mounted), + not present when the service is started (after the boot + partition is mounted), the service fails and the EFI partition is never mounted. - The new approach does the trick pretty well - /mnt/boot/EFI is a symlink + The new approach does the trick pretty well - /mnt/boot/EFI is a + symlink - if the EFI partition is split and a regular directory in case there is a single + if the EFI partition is split and a regular directory in case + there is a single - boot partition. That said the service is only started when necessary + boot partition. That said the service is only started when + necessary - and the waiting for udev is implemented as a part of the mount script. + and the waiting for udev is implemented as a part of the mount + script. footer: Change-type: patch change-type: patch @@ -25422,7 +26184,8 @@ Recent versions of GRUB default to use shim_lock when in secure boot mode. - We do not use shim and do not build the shim_lock module into GRUB EFI binary + We do not use shim and do not build the shim_lock module into + GRUB EFI binary therefore this needs to be disabled. footer: @@ -25463,9 +26226,11 @@ Create a directConnect variable that indicates whether we're connecting - to a local instance of the worker server, including if the connection is + to a local instance of the worker server, including if + the connection is - over a unix domain socket. This allows the suite to skip steps that + over a unix domain socket. This allows the suite to skip + steps that don't pertain to local runs. footer: @@ -25632,9 +26397,11 @@ and the system time does not skew. - The healthcheck will command a burst sync if there is no selected + The healthcheck will command a burst sync if there is no + selected - reachable source, and will restart chronyd if the system clock skews. + reachable source, and will restart chronyd if the system clock + skews. Fixes #2314 @@ -25654,7 +26421,8 @@ body: > Test context is now accessible from self, remove verbose - this.context.get() syntax when calling worker.executeCommandInHostOS. + this.context.get() syntax when calling + worker.executeCommandInHostOS. footer: Change-type: patch change-type: patch @@ -25668,7 +26436,8 @@ When calling waitUntil(), reduce calling intervals and the total number - of retries. This effectively halves the runtime for these tests, as the + of retries. This effectively halves the runtime for these tests, + as the latency for detecting success is much lower. footer: @@ -25754,7 +26523,8 @@ The resin-img is no longer maintained and the deployment of raw images - as well as flasher requires features only available in balena-img. + as well as flasher requires features only available in + balena-img. footer: Change-type: patch change-type: patch @@ -25807,7 +26577,8 @@ As part of rebranding, resin docker repos were renamed to balena, and - resin/resin-img no longer receives updates. Change the image we pull to + resin/resin-img no longer receives updates. Change the image we + pull to process OS images from resin/resin-img to balena/balena-img. footer: @@ -25975,7 +26746,8 @@ When the promise called in waitUntil fails, the function defaults to a - 30s interval before trying again. Reduce this to a 5s interval w/ + 30s interval before trying again. Reduce this to a 5s interval + w/ maximum 5m retry window in the cloud suite. footer: @@ -26088,7 +26860,8 @@ utils.waitUntil expects rejectionFail before _times and _delay, include - this argument to ensure the later arguments have the intended effect + this argument to ensure the later arguments have the intended + effect footer: Change-type: patch change-type: patch @@ -26129,7 +26902,8 @@ Use systemd.waitForServiceState instead of waitUntil. This improves - readability, and reduces the time taken in the case that the test fails + readability, and reduces the time taken in the case that the + test fails and the default interval of waitUntil causes an excessive wait. footer: @@ -26225,7 +26999,8 @@ Log statements for powerOn/powerOff were added for debugging when - refactoring the QEMU worker to operate w/out libvirt. Remove these. + refactoring the QEMU worker to operate w/out libvirt. + Remove these. footer: Change-type: patch change-type: patch @@ -26257,7 +27032,8 @@ Device addresses likely won't change during a single test run, and mDNS - queries can be time consuming. Memoize the result of the query to save + queries can be time consuming. Memoize the result of the + query to save some time. footer: @@ -26304,7 +27080,8 @@ When we patch an ESR branch, for example from v2022.1.0 to v2022.1.1, - do not update the next, current, sunset ESR phases as they remain the + do not update the next, current, sunset ESR phases as they + remain the same. footer: @@ -26332,7 +27109,8 @@ When we patch an ESR branch, for example from v2022.1.0 to v2022.1.1, - do not update the next, current, sunset ESR phases as they remain the + do not update the next, current, sunset ESR phases as they + remain the same. footer: @@ -26443,9 +27221,11 @@ When tests execute quicker, a race condition can occur where config.json - is edited to remove the dnsServers property, but /run/dnsmasq.servers is + is edited to remove the dnsServers property, but + /run/dnsmasq.servers is - not changed yet. This causes the test to fail, as the file is not empty. + not changed yet. This causes the test to fail, as the file is + not empty. not ok 1 - We should have an empty /run/dnsmasq.servers file. --- @@ -26459,7 +27239,8 @@ -/^\s?$/ +"server=1.1.1.1\nserver=1.1.1.1" - Fix this by waiting until the InvocationID of dnsmasq.service changes. + Fix this by waiting until the InvocationID of dnsmasq.service + changes. footer: Change-type: patch change-type: patch @@ -26540,18 +27321,23 @@ The root cause of this issue was an integer overflow in 32-bit platforms - -- specifically, when explicitly converting `io.Reader` offsets + -- specifically, when explicitly converting `io.Reader` + offsets - (`int64`) to slice indices (`int`, which is only 32-bit long on 32-bit + (`int64`) to slice indices (`int`, which is only 32-bit + long on 32-bit platforms). - We already had in place the checks supposed to ensure slice bounds where + We already had in place the checks supposed to ensure + slice bounds where - always within the expected bounds. So, in a way, this commit just + always within the expected bounds. So, in a way, this + commit just - re-organizes the code so that we perform safely this required `int64` to + re-organizes the code so that we perform safely this + required `int64` to `int` type conversion. footer: @@ -26594,7 +27380,8 @@ This is used by the OS builders to deploy releases. This contract contains - details related to the balena-image artifact generated in the balenaOS + details related to the balena-image artifact generated in the + balenaOS build. footer: @@ -26807,7 +27594,8 @@ When KVM is not used we can easily emulate generic-aarch64 - on x86_64 hosts so avoid tying the host and target architectures + on x86_64 hosts so avoid tying the host and target + architectures together in the Makefile. footer: @@ -26966,7 +27754,8 @@ body: > Add a missing asterisk to FILES for iwlwifi-quz-a0-hr-b0 - and iwlwifi-quz-a0-jf-b0 to include compressed firmware files. Without it + and iwlwifi-quz-a0-jf-b0 to include compressed firmware files. + Without it the packages end up empty as we have compression enabled. footer: @@ -27200,7 +27989,8 @@ Splash image backend would throw if the image is not a valid png during - the write step. This could prevent the device from provisioning if some + the write step. This could prevent the device from + provisioning if some corruption happens at some point. footer: @@ -27218,12 +28008,14 @@ This commit updates all backends that write to /mnt/boot to do it - through a new `lib/host-utils` module. Writes are now done using write + + through a new `lib/host-utils` module. Writes are now + done using write + sync as rename is not an atomic operation in vfat. - The change also applies for writes through the `/v1/host-config` + The change also applies for writes through the + `/v1/host-config` endpoint. @@ -27391,10 +28183,12 @@ The status field does not indicate the current state of the Engine on the device. - Rather, it only indicates whether the HTTP response returned successfully or not. + Rather, it only indicates whether the HTTP response + returned successfully or not. - To get the current state of the device, i.e. whether it has applied target state or not, + To get the current state of the device, i.e. whether it + has applied target state or not, use the `appState` field in the response. footer: @@ -27426,18 +28220,24 @@ The linked issue describes the Supervisor not cleaning up locks it creates due - to crashing at just the wrong time. After internal discussion we decided to + to crashing at just the wrong time. After internal + discussion we decided to - differentiate Supervisor-created lockfiles from user-created lockfiles by using + differentiate Supervisor-created lockfiles from + user-created lockfiles by using - the `nobody` UID (65534) for Supervisor-created lockfiles. + the `nobody` UID (65534) for Supervisor-created + lockfiles. - As the existing NPM lockfile lib does not allow creating lockfiles atomically + As the existing NPM lockfile lib does not allow creating + lockfiles atomically - with different UIDs, we move to using the lockfile binary, which is part of the + with different UIDs, we move to using the lockfile + binary, which is part of the - procmail package. To allow nonroot users to write to lock directories, permissions + procmail package. To allow nonroot users to write to + lock directories, permissions are changed to allow write access by nonroot users. footer: @@ -27498,19 +28298,22 @@ Migration `M00008` had a bug with the check for legacy apps, which - resulted in devices that had at some point been updated from a single + resulted in devices that had at some point been updated + from a single container supervisor to get the error ``` - Undefined binding(s) detected when compiling UPDATE. Undefined column(s): [appUuid] query + Undefined binding(s) detected when compiling UPDATE. + Undefined column(s): [appUuid] query ``` - This adds a new migration with the fix to ensure broken fix the + This adds a new migration with the fix to ensure broken + fix the inconsistent database state. footer: @@ -27530,12 +28333,14 @@ This change updates types and database format in order to allow - receiving the new format of the target state from the cloud and allow + receiving the new format of the target state from the + cloud and allow applications to keep working. - This change also updates metadata in the containers, meaning services + This change also updates metadata in the containers, + meaning services will need to be restarted on supervisor update footer: @@ -27552,7 +28357,8 @@ body: > It seems that in some cases the supervisor can report - an image without a `status` field leading to a cloud side 401 response. + an image without a `status` field leading to a cloud + side 401 response. See #1905 for more details. footer: @@ -27876,7 +28682,8 @@ sets the system time to the incorrect rtc time - as soon as timesync-https finished running and setting the correct date + as soon as timesync-https finished running and setting the + correct date from the servers. footer: @@ -27908,23 +28715,29 @@ This reverts 5047757 where we set the global DHCP timeout for ipv4 to - infinity to mitigate problems with routers that do not respond to DHCP + infinity to mitigate problems with routers that do not respond + to DHCP requests in time. - This however causes issues on some routers, which after power cycle fail to + This however causes issues on some routers, which after power + cycle fail to renew leases after they are expired. - The same ipv4.dhcp-timeout can be specified on per connection basis. Since + The same ipv4.dhcp-timeout can be specified on per connection + basis. Since - the case with lease renewal due to router power cycle is a more common case, + the case with lease renewal due to router power cycle is a more + common case, - it is better to keep the global dhcp-timeout default unmodified. If the + it is better to keep the global dhcp-timeout default unmodified. + If the - setting needs adjustment then it can be overwritten in the particular + setting needs adjustment then it can be overwritten in the + particular connection profile. footer: @@ -27985,21 +28798,26 @@ There are known situations in which balenaEngine times out during - initialization (for example, during aufs to overlayfs migrations, or + initialization (for example, during aufs to overlayfs + migrations, or - when restarting a device that was running a large number of containers). + when restarting a device that was running a large number of + containers). - When these time outs occur, Systemd kills the Engine, causing further + When these time outs occur, Systemd kills the Engine, causing + further problems. - To avoid these cases, this commit disables timeouts during the Engine + To avoid these cases, this commit disables timeouts during the + Engine initialization. - This is also aligned with the default Systemd settings distributed with + This is also aligned with the default Systemd settings + distributed with the Moby project. footer: @@ -28073,7 +28891,8 @@ development mode. - A managed device is always accessible via the configured custom keys. + A managed device is always accessible via the configured custom + keys. footer: Change-type: patch change-type: patch @@ -28088,7 +28907,8 @@ The behaviour of the SSH connection depends on whether custom keys are - present or not. This commit calls out to generate the development mode + present or not. This commit calls out to generate the + development mode configuration file on ssh key change. @@ -28177,7 +28997,8 @@ body: > * Fixes #2569 - * ensure OpenVPN client always starts with the latest CA certificate + * ensure OpenVPN client always starts with the latest CA + certificate from API config endpoint as this certificate may have changed and we don't want VPN to be down for ~24 hours until os-config is triggered by systemd timer @@ -28225,9 +29046,11 @@ We currently do not log any information to help us understanding the - underlying issue -- not even to identify what is the exact point in + underlying issue -- not even to identify what is the + exact point in - which the error is raised. This commit improves on this situation. + which the error is raised. This commit improves on this + situation. footer: Signed-off-by: Leandro Motta Barros signed-off-by: Leandro Motta Barros @@ -28245,15 +29068,19 @@ This adds two new integration tests: - * TestDeltaSize: this is meant to catch regressions on delta sizes. It + * TestDeltaSize: this is meant to catch regressions on + delta sizes. It generates deltas and compare their sizes with the delta sizes we get as of now. If the size increases, the test fails. - * TestDeltaCorrectness: checks if applying a delta indeed results in the + * TestDeltaCorrectness: checks if applying a delta + indeed results in the same image as we had originally. - A number of different test cases (different images with distinct + A number of different test cases (different images with + distinct - features) are tested for each of these integration tests. + features) are tested for each of these integration + tests. footer: Signed-off-by: Leandro Motta Barros signed-off-by: Leandro Motta Barros @@ -28378,9 +29205,11 @@ Starting with v2.91.6 device provisioning will not start if the device - was unable to register in the cloud due to lack of internet connectivity. + was unable to register in the cloud due to lack of internet + connectivity. - To avoid this, we switch to retrying 3 times with the usual 2 second delay + To avoid this, we switch to retrying 3 times with the usual 2 + second delay between attempts, after which the flashing process will start. footer: @@ -28483,22 +29312,29 @@ This was originally introduced in combination with a kernel patch backported - from Red Hat kernel that would enable kernel lockdown when secure boot + from Red Hat kernel that would enable kernel lockdown when + secure boot - is enabled. We have since changed the approach, dropped the kernel patch + is enabled. We have since changed the approach, dropped the + kernel patch - and when in secure boot mode use a different GRUB config file that enables + and when in secure boot mode use a different GRUB config file + that enables lockdown on kernel command line unconditionally. - That said, while the patch works fine, we do not really need it and it adds + That said, while the patch works fine, we do not really need it + and it adds - extra overhead porting it to newer yocto versions so there is no point keeping + extra overhead porting it to newer yocto versions so there is no + point keeping - it at this moment. If in the future there is need for the kernel to know + it at this moment. If in the future there is need for the kernel + to know - whether it is in secure boot mode or not, we can roll this back and rebase. + whether it is in secure boot mode or not, we can roll this back + and rebase. footer: Change-type: patch change-type: patch @@ -28520,7 +29356,8 @@ is on the same partition. - This uses a data mount to ensure temporary extracted files of the + This uses a data mount to ensure temporary extracted files of + the compressed image will not fill the target sysroot. footer: @@ -28582,7 +29419,8 @@ preventing the device from booting. - Should this happen, we use sgdisk to check and recover the end gpt + Should this happen, we use sgdisk to check and recover the end + gpt from the main one. footer: @@ -28603,9 +29441,11 @@ A legacy development image will update to development mode enabled - independently of whether the newOS is configured for development mode or + independently of whether the newOS is configured for development + mode or - not. The only case when a hostapp has developmentMode set is when locally + not. The only case when a hostapp has developmentMode set is + when locally building with `OS_DEVELOPMENT`. footer: @@ -28625,9 +29465,11 @@ body: > This caters for the use case of custom device types that are not - registered in balena-cloud but still need to fetch the supervisor from + registered in balena-cloud but still need to fetch the + supervisor from - balena-cloud's registry by querying the `supervisor_version` endpoint, + balena-cloud's registry by querying the `supervisor_version` + endpoint, for example when using openBalena. footer: @@ -28646,9 +29488,11 @@ by the API. - Providing the image name in the command line provides an update path + Providing the image name in the command line provides an update + path - for unmanaged devices, manual updates on ESR devices (which currently + for unmanaged devices, manual updates on ESR devices (which + currently do not allow dashboard based updates) and openBalena use cases. footer: @@ -28672,7 +29516,8 @@ 4.9, the test will not pass. - Let's run this test on kernel versions where the issue was present. + Let's run this test on kernel versions where the issue was + present. footer: Change-type: patch change-type: patch @@ -28869,12 +29714,15 @@ is not generated for two reasons: - - 60-resin-update-state.rules only react to resin-* partition names + - 60-resin-update-state.rules only react to resin-* partition + names - - the root device is not identified correctly in resin_update_state_probe + - the root device is not identified correctly in + resin_update_state_probe for LUKS devices - This patch fixes both the issues and makes use of the by-state symlink + This patch fixes both the issues and makes use of the by-state + symlink instead of by-label. footer: @@ -28912,19 +29760,25 @@ - 3 - Expansion card firmware configs - The full protection is only applied on first boot after provisioning, + The full protection is only applied on first boot after + provisioning, flasher only locks against PCRs 0, 2 and 3. - This is because when using flasher, the environment is not in the same + This is because when using flasher, the environment is not in + the same - state as the one we want to lock to. In particular the boot order is + state as the one we want to lock to. In particular the boot + order is - different (flasher is booted from a different drive than the resulting OS). + different (flasher is booted from a different drive than the + resulting OS). - As for now we were not able to find a better solution than fully locking + As for now we were not able to find a better solution than fully + locking - only on first boot. This means the device must be booted in a secure + only on first boot. This means the device must be booted in a + secure environment at least once after provisioning. footer: @@ -28956,9 +29810,11 @@ jq returns null by default when a given key isn't found, ensure that - when getting the value of deployRawArtifact, we get an empty variable + when getting the value of deployRawArtifact, we get an empty + variable - instead, which is checked later on to determine if that file should be + instead, which is checked later on to determine if that file + should be deployed footer: @@ -29017,15 +29873,18 @@ * the CLI prompts for input during preload - Alternatively, the --pin-device-to-release flag may be used to pin only the + Alternatively, the --pin-device-to-release flag may be used to + pin only the preloaded device to the selected release. - Would you like to disable automatic updates for this fleet now? No + Would you like to disable automatic updates for this fleet now? + No - * we do not want to set the suggested flag and we do not want to touch the fleet release policy for this use case + * we do not want to set the suggested flag and we do not want to + touch the fleet release policy for this use case footer: Change-type: patch change-type: patch @@ -29085,7 +29944,8 @@ Surface the preloaded app commit as a variable that can be overridden in - the build job. Default to "current" to maintain existing behavior when + the build job. Default to "current" to maintain existing + behavior when the variable isn't set. footer: @@ -29257,7 +30117,8 @@ Update balena-supervisor from 12.11.32 to 12.11.36 - Includes a fix for https://github.com/balena-os/balena-supervisor/issues/1890 + Includes a fix for + https://github.com/balena-os/balena-supervisor/issues/1890 footer: Change-type: patch change-type: patch @@ -29274,13 +30135,17 @@ The moby engine v20.x.y adds some selinux [security configurations](https://docs.docker.com/engine/reference/run/#security-configuration) - depending on the [container configuration](https://github.com/moby/moby/blob/master/daemon/create.go#L214). + depending on the [container + configuration](https://github.com/moby/moby/blob/master/daemon/create.go#L214). - This would cause the supervisor to enter a service restart loop as the + This would cause the supervisor to enter a service + restart loop as the - current and target service configurations will never match. The + current and target service configurations will never + match. The - supervisor now ignores selinux specific security options since those are + supervisor now ignores selinux specific security options + since those are not supported by balenaOS. footer: @@ -29324,7 +30189,8 @@ This is necessary with the changes as of balenaOS 2.82.6, which watches config.json - and will restart balena-hostname and some other services automatically on file change. + and will restart balena-hostname and some other services + automatically on file change. footer: Change-type: patch change-type: patch @@ -29344,7 +30210,8 @@ With more and more devices in ipv6 only networks, this ensures the - local addresses are reported to the cloud as part of the state patch. + local addresses are reported to the cloud as part of the + state patch. footer: Change-type: patch change-type: patch @@ -29408,7 +30275,8 @@ the only hard error is if rollback (failcleanup) fails, in all other - scenarios we want the daemon to continue starting with the new + scenarios we want the daemon to continue starting with + the new graphdriver footer: @@ -29424,12 +30292,14 @@ previously switch would treat S_IFIFO and S_IFSOCK as the same, passing - both of the to mkfifo, which lead to EINVAL errors when trying to create + both of the to mkfifo, which lead to EINVAL errors when + trying to create the socket, we instead handle socket separately. - Also adds cases for this to the unit and integration tests of the + Also adds cases for this to the unit and integration + tests of the migrator. footer: @@ -29469,7 +30339,9 @@ body: > See https://github.com/containerd/containerd/pull/4530 - and `git log ad25c1a9c34361e4071f508b9a91946b05fce165^..2055e12953bb538228d8d9fe627fa545d7cf82be ./platforms/` + and `git log + ad25c1a9c34361e4071f508b9a91946b05fce165^..2055e12953bb538228d8d9fe627fa545d7cf82be + ./platforms/` in the containerd repo footer: @@ -29485,7 +30357,8 @@ reorder the defer statements in the migrate function to only teardown - the logger after the failcleanup function ran. otherwise errors logged + the logger after the failcleanup function ran. otherwise + errors logged there won't show up in the logfile footer: @@ -29501,9 +30374,11 @@ This brings all migration logic into a single call into the - storagemigration package, which should make future maintenance easier + storagemigration package, which should make future + maintenance easier - and fixes the cleanup logic bug, where the old aufs root would never be + and fixes the cleanup logic bug, where the old aufs root + would never be cleaned up. footer: @@ -29527,7 +30402,8 @@ This commit changes the way we retry layer downloads after failures with - the goal of making it more resilient, especially for cases involving + the goal of making it more resilient, especially for + cases involving large layers and unreliable network connections. @@ -29535,15 +30411,18 @@ These are the changes: - * Make sure we also retry after failures in `v2LayerDescriptor.reset()`. + * Make sure we also retry after failures in + `v2LayerDescriptor.reset()`. This method creates a new HTTP request to resume a failed download, and therefore depends on a working network to succeed. - * Wait exponentially longer times between retries (instead of retrying + * Wait exponentially longer times between retries + (instead of retrying immediately as before). This shall increase of success in case of network issues that take longer to get resolved. * Increase the number of retries to 10. - * Reset retry count whenever we successfully download anything at all. + * Reset retry count whenever we successfully download + anything at all. The idea is that we want to give up downloading only after a long continuous period of failures. Combined with the exponential back-off strategy and increased number of retries described above, a layer pull @@ -29581,7 +30460,8 @@ body: > This can be used to keep a record of failed migrations. - Only runs if BALENA_MIGRATE_OVERLAY_LOGFILE is set to a path on disk. + Only runs if BALENA_MIGRATE_OVERLAY_LOGFILE is set to a + path on disk. The log file will be deleted if there are no errors. footer: @@ -29597,7 +30477,8 @@ With this change the aufs data is kept around until the next time we - start. If we find both an aufs AND an overlay2 storage root, we cleanup + start. If we find both an aufs AND an overlay2 storage + root, we cleanup the aufs data. footer: @@ -29613,13 +30494,16 @@ During fingerpinting of the source image the destination layers are not - exepmt from being released (e.g. when `balena image rm `) is run + exepmt from being released (e.g. when `balena image rm + `) is run simultaneously. - Similarly when processing the destination layers to generate deltas we + Similarly when processing the destination layers to + generate deltas we - only hold one reference at a time, leaving the subsequent layers + only hold one reference at a time, leaving the + subsequent layers vulnerable to the same issues. footer: @@ -29680,9 +30564,11 @@ Busybox in balenaOS is compiled with desktop mode disabled, - so features like `-ef` and providing pids via `-q` are not + so features like `-ef` and providing pids via `-q` are + not - supported. Add a 3rd condition to try ps with no args and allow + supported. Add a 3rd condition to try ps with no args + and allow parsePSOutput to filter by pid. @@ -29743,7 +30629,8 @@ body: > https://github.com/balena-os/balena-engine-cli/commit/20c19830a95455e8562551aad52c715ad0807cc6 - moves the versioning variables to a separate package. We have to adjust + moves the versioning variables to a separate package. We + have to adjust the location in hack/make.sh too footer: @@ -29819,12 +30706,15 @@ Earlier engine versions were not properly persisting cacheID - in layer metadata. As a result, because of abruptly terminated transactions, + in layer metadata. As a result, because of abruptly + terminated transactions, - a lot of devices have unreferenced graphdriver layers on disk. + a lot of devices have unreferenced graphdriver layers on + disk. - With this change, the engine will be able to clean up such unreferenced layers. + With this change, the engine will be able to clean up + such unreferenced layers. footer: Change-type: patch change-type: patch @@ -29838,11 +30728,14 @@ When layer store is created, its tmp directory may contain information - about transactions that were abruptly treminated during the previous process run. + about transactions that were abruptly treminated during + the previous process run. - Such data is now identified before any new transactions can be created, + Such data is now identified before any new transactions + can be created, - and a background process is started to delete both meta data and graph driver layeres. + and a background process is started to delete both meta + data and graph driver layeres. footer: Change-type: patch change-type: patch @@ -29856,19 +30749,25 @@ If the engine process is terminated during the layer extraction transaction, - before Commit or Cancel is called on the transaction, a new FS layer can be created + before Commit or Cancel is called on the transaction, a + new FS layer can be created - by the graph driver without any link to the layers metadata. + by the graph driver without any link to the layers + metadata. - This change ensures we don't perform any actions on the graph driver storage until + This change ensures we don't perform any actions on the + graph driver storage until - the FS layer ID (the cacheID) is persisted as a part of the transaction data. + the FS layer ID (the cacheID) is persisted as a part of + the transaction data. - We can use this data to clean up the graph driver storage on next process start + We can use this data to clean up the graph driver + storage on next process start - deleting all data associated with the transactions terminated abruptly. + deleting all data associated with the transactions + terminated abruptly. footer: Change-type: patch change-type: patch @@ -29882,7 +30781,8 @@ On macOS, unit tests where failing with - root@c4101a75c792:/go/src/github.com/docker/docker/pkg/authorization# go test . + root@c4101a75c792:/go/src/github.com/docker/docker/pkg/authorization# + go test . --- FAIL: TestAuthZRequestPluginError (0.00s) authz_unix_test.go:295: listen unix authz-test-plugin.sock: bind: file name too long @@ -29890,7 +30790,8 @@ authz_unix_test.go:295: listen unix authz-test-plugin.sock: bind: file name too long --- FAIL: TestAuthZResponsePlugin (0.00s) authz_unix_test.go:295: listen unix authz-test-plugin.sock: bind: file name too long - time="2020-04-07T10:07:04Z" level=warning msg="Request body is larger than: '1048576' skipping body" + time="2020-04-07T10:07:04Z" level=warning msg="Request + body is larger than: '1048576' skipping body" --- FAIL: TestMiddlewareWrapHandler (0.00s) authz_unix_test.go:295: listen unix authz-test-plugin.sock: bind: file name too long @@ -29899,7 +30800,8 @@ FAIL github.com/docker/docker/pkg/authorization 0.120s - This change moves the socket creation from a working test directory to a tmp directory, + This change moves the socket creation from a working + test directory to a tmp directory, so the path is shorter. footer: @@ -29939,7 +30841,8 @@ We don't support these on balenaOS anyway and we are planning to drop - support for them once we move to the new balenaCI-based pipeline. + support for them once we move to the new balenaCI-based + pipeline. footer: Change-type: patch change-type: patch @@ -29977,7 +30880,8 @@ This adds a new ContainerIDEnv field to HostConfig that can pass an - environment variable name, which will be set to the container ID and + environment variable name, which will be set to the + container ID and passed to the container environment. footer: @@ -29995,7 +30899,8 @@ First warn the user that balena-engine-daemon needs to be started. - Including instructions on how to make the system ready for that: + Including instructions on how to make the system ready + for that: - service files @@ -30063,7 +30968,8 @@ 684d8ba6109c853b355bf11ca3733c4099f14b92. - The default is still to sync all currently mounted filesystems before + The default is still to sync all currently mounted + filesystems before reporting an ApplyDiff as successful. footer: @@ -30092,16 +30998,19 @@ The only test from integration/ that covers any resource constrained - container scenarios is the OomKilled check in integration/container/kill_test.go + container scenarios is the OomKilled check in + integration/container/kill_test.go - This adds two addional checks that try to create, startk, stop and + This adds two addional checks that try to create, + startk, stop and inspect a busybox container with: a) a memory constraint like: balena-engine run -m 32m .. - b) a memory constraint like: balena-engine run -cpus ".5" .. + b) a memory constraint like: balena-engine run -cpus + ".5" .. footer: Change-type: patch change-type: patch @@ -30127,7 +31036,8 @@ set up the Go environment. - See https://docs.travis-ci.com/user/languages/minimal-and-generic/ + See + https://docs.travis-ci.com/user/languages/minimal-and-generic/ footer: Change-type: patch change-type: patch @@ -30219,14 +31129,17 @@ The prepare-openvpn service needs to be restarted to regenerate the - VPN keys when provisioning is finished and the `apiKey` is removed. + VPN keys when provisioning is finished and the `apiKey` is + removed. - Note that flasher images make changes to `/mnt/boottmp/config.json` so + Note that flasher images make changes to + `/mnt/boottmp/config.json` so we still need to manually kick `prepare-openvpn` in - `resin-device-register`, which is preferable to modifying services to + `resin-device-register`, which is preferable to modifying + services to watch multiple paths. footer: @@ -30247,7 +31160,8 @@ device ID. - This might also resolve a race condition that has been difficult to + This might also resolve a race condition that has been difficult + to track down. footer: @@ -30267,7 +31181,8 @@ This is just a name change that reflects the wider change that the - supervisor is no longer fetched from dockerhub but from Balena's registry. + supervisor is no longer fetched from dockerhub but from Balena's + registry. footer: Change-type: minor change-type: minor @@ -30321,7 +31236,8 @@ Before registration the VPN credentials use the `apiKey`, once the - device is registered we need to regenerate the credentials to use the + device is registered we need to regenerate the credentials to + use the `deviceApiKey` instead. footer: @@ -30380,20 +31296,26 @@ is not reachable. - The rationale for adding the dependency in the first place was that + The rationale for adding the dependency in the first place was + that - without an initial timesync certificate checks may fail. This can still + without an initial timesync certificate checks may fail. This + can still - happen, but the VPN will retry continuously and eventually succeed once + happen, but the VPN will retry continuously and eventually + succeed once the time is synched. - What happens now is that the VPN is delayed until the http sync, so if + What happens now is that the VPN is delayed until the http sync, + so if - the connectivity URL is blocked or unreachable, but the internet is + the connectivity URL is blocked or unreachable, but the internet + is - accessible, VPN will not even launch and no remote debugging is possible. + accessible, VPN will not even launch and no remote debugging is + possible. Fixes #2508 @@ -30470,7 +31392,8 @@ This will allow us to make changes to config.js in meta-balena without - breaking the deploy steps. If additional changes are needed at runtime + breaking the deploy steps. If additional changes are needed at + runtime the substitutions can be made by the leviathan Jenkins job. footer: @@ -30489,7 +31412,8 @@ body: > If the submodule was recently added to meta-balena, the checkout - command will not initialize it without a separate submodule update + command will not initialize it without a separate submodule + update command. footer: @@ -30636,9 +31560,11 @@ per-unit configuration files extracts. - These will then be monitored by the respective service units to trigger + These will then be monitored by the respective service units to + trigger - restarts, so that service units will only be restarted when there are + restarts, so that service units will only be restarted when + there are configuration changes that apply to them. footer: @@ -30657,14 +31583,17 @@ on config.json changes are part of this target. - This causes a burst of service restarts each time config.json changes, + This causes a burst of service restarts each time config.json + changes, - independently of whether the configuration changes applies to the unit + independently of whether the configuration changes applies to + the unit or not. - This commit removes all config-json.target instances in preparation for + This commit removes all config-json.target instances in + preparation for the introduction of a better, more fine grained mechanism. footer: @@ -30707,11 +31636,14 @@ In order to avoid the need to unlock encrypted partitions in GRUB we want - to use a custom stage2 bootloader. Since that is not ready yet, emulate that + to use a custom stage2 bootloader. Since that is not ready yet, + emulate that - by copying flasher kernel to the EFI partition, start it from GRUB, have it + by copying flasher kernel to the EFI partition, start it from + GRUB, have it - unlock all the partitions, find the real kernel and kexec into it. + unlock all the partitions, find the real kernel and kexec into + it. footer: Change-type: patch change-type: patch @@ -30728,7 +31660,8 @@ - Split resin-boot into EFI and linux boot - - LUKS-format the partitions, copy image contents after unlocking + - LUKS-format the partitions, copy image contents after + unlocking - Propagate signatures for secure boot @@ -30828,7 +31761,8 @@ After moving the partition resizing code to execute on each boot, - we made it unreachable on first boot. We must not exit the script + we made it unreachable on first boot. We must not exit the + script after resizing the partition only because that way the resizing @@ -30866,7 +31800,8 @@ This will allow us to refer to the supervisor image by the repo name - in docker commands, like docker inspect, and prevent re-downloading the + in docker commands, like docker inspect, and prevent + re-downloading the image even though it already exists as an untagged digest. footer: @@ -30888,14 +31823,17 @@ it because it's dirty, the partition gets resized, but not the - filesystem. The script will not attempt to resize the filesystem again, + filesystem. The script will not attempt to resize the filesystem + again, as it detects the partition has already been resized. - Split these actions apart, so that the filesystem resize is always + Split these actions apart, so that the filesystem resize is + always - attempted. If resize2fs detects that the filesystem is already filling + attempted. If resize2fs detects that the filesystem is already + filling available space, it will exit with no action taken. footer: @@ -30957,7 +31895,8 @@ body: > Some BSPs might only make use of UBOOT_MACHINE so let's consider - this case in addition to UBOOT_CONFIG when setting dependency for + this case in addition to UBOOT_CONFIG when setting dependency + for u-boot's do_deploy task. footer: diff --git a/CHANGELOG.md b/CHANGELOG.md index d548a06f..93a43499 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,6 +1,21 @@ Change log ----------- +# v5.1.48 +## (2024-03-06) + + +
+ Update layers/meta-balena to 6898f00e2d312aa468a44a98880882c31e81b26e [Self-hosted Renovate Bot] + +> ## meta-balena-5.1.48 +> ### (2024-03-06) +> +> * hostapp-update-hooks: Soft include balena-config-defaults [Michal Toman] +> + +
+ # v5.1.47+rev3 ## (2024-03-05) diff --git a/VERSION b/VERSION index 447c61eb..c9019730 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -5.1.47+rev3 \ No newline at end of file +5.1.48 \ No newline at end of file