OS Update Failure Not Displayed in UI

[petermcd]

Describe the bug
I have attempted to build the BMC firmware on my Mac and upload it to my Turing Pi. The build seemed successful but when I tried to update the Turing Pi using the OTA method the progress bar remains green and displays the message "Verifying checksum and finalizing upgrade..."

When looking at the browsers developer tools it is in fact receiving the following response continuously:

{
    "Error": "failed os_update (exit status: 255)"
}

To Reproduce
Steps to reproduce the behaviour:

Build custom OS using instructions for Mac
Use BMC UI to upload

Expected behavior
Upgrade to be successful or to output the failure message

Screenshots
If applicable, add screenshots to help explain your problem.

Versions
linux version=Linux turing-master-2 5.4.61 #1 SMP PREEMPT Tue Nov 28 13:52:46 UTC 2023 armv7l GNU/Linux
bmc version=Daemon version: 2.0.5
Build time: 2023-11-28 14:01:07-00:00
Build version: 2023.08
Buildroot release:
Buildroot 2023.08
API version: 1.1
BMC UI: v1.0.3

Additional context

Although I have built the firmware using the instructions in the README.md file in the BMC repo there is every possibility there is something wrong, this should not however cause the WEB UI not to react to the failure received.

[petermcd]

I have taken a closer look at this and can see that the response received has the following

Summary
URL: https://192.168.1.175/api/bmc?opt=get&type=firmware
Status: 200
Source: Network
Address: 192.168.1.175:443
Initiator: 
app.js:2:85593

This is not being caught by the error handler in handleFirmwareUpload() as the http response is a 200 response. BMC should not be responding with a 200 response code on this.

[barrenechea]

Hey Peter! I hope you're doing good!

Just to confirm that it is not related to your local environment, can you try flashing an officially built beta firmware? The bmc-firmware repo executes stateless builds as soon as a change is made to the codebase. I did a quick guide to download the latest firmware. I hope it's clear enough!

Regarding the unresponsive UI, I believe that is being handled in the latest bmc-ui 😄 I'd need to confirm, though. I've never seen that "Error": "failed os_update (exit status: 255)" before, but it should be handled properly.

[petermcd]

I get the same issue with the latest build. Each of the entries called BMC in the below screenshot has this output.

[petermcd]

Here is the same issue attempting the same from Firefox:

I have also tried from an iPhone and had the same issue (but can't see the responses, just that it doesn't complete)

[petermcd]

In case it matters, this occurs on 2 different boards, both were from the Kickstarter campaign. If I use the 2.0.5 release this flashes fine using the same browsers (although both are already on 2.0.5).

[svenrademakers]

Hi @petermcd, can you capture some of the logging on the BMC? Logging is just printed onto the console
It should give us more information as to why you get a 255 on the upgrade script.

https://docs.turingpi.com/docs/turing-pi2-bmc-how-to-get-started should give some more info on how to connect to the BMC

[petermcd]

HI @svenrademakers I logged into the BMC using SSH but received no logging straight out onto the console.

To rule out my own home environment (for example router or switches interfering) I used another laptop, connected it directly to the Turing Pi, configured my NIC to be on the subnet the Turing Pi is on. Connected using Putty and then attempted the firmware upgrade using the web UI.

Attached is the output (apologies for the crud image but took it on my phone as it wasn't connected to anything but the Turing Pi. As you can see the output is exactly the same.

[petermcd]

After doing that It has dawned on me you didn't realise SSH. I can't connect using serial at the moment, I will have to give that a try tomorrow.

[petermcd]

I think I have realised what has happened. When I flashed the firmware from 1x - 2x I missed the last step and didn't remove the SD Card, the BMC is running from the SDCard. I will remove the card and see if this resolved the issue.

Although I have flashed the drive to 2.0.5 since so I might be wrong as to the cause.

[petermcd]

Turns put this was in fact caused by me leaving the SD Card in the Turing Pi after upgrading from 1x to 2x. Removing the SD Card resolved the issue.

As this is an error type that the Turing Pi 2 can give out it is probably be worth checking if the error key exists in the response handler for the API and deal with it appropriately such as outputting an error and getting it to stop trying.

[svenrademakers]

Actually this is a good check to implement. Thanks for getting to the bottom of this @petermcd !