Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

phosphor-settings-manager repeatedly terminating when setting is out of bounds #15

Open
cjcain opened this issue Mar 31, 2022 · 0 comments

Comments

@cjcain
Copy link

cjcain commented Mar 31, 2022

If a system has a user power cap set that is outside of the settings bounds, settings-manager terminates repeatedly and leaves system in a bad state. It stays in this bad state forever.
Scenario:

  1. user power cap was set within valid settings bounds
  2. code update happens which introduces new bounds that the user power cap does not fall into.

How discovered: A change was made to override the bounds checking of the user power cap value:
meta-ibm: Remove Power.Cap hardcoded range

The settings bounds for user power cap:
mrw-override-settings.yaml

# obmcutil state
Error finding '/xyz/openbmc_project/state/bmc0' service: Input/output error

From journal:

Mar 31 16:55:10 perfrain86bmc systemd[1]: Starting Phosphor Settings Daemon...
Mar 31 16:55:10 perfrain86bmc systemd[1]: Starting Phosphor Download Manager...
Mar 31 16:55:10 perfrain86bmc systemd[1]: Starting Post code manager...
Mar 31 16:55:10 perfrain86bmc systemd[1]: Starting Rsyslog config updater...
Mar 31 16:55:10 perfrain86bmc systemd[1]: Starting Telemetry...
Mar 31 16:55:10 perfrain86bmc systemd[1]: Starting Phosphor User Manager...
Mar 31 16:55:10 perfrain86bmc phosphor-settings-manager[335]: Input parameter for PowerCap is invalid Input: 1002in uint: Watts is not in range:0..1000
Mar 31 16:55:10 perfrain86bmc phosphor-settings-manager[335]: Invalid argument was given.
Mar 31 16:55:10 perfrain86bmc post-code-manager[337]: Start post code manager service...
Mar 31 16:55:10 perfrain86bmc phosphor-settings-manager[335]: terminate called after throwing an instance of 'sdbusplus::exception::SdBusError'
Mar 31 16:55:10 perfrain86bmc phosphor-settings-manager[335]:   what():  sd_bus_call: org.freedesktop.DBus.Error.ServiceUnknown: The name is not activatable
Mar 31 16:55:15 perfrain86bmc systemd[1]: Started Phosphor Settings Daemon.
Mar 31 16:55:29 perfrain86bmc phosphor-mapper[334]: Introspect call failed with error: generic:113, No route to host on process: xyz.openbmc_project.Settings path: /
Mar 31 16:55:29 perfrain86bmc ibm-vpd-parser[716]: terminate called after throwing an instance of 'std::runtime_error'
Mar 31 16:55:29 perfrain86bmc ibm-vpd-parser[716]:   what():  Error in invoking D-Bus logging create interface to register PEL
Mar 31 16:55:29 perfrain86bmc phosphor-bmc-state-manager[506]: terminate called after throwing an instance of 'std::runtime_error'
Mar 31 16:55:29 perfrain86bmc phosphor-bmc-state-manager[506]:   what():  Error in invoking D-Bus logging create interface
Mar 31 16:55:29 perfrain86bmc systemd[1]: set-spi-mux.service: Deactivated successfully.
Mar 31 16:55:29 perfrain86bmc systemd[1]: Finished Set the SPI mux.
Mar 31 16:55:29 perfrain86bmc systemd-coredump[511]: Process 335 (phosphor-settin) of user 0 dumped core.
Mar 31 16:55:29 perfrain86bmc phosphor-log-manager[333]: Error reading QuiesceOnHwError property: sd_bus_call: org.freedesktop.DBus.Error.NoReply: Remote peer disconnected
Mar 31 16:55:29 perfrain86bmc phosphor-bmc-state-manager[506]: sdbusplus D-Bus call exception, error sd_bus_call: org.freedesktop.DBus.Error.NoReply: Remote peer disconnected trying to create an error with xyz.openbmc_project.State.PinholeReset
Mar 31 16:55:29 perfrain86bmc phosphor-log-manager[333]: Error reading QuiesceOnHwError property: sd_bus_call: org.freedesktop.DBus.Error.ServiceUnknown: The name is not activatable
Mar 31 16:55:29 perfrain86bmc systemd[1]: xyz.openbmc_project.Settings.service: Main process exited, code=dumped, status=6/ABRT
Mar 31 16:55:29 perfrain86bmc systemd[1]: xyz.openbmc_project.Settings.service: Failed with result 'core-dump'.
Mar 31 16:55:29 perfrain86bmc systemd[1]: [email protected]: Deactivated successfully.
Mar 31 16:55:30 perfrain86bmc ibm-panel[649]: Resolution is empty for PEL = /xyz/openbmc_project/logging/entry/269
Mar 31 16:55:30 perfrain86bmc ibm-panel[649]: sd_bus_call: org.freedesktop.DBus.Error.ServiceUnknown: The name is not activatablesd_bus_call: org.freedesktop.DBus.Error.ServiceUnknown: The name is not activatableSystem operating mode set to Normal as power policy = and reboot policy = 0
Mar 31 16:55:30 perfrain86bmc systemd[1]: xyz.openbmc_project.Settings.service: Scheduled restart job, restart counter is at 1.

To recreate:
Driver: fw1020.00-49.3

  1. Edit /var/lib/phosphor-settings-manager/settings/xyz/openbmc_project/control/host0/power_cap__
    Use values outside of the 0-1000 range and enable the user cap. Example:
    "value0": 1002,
    "value1": true
  2. Reboot the BMC
    When system comes back up, it should be stuck in the bad/failed state.

To fix: Edit /var/lib/phosphor-settings-manager/settings/xyz/openbmc_project/control/host0/power_cap__
Set the values back to their defaults of 0 / disabled:
"value0": 0,
"value1": false

@cjcain cjcain changed the title phosphor-settings-manager repeatedly terminating went setting is out of bounds phosphor-settings-manager repeatedly terminating when setting is out of bounds Mar 31, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant