increase timeout on systemd Subscribe method call
An issue was recently hit in IBM manufacturing where this call to the
Subscribe method provided by systemd returned a dbus timeout. The
phosphor-host-state-manager service was restarted and the call then
passed, but by this time the first fail generated a core dump
and resulted in a BMC Quiesced state.
The situation is not easily recreatable and was only seen on our largest
configured systems, and only after a factory reset.
No searching of systemd bugs has turned up anything in this area so it
appears to be specific to OpenBMC. Most likely related to our embedded
env and large amount of services started during BMC startup, especially
after a factory reset.
Worst case time on our max configured system was around 30s so double
that and make it our timeout for this call.
Tested
- Basic good path testing to ensure no regressions
- 100+ factory resets done on a variety of systems to ensure the timeout
was no longer seen
Signed-off-by: Andrew Geissler <geissonator@yahoo.com>
Change-Id: Id32017c3d0ca7d12091818467b15adf0e1a7aaa9
diff --git a/utils.cpp b/utils.cpp
index ec85e41..124f28f 100644
--- a/utils.cpp
+++ b/utils.cpp
@@ -8,6 +8,7 @@
#include <phosphor-logging/lg2.hpp>
+#include <chrono>
#include <filesystem>
namespace phosphor
@@ -19,6 +20,8 @@
namespace utils
{
+using namespace std::literals::chrono_literals;
+
PHOSPHOR_LOG2_USING;
constexpr auto SYSTEMD_SERVICE = "org.freedesktop.systemd1";
@@ -37,7 +40,12 @@
try
{
- bus.call(method);
+ // On OpenBMC based systems, systemd has had a few situations where it
+ // has been unable to respond to this call within the default d-bus
+ // timeout of 25 seconds. This is due to the large amount of work being
+ // done by systemd during OpenBMC startup. Set the timeout for this call
+ // to 60 seconds (worst case seen was around 30s so double it).
+ bus.call(method, 60s);
}
catch (const sdbusplus::exception_t& e)
{