Auto restart mission critical BMC services
This is to ensure these services are running and active at all times.
Resolves openbmc/openbmc-test-automation#130
Change-Id: I6a9325582b1fa80ff826165f2cb0e9fb05e2dbc3
Signed-off-by: George Keishing <gkeishin@in.ibm.com>
diff --git a/lib/utils.robot b/lib/utils.robot
index dbfcb77..87f1f52 100755
--- a/lib/utils.robot
+++ b/lib/utils.robot
@@ -634,3 +634,11 @@
Should Be Empty ${stderr}
${btime}= Convert To Integer ${output}
[Return] ${btime}
+
+
+Execute Command On BMC
+ [Documentation] Execute given command on BMC and return output.
+ [Arguments] ${command}
+ ${stdout} ${stderr}= Execute Command ${command} return_stderr=True
+ Should Be Empty ${stderr}
+ [Return] ${stdout}
diff --git a/test_lists/skip_test b/test_lists/skip_test
index 7635f74..c510a93 100644
--- a/test_lists/skip_test
+++ b/test_lists/skip_test
@@ -15,9 +15,6 @@
-e Validate_Heartbeat_LEDs_Test_Cases
-e Validate_Identify_LEDs_Test_Cases
-e Validate_Beep_LEDs_Test_Cases
-#https://github.com/openbmc/openbmc-test-automation/issues/130
--e Test_OpenBMC_Services_Autorestart_Policy
--e Test_Restart_Policy_for_openbmc_service
#https://github.com/openbmc/openbmc-test-automation/issues/90
-e Verify_System_VPD_Properties
#https://github.com/openbmc/openbmc/issues/93
diff --git a/test_lists/skip_test_palmetto b/test_lists/skip_test_palmetto
index abc462f..7ceadf5 100644
--- a/test_lists/skip_test_palmetto
+++ b/test_lists/skip_test_palmetto
@@ -31,9 +31,6 @@
-e Disable_PowerCap
#https://github.com/openbmc/openbmc-test-automation/issues/123
-e Execute_ipmi_BT_capabilities_command
-#https://github.com/openbmc/openbmc-test-automation/issues/130
--e Test_OpenBMC_Services_Autorestart_Policy
--e Test_Restart_Policy_for_openbmc_service
#https://github.com/openbmc/openbmc-test-automation/issues/22
-e Validate_Heartbeat_LEDs_Test_Cases
-e Validate_Identify_LEDs_Test_Cases
diff --git a/test_lists/skip_test_witherspoon b/test_lists/skip_test_witherspoon
index 1f5ed7a..9780d6d 100644
--- a/test_lists/skip_test_witherspoon
+++ b/test_lists/skip_test_witherspoon
@@ -15,9 +15,6 @@
-e Validate_Heartbeat_LEDs_Test_Cases
-e Validate_Identify_LEDs_Test_Cases
-e Validate_Beep_LEDs_Test_Cases
-#https://github.com/openbmc/openbmc-test-automation/issues/130
--e Test_OpenBMC_Services_Autorestart_Policy
--e Test_Restart_Policy_for_openbmc_service
#https://github.com/openbmc/openbmc-test-automation/issues/90
-e Verify_System_VPD_Properties
#https://github.com/openbmc/openbmc/issues/93
diff --git a/tests/test_openbmc_service_restart_policy.robot b/tests/test_openbmc_service_restart_policy.robot
index d52a51f..8b7b3c5 100644
--- a/tests/test_openbmc_service_restart_policy.robot
+++ b/tests/test_openbmc_service_restart_policy.robot
@@ -1,98 +1,80 @@
*** Settings ***
-Documentation This testsuites tests the autorestart policy for
-... OpenBMC project
+Documentation Verify Auto Restart policy for set of mission critical
+... services needed for functioning on BMC.
-Resource ../lib/resource.txt
-Resource ../lib/connection_client.robot
-Resource ../lib/openbmc_ffdc.robot
+Resource ../lib/resource.txt
+Resource ../lib/connection_client.robot
+Resource ../lib/openbmc_ffdc.robot
+Resource ../lib/utils.robot
-Suite Setup Open Connection And Log In
-Suite Teardown Close All Connections
-Test Teardown FFDC On Test Case Fail
+Suite Setup Open Connection And Log In
+Suite Teardown Close All Connections
+Test Teardown FFDC On Test Case Fail
+
+*** Variables ***
+${LOG_SERVICE} xyz.openbmc_project.Logging.service
*** Test Cases ***
-Test OpenBMC Services Autorestart Policy
- [Documentation] This testcases is for checking all the openbmc services
- ... restart policy is set to active
- ... Disabling this test as use case is not well define and
- ... developement point of view this may keep changing. So
- ... untill then, this remains commented piece of test.
- [Tags] Test_OpenBMC_Services_Autorestart_Policy
- @{services}= Create List btbridged.service
- ... host-ipmid.service
- ... inarp.service
- ... network.service
- ... network-update-dns.service
- ... obmc-console.service
- ... obmc-hwmon.service
- ... obmc-phosphor-chassisd.service
- ... obmc-phosphor-event.service
- ... obmc-phosphor-fand.service
- ... obmc-phosphor-flashd.service
- ... obmc-phosphor-policyd.service
- ... obmc-phosphor-sensord.service
- ... obmc-phosphor-sysd.service
- ... obmc-phosphor-user.service
- ... org.openbmc.buttons.Power.service
- ... org.openbmc.buttons.reset.service
- ... org.openbmc.control.BmcFlash.service
- ... org.openbmc.control.Bmc.service
- ... org.openbmc.control.Chassis.service
- ... org.openbmc.control.Checkstop.service
- ... org.openbmc.control.Fans.service
- ... org.openbmc.control.Flash.service
- ... org.openbmc.control.Host.service
- ... org.openbmc.control.led.service
- ... org.openbmc.control.Power.service
- ... org.openbmc.examples.PythonService.service
- ... org.openbmc.examples.SDBusService.service
- ... org.openbmc.Inventory.service
- ... org.openbmc.managers.Download.service
- ... org.openbmc.managers.System.service
- ... org.openbmc.ObjectMapper.service
- ... org.openbmc.Sensors.service
- ... org.openbmc.watchdog.Host.service
- ... phosphor-rest.service
- ... rest-dbus.service
- ... settings.service
- : FOR ${SERVICE} IN @{services}
- \ Check Service Autorestart ${SERVICE}
+
+Verify OpenBMC Services Auto Restart Policy
+ [Documentation] Kill active services and expect auto restart.
+ [Tags] Verify_OpenBMC_Services_Auto_Restart_Policy
+ # The services listed bellow restart policy should be "always"
+ # Command output:
+ # systemctl -p Restart show xyz.openbmc_project.Logging.service | cat
+ # Restart=always
+ @{services}=
+ ... Create List xyz.openbmc_project.Logging.service
+ ... xyz.openbmc_project.ObjectMapper.service
+ ... xyz.openbmc_project.State.BMC.service
+ ... xyz.openbmc_project.State.Chassis.service
+ ... xyz.openbmc_project.State.Host.service
+ : FOR ${SERVICE} IN @{services}
+ \ Check Service Autorestart ${SERVICE}
-Test Restart Policy for openbmc service
- [Documentation] This testcase will kill the service and make sure it
- ... does restart after that
- [Tags] Test_Restart_Policy_for_openbmc_service
+Kill Services And Expect Service Restart
+ [Documentation] Kill the service and it must restart.
+ [Tags] Kill_Services_And_Expect_Service_Restart
- ${MainPID}= Execute Restart Policy Command
- ... systemctl -p MainPID show phosphor-settings.service| cut -d = -f2
- Should Not Be Equal 0 ${MainPID}
+ # Get the MainPID and service state.
+ ${MainPID}= Get Service Attribute MainPID ${LOG_SERVICE}
+ Should Not Be Equal ${0} ${MainPID}
+ ... msg=Logging service not restarted.
- Execute Restart Policy Command kill -9 ${MainPID}
- Sleep 30s reason=Wait for service to restart properly
+ ${ActiveState}= Get Service Attribute ActiveState ${LOG_SERVICE}
+ Should Be Equal active ${ActiveState}
+ ... msg=Logging Service not in active state.
- ${ActiveState}= Execute Restart Policy Command
- ... systemctl -p ActiveState show phosphor-settings.service| cut -d = -f2
- Should Be Equal active ${ActiveState}
+ Execute Command On BMC kill -9 ${MainPID}
+ Sleep 10s reason=Wait for service to restart.
- ${MainPID}= Execute Restart Policy Command
- ... systemctl -p MainPID show phosphor-settings.service| cut -d = -f2
- Should Not Be Equal 0 ${MainPID}
+ ${MainPID}= Get Service Attribute MainPID ${LOG_SERVICE}
+ Should Not Be Equal ${0} ${MainPID}
+ ... msg=Logging service not restarted.
+
+ ${ActiveState}= Get Service Attribute ActiveState ${LOG_SERVICE}
+ Should Be Equal active ${ActiveState}
+ ... msg=Logging service not in active state.
*** Keywords ***
Check Service Autorestart
- [Arguments] ${servicename}
- ${restart_policy}=
- ... Execute Restart Policy Command
- ... systemctl -p Restart show ${servicename} | cut -d = -f2
- Should Be Equal always ${restart_policy}
- ... msg=restart policy is not always for ${servicename}
+ [Documentation] Check if given policy is "always".
+ [Arguments] ${servicename}
+ # servicename Qualified service name
+ ${restart_policy}= Get Service Attribute Restart ${servicename}
+ Should Be Equal always ${restart_policy}
+ ... msg=Incorrect policy for ${servicename}
-Execute Restart Policy Command
- [Arguments] ${command}
- ${stdout} ${stderr}= Execute Command ${command} return_stderr=True
- Should Be Empty ${stderr}
- [Return] ${stdout}
+Get Service Attribute
+ [Documentation] Get service attribute policy output.
+ [Arguments] ${option} ${servicename}
+ # option systemctl supported options
+ # servicename Qualified service name
+ ${cmd}= Set Variable
+ ... systemctl -p ${option} show ${servicename} | cut -d = -f2
+ ${attr}= Execute Command On BMC ${cmd}
+ [Return] ${attr}