Added skip list for Palmetto
This changes addresses the following:
- Created argument file skip_test_palmetto to
Skip TCs which are failing due to existing fw issue on Palmetto
Skip TCs which are not applicable for Palmetto system
- Moved all argument files into new test_lists folder
Resolves openbmc/openbmc-test-automation#145
openbmc/openbmc-test-automation#138
Change-Id: Icddd873a3ea23d676ea2e397528a82b8923c7c2e
Signed-off-by: Rahul Maheshwari <rahulmaheshwari@in.ibm.com>
diff --git a/README.md b/README.md
index 3a99ddd..c00aa00 100755
--- a/README.md
+++ b/README.md
@@ -86,8 +86,8 @@
$ OPENBMC_HOST=x.x.x.x tox -e barreleye -- tests
No preset environment variables, the entire test suite excluding test
- cases failing due to known issues. The skip_list file contains tag list which are to be skipped
- $ OPENBMC_HOST=x.x.x.x tox -e barreleye -- --argumentfile skip_list tests
+ cases using argument file.
+ $ OPENBMC_HOST=x.x.x.x tox -e barreleye -- --argumentfile test_lists/skip_test tests
```
It can also be run by pasing variables from the cli...
@@ -120,6 +120,6 @@
```shell
HW CI tox command
Set the preset environment variables, run HW CI test for a barreleye system
- $ OPENBMC_HOST=x.x.x.x tox -e barreleye -- --argumentfile HW_CI tests
+ $ OPENBMC_HOST=x.x.x.x tox -e barreleye -- --argumentfile test_lists/HW_CI tests
```
diff --git a/HW_CI b/test_lists/HW_CI
similarity index 100%
rename from HW_CI
rename to test_lists/HW_CI
diff --git a/skip_list b/test_lists/skip_test
similarity index 100%
rename from skip_list
rename to test_lists/skip_test
diff --git a/test_lists/skip_test_palmetto b/test_lists/skip_test_palmetto
new file mode 100644
index 0000000..33d9363
--- /dev/null
+++ b/test_lists/skip_test_palmetto
@@ -0,0 +1,28 @@
+-e known_issue
+#https://github.com/openbmc/openbmc/issues/611
+-e Association_with_virtual_sensor
+#https://github.com/openbmc/openbmc/issues/539
+-e Test_Wrong_Reservation_ID
+#https://github.com/openbmc/openbmc/issues/539
+-e Test_Correct_Reservation_ID
+#https://github.com/openbmc/openbmc/issues/604
+-e minimal_memory_buffer_inventory
+-e minimal_main_planar_inventory
+-e Centaur0_Present
+-e Centaur0_fault
+#https://github.com/openbmc/openbmc/issues/93
+-e minimal_fan_inventory
+-e minimal_system_inventory
+-e System_Present
+-e Chassis_Present
+#https://github.com/openbmc/openbmc/issues/615
+-e Create_and_delete_user_without_group_name
+-e Create_and_delete_user_with_user_group_name
+-e Create_multiple_users
+-e Create_and_delete_user_without_password
+-e Set_password_for_existing_user
+-e Set_password_with_empty_password_for_existing
+-e Create_existing_user
+#Testcase NA for Palmetto system
+-e io_board_Present
+-e io_board_Fault
diff --git a/tests/test_association.robot b/tests/test_association.robot
index 86c98cc..fdc7b0e 100755
--- a/tests/test_association.robot
+++ b/tests/test_association.robot
@@ -159,6 +159,7 @@
[Documentation] ***GOOD PATH***
... Create an error log on virtual sensor and
... verify its association.\n
+ [Tags] Association_with_virtual_sensor
Clear all logs
diff --git a/tests/test_inventory.robot b/tests/test_inventory.robot
index d007638..7542778 100644
--- a/tests/test_inventory.robot
+++ b/tests/test_inventory.robot
@@ -32,6 +32,8 @@
Should Be True ${count}>${0}
minimal memory buffer inventory
+ [Tags] minimal_memory_buffer_inventory
+
${count} = Get Total Present membuf
Should Be True ${count}>${0}
@@ -41,10 +43,14 @@
Should Be True ${count}>${2}
minimal main planar inventory
+ [Tags] minimal_main_planar_inventory
+
${count} = Get Total Present motherboard
Should Be True ${count}>${0}
minimal system inventory
+ [Tags] minimal_system_inventory
+
${count} = Get Total Present system
Should Be True ${count}>${0}
diff --git a/tests/test_sensors.robot b/tests/test_sensors.robot
index 48bf97d..ebbbe78 100644
--- a/tests/test_sensors.robot
+++ b/tests/test_sensors.robot
@@ -199,6 +199,8 @@
Response Should Be Equal False
Centaur0 Present
+ [Tags] Centaur0_Present
+
${uri} = Get System component membuf
${x} = Get Inventory Sensor Number ${uri}
@@ -207,6 +209,8 @@
Response Should Be Equal True
Centaur0 not Present
+ [Tags] Centaur0_not_Present
+
${uri} = Get System component membuf
${x} = Get Inventory Sensor Number ${uri}
@@ -231,16 +235,20 @@
Response Should Be Equal False
System Present
+ [Tags] System_Present
+
${uri} = Get System component system
Read The Attribute ${uri} present
Response Should Be Equal True
-
+
System Fault
${uri} = Get System component system
Read The Attribute ${uri} fault
Response Should Be Equal False
Chassis Present
+ [Tags] Chassis_Present
+
${uri} = Get System component chassis
Read The Attribute /org/openbmc/inventory/system/chassis present
Response Should Be Equal True
@@ -251,11 +259,13 @@
Response Should Be Equal False
io_board Present
+ [Tags] io_board_Present
${uri} = Get System component io_board
Read The Attribute ${uri} present
Response Should Be Equal True
io_board Fault
+ [Tags] io_board_Fault
${uri} = Get System component io_board
Read The Attribute ${uri} fault
Response Should Be Equal False
diff --git a/tests/test_user.robot b/tests/test_user.robot
index ae64cde..9418153 100755
--- a/tests/test_user.robot
+++ b/tests/test_user.robot
@@ -36,10 +36,11 @@
${resp} = Delete Group ${groupname}
Should Be Equal ${resp} ok
-Create and delete user without groupname
+Create and delete user without group name
[Documentation] ***GOOD PATH***
... This testcase is for testing user creation with
... without groupname in open bmc.\n
+ [Tags] Create_and_delete_user_without_group_name
${username} = Generate Random String ${RANDOM_STRING_LENGTH}
${password} = Generate Random String ${RANDOM_STRING_LENGTH}
@@ -62,6 +63,7 @@
... This testcase is for testing user creation with
... user name, password, comment and group name(user group)
... in open bmc.\n
+ [Tags] Create_and_delete_user_with_user_group_name
${username} = Generate Random String ${RANDOM_STRING_LENGTH}
${password} = Generate Random String ${RANDOM_STRING_LENGTH}
@@ -88,6 +90,7 @@
[Documentation] ***GOOD PATH***
... This testcase is to verify that multiple users creation
... in open bmc.\n
+ [Tags] Create_multiple_users
: FOR ${INDEX} IN RANGE 1 10
\ Log ${INDEX}
@@ -106,6 +109,7 @@
[Documentation] ***GOOD PATH***
... This testcase is to create and delete a user without password
... in open bmc.\n
+ [Tags] Create_and_delete_user_without_password
${username} = Generate Random String ${RANDOM_STRING_LENGTH}
${password} = Generate Random String ${RANDOM_STRING_LENGTH}
@@ -132,6 +136,7 @@
[Documentation] ***GOOD PATH***
... This testcase is for testing password set for user
... in open bmc.\n
+ [Tags] Set_password_for_existing_user
${username} = Generate Random String ${RANDOM_STRING_LENGTH}
${password} = Generate Random String ${RANDOM_STRING_LENGTH}
@@ -160,6 +165,7 @@
[Documentation] ***GOOD PATH***
... This testcase is to verify that empty password can be set
... for a existing user.\n
+ [Tags] Set_password_with_empty_password_for_existing
${username} = Generate Random String ${RANDOM_STRING_LENGTH}
${password} = Generate Random String ${RANDOM_STRING_LENGTH}
@@ -193,6 +199,7 @@
[Documentation] ***BAD PATH***
... This testcase is for checking that user creation is not allowed
... for existing user in open bmc.\n
+ [Tags] Create_existing_user
${username} = Generate Random String ${RANDOM_STRING_LENGTH}
${password} = Generate Random String ${RANDOM_STRING_LENGTH}