Add test for number of BMC dumps

Add 'Dump Out Of Space Test' to module test_bmc_dump.robot.
Add parameter to 'Create User Initiated Dump' in module
dump_utils.robot to enable out of dump space checking.

Resolves openbmc/openbmc-test-automation#1353

Change-Id: Ia73b327e7177685b5efb22a76dbf00cadb7c423e
Signed-off-by: Steven Sombar <ssombar@us.ibm.com>
diff --git a/lib/dump_utils.robot b/lib/dump_utils.robot
index 778d089..30b5ef4 100644
--- a/lib/dump_utils.robot
+++ b/lib/dump_utils.robot
@@ -9,21 +9,45 @@
 
 Create User Initiated Dump
     [Documentation]  Generate user initiated dump and return
-    ...  dump id (e.g 1, 2 etc).
+    ...  the dump id number (e.g., "5").  Optionally return EMPTY
+    ...  if out of dump space.
+    [Arguments]   ${check_out_of_space}=${False}
+
+    # Description of Argument(s):
+    # check_out_of_space   If ${False}, a dump will be created and
+    #                      its dump_id will be returned.
+    #                      If ${True}, either the dump_id will be
+    #                      returned, or the value ${EMPTY} will be
+    #                      returned if out of dump space was
+    #                      detected when creating the dump.
 
     ${data}=  Create Dictionary  data=@{EMPTY}
     ${resp}=  OpenBMC Post Request
-    ...  ${DUMP_URI}action/CreateDump  data=${data}
+    ...  ${DUMP_URI}action/CreateDump  data=${data}  quiet=${1}
+
+    Run Keyword If  '${check_out_of_space}' == '${False}'
+    ...  Run Keyword And Return  Get The Dump Id  ${resp}
+    ...  ELSE   Run Keyword And Return  Check For Too Many Dumps  ${resp}
+
+
+Get The Dump Id
+    [Documentation]  Wait for the dump to be created. Return the
+    ...  dump id number (e.g., "5").
+    [Arguments]  ${resp}
+
+    # Description of Argument(s):
+    # resp   Response object from action/Create Dump attempt.
+    #        Example object:
+    #        {
+    #           "data": 5,
+    #           "message": "200 OK",
+    #           "status": "ok"
+    #        },
+    #        The "data" field conveys the id number of the created dump.
 
     Should Be Equal As Strings  ${resp.status_code}  ${HTTP_OK}
     ${json}=  To JSON  ${resp.content}
 
-    # REST "CreateDump" JSON response.
-    # {
-    #    "data": null,
-    #    "message": "200 OK",
-    #    "status": "ok"
-    # }
     Run Keyword If  ${json["data"]} == ${None}
     ...  Fail  Dump id returned null.
 
@@ -35,6 +59,44 @@
     [Return]  ${dump_id}
 
 
+Check For Too Many Dumps
+    [Documentation]  Return the dump_id number, or return ${EMPTY} if dump
+    ...  creation failed due to too many dumps.
+    [Arguments]  ${resp}
+
+    # Description of Argument(s):
+    # resp   Response object from action/Create Dump attempt.
+    #        Example object if there are too many dumps:
+    #        {
+    #           "data": {
+    #              "description": "Internal Server Error",
+    #              "exception": "'Dump not captured due to a cap.'",
+    #              "traceback": [
+    #              "Traceback (most recent call last):",
+    #                ...
+    #              "DBusException: Create.Error.QuotaExceeded"
+    #                           ]
+    #              },
+    #           "message": "500 Internal Server Error",
+    #           "status": "error"
+    #        }
+
+    # If dump was created normally, return the dump_id number.
+    Run Keyword If  '${resp.status_code}' == '${HTTP_OK}'
+    ...  Run Keyword And Return  Get The Dump Id  ${resp}
+
+    ${exception}=  Set Variable  ${resp.json()['data']['exception']}
+    ${at_capacity}=  Set Variable  Dump not captured due to a cap
+    ${too_many_dumps}=  Evaluate  $at_capacity in $exception
+    Rprintn
+    Rprint Vars   exception  too_many_dumps
+    # If there are too many dumps, return ${EMPTY}, otherwise Fail.
+    ${status}=  Run Keyword If  ${too_many_dumps}  Set Variable  ${EMPTY}
+    ...  ELSE  Fail  msg=${exception}.
+
+    [Return]  ${status}
+
+
 Verify No Dump In Progress
     [Documentation]  Verify no dump in progress.
 
diff --git a/tests/test_bmc_dump.robot b/tests/test_bmc_dump.robot
index 9686d9c..109eb95 100644
--- a/tests/test_bmc_dump.robot
+++ b/tests/test_bmc_dump.robot
@@ -27,10 +27,11 @@
     ...  verify dump entry for it.
     [Tags]  Verify_User_Initiated_BMC_Dump_When_Powered_Off
 
-    REST Power Off  stack_mode=skip  quiet=1
+    REST Power Off  stack_mode=skip
     ${dump_id}=  Create User Initiated Dump
     Check Existence of BMC Dump file  ${dump_id}
 
+
 Verify Dump Persistency On Service Restart
     [Documentation]  Create user dump, restart BMC service and verify dump
     ...  persistency.
@@ -125,7 +126,7 @@
     ...  verify dump entry for it.
     [Tags]  Verify_User_Initiated_BMC_Dump_When_Host_Booted
 
-    REST Power On  stack_mode=skip  quiet=1
+    REST Power On  stack_mode=skip
     Create User Initiated Dump
 
 
@@ -144,6 +145,33 @@
     Should Be True  0 < ${dump_size} < 204800  msg=Size of dump is incorrect.
 
 
+Dump Out Of Space Test
+    [Documentation]  Verify out of dump space is reported when attempt
+    ...  to create too many dumps.
+    [Tags]  Dump_Out_Of_Space_Test
+
+    # Systems typically hold 8-14 dumps before running out of dump space.
+    # Attempt to create too_many_dumps.  Expect to run out of space
+    # before this.
+    ${too_many_dumps}  Set Variable  ${100}
+
+    # Should be able to create at least this many dumps.
+    ${minimum_number_of_dumps}   Set Variable  ${7}
+
+    # Loop, creating a dump each iteration.  Will either get dump_id or
+    # will get EMPTY when out of dump space.
+    :FOR  ${n}  IN RANGE  ${too_many_dumps}
+    \  ${dump_id}=  Create User Initiated Dump  check_out_of_space=${True}
+    \  Exit For Loop If  '${dump_id}' == '${EMPTY}'
+    \  Check Existence of BMC Dump file  ${dump_id}
+
+    Run Keyword If  '${dump_id}' != '${EMPTY}'  Fail
+    ...  msg=Did not run out of dump space as expected.
+
+    Run Keyword If  ${n} < ${minimum_number_of_dumps}  Fail
+    ...  msg=Insufficient space for at least ${minimum_number_of_dumps} dumps.
+
+
 Post Dump BMC Performance Test
     [Documentation]  Check performance of memory, CPU & file system of BMC.
     [Tags]  Post_Dump_BMC_Performance_Test
@@ -151,6 +179,7 @@
     Open Connection And Log In
     Check BMC Performance
 
+
 Post Dump Core Dump Check
     [Documentation]  Check core dump existence on BMC after code update.
     [Tags]  Post_Dump_Core_Dump_Check