Brad Bishop | d7bf8c1 | 2018-02-25 22:55:05 -0500 | [diff] [blame] | 1 | = OEQA (v2) Framework = |
Brad Bishop | 6e60e8b | 2018-02-01 10:27:11 -0500 | [diff] [blame] | 2 | |
| 3 | == Introduction == |
| 4 | |
Brad Bishop | d7bf8c1 | 2018-02-25 22:55:05 -0500 | [diff] [blame] | 5 | This is version 2 of the OEQA framework. Base clases are located in the |
| 6 | 'oeqa/core' directory and subsequent components must extend from these. |
Brad Bishop | 6e60e8b | 2018-02-01 10:27:11 -0500 | [diff] [blame] | 7 | |
Brad Bishop | d7bf8c1 | 2018-02-25 22:55:05 -0500 | [diff] [blame] | 8 | The main design consideration was to implement the needed functionality on |
| 9 | top of the Python unittest framework. To achieve this goal, the following |
| 10 | modules are used: |
Brad Bishop | 6e60e8b | 2018-02-01 10:27:11 -0500 | [diff] [blame] | 11 | |
Brad Bishop | d7bf8c1 | 2018-02-25 22:55:05 -0500 | [diff] [blame] | 12 | * oeqa/core/runner.py: Provides OETestResult and OETestRunner base |
| 13 | classes extending the unittest class. These classes support exporting |
| 14 | results to different formats; currently RAW and XML support exist. |
Brad Bishop | 6e60e8b | 2018-02-01 10:27:11 -0500 | [diff] [blame] | 15 | |
Brad Bishop | d7bf8c1 | 2018-02-25 22:55:05 -0500 | [diff] [blame] | 16 | * oeqa/core/loader.py: Provides OETestLoader extending the unittest class. |
| 17 | It also features a unified implementation of decorator support and |
| 18 | filtering test cases. |
Brad Bishop | 6e60e8b | 2018-02-01 10:27:11 -0500 | [diff] [blame] | 19 | |
Brad Bishop | d7bf8c1 | 2018-02-25 22:55:05 -0500 | [diff] [blame] | 20 | * oeqa/core/case.py: Provides OETestCase base class extending |
| 21 | unittest.TestCase and provides access to the Test data (td), Test context |
| 22 | and Logger functionality. |
Brad Bishop | 6e60e8b | 2018-02-01 10:27:11 -0500 | [diff] [blame] | 23 | |
Brad Bishop | d7bf8c1 | 2018-02-25 22:55:05 -0500 | [diff] [blame] | 24 | * oeqa/core/decorator: Provides OETestDecorator, a new class to implement |
| 25 | decorators for Test cases. |
Brad Bishop | 6e60e8b | 2018-02-01 10:27:11 -0500 | [diff] [blame] | 26 | |
Brad Bishop | d7bf8c1 | 2018-02-25 22:55:05 -0500 | [diff] [blame] | 27 | * oeqa/core/context: Provides OETestContext, a high-level API for |
| 28 | loadTests and runTests of certain Test component and |
| 29 | OETestContextExecutor a base class to enable oe-test to discover/use |
| 30 | the Test component. |
Brad Bishop | 6e60e8b | 2018-02-01 10:27:11 -0500 | [diff] [blame] | 31 | |
Brad Bishop | d7bf8c1 | 2018-02-25 22:55:05 -0500 | [diff] [blame] | 32 | Also, a new 'oe-test' runner is located under 'scripts', allowing scans for components |
| 33 | that supports OETestContextExecutor (see below). |
Brad Bishop | 6e60e8b | 2018-02-01 10:27:11 -0500 | [diff] [blame] | 34 | |
Brad Bishop | d7bf8c1 | 2018-02-25 22:55:05 -0500 | [diff] [blame] | 35 | == Terminology == |
| 36 | |
| 37 | * Test component: The area of testing in the Project, for example: runtime, SDK, eSDK, selftest. |
| 38 | |
| 39 | * Test data: Data associated with the Test component. Currently we use bitbake datastore as |
| 40 | a Test data input. |
| 41 | |
| 42 | * Test context: A context of what tests needs to be run and how to do it; this additionally |
| 43 | provides access to the Test data and could have custom methods and/or attrs. |
| 44 | |
| 45 | == oe-test == |
| 46 | |
| 47 | The new tool, oe-test, has the ability to scan the code base for test components and provide |
| 48 | a unified way to run test cases. Internally it scans folders inside oeqa module in order to find |
| 49 | specific classes that implement a test component. |
| 50 | |
| 51 | == Usage == |
| 52 | |
| 53 | Executing the example test component |
| 54 | |
| 55 | $ source oe-init-build-env |
| 56 | $ oe-test core |
| 57 | |
| 58 | Getting help |
| 59 | |
| 60 | $ oe-test -h |
| 61 | |
| 62 | == Creating new Test Component == |
| 63 | |
| 64 | Adding a new test component the developer needs to extend OETestContext/OETestContextExecutor |
| 65 | (from context.py) and OETestCase (from case.py) |
| 66 | |
| 67 | == Selftesting the framework == |
Brad Bishop | 6e60e8b | 2018-02-01 10:27:11 -0500 | [diff] [blame] | 68 | |
| 69 | Run all tests: |
| 70 | |
Brad Bishop | d7bf8c1 | 2018-02-25 22:55:05 -0500 | [diff] [blame] | 71 | $ PATH=$PATH:../../ python3 -m unittest discover -s tests |
Brad Bishop | 6e60e8b | 2018-02-01 10:27:11 -0500 | [diff] [blame] | 72 | |
| 73 | Run some test: |
| 74 | |
Brad Bishop | d7bf8c1 | 2018-02-25 22:55:05 -0500 | [diff] [blame] | 75 | $ cd tests/ |
| 76 | $ ./test_data.py |