tree acbed54411958d3e4fdc89db96f53edfd75c2108
parent ca8c4c61ddcbe1766d7e33b1a6780506dff294cf
author Michael Walsh <micwalsh@us.ibm.com> 1475594841 -0500
committer Andrew Geissler <geissonator@yahoo.com> 1476474007 +0000

Providing plug-in support:

Typically, a test program is written to perform certain basic tests on a test
machine.  For example, one might write an "obmc_boot" program that performs
various boot tests on the Open BMC machine.

Experience has shown that over time, additional testing needs often arise.
Examples of such additional testing needs might include:

- Data base logging of results
- Performance measurements
- Memory leak analysis
- Hardware verification
- Error log (sels) analysis
- SOL_console

The developer could add additional parms to obmc_boot and likewise add
supporting code in obmc_boot each time a need arises.  Users would employ
these new functions as follows:

obmc_boot --perf=1 --mem_leak=1 --db_logging=1 --db_userid=xxxx

However, another option would be to add general-purpose plug-in support to
obmc_boot.  This would allow the user to indicate to obmc_boot which plug-in
packages it ought to run.  Such plug-in packages could be written in any
langauge whatsoever: Robot, python, bash, perl, C++.

An example call to obmc_boot would then look something like this:

obmc_boot --plug_in_dir_paths="Perf:Mem_leak:DB_logging"

Now all the obmc_boot developer needs to do is call the plug-in processing
module (process_plug_in_packages.py) at various call points which are agreed
upon by the obmc_boot developer and the plug-in developers.  Example call
points which can be implemented are:

setup - Called at the start of obmc_boot
pre_boot - Called before each boot test initiated by obmc_boot
post_boot - Called after each boot test initiated by obmc_boot
cleanup - Called at the end of obmc_boot

This allows the choice of options to be passed as data to obmc_boot.  The
advantages of this approach are:
- Much less maintenance of the original test program (obmc_boot).
- Since plug-ins are separate from the main test program, users are free to
have plug-ins that suit their environments.  One user may wish to log results
to a database that is of no interest to the rest of the world.  Such a plug-in
can be written and need never be pushed to gerrit/github.
- One can even write temporary plug-ins designed just to collect data or stop
when a particular defect occurs.

In our current environment, the concept has proven exceedingly useful.  We
have over 40 permanent plug-ins and in our temp plug-in directory, we still
have over 80 plug-ins.

Change-Id: Iee0ea950cffaef202d56da4dae7c044b6366a59c
Signed-off-by: Michael Walsh <micwalsh@us.ibm.com>