Providing plug-in support:

Typically, a test program is written to perform certain basic tests on a test
machine.  For example, one might write an "obmc_boot" program that performs
various boot tests on the Open BMC machine.

Experience has shown that over time, additional testing needs often arise.
Examples of such additional testing needs might include:

- Data base logging of results
- Performance measurements
- Memory leak analysis
- Hardware verification
- Error log (sels) analysis
- SOL_console

The developer could add additional parms to obmc_boot and likewise add
supporting code in obmc_boot each time a need arises.  Users would employ
these new functions as follows:

obmc_boot --perf=1 --mem_leak=1 --db_logging=1 --db_userid=xxxx

However, another option would be to add general-purpose plug-in support to
obmc_boot.  This would allow the user to indicate to obmc_boot which plug-in
packages it ought to run.  Such plug-in packages could be written in any
langauge whatsoever: Robot, python, bash, perl, C++.

An example call to obmc_boot would then look something like this:

obmc_boot --plug_in_dir_paths="Perf:Mem_leak:DB_logging"

Now all the obmc_boot developer needs to do is call the plug-in processing
module (process_plug_in_packages.py) at various call points which are agreed
upon by the obmc_boot developer and the plug-in developers.  Example call
points which can be implemented are:

setup - Called at the start of obmc_boot
pre_boot - Called before each boot test initiated by obmc_boot
post_boot - Called after each boot test initiated by obmc_boot
cleanup - Called at the end of obmc_boot

This allows the choice of options to be passed as data to obmc_boot.  The
advantages of this approach are:
- Much less maintenance of the original test program (obmc_boot).
- Since plug-ins are separate from the main test program, users are free to
have plug-ins that suit their environments.  One user may wish to log results
to a database that is of no interest to the rest of the world.  Such a plug-in
can be written and need never be pushed to gerrit/github.
- One can even write temporary plug-ins designed just to collect data or stop
when a particular defect occurs.

In our current environment, the concept has proven exceedingly useful.  We
have over 40 permanent plug-ins and in our temp plug-in directory, we still
have over 80 plug-ins.

Change-Id: Iee0ea950cffaef202d56da4dae7c044b6366a59c
Signed-off-by: Michael Walsh <micwalsh@us.ibm.com>
diff --git a/bin/process_plug_in_packages.py b/bin/process_plug_in_packages.py
new file mode 100755
index 0000000..f1bf3d9
--- /dev/null
+++ b/bin/process_plug_in_packages.py
@@ -0,0 +1,348 @@
+#!/usr/bin/env python
+
+import sys
+import __builtin__
+import subprocess
+import os
+import argparse
+
+# python puts the program's directory path in sys.path[0].  In other words,
+# the user ordinarily has no way to override python's choice of a module from
+# its own dir.  We want to have that ability in our environment.  However, we
+# don't want to break any established python modules that depend on this
+# behavior.  So, we'll save the value from sys.path[0], delete it, import our
+# modules and then restore sys.path to its original value.
+
+save_path_0 = sys.path[0]
+del sys.path[0]
+
+from gen_print import *
+from gen_valid import *
+from gen_arg import *
+from gen_plug_in import *
+
+# Restore sys.path[0].
+sys.path.insert(0, save_path_0)
+# I use this variable in calls to print_var.
+hex = 1
+
+###############################################################################
+# Create parser object to process command line parameters and args.
+
+# Create parser object.
+parser = argparse.ArgumentParser(
+    usage='%(prog)s [OPTIONS]',
+    description="%(prog)s will process the plug-in packages passed to it." +
+                "  A plug-in package is essentially a directory containing" +
+                " one or more call point programs.  Each of these call point" +
+                " programs must have a prefix of \"cp_\".  When calling" +
+                " %(prog)s, a user must provide a call_point parameter" +
+                " (described below).  For each plug-in package passed," +
+                " %(prog)s will check for the presence of the specified call" +
+                " point program in the plug-in directory.  If it is found," +
+                " %(prog)s will run it.  It is the responsibility of the" +
+                " caller to set any environment variables needed by the call" +
+                " point programs.\n\nAfter each call point program" +
+                " has been run, %(prog)s will print the following values in" +
+                " the following formats for use by the calling program:\n" +
+                "  failed_plug_in_name:               <failed plug-in value," +
+                " if any>\n  shell_rc:                          " +
+                "<shell return code value of last call point program - this" +
+                " will be printed in hexadecimal format.  Also, be aware" +
+                " that if a call point program returns a value it will be" +
+                " shifted left 2 bytes (e.g. rc of 2 will be printed as" +
+                " 0x00000200).  That is because the rightmost byte is" +
+                " reserverd for errors in calling the call point program" +
+                " rather than errors generated by the call point program.>",
+    formatter_class=argparse.RawTextHelpFormatter,
+    prefix_chars='-+'
+    )
+
+# Create arguments.
+parser.add_argument(
+    'plug_in_dir_paths',
+    nargs='?',
+    default="",
+    help=plug_in_dir_paths_help_text + default_string
+    )
+
+parser.add_argument(
+    '--call_point',
+    default="setup",
+    required=True,
+    help='The call point program name.  This value must not include the' +
+         ' "cp_" prefix.  For each plug-in package passed to this program,' +
+         ' the specified call_point program will be called if it exists in' +
+         ' the plug-in directory.' + default_string
+    )
+
+parser.add_argument(
+    '--shell_rc',
+    default="0x00000000",
+    help='The user may supply a value other than zero to indicate an' +
+         ' acceptable non-zero return code.  For example, if this value' +
+         ' equals 0x00000200, it means that for each plug-in call point that' +
+         ' runs, a 0x00000200 will not be counted as a failure.  See note' +
+         ' above regarding left-shifting of return codes.' + default_string
+    )
+
+parser.add_argument(
+    '--stop_on_plug_in_failure',
+    default=1,
+    type=int,
+    choices=[1, 0],
+    help='If this parameter is set to 1, this program will stop and return ' +
+         'non-zero if the call point program from any plug-in directory ' +
+         'fails.  Conversely, if it is set to false, this program will run ' +
+         'the call point program from each and every plug-in directory ' +
+         'regardless of their return values.  Typical example cases where ' +
+         'you\'d want to run all plug-in call points regardless of success ' +
+         'or failure would be "cleanup" or "ffdc" call points.'
+    )
+
+parser.add_argument(
+    '--stop_on_non_zero_rc',
+    default=0,
+    type=int,
+    choices=[1, 0],
+    help='If this parm is set to 1 and a plug-in call point program returns ' +
+         'a valid non-zero return code (see "shell_rc" parm above), this' +
+         ' program will stop processing and return 0 (success).  Since this' +
+         ' constitutes a successful exit, this would normally be used where' +
+         ' the caller wishes to stop processing if one of the plug-in' +
+         ' directory call point programs returns a special value indicating' +
+         ' that some special case has been found.  An example might be in' +
+         ' calling some kind of "check_errl" call point program.  Such a' +
+         ' call point program might return a 2 (i.e. 0x00000200) to indicate' +
+         ' that a given error log entry was found in an "ignore" list and is' +
+         ' therefore to be ignored.  That being the case, no other' +
+         ' "check_errl" call point program would need to be called.' +
+         default_string
+    )
+
+parser.add_argument(
+    '--mch_class',
+    default="obmc",
+    help=mch_class_help_text + default_string
+    )
+
+# The stock_list will be passed to gen_get_options.  We populate it with the
+# names of stock parm options we want.  These stock parms are pre-defined by
+# gen_get_options.
+stock_list = [("test_mode", 0), ("quiet", 1), ("debug", 0)]
+###############################################################################
+
+
+###############################################################################
+def exit_function(signal_number=0,
+                  frame=None):
+
+    r"""
+    Execute whenever the program ends normally or with the signals that we
+    catch (i.e. TERM, INT).
+    """
+
+    dprint_executing()
+    dprint_var(signal_number)
+
+    qprint_pgm_footer()
+
+###############################################################################
+
+
+###############################################################################
+def signal_handler(signal_number, frame):
+
+    r"""
+    Handle signals.  Without a function to catch a SIGTERM or SIGINT, our
+    program would terminate immediately with return code 143 and without
+    calling our exit_function.
+    """
+
+    # Our convention is to set up exit_function with atexit.registr() so
+    # there is no need to explicitly call exit_function from here.
+
+    dprint_executing()
+
+    # Calling exit prevents us from returning to the code that was running
+    # when we received the signal.
+    exit(0)
+
+###############################################################################
+
+
+###############################################################################
+def validate_parms():
+
+    r"""
+    Validate program parameters, etc.  Return True or False accordingly.
+    """
+
+    if not valid_value(call_point):
+        return False
+
+    gen_post_validation(exit_function, signal_handler)
+
+    return True
+
+###############################################################################
+
+
+###############################################################################
+def run_pgm(plug_in_dir_path,
+            call_point,
+            caller_shell_rc):
+
+    r"""
+    Run the call point program in the given plug_in_dir_path.  Return the
+    following:
+    rc                              The return code - 0 = PASS, 1 = FAIL.
+    shell_rc                        The shell return code returned by
+                                    process_plug_in_packages.py.
+    failed_plug_in_name             The failed plug in name (if any).
+
+    Description of arguments:
+    plug_in_dir_path                The directory path where the call_point
+                                    program may be located.
+    call_point                      The call point (e.g. "setup").  This
+                                    program will look for a program named
+                                    "cp_" + call_point in the
+                                    plug_in_dir_path.  If no such call point
+                                    program is found, this function returns an
+                                    rc of 0 (i.e. success).
+    caller_shell_rc                 The user may supply a value other than
+                                    zero to indicate an acceptable non-zero
+                                    return code.  For example, if this value
+                                    equals 0x00000200, it means that for each
+                                    plug-in call point that runs, a 0x00000200
+                                    will not be counted as a failure.  See
+                                    note above regarding left-shifting of
+                                    return codes.
+    """
+
+    rc = 0
+    failed_plug_in_name = ""
+    shell_rc = 0x00000000
+
+    cp_prefix = "cp_"
+    plug_in_pgm_path = plug_in_dir_path + cp_prefix + call_point
+    if not os.path.exists(plug_in_pgm_path):
+        # No such call point in this plug in dir path.  This is legal so we
+        # return 0, etc.
+        return rc, shell_rc, failed_plug_in_name
+
+    # Get some stats on the file.
+    cmd_buf = "stat -c '%n %s %z' " + plug_in_pgm_path
+    dissuing(cmd_buf)
+    sub_proc = subprocess.Popen(cmd_buf, shell=True, stdout=subprocess.PIPE,
+        stderr=subprocess.STDOUT)
+    out_buf, err_buf = sub_proc.communicate()
+    shell_rc = sub_proc.returncode
+    if shell_rc != 0:
+        rc = 1
+        print_var(shell_rc, hex)
+        failed_plug_in_name = \
+            os.path.basename(os.path.normpath(plug_in_dir_path))
+        print(out_buf)
+        return rc, shell_rc, failed_plug_in_name
+
+    print("------------------------------------------------ Starting plug-in" +
+          " ------------------------------------------------")
+    print(out_buf)
+    cmd_buf = "PATH=" + plug_in_dir_path + ":${PATH} ; " + cp_prefix +\
+              call_point
+    issuing(cmd_buf)
+
+    sub_proc = subprocess.Popen(cmd_buf, shell=True, stdout=subprocess.PIPE,
+        stderr=subprocess.STDOUT)
+    out_buf, err_buf = sub_proc.communicate()
+    shell_rc = sub_proc.returncode
+    if shell_rc != 0 and shell_rc != int(caller_shell_rc, 16):
+        rc = 1
+        failed_plug_in_name = \
+            os.path.basename(os.path.normpath(plug_in_dir_path))
+
+    print(out_buf)
+    if rc == 1 and out_buf.find('**ERROR**') == -1:
+        # Plug-in output contains no "**ERROR**" text so we'll generate it.
+        print_error_report("Plug-in failed.\n")
+    print("------------------------------------------------- Ending plug-in" +
+          " -------------------------------------------------")
+
+    return rc, shell_rc, failed_plug_in_name
+
+###############################################################################
+
+
+###############################################################################
+def main():
+
+    r"""
+    This is the "main" function.  The advantage of having this function vs
+    just doing this in the true mainline is that you can:
+    - Declare local variables
+    - Use "return" instead of "exit".
+    - Indent 4 chars like you would in any function.
+    This makes coding more consistent, i.e. it's easy to move code from here
+    into a function and vice versa.
+    """
+
+    if not gen_get_options(parser, stock_list):
+        return False
+
+    if not validate_parms():
+        return False
+
+    qprint_pgm_header()
+
+    # Access program parameter globals.
+    global plug_in_dir_paths
+    global mch_class
+    global shell_rc
+    global stop_on_plug_in_failure
+    global stop_on_non_zero_rc
+
+    plug_in_packages_list = return_plug_in_packages_list(plug_in_dir_paths,
+                                                         mch_class)
+
+    qpvar(plug_in_packages_list)
+
+    qprint("\n")
+
+    caller_shell_rc = shell_rc
+    failed_plug_in_name = ""
+
+    ret_code = 0
+    for plug_in_dir_path in plug_in_packages_list:
+        rc, shell_rc, failed_plug_in_name = \
+            run_pgm(plug_in_dir_path, call_point, caller_shell_rc)
+        print_var(failed_plug_in_name)
+        print_var(shell_rc, hex)
+        if rc != 0:
+            ret_code = 1
+            if stop_on_plug_in_failure:
+                break
+        if shell_rc != 0 and stop_on_non_zero_rc:
+            qprint_time("Stopping on non-zero shell return code as requested" +
+                        " by caller.\n")
+            break
+
+    if ret_code == 0:
+        return True
+    else:
+        if not stop_on_plug_in_failure:
+            # We print a summary error message to make the failure more
+            # obvious.
+            print_error_report("At least one plug-in failed.\n")
+        return False
+
+###############################################################################
+
+
+###############################################################################
+# Main
+
+if not main():
+    exit(1)
+
+###############################################################################
diff --git a/bin/validate_plug_ins.py b/bin/validate_plug_ins.py
new file mode 100755
index 0000000..d0e541d
--- /dev/null
+++ b/bin/validate_plug_ins.py
@@ -0,0 +1,157 @@
+#!/usr/bin/env python
+
+import sys
+import __builtin__
+import os
+
+# python puts the program's directory path in sys.path[0].  In other words,
+# the user ordinarily has no way to override python's choice of a module from
+# its own dir.  We want to have that ability in our environment.  However, we
+# don't want to break any established python modules that depend on this
+# behavior.  So, we'll save the value from sys.path[0], delete it, import our
+# modules and then restore sys.path to its original value.
+
+save_path_0 = sys.path[0]
+del sys.path[0]
+
+from gen_print import *
+from gen_arg import *
+from gen_plug_in import *
+
+# Restore sys.path[0].
+sys.path.insert(0, save_path_0)
+
+
+###############################################################################
+# Create parser object to process command line parameters and args.
+
+# Create parser object.
+parser = argparse.ArgumentParser(
+    usage='%(prog)s [OPTIONS] [PLUG_IN_DIR_PATHS]',
+    description="%(prog)s will validate the plug-in packages passed to it." +
+                "  It will also print a list of the absolute plug-in" +
+                " directory paths for use by the calling program.",
+    formatter_class=argparse.RawTextHelpFormatter,
+    prefix_chars='-+'
+    )
+
+# Create arguments.
+parser.add_argument(
+    'plug_in_dir_paths',
+    nargs='?',
+    default="",
+    help=plug_in_dir_paths_help_text + default_string
+    )
+
+parser.add_argument(
+    '--mch_class',
+    default="obmc",
+    help=mch_class_help_text + default_string
+    )
+
+# The stock_list will be passed to gen_get_options.  We populate it with the
+# names of stock parm options we want.  These stock parms are pre-defined by
+# gen_get_options.
+stock_list = [("test_mode", 0), ("quiet", 1), ("debug", 0)]
+
+###############################################################################
+
+
+###############################################################################
+def exit_function(signal_number=0,
+                  frame=None):
+
+    r"""
+    Execute whenever the program ends normally or with the signals that we
+    catch (i.e. TERM, INT).
+    """
+
+    dprint_executing()
+    dprint_var(signal_number)
+
+    qprint_pgm_footer()
+
+###############################################################################
+
+
+###############################################################################
+def signal_handler(signal_number, frame):
+
+    r"""
+    Handle signals.  Without a function to catch a SIGTERM or SIGINT, our
+    program would terminate immediately with return code 143 and without
+    calling our exit_function.
+    """
+
+    # Our convention is to set up exit_function with atexit.registr() so
+    # there is no need to explicitly call exit_function from here.
+
+    dprint_executing()
+
+    # Calling exit prevents us from returning to the code that was running
+    # when we received the signal.
+    exit(0)
+
+###############################################################################
+
+
+###############################################################################
+def validate_parms():
+
+    r"""
+    Validate program parameters, etc.  Return True or False accordingly.
+    """
+
+    gen_post_validation(exit_function, signal_handler)
+
+    return True
+
+###############################################################################
+
+
+###############################################################################
+def main():
+
+    r"""
+    This is the "main" function.  The advantage of having this function vs
+    just doing this in the true mainline is that you can:
+    - Declare local variables
+    - Use "return" instead of "exit".
+    - Indent 4 chars like you would in any function.
+    This makes coding more consistent, i.e. it's easy to move code from here
+    into a function and vice versa.
+    """
+
+    if not gen_get_options(parser, stock_list):
+        return False
+
+    if not validate_parms():
+        return False
+
+    qprint_pgm_header()
+
+    # Access program parameter globals.
+    global plug_in_dir_paths
+    global mch_class
+
+    plug_in_packages_list = return_plug_in_packages_list(plug_in_dir_paths,
+                                                         mch_class)
+    qpvar(plug_in_packages_list)
+
+    # As stated in the help text, this program must print the full paths of
+    # each selected plug in.
+    for plug_in_dir_path in plug_in_packages_list:
+        print(plug_in_dir_path)
+
+    return True
+
+###############################################################################
+
+
+###############################################################################
+# Main
+
+if not main():
+    exit(1)
+
+###############################################################################
diff --git a/lib/gen_arg.py b/lib/gen_arg.py
new file mode 100755
index 0000000..1a57411
--- /dev/null
+++ b/lib/gen_arg.py
@@ -0,0 +1,216 @@
+#!/usr/bin/env python
+
+r"""
+This module provides valuable argument processing functions like
+gen_get_options and sprint_args.
+"""
+
+import sys
+import __builtin__
+import atexit
+import signal
+import argparse
+
+import gen_print as gp
+
+default_string = '  The default value is "%(default)s".'
+
+
+###############################################################################
+def gen_get_options(parser,
+                    stock_list=[]):
+
+    r"""
+    Parse the command line arguments using the parser object passed and return
+    True/False (i.e. pass/fail).  Also set the following built in values:
+
+    __builtin__.quiet      This value is used by the qprint functions.
+    __builtin__.test_mode  This value is used by command processing functions.
+    __builtin__.debug      This value is used by the dprint functions.
+    __builtin__.arg_obj    This value is used by print_program_header, etc.
+    __builtin__.parser     This value is used by print_program_header, etc.
+
+    Description of arguments:
+    parser                          A parser object.  See argparse module
+                                    documentation for details.
+    stock_list                      The caller can use this parameter to
+                                    request certain stock parameters offered
+                                    by this function.  For example, this
+                                    function will define a "quiet" option upon
+                                    request.  This includes stop help text and
+                                    parm checking.  The stock_list is a list
+                                    of tuples each of which consists of an
+                                    arg_name and a default value.  Example:
+                                    stock_list = [("test_mode", 0), ("quiet",
+                                    1), ("debug", 0)]
+    """
+
+    # This is a list of stock parms that we support.
+    master_stock_list = ["quiet", "test_mode", "debug", "loglevel"]
+
+    # Process stock_list.
+    for ix in range(0, len(stock_list)):
+        if len(stock_list[ix]) < 1:
+            gp.print_error_report("Programmer error - stock_list[" + str(ix) +
+                                  "] is supposed to be a tuple containing at" +
+                                  " least one element which is the name of" +
+                                  " the desired stock parameter:\n" +
+                                  gp.sprint_var(stock_list))
+            return False
+        if type(stock_list[ix]) is tuple:
+            arg_name = stock_list[ix][0]
+            default = stock_list[ix][1]
+        else:
+            arg_name = stock_list[ix]
+            default = None
+
+        if arg_name not in master_stock_list:
+            gp.pvar(arg_name)
+            gp.print_error_report("Programmer error - \"" + arg_name +
+                                  "\" not found found in stock list:\n" +
+                                  gp.sprint_var(master_stock_list))
+            return False
+
+        if arg_name == "quiet":
+            if default is None:
+                default = 0
+            parser.add_argument(
+                '--quiet',
+                default=default,
+                type=int,
+                choices=[1, 0],
+                help='If this parameter is set to "1", %(prog)s' +
+                     ' will print only essential information, i.e. it will' +
+                     ' not echo parameters, echo commands, print the total' +
+                     ' run time, etc.' + default_string
+                )
+        elif arg_name == "test_mode":
+            if default is None:
+                default = 0
+            parser.add_argument(
+                '--test_mode',
+                default=default,
+                type=int,
+                choices=[1, 0],
+                help='This means that %(prog)s should go through all the' +
+                     ' motions but not actually do anything substantial.' +
+                     '  This is mainly to be used by the developer of' +
+                     ' %(prog)s.' + default_string
+                )
+        elif arg_name == "debug":
+            if default is None:
+                default = 0
+            parser.add_argument(
+                '--debug',
+                default=default,
+                type=int,
+                choices=[1, 0],
+                help='If this parameter is set to "1", %(prog)s will print' +
+                     ' additional debug information.  This is mainly to be' +
+                     ' used by the developer of %(prog)s.' + default_string
+                )
+        elif arg_name == "loglevel":
+            if default is None:
+                default = "info"
+            parser.add_argument(
+                '--loglevel',
+                default=default,
+                type=str,
+                choices=['DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL',
+                         'debug', 'info', 'warning', 'error', 'critical'],
+                help='If this parameter is set to "1", %(prog)s will print' +
+                     ' additional debug information.  This is mainly to be' +
+                     ' used by the developer of %(prog)s.' + default_string
+                )
+
+    arg_obj = parser.parse_args()
+
+    __builtin__.quiet = 0
+    __builtin__.test_mode = 0
+    __builtin__.debug = 0
+    __builtin__.loglevel = 'WARNING'
+    for ix in range(0, len(stock_list)):
+        if type(stock_list[ix]) is tuple:
+            arg_name = stock_list[ix][0]
+            default = stock_list[ix][1]
+        else:
+            arg_name = stock_list[ix]
+            default = None
+        if arg_name == "quiet":
+            __builtin__.quiet = arg_obj.quiet
+        elif arg_name == "test_mode":
+            __builtin__.test_mode = arg_obj.test_mode
+        elif arg_name == "debug":
+            __builtin__.debug = arg_obj.debug
+        elif arg_name == "loglevel":
+            __builtin__.loglevel = arg_obj.loglevel
+
+    __builtin__.arg_obj = arg_obj
+    __builtin__.parser = parser
+
+    # For each command line parameter, create a corresponding global variable
+    # and assign it the appropriate value.  For example, if the command line
+    # contained "--last_name='Smith', we'll create a global variable named
+    # "last_name" with the value "Smith".
+    module = sys.modules['__main__']
+    for key in arg_obj.__dict__:
+        setattr(module, key, getattr(__builtin__.arg_obj, key))
+
+    return True
+
+###############################################################################
+
+
+# Put this in gen_opt.py or gen_parm.py or gen_arg.py.
+###############################################################################
+def sprint_args(arg_obj,
+                indent=0):
+
+    r"""
+    sprint_var all of the arguments found in arg_obj and return the result as
+    a string.
+
+    Description of arguments:
+    arg_obj                         An argument object such as is returned by
+                                    the argparse parse_args() method.
+    indent                          The number of spaces to indent each line
+                                    of output.
+    """
+
+    buffer = ""
+
+    for key in arg_obj.__dict__:
+        buffer += gp.sprint_varx(key, getattr(arg_obj, key), 0, indent)
+
+    return buffer
+
+###############################################################################
+
+
+###############################################################################
+def gen_post_validation(exit_function=None,
+                        signal_handler=None):
+
+    r"""
+    Do generic post-validation processing.  By "post", we mean that this is to
+    be called from a validation function after the caller has done any
+    validation desired.  If the calling program passes exit_function and
+    signal_handler parms, this function will register them.  In other words,
+    it will make the signal_handler functions get called for SIGINT and
+    SIGTERM and will make the exit_function function run prior to the
+    termination of the program.
+
+    Description of arguments:
+    exit_function                   A function object pointing to the caller's
+                                    exit function.
+    signal_handler                  A function object pointing to the caller's
+                                    signal_handler function.
+    """
+
+    if exit_function is not None:
+        atexit.register(exit_function)
+    if signal_handler is not None:
+        signal.signal(signal.SIGINT, signal_handler)
+        signal.signal(signal.SIGTERM, signal_handler)
+
+###############################################################################
diff --git a/lib/gen_misc.py b/lib/gen_misc.py
index 167a3d9..30ab443 100755
--- a/lib/gen_misc.py
+++ b/lib/gen_misc.py
@@ -1,6 +1,8 @@
 #!/usr/bin/env python
 
-# This module provides many valuable functions such as my_parm_file.
+r"""
+This module provides many valuable functions such as my_parm_file.
+"""
 
 # sys and os are needed to get the program dir path and program name.
 import sys
@@ -8,6 +10,8 @@
 import ConfigParser
 import StringIO
 
+import gen_print as gp
+
 
 ###############################################################################
 def my_parm_file(prop_file_path):
@@ -26,7 +30,8 @@
     This one
 
     Description of arguments:
-    prop_file_path  The caller should pass the path to the properties file.
+    prop_file_path                  The caller should pass the path to the
+                                    properties file.
     """
 
     # ConfigParser expects at least one section header in the file (or you
@@ -51,3 +56,20 @@
     return dict(config_parser.items('dummysection'))
 
 ###############################################################################
+
+
+###############################################################################
+def return_path_list():
+
+    r"""
+    This function will split the PATH environment variable into a PATH_LIST
+    and return it.  Each element in the list will be normalized and have a
+    trailing slash added.
+    """
+
+    PATH_LIST = os.environ['PATH'].split(":")
+    PATH_LIST = [os.path.normpath(path) + os.sep for path in PATH_LIST]
+
+    return PATH_LIST
+
+###############################################################################
diff --git a/lib/gen_plug_in.py b/lib/gen_plug_in.py
new file mode 100755
index 0000000..4c90473
--- /dev/null
+++ b/lib/gen_plug_in.py
@@ -0,0 +1,217 @@
+#!/usr/bin/env python
+
+r"""
+This module provides functions which are useful for running plug-ins.
+"""
+
+import sys
+import os
+import commands
+import glob
+
+import gen_print as gp
+import gen_misc as gm
+
+# Some help text that is common to more than one program.
+plug_in_dir_paths_help_text = \
+    'This is a colon-separated list of plug-in directory paths.  If one' +\
+    ' of the entries in the list is a plain directory name (i.e. no' +\
+    ' path info), it will be taken to be a native plug-in.  In that case,' +\
+    ' %(prog)s will search for the native plug-in in the "plug-ins"' +\
+    ' subdirectory of each path in the PATH environment variable until it' +\
+    ' is found.  Also, integrated plug-ins will automatically be appended' +\
+    ' to your plug_in_dir_paths list.  An integrated plug-in is any plug-in' +\
+    ' found using the PATH variable that contains a file named "integrated".'
+
+mch_class_help_text = \
+    'The class of machine that we are testing (e.g. "op" = "open power",' +\
+    ' "obmc" = "open bmc", etc).'
+
+PATH_LIST = gm.return_path_list()
+
+
+###############################################################################
+def get_plug_in_base_paths():
+
+    r"""
+    Get plug-in base paths and return them as a list.
+
+    This function searches the PATH_LIST (created from PATH environment
+    variable) for any paths that have a "plug_ins" subdirectory.  All such
+    paths are considered plug_in_base paths.
+    """
+
+    global PATH_LIST
+
+    plug_in_base_path_list = []
+
+    for path in PATH_LIST:
+        candidate_plug_in_base_path = path + "plug_ins/"
+        if os.path.isdir(candidate_plug_in_base_path):
+            plug_in_base_path_list.append(candidate_plug_in_base_path)
+
+    return plug_in_base_path_list
+
+###############################################################################
+# Define global plug_in_base_path_list and call get_plug_in_base_paths to set
+# its value.
+plug_in_base_path_list = get_plug_in_base_paths()
+
+
+###############################################################################
+def find_plug_in_package(plug_in_name):
+
+    r"""
+    Find and return the normalized directory path of the specified plug in.
+    This is done by searching the global plug_in_base_path_list.
+
+    Description of arguments:
+    plug_in_name                    The unqualified name of the plug-in
+                                    package.
+    """
+
+    global plug_in_base_path_list
+    for plug_in_base_dir_path in plug_in_base_path_list:
+        candidate_plug_in_dir_path = os.path.normpath(plug_in_base_dir_path +
+                                                      plug_in_name) + \
+                                                      os.sep
+        if os.path.isdir(candidate_plug_in_dir_path):
+            return candidate_plug_in_dir_path
+
+    return ""
+
+###############################################################################
+
+
+###############################################################################
+def validate_plug_in_package(plug_in_dir_path,
+                             mch_class="obmc"):
+
+    r"""
+    Validate the plug in package and return the normalized plug-in directory
+    path.
+
+    Description of arguments:
+    plug_in_dir_path                The "relative" or absolute path to a plug
+                                    in package directory.
+    mch_class                       The class of machine that we are testing
+                                    (e.g. "op" = "open power", "obmc" = "open
+                                    bmc", etc).
+    """
+
+    gp.dprint_executing()
+
+    if os.path.isabs(plug_in_dir_path):
+        # plug_in_dir_path begins with a slash so it is an absolute path.
+        candidate_plug_in_dir_path = os.path.normpath(plug_in_dir_path) +\
+                                     os.sep
+        if not os.path.isdir(candidate_plug_in_dir_path):
+            gp.print_error_report("Plug-in directory path \"" +
+                                  plug_in_dir_path + "\" does not exist.\n")
+            exit(1)
+    else:
+        # The plug_in_dir_path is actually a simple name (e.g.
+        # "OBMC_Sample")...
+        candidate_plug_in_dir_path = find_plug_in_package(plug_in_dir_path)
+        if candidate_plug_in_dir_path == "":
+            global PATH_LIST
+            gp.print_error_report("Plug-in directory path \"" +
+                                  plug_in_dir_path + "\" could not be found" +
+                                  " in any of the following directories:\n" +
+                                  gp.sprint_var(PATH_LIST))
+            exit(1)
+    # Make sure that this plug-in supports us...
+    supports_file_path = candidate_plug_in_dir_path + "supports_" + mch_class
+    if not os.path.exists(supports_file_path):
+        gp.print_error_report("The following file path could not be" +
+                              " found:\n" +
+                              gp.sprint_varx("supports_file_path",
+                                             supports_file_path) +
+                              "\nThis file is necessary to indicate that" +
+                              " the given plug-in supports the class of" +
+                              " machine we are testing, namely \"" +
+                              mch_class + "\".\n")
+        exit(1)
+
+    return candidate_plug_in_dir_path
+
+###############################################################################
+
+
+###############################################################################
+def return_integrated_plug_ins(mch_class="obmc"):
+
+    r"""
+    Return a list of integrated plug-ins.  Integrated plug-ins are plug-ins
+    which are selected without regard for whether the user has specified them.
+    In other words, they are "integrated" into the program suite.  The
+    programmer designates a plug-in as integrated by putting a file named
+    "integrated" into the plug-in package directory.
+
+    Description of arguments:
+    mch_class                       The class of machine that we are testing
+                                    (e.g. "op" = "open power", "obmc" = "open
+                                    bmc", etc).
+    """
+
+    global plug_in_base_path_list
+
+    integrated_plug_ins_list = []
+
+    for plug_in_base_path in plug_in_base_path_list:
+        # Get a list of all plug-in paths that support our mch_class.
+        mch_class_candidate_list = glob.glob(plug_in_base_path +
+                                             "*/supports_" + mch_class)
+        for candidate_path in mch_class_candidate_list:
+            integrated_plug_in_dir_path = os.path.dirname(candidate_path) +\
+                                          os.sep
+            integrated_file_path = integrated_plug_in_dir_path + "integrated"
+            if os.path.exists(integrated_file_path):
+                plug_in_name = \
+                    os.path.basename(os.path.dirname(candidate_path))
+                if plug_in_name not in integrated_plug_ins_list:
+                    # If this plug-in has not already been added to the list...
+                    integrated_plug_ins_list.append(plug_in_name)
+
+    return integrated_plug_ins_list
+
+###############################################################################
+
+
+###############################################################################
+def return_plug_in_packages_list(plug_in_dir_paths,
+                                 mch_class="obmc"):
+
+    r"""
+    Return a list of plug-in packages given the plug_in_dir_paths string.
+    This function calls validate_plug_in_package so it will fail if
+    plug_in_dir_paths contains any invalid plug-ins.
+
+    Description of arguments:
+    plug_in_dir_path                The "relative" or absolute path to a plug
+                                    in package directory.
+    mch_class                       The class of machine that we are testing
+                                    (e.g. "op" = "open power", "obmc" = "open
+                                    bmc", etc).
+    """
+
+    if plug_in_dir_paths != "":
+        plug_in_packages_list = plug_in_dir_paths.split(":")
+    else:
+        plug_in_packages_list = []
+
+    # Get a list of integrated plug-ins (w/o full path names).
+    integrated_plug_ins_list = return_integrated_plug_ins(mch_class)
+    # Put both lists together in plug_in_packages_list with no duplicates.
+    # NOTE: This won't catch duplicates if the caller specifies the full path
+    # name of a native plug-in but that should be rare enough.
+
+    plug_in_packages_list = plug_in_packages_list + integrated_plug_ins_list
+
+    plug_in_packages_list = \
+        list(set([validate_plug_in_package(path, mch_class)
+                  for path in plug_in_packages_list]))
+
+    return plug_in_packages_list
+
+###############################################################################
diff --git a/lib/gen_print.py b/lib/gen_print.py
index c4618cc..6310122 100755
--- a/lib/gen_print.py
+++ b/lib/gen_print.py
@@ -13,11 +13,17 @@
 import grp
 import socket
 import argparse
+import __builtin__
+import logging
+
+import gen_arg as ga
 
 # Setting these variables for use both inside this module and by programs
 # importing this module.
 pgm_dir_path = sys.argv[0]
 pgm_name = os.path.basename(pgm_dir_path)
+pgm_dir_name = re.sub("/" + pgm_name, "", pgm_dir_path) + "/"
+
 
 # Some functions (e.g. sprint_pgm_header) have need of a program name value
 # that looks more like a valid variable name.  Therefore, we'll swap odd
@@ -54,6 +60,13 @@
 start_time = time.time()
 sprint_time_last_seconds = start_time
 
+try:
+    # The user can set environment variable "GEN_PRINT_DEBUG" to get debug
+    # output from this module.
+    gen_print_debug = os.environ['GEN_PRINT_DEBUG']
+except KeyError:
+    gen_print_debug = 0
+
 
 ###############################################################################
 def sprint_func_name(stack_frame_ix=None):
@@ -193,13 +206,16 @@
     composite_line = lines[0].strip()
 
     called_func_name = sprint_func_name(stack_frame_ix)
-    # 2016/09/01 Mike Walsh (xzy0065) - I added code to handle pvar alias.
-    # pvar is an alias for print_var.  However, when it is used,
-    # sprint_func_name() returns the non-alias version, i.e. "print_var".
-    # Adjusting for that here.
-    substring = composite_line[0:4]
-    if substring == "pvar":
-        called_func_name = "pvar"
+    if not re.match(r".*" + called_func_name, composite_line):
+        # The called function name was not found in the composite line.  The
+        # caller may be using a function alias.
+        # I added code to handle pvar, qpvar, dpvar, etc. aliases.
+        # pvar is an alias for print_var.  However, when it is used,
+        # sprint_func_name() returns the non-alias version, i.e. "print_var".
+        # Adjusting for that here.
+        alias = re.sub("print_var", "pvar", called_func_name)
+        called_func_name = alias
+
     arg_list_etc = re.sub(".*" + called_func_name, "", composite_line)
     if local_debug:
         print_varx("called_func_name", called_func_name, 0, debug_indent)
@@ -210,7 +226,7 @@
     # Initialize...
     nest_level = -1
     arg_ix = 0
-    args_arr = [""]
+    args_list = [""]
     for ix in range(0, len(arg_list_etc)):
         char = arg_list_etc[ix]
         # Set the nest_level based on whether we've encounted a parenthesis.
@@ -224,30 +240,30 @@
                 break
 
         # If we reach a comma at base nest level, we are done processing an
-        # argument so we increment arg_ix and initialize a new args_arr entry.
+        # argument so we increment arg_ix and initialize a new args_list entry.
         if char == "," and nest_level == 0:
             arg_ix += 1
-            args_arr.append("")
+            args_list.append("")
             continue
 
-        # For any other character, we append it it to the current arg array
+        # For any other character, we append it it to the current arg list
         # entry.
-        args_arr[arg_ix] += char
+        args_list[arg_ix] += char
 
     # Trim whitespace from each list entry.
-    args_arr = [arg.strip() for arg in args_arr]
+    args_list = [arg.strip() for arg in args_list]
 
-    if arg_num > len(args_arr):
+    if arg_num > len(args_list):
         print_error("Programmer error - The caller has asked for the name of" +
                     " argument number \"" + str(arg_num) + "\" but there " +
-                    "were only \"" + str(len(args_arr)) + "\" args used:\n" +
-                    sprint_varx("args_arr", args_arr))
+                    "were only \"" + str(len(args_list)) + "\" args used:\n" +
+                    sprint_varx("args_list", args_list))
         return
 
-    argument = args_arr[arg_num - 1]
+    argument = args_list[arg_num - 1]
 
     if local_debug:
-        print_varx("args_arr", args_arr, 0, debug_indent)
+        print_varx("args_list", args_list, 0, debug_indent)
         print_varx("argument", argument, 0, debug_indent)
 
     return argument
@@ -394,7 +410,8 @@
                 var_value,
                 hex=0,
                 loc_col1_indent=col1_indent,
-                loc_col1_width=col1_width):
+                loc_col1_width=col1_width,
+                trailing_char="\n"):
 
     r"""
     Print the var name/value passed to it.  If the caller lets loc_col1_width
@@ -447,51 +464,73 @@
                                     this is adjusted so that the var_value
                                     lines up with text printed via the
                                     print_time function.
-   """
-
-    # Adjust loc_col1_width.
-    loc_col1_width = loc_col1_width - loc_col1_indent
+    trailing_char                   The character to be used at the end of the
+                                    returned string.  The default value is a
+                                    line feed.
+    """
 
     # Determine the type
     if type(var_value) in (int, float, bool, str, unicode) \
        or var_value is None:
         # The data type is simple in the sense that it has no subordinate
         # parts.
+        # Adjust loc_col1_width.
+        loc_col1_width = loc_col1_width - loc_col1_indent
         # See if the user wants the output in hex format.
         if hex:
             value_format = "0x%08x"
         else:
             value_format = "%s"
         format_string = "%" + str(loc_col1_indent) + "s%-" \
-            + str(loc_col1_width) + "s" + value_format + "\n"
+            + str(loc_col1_width) + "s" + value_format + trailing_char
         return format_string % ("", var_name + ":", var_value)
     else:
         # The data type is complex in the sense that it has subordinate parts.
         format_string = "%" + str(loc_col1_indent) + "s%s\n"
         buffer = format_string % ("", var_name + ":")
         loc_col1_indent += 2
+        try:
+            length = len(var_value)
+        except TypeError:
+            pass
+        ix = 0
+        loc_trailing_char = "\n"
         if type(var_value) is dict:
             for key, value in var_value.iteritems():
+                ix += 1
+                if ix == length:
+                    loc_trailing_char = trailing_char
                 buffer += sprint_varx(var_name + "[" + key + "]", value, hex,
-                                      loc_col1_indent)
-        elif type(var_value) in (list, tuple):
+                                      loc_col1_indent, loc_col1_width,
+                                      loc_trailing_char)
+        elif type(var_value) in (list, tuple, set):
             for key, value in enumerate(var_value):
+                ix += 1
+                if ix == length:
+                    loc_trailing_char = trailing_char
                 buffer += sprint_varx(var_name + "[" + str(key) + "]", value,
-                                      hex, loc_col1_indent)
+                                      hex, loc_col1_indent, loc_col1_width,
+                                      loc_trailing_char)
         elif type(var_value) is argparse.Namespace:
             for key in var_value.__dict__:
+                ix += 1
+                if ix == length:
+                    loc_trailing_char = trailing_char
                 cmd_buf = "buffer += sprint_varx(var_name + \".\" + str(key)" \
-                          + ", var_value." + key + ", hex, loc_col1_indent)"
+                          + ", var_value." + key + ", hex, loc_col1_indent," \
+                          + " loc_col1_width, loc_trailing_char)"
                 exec(cmd_buf)
         else:
             var_type = type(var_value).__name__
             func_name = sys._getframe().f_code.co_name
-            var_value = "<" + var_type + " type not supported by " \
-                        + func_name + "()>"
+            var_value = "<" + var_type + " type not supported by " + \
+                        func_name + "()>"
             value_format = "%s"
             loc_col1_indent -= 2
+            # Adjust loc_col1_width.
+            loc_col1_width = loc_col1_width - loc_col1_indent
             format_string = "%" + str(loc_col1_indent) + "s%-" \
-                + str(loc_col1_width) + "s" + value_format + "\n"
+                + str(loc_col1_width) + "s" + value_format + trailing_char
             return format_string % ("", var_name + ":", var_value)
         return buffer
 
@@ -512,8 +551,8 @@
 
     # Get the name of the first variable passed to this function.
     stack_frame = 2
-    calling_func_name = sprint_func_name(2)
-    if calling_func_name == "print_var":
+    caller_func_name = sprint_func_name(2)
+    if caller_func_name.endswith("print_var"):
         stack_frame += 1
     var_name = get_arg_name(None, 1, stack_frame)
     return sprint_varx(var_name, *args)
@@ -522,9 +561,49 @@
 
 
 ###############################################################################
-def sprint_dashes(loc_col1_indent=col1_indent,
-                  col_width=80,
-                  line_feed=1):
+def lprint_varx(var_name,
+                var_value,
+                hex=0,
+                loc_col1_indent=col1_indent,
+                loc_col1_width=col1_width,
+                log_level=getattr(logging, 'INFO')):
+
+    r"""
+    Send sprint_varx output to logging.
+    """
+
+    logging.log(log_level, sprint_varx(var_name, var_value, hex,
+                loc_col1_indent, loc_col1_width, ""))
+
+###############################################################################
+
+
+###############################################################################
+def lprint_var(*args):
+
+    r"""
+    Figure out the name of the first argument for you and then call
+    lprint_varx with it.  Therefore, the following 2 calls are equivalent:
+    lprint_varx("var1", var1)
+    lprint_var(var1)
+    """
+
+    # Get the name of the first variable passed to this function.
+    stack_frame = 2
+    caller_func_name = sprint_func_name(2)
+    if caller_func_name.endswith("print_var"):
+        stack_frame += 1
+    var_name = get_arg_name(None, 1, stack_frame)
+    lprint_varx(var_name, *args)
+
+###############################################################################
+
+
+###############################################################################
+def sprint_dashes(indent=col1_indent,
+                  width=80,
+                  line_feed=1,
+                  char="-"):
 
     r"""
     Return a string of dashes to the caller.
@@ -535,10 +614,12 @@
     width                           The width of the string of dashes.
     line_feed                       Indicates whether the output should end
                                     with a line feed.
+    char                            The character to be repeated in the output
+                                    string.
     """
 
-    col_width = int(col_width)
-    buffer = " "*int(loc_col1_indent) + "-"*col_width
+    width = int(width)
+    buffer = " "*int(indent) + char*width
     if line_feed:
         buffer += "\n"
 
@@ -548,7 +629,30 @@
 
 
 ###############################################################################
-def sprint_call_stack():
+def sindent(text="",
+            indent=0):
+
+    r"""
+    Pre-pend the specified number of characters to the text string (i.e.
+    indent it) and return it.
+
+    Description of arguments:
+    text                            The string to be indented.
+    indent                          The number of characters to indent the
+                                    string.
+    """
+
+    format_string = "%" + str(indent) + "s%s"
+    buffer = format_string % ("", text)
+
+    return buffer
+
+###############################################################################
+
+
+###############################################################################
+def sprint_call_stack(indent=0,
+                      stack_frame_ix=0):
 
     r"""
     Return a call stack report for the given point in the program with line
@@ -575,18 +679,21 @@
     """
 
     buffer = ""
-
-    buffer += sprint_dashes()
-    buffer += "Python function call stack\n\n"
-    buffer += "Line # Function name and arguments\n"
-    buffer += sprint_dashes(0, 6, 0) + " " + sprint_dashes(0, 73)
+    buffer += sprint_dashes(indent)
+    buffer += sindent("Python function call stack\n\n", indent)
+    buffer += sindent("Line # Function name and arguments\n", indent)
+    buffer += sprint_dashes(indent, 6, 0) + " " + sprint_dashes(0, 73)
 
     # Grab the current program stack.
     current_stack = inspect.stack()
 
     # Process each frame in turn.
     format_string = "%6s %s\n"
+    ix = 0
     for stack_frame in current_stack:
+        if ix < stack_frame_ix:
+            ix += 1
+            continue
         lineno = str(stack_frame[2])
         func_name = str(stack_frame[3])
         if func_name == "?":
@@ -594,8 +701,8 @@
             func_name = "(none)"
 
         if func_name == "<module>":
-            # If the func_name is the "main" program, we simply get the command
-            # line call string.
+            # If the func_name is the "main" program, we simply get the
+            # command line call string.
             func_and_args = ' '.join(sys.argv)
         else:
             # Get the program arguments.
@@ -603,19 +710,20 @@
             function_parms = arg_vals[0]
             frame_locals = arg_vals[3]
 
-            args_arr = []
+            args_list = []
             for arg_name in function_parms:
                 # Get the arg value from frame locals.
                 arg_value = frame_locals[arg_name]
-                args_arr.append(arg_name + " = " + repr(arg_value))
-            args_str = "(" + ', '.join(map(str, args_arr)) + ")"
+                args_list.append(arg_name + " = " + repr(arg_value))
+            args_str = "(" + ', '.join(map(str, args_list)) + ")"
 
             # Now we need to print this in a nicely-wrapped way.
             func_and_args = func_name + " " + args_str
 
-        buffer += format_string % (lineno, func_and_args)
+        buffer += sindent(format_string % (lineno, func_and_args), indent)
+        ix += 1
 
-    buffer += sprint_dashes()
+    buffer += sprint_dashes(indent)
 
     return buffer
 
@@ -648,7 +756,7 @@
     if stack_frame_ix is None:
         func_name = sys._getframe().f_code.co_name
         caller_func_name = sys._getframe(1).f_code.co_name
-        if func_name[1:] == caller_func_name:
+        if caller_func_name.endswith(func_name[1:]):
             stack_frame_ix = 2
         else:
             stack_frame_ix = 1
@@ -670,12 +778,12 @@
         function_parms = arg_vals[0]
         frame_locals = arg_vals[3]
 
-        args_arr = []
+        args_list = []
         for arg_name in function_parms:
             # Get the arg value from frame locals.
             arg_value = frame_locals[arg_name]
-            args_arr.append(arg_name + " = " + repr(arg_value))
-        args_str = "(" + ', '.join(map(str, args_arr)) + ")"
+            args_list.append(arg_name + " = " + repr(arg_value))
+        args_str = "(" + ', '.join(map(str, args_list)) + ")"
 
         # Now we need to print this in a nicely-wrapped way.
         func_and_args = func_name + " " + args_str
@@ -686,32 +794,45 @@
 
 
 ###############################################################################
-def sprint_pgm_header():
+def sprint_pgm_header(indent=0):
 
     r"""
     Return a standardized header that programs should print at the beginning
     of the run.  It includes useful information like command line, pid,
     userid, program parameters, etc.
 
+    Description of arguments:
+    indent                          The number of characters to indent each
+                                    line of output.
     """
 
     buffer = "\n"
-    buffer += sprint_time() + "Running " + pgm_name + ".\n"
-    buffer += sprint_time() + "Program parameter values, etc.:\n\n"
-    buffer += sprint_varx("command_line", ' '.join(sys.argv))
-    # We want the output to show a customized name for the pid and pgid but we
-    # want it to look like a valid variable name.  Therefore, we'll use
+
+    buffer += sindent(sprint_time() + "Running " + pgm_name + ".\n", indent)
+    buffer += sindent(sprint_time() + "Program parameter values, etc.:\n\n",
+                      indent)
+    buffer += sprint_varx("command_line", ' '.join(sys.argv), 0, indent)
+    # We want the output to show a customized name for the pid and pgid but
+    # we want it to look like a valid variable name.  Therefore, we'll use
     # pgm_name_var_name which was set when this module was imported.
-    buffer += sprint_varx(pgm_name_var_name + "_pid", os.getpid())
-    buffer += sprint_varx(pgm_name_var_name + "_pgid", os.getpgrp())
+    buffer += sprint_varx(pgm_name_var_name + "_pid", os.getpid(), 0, indent)
+    buffer += sprint_varx(pgm_name_var_name + "_pgid", os.getpgrp(), 0, indent)
     buffer += sprint_varx("uid", str(os.geteuid()) + " (" + os.getlogin() +
-                          ")")
+                          ")", 0, indent)
     buffer += sprint_varx("gid", str(os.getgid()) + " (" +
-                          str(grp.getgrgid(os.getgid()).gr_name) + ")")
-    buffer += sprint_varx("host_name", socket.gethostname())
-    buffer += sprint_varx("DISPLAY", os.environ['DISPLAY'])
+                          str(grp.getgrgid(os.getgid()).gr_name) + ")", 0,
+                          indent)
+    buffer += sprint_varx("host_name", socket.gethostname(), 0, indent)
+    buffer += sprint_varx("DISPLAY", os.environ['DISPLAY'], 0, indent)
     # I want to add code to print caller's parms.
 
+    # __builtin__.arg_obj is created by the get_arg module function,
+    # gen_get_options.
+    try:
+        buffer += ga.sprint_args(__builtin__.arg_obj, indent)
+    except AttributeError:
+        pass
+
     buffer += "\n"
 
     return buffer
@@ -720,6 +841,43 @@
 
 
 ###############################################################################
+def sprint_error_report(error_text="\n",
+                        indent=2):
+
+    r"""
+    Return a string with a standardized report which includes the caller's
+    error text, the call stack and the program header.
+
+    Description of args:
+    error_text                      The error text to be included in the
+                                    report.  The caller should include any
+                                    needed linefeeds.
+    indent                          The number of characters to indent each
+                                    line of output.
+    """
+
+    buffer = ""
+    buffer += sprint_dashes(width=120, char="=")
+    buffer += sprint_error(error_text)
+    buffer += "\n"
+    # Calling sprint_call_stack with stack_frame_ix of 0 causes it to show
+    # itself and this function in the call stack.  This is not helpful to a
+    # debugger and is therefore clutter.  We will adjust the stack_frame_ix to
+    # hide that information.
+    stack_frame_ix = 2
+    caller_func_name = sprint_func_name(2)
+    if caller_func_name.endswith("print_error_report"):
+        stack_frame_ix += 1
+    buffer += sprint_call_stack(indent, stack_frame_ix)
+    buffer += sprint_pgm_header(indent)
+    buffer += sprint_dashes(width=120, char="=")
+
+    return buffer
+
+###############################################################################
+
+
+###############################################################################
 def sissuing(cmd_buf):
 
     r"""
@@ -752,7 +910,24 @@
     total_time = time.time() - start_time
     total_time_string = "%0.6f" % total_time
 
-    buffer += sprint_varx(pgm_name_var_name + "runtime", total_time_string)
+    buffer += sprint_varx(pgm_name_var_name + "_runtime", total_time_string)
+
+    return buffer
+
+###############################################################################
+
+
+###############################################################################
+def sprint(buffer=""):
+
+    r"""
+    Simply return the user's buffer.  This function is used by the qprint and
+    dprint functions defined dynamically below, i.e. it would not normally be
+    called for general use.
+
+    Description of arguments.
+    buffer                          This will be returned to the caller.
+    """
 
     return buffer
 
@@ -763,8 +938,8 @@
 # In the following section of code, we will dynamically create print versions
 # for each of the sprint functions defined above.  So, for example, where we
 # have an sprint_time() function defined above that returns the time to the
-# caller in a string, we will create a corresponding print_time() function that
-# will print that string directly to stdout.
+# caller in a string, we will create a corresponding print_time() function
+# that will print that string directly to stdout.
 
 # It can be complicated to follow what's being creaed by the exec statement
 # below.  Here is an example of the print_time() function that will be created:
@@ -778,23 +953,28 @@
 # Calculate the "s" version of this function name (e.g. if this function name
 # is print_time, we want s_funcname to be "sprint_time".
 # Put a reference to the "s" version of this function in s_func.
-# Call the "s" version of this function passing it all of our arguments.  Write
-# the result to stdout.
+# Call the "s" version of this function passing it all of our arguments.
+# Write the result to stdout.
 
 # func_names contains a list of all print functions which should be created
 # from their sprint counterparts.
 func_names = ['print_time', 'print_timen', 'print_error', 'print_varx',
               'print_var', 'print_dashes', 'print_call_stack',
               'print_func_name', 'print_executing', 'print_pgm_header',
-              'issuing', 'print_pgm_footer']
+              'issuing', 'print_pgm_footer', 'print_error_report', 'print']
 
 for func_name in func_names:
+    if func_name == "print":
+        continue
     # Create abbreviated aliases (e.g. spvar is an alias for sprint_var).
     alias = re.sub("print_", "p", func_name)
-    exec("s" + alias + " = s" + func_name)
+    pgm_definition_string = "s" + alias + " = s" + func_name
+    if gen_print_debug:
+        print(pgm_definition_string)
+    exec(pgm_definition_string)
 
 for func_name in func_names:
-    if func_name == "print_error":
+    if func_name == "print_error" or func_name == "print_error_report":
         output_stream = "stderr"
     else:
         output_stream = "stdout"
@@ -806,11 +986,88 @@
             "  sys." + output_stream + ".write(s_func(*args))",
             "  sys." + output_stream + ".flush()"
         ]
+    if func_name != "print":
+        pgm_definition_string = '\n'.join(func_def)
+        if gen_print_debug:
+            print(pgm_definition_string)
+        exec(pgm_definition_string)
+
+    # Now define "q" versions of each print function.
+    func_def = \
+        [
+            "def q" + func_name + "(*args):",
+            "  if __builtin__.quiet: return",
+            "  s_func_name = \"s" + func_name + "\"",
+            "  s_func = getattr(sys.modules[__name__], s_func_name)",
+            "  sys." + output_stream + ".write(s_func(*args))",
+            "  sys." + output_stream + ".flush()"
+        ]
+
     pgm_definition_string = '\n'.join(func_def)
+    if gen_print_debug:
+        print(pgm_definition_string)
     exec(pgm_definition_string)
 
+    # Now define "d" versions of each print function.
+    func_def = \
+        [
+            "def d" + func_name + "(*args):",
+            "  if not __builtin__.debug: return",
+            "  s_func_name = \"s" + func_name + "\"",
+            "  s_func = getattr(sys.modules[__name__], s_func_name)",
+            "  sys." + output_stream + ".write(s_func(*args))",
+            "  sys." + output_stream + ".flush()"
+        ]
+
+    pgm_definition_string = '\n'.join(func_def)
+    if gen_print_debug:
+        print(pgm_definition_string)
+    exec(pgm_definition_string)
+
+    # Now define "l" versions of each print function.
+    func_def = \
+        [
+            "def l" + func_name + "(*args):",
+            "  s_func_name = \"s" + func_name + "\"",
+            "  s_func = getattr(sys.modules[__name__], s_func_name)",
+            "  logging.log(getattr(logging, 'INFO'), s_func(*args))",
+        ]
+
+    if func_name != "print_varx" and func_name != "print_var":
+        pgm_definition_string = '\n'.join(func_def)
+        if gen_print_debug:
+            print(pgm_definition_string)
+        exec(pgm_definition_string)
+
+    if func_name == "print":
+        continue
+
     # Create abbreviated aliases (e.g. pvar is an alias for print_var).
     alias = re.sub("print_", "p", func_name)
-    exec(alias + " = " + func_name)
+    pgm_definition_string = alias + " = " + func_name
+    if gen_print_debug:
+        print(pgm_definition_string)
+    exec(pgm_definition_string)
+
+    # Create abbreviated aliases (e.g. qpvar is an alias for qprint_var).
+    alias = re.sub("print_", "p", func_name)
+    pgm_definition_string = "q" + alias + " = q" + func_name
+    if gen_print_debug:
+        print(pgm_definition_string)
+    exec(pgm_definition_string)
+
+    # Create abbreviated aliases (e.g. dpvar is an alias for dprint_var).
+    alias = re.sub("print_", "p", func_name)
+    pgm_definition_string = "d" + alias + " = d" + func_name
+    if gen_print_debug:
+        print(pgm_definition_string)
+    exec(pgm_definition_string)
+
+    # Create abbreviated aliases (e.g. lpvar is an alias for lprint_var).
+    alias = re.sub("print_", "p", func_name)
+    pgm_definition_string = "l" + alias + " = l" + func_name
+    if gen_print_debug:
+        print(pgm_definition_string)
+    exec(pgm_definition_string)
 
 ###############################################################################
diff --git a/lib/gen_robot_print.py b/lib/gen_robot_print.py
index 803aa4a..331690b 100755
--- a/lib/gen_robot_print.py
+++ b/lib/gen_robot_print.py
@@ -6,7 +6,9 @@
 
 import sys
 import re
+
 import gen_print as gp
+
 from robot.libraries.BuiltIn import BuiltIn
 from robot.api import logger
 
@@ -20,8 +22,8 @@
 # string directly to stdout.
 
 # It can be complicated to follow what's being creaed by the exec statement
-# below.  Here is an example of the rprint_time() function that will be created
-# (as of the time of this writing):
+# below.  Here is an example of the rprint_time() function that will be
+# created (as of the time of this writing):
 
 # def rprint_time(*args):
 #   s_func = getattr(gp, "sprint_time")
@@ -31,13 +33,14 @@
 
 # Here are comments describing the lines in the body of the created function.
 # Put a reference to the "s" version of this function in s_func.
-# Call the "s" version of this function passing it all of our arguments.  Write
-# the result to stdout.
+# Call the "s" version of this function passing it all of our arguments.
+# Write the result to stdout.
 
 robot_prefix = "r"
 for func_name in gp.func_names:
-    # The print_var function's job is to figure out the name of arg 1 and then
-    # call print_varx.  This is not currently supported for robot programs.
+    # The print_var function's job is to figure out the name of arg 1 and
+    # then call print_varx.  This is not currently supported for robot
+    # programs.  Though it IS supported for python modules.
     if func_name == "print_error":
         output_stream = "STDERR"
     else:
@@ -47,7 +50,7 @@
             "def " + robot_prefix + func_name + "(*args):",
             "  s_func = getattr(gp, \"s" + func_name + "\")",
             "  BuiltIn().log_to_console(s_func(*args),"
-            " stream = '" + output_stream + "',"
+            " stream='" + output_stream + "',"
             " no_newline=True)"
         ]
 
@@ -63,37 +66,39 @@
 
 
 ###############################################################################
-def rprint(buffer=""):
+def rprint(buffer="",
+           stream="STDOUT"):
 
     r"""
     rprint stands for "Robot Print".  This keyword will print the user's
     buffer to the console.  This keyword does not write a linefeed.  It is the
     responsibility of the caller to include a line feed if desired.  This
     keyword is essentially an alias for "Log to Console  <string>
-    no_newline=True".
+    <stream>".
 
     Description of arguments:
     buffer                          The value that is to written to stdout.
     """
 
-    BuiltIn().log_to_console(buffer, no_newline=True)
+    BuiltIn().log_to_console(buffer, no_newline=True, stream=stream)
 
 ###############################################################################
 
 
 ###############################################################################
-def rprintn(buffer=""):
+def rprintn(buffer="",
+            stream='STDOUT'):
 
     r"""
     rprintn stands for "Robot print with linefeed".  This keyword will print
     the user's buffer to the console along with a linefeed.  It is basically
-    an abbreviated form of "Log go Console  <string>"
+    an abbreviated form of "Log go Console  <string>  <stream>"
 
     Description of arguments:
     buffer                          The value that is to written to stdout.
     """
 
-    BuiltIn().log_to_console(buffer, no_newline=False)
+    BuiltIn().log_to_console(buffer, no_newline=False, stream=stream)
 
 ###############################################################################
 
@@ -157,6 +162,6 @@
 ###############################################################################
 
 
-# Define an alias.  rpvar is just a special case of rpvars where the var_names
-# list contains only one element.
+# Define an alias.  rpvar is just a special case of rpvars where the
+# var_names list contains only one element.
 rpvar = rpvars
diff --git a/lib/gen_valid.py b/lib/gen_valid.py
new file mode 100755
index 0000000..1a52ace
--- /dev/null
+++ b/lib/gen_valid.py
@@ -0,0 +1,104 @@
+#!/usr/bin/env python
+
+r"""
+This module provides valuable argument processing functions like
+gen_get_options and sprint_args.
+"""
+
+import sys
+
+import gen_print as gp
+
+
+
+###############################################################################
+def valid_value(var_value,
+                invalid_values=[""],
+                valid_values=[]):
+
+    r"""
+    Return True if var_value is a valid value.  Otherwise, return False and
+    print an error message to stderr.
+
+    Description of arguments:
+    var_value                       The value being validated.
+    invalid_values                  A list of invalid values.  If var_value is
+                                    equal to any of these, it is invalid.
+                                    Note that if you specify anything for
+                                    invalid_values (below), the valid_values
+                                    list is not even processed.
+    valid_values                    A list of invalid values.  var_value must
+                                    be equal to one of these values to be
+                                    considered valid.
+    """
+
+    len_valid_values = len(valid_values)
+    len_invalid_values = len(invalid_values)
+    if len_valid_values > 0 and len_invalid_values > 0:
+        gp.print_error_report("Programmer error - You must provide either an" +
+                              " invalid_values list or a valid_values" +
+                              " list but NOT both.")
+        return False
+
+    if len_valid_values > 0:
+        # Processing the valid_values list.
+        if var_value in valid_values:
+            return True
+        var_name = gp.get_arg_name(0, 1, 2)
+        gp.print_error_report("The following variable has an invalid" +
+                              " value:\n" +
+                              gp.sprint_varx(var_name, var_value) +
+                              "\nIt must be one of the following values:\n" +
+                              gp.sprint_varx("valid_values", valid_values))
+        return False
+
+    if len_invalid_values == 0:
+        gp.print_error_report("Programmer error - You must provide either an" +
+                              " invalid_values list or a valid_values" +
+                              " list.  Both are empty.")
+        return False
+
+    # Assertion: We have an invalid_values list.  Processing it now.
+    if var_value not in invalid_values:
+        return True
+
+    var_name = gp.get_arg_name(0, 1, 2)
+    gp.print_error_report("The following variable has an invalid value:\n" +
+                          gp.sprint_varx(var_name, var_value) + "\nIt must" +
+                          " NOT be one of the following values:\n" +
+                          gp.sprint_varx("invalid_values", invalid_values))
+    return False
+
+###############################################################################
+
+
+###############################################################################
+def valid_integer(var_value):
+
+    r"""
+    Return True if var_value is a valid integer.  Otherwise, return False and
+    print an error message to stderr.
+
+    Description of arguments:
+    var_value                       The value being validated.
+    """
+
+    # This currently allows floats which is not good.
+
+    try:
+        if type(int(var_value)) is int:
+            return True
+    except ValueError:
+        pass
+
+    # If we get to this point, the validation has failed.
+
+    var_name = gp.get_arg_name(0, 1, 2)
+    gp.print_varx("var_name", var_name)
+
+    gp.print_error_report("Invalid integer value:\n" +
+                          gp.sprint_varx(var_name, var_value))
+
+    return False
+
+###############################################################################