Re: [edk2-test][RFC] Integrating SBBR tests into SCT - A design proposal

[Supreeth Venkatesh <Supreeth.Venkatesh@arm.com>]

Sakar,

Looks good.
However, Why do we need two  additional parameters SBBR and SBBR_EXCL?

Since SBBR is a subset/superset of UEFI specifications,
would "sbbr_standalone" or similar (just sbbr) which would run all the sbbr tests including ones that is already part of UEFI-SCT
Suffice?

Lets wait for additional comments from Eric.

Thanks,
Supreeth

-----Original Message-----
From: Sakar Arora <Sakar.Arora@arm.com>
Sent: Monday, December 31, 2018 12:50 AM
To: edk2-devel@lists.01.org; eric.jin@intel.com; Supreeth Venkatesh <Supreeth.Venkatesh@arm.com>
Cc: Prasanth Pulla <Prasanth.Pulla@arm.com>
Subject: [edk2][edk2-test][RFC] Integrating SBBR tests into SCT - A design proposal

Hi All,

Introduction -->

The intent is to run SBBR  tests using SCT infrastructure. This is the SBBR specification link for reference http://infocenter.arm.com/help/topic/com.arm.doc.den0044c/Server_Base_Boot_Requirements_v1_1_Arm_DEN_0044C.pdf

Majority of SBBR requirements can be tested by current SCT code as is. A few more SBBR specific tests are needed to cover complete spec.

Proposal -->

An ideal setup is new SBBR specific code residing in edk2-test master. SCT test infrastructure will run tests for UEFI spec compliance or just SBBR spec compliance based on command line arguments. This would require run time classification of tests.

- Details

Each SCT test provides a list of sub-tests to verify various aspects of the requirement. Each sub-test is assigned some attributes (AUTO, MANUAL, DESTRUCTIVE, or RESET_REQUIRED) which are referred to by the test infrastructure to do pre and post processing for it.

The proposal is to add 2 new attributes SBBR and SBBR_EXCL to the existing set. The test infrastructure will decide whether to run or skip a test based on command line parameters and these new attributes.

- The new attributes

SBBR : Tests with this attribute are required by both SBBR and UEFI.
SBBR_EXCL : Tests with this attribute are required by SBBR exclusively.

- Examples

Shell> Sct.efi -a -sbbr

Would run tests required by SBBR, i.e., tests with attributes SBBR or SBBR_EXCL.

Shell> Sct.efi -a

Would run tests required by UEFI, i.e., all tests excluding those with attribute SBBR_EXCL.

-  SCT test pre-processing flow

The following logic will be added to the pre-processing flow.

If (SCT_FOR_SBBR) {
       If (!TEST_CASE_FOR_SBBR(TestCaseAttribute))
            goto SkipTest;
} else {
    If (TEST_CASE_FOR_SBBR_EXCL(TestCaseAttribute))
           goto SkipTest;
}

Comments are welcome.

Thanks,
Sakar
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.