From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=40.107.14.81; helo=eur01-ve1-obe.outbound.protection.outlook.com; envelope-from=supreeth.venkatesh@arm.com; receiver=edk2-devel@lists.01.org Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-eopbgr140081.outbound.protection.outlook.com [40.107.14.81]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 29D2221180F27 for ; Mon, 7 Jan 2019 12:13:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector1-arm-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=bnxStQb87niSRo8pjpUByucoWpwM13Q5qWvzJX5eIpM=; b=mvuwii6SEWk1F4WYSQY8vUWEIYppf21hgYOsTosxrT/GQIoRAa+5PUdBRcTczXmXQEr/vbHSBIifuWN7J+nauezURuIZlt+4w15yxFuaGL4B4PpcoCfp3IADawhmUTMR5HiZIrI3dLWWSCqt5ZoZ7/AFq2xtgR5PdU3DOyVCbP0= Received: from AM4PR08MB2788.eurprd08.prod.outlook.com (10.171.191.18) by AM4PR08MB2690.eurprd08.prod.outlook.com (10.171.190.155) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1495.7; Mon, 7 Jan 2019 20:13:51 +0000 Received: from AM4PR08MB2788.eurprd08.prod.outlook.com ([fe80::598a:a464:e5d8:5307]) by AM4PR08MB2788.eurprd08.prod.outlook.com ([fe80::598a:a464:e5d8:5307%4]) with mapi id 15.20.1495.011; Mon, 7 Jan 2019 20:13:51 +0000 From: Supreeth Venkatesh To: Sakar Arora , "edk2-devel@lists.01.org" , "eric.jin@intel.com" Thread-Topic: [edk2][edk2-test][RFC] Integrating SBBR tests into SCT - A design proposal Thread-Index: AdSgwgEf4U2uyT4uRYG+AR9zd3VV2QADkKdgAX1u4XA= Date: Mon, 7 Jan 2019 20:13:51 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=Supreeth.Venkatesh@arm.com; x-originating-ip: [2600:1700:8c61:400:686d:6b4f:664f:206a] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; AM4PR08MB2690; 6:XsGvRIKhJ6oYZNNN/VKYp7YyUu1RWSBhDWD7UL0ggiXFlHcp9aenjwbelRfu6GJXAhWrS8LTdBaZNhdrkBqqKcw9htdEHOeioKo2cpHWFPKz5AINFD9YD8DsZ1Y5Cp/GAtp0ZoTaX1aCXQtG2i4D4LKjLgb7NbDPSwS++qXIxrs3QHJxIMPyHYbM7t/UWYrkqdptT7uzLk+QFErWiqJcNY9ssgMkFUcCGlbnrEG6CE6/zvDdmZLHVmtSwICdc573rE+akn4s46ElY3XrNVG+Vp+byXkQonaOSZmJOodgo3br9rl0XZ8o6EUJew8aXqpP9oDu+jGSPeCUGjr1Bb9ajwtngCaZ348YAEL02wPz0PG/sFVzcovyl75UPHJA7ZT/emPiM/Cp9JRrO3N16579MemAzN3QEYIj9Goj75D+abqQcoKuiCRf76E/+AYclEstnSMQJoOc7WSNt8kRaOlHtQ==; 5:41xx34KrawOm3qjaVxrP998caD0LtQ4qjv97Bdd8dVnZm4EmPagTtHUfe03kbHBtjjSpnF6IPASAIluuDPu4Vdu18y8NX/rMcZZusdZbLLTu9/yC/5qV8wMC995DyhjrwOx/VEGXTCGPBBu/6BnkGRARU51PKR7f+GgkLWzsingBjgXRR43PIrC7kdF0u+fp7doUiPS7QYDT3mVfvilL1Q==; 7:fQqG2OHAf9eHD1aVJVBeJHcC+4nQwQ5CI48lltVo0LSN6Aql1EhzcT/9pA3p7fxD3F6b8DS9H9evVXaqVL8CbGrOcLb3pkM7Iq10YdMNCr8sl9AmNAXE810Hi9XWYf8vVFwJPasM7iBTHDt3feRr4A== x-ms-exchange-antispam-srfa-diagnostics: SOS;SOR; x-ms-office365-filtering-correlation-id: 12c8b254-b816-4ed7-487c-08d674dca407 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600109)(711020)(4618075)(2017052603328)(7153060)(7193020); SRVR:AM4PR08MB2690; x-ms-traffictypediagnostic: AM4PR08MB2690: x-microsoft-antispam-prvs: x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(3230021)(908002)(999002)(5005026)(6040522)(8220060)(2401047)(8121501046)(3231475)(944501520)(4982022)(52105112)(93006095)(93001095)(10201501046)(3002001)(6055026)(6041310)(20161123558120)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123562045)(20161123560045)(20161123564045)(201708071742011)(7699051)(76991095); SRVR:AM4PR08MB2690; BCL:0; PCL:0; RULEID:; SRVR:AM4PR08MB2690; x-forefront-prvs: 0910AAF391 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(346002)(366004)(39860400002)(376002)(136003)(396003)(53754006)(199004)(189003)(13464003)(40434004)(305945005)(316002)(2501003)(7736002)(110136005)(102836004)(76176011)(81156014)(33656002)(8676002)(81166006)(186003)(561944003)(966005)(478600001)(14454004)(74316002)(256004)(5024004)(72206003)(8936002)(6246003)(53936002)(7696005)(25786009)(53546011)(68736007)(97736004)(446003)(229853002)(476003)(6506007)(55016002)(5660300001)(4326008)(106356001)(11346002)(486006)(14444005)(2906002)(6436002)(86362001)(6116002)(9686003)(99286004)(71190400001)(46003)(6306002)(71200400001)(105586002)(2201001); DIR:OUT; SFP:1101; SCL:1; SRVR:AM4PR08MB2690; H:AM4PR08MB2788.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: bxpOsiNYhuSySuylUO9YyNKX8FjselG+aWyh54gdwZ6L/YGrwN/mGUecnWApCJvryw/l7Ei3N1uSOUpdH/yJ3NV7zntIrQAQcW5ofecH4qm7Q3jzGLJEoanXjeP8L3lCG50/khmmoqrH/v4U2Ss0D9eHZvyb6W8MNhY7pAl9o+xnafJxiXhhr4mSws2PKlWJpy9lNMK88KbUOKkqMmwSWfHdNXyOf3CQcTn7oOzDEJmfKGUR+mY0FkA9xZ36/9dN3Dj50jLrKtzs/bMnqabu9hznprI0zDv4+l1EPbNt7tJx0SOuS6tyTcImkMTG/gp1 spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-Network-Message-Id: 12c8b254-b816-4ed7-487c-08d674dca407 X-MS-Exchange-CrossTenant-originalarrivaltime: 07 Jan 2019 20:13:51.2021 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM4PR08MB2690 Subject: Re: [edk2-test][RFC] Integrating SBBR tests into SCT - A design proposal X-BeenThere: edk2-devel@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: EDK II Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jan 2019 20:13:56 -0000 Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Sakar, Looks good. However, Why do we need two additional parameters SBBR and SBBR_EXCL? Since SBBR is a subset/superset of UEFI specifications, would "sbbr_standalone" or similar (just sbbr) which would run all the sbbr= tests including ones that is already part of UEFI-SCT Suffice? Lets wait for additional comments from Eric. Thanks, Supreeth -----Original Message----- From: Sakar Arora Sent: Monday, December 31, 2018 12:50 AM To: edk2-devel@lists.01.org; eric.jin@intel.com; Supreeth Venkatesh Cc: Prasanth Pulla Subject: [edk2][edk2-test][RFC] Integrating SBBR tests into SCT - A design = proposal Hi All, Introduction --> The intent is to run SBBR tests using SCT infrastructure. This is the SBBR= specification link for reference http://infocenter.arm.com/help/topic/com.= arm.doc.den0044c/Server_Base_Boot_Requirements_v1_1_Arm_DEN_0044C.pdf Majority of SBBR requirements can be tested by current SCT code as is. A fe= w more SBBR specific tests are needed to cover complete spec. Proposal --> An ideal setup is new SBBR specific code residing in edk2-test master. SCT = test infrastructure will run tests for UEFI spec compliance or just SBBR sp= ec compliance based on command line arguments. This would require run time = classification of tests. - Details Each SCT test provides a list of sub-tests to verify various aspects of the= requirement. Each sub-test is assigned some attributes (AUTO, MANUAL, DEST= RUCTIVE, or RESET_REQUIRED) which are referred to by the test infrastructur= e to do pre and post processing for it. The proposal is to add 2 new attributes SBBR and SBBR_EXCL to the existing = set. The test infrastructure will decide whether to run or skip a test base= d on command line parameters and these new attributes. - The new attributes SBBR : Tests with this attribute are required by both SBBR and UEFI. SBBR_EXCL : Tests with this attribute are required by SBBR exclusively. - Examples Shell> Sct.efi -a -sbbr Would run tests required by SBBR, i.e., tests with attributes SBBR or SBBR_= EXCL. Shell> Sct.efi -a Would run tests required by UEFI, i.e., all tests excluding those with attr= ibute SBBR_EXCL. - SCT test pre-processing flow The following logic will be added to the pre-processing flow. If (SCT_FOR_SBBR) { If (!TEST_CASE_FOR_SBBR(TestCaseAttribute)) goto SkipTest; } else { If (TEST_CASE_FOR_SBBR_EXCL(TestCaseAttribute)) goto SkipTest; } Comments are welcome. Thanks, Sakar IMPORTANT NOTICE: The contents of this email and any attachments are confid= ential and may also be privileged. If you are not the intended recipient, p= lease notify the sender immediately and do not disclose the contents to any= other person, use it for any purpose, or store or copy the information in = any medium. Thank you.