From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=40.107.14.55; helo=eur01-ve1-obe.outbound.protection.outlook.com; envelope-from=sakar.arora@arm.com; receiver=edk2-devel@lists.01.org Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-eopbgr140055.outbound.protection.outlook.com [40.107.14.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 9C6D1211A6D54 for ; Sun, 30 Dec 2018 22:50:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector1-arm-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=KY/p8OEOdusjE2WmgGvQNFnKcustoBV7ys4t+Kf/sOg=; b=c2c3q0zSK90OQwuTgCquZTogyNucCKP2OSsv570sJviBj5NQ9THSO4t1adGOAizgaZ9dUBUVPRQhPw+T5kRBSmdPUrnGeg8Gip6a0miOa78vFe+19l6rqpW5Cdud0G10vpIsQcdJj+9UfLwql+R7Pdl8puv309kTaK8IPmfHuto= Received: from DB7PR08MB3209.eurprd08.prod.outlook.com (52.134.110.159) by DB7PR08MB3017.eurprd08.prod.outlook.com (52.134.109.159) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1471.20; Mon, 31 Dec 2018 06:50:15 +0000 Received: from DB7PR08MB3209.eurprd08.prod.outlook.com ([fe80::28c8:85c9:89bf:f21b]) by DB7PR08MB3209.eurprd08.prod.outlook.com ([fe80::28c8:85c9:89bf:f21b%2]) with mapi id 15.20.1471.019; Mon, 31 Dec 2018 06:50:15 +0000 From: Sakar Arora To: "edk2-devel@lists.01.org" , "eric.jin@intel.com" , Supreeth Venkatesh Thread-Topic: [edk2][edk2-test][RFC] Integrating SBBR tests into SCT - A design proposal Thread-Index: AdSgwgEf4U2uyT4uRYG+AR9zd3VV2QADkKdg Date: Mon, 31 Dec 2018 06:50:15 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [217.140.105.40] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; DB7PR08MB3017; 6:AfS+9XKrhdXbspCYw3xnzd+kb2c5TQUPqdv0c0x1i17Qjh7l/Da/RAYLzJmez3BW7tIukGgE5gIxz6PruPfh9euJJz4ARZMYtE1dNRrTWyBKqYP+g+dDbinlPqzU0FVkJQMhPVE2prU/fu/sXAs1itLrQ/rvqN/I9P/G23JUGGrcMJ59TZHzfuCvXB4XRwU7Vz9iWySy6fuBC7+OlIlcz+7lsFUOtOY99U2A0JcmnC/GpmH7VcO6On5TfyjnH3PS4FG5h1WlaToyCG/2rjbvPbVgk+iBy49J6EZgq5jBgO6p0uaN2q0Yv3FwyBcaXKWMZ7YFlChVn2qglQsBNhzjlgwQPUtGapKhzhivyxCTCYqj4/0BhvdLU3jqOIJnyg20Nv6cTgN5hKIpJ4pDsq9v7Tf5naamSOBa+BsXQvzLmmbn88gudZ1FtLkMXOOwi7iVuTbshm3/vrKETmbopq8vLQ==; 5:K8cg0oMtZMebedq2ImgWaAaXpnUzJ6XtZktmGpMbBBbHHBwLXnY5eSPiVdl4G7szE6oLtCOkyL91nmNcMrRZGVR27ID+lk9QxnLoWlkRPtEkkmcQOtsMPBmEBTVYr589uUPlec7xUjfFFgdw4cAN4TODuGL0jzOGwpic3vqbutM=; 7:Fa3EvOQecFrBGhG2TnJEIXl88vAmsyggznhbBvjJ7r+Tuv9lOGTv/zvbuYTTMlPEKrORogBY2cjuQPApD1lNpnX1zh2LENLSS54emXr15Dv3aAa6ROxwYXZkYg9/QLuTN3BoFgflm4lJA3jg7IXhUQ== x-ms-exchange-antispam-srfa-diagnostics: SOS;SOR; x-ms-office365-filtering-correlation-id: d638fbd7-dbc7-4ca8-87c5-08d66eec3865 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(5600109)(711020)(4618075)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(2017052603328)(7153060)(7193020); SRVR:DB7PR08MB3017; x-ms-traffictypediagnostic: DB7PR08MB3017: x-microsoft-antispam-prvs: x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(3230021)(908002)(999002)(5005026)(6040522)(8220055)(2401047)(8121501046)(3002001)(3231475)(944501520)(4982022)(52105112)(93006095)(93001095)(10201501046)(6055026)(6041310)(20161123558120)(20161123564045)(20161123560045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123562045)(201708071742011)(7699051)(76991095); SRVR:DB7PR08MB3017; BCL:0; PCL:0; RULEID:; SRVR:DB7PR08MB3017; x-forefront-prvs: 0903DD1D85 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(39850400004)(396003)(136003)(376002)(366004)(346002)(53754006)(40434004)(189003)(199004)(8936002)(7736002)(2906002)(81166006)(97736004)(6116002)(3846002)(81156014)(6636002)(186003)(102836004)(561944003)(6506007)(7696005)(72206003)(305945005)(8676002)(6346003)(26005)(33656002)(478600001)(55016002)(68736007)(966005)(66066001)(14454004)(6436002)(53936002)(74316002)(5660300001)(76176011)(4326008)(9686003)(2501003)(25786009)(6306002)(106356001)(105586002)(5024004)(446003)(316002)(93156006)(86362001)(2940100002)(11346002)(486006)(110136005)(14444005)(476003)(256004)(71190400001)(99286004)(71200400001); DIR:OUT; SFP:1101; SCL:1; SRVR:DB7PR08MB3017; H:DB7PR08MB3209.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) authentication-results: spf=none (sender IP is ) smtp.mailfrom=Sakar.Arora@arm.com; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: QwN5v+MOpTNCjui7ntNdx0t1wM8YlHEBDAI6E7MiA5zm2uloZ8qZVlY73v52AkWMGyYLwUOyEThy/yEKgpPOd1wiTv8fZdepCbUqX2ASHs08taZ7LZStOneXEwJRp+I5AhNOmD+vq0pi0AbrvW/ukfUFj0FemrSYJ2ssd+MRRJ0etHctwidChLufmSJ5OUiz1ZuGkxWXM5G1guWxDZUi9QXYuX7DQQKyMUCsNhJ2YBhEo6NgVVubzDtso/WWFSIqQQsU04+m5SZrpE5j6d1dK2cFKBHFrq1q0WO/GoHKPDk1EqFJu0z7v5jmxFjloNoL spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-Network-Message-Id: d638fbd7-dbc7-4ca8-87c5-08d66eec3865 X-MS-Exchange-CrossTenant-originalarrivaltime: 31 Dec 2018 06:50:15.5925 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB7PR08MB3017 Subject: [edk2-test][RFC] Integrating SBBR tests into SCT - A design proposal X-BeenThere: edk2-devel@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: EDK II Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Dec 2018 06:50:20 -0000 Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi All, Introduction --> The intent is to run SBBR tests using SCT infrastructure. This is the SBBR= specification link for reference http://infocenter.arm.com/help/topic/com.= arm.doc.den0044c/Server_Base_Boot_Requirements_v1_1_Arm_DEN_0044C.pdf Majority of SBBR requirements can be tested by current SCT code as is. A fe= w more SBBR specific tests are needed to cover complete spec. Proposal --> An ideal setup is new SBBR specific code residing in edk2-test master. SCT = test infrastructure will run tests for UEFI spec compliance or just SBBR sp= ec compliance based on command line arguments. This would require run time = classification of tests. - Details Each SCT test provides a list of sub-tests to verify various aspects of the= requirement. Each sub-test is assigned some attributes (AUTO, MANUAL, DEST= RUCTIVE, or RESET_REQUIRED) which are referred to by the test infrastructur= e to do pre and post processing for it. The proposal is to add 2 new attributes SBBR and SBBR_EXCL to the existing = set. The test infrastructure will decide whether to run or skip a test base= d on command line parameters and these new attributes. - The new attributes SBBR : Tests with this attribute are required by both SBBR and UEFI. SBBR_EXCL : Tests with this attribute are required by SBBR exclusively. - Examples Shell> Sct.efi -a -sbbr Would run tests required by SBBR, i.e., tests with attributes SBBR or SBBR_= EXCL. Shell> Sct.efi -a Would run tests required by UEFI, i.e., all tests excluding those with attr= ibute SBBR_EXCL. - SCT test pre-processing flow The following logic will be added to the pre-processing flow. If (SCT_FOR_SBBR) { If (!TEST_CASE_FOR_SBBR(TestCaseAttribute)) goto SkipTest; } else { If (TEST_CASE_FOR_SBBR_EXCL(TestCaseAttribute)) goto SkipTest; } Comments are welcome. Thanks, Sakar IMPORTANT NOTICE: The contents of this email and any attachments are confid= ential and may also be privileged. If you are not the intended recipient, p= lease notify the sender immediately and do not disclose the contents to any= other person, use it for any purpose, or store or copy the information in = any medium. Thank you.