From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=192.55.52.88; helo=mga01.intel.com; envelope-from=eric.jin@intel.com; receiver=edk2-devel@lists.01.org Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id B2FF5211B5A43 for ; Mon, 14 Jan 2019 23:22:51 -0800 (PST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 14 Jan 2019 23:22:50 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,481,1539673200"; d="scan'208";a="134680322" Received: from fmsmsx107.amr.corp.intel.com ([10.18.124.205]) by fmsmga002.fm.intel.com with ESMTP; 14 Jan 2019 23:22:50 -0800 Received: from fmsmsx156.amr.corp.intel.com (10.18.116.74) by fmsmsx107.amr.corp.intel.com (10.18.124.205) with Microsoft SMTP Server (TLS) id 14.3.408.0; Mon, 14 Jan 2019 23:22:50 -0800 Received: from shsmsx152.ccr.corp.intel.com (10.239.6.52) by fmsmsx156.amr.corp.intel.com (10.18.116.74) with Microsoft SMTP Server (TLS) id 14.3.408.0; Mon, 14 Jan 2019 23:22:50 -0800 Received: from shsmsx103.ccr.corp.intel.com ([169.254.4.150]) by SHSMSX152.ccr.corp.intel.com ([169.254.6.44]) with mapi id 14.03.0415.000; Tue, 15 Jan 2019 15:22:48 +0800 From: "Jin, Eric" To: Sakar Arora , Supreeth Venkatesh CC: "edk2-devel@lists.01.org" , "Jin, Eric" Thread-Topic: [edk2][edk2-test][RFC] Integrating SBBR tests into SCT - A design proposal Thread-Index: AdSgwgEf4U2uyT4uRYG+AR9zd3VV2QADkKdgAX1u4XAACrv4QACqU/agAIHQtBA= Date: Tue, 15 Jan 2019 07:22:48 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.239.127.40] MIME-Version: 1.0 Subject: Re: [edk2-test][RFC] Integrating SBBR tests into SCT - A design proposal X-BeenThere: edk2-devel@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: EDK II Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jan 2019 07:22:51 -0000 Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi Sakar, Thank for the clarification. Now, I understand that the SBBR wants to add/modify in existing edk2-test/u= efi-sct. In your description, '-s' is not suitable for sbbr purpose.=20 The fundamental point to be discussed is that edk2-test/uefi-sct is UEFI Se= lf Compliance Test and should keep the alignment to UEFI spec. To my understanding, the SBBR is published by arm and gives special require= ments on UEFI/ACPI/SMBIOS areas.=20 I remember some new files/new directories with the prefix "sbbr" in file/di= rectory name are adding into the edk2-test/uefi-sct In previous push reques= t emails, it makes me confused. And it is the reason that I ask for the pos= sible standalone space. Could you please send the patches for review? We can discuss further based = on them and explore the correct direction. At least, the baseline is it doe= sn't impact existing test result (I want to hear the comments from Supreeth= on this baseline).=20 Best Regards Eric -----Original Message----- From: Sakar Arora =20 Sent: Friday, January 11, 2019 8:23 PM To: Jin, Eric ; Supreeth Venkatesh ; edk2-devel@lists.01.org Cc: Prasanth Pulla Subject: RE: [edk2][edk2-test][RFC] Integrating SBBR tests into SCT - A des= ign proposal Hi Supreeth, Eric, My replies inlined. Thanks, Sakar -----Original Message----- From: Jin, Eric Sent: Tuesday, January 8, 2019 7:30 AM To: Supreeth Venkatesh ; Sakar Arora ; edk2-devel@lists.01.org Cc: Prasanth Pulla Subject: RE: [edk2][edk2-test][RFC] Integrating SBBR tests into SCT - A des= ign proposal Hello Sakar, SBBR spec is not published by UEFI Forum and can be considered as a supplem= ent to ARM UEFI. The evolution of edk2-test/uefi-sct should be tagged to UE= FI Spec only. Can we land SBBR test on standalone space under edk2-test currently or in s= hort-term? I mean SBBR test can leverage edk2-test/uefi-sct (as sub-module= )or not to build test efi files itself and these test efi files can be inte= grated into UEFI SCT to execute with user usage. [Sakar] If I understand correctly, you want a separate directory in edk2-te= st for SBBR code, rather than having it in edk2-test/uefi-sct. The problem I see here is that apart from standalone SBBR test cases, there= are some changes to the existing test cases inside edk2-test/uefi-sct, tha= t we would need pushed for SBBR compliance (These changes will only take ef= fect if "-sbbr" parameter is passed while running SCT). So SBBR tests are n= ot exactly standalone. Keeping all SBBR specific stuff away, would mean mai= ntaining patches to SCT code, which will be difficult. I hope I am making s= ense. For the new attributes - SBBR and SBBR_EXCL, I agree with Supreeth that the= y are more like the additional parameters. So they are not the similar case= s to (AUTO, MANUAL, DESTRUCTIVE, or RESET_REQUIRED). [Sakar] The reasoning for proposing these 2 attributes is this. There are some SBBR tests that are needed only for SBBR compliance. When SC= T app is run *without* the "-sbbr" cmdline parameter, it should run all tes= ts, excluding those specific to SBBR. So it needs to differentiate, at runt= ime, tests that are valid for both SBBR and UEFI compliance from the ones t= hat are exclusively for SBBR. Tests with attribute SBBR are meant to run fo= r both UEFI and SBBR compliance, while those with attribute SBBR_EXCL are o= nly meant for SBBR compliance. If the intention is to distinguish SBBR test from existing test, It can be = implemented with "-s" parameter plus the SBBR test sequence easily. [Sakar] Using the "-s" parameter with test sequence file is an option I exp= lored a bit. It doesn't seem to offer control over what sub-tests to run within a test s= cenario, which is something we want while adding SBBR support in SCT. For = example, there are some sub-tests that need to be removed from some main te= sts, for SBBR compliance. As far as I understand, sequence file does not pr= ovide that level of control. Please correct me if I am missing something. Above is my quick response and I will continue to consider it these days an= d look at SBBR spec. Any input is welcome. Best Regards Eric -----Original Message----- From: Supreeth Venkatesh Sent: Tuesday, January 8, 2019 4:14 AM To: Sakar Arora ; edk2-devel@lists.01.org; Jin, Eric <= eric.jin@intel.com> Cc: Prasanth Pulla Subject: RE: [edk2][edk2-test][RFC] Integrating SBBR tests into SCT - A des= ign proposal Sakar, Looks good. However, Why do we need two additional parameters SBBR and SBBR_EXCL? Since SBBR is a subset/superset of UEFI specifications, would "sbbr_standal= one" or similar (just sbbr) which would run all the sbbr tests including on= es that is already part of UEFI-SCT Suffice? Lets wait for additional comments from Eric. Thanks, Supreeth -----Original Message----- From: Sakar Arora Sent: Monday, December 31, 2018 12:50 AM To: edk2-devel@lists.01.org; eric.jin@intel.com; Supreeth Venkatesh Cc: Prasanth Pulla Subject: [edk2][edk2-test][RFC] Integrating SBBR tests into SCT - A design = proposal Hi All, Introduction --> The intent is to run SBBR tests using SCT infrastructure. This is the SBBR= specification link for reference http://infocenter.arm.com/help/topic/com.= arm.doc.den0044c/Server_Base_Boot_Requirements_v1_1_Arm_DEN_0044C.pdf Majority of SBBR requirements can be tested by current SCT code as is. A fe= w more SBBR specific tests are needed to cover complete spec. Proposal --> An ideal setup is new SBBR specific code residing in edk2-test master. SCT = test infrastructure will run tests for UEFI spec compliance or just SBBR sp= ec compliance based on command line arguments. This would require run time = classification of tests. - Details Each SCT test provides a list of sub-tests to verify various aspects of the= requirement. Each sub-test is assigned some attributes (AUTO, MANUAL, DEST= RUCTIVE, or RESET_REQUIRED) which are referred to by the test infrastructur= e to do pre and post processing for it. The proposal is to add 2 new attributes SBBR and SBBR_EXCL to the existing = set. The test infrastructure will decide whether to run or skip a test base= d on command line parameters and these new attributes. - The new attributes SBBR : Tests with this attribute are required by both SBBR and UEFI. SBBR_EXCL : Tests with this attribute are required by SBBR exclusively. - Examples Shell> Sct.efi -a -sbbr Would run tests required by SBBR, i.e., tests with attributes SBBR or SBBR_= EXCL. Shell> Sct.efi -a Would run tests required by UEFI, i.e., all tests excluding those with attr= ibute SBBR_EXCL. - SCT test pre-processing flow The following logic will be added to the pre-processing flow. If (SCT_FOR_SBBR) { If (!TEST_CASE_FOR_SBBR(TestCaseAttribute)) goto SkipTest; } else { If (TEST_CASE_FOR_SBBR_EXCL(TestCaseAttribute)) goto SkipTest; } Comments are welcome. Thanks, Sakar IMPORTANT NOTICE: The contents of this email and any attachments are confid= ential and may also be privileged. If you are not the intended recipient, p= lease notify the sender immediately and do not disclose the contents to any= other person, use it for any purpose, or store or copy the information in = any medium. Thank you. IMPORTANT NOTICE: The contents of this email and any attachments are confid= ential and may also be privileged. If you are not the intended recipient, p= lease notify the sender immediately and do not disclose the contents to any= other person, use it for any purpose, or store or copy the information in = any medium. Thank you.