From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by mx.groups.io with SMTP id smtpd.web12.11659.1602162647156011880 for ; Thu, 08 Oct 2020 06:10:47 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=BBRnpVrC; spf=pass (domain: redhat.com, ip: 216.205.24.124, mailfrom: lersek@redhat.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1602162646; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=h+jOcw+Seug3DKrTjBIoIxVGpkadOUrNSvoZw1QPwxA=; b=BBRnpVrCmZjfETBI+5GkiL77PmgvgKfpkIAtHEobrM8YMVULgyMhGFI9o5QR3piB5Zt7lC uhxrzbXN/jgGbH4Gu3vYYYAsBE+vCkvbbMmFPUPopRizivurkz6GhagGylpxMJtsGlXKzQ B9s3zP8If21fY0Yj4Z1pKkW0xyQBa7c= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-283-kyBv5FAfNB6sOwPImaZi8Q-1; Thu, 08 Oct 2020 09:10:42 -0400 X-MC-Unique: kyBv5FAfNB6sOwPImaZi8Q-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 3781F81EE65; Thu, 8 Oct 2020 13:10:41 +0000 (UTC) Received: from lacos-laptop-7.usersys.redhat.com (ovpn-115-66.ams2.redhat.com [10.36.115.66]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0F4856EF5D; Thu, 8 Oct 2020 13:10:39 +0000 (UTC) Subject: Re: [EXTERNAL] Re: [edk2-devel] VariablePolicy: Final Changes Thread 2 - ECC & UnitTest To: Bret Barkelew , Andrew Fish , "devel@edk2.groups.io" Cc: "Kinney, Michael D" References: <742c37aa-59a8-ac80-ee61-5173be35afea@redhat.com> <6AB2D4A8-B715-4755-A2A0-804BBC292AA3@apple.com> From: "Laszlo Ersek" Message-ID: <529d2f8d-ee1a-33fe-c8e3-d0b1a9491447@redhat.com> Date: Thu, 8 Oct 2020 15:10:39 +0200 MIME-Version: 1.0 In-Reply-To: X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=lersek@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit On 10/07/20 18:44, Bret Barkelew wrote: > I can test it again, but I hit a compiler complaint around the STATIC CONST solution. I'm curious. > Also, it doesn’t apply in all cases because I have at lease half a dozen cases that say “test X, prove negative, twiddle value, test Y, prove positive”. That works with the last example I gave; "STATIC CONST TestDataTemplate" + normal local TestData. CopyMem from template to local variable, check and tweak local var as needed. Please check the end of my previous email again. EFI_STATUS FoobarTest ( VOID ) { STATIC CONST TEST_DATA TestDataTemplate = { 42 }; TEST_DATA TestData; CopyMem (&TestData, TestDataTemplate, sizeof (TEST_DATA)); // // run a test on TestData, verify that it fails // TestData.Value = 43; // // run the same test on TestData, verify that it succeeds // } thanks Laszlo > > I’ll try a few more things, but I may reissue the patch series and withhold the tests until they can be rewritten to match. I don’t want them to hold up VarPol any longer. > > - Bret > > From: Laszlo Ersek > Sent: Wednesday, October 7, 2020 8:51 AM > To: Andrew Fish; devel@edk2.groups.io > Cc: Kinney, Michael D; Bret Barkelew > Subject: [EXTERNAL] Re: [edk2-devel] VariablePolicy: Final Changes Thread 2 - ECC & UnitTest > > On 10/07/20 16:27, Andrew Fish wrote: >> For case 1 I thought the size had to be > 8 bytes, not just a struct? Maybe that is compiler specific? > > Honestly, I've got no clue. I just remember we must avoid initializers > for objects that do not have static storage duration. > > Laszlo > >> >> Sent from my iPhone >> >>> On Oct 7, 2020, at 6:43 AM, Laszlo Ersek wrote: >>> >>> On 10/07/20 03:46, Michael D Kinney wrote: >>>> >>>> Bret, >>>> >>>> Initializing variable in declaration for structures and arrays >>>> introduces use of intrinsics. Since it is possible for unit test >>>> sources to be used for both host and target tests, I recommend we >>>> continue to follow the EDK II coding style for unit tests to support >>>> maximum compatibility and code reuse. >>>> >>>> Using a module global variable with initializers instead of >>>> initializing a local declaration is the same amount of work, so I do >>>> not believe that will result in fewer tests. >>>> >>>> I agree it is useful to have the test data next to the test code. This >>>> can be accomplished by breaking up into more files so the test data is >>>> immediately above the test function the test data is used. Does ECC >>>> raise an error if a module global is placed between 2 functions? A >>>> 2nd approach to put the module global immediately above the test >>>> function the test data is used. >>> >>> Consider the following example structure type, for the sake of >>> discussion: >>> >>> typedef struct { >>> UINT32 Value; >>> } TEST_DATA; >>> >>> >>> * Case#1: block scope, automatic storage duration >>> >>> EFI_STATUS >>> FoobarTest ( >>> VOID >>> ) >>> { >>> TEST_DATA TestData = { 42 }; >>> // ... >>> } >>> >>> Problem: uses intrinsics. >>> >>> >>> * Case#2: file scope, static storage duration. >>> >>> STATIC CONST TEST_DATA mTestData = { 42 }; >>> >>> EFI_STATUS >>> FoobarTest ( >>> VOID >>> ) >>> { >>> // ... >>> } >>> >>> Problem: either "mTestData" is textually far from FoobarTest(), or -- if >>> we keep them close to each other -- we mix variable definitions with >>> function definitions, at file scope. >>> >>> >>> * Case #3: block scope, static storage duration. >>> >>> EFI_STATUS >>> FoobarTest ( >>> VOID >>> ) >>> { >>> STATIC CONST TEST_DATA TestData = { 42 }; >>> // ... >>> } >>> >>> Problem: there should be none. Does not involve intrinsics, and the >>> object definition is part of the function's scope. >>> >>> >>> If ECC does not recognize case#3 as valid, then that is an *ECC bug*. >>> >>> ECC has no reason to prevent case#3, as case#3 does not involve >>> intrinsics, and is a generally valid and useful C language construct (it >>> combines the life cycle of case#2 with the visibility of case#1). >>> >>> Again, if ECC rejects case#3, that's *definitely* a bug in ECC, and we >>> should fix it first. Given that ECC includes a full-blown C language >>> parser, the fix should not be too difficult -- check if the declaration >>> has the "static" storage-class specifier. >>> >>> ... In fact, I think that purely CONST-qualifying TestData might suffice >>> for shutting up ECC. See the following in >>> "BaseTools/Source/Python/Ecc/c.py", method >>> "CheckFuncLayoutLocalVariable": >>> >>>> for Result in ResultSet: >>>> if len(Result[1]) > 0 and 'CONST' not in Result[3]: >>>> PrintErrorMsg(ERROR_C_FUNCTION_LAYOUT_CHECK_NO_INIT_OF_VARIABLE, 'Variable Name: %s' % Result[0], FileTable, Result[2]) >>> >>> So case#3 should work through that avenue already, because case#3 has >>> CONST *too*. >>> >>> Now, in case#3, if "TestData" needs to undergo modifications, and so >>> CONST is not immediately desirable, that's solvable: >>> >>> EFI_STATUS >>> FoobarTest ( >>> VOID >>> ) >>> { >>> STATIC CONST TEST_DATA TestDataTemplate = { 42 }; >>> TEST_DATA TestData; >>> >>> CopyMem (&TestData, TestDataTemplate, sizeof (TEST_DATA)); >>> // ... >>> } >>> >>> Thanks >>> Laszlo >>> >>>> >>>> Best regards, >>>> >>>> Mike >>>> >>>> From: devel@edk2.groups.io On Behalf Of Bret Barkelew via groups.io >>>> Sent: Tuesday, October 6, 2020 5:28 PM >>>> To: devel@edk2.groups.io >>>> Subject: [edk2-devel] VariablePolicy: Final Changes Thread 2 - ECC & UnitTest >>>> >>>> Ive worked through all the ECC issues with Variable Policy (AND the UnitTests) on this branch: >>>> Commits · corthon/edk2 (github.com) >>>> >>>> I even wrote the Main() entry point lib that Laszlo suggested (it works rather nicely): >>>> TEMP: Staging for HostTest entry point · corthon/edk2@4ce5210 (github.com) >>>> >>>> However, theres one that I just cant get past and I would like to take it up with the community. I dont think that UnitTests should have to deal with the cant initialize variables in declaration check. Almost none of the solutions that I tested worked, and the ones that did were too cumbersome. They failed on two key points that are important for test writing: >>>> >>>> * They were annoying to write ===> fewer tests. >>>> * They moved even more of the test case data away from the test ===> harder to read tests. >>>> >>>> I would like to move for an exception for unit tests (or at least host-based unit tests), but I dont know how to accomplish that from a technical standpoint. >>>> >>>> Thoughts? >>>> >>>> - Bret >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>> >>> >>> >>> >>> >>> >> >