[PATCH v3 0/3] Return GetVariable() attr if EFI_BUFFER_TOO_SMALL

[PATCH v3 0/3] Return GetVariable() attr if EFI_BUFFER_TOO_SMALL

[Michael Kubacki]

From: Michael Kubacki <michael.kubacki@microsoft.com>

This patch series updates the GetVariable() implementation
to return Attributes in the case EFI_BUFFER_TOO_SMALL is returned.

* [PATCH v3 1/3] Makes the functional change in the DXE/MM variable driver.
* [PATCH v3 2/3] Makes the functional change in the PEI variable driver.
* [PATCH v3 3/3] Removes a change made in NetworkPkg that worked around the
  previous behavior when EFI_BUFFER_TOO_SMALL is returned.

V3 changes:
* Apply the same GetVariable() behavior in VariablePei so it is consistent
  with the DXE/MM variable driver implementation.
V2 changes:
* Revert commit 6896efdec2 in NetworkPkg since the variable Attributes are
  now returned in the case of EFI_BUFFER_TOO_SMALL.

Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Siyuan Fu <siyuan.fu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Maciej Rabeda <maciej.rabeda@linux.intel.com>
Cc: Jian J Wang <jian.j.wang@intel.com>
Cc: Jiaxin Wu <jiaxin.wu@intel.com>
Cc: Hao A Wu <hao.a.wu@intel.com>
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>

Michael Kubacki (3):
  MdeModulePkg Variable: Return GetVariable() attr if
    EFI_BUFFER_TOO_SMALL
  MdeModulePkg VariablePei: Return GetVariable() attr if
    EFI_BUFFER_TOO_SMALL
  Revert "NetworkPkg/TlsAuthConfigDxe: fix TlsCaCertificate attributes
    retrieval"

 MdeModulePkg/Universal/Variable/Pei/Variable.c                     | 19 +++++++-------
 MdeModulePkg/Universal/Variable/RuntimeDxe/Variable.c              | 10 +++++---
 MdeModulePkg/Universal/Variable/RuntimeDxe/VariableSmmRuntimeDxe.c | 10 +++++---
 NetworkPkg/TlsAuthConfigDxe/TlsAuthConfigImpl.c                    | 27 +-------------------
 4 files changed, 24 insertions(+), 42 deletions(-)

-- 
2.16.3.windows.1

Re: [edk2-devel] [PATCH v3 0/3] Return GetVariable() attr if EFI_BUFFER_TOO_SMALL

[Laszlo Ersek]

Hi Michael,

On 03/27/20 22:55, Michael Kubacki wrote:
> From: Michael Kubacki <michael.kubacki@microsoft.com>
> 
> This patch series updates the GetVariable() implementation
> to return Attributes in the case EFI_BUFFER_TOO_SMALL is returned.
> 
> * [PATCH v3 1/3] Makes the functional change in the DXE/MM variable driver.
> * [PATCH v3 2/3] Makes the functional change in the PEI variable driver.
> * [PATCH v3 3/3] Removes a change made in NetworkPkg that worked around the
>   previous behavior when EFI_BUFFER_TOO_SMALL is returned.
> 
> V3 changes:
> * Apply the same GetVariable() behavior in VariablePei so it is consistent
>   with the DXE/MM variable driver implementation.

Can you please configure your git-send-email machinery to post patch
series with "shallow threading"? Otherwise the patch emails are not
linked under the cover letter, and they fly apart.

I'm asking in particular because this series modifies both MdeModulePkg
and NetworkPkg. In such cases usually one of the affected maintainers
volunteers for merging the full set (once review is complete for the
entire set). And collecting the individual patches without proper
shallow threading is very difficult.

The git options are the following:

git config sendemail.chainreplyto false
git config sendemail.thread       true

They are also put in place (with many other useful settings) by
"BaseTools/Scripts/SetupGit.py".

Or is there another problem with posting the patches? (I assume you post
the full set with a single git-send-email invocation.)

Thanks!
Laszlo

Re: [edk2-devel] [PATCH v3 0/3] Return GetVariable() attr if EFI_BUFFER_TOO_SMALL

[Michael Kubacki]

I have those options set correctly in git config.

After a quick look, as far as I can tell, this is because the Message-Id 
in my email is modified such the the In-Reply-To and References headers 
do not refer to the actual Message-Id in the cover letter:

Cover letter:

Subject: [edk2-devel] [PATCH v3 0/3] Return GetVariable() attr if 
EFI_BUFFER_TOO_SMALL
Date: Fri, 27 Mar 2020 14:55:33 -0700
Message-Id: <160047F24E1D38F5.15390@groups.io>

The original Message-Id in the cover letter was moved to 
X-Microsoft-Original-Message-Id:

X-Microsoft-Original-Message-ID:
  <20200327215536.9556-1-michael.kubacki@outlook.com>

The first patch in the series:

Subject: [edk2-devel] [PATCH v3 1/3] MdeModulePkg Variable: Return 
GetVariable() attr if EFI_BUFFER_TOO_SMALL
Date: Fri, 27 Mar 2020 14:55:34 -0700
Message-Id: <160047F33E58AD06.17905@groups.io>
In-Reply-To: <20200327215536.9556-1-michael.kubacki@outlook.com>
References: <20200327215536.9556-1-michael.kubacki@outlook.com>

Please let me know if you have suggestions. I'll look into it more.

Thanks,
Michael

On 3/31/2020 4:55 PM, Laszlo Ersek wrote:
> Hi Michael,
> 
> On 03/27/20 22:55, Michael Kubacki wrote:
>> From: Michael Kubacki <michael.kubacki@microsoft.com>
>>
>> This patch series updates the GetVariable() implementation
>> to return Attributes in the case EFI_BUFFER_TOO_SMALL is returned.
>>
>> * [PATCH v3 1/3] Makes the functional change in the DXE/MM variable driver.
>> * [PATCH v3 2/3] Makes the functional change in the PEI variable driver.
>> * [PATCH v3 3/3] Removes a change made in NetworkPkg that worked around the
>>    previous behavior when EFI_BUFFER_TOO_SMALL is returned.
>>
>> V3 changes:
>> * Apply the same GetVariable() behavior in VariablePei so it is consistent
>>    with the DXE/MM variable driver implementation.
> 
> Can you please configure your git-send-email machinery to post patch
> series with "shallow threading"? Otherwise the patch emails are not
> linked under the cover letter, and they fly apart.
> 
> I'm asking in particular because this series modifies both MdeModulePkg
> and NetworkPkg. In such cases usually one of the affected maintainers
> volunteers for merging the full set (once review is complete for the
> entire set). And collecting the individual patches without proper
> shallow threading is very difficult.
> 
> The git options are the following:
> 
> git config sendemail.chainreplyto false
> git config sendemail.thread       true
> 
> They are also put in place (with many other useful settings) by
> "BaseTools/Scripts/SetupGit.py".
> 
> Or is there another problem with posting the patches? (I assume you post
> the full set with a single git-send-email invocation.)
> 
> Thanks!
> Laszlo
> 

Re: [edk2-devel] [PATCH v3 0/3] Return GetVariable() attr if EFI_BUFFER_TOO_SMALL

[Laszlo Ersek]

On 04/01/20 03:15, Michael Kubacki wrote:
> I have those options set correctly in git config.
> 
> After a quick look, as far as I can tell, this is because the
> Message-Id in my email is modified such the the In-Reply-To and
> References headers do not refer to the actual Message-Id in the cover
> letter:
> 
> Cover letter:
> 
> Subject: [edk2-devel] [PATCH v3 0/3] Return GetVariable() attr if
> EFI_BUFFER_TOO_SMALL
> Date: Fri, 27 Mar 2020 14:55:33 -0700
> Message-Id: <160047F24E1D38F5.15390@groups.io>
> 
> The original Message-Id in the cover letter was moved to
> X-Microsoft-Original-Message-Id:
> 
> X-Microsoft-Original-Message-ID:
>  <20200327215536.9556-1-michael.kubacki@outlook.com>
> 
> The first patch in the series:
> 
> Subject: [edk2-devel] [PATCH v3 1/3] MdeModulePkg Variable: Return
> GetVariable() attr if EFI_BUFFER_TOO_SMALL
> Date: Fri, 27 Mar 2020 14:55:34 -0700
> Message-Id: <160047F33E58AD06.17905@groups.io>
> In-Reply-To: <20200327215536.9556-1-michael.kubacki@outlook.com>
> References: <20200327215536.9556-1-michael.kubacki@outlook.com>
> 
> Please let me know if you have suggestions. I'll look into it more.

(1) *Normally*, here's what I would tell you:

Please log in to your groups.io account, and navigate to the following
setting:

Your name (upper right corner)
  Account
    Preferences (top of left sidebar)
      I always want copies of my own emails (checkbox at the bottom)

Please *untick* this checkbox.

The name of the checkbox is very misleading. The actual behavior is the
following:

  "I am a gmail user, and gmail de-duplicates messages that are sent to
   me both directly, and reflected through the mailing list. As a
   special case, the same de-duplication applies to messages I send to
   the list, and are reflected to me by the list -- gmail just hides
   those from me as "seen". I don't like this; I want my reflected
   copies of messages that I sent. Because the de-duplication is
   Message-ID based, groups.io should please falsify the Message-ID on
   those messages that I send *AND* are delivered back to me *only*"

Meaning, if you have this box checked, then groups.io places
<....@groups.io>-style message IDs in the emails that it sends back *to
you* (only to you) *and* that originate from you.


(2) *However*, that is not the problem here. (Or, more precisely, not
the only problem.) Because, as I say above, this message-ID
re-generation by groups.io only applies to *you*; so the threading of
your patch series would only be broken in *your* list folder, not in
mine (or in any other list subscriber's).

But, the threading *is* broken on my end too.

And the reason for that is that the SMTP server that you use, ignores
the Message-ID put in place by git-send-email, and generates its own.

Your original message ID (as stated above), from git-send-email, is:

  20200327215536.9556-1-michael.kubacki@outlook.com

But the message ID in my list folder is:

  MWHPR07MB3440A69B5AFAD0E373974FBFE9CC0@MWHPR07MB3440.namprd07.prod.outlook.com

Therefore, the original message-id (from git-send-email) is not moved to
the X-Microsoft-Original-Message-ID header by groups.io -- instead, it
is moved there by your SMTP server, at <outlook.com>.

Can you disable that somehow? It is incorrect behavior. An SMTP server
itself should only put a Message-ID on an outgoing email if that email
doesn't already come with one.

To summarize, there are two issues:

(a) your groups.io account settings have the gmail-oriented Message-ID
falsification enabled, which breaks threading for you (and only for you)

(b) your SMTP server overwrites the git-send-email-generated Message-ID
with its own, which breaks threading for every list subscriber
(including you).

You can easily remedy (a) in your groups.io account settings. Not sure
how you can fix problem (b) -- can you use a different SMTP server
maybe?

(I've been very happy that we've finally seen patches posted to the list
from a @microsoft.com email address -- it would be fantastic if that
continued, with the threading fixed even!)

Thanks!
Laszlo