From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by mx.groups.io with SMTP id smtpd.web11.10076.1589889760622241429 for ; Tue, 19 May 2020 05:02:40 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=none, err=permanent DNS error (domain: linux.intel.com, ip: 192.55.52.43, mailfrom: maciej.rabeda@linux.intel.com) IronPort-SDR: t7lwOQN4oSHZruTEfwcucvevawCHxdV89xAzf2W2idTDijrME7pth9498Om0VEiLpn3rePCADG ilode6BsAO+w== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 May 2020 05:02:39 -0700 IronPort-SDR: 6RcEfc4U/jsP5wJIsMe4HJ/HkAXqL8zOZ3h/ofzLSBcVZCa0SyPK9/9uJDrEh6q+r3fv3JMcpM 8LJiltGFCewQ== X-IronPort-AV: E=Sophos;i="5.73,410,1583222400"; d="scan'208";a="439595628" Received: from mrabeda-mobl.ger.corp.intel.com (HELO [10.213.7.209]) ([10.213.7.209]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 May 2020 05:02:38 -0700 Subject: Re: [edk2][PATCH 1/1] HttpBoot: handle servers which may FIN after file sizing in HttpBootLoadFile To: Andrei Warkentin , devel@edk2.groups.io Cc: jiaxin.wu@intel.com, siyuan.fu@intel.com References: <20200514070223.48194-1-andrey.warkentin@gmail.com> From: "Maciej Rabeda" Message-ID: <0b179e68-1454-75a1-18c8-fb26aab4dc2a@linux.intel.com> Date: Tue, 19 May 2020 14:02:24 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.8.0 MIME-Version: 1.0 In-Reply-To: <20200514070223.48194-1-andrey.warkentin@gmail.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: pl Hi Andrei, Thanks for catching this. Smells a bit like a potential issue with Python's implementation of underlying socketserver and interoperability with Windows. https://stackoverflow.com/questions/46785946/python-socketserver-sends-fin-flag Nevertheless, HTTP client should be aware that it can receive TCP FIN from HTTP server at any time. As for your patch: 1. Please add maintainer and reviewers as CC: to the patch before 'Signed-off-by:'. Example: https://edk2.groups.io/g/devel/message/58045 2. Patch & commit title should contain names of package and driver that are subject to changes - so it should start with 'NetworkPkg/HttpBootDxe:' 3. HttpBootDxeLoadFile is an implementation of EFI_LOAD_FILE_PROTOCOL.LoadFile() protocol function and UEFI specification clearly describes possible error codes. Whatever code is handling boot menu and that protocol, it may not be able to react correctly to that error code. From the error codes that are available, EFI_ABORTED best fits the 'Load process was cancelled by the server' narrative. HttpBootDxeLoadFile, line 602 contains an if (EFI_ERROR (Status)) clause. Please, catch EFI_CONNECTION_FIN error inside that clause. Use AsciiPrint to report the following message to the user: "\n Error: Server has terminated the connection." Set Status to EFI_ABORTED before the whole function returns it. Thanks, Maciej On 14-May-20 09:02, Andrei Warkentin wrote: > Python http.server seems to FIN after the first HEAD request to size > the loaded file is completed and ACKed. What happens next is interesting. > On low latency connections, the GET request to download may get sent > after the server sends the FIN but before the client has a chance to > process it. The net result is: > - Server ignores GET > - HttpBootLoadFile returns EFI_CONNECTION_FIN. Boot fails. > > In the other case, client handles the FIN before attempting the GET, > so there's a proper three-way handshake as part of GET. > > The solution is to retry HttpBootLoadFile 2 times if it returns > EFI_CONNECTION_FIN. This is because HttpBootLoadFile may issue up to > 3 requests: HEAD/GET to get size and final GET to load. Some servers > may send a FIN after each request. The first request (HEAD) is not > supposed to fail this way, so only the two subsequent GET request > may result in a FIN. > > Fixes https://bugzilla.tianocore.org/show_bug.cgi?id=2720 > > Signed-off-by: Andrei Warkentin > --- > NetworkPkg/HttpBootDxe/HttpBootImpl.c | 27 +++++++++++++++++++- > NetworkPkg/HttpDxe/HttpImpl.c | 5 ++++ > 2 files changed, 31 insertions(+), 1 deletion(-) > > diff --git a/NetworkPkg/HttpBootDxe/HttpBootImpl.c b/NetworkPkg/HttpBootDxe/HttpBootImpl.c > index 4a51f35cdd..2d74d5f293 100644 > --- a/NetworkPkg/HttpBootDxe/HttpBootImpl.c > +++ b/NetworkPkg/HttpBootDxe/HttpBootImpl.c > @@ -288,6 +288,7 @@ HttpBootDhcp ( > @retval EFI_NOT_STARTED The driver is in stopped state. > @retval EFI_BUFFER_TOO_SMALL The BufferSize is too small to read the boot file. BufferSize has > been updated with the size needed to complete the request. > + @retval EFI_CONNECTION_FIN Server had closed the connection while we were waiting downloading. > @retval EFI_DEVICE_ERROR An unexpected network error occurred. > @retval Others Other errors as indicated. > > @@ -535,6 +536,7 @@ HttpBootStop ( > @retval EFI_BUFFER_TOO_SMALL The BufferSize is too small to read the current directory entry. > BufferSize has been updated with the size needed to complete > the request. > + @retval EFI_CONNECTION_FIN Server had closed the connection while we were waiting for a response. > > **/ > EFI_STATUS > @@ -553,6 +555,7 @@ HttpBootDxeLoadFile ( > BOOLEAN UsingIpv6; > EFI_STATUS Status; > HTTP_BOOT_IMAGE_TYPE ImageType; > + UINTN MaxTries; > > if (This == NULL || BufferSize == NULL || FilePath == NULL) { > return EFI_INVALID_PARAMETER; > @@ -598,7 +601,29 @@ HttpBootDxeLoadFile ( > // Load the boot file. > // > ImageType = ImageTypeMax; > - Status = HttpBootLoadFile (Private, BufferSize, Buffer, &ImageType); > + // > + // HttpBootLoadFile may issue up to 2 requests: HEAD/GET to get > + // size and final GET to load. Some servers may send a FIN after > + // each request. The first request (HEAD) is not supposed to > + // fail this way, so only the two possible GETs need the special > + // handling. > + // > + MaxTries = 2; > + do { > + Status = HttpBootLoadFile (Private, BufferSize, Buffer, &ImageType); > + if (Status == EFI_CONNECTION_FIN) { > + if (Private->HttpCreated) { > + // > + // Tear down HTTP/TCP state entirely. Http->Configure (NULL) is not > + // sufficient (EFI_ACCESS_DENIED from TCP stack on subsequent > + // HttpBootLoadFile. > + // > + HttpIoDestroyIo (&Private->HttpIo); > + Private->HttpCreated = FALSE; > + } > + } > + } while (MaxTries-- && Status == EFI_CONNECTION_FIN); > + > if (EFI_ERROR (Status)) { > if (Status == EFI_BUFFER_TOO_SMALL && (ImageType == ImageTypeVirtualCd || ImageType == ImageTypeVirtualDisk)) { > Status = EFI_WARN_FILE_SYSTEM; > diff --git a/NetworkPkg/HttpDxe/HttpImpl.c b/NetworkPkg/HttpDxe/HttpImpl.c > index 5a6ecbc9d9..34a33b09f7 100644 > --- a/NetworkPkg/HttpDxe/HttpImpl.c > +++ b/NetworkPkg/HttpDxe/HttpImpl.c > @@ -959,6 +959,8 @@ HttpBodyParserCallback ( > @retval EFI_OUT_OF_RESOURCES Failed to complete the operation due to lack of resources. > @retval EFI_NOT_READY Can't find a corresponding Tx4Token/Tx6Token or > the EFI_HTTP_UTILITIES_PROTOCOL is not available. > + @retval EFI_CONNECTION_FIN Server had closed the connection while we were waiting for > + a response. > > **/ > EFI_STATUS > @@ -1528,6 +1530,9 @@ Error: > @retval EFI_OUT_OF_RESOURCES Could not allocate enough system resources. > @retval EFI_ACCESS_DENIED An open TCP connection is not present with the host > specified by response URL. > + > + @retval EFI_CONNECTION_FIN Server had closed the connection while we were waiting for > + a response. > **/ > EFI_STATUS > EFIAPI