HTTP boot failed on timeout

HTTP boot failed on timeout

[doron.bleiberg]

[-- Attachment #1: Type: text/plain, Size: 3143 bytes --]

Hi Community,

I've also opened same topic at general discussion group, but after digging more into this issue I think the relevant group is here where all the technical stuff happens.
A short introduction to my problem:
I'm trying to boot QEMU VM using HTTP boot.
I've modified Conf/target.txt as follows:
ACTIVE_PLATFORM       = OvmfPkg/OvmfPkgX64.dsc
TARGET_ARCH           = X64
TOOL_CHAIN_TAG        = GCC48
TARGET                = RELEASE

and OvmfPkg/OvmfPkgX64.dsc as follows:
DEFINE NETWORK_HTTP_BOOT_ENABLE     = TRUE
DEFINE SECURE_BOOT_ENABLE      = TRUE
BUILD_TARGETS                  = RELEASE

and compiled compiled OvmfPkg/OvmfPkgX64.dsc

I'm loading the OVMF.fd to QEMU VM in this way:
/usr/bin/qemu-system-x86_64 -name UEFI-HTTP-VM
-m 8192M -smp cpus=1
-enable-kvm -machine smm=off
-boot order=c *-bios /opt/gns3/images/QEMU/Ovmf.fd*
-drive file=/opt/gns3/projects/1a83274a-c57f-4337-8a0d-1e68a9312e9a/project-files/qemu/d8f37f0b-2b63-455f-b536-b309b9020e36/hda_disk.qcow2,if=ide,index=0,media=disk
-uuid d8f37f0b-2b63-455f-b536-b309b9020e36
-vnc 0.0.0.0:3
-monitor tcp:127.0.0.1:33919,server,nowait
-net none -device virtio-net-pci,mac=0c:2e:9a:0e:36:00,netdev=gns3-0 -netdev socket,id=gns3-0,udp=127.0.0.1:10017,localaddr=127.0.0.1:10016
-nographic

The VM starts and boots, the HTTP boot is kicking in and the download is starting. However, after few seconds the download is stopped and the boot terminates with 'Error: Server response timeout'.
The process is repeatable and always terminates at the same point (42% upload completed in my case).

I've drilled down the code and located the 'offending' code here:
file: NetworkPkg/HttpBootDxe/HttpBootSupport.c
line#: 1012
Offending Code: while (!HttpIo->IsRxDone && ((HttpIo->TimeoutEvent == NULL) || EFI_ERROR (gBS->CheckEvent (HttpIo->TimeoutEvent))))
Error Handler at line #1022: if (!HttpIo-> IsRxDone )

What I did i order to investigate:
I've also tried to add HTTP header: "Connection": "Keep-Alive" hoping it will result the HTTP server keeping sessions open.
Used Wireshark to capture and analyze the packets during download - I could not spot anything unusual. Eventually the packed transfer just stops
Raised HTTP server debug level - could not find anything valuable in the logs. As far as I can tell from the logs, the HTTP server works correctly and it seems the problem is in the client side (UEFI)
I'm downloading large file, so I've checked I have enough RAM to hold the image - seems OK. No complains regarding buffer size

Few inputs:
My HTTP server version: Apache httpd/2.4.6 (CentOS)
File size is I'm trying to download and boot: ~420MB

Error observed in the UEFI log when download terminates:
Error: Server response timeout.
BdsDxe: failed to load Boot0005 "UEFI HTTPv4 (MAC:0C2E9A0E3600)" from PciRoot(0x0)/Pci(0x3,0x0)/MAC(0C2E9A0E3600,0x1)/IPv4(0.0.0.0,0x0,DHCP,0.0.0.0,0.0.0.0,0.0.0.0)/Uri(): Not Found

For me it seems like a potential problem on the UEFI side, maybe related to Poll or other networking stuff.

I'll appreciate any help.

Doron

[-- Attachment #2: Type: text/html, Size: 4974 bytes --]

Re: [edk2-devel] HTTP boot failed on timeout

[Laszlo Ersek]

On 02/17/20 09:17, doron.bleiberg@ecitele.com wrote:
> Hi Community,
> 
> I've also opened same topic at general discussion group, but after digging more into this issue I think the relevant group is here where all the technical stuff happens.
> A short introduction to my problem:
> I'm trying to boot QEMU VM using HTTP boot.
> I've modified Conf/target.txt as follows:
> ACTIVE_PLATFORM       = OvmfPkg/OvmfPkgX64.dsc
> TARGET_ARCH           = X64
> TOOL_CHAIN_TAG        = GCC48
> TARGET                = RELEASE
> 
> and OvmfPkg/OvmfPkgX64.dsc as follows:
> DEFINE NETWORK_HTTP_BOOT_ENABLE     = TRUE
> DEFINE SECURE_BOOT_ENABLE      = TRUE
> BUILD_TARGETS                  = RELEASE
> 
> and compiled compiled OvmfPkg/OvmfPkgX64.dsc
> 
> I'm loading the OVMF.fd to QEMU VM in this way:
> /usr/bin/qemu-system-x86_64 -name UEFI-HTTP-VM
> -m 8192M -smp cpus=1
> -enable-kvm -machine smm=off
> -boot order=c *-bios /opt/gns3/images/QEMU/Ovmf.fd*
> -drive file=/opt/gns3/projects/1a83274a-c57f-4337-8a0d-1e68a9312e9a/project-files/qemu/d8f37f0b-2b63-455f-b536-b309b9020e36/hda_disk.qcow2,if=ide,index=0,media=disk
> -uuid d8f37f0b-2b63-455f-b536-b309b9020e36
> -vnc 0.0.0.0:3
> -monitor tcp:127.0.0.1:33919,server,nowait
> -net none -device virtio-net-pci,mac=0c:2e:9a:0e:36:00,netdev=gns3-0 -netdev socket,id=gns3-0,udp=127.0.0.1:10017,localaddr=127.0.0.1:10016
> -nographic
> 
> The VM starts and boots, the HTTP boot is kicking in and the download is starting. However, after few seconds the download is stopped and the boot terminates with 'Error: Server response timeout'.
> The process is repeatable and always terminates at the same point (42% upload completed in my case).
> 
> I've drilled down the code and located the 'offending' code here:
> file: NetworkPkg/HttpBootDxe/HttpBootSupport.c
> line#: 1012
> Offending Code: while (!HttpIo->IsRxDone && ((HttpIo->TimeoutEvent == NULL) || EFI_ERROR (gBS->CheckEvent (HttpIo->TimeoutEvent))))
> Error Handler at line #1022: if (!HttpIo-> IsRxDone )
> 
> What I did i order to investigate:
> I've also tried to add HTTP header: "Connection": "Keep-Alive" hoping it will result the HTTP server keeping sessions open.
> Used Wireshark to capture and analyze the packets during download - I could not spot anything unusual. Eventually the packed transfer just stops
> Raised HTTP server debug level - could not find anything valuable in the logs. As far as I can tell from the logs, the HTTP server works correctly and it seems the problem is in the client side (UEFI)
> I'm downloading large file, so I've checked I have enough RAM to hold the image - seems OK. No complains regarding buffer size
> 
> Few inputs:
> My HTTP server version: Apache httpd/2.4.6 (CentOS)
> File size is I'm trying to download and boot: ~420MB
> 
> Error observed in the UEFI log when download terminates:
> Error: Server response timeout.
> BdsDxe: failed to load Boot0005 "UEFI HTTPv4 (MAC:0C2E9A0E3600)" from PciRoot(0x0)/Pci(0x3,0x0)/MAC(0C2E9A0E3600,0x1)/IPv4(0.0.0.0,0x0,DHCP,0.0.0.0,0.0.0.0,0.0.0.0)/Uri(): Not Found
> 
> For me it seems like a potential problem on the UEFI side, maybe related to Poll or other networking stuff.
> 
> I'll appreciate any help.

As I mentioned on edk2-discuss, I'd suggest two tweaks (independently of
each other):

* try the builting virtio-net driver rather than the iPXE one. For that,
use:

  -device virtio-net-pci,[other properties],romfile=""

that is, add a 'romfile=""' property

* Try using a different -netdev backend than a UDP socket.

Thanks
Laszlo

Re: [edk2-devel] HTTP boot failed on timeout

[Laszlo Ersek]

On 02/17/20 11:29, Laszlo Ersek wrote:
> On 02/17/20 09:17, doron.bleiberg@ecitele.com wrote:
>> Hi Community,
>>
>> I've also opened same topic at general discussion group, but after digging more into this issue I think the relevant group is here where all the technical stuff happens.
>> A short introduction to my problem:
>> I'm trying to boot QEMU VM using HTTP boot.
>> I've modified Conf/target.txt as follows:
>> ACTIVE_PLATFORM       = OvmfPkg/OvmfPkgX64.dsc
>> TARGET_ARCH           = X64
>> TOOL_CHAIN_TAG        = GCC48
>> TARGET                = RELEASE
>>
>> and OvmfPkg/OvmfPkgX64.dsc as follows:
>> DEFINE NETWORK_HTTP_BOOT_ENABLE     = TRUE
>> DEFINE SECURE_BOOT_ENABLE      = TRUE
>> BUILD_TARGETS                  = RELEASE
>>
>> and compiled compiled OvmfPkg/OvmfPkgX64.dsc
>>
>> I'm loading the OVMF.fd to QEMU VM in this way:
>> /usr/bin/qemu-system-x86_64 -name UEFI-HTTP-VM
>> -m 8192M -smp cpus=1
>> -enable-kvm -machine smm=off
>> -boot order=c *-bios /opt/gns3/images/QEMU/Ovmf.fd*
>> -drive file=/opt/gns3/projects/1a83274a-c57f-4337-8a0d-1e68a9312e9a/project-files/qemu/d8f37f0b-2b63-455f-b536-b309b9020e36/hda_disk.qcow2,if=ide,index=0,media=disk
>> -uuid d8f37f0b-2b63-455f-b536-b309b9020e36
>> -vnc 0.0.0.0:3
>> -monitor tcp:127.0.0.1:33919,server,nowait
>> -net none -device virtio-net-pci,mac=0c:2e:9a:0e:36:00,netdev=gns3-0 -netdev socket,id=gns3-0,udp=127.0.0.1:10017,localaddr=127.0.0.1:10016
>> -nographic
>>
>> The VM starts and boots, the HTTP boot is kicking in and the download is starting. However, after few seconds the download is stopped and the boot terminates with 'Error: Server response timeout'.
>> The process is repeatable and always terminates at the same point (42% upload completed in my case).
>>
>> I've drilled down the code and located the 'offending' code here:
>> file: NetworkPkg/HttpBootDxe/HttpBootSupport.c
>> line#: 1012
>> Offending Code: while (!HttpIo->IsRxDone && ((HttpIo->TimeoutEvent == NULL) || EFI_ERROR (gBS->CheckEvent (HttpIo->TimeoutEvent))))
>> Error Handler at line #1022: if (!HttpIo-> IsRxDone )
>>
>> What I did i order to investigate:
>> I've also tried to add HTTP header: "Connection": "Keep-Alive" hoping it will result the HTTP server keeping sessions open.
>> Used Wireshark to capture and analyze the packets during download - I could not spot anything unusual. Eventually the packed transfer just stops
>> Raised HTTP server debug level - could not find anything valuable in the logs. As far as I can tell from the logs, the HTTP server works correctly and it seems the problem is in the client side (UEFI)
>> I'm downloading large file, so I've checked I have enough RAM to hold the image - seems OK. No complains regarding buffer size
>>
>> Few inputs:
>> My HTTP server version: Apache httpd/2.4.6 (CentOS)
>> File size is I'm trying to download and boot: ~420MB
>>
>> Error observed in the UEFI log when download terminates:
>> Error: Server response timeout.
>> BdsDxe: failed to load Boot0005 "UEFI HTTPv4 (MAC:0C2E9A0E3600)" from PciRoot(0x0)/Pci(0x3,0x0)/MAC(0C2E9A0E3600,0x1)/IPv4(0.0.0.0,0x0,DHCP,0.0.0.0,0.0.0.0,0.0.0.0)/Uri(): Not Found
>>
>> For me it seems like a potential problem on the UEFI side, maybe related to Poll or other networking stuff.
>>
>> I'll appreciate any help.
> 
> As I mentioned on edk2-discuss, I'd suggest two tweaks (independently of
> each other):
> 
> * try the builting virtio-net driver rather than the iPXE one. For that,
> use:
> 
>   -device virtio-net-pci,[other properties],romfile=""
> 
> that is, add a 'romfile=""' property
> 
> * Try using a different -netdev backend than a UDP socket.

Also QEMU v2.5.0 was tagged on 2015-12-16; please try a more recent QEMU
release (I'd suggest 4.2.0).

Thanks
Laszlo

Re: [edk2-devel] HTTP boot failed on timeout

[Maciej Rabeda]

Hi Doron,

As for -netdev backend, you can try using tap.
See: 
https://gist.github.com/extremecoders-re/e8fd8a67a515fee0c873dcafc81d811c
Looking forward to test results.

Thanks,
Maciej

On 17-Feb-20 11:32, Laszlo Ersek wrote:
> On 02/17/20 11:29, Laszlo Ersek wrote:
>> On 02/17/20 09:17, doron.bleiberg@ecitele.com wrote:
>>> Hi Community,
>>>
>>> I've also opened same topic at general discussion group, but after digging more into this issue I think the relevant group is here where all the technical stuff happens.
>>> A short introduction to my problem:
>>> I'm trying to boot QEMU VM using HTTP boot.
>>> I've modified Conf/target.txt as follows:
>>> ACTIVE_PLATFORM       = OvmfPkg/OvmfPkgX64.dsc
>>> TARGET_ARCH           = X64
>>> TOOL_CHAIN_TAG        = GCC48
>>> TARGET                = RELEASE
>>>
>>> and OvmfPkg/OvmfPkgX64.dsc as follows:
>>> DEFINE NETWORK_HTTP_BOOT_ENABLE     = TRUE
>>> DEFINE SECURE_BOOT_ENABLE      = TRUE
>>> BUILD_TARGETS                  = RELEASE
>>>
>>> and compiled compiled OvmfPkg/OvmfPkgX64.dsc
>>>
>>> I'm loading the OVMF.fd to QEMU VM in this way:
>>> /usr/bin/qemu-system-x86_64 -name UEFI-HTTP-VM
>>> -m 8192M -smp cpus=1
>>> -enable-kvm -machine smm=off
>>> -boot order=c *-bios /opt/gns3/images/QEMU/Ovmf.fd*
>>> -drive file=/opt/gns3/projects/1a83274a-c57f-4337-8a0d-1e68a9312e9a/project-files/qemu/d8f37f0b-2b63-455f-b536-b309b9020e36/hda_disk.qcow2,if=ide,index=0,media=disk
>>> -uuid d8f37f0b-2b63-455f-b536-b309b9020e36
>>> -vnc 0.0.0.0:3
>>> -monitor tcp:127.0.0.1:33919,server,nowait
>>> -net none -device virtio-net-pci,mac=0c:2e:9a:0e:36:00,netdev=gns3-0 -netdev socket,id=gns3-0,udp=127.0.0.1:10017,localaddr=127.0.0.1:10016
>>> -nographic
>>>
>>> The VM starts and boots, the HTTP boot is kicking in and the download is starting. However, after few seconds the download is stopped and the boot terminates with 'Error: Server response timeout'.
>>> The process is repeatable and always terminates at the same point (42% upload completed in my case).
>>>
>>> I've drilled down the code and located the 'offending' code here:
>>> file: NetworkPkg/HttpBootDxe/HttpBootSupport.c
>>> line#: 1012
>>> Offending Code: while (!HttpIo->IsRxDone && ((HttpIo->TimeoutEvent == NULL) || EFI_ERROR (gBS->CheckEvent (HttpIo->TimeoutEvent))))
>>> Error Handler at line #1022: if (!HttpIo-> IsRxDone )
>>>
>>> What I did i order to investigate:
>>> I've also tried to add HTTP header: "Connection": "Keep-Alive" hoping it will result the HTTP server keeping sessions open.
>>> Used Wireshark to capture and analyze the packets during download - I could not spot anything unusual. Eventually the packed transfer just stops
>>> Raised HTTP server debug level - could not find anything valuable in the logs. As far as I can tell from the logs, the HTTP server works correctly and it seems the problem is in the client side (UEFI)
>>> I'm downloading large file, so I've checked I have enough RAM to hold the image - seems OK. No complains regarding buffer size
>>>
>>> Few inputs:
>>> My HTTP server version: Apache httpd/2.4.6 (CentOS)
>>> File size is I'm trying to download and boot: ~420MB
>>>
>>> Error observed in the UEFI log when download terminates:
>>> Error: Server response timeout.
>>> BdsDxe: failed to load Boot0005 "UEFI HTTPv4 (MAC:0C2E9A0E3600)" from PciRoot(0x0)/Pci(0x3,0x0)/MAC(0C2E9A0E3600,0x1)/IPv4(0.0.0.0,0x0,DHCP,0.0.0.0,0.0.0.0,0.0.0.0)/Uri(): Not Found
>>>
>>> For me it seems like a potential problem on the UEFI side, maybe related to Poll or other networking stuff.
>>>
>>> I'll appreciate any help.
>> As I mentioned on edk2-discuss, I'd suggest two tweaks (independently of
>> each other):
>>
>> * try the builting virtio-net driver rather than the iPXE one. For that,
>> use:
>>
>>    -device virtio-net-pci,[other properties],romfile=""
>>
>> that is, add a 'romfile=""' property
>>
>> * Try using a different -netdev backend than a UDP socket.
> Also QEMU v2.5.0 was tagged on 2015-12-16; please try a more recent QEMU
> release (I'd suggest 4.2.0).
>
> Thanks
> Laszlo
>
>
> 
>