* EDK II Network Stack Issue
@ 2019-01-04 14:02 Karin Willers
2019-01-07 19:27 ` Laszlo Ersek
0 siblings, 1 reply; 7+ messages in thread
From: Karin Willers @ 2019-01-04 14:02 UTC (permalink / raw)
To: edk2-devel
G'Day!
I'm trying to get networking under edk2 up and running. I tried
AppPkg/Applications/Sockets/RawIp4Tx
under OVMF. The raw packet is sent out on the network, but the
application never returns from the
socket close routine.
I'm currently using UDK2017 with the latest security patches (downloaded
December 18 2018). The network
driver under OVMF is the e1000 driver.
The effect that the socket close never returns is also visible when
running RawIp4Tx on real hardware,
so I think the behavior has nothing to do with OVMF or the UEFI itself.
Does anyone see similar effects? Any hints on setting up networking
under edk2 correctly?
Thanks, Karin
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: EDK II Network Stack Issue
2019-01-04 14:02 EDK II Network Stack Issue Karin Willers
@ 2019-01-07 19:27 ` Laszlo Ersek
2019-01-10 12:40 ` Karin Willers
0 siblings, 1 reply; 7+ messages in thread
From: Laszlo Ersek @ 2019-01-07 19:27 UTC (permalink / raw)
To: Karin Willers, edk2-devel
On 01/04/19 15:02, Karin Willers wrote:
> G'Day!
>
> I'm trying to get networking under edk2 up and running. I tried
> AppPkg/Applications/Sockets/RawIp4Tx
> under OVMF. The raw packet is sent out on the network, but the
> application never returns from the
> socket close routine.
>
> I'm currently using UDK2017 with the latest security patches (downloaded
> December 18 2018). The network
> driver under OVMF is the e1000 driver.
>
> The effect that the socket close never returns is also visible when
> running RawIp4Tx on real hardware,
> so I think the behavior has nothing to do with OVMF or the UEFI itself.
>
> Does anyone see similar effects? Any hints on setting up networking
> under edk2 correctly?
The socket (= libc-level networking) APIs are not the most robust ones
in edk2, in my -- admittedly limited -- experience. I'd suggest using
applications and shell commands that use UEFI protocols instead, for
networking. (I understand that could be a challenge if you are porting a
standard C program to UEFI.)
Thanks
Laszlo
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: EDK II Network Stack Issue
2019-01-07 19:27 ` Laszlo Ersek
@ 2019-01-10 12:40 ` Karin Willers
2019-01-16 12:57 ` Karin Willers
0 siblings, 1 reply; 7+ messages in thread
From: Karin Willers @ 2019-01-10 12:40 UTC (permalink / raw)
To: Laszlo Ersek; +Cc: edk2-devel
On 2019-01-07 20:27, Laszlo Ersek wrote:
> On 01/04/19 15:02, Karin Willers wrote:
>> G'Day!
>>
>> I'm trying to get networking under edk2 up and running. I tried
>> AppPkg/Applications/Sockets/RawIp4Tx
>> under OVMF. The raw packet is sent out on the network, but the
>> application never returns from the
>> socket close routine.
>>
>> I'm currently using UDK2017 with the latest security patches
>> (downloaded
>> December 18 2018). The network
>> driver under OVMF is the e1000 driver.
>>
>> The effect that the socket close never returns is also visible when
>> running RawIp4Tx on real hardware,
>> so I think the behavior has nothing to do with OVMF or the UEFI
>> itself.
>>
>> Does anyone see similar effects? Any hints on setting up networking
>> under edk2 correctly?
>
> The socket (= libc-level networking) APIs are not the most robust ones
> in edk2, in my -- admittedly limited -- experience. I'd suggest using
> applications and shell commands that use UEFI protocols instead, for
> networking. (I understand that could be a challenge if you are porting
> a
> standard C program to UEFI.)
>
> Thanks
> Laszlo
The reason for this undertaking initially was to compile a Python.efi
that
supports sockets. We wanted that to be able to run the Chipsec suite
with
external network test partners under UEFI. The Chipsec guys do provide
pre-compiled Pyhon executables, but these do not include the network
stack.
I think, I have to debug the issue myself ...
Greetings, Karin
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: EDK II Network Stack Issue
2019-01-10 12:40 ` Karin Willers
@ 2019-01-16 12:57 ` Karin Willers
2019-01-16 16:51 ` Laszlo Ersek
0 siblings, 1 reply; 7+ messages in thread
From: Karin Willers @ 2019-01-16 12:57 UTC (permalink / raw)
To: Laszlo Ersek; +Cc: edk2-devel
On 2019-01-10 13:40, Karin Willers wrote:
> On 2019-01-07 20:27, Laszlo Ersek wrote:
>> On 01/04/19 15:02, Karin Willers wrote:
>>> G'Day!
>>>
>>> I'm trying to get networking under edk2 up and running. I tried
>>> AppPkg/Applications/Sockets/RawIp4Tx
>>> under OVMF. The raw packet is sent out on the network, but the
>>> application never returns from the
>>> socket close routine.
>>>
>>> I'm currently using UDK2017 with the latest security patches
>>> (downloaded
>>> December 18 2018). The network
>>> driver under OVMF is the e1000 driver.
>>>
>>> The effect that the socket close never returns is also visible when
>>> running RawIp4Tx on real hardware,
>>> so I think the behavior has nothing to do with OVMF or the UEFI
>>> itself.
>>>
>>> Does anyone see similar effects? Any hints on setting up networking
>>> under edk2 correctly?
>>
>> The socket (= libc-level networking) APIs are not the most robust ones
>> in edk2, in my -- admittedly limited -- experience. I'd suggest using
>> applications and shell commands that use UEFI protocols instead, for
>> networking. (I understand that could be a challenge if you are porting
>> a
>> standard C program to UEFI.)
>>
>> Thanks
>> Laszlo
>
> The reason for this undertaking initially was to compile a Python.efi
> that
> supports sockets. We wanted that to be able to run the Chipsec suite
> with
> external network test partners under UEFI. The Chipsec guys do provide
> pre-compiled Pyhon executables, but these do not include the network
> stack.
>
> I think, I have to debug the issue myself ...
>
> Greetings, Karin
Could please someone shed some light on how to enable debug prints using
UefiDebugLibConOut
in EfiSocketLib/Socket.c. Enabling debug prints is, IMHO, not very
intuitive in edk2 ...
Thanks, Karin
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: EDK II Network Stack Issue
2019-01-16 12:57 ` Karin Willers
@ 2019-01-16 16:51 ` Laszlo Ersek
[not found] ` <b747eeda2cb9af7318986091b1645fb0@webmail-alfa3035.alfahosting-server.de>
0 siblings, 1 reply; 7+ messages in thread
From: Laszlo Ersek @ 2019-01-16 16:51 UTC (permalink / raw)
To: Karin Willers; +Cc: edk2-devel
On 01/16/19 13:57, Karin Willers wrote:
> On 2019-01-10 13:40, Karin Willers wrote:
>> On 2019-01-07 20:27, Laszlo Ersek wrote:
>>> On 01/04/19 15:02, Karin Willers wrote:
>>>> G'Day!
>>>>
>>>> I'm trying to get networking under edk2 up and running. I tried
>>>> AppPkg/Applications/Sockets/RawIp4Tx
>>>> under OVMF. The raw packet is sent out on the network, but the
>>>> application never returns from the
>>>> socket close routine.
>>>>
>>>> I'm currently using UDK2017 with the latest security patches
>>>> (downloaded
>>>> December 18 2018). The network
>>>> driver under OVMF is the e1000 driver.
>>>>
>>>> The effect that the socket close never returns is also visible when
>>>> running RawIp4Tx on real hardware,
>>>> so I think the behavior has nothing to do with OVMF or the UEFI itself.
>>>>
>>>> Does anyone see similar effects? Any hints on setting up networking
>>>> under edk2 correctly?
>>>
>>> The socket (= libc-level networking) APIs are not the most robust ones
>>> in edk2, in my -- admittedly limited -- experience. I'd suggest using
>>> applications and shell commands that use UEFI protocols instead, for
>>> networking. (I understand that could be a challenge if you are porting a
>>> standard C program to UEFI.)
>>>
>>> Thanks
>>> Laszlo
>>
>> The reason for this undertaking initially was to compile a Python.efi
>> that
>> supports sockets. We wanted that to be able to run the Chipsec suite with
>> external network test partners under UEFI. The Chipsec guys do provide
>> pre-compiled Pyhon executables, but these do not include the network
>> stack.
>>
>> I think, I have to debug the issue myself ...
>>
>> Greetings, Karin
>
> Could please someone shed some light on how to enable debug prints using
> UefiDebugLibConOut
> in EfiSocketLib/Socket.c. Enabling debug prints is, IMHO, not very
> intuitive in edk2 ...
Can you build your platform and/or application with the --log and
--report-file options, and share the outputs? The following could all
matter:
- what you pass to "build" with "-b" (NOOPT/DEBUG/RELEASE),
- gEfiMdePkgTokenSpaceGuid.PcdDebugPropertyMask,
- gEfiMdePkgTokenSpaceGuid.PcdDebugPrintErrorLevel,
- gEfiMdePkgTokenSpaceGuid.PcdFixedDebugPrintErrorLevel,
- what lib instance you resolve the DebugLib class to,
- whether you apply the above just to your application, or all other DXE
components in your platform.
Thanks
Laszlo
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: EDK II Network Stack Issue
[not found] ` <b747eeda2cb9af7318986091b1645fb0@webmail-alfa3035.alfahosting-server.de>
@ 2019-01-22 17:34 ` Laszlo Ersek
2019-01-23 22:14 ` Karin Willers
0 siblings, 1 reply; 7+ messages in thread
From: Laszlo Ersek @ 2019-01-22 17:34 UTC (permalink / raw)
To: Karin Willers; +Cc: edk2-devel-ml01
Adding the list back in -- I can't really justify spending an hour or
more on emails like this if I don't send them to the list. More
importantly, I could be wrong about some details, and keeping the
discussion on-list gives others a chance to correct me.
Comments below:
On 01/22/19 09:56, Karin Willers wrote:
> On 2019-01-16 17:51, Laszlo Ersek wrote:
>
>>>
>>> Could please someone shed some light on how to enable debug prints
>>> using UefiDebugLibConOut in EfiSocketLib/Socket.c. Enabling debug
>>> prints is, IMHO, not very intuitive in edk2 ...
>>
>> Can you build your platform and/or application with the --log and
>> --report-file options, and share the outputs? The following could all
>> matter:
>
> See attached ZIP-File.
>
>>
>> - what you pass to "build" with "-b" (NOOPT/DEBUG/RELEASE),
>
> DEBUG is selected in target.txt
>
>> - gEfiMdePkgTokenSpaceGuid.PcdDebugPropertyMask,
>> - gEfiMdePkgTokenSpaceGuid.PcdDebugPrintErrorLevel,
>> - gEfiMdePkgTokenSpaceGuid.PcdFixedDebugPrintErrorLevel,
>
> In StdLib.dsc I have:
>
> #
> # Debug output control
> #
> DEFINE DEBUG_ENABLE_OUTPUT = TRUE # Set to TRUE to enable
> debug output
> # DEFINE DEBUG_PRINT_ERROR_LEVEL = 0x80000000 # Flags to control
> amount of debug output
> DEFINE DEBUG_PRINT_ERROR_LEVEL = 0xffffffff # Flags to control
> amount of debug output
> DEFINE DEBUG_PROPERTY_MASK = 0x0f
>
> [PcdsFeatureFlag]
>
> [PcdsFixedAtBuild]
> gEfiMdePkgTokenSpaceGuid.PcdDebugPropertyMask|$(DEBUG_PROPERTY_MASK)
>
> gEfiMdePkgTokenSpaceGuid.PcdDebugPrintErrorLevel|$(DEBUG_PRINT_ERROR_LEVEL)
>
> This enables UefiDebugLibConOut.inf (in StdLib.dsc) as follows:
>
> !if $(DEBUG_ENABLE_OUTPUT)
> DebugLib|MdePkg/Library/UefiDebugLibConOut/UefiDebugLibConOut.inf
>
> DebugPrintErrorLevelLib|MdePkg/Library/BaseDebugPrintErrorLevelLib/BaseDebugPrintErrorLevelLib.inf
>
> !else ## DEBUG_ENABLE_OUTPUT
> DebugLib|MdePkg/Library/BaseDebugLibNull/BaseDebugLibNull.inf
> !endif ## DEBUG_ENABLE_OUTPUT
>
> Just to be sure (don't know whether this is really necessary) in
> MdePkg.dsc I have:
> gEfiMdePkgTokenSpaceGuid.PcdDebugPrintErrorLevel|0xffffffff
>
>> - what lib instance you resolve the DebugLib class to,
>
> In 'StdLib.inc' (included from 'StdLib.dsc') I inserted:
> DebugLib|MdePkg/Library/UefiDebugLibConOut/UefiDebugLibConOut.inf #
> Debugging Support
>
>> - whether you apply the above just to your application, or all other
>> DXE
>
> I suppose, just my application?!
>
>> components in your platform.
>
> I compile the parts (which I think I need) individually:
>
> o MdePkg
> o StdLib
> o AppPkg
>
> and then copy everything to a folder for use with OVMF.
>
> The packages were built in this exact order:
>
> build --log MdePkg-log.txt --report-file MdePkg-report-file.txt
> build --log StdLib-log.txt --report-file StdLib-report-file.txt
> build --log AppPkg-log.txt --report-file AppPkg-report-file.txt
>
> When running qemu on the OVMF image (built previously) using the files
> just generated,
> I do not see any debug prints. My qemu sequence is as follows:
>
> #!/bin/sh
>
> set -x
>
> qemu-system-x86_64 -L . --bios ./OVMF.fd \
> -hda fat:rw:hda-contents \
> -global e1000.romfile="" \
> -netdev tap,id=hostnet0,ifname=tap0,script=no,downscript=no \
> -device e1000,netdev=hostnet0,mac=52:54:00:56:34:12
>
>>
>> Thanks
>> Laszlo
>
> The next thing is to try another network driver - the e1000 driver is
> only available as a binary from Intel
> and not in source form.
>
> Thanks in advance for your help!
In "AppPkg-report-file.txt", the "RawIp4Tx" module's section lists the
following library instances (reformatted for readability):
DebugPrintErrorLevelLib --> MdePkg/Library/BaseDebugPrintErrorLevelLib/BaseDebugPrintErrorLevelLib.inf
PcdLib --> MdePkg/Library/BasePcdLibNull/BasePcdLibNull.inf
DebugLib --> MdePkg/Library/UefiDebugLibConOut/UefiDebugLibConOut.inf
EfiSocketLib --> StdLib/EfiSocketLib/EfiSocketLib.inf
The report file also confirms that the build target was DEBUG.
This means that DEBUG macro invocations in StdLib/EfiSocketLib code will
be resolved to MdePkg/Library/UefiDebugLibConOut, in the final
"RawIp4Tx.efi" binary.
The PcdLib class resolution means that dynamic PCDs won't work; all the
relevant PCDs have to be patched-in-module, or fixed-at-build. This is
actually good (see details below).
The chosen DebugLib instance depends on PcdDebugPropertyMask and
PcdFixedDebugPrintErrorLevel. The former PCD is relevant here for
DebugPrintEnabled() and DebugCodeEnabled(), the latter is relevant for
DebugPrintLevelEnabled().
In addition, the chosen DebugPrintErrorLevelLib instance consumes
PcdDebugPrintErrorLevel, via GetDebugPrintErrorLevel().
Now, assume that the DEBUG() function-like macro is invoked within a
DEBUG_CODE() section in StdLib/EfiSocketLib. This is what happens, after
the macros are all expanded:
- DebugCodeEnabled() is called first. This checks, in
UefiDebugLibConOut, whether DEBUG_PROPERTY_DEBUG_CODE_ENABLED (0x04)
is set in PcdDebugPropertyMask.
- DebugPrintEnabled() is called. This checks, in UefiDebugLibConOut,
whether DEBUG_PROPERTY_DEBUG_PRINT_ENABLED (0x02) is set in
PcdDebugPropertyMask.
- DebugPrintLevelEnabled() is called. This checks, in
UefiDebugLibConOut, whether the specified error level (or more
precisely, error mask) intersects with PcdFixedDebugPrintErrorLevel.
- DebugPrint() is called. This function is again implemented in
UefiDebugLibConOut, and it calls GetDebugPrintErrorLevel(). The retval
is checked against the caller-specified error mask.
- GetDebugPrintErrorLevel() is implemented in
BaseDebugPrintErrorLevelLib, and it retrieves PcdDebugPrintErrorLevel.
- If all the checks pass, and you have a non-NULL UEFI console
(gST->ConOut), then DebugPrint() outputs the message.
That is:
- PcdDebugPropertyMask must have bit value 0x04 set.
- PcdDebugPropertyMask must have bit value 0x02 set.
- PcdFixedDebugPrintErrorLevel must intersect with the error mask that
the given DEBUG macro invocation uses.
- PcdDebugPrintErrorLevel must intersect with the error mask that the
given DEBUG macro invocation uses.
In the same section of the AppPkg report file (under RawIp4Tx), I see
the following PCD settings:
> PcdDebugPropertyMask : FIXED (UINT8) = 0
>
> PcdFixedDebugPrintErrorLevel : FIXED (UINT32) = 0xFFFFFFFF
>
> *P PcdDebugPrintErrorLevel : FIXED (UINT32) = 0x80000040
> DEC DEFAULT = 0x80000000
The PcdDebugPropertyMask requirement is not satisfied. You have to set
it to the bitmask 0x06 (or a superset of it) in either AppPkg.dsc, or
else on the "build" command line, with the --pcd switch.
The PcdFixedDebugPrintErrorLevel requirement is satisfied (it has all
imaginable bits set).
The PcdDebugPrintErrorLevel requirement is not satisfied. Although
AppPkg.dsc already overrides the default from MdePkg.dec (hence the "*P"
mark, for "platform override"), value 0x80000040 only stands for
DEBUG_ERROR | DEBUG_INFO (see "MdePkg/Include/Library/DebugLib.h").
However, "StdLib/EfiSocketLib" uses non-standard error masks such as
DEBUG_BIND, DEBUG_ACCEPT, and so on (see
"StdLib/EfiSocketLib/Socket.h"). If you want those to appear as well,
then you'll have to include the corresponding bits in
PcdDebugPrintErrorLevel. Use the --pcd switch again, or modify the
AppPkg.dsc file, or use -D DEBUG_PRINT_ERROR_LEVEL=... on the "build"
command line, when building AppPkg (because AppPkg.dsc sets the PCD to
that macro, as a convenience).
A PCD has several aspects. Any given PCD has:
- A set of permissible PCD storage types. This set comes from the
package DEC (declaration) file, where the PCD is declared. It
determines the storage types (plural) that any given platform, or
module, may choose for the PCD. Storage types include: fixed-at-build
(basically: a build constant), patchable-in-module (a build constant
capable of being statically updated in firmware images by an external,
binary patching tool), dynamic (read/write, stored in memory,
accessible by almost everything in the firmware), HII (stored in NVRAM
/ UEFI variables).
- A data type, such as UINT32. This comes from the DEC file.
- A default value, such as 0 or 1. Comes from the DEC file.
- In the platform DSC file, the platform chooses the actual storage type
for the PCD (selecting one of the DEC-permitted types), and may
provide a platform-specific default value (overriding the default from
the DEC file).
- In the platform DSC file, individual modules too may choose different
storage types (overriding the DSC default) and different default
values (ditto).
In your case:
- Compiling MdePkg and StdLib modules in separation, that is, through
the "MdePkg.dsc" and "StdLib.dsc" platform description files, is
entirely superfluous. The build output fom those builds will not be
used for anything. When you compile AppPkg, the build utility will
collect dependencies from across the tree, and build everything
necessary under Build/, such that the build artifacts are going to be
specific to AppPkg, even if they come from under MdePkg or StdLib.
And, in this build, the AppPkg.dsc settings will apply. The only PCD
settings that will take effect from MdePkg and StdLib are those where
AppPkg doesn't override the MdePkg / StdLib DEC defaults. This is why
I only looked at the AppPkg build report file, from your attachment.
- If you build EFI binaries from AppPkg, and then run them on an OVMF
platform firmware, then fixed-at-build PCDs may have entirely
different values in the AppPkg binaries, from the values that the same
PCDs have in OVMF modules. The same applies to the resolutions of
library classes to library instances. For example, modules built into
OVMF will print their debug messages to the QEMU debug port, not to
the UEFI console.
- Worse, if both sets of modules use dynamic storage for a given PCD,
then they might disagree on either the precise storage location itself
(with catastrophic results [I'm ignoring some details here]), or on
the meaning of the individual integer values. Dynamic PCDs exist for
cross-driver communication, and they establish a tight coupling
between modules. Therefore UEFI applications (which are generally
considered "portable") should generally not use dynamic PCDs at all.
Fixed PCDs are fine, but then see the previous bullet -- the same PCDs
may have different assignments in the platform firmware.
This is why I asked earlier, "whether you apply the above just to your
application, or all other DXE" -- the question referred to whether you
applied the fixed PCD settings to all of OVMF (via the OVMF DSC files),
or just your standalone EFI application (built from AppPkg).
Okay, summary:
- forget the MdePkg and StdLib DSC files, build only AppPkg (or whatever
platform DSC it is that produces your UEFI application)
- make sure PcdDebugPropertyMask has bitmask 0x06 set when you build
AppPkg (or the appropriate platform DSC in question); either modify
the same DSC for this, or use the --pcd command line switch
- make sure PcdDebugPrintErrorLevel has all the DEBUG_* bits set from
"StdLib/EfiSocketLib/Socket.h" that you care about; the methods for
setting this PCD are identical to those seen in the previous point.
Hope this helps,
Laszlo
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: EDK II Network Stack Issue
2019-01-22 17:34 ` Laszlo Ersek
@ 2019-01-23 22:14 ` Karin Willers
0 siblings, 0 replies; 7+ messages in thread
From: Karin Willers @ 2019-01-23 22:14 UTC (permalink / raw)
To: Laszlo Ersek; +Cc: edk2-devel-ml01
On 1/22/19 6:34 PM, Laszlo Ersek wrote:
> Okay, summary:
>
> - forget the MdePkg and StdLib DSC files, build only AppPkg (or whatever
> platform DSC it is that produces your UEFI application)
>
> - make sure PcdDebugPropertyMask has bitmask 0x06 set when you build
> AppPkg (or the appropriate platform DSC in question); either modify
> the same DSC for this, or use the --pcd command line switch
>
> - make sure PcdDebugPrintErrorLevel has all the DEBUG_* bits set from
> "StdLib/EfiSocketLib/Socket.h" that you care about; the methods for
> setting this PCD are identical to those seen in the previous point.
>
> Hope this helps,
> Laszlo
>
Hello Laszlo,
debug prints are working now - thanks for your advice!
Greetings, Karin
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2019-01-23 22:14 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2019-01-04 14:02 EDK II Network Stack Issue Karin Willers
2019-01-07 19:27 ` Laszlo Ersek
2019-01-10 12:40 ` Karin Willers
2019-01-16 12:57 ` Karin Willers
2019-01-16 16:51 ` Laszlo Ersek
[not found] ` <b747eeda2cb9af7318986091b1645fb0@webmail-alfa3035.alfahosting-server.de>
2019-01-22 17:34 ` Laszlo Ersek
2019-01-23 22:14 ` Karin Willers
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox