Re: OVMF.fd and placement of EfiBootServicesData

[spam collector <spamcollector@cableone.net>]

----- Original Message -----
> From: "Laszlo Ersek" <lersek@redhat.com>
> To: "spam collector" <spamcollector@cableone.net>
> Cc: edk2-devel@ml01.01.org
> Sent: Tuesday, October 4, 2016 12:52:00 AM
> Subject: Re: [edk2] OVMF.fd and placement of EfiBootServicesData
> 
> On 10/04/16 04:45, spam collector wrote:
> > 
> > 
> > ----- Original Message -----
> >> From: "Laszlo Ersek" <lersek@redhat.com>
> >> To: "spam collector" <spamcollector@cableone.net>
> >> Cc: edk2-devel@ml01.01.org
> >> Sent: Monday, October 3, 2016 2:42:08 AM
> >> Subject: Re: [edk2] OVMF.fd and placement of EfiBootServicesData
> >>
> >> On 10/03/16 05:16, spam collector wrote:
> >>> Hello,
> >>> Forgive me for not searching for this first, but gmane.org has
> >>> been down for a little while. Therefore, forgive me if this
> >>> question has been asked and answered.
> >>> I am using OVMF.fd and QEMU for Windows and tried to load
> >>> a large file to physical address 0x00800000. When an error
> >>> was returned, I found that OVMF has reserved an amount of
> >>> EfiBootServicesData within/around that location.
> >>> Here is a memory dump using the BootServices MemoryMap Service:
> >>> Start: 0x00000000->0x0009FFFF, Pgs: 160, EfiConventionalMem
> >>> Start: 0x00100000->0x003FFFFF, Pgs: 768, EfiConventionalMem
> >>> Start: 0x00400000->0x0040BFFF, Pgs: 12, EfiBootServicesCode
> >>> Start: 0x0040C000->0x0081FFFF, Pgs: 1044, EfiConventionalMem
> >>> Start: 0x00820000->0x00FFFFFF, Pgs: 2016, EfiBootServicesData
> >>> Start: 0x01000000->0x0BFFFFFF, Pgs: 1044, EfiConventionalMem
> >>> ... and so on
> >>> Without the idea of "you should make your code relocatable,
> >>> i.e.: not care where it is in memory", or if I say that I *must* have
> >>> the memory from 0x00800000 to 0x00FFFFFFavailable,
> >>> without re-building OVMF.fd, is there a way to tell the system
> >>> to use a different address for that portion of EfiBootServiceCode?
> >>> I tried manipulating NvVars with a few entries to no avail.
> >>> I added the "LoadFixedAddressConfigurationTable" entry hoping
> >>> that I could set a minimum location, too without success.
> >>> So, without rebuilding OVMF.fd, which would require everyone
> >>> whom wished to use my code to do so, or at least download it
> >>> from me, and I don't like modified packages of other's work
> >>> floating around, is there a why to tell OVMF.fd/the EFI system
> >>> to not use memory below a certain address?
> >>> Thank you in advance for your suggestions,
> >>
> >> Sorry, I don't think I can help.
> >>
> >> You can read about the OVMF memory map in the (now somewhat outdated)
> >> OVMF whitepaper at
> >>
> >>   http://www.linux-kvm.org/downloads/lersek/ovmf-whitepaper-c770f8c.txt
> >>
> >> section
> >>
> >>   A comprehensive memory map of OVMF
> >>
> >> You'd like to place your stuff at 8192 KB .. 16384 KB, but as you can
> >> see from the map, that exact range is heavily used by OVMF itself.
> >>
> >> (As I said, some of those exact values are no longer current, due to the
> >> following commits since:
> >>
> >>   08df58ec3043 OvmfPkg: raise DXEFV size to 9 MB
> >>   2f7b34b20842 OvmfPkg: raise DXEFV size to 10 MB
> >>   45d870815156 OvmfPkg/PlatformPei: rebase and resize the permanent PEI
> >>                memory for S3
> >>   6b04cca4d697 OvmfPkg: remove PcdS3AcpiReservedMemoryBase,
> >>                PcdS3AcpiReservedMemorySize
> >>
> >> but the main layout remains the same.)
> >>
> >> Please consider making your code relocatable.
> > 
> > Without sounding rude, and please do not take it as so, making my
> > code relocatable due to the fact that the other code is not relocatable,
> > sounds a little hypocritical. :-)
> 
> Not taking it as rude, but you are wrong about this. OVMF is a platform
> firmware, including SEC and PEI phase modules. Such modules are fully
> expected to operate with a fixed address map.
> 
> In comparison your module is (I assume) a UEFI_DRIVER /
> UEFI_APPLICATION, or maybe a DXE_DRIVER.
> - If it's a UEFI_DRIVER or UEFI_APPLICATION, then it is expected to work
> without hardcoded addresses.
> - If it is a DXE_DRIVER, then it might be okay for it to operate with
> fixed addresses. However, a DXE_DRIVER module is meant to be shipped by
> the platform vendor. For that, your module would have to be integrated
> with OVMF -- it should either be part of the public edk2 tree (or your
> edk2 tree), or you should be providing a prebuilt OVMF binary that
> contained it.
> 
> > Anyway, please don't take offense to that, it is just what first
> > came to mind when I read it.  Eventually I will make my code relocatable
> > but at the moment that would be quite a re-write of many other parts
> > of this project, and I am not ready or willing to do that yet. Yet...
> > 
> > May I please suggest to the maintainers of this project to add a function
> > that will allow a NvVars entry to indicate where the base of this
> > "comprehensive memory map of OVMF" starts.  Allow an entry within the
> > NvVars file to "suggest" a base address and let the code accommodate
> > as close as it can.
> 
> (1) You mention the NvVars file as storage for non-volatile variables.
> If you are using OVMF with QEMU, I strongly suggest to abandon this
> setup, and adopt the pflash based variable store instead. The behavior
> of the NvVars file is not fully compatible with the UEFI spec, and it
> had been invented as a stop-gap solution until real variable storage
> would be implemented.
> 
> (2) The addresses that you seem interested in are build-time constants,
> and even reset vector and SEC phase code depend on them. In those
> phases, variable services don't exist.
> 
> If you want to move OVMF's fixed memory map to a different spot, for
> your own builds, please grep OvmfPkg for the PCDs that are listed to the
> right of the table that is in section "A comprehensive memory map of
> OVMF" of the whitepaper, and customize them as you see fit. (And then
> rebuild OVMF, of course.)
> 
> We have sought to make this as painless as possible -- for our own
> sanity's sake too --; that is, most of those PCDs are already being
> computed from other (more basic) PCDs in the same set, either at build
> time or at runtime. It's just that the "root" PCDs have to be build-time
> constants. For example, please refer to the MEMFD_BASE_ADDRESS macro in
> "OvmfPkg.fdf.inc" and elsewhere.
> 
> > Or even just a flag that states to use higher
> > area memory when set, or lower area memory when clear.  Some OS writers
> > like to use high memory and leave low memory available while others
> > like to use low memory and leave the remaining higher memory available.
> 
> If you want to move OVMF's fixed areas to different places, specific
> addresses are necessary. Please refer to the PCDs mentioned above.
> 
> The current map works with at least Linux guests, Windows guests, OSX
> guests (I've heard from users), and FreeBSD guests (based on their
> Wiki). I'm quite sure the address constraints currently encoded in your
> module (a boot loader perhaps?) could break on suitable physical
> firmware as well; OVMF does nothing invalid in this regard.
> 
> Laszlo

Thank you for your comments and suggestions.  I have glanced over
the pflash storage a little, but not fully investigated it at all.

I appreciate the work and effort of those whom have created OVMF and
admire their commitment.

Again, thank you for your comments.  Maybe it is time to spend the
effort and time to make my code relocatable.

On a slightly different note, I was surprised to see that QEMU and/or
OVMF requires the NvVars file to exist on the partition, else OVMF
does not even show the shell, even on a perfectly valid partitioned
image.  I don't know if this is QEMU's fault or OVMF's fault.  If 
the NvVars file does not exist, is it the desired option to fail 
the loading of the shell? If the file does not exist, can OVMF use 
default values and still load the shell?

I ask because I spent a little while thinking that my GPT partitioned
image was in error since I could not get a shell to load.  Once I
added the NvVars file to the root directory, the shell loaded as
expected.  If NvVars is not compatible with UEFI, which I don't
believe it is, should OVMF still require it to boot to the shell?
Maybe if I study more on the pflash option and use it, OVMF will
not require the NvVars file.

I ask with the intent of better understanding the reason it
requires the NvVars file, as opposed to being sarcastic as it may
sound. :-)

Thank you once again,
Ben