From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-ed1-f43.google.com (mail-ed1-f43.google.com [209.85.208.43]) by mx.groups.io with SMTP id smtpd.web12.3871.1615456178380719107 for ; Thu, 11 Mar 2021 01:49:38 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@solid-run-com.20150623.gappssmtp.com header.s=20150623 header.b=zl+AJgOv; spf=pass (domain: solid-run.com, ip: 209.85.208.43, mailfrom: jon@solid-run.com) Received: by mail-ed1-f43.google.com with SMTP id bf3so1798766edb.6 for ; Thu, 11 Mar 2021 01:49:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=solid-run-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=3t5ZTjxiuA0QRiWhAD/CV9/q+P/sCKt99dE6kFZW71c=; b=zl+AJgOv/3i8bv2JC4SyfKwI3VinFPBXcyL/89ivN7YJCib9/qO7Qb6XH4LgoCRte3 0p/tZaAK2feRbaXyyMaQ2g8ujzd33IqdkYlXJlKhW3+OozwDItiOKs2VqwoYVXsrmVY8 5+2NiOlKksUribYq6YdaXJgtnI6YwcdcL6JEAvbxuXcZB3l4gmq6T2IXYk3/h6A9GVAD P15qFhFGC0JnOcEj+1e/yY/rLsu26/ODw853bj5LyzF06Sn1dD1UBJvG61M5pMgbFMZM UzaRApxc57F2qIsoWmdnvM6+Mc/yAA80LbjyDcHac2HnFwS+X3RNU3j9UwUgeejS8n+K p/Ig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=3t5ZTjxiuA0QRiWhAD/CV9/q+P/sCKt99dE6kFZW71c=; b=Q08QbhTO1D6Xtw/eeWq/9ZYjr4QaRNiRRm6xgdGZ5+ZHh09yMcwHEGJrO2tn0GfFXA YAO7Gp7nVS2wBQyOIxwLATHv404SPIr2uAmgIQJ+mmAzbgKDmGXYyHmsD7gKsWPnkEY8 Y9dGip0NpuZ/JGB1/oNUrWqFQRLT1+laQqt0IvBpBfUZBpSOM2yaqMXRMFyiAfRkpvwr MrhaGPR7F6Sc/TZDOUoErqOtdclDNaPKHaNAHb47Xt/Q9gBsAgnHFmxCgNLipFMIXX10 LXNpiHjl/rAzMVckETr5zNIxU4QIjr0Dsl6oW9lv1nfNoJn7ROXcpbAqnJlJK5+YtMtk JWbg== X-Gm-Message-State: AOAM530ArJUv1l99gkVKKE6rTAqe0OE3oZ5C0d3ssenx3XsnJjUUnQhS zjueqdAE0d9uC48GX2bWOIWtMuf3vKmevc5PrRRJPqzNIIcYmbMV X-Google-Smtp-Source: ABdhPJwwq+bIY03XkO8XxclNYp52kxxpuadUV1/97A42sv8/3aLP/3iSpr5XPgJNnT+T1IHgl9DrKsf+UpEKJlBa8nQ= X-Received: by 2002:a05:6402:3089:: with SMTP id de9mr7789573edb.10.1615456176248; Thu, 11 Mar 2021 01:49:36 -0800 (PST) MIME-Version: 1.0 References: <5363bdf0-afac-73bf-d001-77949916f511@redhat.com> <166B374585A9D8FC.18699@groups.io> In-Reply-To: <166B374585A9D8FC.18699@groups.io> From: "Jon Nettleton" Date: Thu, 11 Mar 2021 10:48:57 +0100 Message-ID: Subject: Re: [edk2-devel] Conflicting virtual addresses causing Runtime Services issues To: devel@edk2.groups.io, Jon Nettleton Cc: Laszlo Ersek Content-Type: text/plain; charset="UTF-8" On Thu, Mar 11, 2021 at 7:54 AM Jon Nettleton via groups.io wrote: > > On Wed, Mar 10, 2021 at 3:52 PM Laszlo Ersek wrote: > > > > On 03/10/21 09:04, Jon Nettleton wrote: > > > I am debugging a failure that I am seeing while using the HoneyComb's > > > spi-nor flash for runtime variable storage. I am hoping someone on > > > the list can give me some insight as what may be the problem. > > > > > > The problem showed up when we switched the MMIO region for the fspi > > > flash device to be marked as non executable. reading variables is > > > fine, however writes began throwing an error. > > > > > > [ 556.709828] Unable to handle kernel execute from non-executable > > > memory at virtual address 00000000206a3968 > > > > > > I have patched the kernel and removed the X86 requirement and enabled > > > the sysfs runtime mappings kernel config so I can get an easy view of > > > the mappings the kernel carries for runtime services. I then track > > > that virtual address to the MMIO region of nor flash, which makes > > > sense that region is marked as non executable. The question is why is > > > code being executed from this address range > > > > > > attribute > > > :::::::::::::: > > > 0x8000000000000001 > > > :::::::::::::: > > > num_pages > > > :::::::::::::: > > > 0x40 > > > :::::::::::::: > > > phys_addr > > > :::::::::::::: > > > 0x20500000 > > > :::::::::::::: > > > type > > > :::::::::::::: > > > 0xb > > > :::::::::::::: > > > virt_addr > > > :::::::::::::: > > > 0x20680000 > > > > > > So then I patched the PL011 serial driver to be able to log to the > > > console in runtime and I track down the access to Status = > > > Fvb->GetPhysicalAddress(Fvb, &FvVolHdr); in UpdateVariableStore(). > > > What I don't understand is why EfiConvertPointer is mapping that > > > pointer into the Virtual address space occupied by the runtime mmio of > > > the flash. The pointer is being properly remapped. Here are the > > > pointer addresses in EFI and Kernel Runtime > > > > > > EFI: > > > UpdateVariableStore:156 ECE33968 > > > FvbGetPhysicalAddress(BaseAddress=0x20000000) > > > > > > KERNEL: > > > UpdateVariableStore:156 206A3968 > > > [ 556.709828] Unable to handle kernel execute from non-executable > > > memory at virtual address 00000000206a3968 > > > > > > Any insight that anyone could provide would be much appreciated. > > > > Your platform appears misconfigured -- the flash MMIO range appears to > > overlap runtime services code even before SetVirtualAddressMap. The > > virtual address conflict is likely the result of the original physical > > address conflict. > > There is no physical address conflict. Running in physical mode the pointer > for GetPhysicalAddress is at 0xECE33968 and the MMIO physical > region is 0x20000000 - 0x2FFFFFFF. Shown as the BaseAddress above. > Obviously I don't use that full MMIO region because our flash is not > 256MB. > > > > > After the virtual address updates, the EfiMemoryMappedIO (0xb) type > > range is mapped at virtual address range [0x20680000, 0x206C0000). The > > GetPhysicalAddress function seems to be located at offset 0x23968 in > > that range (with 0x1C698 bytes to go in the range). That's inexplicable. > > Yes, exactly why I am reaching out. > > > > > What is the physical base address of the flash? What are the PCDs used > > in "ArmPlatformPkg/Drivers/NorFlashDxe/NorFlashFvb.c"? > > We don't use the ArmPlatformPkg > > The code is available here. > https://github.com/SolidRun/edk2-platforms/tree/LX2160_UEFI_ACPI_EAR3-lx2160cex7/Silicon/NXP/Drivers/SpiNorFlashDxe > > Thanks > > Found the root of my problem and fixed it. As used from other devices they are relocating the individual pointer for each function in FvbProtocol of the DXE. This is done in ArmPlatformPkg/Drivers/NorFlashDxe/NorFlashFvb.c, as well as lots of vendor drivers. I was able to dump the addresses being used in RuntimeDriverConvertPointer and could locate the pointer to GetPhysicalAddress. Convert 0xEC8C1648:0xEC8B0000:0x203D0000 New virtual address is then 203E1648 which is fine. However then later down in the remapping I found Convert 0x203E1648:0x20000000:0x20800000 And this is where the pointer gets remapped again and into the MMIO space of the nor flash. If I remove the calls to ConvertPointer for the FvbProtocol I am still seeing those addresses getting remapped but only once and runtime works as expected. I am seeing that in MdeModulePkg/Universal/Variable/RuntimeDxe/VariableDxe.c &mVariableModuleGlobal->FvbInstance->* are all being converted. It is possible this is a long standing bug and it just so happens that our configuration has caused a conflict and exposed it. -Jon