Hi Lazslo,
OK, I will regenerate the GUID in next commit.On 1/26/24 07:29, Chao Li wrote:Add a new library named CpuMmuLib and add a LoongArch64 instance with in the library. It provides two-stage MMU libraryinstances, PEI and DXE. BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=4584 Cc: Eric Dong <eric.dong@intel.com> Cc: Ray Ni <ray.ni@intel.com> Cc: Laszlo Ersek <lersek@redhat.com> Cc: Rahul Kumar <rahul1.kumar@intel.com> Cc: Gerd Hoffmann <kraxel@redhat.com> Signed-off-by: Chao Li <lichao@loongson.cn> Co-authored-by: Baoqi Zhang <zhangbaoqi@loongson.cn> Co-authored-by: Dongyan Qian <qiandongyan@loongson.cn> Co-authored-by: Xianglai Li <lixianglai@loongson.cn> Co-authored-by: Bibo Mao <maobibo@loongson.cn> --- UefiCpuPkg/Library/CpuMmuLib/DxeCpuMmuLib.inf | 36 + UefiCpuPkg/Library/CpuMmuLib/DxeCpuMmuLib.uni | 14 + .../CpuMmuLib/LoongArch64/CommonMmuLib.c | 988 ++++++++++++++++++ .../CpuMmuLib/LoongArch64/CommonMmuLib.h | 43 + .../Library/CpuMmuLib/LoongArch64/Page.h | 279 +++++ .../CpuMmuLib/LoongArch64/PeiCpuMmuLib.c | 178 ++++ .../Library/CpuMmuLib/LoongArch64/Tlb.h | 48 + .../CpuMmuLib/LoongArch64/TlbOperation.S | 44 + UefiCpuPkg/Library/CpuMmuLib/PeiCpuMmuLib.inf | 44 + UefiCpuPkg/Library/CpuMmuLib/PeiCpuMmuLib.uni | 14 + UefiCpuPkg/UefiCpuPkg.dsc | 4 + 11 files changed, 1692 insertions(+) create mode 100644 UefiCpuPkg/Library/CpuMmuLib/DxeCpuMmuLib.inf create mode 100644 UefiCpuPkg/Library/CpuMmuLib/DxeCpuMmuLib.uni create mode 100644 UefiCpuPkg/Library/CpuMmuLib/LoongArch64/CommonMmuLib.c create mode 100644 UefiCpuPkg/Library/CpuMmuLib/LoongArch64/CommonMmuLib.h create mode 100644 UefiCpuPkg/Library/CpuMmuLib/LoongArch64/Page.h create mode 100644 UefiCpuPkg/Library/CpuMmuLib/LoongArch64/PeiCpuMmuLib.c create mode 100644 UefiCpuPkg/Library/CpuMmuLib/LoongArch64/Tlb.h create mode 100644 UefiCpuPkg/Library/CpuMmuLib/LoongArch64/TlbOperation.S create mode 100644 UefiCpuPkg/Library/CpuMmuLib/PeiCpuMmuLib.inf create mode 100644 UefiCpuPkg/Library/CpuMmuLib/PeiCpuMmuLib.uni diff --git a/UefiCpuPkg/Library/CpuMmuLib/DxeCpuMmuLib.inf b/UefiCpuPkg/Library/CpuMmuLib/DxeCpuMmuLib.inf new file mode 100644 index 0000000000..bfce3ce96d --- /dev/null +++ b/UefiCpuPkg/Library/CpuMmuLib/DxeCpuMmuLib.inf @@ -0,0 +1,36 @@ +## @file +# CPU Memory Map Unit DXE phase driver. +# +# Copyright (c) 2024 Loongson Technology Corporation Limited. All rights reserved.<BR> +# +# SPDX-License-Identifier: BSD-2-Clause-Patent +# +## + +[Defines] + INF_VERSION = 1.29 + BASE_NAME = DxeCpuMmuLib + MODULE_UNI_FILE = DxeCpuMmuLib.uni + FILE_GUID = DA8F0232-FB14-42F0-922C-63104D2C70BE(1) This FILE_GUID was created from the FILE_GUID of "ArmPkg/Library/ArmMmuLib/ArmMmuBaseLib.inf" by adding 1. That's not an acceptable method for GUID generation. A method is only acceptable for GUID generation if multiple (= an unlimited number of) parties can execute the method at any time, and the output remains conflict-free. Taking an existent (known) FILE_GUID and incrementing it by 1 is not such a method. Please regenerate the GUID with "uuidgen". Please also review the rest of your new GUIDs over this series (not only FILE_GUIDs in INF files, but any other GUIDs, too).
OK.+ MODULE_TYPE = DXE_DRIVER + VERSION_STRING = 1.0 + LIBRARY_CLASS = CpuMmuLib | DXE_DRIVER + CONSTRUCTOR = MmuInitialize + +# +# VALID_ARCHITECTURES = LOONGARCH64 +# + +[Sources.LoongArch64] + LoongArch64/TlbOperation.S | GCC + LoongArch64/CommonMmuLib.c + LoongArch64/Page.h + LoongArch64/Tlb.h + +[Packages] + MdePkg/MdePkg.dec + UefiCpuPkg/UefiCpuPkg.dec + +[LibraryClasses] + DebugLib + MemoryAllocationLib diff --git a/UefiCpuPkg/Library/CpuMmuLib/DxeCpuMmuLib.uni b/UefiCpuPkg/Library/CpuMmuLib/DxeCpuMmuLib.uni new file mode 100644 index 0000000000..7342249516 --- /dev/null +++ b/UefiCpuPkg/Library/CpuMmuLib/DxeCpuMmuLib.uni @@ -0,0 +1,14 @@ +// /** @file +// CPU Memory Manager Unit library instance for DXE modules. +// +// CPU Memory Manager Unit library instance for DXE modules. +// +// Copyright (c) 2024, Loongson Technology Corporation Limited. All rights reserved.<BR> +// +// SPDX-License-Identifier: BSD-2-Clause-Patent +// +// **/ + +#string STR_MODULE_ABSTRACT #language en-US "CPU Memory Manager Unit library instance for DXE modules." + +#string STR_MODULE_DESCRIPTION #language en-US "CPU Memory Manager Unit library instance for DXE modules." diff --git a/UefiCpuPkg/Library/CpuMmuLib/LoongArch64/CommonMmuLib.c b/UefiCpuPkg/Library/CpuMmuLib/LoongArch64/CommonMmuLib.c new file mode 100644 index 0000000000..2e852c3371 --- /dev/null +++ b/UefiCpuPkg/Library/CpuMmuLib/LoongArch64/CommonMmuLib.c @@ -0,0 +1,988 @@ +/** @file + + CPU Memory Map Unit Handler Library common functions. + + Copyright (c) 2024 Loongson Technology Corporation Limited. All rights reserved.<BR> + + SPDX-License-Identifier: BSD-2-Clause-Patent + + @par Glossary: + - Pgd or Pgd or PGD - Page Global Directory + - Pud or Pud or PUD - Page Upper Directory + - Pmd or Pmd or PMD - Page Middle Directory + - Pte or pte or PTE - Page Table Entry + - Val or VAL or val - Value + - Dir - Directory +**/ +#include <Uefi.h> +#include <Library/BaseLib.h> +#include <Library/BaseMemoryLib.h> +#include <Library/CpuMmuLib.h> +#include <Library/DebugLib.h> +#include <Library/MemoryAllocationLib.h> +#include <Register/LoongArch64/Csr.h> +#include "Tlb.h" +#include "Page.h" + +#define SWAP_PAGE_DIR CsrRead(LOONGARCH_CSR_PGDL)(2) Missing space after "CsrRead".
OK, I will rename with out EFI_perfix next time.+#define EFI_MEMORY_CACHETYPE_MASK (EFI_MEMORY_UC | \ + EFI_MEMORY_WC | \ + EFI_MEMORY_WT | \ + EFI_MEMORY_WB | \ + EFI_MEMORY_UCE \ + )(3) This seems to come from "ArmPkg/Include/Library/ArmLib.h"; it's not great regardless: we shouldn't use the EFI_ prefix for symbols that are not standard. (Put differently, EFI_ is a reserved namespace prefix for the UEFI and PI specs.)
+ +BOOLEAN mMmuInited = FALSE;(4) This should be STATIC, I believe. (5) So this is the point where I realize that the library design makes no sense to me. Normally you create SEC/PEI vs. DXE phase instances of a library because, using writable global variables, you can gain performance (you can remember the initialization) in DXE, but in SEC/PEI, you don't have writeable global variables. This pattern does not seem to apply here, or at least it doesn't seem to work. Here's why: - the variable mMmuInited and the function MmuInitialize (which contains an assignment to the variable) are in "CommonMmuLib.c", which gets built into both library instances. This makes no sense; that assignment cannot work in SEC/PEI (I presume you're not going to have writeable globals -- executing from flash). - I think you may be trying to make up for that problem by checking SWAP_PAGE_DIR (the LOONGARCH_CSR_PGDL register) in MmuInitialize() and MmuIsInit(). That doesn't seem right.
Yes, you are right, the PEI stage may be executed from flash, in this case, we have no way to write global variables, so we can only check whether CSR_PGDL is NULL in the DXE stage to get whether the MMU has been initialized.
- The PEI instance of the library contains an EFIAPI function called ConfigureMemoryManagementUnit(). This function is never called in this patch, which makes me think it's supposed to be called from driver or application code (i.e., not from within the library itself, but from client code). However, ConfigureMemoryManagementUnit() is also not declared in the previous patch (in "UefiCpuPkg/Include/Library/CpuMmuLib.h"); therefore client code cannot reach it at all -- so that function doesn't even belong in this library.
This API was discussed with Ray in the V3 patch 13. Actually, the CpuMmuLib.h include more APIs before, Ray believed that other APIs should not be open to users or some APIs can be done by the base APIs, and in my opinion, the ConfigureMemoryManagementUnit() is a private API, so in V4, it has been removed from CpuMmuLib.h.
So what do you think? the ConfigureMemoryManagementUnit should be
added back?
- MmuInitialize() doesn't actually do anything, it just checks (via the CSR) whether some other component has already initialized the MMU (likely with ConfigureMemoryManagementUnit()). And the sole purpose of MmuIsInit() appears to be to return from the public library APIs SetMemoryRegionAttributes() and GetMemoryRegionAttributes() early, if that "other" (unknown) component has not called ConfigureMemoryManagementUnit() yet. So, I don't understand what you are trying to do. I could explain how to keep the initialization logic *differences* minimal between the SEC/PEI and the DXE library instances, but I don't understand how / when the MMU initialization is supposed to occur in the first place.
Let's me tell you this library how to work:
In PEI stage, in addition to ensuring that the MMU is not used, the user must call the ConfigureMemoryManagementUnit toinitialization the MMU, such as filling the static page tables, set the TLB refill exception entry point, set the page size etc. the ConfigureMemoryManagementUnit is a private API but related to ARCH.
During DXE stage, this library will provide services for CpuDxe and other drivers, but almost changes to memory page attributes use the protocols provided by CpuDxe. That is why the CpuMmuLib.h only contains two APIs, it only can get/set the attribute.
In short, the PEI stage needs to initialize and set up TLB refill
entry point and method(dynamically populate the TLB, keep the PA
== VA), and DXE stage is provides services for changing the memory
attributes.
OK, I will try it.(6) The patch is too large in general. You should construct these library instances over multiple patches. The first patch could add some declarations / macro definitions, such as "Page.h" and "Tlb.h". Another patch could add the assembly language helper functions. Another patch could add the PEI phase library instance (including the common code). A final patch could add the DXE-phase bits. The idea is to proceed in layers, logically building one on top of the other. It's fine if the library doesn't build initially; there is no attempt to build it anyway until you actually reference the INF files in some DSC files. So please split this at least in 4 patches.
OK.(7) The commit message should explain the expected usage model in detail.
Best regards, Laszlo+ +/** + Check to see if mmu successfully initializes. + + @param VOID. + + @retval TRUE Initialization has been completed. + FALSE Initialization did not complete. +**/ +STATIC +BOOLEAN +MmuIsInit ( + VOID + ) +{ + if (mMmuInited || (SWAP_PAGE_DIR != 0)) { + return TRUE; + } + + return FALSE; +} + +/** + Iterates through the page directory to initialize it. + + @param Dst A pointer to the directory of the page to initialize. + @param Num The number of page directories to initialize. + @param Src A pointer to the data used to initialize the page directory. + + @return VOID. +**/ +STATIC +VOID +PageDirInit ( + IN VOID *Dst, + IN UINTN Num, + IN VOID *Src + ) +{ + UINTN *Ptr; + UINTN *End; + UINTN Entry; + + Entry = (UINTN)Src; + Ptr = (UINTN *)Dst; + End = Ptr + Num; + + for ( ; Ptr < End; Ptr++) { + *Ptr = Entry; + } + + return; +} + +/** + Gets the virtual address corresponding to the page global directory table entry. + + @param Address the virtual address for the table entry. + + @retval PGD A pointer to get the table item. +**/ +STATIC +PGD * +PgdOffset ( + IN UINTN Address + ) +{ + return (PGD *)(SWAP_PAGE_DIR) + PGD_INDEX (Address); +} + +/** + Gets the virtual address corresponding to the page upper directory table entry. + + @param Pgd A pointer to a page global directory table entry. + @param Address the virtual address for the table entry. + + @retval PUD A pointer to get the table item. +**/ +STATIC +PUD * +PudOffset ( + IN PGD *Pgd, + IN UINTN Address + ) +{ + UINTN PgdVal; + + PgdVal = (UINTN)PGD_VAL (*Pgd); + + return (PUD *)PgdVal + PUD_INDEX (Address); +} + +/** + Gets the virtual address corresponding to the page middle directory table entry. + + @param Pud A pointer to a page upper directory table entry. + @param Address the virtual address for the table entry. + + @retval PMD A pointer to get the table item. +**/ +STATIC +PMD * +PmdOffset ( + IN PUD *Pud, + IN UINTN Address + ) +{ + UINTN PudVal; + + PudVal = PUD_VAL (*Pud); + + return (PMD *)PudVal + PMD_INDEX (Address); +} + +/** + Gets the virtual address corresponding to the page table entry. + + @param Pmd A pointer to a page middle directory table entry. + @param Address the virtual address for the table entry. + + @retval PTE A pointer to get the table item. +**/ +STATIC +PTE * +PteOffset ( + IN PMD *Pmd, + IN UINTN Address + ) +{ + UINTN PmdVal; + + PmdVal = (UINTN)PMD_VAL (*Pmd); + + return (PTE *)PmdVal + PTE_INDEX (Address); +} + +/** + Sets the value of the page table entry. + + @param Pte A pointer to a page table entry. + @param PteVal The value of the page table entry to set. + +**/ +STATIC +VOID +SetPte ( + IN PTE *Pte, + IN PTE PteVal + ) +{ + *Pte = PteVal; +} + +/** + Sets the value of the page global directory. + + @param Pgd A pointer to a page global directory. + @param Pud The value of the page global directory to set. + +**/ +STATIC +VOID +SetPgd ( + IN PGD *Pgd, + IN PUD *Pud + ) +{ + *Pgd = (PGD) { + ((UINTN)Pud) + }; +} + +/** + Sets the value of the page upper directory. + + @param Pud A pointer to a page upper directory. + @param Pmd The value of the page upper directory to set. + +**/ +STATIC +VOID +SetPud ( + IN PUD *Pud, + IN PMD *Pmd + ) +{ + *Pud = (PUD) { + ((UINTN)Pmd) + }; +} + +/** + Sets the value of the page middle directory. + + @param Pmd A pointer to a page middle directory. + @param Pte The value of the page middle directory to set. + +**/ +STATIC +VOID +SetPmd ( + IN PMD *Pmd, + IN PTE *Pte + ) +{ + *Pmd = (PMD) { + ((UINTN)Pte) + }; +} + +/** + Free up memory space occupied by page tables. + + @param Pte A pointer to the page table. + +**/ +VOID +PteFree ( + IN PTE *Pte + ) +{ + FreePages ((VOID *)Pte, 1); +} + +/** + Free up memory space occupied by page middle directory. + + @param Pmd A pointer to the page middle directory. + +**/ +VOID +PmdFree ( + IN PMD *Pmd + ) +{ + FreePages ((VOID *)Pmd, 1); +} + +/** + Free up memory space occupied by page upper directory. + + @param Pud A pointer to the page upper directory. + +**/ +VOID +PudFree ( + IN PUD *Pud + ) +{ + FreePages ((VOID *)Pud, 1); +} + +/** + Requests the memory space required for the page upper directory, + initializes it, and places it in the specified page global directory + + @param Pgd A pointer to the page global directory. + + @retval EFI_SUCCESS Memory request successful. + @retval EFI_OUT_OF_RESOURCES Resource exhaustion cannot be requested to memory. +**/ +STATIC +EFI_STATUS +PudAlloc ( + IN PGD *Pgd + ) +{ + PUD *Pud; + + Pud = (PUD *)AllocatePages (1); + if (Pud == NULL) { + return EFI_OUT_OF_RESOURCES; + } + + PageDirInit ((VOID *)Pud, ENTRYS_PER_PUD, (VOID *)INVALID_PAGE); + + SetPgd (Pgd, Pud); + + return EFI_SUCCESS; +} + +/** + Requests the memory space required for the page middle directory, + initializes it, and places it in the specified page upper directory + + @param Pud A pointer to the page upper directory. + + @retval EFI_SUCCESS Memory request successful. + @retval EFI_OUT_OF_RESOURCES Resource exhaustion cannot be requested to memory. +**/ +STATIC +EFI_STATUS +PmdAlloc ( + IN PUD *Pud + ) +{ + PMD *Pmd; + + Pmd = (PMD *)AllocatePages (1); + if (!Pmd) { + return EFI_OUT_OF_RESOURCES; + } + + PageDirInit ((VOID *)Pmd, ENTRYS_PER_PMD, (VOID *)INVALID_PAGE); + + SetPud (Pud, Pmd); + + return EFI_SUCCESS; +} + +/** + Requests the memory space required for the page table, + initializes it, and places it in the specified page middle directory + + @param Pmd A pointer to the page middle directory. + + @retval EFI_SUCCESS Memory request successful. + @retval EFI_OUT_OF_RESOURCES Resource exhaustion cannot be requested to memory. +**/ +STATIC +EFI_STATUS +PteAlloc ( + IN PMD *Pmd + ) +{ + PTE *Pte; + + Pte = (PTE *)AllocatePages (1); + if (!Pte) { + return EFI_OUT_OF_RESOURCES; + } + + Pte = ZeroMem (Pte, EFI_PAGE_SIZE); + + SetPmd (Pmd, Pte); + + return EFI_SUCCESS; +} + +/** + Requests the memory space required for the page upper directory, + initializes it, and places it in the specified page global directory, + and get the page upper directory entry corresponding to the virtual address. + + @param Pgd A pointer to the page global directory. + @param Address The corresponding virtual address of the page table entry. + + @retval A pointer to the page upper directory entry. Return NULL, if + allocate the memory buffer is fail. +**/ +STATIC +PUD * +PudAllocGet ( + IN PGD *Pgd, + IN UINTN Address + ) +{ + EFI_STATUS Status; + + if (PGD_IS_EMPTY (*Pgd)) { + Status = PudAlloc (Pgd); + ASSERT_EFI_ERROR (Status); + if (EFI_ERROR (Status)) { + return NULL; + } + } + + return PudOffset (Pgd, Address); +} + +/** + Requests the memory space required for the page middle directory, + initializes it, and places it in the specified page upper directory, + and get the page middle directory entry corresponding to the virtual address. + + @param Pud A pointer to the page upper directory. + @param Address The corresponding virtual address of the page table entry. + + @retval A pointer to the page middle directory entry. Return NULL, if + allocate the memory buffer is fail. +**/ +STATIC +PMD * +PmdAllocGet ( + IN PUD *Pud, + IN UINTN Address + ) +{ + EFI_STATUS Status; + + if (PUD_IS_EMPTY (*Pud)) { + Status = PmdAlloc (Pud); + ASSERT_EFI_ERROR (Status); + if (EFI_ERROR (Status)) { + return NULL; + } + } + + return PmdOffset (Pud, Address); +} + +/** + Requests the memory space required for the page table, + initializes it, and places it in the specified page middle directory, + and get the page table entry corresponding to the virtual address. + + @param Pmd A pointer to the page upper directory. + @param Address The corresponding virtual address of the page table entry. + + @retval A pointer to the page table entry. Return NULL, if allocate + the memory buffer is fail. +**/ +STATIC +PTE * +PteAllocGet ( + IN PMD *Pmd, + IN UINTN Address + ) +{ + EFI_STATUS Status; + + if (PMD_IS_EMPTY (*Pmd)) { + Status = PteAlloc (Pmd); + ASSERT_EFI_ERROR (Status); + if (EFI_ERROR (Status)) { + return NULL; + } + } + + return PteOffset (Pmd, Address); +} + +/** + Gets the physical address of the page table entry corresponding to the specified virtual address. + + @param Address The corresponding virtual address of the page table entry. + + @retval A pointer to the page table entry. + @retval NULL +**/ +STATIC +PTE * +GetPteAddress ( + IN UINTN Address + ) +{ + PGD *Pgd; + PUD *Pud; + PMD *Pmd; + + Pgd = PgdOffset (Address); + + if (PGD_IS_EMPTY (*Pgd)) { + return NULL; + } + + Pud = PudOffset (Pgd, Address); + + if (PUD_IS_EMPTY (*Pud)) { + return NULL; + } + + Pmd = PmdOffset (Pud, Address); + if (PMD_IS_EMPTY (*Pmd)) { + return NULL; + } + + if (IS_HUGE_PAGE (Pmd->PmdVal)) { + return ((PTE *)Pmd); + } + + return PteOffset (Pmd, Address); +} + +/** + Gets the Attributes of Huge Page. + + @param Pmd A pointer to the page middle directory. + + @retval Value of Attributes. +**/ +STATIC +UINTN +GetHugePageAttributes ( + IN PMD *Pmd + ) +{ + UINTN Attributes; + UINTN GlobalFlag; + UINTN HugeVal; + + HugeVal = PMD_VAL (*Pmd); + Attributes = HugeVal & (~HUGEP_PAGE_MASK); + GlobalFlag = ((Attributes & (1 << PAGE_HGLOBAL_SHIFT)) >> PAGE_HGLOBAL_SHIFT) << PAGE_GLOBAL_SHIFT; + Attributes &= ~(1 << PAGE_HGLOBAL_SHIFT); + Attributes |= GlobalFlag; + return Attributes; +} + +/** + Establishes a page table entry based on the specified memory region. + + @param Pmd A pointer to the page middle directory. + @param Address The memory space start address. + @param End The end address of the memory space. + @param Attributes Memory space Attributes. + + @retval EFI_SUCCESS The page table entry was created successfully. + @retval EFI_OUT_OF_RESOURCES Page table entry establishment failed due to resource exhaustion. +**/ +STATIC +EFI_STATUS +MemoryMapPteRange ( + IN PMD *Pmd, + IN UINTN Address, + IN UINTN End, + IN UINTN Attributes + ) +{ + PTE *Pte; + PTE PteVal; + BOOLEAN UpDate; + + Pte = PteAllocGet (Pmd, Address); + if (!Pte) { + return EFI_OUT_OF_RESOURCES; + } + + DEBUG (( + DEBUG_INFO, + "%a %d Address %p End %p Attributes %llx\n", + __func__, + __LINE__, + Address, + End, + Attributes + )); + + do { + UpDate = FALSE; + PteVal = MAKE_PTE (Address, Attributes); + + if ((!PTE_IS_EMPTY (*Pte)) && + (PTE_VAL (*Pte) != PTE_VAL (PteVal))) + { + UpDate = TRUE; + } + + SetPte (Pte, PteVal); + if (UpDate) { + InvalidTlb (Address); + } + } while (Pte++, Address += EFI_PAGE_SIZE, Address != End); + + return EFI_SUCCESS; +} + +/** + Convert Huge Page to Page. + + @param Pmd A pointer to the page middle directory. + @param Address The memory space start address. + @param End The end address of the memory space. + @param Attributes Memory space Attributes. + + @retval EFI_SUCCESS The page table entry was created successfully. + @retval EFI_OUT_OF_RESOURCES Page table entry establishment failed due to resource exhaustion. +**/ +STATIC +EFI_STATUS +ConvertHugePageToPage ( + IN PMD *Pmd, + IN UINTN Address, + IN UINTN End, + IN UINTN Attributes + ) +{ + UINTN OldAttributes; + UINTN HugePageEnd; + UINTN HugePageStart; + EFI_STATUS Status; + + Status = EFI_SUCCESS; + + if ((PMD_IS_EMPTY (*Pmd)) || + (!IS_HUGE_PAGE (Pmd->PmdVal))) + { + Status |= MemoryMapPteRange (Pmd, Address, End, Attributes); + } else { + OldAttributes = GetHugePageAttributes (Pmd); + if (Attributes == OldAttributes) { + return Status; + } + + SetPmd (Pmd, (PTE *)(INVALID_PAGE)); + HugePageStart = Address & PMD_MASK; + HugePageEnd = HugePageStart + HUGE_PAGE_SIZE; + ASSERT (HugePageEnd >= End); + + if (Address > HugePageStart) { + Status |= MemoryMapPteRange (Pmd, HugePageStart, Address, OldAttributes); + } + + Status |= MemoryMapPteRange (Pmd, Address, End, Attributes); + + if (End < HugePageEnd) { + Status |= MemoryMapPteRange (Pmd, End, HugePageEnd, OldAttributes); + } + } + + return Status; +} + +/** + Establishes a page middle directory based on the specified memory region. + + @param Pud A pointer to the page upper directory. + @param Address The memory space start address. + @param End The end address of the memory space. + @param Attributes Memory space Attributes. + + @retval EFI_SUCCESS The page middle directory was created successfully. + @retval EFI_OUT_OF_RESOURCES Page middle directory establishment failed due to resource exhaustion. +**/ +STATIC +EFI_STATUS +MemoryMapPmdRange ( + IN PUD *Pud, + IN UINTN Address, + IN UINTN End, + IN UINTN Attributes + ) +{ + PMD *Pmd; + UINTN Next; + PTE PteVal; + BOOLEAN UpDate; + + Pmd = PmdAllocGet (Pud, Address); + if (Pmd == NULL) { + return EFI_OUT_OF_RESOURCES; + } + + do { + Next = PMD_ADDRESS_END (Address, End); + if (((Address & (~PMD_MASK)) == 0) && + ((Next & (~PMD_MASK)) == 0) && + (PMD_IS_EMPTY (*Pmd) || IS_HUGE_PAGE (Pmd->PmdVal))) + { + UpDate = FALSE; + PteVal = MAKE_HUGE_PTE (Address, Attributes); + + if ((!PMD_IS_EMPTY (*Pmd)) && + (PMD_VAL (*Pmd) != PTE_VAL (PteVal))) + { + UpDate = TRUE; + } + + SetPmd (Pmd, (PTE *)PteVal.PteVal); + if (UpDate) { + InvalidTlb (Address); + } + } else { + ConvertHugePageToPage (Pmd, Address, Next, Attributes); + } + } while (Pmd++, Address = Next, Address != End); + + return EFI_SUCCESS; +} + +/** + Establishes a page upper directory based on the specified memory region. + + @param Pgd A pointer to the page global directory. + @param Address The memory space start address. + @param End The end address of the memory space. + @param Attributes Memory space Attributes. + + @retval EFI_SUCCESS The page upper directory was created successfully. + @retval EFI_OUT_OF_RESOURCES Page upper directory establishment failed due to resource exhaustion. +**/ +STATIC +EFI_STATUS +MemoryMapPudRange ( + IN PGD *Pgd, + IN UINTN Address, + IN UINTN End, + IN UINTN Attributes + ) +{ + PUD *Pud; + UINTN Next; + + Pud = PudAllocGet (Pgd, Address); + if (Pud == NULL) { + return EFI_OUT_OF_RESOURCES; + } + + do { + Next = PUD_ADDRESS_END (Address, End); + if (EFI_ERROR (MemoryMapPmdRange (Pud, Address, Next, Attributes))) { + return EFI_OUT_OF_RESOURCES; + } + } while (Pud++, Address = Next, Address != End); + + return EFI_SUCCESS; +} + +/** + Establishes a page global directory based on the specified memory region. + + @param Start The memory space start address. + @param End The end address of the memory space. + @param Attributes Memory space Attributes. + + @retval EFI_SUCCESS The page global directory was created successfully. + @retval EFI_OUT_OF_RESOURCES Page global directory establishment failed due to resource exhaustion. +**/ +STATIC +EFI_STATUS +MemoryMapPageRange ( + IN UINTN Start, + IN UINTN End, + IN UINTN Attributes + ) +{ + PGD *Pgd; + UINTN Next; + UINTN Address; + EFI_STATUS Err; + + Address = Start; + + /* Get PGD(PTE PMD PUD PGD) in PageTables */ + Pgd = PgdOffset (Address); + do { + Next = PGD_ADDRESS_END (Address, End); + /* Get Next Align Page to Map */ + Err = MemoryMapPudRange (Pgd, Address, Next, Attributes); + if (Err) { + return Err; + } + } while (Pgd++, Address = Next, Address != End); + + return EFI_SUCCESS; +} + +/** + Page tables are established from memory-mapped tables. + + @param MemoryRegion A pointer to a memory-mapped table entry. + + @retval EFI_SUCCESS The page table was created successfully. + @retval EFI_OUT_OF_RESOURCES Page table establishment failed due to resource exhaustion. +**/ +EFI_STATUS +FillTranslationTable ( + IN MEMORY_REGION_DESCRIPTOR *MemoryRegion + ) +{ + return MemoryMapPageRange ( + MemoryRegion->VirtualBase, + (MemoryRegion->Length + MemoryRegion->VirtualBase), + MemoryRegion->Attributes + ); +} + +/** + Convert EFI Attributes to Loongarch Attributes. + + @param[in] EfiAttributes Efi Attributes. + + @retval Corresponding architecture attributes. +**/ +UINTN +EFIAPI +EfiAttributeConverse ( + IN UINTN EfiAttributes + ) +{ + UINTN LoongArchAttributes; + + LoongArchAttributes = PAGE_VALID | PAGE_DIRTY | PLV_KERNEL | PAGE_GLOBAL; + + switch (EfiAttributes & EFI_MEMORY_CACHETYPE_MASK) { + case EFI_MEMORY_UC: + LoongArchAttributes |= CACHE_SUC; + break; + case EFI_MEMORY_WC: + LoongArchAttributes |= CACHE_WUC; + break; + case EFI_MEMORY_WT: + case EFI_MEMORY_WB: + LoongArchAttributes |= CACHE_CC; + break; + default: + LoongArchAttributes |= CACHE_CC; + break; + } + + // Write protection attributes + if (((EfiAttributes & EFI_MEMORY_RO) != 0) || + ((EfiAttributes & EFI_MEMORY_WP) != 0)) + { + LoongArchAttributes &= ~PAGE_DIRTY; + } + + if ((EfiAttributes & EFI_MEMORY_RP) != 0) { + LoongArchAttributes |= PAGE_NO_READ; + } + + // eXecute protection attribute + if ((EfiAttributes & EFI_MEMORY_XP) != 0) { + LoongArchAttributes |= PAGE_NO_EXEC; + } + + return LoongArchAttributes; +} + +/** + Finds the first of the length and memory properties of the memory region corresponding + to the specified base address. + + @param[in] BaseAddress To find the base address of the memory region. + @param[in, out] RegionLength Pointer holding: + - At entry, the length of the memory region + expected to be found. + - At exit, the length of the memory region found. + @param[out] RegionAttributes Properties of the memory region found. + + @retval EFI_SUCCESS The corresponding memory area was successfully found + EFI_NOT_FOUND No memory area found + EFI_OUT_OF_RESOURCES Base address or expected memory region exceeds the maximum + address. +**/ +EFI_STATUS +EFIAPI +GetMemoryRegionAttributes ( + IN UINTN BaseAddress, + IN OUT UINTN *RegionLength, + OUT UINTN *RegionAttributes + ) +{ + PTE *Pte; + UINTN Attributes; + UINTN AttributesTmp; + UINTN MaxAddress; + UINTN EndAddress; + UINTN AddSize; + + if (!MmuIsInit ()) { + return EFI_UNSUPPORTED; + } + + EndAddress = BaseAddress + *RegionLength; + MaxAddress = LShiftU64 (1ULL, MAX_VA_BITS) - 1; + + // Clean the value to prepare output to find region size. + *RegionLength = 0x0; + + if ((BaseAddress >= MaxAddress) || (EndAddress >= MaxAddress)) { + return EFI_OUT_OF_RESOURCES; + } + + Pte = GetPteAddress (BaseAddress); + + if (Pte == NULL) { + return EFI_NOT_FOUND; + } + + Attributes = GET_PAGE_ATTRIBUTES (*Pte); + if (IS_HUGE_PAGE (Pte->PteVal)) { + *RegionAttributes = Attributes & (~(PAGE_HUGE)); + } else { + *RegionAttributes = Attributes; + } + + do { + Pte = GetPteAddress (BaseAddress); + if (Pte == NULL) { + return EFI_SUCCESS; + } + + AttributesTmp = GET_PAGE_ATTRIBUTES (*Pte); + if (AttributesTmp == Attributes) { + if (IS_HUGE_PAGE (Pte->PteVal)) { + AddSize = HUGE_PAGE_SIZE; + } else { + AddSize = EFI_PAGE_SIZE; + } + + *RegionLength += AddSize; + BaseAddress += AddSize; + } else { + return EFI_SUCCESS; + } + } while (BaseAddress <= EndAddress); + + return EFI_SUCCESS; +} + +/** + Sets the Attributes of the specified memory region + + @param[in] BaseAddress The base address of the memory region to set the Attributes. + @param[in] Length The length of the memory region to set the Attributes. + @param[in] Attributes The Attributes to be set. + @param[in] AttributeMask Mask of memory attributes to take into account. + + @retval EFI_SUCCESS The Attributes was set successfully +**/ +EFI_STATUS +EFIAPI +SetMemoryRegionAttributes ( + IN EFI_PHYSICAL_ADDRESS BaseAddress, + IN UINTN Length, + IN UINTN Attributes, + IN UINT64 AttributeMask + ) +{ + EFI_STATUS Status; + + if (!MmuIsInit ()) { + return EFI_UNSUPPORTED; + } + + Attributes = EfiAttributeConverse (Attributes); + Status = MemoryMapPageRange (BaseAddress, BaseAddress + Length, Attributes); + ASSERT_EFI_ERROR (Status); + + return Status; +} + +/** + Check to see if mmu successfully initializes and saves the result. + + @param[in] ImageHandle The firmware allocated handle for the EFI image. + @param[in] SystemTable A pointer to the EFI System Table. + + @retval RETURN_SUCCESS Initialization succeeded. +**/ +RETURN_STATUS +MmuInitialize ( + IN EFI_HANDLE ImageHandle, + IN EFI_SYSTEM_TABLE *SystemTable + ) +{ + if (SWAP_PAGE_DIR != 0) { + mMmuInited = TRUE; + } + + return RETURN_SUCCESS; +} diff --git a/UefiCpuPkg/Library/CpuMmuLib/LoongArch64/CommonMmuLib.h b/UefiCpuPkg/Library/CpuMmuLib/LoongArch64/CommonMmuLib.h new file mode 100644 index 0000000000..d8c922c8fa --- /dev/null +++ b/UefiCpuPkg/Library/CpuMmuLib/LoongArch64/CommonMmuLib.h @@ -0,0 +1,43 @@ +/** @file + + Copyright (c) 2024 Loongson Technology Corporation Limited. All rights reserved.<BR> + + SPDX-License-Identifier: BSD-2-Clause-Patent + + @par Glossary: + - Dir - Directory +**/ + +#ifndef MMU_LIB_CORE_H_ +#define MMU_LIB_CORE_H_ + +/** + Iterates through the page directory to initialize it. + + @param Dst A pointer to the directory of the page to initialize. + @param Num The number of page directories to initialize. + @param Src A pointer to the data used to initialize the page directory. + + @retval VOID. +**/ +VOID +PageDirInit ( + IN VOID *dest, + IN UINTN Count, + IN VOID *src + ); + +/** + Page tables are established from memory-mapped tables. + + @param MemoryRegion A pointer to a memory-mapped table entry. + + @retval EFI_SUCCESS The page table was created successfully. + @retval EFI_OUT_OF_RESOURCES Page table establishment failed due to resource exhaustion. +**/ +EFI_STATUS +FillTranslationTable ( + IN MEMORY_REGION_DESCRIPTOR *MemoryRegion + ); + +#endif // MMU_LIB_CORE_H_ diff --git a/UefiCpuPkg/Library/CpuMmuLib/LoongArch64/Page.h b/UefiCpuPkg/Library/CpuMmuLib/LoongArch64/Page.h new file mode 100644 index 0000000000..bac4f52327 --- /dev/null +++ b/UefiCpuPkg/Library/CpuMmuLib/LoongArch64/Page.h @@ -0,0 +1,279 @@ +/** @file + + Copyright (c) 2024 Loongson Technology Corporation Limited. All rights reserved.<BR> + + SPDX-License-Identifier: BSD-2-Clause-Patent + + @par Glossary: + - Pgd or Pgd or PGD - Page Global Directory + - Pud or Pud or PUD - Page Upper Directory + - Pmd or Pmd or PMD - Page Middle Directory + - Pte or pte or PTE - Page Table Entry + - Val or VAL or val - Value + - Dir - Directory +**/ + +#ifndef PAGE_H_ +#define PAGE_H_ + +#include <Library/CpuMmuLib.h> + +#define MAX_VA_BITS 47 +#define PGD_WIDE (8) +#define PUD_WIDE (9) +#define PMD_WIDE (9) +#define PTE_WIDE (9) + +#define ENTRYS_PER_PGD (1 << PGD_WIDE) +#define ENTRYS_PER_PUD (1 << PUD_WIDE) +#define ENTRYS_PER_PMD (1 << PMD_WIDE) +#define ENTRYS_PER_PTE (1 << PTE_WIDE) + +#define PGD_SHIFT (PUD_SHIFT + PUD_WIDE) +#define PUD_SHIFT (PMD_SHIFT + PMD_WIDE) +#define PMD_SHIFT (EFI_PAGE_SHIFT + PTE_WIDE) +#define PTE_SHIFT (EFI_PAGE_SHIFT) + +#define PGD_SIZE (1UL << PGD_SHIFT) +#define PUD_SIZE (1UL << PUD_SHIFT) +#define PMD_SIZE (1UL << PMD_SHIFT) + +#define PGD_MASK (~(PGD_SIZE-1)) +#define PUD_MASK (~(PUD_SIZE-1)) +#define PMD_MASK (~(PMD_SIZE-1)) +#define PAGE_MASK (~(EFI_PAGE_SIZE - 1)) +#define PFN_MASK (~(((UINTN)(1) << (EFI_PAGE_SHIFT)) - 1) & \ + (((UINTN)(1) << (PAGE_PFN_END_SHIFT)) - 1)) + +#define HUGEP_PAGE_MASK (~(((UINTN)(1) << (PMD_SHIFT)) - 1) & \ + (((UINTN)(1) << (PAGE_PFN_END_SHIFT)) - 1)) + +#define INVALID_PAGE 0 + +typedef struct { + UINTN PgdVal; +} PGD; +typedef struct { + UINTN PudVal; +} PUD; +typedef struct { + UINTN PmdVal; +} PMD; +typedef struct { + UINTN PteVal; +} PTE; + +/** + Gets the value of the page global directory table entry. + + @param x Page global directory struct variables. + + @retval the value of the page global directory table entry. + **/ +#define PGD_VAL(x) ((x).PgdVal) + +/** + Gets the value of the page upper directory table entry. + + @param x Page upper directory struct variables. + + @retval the value of the page upper directory table entry. + **/ +#define PUD_VAL(x) ((x).PudVal) + +/** + Gets the value of the page middle directory table entry. + + @param x Page middle directory struct variables. + + @retval the value of the page middle directory table entry. + **/ +#define PMD_VAL(x) ((x).PmdVal) + +/** + Gets the value of the page table entry. + + @param x Page table entry struct variables. + + @retval the value of the page table entry. + **/ +#define PTE_VAL(x) ((x).PteVal) + +#define PGD_TABLE_SIZE (ENTRYS_PER_PGD * sizeof(PGD)) +#define PUD_TABLE_SIZE (ENTRYS_PER_PUD * sizeof(PUD)) +#define PMD_TABLE_SIZE (ENTRYS_PER_PMD * sizeof(PMD)) +#define PTE_TABLE_SIZE (ENTRYS_PER_PTE * sizeof(PTE)) + +/** + Gets the physical address of the record in the page table entry. + + @param x Page table entry struct variables. + + @retval the value of the physical address. + **/ +#define GET_PAGE_ATTRIBUTES(x) (UINTN) {(PTE_VAL(x) & ~PFN_MASK)} + +/** + Gets the virtual address of the next block of the specified virtual address + that is aligned with the size of the global page directory mapping. + + @param Address Specifies the virtual address. + @param End The end address of the memory region. + + @retval the specified virtual address of the next block. + **/ +#define PGD_ADDRESS_END(Address, End) \ +({ \ + UINTN Boundary = ((Address) + PGD_SIZE) & PGD_MASK; \ + (Boundary - 1 < (End) - 1)? Boundary: (End); \ +}) + +/** + Gets the virtual address of the next block of the specified virtual address + that is aligned with the size of the page upper directory mapping. + + @param Address Specifies the virtual address. + @param End The end address of the memory region. + + @retval the specified virtual address of the next block. + **/ +#define PUD_ADDRESS_END(Address, End) \ +({ \ + UINTN Boundary = ((Address) + PUD_SIZE) & PUD_MASK; \ + (Boundary - 1 < (End) - 1)? Boundary: (End); \ +}) + +/** + Gets the virtual address of the next block of the specified virtual address + that is aligned with the size of the page middle directory mapping. + + @param Address Specifies the virtual address. + @param End The end address of the memory region. + + @retval the specified virtual address of the next block. + **/ +#define PMD_ADDRESS_END(Address, End) \ +({ \ + UINTN Boundary = ((Address) + PMD_SIZE) & PMD_MASK; \ + (Boundary - 1 < (End) - 1)? Boundary: (End); \ +}) + +/** + Get Specifies the virtual address corresponding to the index of the page global directory table entry. + + @param Address Specifies the virtual address. + + @retval the index of the page global directory table entry. + **/ +#define PGD_INDEX(Address) (((Address) >> PGD_SHIFT) & (ENTRYS_PER_PGD-1)) + +/** + Get Specifies the virtual address corresponding to the index of the page upper directory table entry. + + @param Address Specifies the virtual address. + @param End The end address of the memory region. + + @retval the index of the page upper directory table entry. + **/ +#define PUD_INDEX(Address) (((Address) >> PUD_SHIFT) & (ENTRYS_PER_PUD - 1)) + +/** + Get Specifies the virtual address corresponding to the index of the page middle directory table entry. + + @param Address Specifies the virtual address. + + @retval the index of the page middle directory table entry. + **/ +#define PMD_INDEX(Address) (((Address) >> PMD_SHIFT) & (ENTRYS_PER_PMD - 1)) + +/** + Get Specifies the virtual address corresponding to the index of the page table entry. + + @param Address Specifies the virtual address. + + @retval the index of the page table entry. + **/ +#define PTE_INDEX(Address) (((Address) >> EFI_PAGE_SHIFT) & (ENTRYS_PER_PTE - 1)) + +/** + Calculates the value of the page table entry based on the specified virtual address and properties. + + @param Address Specifies the virtual address. + @param Attributes Specifies the Attributes. + + @retval the value of the page table entry. + **/ +#define MAKE_PTE(Address, Attributes) (PTE){((((Address) >> EFI_PAGE_SHIFT) << 12) | (Attributes))} + +/** + Get Global bit from Attributes + + @param Attributes Specifies the Attributes. + * */ +#define GET_GLOBALBIT(Attributes) ((Attributes & PAGE_GLOBAL) >> PAGE_GLOBAL_SHIFT) + +/** + Calculates the value of the Huge page table entry based on the specified virtual address and properties. + + @param Address Specifies the virtual address. + @param Attributes Specifies the Attributes. + + @retval the value of the HUGE page table entry. + **/ +#define MAKE_HUGE_PTE(Address, Attributes) (PTE){(((((Address) >> PMD_SHIFT) << PMD_SHIFT) | \ + ((Attributes) | (GET_GLOBALBIT(Attributes) << PAGE_HGLOBAL_SHIFT) | \ + PAGE_HUGE)))} + +/** + Check whether the large page table entry is. + + @param Val The value of the page table entry. + + @retval 1 Is huge page table entry. + @retval 0 Isn't huge page table entry. +**/ +#define IS_HUGE_PAGE(Val) ((((Val) & PAGE_HUGE) == PAGE_HUGE) && \ + (((Val) & PAGE_HGLOBAL) == PAGE_HGLOBAL)) + +#define HUGE_PAGE_SIZE (PMD_SIZE) + +/** + Check that the global page directory table entry is empty. + + @param pgd the global page directory struct variables. + + @retval 1 The page table is invalid. + @retval 0 The page table is valid. +**/ +#define PGD_IS_EMPTY(Val) (PGD_VAL(Val) == INVALID_PAGE) + +/** + Check that the page upper directory table entry is empty. + + @param pud Page upper directory struct variables. + + @retval 1 The page table is invalid. + @retval 0 The page table is valid. +**/ +#define PUD_IS_EMPTY(Val) (PUD_VAL(Val) == INVALID_PAGE) + +/** + Check that the page middle directory table entry is empty. + + @param pmd Page middle directory struct variables. + + @retval 1 The page table is invalid. + @retval 0 The page table is valid. +**/ +#define PMD_IS_EMPTY(Val) (PMD_VAL(Val) == INVALID_PAGE) + +/** + Check that the page the page table entry is empty. + + @param pte Page table entry struct variables. + + @retval 1 The page table is invalid. + @retval 0 The page table is valid. +**/ +#define PTE_IS_EMPTY(Val) (!(PTE_VAL(Val) & (~PAGE_VALID))) +#endif // PAGE_H_ diff --git a/UefiCpuPkg/Library/CpuMmuLib/LoongArch64/PeiCpuMmuLib.c b/UefiCpuPkg/Library/CpuMmuLib/LoongArch64/PeiCpuMmuLib.c new file mode 100644 index 0000000000..c214e8d847 --- /dev/null +++ b/UefiCpuPkg/Library/CpuMmuLib/LoongArch64/PeiCpuMmuLib.c @@ -0,0 +1,178 @@ +/** @file + CPU Memory Map Unit PEI phase driver. + + Copyright (c) 2024 Loongson Technology Corporation Limited. All rights reserved.<BR> + + SPDX-License-Identifier: BSD-2-Clause-Patent + + @par Glossary: + - Tlb - Translation Lookaside Buffer +**/ + +#include <Uefi.h> +#include <Library/BaseLib.h> +#include <Library/BaseMemoryLib.h> +#include <Library/CacheMaintenanceLib.h> +#include <Library/CpuMmuLib.h> +#include <Library/DebugLib.h> +#include <Library/MemoryAllocationLib.h> +#include <Library/PcdLib.h> +#include <Register/LoongArch64/Csr.h> + +#include "Page.h" +#include "Tlb.h" +#include "CommonMmuLib.h" + +// +// For coding convenience, define the maximum valid +// LoongArch exception. +// Since UEFI V2.11, it will be present in DebugSupport.h. +// +#define MAX_LOONGARCH_EXCEPTION 64 + +/** + Create a page table and initialize the memory management unit(MMU). + + @param[in] MemoryTable A pointer to a memory ragion table. + @param[out] TranslationTableBase A pointer to a translation table base address. + @param[out] TranslationTableSize A pointer to a translation table base size. + + @retval EFI_SUCCESS Configure MMU successfully. + EFI_INVALID_PARAMETER MemoryTable is NULL. + EFI_UNSUPPORTED Out of memory space or size not aligned. +**/ +EFI_STATUS +EFIAPI +ConfigureMemoryManagementUnit ( + IN MEMORY_REGION_DESCRIPTOR *MemoryTable, + OUT VOID **TranslationTableBase OPTIONAL, + OUT UINTN *TranslationTableSize OPTIONAL + ) +{ + PGD *SwapperPageDir; + UINTN PgdShift; + UINTN PgdWide; + UINTN PudShift; + UINTN PudWide; + UINTN PmdShift; + UINTN PmdWide; + UINTN PteShift; + UINTN PteWide; + UINTN Length; + UINTN TlbReEntry; + UINTN TlbReEntryOffset; + UINTN Remaining; + RETURN_STATUS Status; + + SwapperPageDir = NULL; + PgdShift = PGD_SHIFT; + PgdWide = PGD_WIDE; + PudShift = PUD_SHIFT; + PudWide = PUD_WIDE; + PmdShift = PMD_SHIFT; + PmdWide = PMD_WIDE; + PteShift = PTE_SHIFT; + PteWide = PTE_WIDE; + + if (MemoryTable == NULL) { + ASSERT (MemoryTable != NULL); + return EFI_INVALID_PARAMETER; + } + + SwapperPageDir = AllocatePages (EFI_SIZE_TO_PAGES (PGD_TABLE_SIZE)); + ZeroMem (SwapperPageDir, PGD_TABLE_SIZE); + + if (SwapperPageDir == NULL) { + goto FreeTranslationTable; + } + + CsrWrite (LOONGARCH_CSR_PGDL, (UINTN)SwapperPageDir); + + while (MemoryTable->Length != 0) { + DEBUG (( + DEBUG_INFO, + "%a %d VirtualBase %p VirtualEnd %p Attributes %p .\n", + __func__, + __LINE__, + MemoryTable->VirtualBase, + (MemoryTable->Length + MemoryTable->VirtualBase), + MemoryTable->Attributes + )); + + Status = FillTranslationTable (MemoryTable); + if (EFI_ERROR (Status)) { + goto FreeTranslationTable; + } + + MemoryTable++; + } + + // + // TLB Re-entry address at the end of exception vector, a vector is up to 512 bytes, + // so the starting address is: total exception vector size + total interrupt vector size + base. + // The total size of TLB handler and exception vector size and interrupt vector size should not + // be lager than 64KB. + // + Length = (UINTN)HandleTlbRefillEnd - (UINTN)HandleTlbRefillStart; + TlbReEntryOffset = (MAX_LOONGARCH_EXCEPTION + MAX_LOONGARCH_INTERRUPT) * 512; + Remaining = TlbReEntryOffset % SIZE_4KB; + if (Remaining != 0x0) { + TlbReEntryOffset += (SIZE_4KB - Remaining); + } + + TlbReEntry = PcdGet64 (PcdCpuExceptionVectorBaseAddress) + TlbReEntryOffset; + if ((TlbReEntryOffset + Length) > SIZE_64KB) { + goto FreeTranslationTable; + } + + // + // Ensure that TLB refill exception base address alignment is equals to 4KB and is valid. + // + if (TlbReEntry & (SIZE_4KB - 1)) { + goto FreeTranslationTable; + } + + CopyMem ((VOID *)TlbReEntry, HandleTlbRefillStart, Length); + InvalidateInstructionCacheRange ((VOID *)(UINTN)HandleTlbRefillStart, Length); + + DEBUG (( + DEBUG_INFO, + "%a %d PteShift %d PteWide %d PmdShift %d PmdWide %d PudShift %d PudWide %d PgdShift %d PgdWide %d.\n", + __func__, + __LINE__, + PteShift, + PteWide, + PmdShift, + PmdWide, + PudShift, + PudWide, + PgdShift, + PgdWide + )); + + // + // Set the address of TLB refill exception handler + // + SetTlbRebaseAddress ((UINTN)TlbReEntry); + + // + // Set page size + // + CsrXChg (LOONGARCH_CSR_TLBIDX, (DEFAULT_PAGE_SIZE << CSR_TLBIDX_SIZE), CSR_TLBIDX_SIZE_MASK); + CsrWrite (LOONGARCH_CSR_STLBPGSIZE, DEFAULT_PAGE_SIZE); + CsrXChg (LOONGARCH_CSR_TLBREHI, (DEFAULT_PAGE_SIZE << CSR_TLBREHI_PS_SHIFT), CSR_TLBREHI_PS); + + CsrWrite (LOONGARCH_CSR_PWCTL0, (PteShift | PteWide << 5 | PmdShift << 10 | PmdWide << 15 | PudShift << 20 | PudWide << 25)); + CsrWrite (LOONGARCH_CSR_PWCTL1, (PgdShift | PgdWide << 6)); + + DEBUG ((DEBUG_INFO, "%a %d Enable Mmu Start PageBassAddress %p.\n", __func__, __LINE__, SwapperPageDir)); + + return EFI_SUCCESS; + +FreeTranslationTable: + if (SwapperPageDir != NULL) { + FreePages (SwapperPageDir, EFI_SIZE_TO_PAGES (PGD_TABLE_SIZE)); + } + + return EFI_UNSUPPORTED; +} diff --git a/UefiCpuPkg/Library/CpuMmuLib/LoongArch64/Tlb.h b/UefiCpuPkg/Library/CpuMmuLib/LoongArch64/Tlb.h new file mode 100644 index 0000000000..9a681ce8e1 --- /dev/null +++ b/UefiCpuPkg/Library/CpuMmuLib/LoongArch64/Tlb.h @@ -0,0 +1,48 @@ +/** @file + + Copyright (c) 2024 Loongson Technology Corporation Limited. All rights reserved.<BR> + + SPDX-License-Identifier: BSD-2-Clause-Patent + +**/ + +#ifndef TLB_H_ +#define TLB_H_ + +/** + Invalid corresponding TLB entries are based on the address given + + @param Address The address corresponding to the invalid page table entry + + @retval none +**/ +VOID +InvalidTlb ( + UINTN Address + ); + +/** + TLB refill handler start. + + @param none + + @retval none +**/ +VOID +HandleTlbRefillStart ( + VOID + ); + +/** + TLB refill handler end. + + @param none + + @retval none +**/ +VOID +HandleTlbRefillEnd ( + VOID + ); + +#endif // TLB_H_ diff --git a/UefiCpuPkg/Library/CpuMmuLib/LoongArch64/TlbOperation.S b/UefiCpuPkg/Library/CpuMmuLib/LoongArch64/TlbOperation.S new file mode 100644 index 0000000000..c9a8c16336 --- /dev/null +++ b/UefiCpuPkg/Library/CpuMmuLib/LoongArch64/TlbOperation.S @@ -0,0 +1,44 @@ +#------------------------------------------------------------------------------ +# +# TLB operation functions +# +# Copyright (c) 2024 Loongson Technology Corporation Limited. All rights reserved.<BR> +# +# SPDX-License-Identifier: BSD-2-Clause-Patent +# +#----------------------------------------------------------------------------- + +#include <Register/LoongArch64/Csr.h> + +ASM_GLOBAL ASM_PFX(HandleTlbRefillStart) +ASM_GLOBAL ASM_PFX(HandleTlbRefillEnd) +ASM_GLOBAL ASM_PFX(InvalidTlb) + +# +# Refill the page table. +# @param VOID +# @retval VOID +# +ASM_PFX(HandleTlbRefillStart): + csrwr $t0, LOONGARCH_CSR_TLBRSAVE + csrrd $t0, LOONGARCH_CSR_PGD + lddir $t0, $t0, 3 #Put pud BaseAddress into T0 + lddir $t0, $t0, 2 #Put pmd BaseAddress into T0 + lddir $t0, $t0, 1 #Put pte BaseAddress into T0 + ldpte $t0, 0 + ldpte $t0, 1 + tlbfill // refill hi,lo0,lo1 + csrrd $t0, LOONGARCH_CSR_TLBRSAVE + ertn +ASM_PFX(HandleTlbRefillEnd): + +# +# Invalid corresponding TLB entries are based on the address given +# @param a0 The address corresponding to the invalid page table entry +# @retval none +# +ASM_PFX(InvalidTlb): + invtlb INVTLB_ADDR_GTRUE_OR_ASID, $zero, $a0 + jirl $zero, $ra, 0 + + .end diff --git a/UefiCpuPkg/Library/CpuMmuLib/PeiCpuMmuLib.inf b/UefiCpuPkg/Library/CpuMmuLib/PeiCpuMmuLib.inf new file mode 100644 index 0000000000..45b15db4c9 --- /dev/null +++ b/UefiCpuPkg/Library/CpuMmuLib/PeiCpuMmuLib.inf @@ -0,0 +1,44 @@ +## @file +# CPU Memory Map Unit PEI phase driver. +# +# Copyright (c) 2024 Loongson Technology Corporation Limited. All rights reserved.<BR> +# +# SPDX-License-Identifier: BSD-2-Clause-Patent +# +## + +[Defines] + INF_VERSION = 1.29 + BASE_NAME = PeiCpuMmuLib + MODULE_UNI_FILE = PeiCpuMmuLib.uni + FILE_GUID = F67EB983-AC2A-7550-AB69-3BC51A1C895B + MODULE_TYPE = PEIM + VERSION_STRING = 1.0 + LIBRARY_CLASS = CpuMmuLib | SEC PEIM + +# +# VALID_ARCHITECTURES = LOONGARCH64 +# + +[Sources.LoongArch64] + LoongArch64/TlbOperation.S | GCC + LoongArch64/CommonMmuLib.c + LoongArch64/PeiCpuMmuLib.c + LoongArch64/CommonMmuLib.h + LoongArch64/Tlb.h + LoongArch64/Page.h + +[Packages] + MdePkg/MdePkg.dec + MdeModulePkg/MdeModulePkg.dec + UefiCpuPkg/UefiCpuPkg.dec + +[PCD] + gEfiMdeModulePkgTokenSpaceGuid.PcdNullPointerDetectionPropertyMask + gUefiCpuPkgTokenSpaceGuid.PcdCpuExceptionVectorBaseAddress + +[LibraryClasses] + CacheMaintenanceLib + DebugLib + MemoryAllocationLib + PcdLib diff --git a/UefiCpuPkg/Library/CpuMmuLib/PeiCpuMmuLib.uni b/UefiCpuPkg/Library/CpuMmuLib/PeiCpuMmuLib.uni new file mode 100644 index 0000000000..3e21334f3e --- /dev/null +++ b/UefiCpuPkg/Library/CpuMmuLib/PeiCpuMmuLib.uni @@ -0,0 +1,14 @@ +// /** @file +// CPU Memory Manager Unit library instance for PEI modules. +// +// CPU Memory Manager Unit library instance for PEI modules. +// +// Copyright (c) 2024, Loongson Technology Corporation Limited. All rights reserved.<BR> +// +// SPDX-License-Identifier: BSD-2-Clause-Patent +// +// **/ + +#string STR_MODULE_ABSTRACT #language en-US "CPU Memory Manager Unit library instance for PEI modules." + +#string STR_MODULE_DESCRIPTION #language en-US "CPU Memory Manager Unit library instance for PEI modules." diff --git a/UefiCpuPkg/UefiCpuPkg.dsc b/UefiCpuPkg/UefiCpuPkg.dsc index 28eed85bce..178dc3c0f9 100644 --- a/UefiCpuPkg/UefiCpuPkg.dsc +++ b/UefiCpuPkg/UefiCpuPkg.dsc @@ -207,5 +207,9 @@ UefiCpuPkg/CpuTimerDxeRiscV64/CpuTimerDxeRiscV64.inf UefiCpuPkg/CpuDxeRiscV64/CpuDxeRiscV64.inf +[Components.LOONGARCH64] + UefiCpuPkg/Library/CpuMmuLib/DxeCpuMmuLib.inf + UefiCpuPkg/Library/CpuMmuLib/PeiCpuMmuLib.inf + [BuildOptions] *_*_*_CC_FLAGS = -D DISABLE_NEW_DEPRECATED_INTERFACES