[PATCH v2 0/7] edksetup.bat, BaseTools: Improve Windows environment setup and BaseTools C compilation

[PATCH v2 0/7] edksetup.bat, BaseTools: Improve Windows environment setup and BaseTools C compilation

[Rebecca Cran]

There are remnants of Python 2 support in BaseTools/toolsetup.bat that it's
probably time to remove since we only support Python 3.6 and newer these days.
So, remove the variables that enable Python3 support and simplify the batch
script. I've also seen errors where after running edksetup.bat the build
command isn't available because PYTHONPATH wasn't being set, so fix that
when the Pip BaseTools are being used.

At the same time, let's add a check that we meet the minimum version
requirement so we don't end up failing with an obscure error.

Building BaseTools causes a warning about threading.currentThread being
deprecated, so update code in NmakeSubdirs.py to switch to
threading.current_thread.

There needs to be further work, because if PYTHON_COMMAND isn't specified then
it defaults to "py -3", where py is C:\Windows\py.exe, which doesn't work if
you're using a virtualenv since it installs python.exe and pythonw.exe in
venv\Scripts. toolsetup.bat therefore fails to detect the Pip BaseTools and
uses the in-source Basetools.

GitHub PR: https://github.com/tianocore/edk2/pull/4302
GitHub branch: https://github.com/bcran/edk2/tree/py3

Changes between v1 and v2
=========================

- Require Python 3.6 or newer: 3.6 was when PEP 526 was added, which we use.
- Fix Tests/RunTests.py on Windows.

Dongyan Qian (1):
  BaseSynchronizationLib: Fix LoongArch64 synchronization functions

Gerd Hoffmann (2):
  OvmfPkg: move OvmfTpmPei.fdf.inc to Include/Fdf
  OvmfPkg: move OvmfTpmDxe.fdf.inc to Include/Fdf

Rebecca Cran (4):
  BaseTools: Remove Python2/Python3 detection from toolset.bat
  BaseTools: use threading.current_thread in NmakeSubdirs.py
  edksetup.bat: if toolsetup.bat fails, just exit
  BaseTools: Update toolsetup.bat and Tests/PythonTest.py to check ver

 BaseTools/Source/C/Makefiles/NmakeSubdirs.py                           |   2 +-
 BaseTools/Tests/PythonTest.py                                          |  22 +++-
 BaseTools/toolsetup.bat                                                | 119 ++++++++------------
 MdePkg/Library/BaseSynchronizationLib/LoongArch64/AsmSynchronization.S |  30 ++---
 MdePkg/Library/BaseSynchronizationLib/LoongArch64/Synchronization.c    |   2 +-
 OvmfPkg/AmdSev/AmdSevX64.fdf                                           |   4 +-
 OvmfPkg/CloudHv/CloudHvX64.fdf                                         |   4 +-
 OvmfPkg/{ => Include/Fdf}/OvmfTpmDxe.fdf.inc                           |   0
 OvmfPkg/{ => Include/Fdf}/OvmfTpmPei.fdf.inc                           |   0
 OvmfPkg/OvmfPkgIa32.fdf                                                |   4 +-
 OvmfPkg/OvmfPkgIa32X64.fdf                                             |   4 +-
 OvmfPkg/OvmfPkgX64.fdf                                                 |   4 +-
 edksetup.bat                                                           |   1 +
 13 files changed, 92 insertions(+), 104 deletions(-)
 rename OvmfPkg/{ => Include/Fdf}/OvmfTpmDxe.fdf.inc (100%)
 rename OvmfPkg/{ => Include/Fdf}/OvmfTpmPei.fdf.inc (100%)

-- 
2.40.0.windows.1

[PATCH v2 1/7] BaseSynchronizationLib: Fix LoongArch64 synchronization functions

[Rebecca Cran]

From: Dongyan Qian <qiandongyan@loongson.cn>

REF: https://bugzilla.tianocore.org/show_bug.cgi?id=4432

There is a return value bug:
The sc.w/sc.d instruction will destroy the reg_t0,
use reg_t1 to avoid context reg_t0 being corrupted.
Adjust Check that ptr align is UINT16.
Optimize function SyncIncrement and SyncDecrement.

Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Zhiguang Liu <zhiguang.liu@intel.com>
Cc: Chao Li <lichao@loongson.cn>
Signed-off-by: Dongyan Qian <qiandongyan@loongson.cn>
Reviewed-by: Chao Li <lichao@loongson.cn>
---
 MdePkg/Library/BaseSynchronizationLib/LoongArch64/AsmSynchronization.S | 30 ++++++++------------
 MdePkg/Library/BaseSynchronizationLib/LoongArch64/Synchronization.c    |  2 +-
 2 files changed, 13 insertions(+), 19 deletions(-)

diff --git a/MdePkg/Library/BaseSynchronizationLib/LoongArch64/AsmSynchronization.S b/MdePkg/Library/BaseSynchronizationLib/LoongArch64/AsmSynchronization.S
index fdd50c54b5f3..03865bf2c966 100644
--- a/MdePkg/Library/BaseSynchronizationLib/LoongArch64/AsmSynchronization.S
+++ b/MdePkg/Library/BaseSynchronizationLib/LoongArch64/AsmSynchronization.S
@@ -53,9 +53,9 @@ ASM_PFX(AsmInternalSyncCompareExchange32):
 1:
   ll.w  $t0, $a0, 0x0
   bne   $t0, $a1, 2f
-  move  $t0, $a2
-  sc.w  $t0, $a0, 0x0
-  beqz  $t0, 1b
+  move  $t1, $a2
+  sc.w  $t1, $a0, 0x0
+  beqz  $t1, 1b
   b     3f
 2:
   dbar  0
@@ -76,9 +76,9 @@ ASM_PFX(AsmInternalSyncCompareExchange64):
 1:
   ll.d  $t0, $a0, 0x0
   bne   $t0, $a1, 2f
-  move  $t0, $a2
-  sc.d  $t0, $a0, 0x0
-  beqz  $t0, 1b
+  move  $t1, $a2
+  sc.d  $t1, $a0, 0x0
+  beqz  $t1, 1b
   b     3f
 2:
   dbar  0
@@ -94,13 +94,10 @@ AsmInternalSyncIncrement (
   )
 **/
 ASM_PFX(AsmInternalSyncIncrement):
-  move     $t0, $a0
-  dbar     0
-  ld.w     $t1, $t0, 0x0
-  li.w     $t2, 1
-  amadd.w  $t1, $t2, $t0
+  li.w     $t0, 1
+  amadd.w  $zero, $t0, $a0
 
-  ld.w     $a0, $t0, 0x0
+  ld.w     $a0, $a0, 0
   jirl     $zero, $ra, 0
 
 /**
@@ -111,12 +108,9 @@ AsmInternalSyncDecrement (
   )
 **/
 ASM_PFX(AsmInternalSyncDecrement):
-  move     $t0, $a0
-  dbar     0
-  ld.w     $t1, $t0, 0x0
-  li.w     $t2, -1
-  amadd.w  $t1, $t2, $t0
+  li.w     $t0, -1
+  amadd.w  $zero, $t0, $a0
 
-  ld.w     $a0, $t0, 0x0
+  ld.w     $a0, $a0, 0
   jirl     $zero, $ra, 0
 .end
diff --git a/MdePkg/Library/BaseSynchronizationLib/LoongArch64/Synchronization.c b/MdePkg/Library/BaseSynchronizationLib/LoongArch64/Synchronization.c
index d696c8ce102f..6baf841c9b09 100644
--- a/MdePkg/Library/BaseSynchronizationLib/LoongArch64/Synchronization.c
+++ b/MdePkg/Library/BaseSynchronizationLib/LoongArch64/Synchronization.c
@@ -81,7 +81,7 @@ InternalSyncCompareExchange16 (
   volatile UINT32  *Ptr32;
 
   /* Check that ptr is naturally aligned */
-  ASSERT (!((UINT64)Value & (sizeof (Value) - 1)));
+  ASSERT (!((UINT64)Value & (sizeof (UINT16) - 1)));
 
   /* Mask inputs to the correct size. */
   Mask               = (((~0UL) - (1UL << (0)) + 1) & (~0UL >> (64 - 1 - ((sizeof (UINT16) * 8) - 1))));
-- 
2.40.0.windows.1

[PATCH v2 2/7] OvmfPkg: move OvmfTpmPei.fdf.inc to Include/Fdf

[Rebecca Cran]

From: Gerd Hoffmann <kraxel@redhat.com>

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Jiewen Yao <jiewen.yao@intel.com>
---
 OvmfPkg/AmdSev/AmdSevX64.fdf                 | 2 +-
 OvmfPkg/CloudHv/CloudHvX64.fdf               | 2 +-
 OvmfPkg/{ => Include/Fdf}/OvmfTpmPei.fdf.inc | 0
 OvmfPkg/OvmfPkgIa32.fdf                      | 2 +-
 OvmfPkg/OvmfPkgIa32X64.fdf                   | 2 +-
 OvmfPkg/OvmfPkgX64.fdf                       | 2 +-
 6 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/OvmfPkg/AmdSev/AmdSevX64.fdf b/OvmfPkg/AmdSev/AmdSevX64.fdf
index 5fb3b5d27632..25e446013cd4 100644
--- a/OvmfPkg/AmdSev/AmdSevX64.fdf
+++ b/OvmfPkg/AmdSev/AmdSevX64.fdf
@@ -162,7 +162,7 @@ [FV.PEIFV]
 INF  UefiCpuPkg/CpuMpPei/CpuMpPei.inf
 INF  OvmfPkg/AmdSev/SecretPei/SecretPei.inf
 
-!include OvmfPkg/OvmfTpmPei.fdf.inc
+!include OvmfPkg/Include/Fdf/OvmfTpmPei.fdf.inc
 
 ################################################################################
 
diff --git a/OvmfPkg/CloudHv/CloudHvX64.fdf b/OvmfPkg/CloudHv/CloudHvX64.fdf
index 0d13d4066d27..62d048bce181 100644
--- a/OvmfPkg/CloudHv/CloudHvX64.fdf
+++ b/OvmfPkg/CloudHv/CloudHvX64.fdf
@@ -168,7 +168,7 @@ [FV.PEIFV]
 !endif
 INF  UefiCpuPkg/CpuMpPei/CpuMpPei.inf
 
-!include OvmfPkg/OvmfTpmPei.fdf.inc
+!include OvmfPkg/Include/Fdf/OvmfTpmPei.fdf.inc
 
 ################################################################################
 
diff --git a/OvmfPkg/OvmfTpmPei.fdf.inc b/OvmfPkg/Include/Fdf/OvmfTpmPei.fdf.inc
similarity index 100%
rename from OvmfPkg/OvmfTpmPei.fdf.inc
rename to OvmfPkg/Include/Fdf/OvmfTpmPei.fdf.inc
diff --git a/OvmfPkg/OvmfPkgIa32.fdf b/OvmfPkg/OvmfPkgIa32.fdf
index 15a25f1be5e5..63b9ef5d9674 100644
--- a/OvmfPkg/OvmfPkgIa32.fdf
+++ b/OvmfPkg/OvmfPkgIa32.fdf
@@ -172,7 +172,7 @@ [FV.PEIFV]
 !endif
 INF  UefiCpuPkg/CpuMpPei/CpuMpPei.inf
 
-!include OvmfPkg/OvmfTpmPei.fdf.inc
+!include OvmfPkg/Include/Fdf/OvmfTpmPei.fdf.inc
 
 ################################################################################
 
diff --git a/OvmfPkg/OvmfPkgIa32X64.fdf b/OvmfPkg/OvmfPkgIa32X64.fdf
index 138cbd382d4e..36eb28c34f89 100644
--- a/OvmfPkg/OvmfPkgIa32X64.fdf
+++ b/OvmfPkg/OvmfPkgIa32X64.fdf
@@ -172,7 +172,7 @@ [FV.PEIFV]
 !endif
 INF  UefiCpuPkg/CpuMpPei/CpuMpPei.inf
 
-!include OvmfPkg/OvmfTpmPei.fdf.inc
+!include OvmfPkg/Include/Fdf/OvmfTpmPei.fdf.inc
 
 ################################################################################
 
diff --git a/OvmfPkg/OvmfPkgX64.fdf b/OvmfPkg/OvmfPkgX64.fdf
index 4f24051ae994..d9ecc837295a 100644
--- a/OvmfPkg/OvmfPkgX64.fdf
+++ b/OvmfPkg/OvmfPkgX64.fdf
@@ -188,7 +188,7 @@ [FV.PEIFV]
 INF  UefiCpuPkg/CpuMpPei/CpuMpPei.inf
 INF  FILE_GUID = $(UP_CPU_PEI_GUID) UefiCpuPkg/CpuMpPei/CpuMpPei.inf
 
-!include OvmfPkg/OvmfTpmPei.fdf.inc
+!include OvmfPkg/Include/Fdf/OvmfTpmPei.fdf.inc
 
 ################################################################################
 
-- 
2.40.0.windows.1

[PATCH v2 3/7] OvmfPkg: move OvmfTpmDxe.fdf.inc to Include/Fdf

[Rebecca Cran]

From: Gerd Hoffmann <kraxel@redhat.com>

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Jiewen Yao <jiewen.yao@intel.com>
---
 OvmfPkg/AmdSev/AmdSevX64.fdf                 | 2 +-
 OvmfPkg/CloudHv/CloudHvX64.fdf               | 2 +-
 OvmfPkg/{ => Include/Fdf}/OvmfTpmDxe.fdf.inc | 0
 OvmfPkg/OvmfPkgIa32.fdf                      | 2 +-
 OvmfPkg/OvmfPkgIa32X64.fdf                   | 2 +-
 OvmfPkg/OvmfPkgX64.fdf                       | 2 +-
 6 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/OvmfPkg/AmdSev/AmdSevX64.fdf b/OvmfPkg/AmdSev/AmdSevX64.fdf
index 25e446013cd4..fec08468d3e0 100644
--- a/OvmfPkg/AmdSev/AmdSevX64.fdf
+++ b/OvmfPkg/AmdSev/AmdSevX64.fdf
@@ -316,7 +316,7 @@ [FV.DXEFV]
 #
 # TPM support
 #
-!include OvmfPkg/OvmfTpmDxe.fdf.inc
+!include OvmfPkg/Include/Fdf/OvmfTpmDxe.fdf.inc
 
 ################################################################################
 
diff --git a/OvmfPkg/CloudHv/CloudHvX64.fdf b/OvmfPkg/CloudHv/CloudHvX64.fdf
index 62d048bce181..72de7bcdad66 100644
--- a/OvmfPkg/CloudHv/CloudHvX64.fdf
+++ b/OvmfPkg/CloudHv/CloudHvX64.fdf
@@ -343,7 +343,7 @@ [FV.DXEFV]
 #
 # TPM support
 #
-!include OvmfPkg/OvmfTpmDxe.fdf.inc
+!include OvmfPkg/Include/Fdf/OvmfTpmDxe.fdf.inc
 
 ################################################################################
 
diff --git a/OvmfPkg/OvmfTpmDxe.fdf.inc b/OvmfPkg/Include/Fdf/OvmfTpmDxe.fdf.inc
similarity index 100%
rename from OvmfPkg/OvmfTpmDxe.fdf.inc
rename to OvmfPkg/Include/Fdf/OvmfTpmDxe.fdf.inc
diff --git a/OvmfPkg/OvmfPkgIa32.fdf b/OvmfPkg/OvmfPkgIa32.fdf
index 63b9ef5d9674..c9c938439759 100644
--- a/OvmfPkg/OvmfPkgIa32.fdf
+++ b/OvmfPkg/OvmfPkgIa32.fdf
@@ -372,7 +372,7 @@ [FV.DXEFV]
 #
 # TPM support
 #
-!include OvmfPkg/OvmfTpmDxe.fdf.inc
+!include OvmfPkg/Include/Fdf/OvmfTpmDxe.fdf.inc
 
 !if $(LOAD_X64_ON_IA32_ENABLE) == TRUE
 INF  OvmfPkg/CompatImageLoaderDxe/CompatImageLoaderDxe.inf
diff --git a/OvmfPkg/OvmfPkgIa32X64.fdf b/OvmfPkg/OvmfPkgIa32X64.fdf
index 36eb28c34f89..f52219e0c26d 100644
--- a/OvmfPkg/OvmfPkgIa32X64.fdf
+++ b/OvmfPkg/OvmfPkgIa32X64.fdf
@@ -379,7 +379,7 @@ [FV.DXEFV]
 #
 # TPM support
 #
-!include OvmfPkg/OvmfTpmDxe.fdf.inc
+!include OvmfPkg/Include/Fdf/OvmfTpmDxe.fdf.inc
 
 ################################################################################
 
diff --git a/OvmfPkg/OvmfPkgX64.fdf b/OvmfPkg/OvmfPkgX64.fdf
index d9ecc837295a..00c7f8849fb8 100644
--- a/OvmfPkg/OvmfPkgX64.fdf
+++ b/OvmfPkg/OvmfPkgX64.fdf
@@ -413,7 +413,7 @@ [FV.DXEFV]
 #
 # TPM support
 #
-!include OvmfPkg/OvmfTpmDxe.fdf.inc
+!include OvmfPkg/Include/Fdf/OvmfTpmDxe.fdf.inc
 
 ################################################################################
 
-- 
2.40.0.windows.1

[PATCH v2 4/7] BaseTools: Remove Python2/Python3 detection from toolset.bat

[Rebecca Cran]

Since Python3 is now required, we can remove the checks for PYTHON3_ENABLE
and PYTHON3 and simplify the code in toolsetup.bat. Also, remove the
leftover from when we supported freezing Python code.

While here, fix a couple of typos and improve error messages.

Signed-off-by: Rebecca Cran <rebecca@bsdio.com>
---
 BaseTools/toolsetup.bat | 64 +++++---------------
 1 file changed, 16 insertions(+), 48 deletions(-)

diff --git a/BaseTools/toolsetup.bat b/BaseTools/toolsetup.bat
index 25d13d559cd6..3d13e9fad286 100755
--- a/BaseTools/toolsetup.bat
+++ b/BaseTools/toolsetup.bat
@@ -305,18 +305,8 @@ goto check_build_environment
   )
 
 :defined_python
-if defined PYTHON_COMMAND if not defined PYTHON3_ENABLE (
-  goto check_python_available
-)
-if defined PYTHON3_ENABLE (
-  if "%PYTHON3_ENABLE%" EQU "TRUE" (
-    set PYTHON_COMMAND=py -3
-    goto check_python_available
-  ) else (
-    goto check_python2
-  )
-)
-if not defined PYTHON_COMMAND if not defined PYTHON3_ENABLE (
+
+if not defined PYTHON_COMMAND (
   set PYTHON_COMMAND=py -3
   py -3 %BASE_TOOLS_PATH%\Tests\PythonTest.py >PythonCheck.txt 2>&1
   setlocal enabledelayedexpansion
@@ -328,56 +318,40 @@ if not defined PYTHON_COMMAND if not defined PYTHON3_ENABLE (
       set PYTHON_COMMAND=
       echo.
       echo !!! ERROR !!! Binary python tools are missing.
-      echo PYTHON_COMMAND, PYTHON3_ENABLE or PYTHON_HOME
-      echo Environment variable is not set successfully.
-      echo They is required to build or execute the python tools.
+      echo PYTHON_COMMAND or PYTHON_HOME
+      echo Environment variable is not set correctly.
+      echo They are required to build or execute the python tools.
       echo.
       goto end
-    ) else (
-      goto check_python2
     )
-  ) else (
-    goto check_freezer_path
   )
 )
 
-:check_python2
 endlocal
+
 if defined PYTHON_HOME (
   if EXIST "%PYTHON_HOME%" (
     set PYTHON_COMMAND=%PYTHON_HOME%\python.exe
-    goto check_python_available
+  ) else (
+    echo .
+    echo !!! ERROR !!!  PYTHON_HOME="%PYTHON_HOME%" does not exist.
+    echo .
+    goto end
   )
 )
-if defined PYTHONHOME (
-  if EXIST "%PYTHONHOME%" (
-    set PYTHON_HOME=%PYTHONHOME%
-    set PYTHON_COMMAND=%PYTHON_HOME%\python.exe
-    goto check_python_available
-  )
-)
-echo.
-echo !!! ERROR !!!  PYTHON_HOME is not defined or The value of this variable does not exist
-echo.
-goto end
-:check_python_available
+
 %PYTHON_COMMAND% %BASE_TOOLS_PATH%\Tests\PythonTest.py >PythonCheck.txt 2>&1
   setlocal enabledelayedexpansion
   set /p PythonCheck=<"PythonCheck.txt"
   del PythonCheck.txt
   if "!PythonCheck!" NEQ "TRUE" (
     echo.
-    echo ! ERROR !  "%PYTHON_COMMAND%" is not installed or added to environment variables
+    echo ! ERROR !  PYTHON_COMMAND="%PYTHON_COMMAND%" is not installed or added to environment variables
     echo.
     goto end
-  ) else (
-    goto check_freezer_path
-  )
+)
 
-
-
-:check_freezer_path
-  endlocal
+endlocal
 
   %PYTHON_COMMAND% -c "import edk2basetools" >NUL 2>NUL
   if %ERRORLEVEL% EQU 0 (
@@ -404,13 +378,7 @@ goto end
 
 :print_python_info
   echo                PATH = %PATH%
-  if defined PYTHON3_ENABLE if "%PYTHON3_ENABLE%" EQU "TRUE" (
-    echo      PYTHON3_ENABLE = %PYTHON3_ENABLE%
-    echo             PYTHON3 = %PYTHON_COMMAND%
-  ) else (
-    echo      PYTHON3_ENABLE = FALSE
-    echo      PYTHON_COMMAND = %PYTHON_COMMAND%
-  )
+  echo      PYTHON_COMMAND = %PYTHON_COMMAND%
   echo          PYTHONPATH = %PYTHONPATH%
   echo.
 
-- 
2.40.0.windows.1

[PATCH v2 5/7] BaseTools: use threading.current_thread in NmakeSubdirs.py

[Rebecca Cran]

threading.currentThread is a deprecated alias for
threading.current_thread, and causes a warning to be displayed when it's
called. Update NmakeSubdirs.py to use the latter method instead.

Signed-off-by: Rebecca Cran <rebecca@bsdio.com>
---
 BaseTools/Source/C/Makefiles/NmakeSubdirs.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/BaseTools/Source/C/Makefiles/NmakeSubdirs.py b/BaseTools/Source/C/Makefiles/NmakeSubdirs.py
index 1f4a45004f4b..7860c040afa0 100644
--- a/BaseTools/Source/C/Makefiles/NmakeSubdirs.py
+++ b/BaseTools/Source/C/Makefiles/NmakeSubdirs.py
@@ -132,7 +132,7 @@ class ThreadControl(object):
                 break
 
         self.runningLock.acquire(True)
-        self.running.remove(threading.currentThread())
+        self.running.remove(threading.current_thread())
         self.runningLock.release()
 
 def Run():
-- 
2.40.0.windows.1

[PATCH v2 6/7] edksetup.bat: if toolsetup.bat fails, just exit

[Rebecca Cran]

If toolsetup.bat fails (i.e. exits with a non-zero %ERRORLEVEL%), don't
try and carry on but just quit.

Signed-off-by: Rebecca Cran <rebecca@bsdio.com>
---
 edksetup.bat | 1 +
 1 file changed, 1 insertion(+)

diff --git a/edksetup.bat b/edksetup.bat
index 2fdf130e00e2..71ceefb32742 100755
--- a/edksetup.bat
+++ b/edksetup.bat
@@ -86,6 +86,7 @@ if exist %EDK_TOOLS_PATH%\Source set BASE_TOOLS_PATH=%EDK_TOOLS_PATH%
 :checkBaseTools
 IF NOT EXIST "%EDK_TOOLS_PATH%\toolsetup.bat" goto BadBaseTools
 call %EDK_TOOLS_PATH%\toolsetup.bat %*
+if %ERRORLEVEL% NEQ 0 goto end
 if /I "%1"=="Reconfig" shift
 goto check_NASM
 goto check_cygwin
-- 
2.40.0.windows.1

[PATCH v2 7/7] BaseTools: Update toolsetup.bat and Tests/PythonTest.py to check ver

[Rebecca Cran]

Update toolsetup.bat and Tests/PythonTest.py to check if we're running a
version of Python that's compatible with BaseTools and the Pip
BaseTools.

BaseTools uses syntax from Python 3.6 or newer, so set that as the minimum
version EDK2 requires.

Signed-off-by: Rebecca Cran <rebecca@bsdio.com>
---
 BaseTools/Tests/PythonTest.py | 22 ++++++-
 BaseTools/toolsetup.bat       | 61 +++++++++++---------
 2 files changed, 54 insertions(+), 29 deletions(-)

diff --git a/BaseTools/Tests/PythonTest.py b/BaseTools/Tests/PythonTest.py
index ec44c7947086..b87c78570eae 100644
--- a/BaseTools/Tests/PythonTest.py
+++ b/BaseTools/Tests/PythonTest.py
@@ -1,9 +1,27 @@
 ## @file
-# Test whether PYTHON_COMMAND is available
+# Test whether PYTHON_COMMAND is available and the
+# minimum Python version is installed.
 #
 # Copyright (c) 2013 - 2018, Intel Corporation. All rights reserved.<BR>
 # SPDX-License-Identifier: BSD-2-Clause-Patent
 #
 
+import sys
+
 if __name__ == '__main__':
-    print('TRUE')
+    # Check if the major and minor versions required were specified.
+    if len(sys.argv) >= 3:
+        req_major_version = int(sys.argv[1])
+        req_minor_version = int(sys.argv[2])
+    else:
+        # If the minimum version wasn't specified on the command line,
+        # default to 3.6 because BaseTools uses syntax from PEP 526
+        # (https://peps.python.org/pep-0526/)
+        req_major_version = 3
+        req_minor_version = 6
+
+    if sys.version_info.major == req_major_version and \
+       sys.version_info.minor >= req_minor_version:
+        sys.exit(0)
+    else:
+        sys.exit(1)
diff --git a/BaseTools/toolsetup.bat b/BaseTools/toolsetup.bat
index 3d13e9fad286..dc6288effd7d 100755
--- a/BaseTools/toolsetup.bat
+++ b/BaseTools/toolsetup.bat
@@ -12,6 +12,8 @@
 @echo off
 pushd .
 set SCRIPT_ERROR=0
+set PYTHON_VER_MAJOR=3
+set PYTHON_VER_MINOR=6
 
 @REM ##############################################################
 @REM # You should not have to modify anything below this line
@@ -304,17 +306,19 @@ goto check_build_environment
      )
   )
 
-:defined_python
+@REM Check Python environment
 
 if not defined PYTHON_COMMAND (
   set PYTHON_COMMAND=py -3
-  py -3 %BASE_TOOLS_PATH%\Tests\PythonTest.py >PythonCheck.txt 2>&1
-  setlocal enabledelayedexpansion
-  set /p PythonCheck=<"PythonCheck.txt"
-  del PythonCheck.txt
-  if "!PythonCheck!" NEQ "TRUE" (
+  py -3 %BASE_TOOLS_PATH%\Tests\PythonTest.py %PYTHON_VER_MAJOR% %PYTHON_VER_MINOR% >NUL 2>NUL
+  if %ERRORLEVEL% EQU 1 (
+    echo.
+    echo !!! ERROR !!! Python %PYTHON_VER_MAJOR%.%PYTHON_VER_MINOR% or newer is required.
+    echo.
+    goto end
+  )
+  if %ERRORLEVEL% NEQ 0 (
     if not defined PYTHON_HOME if not defined PYTHONHOME (
-      endlocal
       set PYTHON_COMMAND=
       echo.
       echo !!! ERROR !!! Binary python tools are missing.
@@ -327,8 +331,6 @@ if not defined PYTHON_COMMAND (
   )
 )
 
-endlocal
-
 if defined PYTHON_HOME (
   if EXIST "%PYTHON_HOME%" (
     set PYTHON_COMMAND=%PYTHON_HOME%\python.exe
@@ -340,27 +342,30 @@ if defined PYTHON_HOME (
   )
 )
 
-%PYTHON_COMMAND% %BASE_TOOLS_PATH%\Tests\PythonTest.py >PythonCheck.txt 2>&1
-  setlocal enabledelayedexpansion
-  set /p PythonCheck=<"PythonCheck.txt"
-  del PythonCheck.txt
-  if "!PythonCheck!" NEQ "TRUE" (
-    echo.
-    echo ! ERROR !  PYTHON_COMMAND="%PYTHON_COMMAND%" is not installed or added to environment variables
-    echo.
-    goto end
+%PYTHON_COMMAND% %BASE_TOOLS_PATH%\Tests\PythonTest.py %PYTHON_VER_MAJOR% %PYTHON_VER_MINOR% >NUL 2>NUL
+if %ERRORLEVEL% EQU 1 (
+  echo.
+  echo !!! ERROR !!! Python %PYTHON_VER_MAJOR%.%PYTHON_VER_MINOR% or newer is required.
+  echo.
+  goto end
+)
+if %ERRORLEVEL% NEQ 0 (
+  echo.
+  echo !!! ERROR !!!  PYTHON_COMMAND="%PYTHON_COMMAND%" does not exist or is not a Python interpreter.
+  echo.
+  goto end
 )
 
 endlocal
 
-  %PYTHON_COMMAND% -c "import edk2basetools" >NUL 2>NUL
-  if %ERRORLEVEL% EQU 0 (
-    goto use_pip_basetools
-  ) else (
-    REM reset ERRORLEVEL
-    type nul>nul
-    goto use_builtin_basetools
-  )
+%PYTHON_COMMAND% -c "import edk2basetools" >NUL 2>NUL
+if %ERRORLEVEL% EQU 0 (
+  goto use_pip_basetools
+) else (
+  REM reset ERRORLEVEL
+  type nul>nul
+  goto use_builtin_basetools
+)
 
 :use_builtin_basetools
   @echo Using EDK2 in-source Basetools
@@ -444,5 +449,7 @@ set VS2019=
 set VS2017=
 set VS2015=
 set VSTool=
+set PYTHON_VER_MAJOR=
+set PYTHON_VER_MINOR=
+set SCRIPT_ERROR=
 popd
-
-- 
2.40.0.windows.1

Re: [edk2-devel] [PATCH v2 0/7] edksetup.bat, BaseTools: Improve Windows environment setup and BaseTools C compilation

[Pedro Falcato]

On Sat, May 6, 2023 at 6:24 PM Rebecca Cran <rebecca@bsdio.com> wrote:
>
> There are remnants of Python 2 support in BaseTools/toolsetup.bat that it's
> probably time to remove since we only support Python 3.6 and newer these days.
> So, remove the variables that enable Python3 support and simplify the batch
> script. I've also seen errors where after running edksetup.bat the build
> command isn't available because PYTHONPATH wasn't being set, so fix that
> when the Pip BaseTools are being used.
>
> At the same time, let's add a check that we meet the minimum version
> requirement so we don't end up failing with an obscure error.
>
> Building BaseTools causes a warning about threading.currentThread being
> deprecated, so update code in NmakeSubdirs.py to switch to
> threading.current_thread.
>
> There needs to be further work, because if PYTHON_COMMAND isn't specified then
> it defaults to "py -3", where py is C:\Windows\py.exe, which doesn't work if
> you're using a virtualenv since it installs python.exe and pythonw.exe in
> venv\Scripts. toolsetup.bat therefore fails to detect the Pip BaseTools and
> uses the in-source Basetools.
>
> GitHub PR: https://github.com/tianocore/edk2/pull/4302
> GitHub branch: https://github.com/bcran/edk2/tree/py3
>
> Changes between v1 and v2
> =========================
>
> - Require Python 3.6 or newer: 3.6 was when PEP 526 was added, which we use.
> - Fix Tests/RunTests.py on Windows.
>
> Dongyan Qian (1):
>   BaseSynchronizationLib: Fix LoongArch64 synchronization functions
>
> Gerd Hoffmann (2):
>   OvmfPkg: move OvmfTpmPei.fdf.inc to Include/Fdf
>   OvmfPkg: move OvmfTpmDxe.fdf.inc to Include/Fdf

Why did you pick these up? Accident?

-- 
Pedro

Re: [edk2-devel] [PATCH v2 0/7] edksetup.bat, BaseTools: Improve Windows environment setup and BaseTools C compilation

[Rebecca Cran]

On 5/6/23 13:16, Pedro Falcato wrote:
> Why did you pick these up? Accident?

Ugh sorry, I'm rushing to try and get as many patches submitted as 
possible before the freeze and included too many patches.

I've just sent out a v3 with just the 4 commits I meant to include.


-- 

Rebecca Cran