From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pl1-f171.google.com (mail-pl1-f171.google.com [209.85.214.171]) by mx.groups.io with SMTP id smtpd.web10.3824.1612831862065700277 for ; Mon, 08 Feb 2021 16:51:02 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20161025 header.b=lbCmedKC; spf=pass (domain: gmail.com, ip: 209.85.214.171, mailfrom: matthewfcarlson@gmail.com) Received: by mail-pl1-f171.google.com with SMTP id u15so8805222plf.1 for ; Mon, 08 Feb 2021 16:51:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=7yx1vIcp1cmWU2xzey58zDI/sWXvj8UqvJVYmWwL2Ig=; b=lbCmedKCplmfRU3dcDac86s7FHNXCF/zPziNJiXP5e1xsjVTYkLUImmLVvb0zcRRpz TCuRmGmNhZFYtLikObRSwJpuIzi8XQc7LwBU6wwT2MJ71tr6kyqGxZpBYPBZKACqSQnE ohcF3Fp0orfkPf+hIxo8ZmPRje8eHm51legaRkeDNrmp4Tvq/rqK0QRKMLfxBnzKSc8R m82MThVwW6EPQmd4znNc4eWoe+NKp7hA5IGaoxMMznDjftfyNF5SXcFHRQBCZQR0aqV8 2kb0nkFojxrxB9A5KQm2IpJgezoCYK6xztrenaGNEDwJKdsqfLROK4RB4xIl21AlI/kq qdaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=7yx1vIcp1cmWU2xzey58zDI/sWXvj8UqvJVYmWwL2Ig=; b=YEAavNDS+JB8PGhjNmZc7zAs6uMxt/9H/QpctSoVXz6Nl9KXzcGEIyBR0pQSyXl07t Czwkr4cxLAWo8zUI2uLTwoG0HlgtgT7RJpu+NjJkCXiuWi7UZEta17kFN+YpKIksFQ1S QloMCr8YKmeI5VcSQTomvuldCYtGl8dWNxXmJBiOHMeApgX5uMHczjdJ9fF4VVMftzh9 LmflOZ2cICucMLvWUCpWa5qtS6R4DNBfaQZvibZjFFso4mCRYKdExnCawmKDrWOCQfXT JZZF1ttmIUganzQi69fcUT6knYaCQTfi1kd7BtLyAKGnwBqjug5zVGZFhpVb2vH6vuiM URGQ== X-Gm-Message-State: AOAM532r4r/wuKhqef8BzbZy6Lt9IBlPplUOMkw3z0DgRrbHpGoptCI4 KAjvaN0gaCUEwMgV54F6S+fS0P9GDgtArA== X-Google-Smtp-Source: ABdhPJz4J95VwgyHIyPZEqLIL3nmjiOQbHVBH4i61DH5RXGR5D0yQ3GqRCeJi11XtAhu3cHY4tOA4A== X-Received: by 2002:a17:902:c3c2:b029:e1:74f5:6a65 with SMTP id j2-20020a170902c3c2b02900e174f56a65mr18008438plj.46.1612831861209; Mon, 08 Feb 2021 16:51:01 -0800 (PST) Return-Path: Received: from localhost.localdomain ([50.34.38.126]) by smtp.gmail.com with ESMTPSA id d13sm453118pjz.3.2021.02.08.16.51.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Feb 2021 16:51:00 -0800 (PST) From: "Matthew Carlson" To: devel@edk2.groups.io Cc: Bob Feng , Liming Gao , Yuwei Chen , Sean Brogan , Michael Kinney Subject: [PATCH v5 0/1] Use Pip version of BaseTools Date: Mon, 8 Feb 2021 16:50:32 -0800 Message-Id: <20210209005033.596-1-matthewfcarlson@gmail.com> X-Mailer: git-send-email 2.29.0.vfs.0.0 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: Matthew Carlson Hello all, This patch adds the capability to leverage the pip based BaseTools. The BaseTools master is current with EDK2 master (as of 3pm Nov 2, 2020). While I could rehash a lot of what has gone out on the mailing list, I think it might be better just to link the discussion. In a nutshell, setup will detect if edk2-basetools is installed, and if it is, use the other verison of BinWrappers (BinPipWrappers). This should provide an easy and convenient way to transition over. By default, CI uses the pip version. The pipeline is setup. Every commit that gets pushed to master on basetools get published to pypi and tagged. This should offer bisectability and easier debugging. Additionally, the pip requirements file tracks a specific version. We'd like to set a date for when PR's no longer go into EDK2 for BaseTools and instead go exclusively into the new basetools repo. V5 Changes: Add Split.bat and removed BrotliCompress.bat V4 Changes: converted files to DOX line ending, added executable chmod to unix binwrappers, and removed unnecessary binwrappers. V3 Changes: converted some files to unix line ending Ref: https://edk2.groups.io/g/devel/topic/76572200#64993 Ref: https://github.com/tianocore/edk2-basetools Ref: https://github.com/tianocore/edk2/pull/1072 Cc: Bob Feng Cc: Liming Gao Cc: Yuwei Chen Cc: Sean Brogan Cc: Michael Kinney Matthew Carlson (1): BaseTools: Use pip module if available, CI uses it by default .pytool/CISettings.py | 51 +++++++++++++++----- BaseTools/BinPipWrappers/PosixLike/AmlToC | 14 ++++++ BaseTools/BinPipWrappers/PosixLike/BPDG | 12 +++++ BaseTools/BinPipWrappers/PosixLike/BrotliCompress | 34 +++++++++++++ BaseTools/BinPipWrappers/PosixLike/DevicePath | 29 +++++++++++ BaseTools/BinPipWrappers/PosixLike/Ecc | 13 +++++ BaseTools/BinPipWrappers/PosixLike/EfiRom | 29 +++++++++++ BaseTools/BinPipWrappers/PosixLike/GenCrc32 | 29 +++++++++++ BaseTools/BinPipWrappers/PosixLike/GenDepex | 12 +++++ BaseTools/BinPipWrappers/PosixLike/GenFds | 12 +++++ BaseTools/BinPipWrappers/PosixLike/GenFfs | 29 +++++++++++ BaseTools/BinPipWrappers/PosixLike/GenFv | 29 +++++++++++ BaseTools/BinPipWrappers/PosixLike/GenFw | 29 +++++++++++ BaseTools/BinPipWrappers/PosixLike/GenPatchPcdTable | 12 +++++ BaseTools/BinPipWrappers/PosixLike/GenSec | 29 +++++++++++ BaseTools/BinPipWrappers/PosixLike/GenerateCapsule | 12 +++++ BaseTools/BinPipWrappers/PosixLike/LzmaCompress | 29 +++++++++++ BaseTools/BinPipWrappers/PosixLike/LzmaF86Compress | 19 ++++++++ BaseTools/BinPipWrappers/PosixLike/PatchPcdValue | 12 +++++ BaseTools/BinPipWrappers/PosixLike/Pkcs7Sign | 12 +++++ BaseTools/BinPipWrappers/PosixLike/Rsa2048Sha256GenerateKeys | 12 +++++ BaseTools/BinPipWrappers/PosixLike/Rsa2048Sha256Sign | 12 +++++ BaseTools/BinPipWrappers/PosixLike/Split | 29 +++++++++++ BaseTools/BinPipWrappers/PosixLike/TargetTool | 12 +++++ BaseTools/BinPipWrappers/PosixLike/TianoCompress | 29 +++++++++++ BaseTools/BinPipWrappers/PosixLike/Trim | 13 +++++ BaseTools/BinPipWrappers/PosixLike/UPT | 12 +++++ BaseTools/BinPipWrappers/PosixLike/VfrCompile | 29 +++++++++++ BaseTools/BinPipWrappers/PosixLike/VolInfo | 29 +++++++++++ BaseTools/BinPipWrappers/PosixLike/build | 12 +++++ BaseTools/{BinWrappers => BinPipWrappers}/PosixLike/posix_path_env.yaml | 21 ++++---- BaseTools/BinPipWrappers/WindowsLike/AmlToC.bat | 3 ++ BaseTools/BinPipWrappers/WindowsLike/BPDG.bat | 3 ++ BaseTools/BinPipWrappers/WindowsLike/Ecc.bat | 3 ++ BaseTools/BinPipWrappers/WindowsLike/GenDepex.bat | 3 ++ BaseTools/BinPipWrappers/WindowsLike/GenFds.bat | 3 ++ BaseTools/BinPipWrappers/WindowsLike/GenPatchPcdTable.bat | 3 ++ BaseTools/BinPipWrappers/WindowsLike/GenerateCapsule.bat | 1 + BaseTools/BinPipWrappers/WindowsLike/PatchPcdValue.bat | 3 ++ BaseTools/BinPipWrappers/WindowsLike/Pkcs7Sign.bat | 3 ++ BaseTools/BinPipWrappers/WindowsLike/Rsa2048Sha256GenerateKeys.bat | 1 + BaseTools/BinPipWrappers/WindowsLike/Rsa2048Sha256Sign.bat | 3 ++ BaseTools/BinPipWrappers/WindowsLike/Split.bat | 3 ++ BaseTools/BinPipWrappers/WindowsLike/TargetTool.bat | 3 ++ BaseTools/BinPipWrappers/WindowsLike/Trim.bat | 3 ++ BaseTools/BinPipWrappers/WindowsLike/UPT.bat | 3 ++ BaseTools/BinPipWrappers/WindowsLike/build.bat | 3 ++ BaseTools/{BinWrappers => BinPipWrappers}/WindowsLike/win_build_tools_path_env.yaml | 3 +- BaseTools/BinWrappers/PosixLike/posix_path_env.yaml | 1 + BaseTools/BinWrappers/WindowsLike/win_build_tools_path_env.yaml | 1 + BaseTools/BuildEnv | 14 +++++- BaseTools/Scripts/PatchCheck.py | 1 + BaseTools/Source/Python/README.md | 29 +++++++++++ BaseTools/toolsetup.bat | 19 ++++++++ pip-requirements.txt | 1 + 55 files changed, 745 insertions(+), 25 deletions(-) create mode 100755 BaseTools/BinPipWrappers/PosixLike/AmlToC create mode 100755 BaseTools/BinPipWrappers/PosixLike/BPDG create mode 100755 BaseTools/BinPipWrappers/PosixLike/BrotliCompress create mode 100755 BaseTools/BinPipWrappers/PosixLike/DevicePath create mode 100755 BaseTools/BinPipWrappers/PosixLike/Ecc create mode 100755 BaseTools/BinPipWrappers/PosixLike/EfiRom create mode 100755 BaseTools/BinPipWrappers/PosixLike/GenCrc32 create mode 100755 BaseTools/BinPipWrappers/PosixLike/GenDepex create mode 100755 BaseTools/BinPipWrappers/PosixLike/GenFds create mode 100755 BaseTools/BinPipWrappers/PosixLike/GenFfs create mode 100755 BaseTools/BinPipWrappers/PosixLike/GenFv create mode 100755 BaseTools/BinPipWrappers/PosixLike/GenFw create mode 100755 BaseTools/BinPipWrappers/PosixLike/GenPatchPcdTable create mode 100755 BaseTools/BinPipWrappers/PosixLike/GenSec create mode 100755 BaseTools/BinPipWrappers/PosixLike/GenerateCapsule create mode 100755 BaseTools/BinPipWrappers/PosixLike/LzmaCompress create mode 100755 BaseTools/BinPipWrappers/PosixLike/LzmaF86Compress create mode 100755 BaseTools/BinPipWrappers/PosixLike/PatchPcdValue create mode 100755 BaseTools/BinPipWrappers/PosixLike/Pkcs7Sign create mode 100755 BaseTools/BinPipWrappers/PosixLike/Rsa2048Sha256GenerateKeys create mode 100755 BaseTools/BinPipWrappers/PosixLike/Rsa2048Sha256Sign create mode 100755 BaseTools/BinPipWrappers/PosixLike/Split create mode 100755 BaseTools/BinPipWrappers/PosixLike/TargetTool create mode 100755 BaseTools/BinPipWrappers/PosixLike/TianoCompress create mode 100755 BaseTools/BinPipWrappers/PosixLike/Trim create mode 100755 BaseTools/BinPipWrappers/PosixLike/UPT create mode 100755 BaseTools/BinPipWrappers/PosixLike/VfrCompile create mode 100755 BaseTools/BinPipWrappers/PosixLike/VolInfo create mode 100755 BaseTools/BinPipWrappers/PosixLike/build copy BaseTools/{BinWrappers => BinPipWrappers}/PosixLike/posix_path_env.yaml (74%) create mode 100644 BaseTools/BinPipWrappers/WindowsLike/AmlToC.bat create mode 100644 BaseTools/BinPipWrappers/WindowsLike/BPDG.bat create mode 100644 BaseTools/BinPipWrappers/WindowsLike/Ecc.bat create mode 100644 BaseTools/BinPipWrappers/WindowsLike/GenDepex.bat create mode 100644 BaseTools/BinPipWrappers/WindowsLike/GenFds.bat create mode 100644 BaseTools/BinPipWrappers/WindowsLike/GenPatchPcdTable.bat create mode 100644 BaseTools/BinPipWrappers/WindowsLike/GenerateCapsule.bat create mode 100644 BaseTools/BinPipWrappers/WindowsLike/PatchPcdValue.bat create mode 100644 BaseTools/BinPipWrappers/WindowsLike/Pkcs7Sign.bat create mode 100644 BaseTools/BinPipWrappers/WindowsLike/Rsa2048Sha256GenerateKeys.bat create mode 100644 BaseTools/BinPipWrappers/WindowsLike/Rsa2048Sha256Sign.bat create mode 100644 BaseTools/BinPipWrappers/WindowsLike/Split.bat create mode 100644 BaseTools/BinPipWrappers/WindowsLike/TargetTool.bat create mode 100644 BaseTools/BinPipWrappers/WindowsLike/Trim.bat create mode 100644 BaseTools/BinPipWrappers/WindowsLike/UPT.bat create mode 100644 BaseTools/BinPipWrappers/WindowsLike/build.bat copy BaseTools/{BinWrappers => BinPipWrappers}/WindowsLike/win_build_tools_path_env.yaml (70%) create mode 100644 BaseTools/Source/Python/README.md -- 2.29.0.vfs.0.0