* [PATCH v3 1/7] .azurepipelines: Add Platform CI template
[not found] <20200424213108.19888-1-michael.kubacki@outlook.com>
@ 2020-04-24 21:31 ` Michael Kubacki
2020-04-24 21:31 ` [PATCH v3 2/7] ArmVirtPkg: Add Platform CI and configuration for Core CI Michael Kubacki
` (5 subsequent siblings)
6 siblings, 0 replies; 13+ messages in thread
From: Michael Kubacki @ 2020-04-24 21:31 UTC (permalink / raw)
To: devel; +Cc: Sean Brogan, Bret Barkelew, Michael D Kinney, Liming Gao
From: Sean Brogan <sean.brogan@microsoft.com>
REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2570
Add template for Platform CI steps for a Pytools based build.
Add README to describe the template and how to use it.
Add helpful information for working with azurepipelines, templates, and
lessons learned.
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Sean Brogan <sean.brogan@microsoft.com>
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
Reviewed-by: Shenglei Zhang <shenglei.zhang@intel.com>
Reviewed-by: Bret Barkelew <bret.barkelew@microsoft.com>
---
.azurepipelines/ReadMe.md | 50 ++++++++
.azurepipelines/templates/ReadMe.md | 59 +++++++++
.azurepipelines/templates/platform-build-run-steps.yml | 134 ++++++++++++++++++++
3 files changed, 243 insertions(+)
diff --git a/.azurepipelines/ReadMe.md b/.azurepipelines/ReadMe.md
new file mode 100644
index 000000000000..cf57d282c197
--- /dev/null
+++ b/.azurepipelines/ReadMe.md
@@ -0,0 +1,50 @@
+# Azure DevOps Pipelines
+
+These yml files are used to provide CI builds using the Azure DevOps Pipeline Service.
+Most of the CI leverages edk2-pytools to support cross platform building and execution.
+
+## Core CI
+
+Focused on building and testing all packages in Edk2 without an actual target platform.
+
+See `.pytools/ReadMe.py` for more details
+
+## Platform CI
+
+Focused on building a single target platform and confirming functionality on that platform.
+
+## Conventions
+
+* Files extension should be *.yml. *.yaml is also supported but in Edk2 we use those for our package configuration.
+* Platform CI files should be in the `<PlatformPkg>/.azurepipelines` folder.
+* Core CI files are in the root folder.
+* Shared templates are in the `templates` folder.
+* Top level CI files should be named `<host os>-<tool_chain_tag>.yml`
+
+## Links
+
+* Basic Azure Landing Site - https://docs.microsoft.com/en-us/azure/devops/pipelines/?view=azure-devops
+* Pipeline jobs - https://docs.microsoft.com/en-us/azure/devops/pipelines/process/phases?view=azure-devops&tabs=yaml
+* Pipeline yml scheme - https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=schema%2Cparameter-schema
+* Pipeline expression - https://docs.microsoft.com/en-us/azure/devops/pipelines/process/expressions?view=azure-devops
+* PyTools - https://github.com/tianocore/edk2-pytool-extensions and https://github.com/tianocore/edk2-pytool-library
+
+## Lessons Learned
+
+### Templates and parameters
+
+They are great but evil. If they are used as part of determining the steps of a build they must resolve before the build starts. They can not use variables set in a yml or determined as part of a matrix. If they are used in a step then they can be bound late.
+
+### File matching patterns
+
+On Linux this can hang if there are too many files in the search list.
+
+### Templates and file splitting
+
+Suggestion is to do one big yaml file that does what you want for one of your targets. Then do the second one and find the deltas. From that you can start to figure out the right split of files, steps, jobs.
+
+### Conditional steps
+
+If you want the step to show up in the log but not run, use a step conditional. This is great when a platform doesn't currently support a feature but you want the builders to know that the features exists and maybe someday it will.
+
+If you want the step to not show up use a template step conditional wrapper. Beware this will be evaluated early (at build start). This can hide things not needed on a given OS for example.
diff --git a/.azurepipelines/templates/ReadMe.md b/.azurepipelines/templates/ReadMe.md
new file mode 100644
index 000000000000..fa433e3ef597
--- /dev/null
+++ b/.azurepipelines/templates/ReadMe.md
@@ -0,0 +1,59 @@
+# CI Templates
+
+This folder contains azure pipeline yml templates for "Core" and "Platform" Continuous Integration and PR validation.
+
+## Common CI templates
+
+### basetools-build-steps.yml
+
+This template compiles the Edk2 basetools from source. The steps in this template are
+conditional and will only run if variable `pkg_count` is greater than 0.
+
+It also has two conditional steps only used when the toolchain contains GCC. These two steps
+use `apt` to update the system packages and add those necessary for Edk2 builds.
+
+## Core CI templates
+
+### pr-gate-build-job.yml
+
+This templates contains the jobs and most importantly the matrix of which packages and
+targets to run for Core CI.
+
+### pr-gate-steps.yml
+
+This template is the main Core CI template. It controls all the steps run and is responsible for most functionality of the Core CI process. This template sets
+the `pkg_count` variable using the `stuart_pr_eval` tool when the
+build type is "pull request"
+
+### spell-check-prereq-steps.yml
+
+This template installs the node based tools used by the spell checker plugin. The steps
+in this template are conditional and will only run if variable `pkg_count` is greater than 0.
+
+## Platform CI templates
+
+### platform-build-run-steps.yml
+
+This template makes heavy use of pytools to build and run a platform in the Edk2 repo
+
+Also uses basetools-build-steps.yml to compile basetools
+
+#### Special Notes
+
+* For a build type of pull request it will conditionally build if the patches change files that impact the platform.
+ * uses `stuart_pr_eval` to determine impact
+* For manual builds or CI builds it will always build the platform
+* It compiles basetools from source
+* Will use `stuart_build --FlashOnly` to attempt to run the built image if the `Run` parameter is set.
+* See the parameters block for expected configuration options
+* Parameter `extra_install_step` allows the caller to insert extra steps. This is useful if additional dependencies, tools, or other things need to be installed. Here is an example of installing qemu on Windows.
+
+ ``` yaml
+ steps:
+ - template: ../../.azurepipelines/templates/build-run-steps.yml
+ parameters:
+ extra_install_step:
+ - powershell: choco install qemu; Write-Host "##vso[task.prependpath]c:\Program Files\qemu"
+ displayName: Install QEMU and Set QEMU on path # friendly name displayed in the UI
+ condition: and(gt(variables.pkg_count, 0), succeeded())
+ ```
diff --git a/.azurepipelines/templates/platform-build-run-steps.yml b/.azurepipelines/templates/platform-build-run-steps.yml
new file mode 100644
index 000000000000..ebf674bfc7d8
--- /dev/null
+++ b/.azurepipelines/templates/platform-build-run-steps.yml
@@ -0,0 +1,134 @@
+
+## @file
+# File steps.yml
+#
+# template file containing the steps to build
+#
+# Copyright (c) Microsoft Corporation.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+
+parameters:
+- name: tool_chain_tag
+ type: string
+ default: ''
+- name: build_pkg
+ type: string
+ default: ''
+- name: build_target
+ type: string
+ default: ''
+- name: build_arch
+ type: string
+ default: ''
+- name: build_file
+ type: string
+ default: ''
+- name: build_flags
+ type: string
+ default: ''
+- name: run_flags
+ type: string
+ default: ''
+
+- name: extra_install_step
+ type: stepList
+ default: []
+
+steps:
+- checkout: self
+ clean: true
+ fetchDepth: 1
+
+- task: UsePythonVersion@0
+ inputs:
+ versionSpec: "3.8.x"
+ architecture: "x64"
+
+- script: pip install -r pip-requirements.txt --upgrade
+ displayName: 'Install/Upgrade pip modules'
+
+# Set default
+- bash: echo "##vso[task.setvariable variable=pkg_count]${{ 1 }}"
+
+# trim the package list if this is a PR
+- task: CmdLine@1
+ displayName: Check if ${{ parameters.build_pkg }} need testing
+ inputs:
+ filename: stuart_pr_eval
+ arguments: -c ${{ parameters.build_file }} -t ${{ parameters.build_target}} -a ${{ parameters.build_arch}} --pr-target origin/$(System.PullRequest.targetBranch) --output-count-format-string "##vso[task.setvariable variable=pkg_count;isOutpout=true]{pkgcount}"
+ condition: eq(variables['Build.Reason'], 'PullRequest')
+
+ # Setup repo
+- task: CmdLine@1
+ displayName: Setup
+ inputs:
+ filename: stuart_setup
+ arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}} -t ${{ parameters.build_target}} -a ${{ parameters.build_arch}} ${{ parameters.build_flags}}
+ condition: and(gt(variables.pkg_count, 0), succeeded())
+
+# Stuart Update
+- task: CmdLine@1
+ displayName: Update
+ inputs:
+ filename: stuart_update
+ arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}} -t ${{ parameters.build_target}} -a ${{ parameters.build_arch}} ${{ parameters.build_flags}}
+ condition: and(gt(variables.pkg_count, 0), succeeded())
+
+# build basetools
+# do this after setup and update so that code base dependencies
+# are all resolved.
+- template: basetools-build-steps.yml
+ parameters:
+ tool_chain_tag: ${{ parameters.tool_chain_tag }}
+
+# Potential Extra steps
+- ${{ parameters.extra_install_step }}
+
+# Build
+- task: CmdLine@1
+ displayName: Build
+ inputs:
+ filename: stuart_build
+ arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}} TARGET=${{ parameters.build_target}} -a ${{ parameters.build_arch}} ${{ parameters.build_flags}}
+ condition: and(gt(variables.pkg_count, 0), succeeded())
+
+# Run
+- task: CmdLine@1
+ displayName: Run to shell
+ inputs:
+ filename: stuart_build
+ arguments: -c ${{ parameters.build_file }} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}} TARGET=${{ parameters.build_target}} -a ${{ parameters.build_arch}} ${{ parameters.build_flags}} ${{ parameters.run_flags }} --FlashOnly
+ condition: and(and(gt(variables.pkg_count, 0), succeeded()), eq(variables['Run'], true))
+ timeoutInMinutes: 1
+
+# Copy the build logs to the artifact staging directory
+- task: CopyFiles@2
+ displayName: "Copy build logs"
+ inputs:
+ targetFolder: "$(Build.ArtifactStagingDirectory)"
+ SourceFolder: "Build"
+ contents: |
+ BUILDLOG_*.txt
+ BUILDLOG_*.md
+ CI_*.txt
+ CI_*.md
+ CISETUP.txt
+ SETUPLOG.txt
+ UPDATE_LOG.txt
+ PREVALLOG.txt
+ TestSuites.xml
+ **/BUILD_TOOLS_REPORT.html
+ **/OVERRIDELOG.TXT
+ BASETOOLS_BUILD*.*
+ flattenFolders: true
+ condition: succeededOrFailed()
+
+# Publish build artifacts to Azure Artifacts/TFS or a file share
+- task: PublishBuildArtifacts@1
+ continueOnError: true
+ displayName: "Publish build logs"
+ inputs:
+ pathtoPublish: "$(Build.ArtifactStagingDirectory)"
+ artifactName: "Build Logs $(System.JobName)"
+ condition: succeededOrFailed()
--
2.16.3.windows.1
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH v3 2/7] ArmVirtPkg: Add Platform CI and configuration for Core CI
[not found] <20200424213108.19888-1-michael.kubacki@outlook.com>
2020-04-24 21:31 ` [PATCH v3 1/7] .azurepipelines: Add Platform CI template Michael Kubacki
@ 2020-04-24 21:31 ` Michael Kubacki
2020-04-27 12:22 ` Leif Lindholm
2020-04-24 21:31 ` [PATCH v3 3/7] EmulatorPkg: " Michael Kubacki
` (4 subsequent siblings)
6 siblings, 1 reply; 13+ messages in thread
From: Michael Kubacki @ 2020-04-24 21:31 UTC (permalink / raw)
To: devel; +Cc: Laszlo Ersek, Ard Biesheuvel, Leif Lindholm
From: Sean Brogan <sean.brogan@microsoft.com>
REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2570
Add new Azure Pipeline definitions to build and run ArmVirtPkg with:
* Ubuntu GCC5
Add PyTool based build of ArmVirtPkg
Add extdep for managing the iasl dependency
Add ArmVirtPkg.ci.yaml for Core CI
Add ReadMe.md for details and instructions
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
Cc: Leif Lindholm <leif@nuviainc.com>
Signed-off-by: Sean Brogan <sean.brogan@microsoft.com>
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
Reviewed-by: Shenglei Zhang <shenglei.zhang@intel.com>
Acked-by: Laszlo Ersek <lersek@redhat.com>
Acked-by: Ard Biesheuvel <ard.biesheuvel@arm.com>
Reviewed-by: Bret Barkelew <bret.barkelew@microsoft.com>
---
ArmVirtPkg/ArmVirtPkg.ci.yaml | 103 ++++++++
ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml | 89 +++++++
ArmVirtPkg/PlatformCI/PlatformBuild.py | 276 ++++++++++++++++++++
ArmVirtPkg/PlatformCI/ReadMe.md | 125 +++++++++
ArmVirtPkg/PlatformCI/iasl_ext_dep.yaml | 21 ++
5 files changed, 614 insertions(+)
diff --git a/ArmVirtPkg/ArmVirtPkg.ci.yaml b/ArmVirtPkg/ArmVirtPkg.ci.yaml
new file mode 100644
index 000000000000..4553725ee528
--- /dev/null
+++ b/ArmVirtPkg/ArmVirtPkg.ci.yaml
@@ -0,0 +1,103 @@
+## @file
+# Core CI configuration for ArmVirtPkg
+#
+# ArmVirtPkg is part of Platform Ci for builds so this is only
+# used for code analysis.
+#
+# Copyright (c) Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ ## options defined .pytool/Plugin/CompilerPlugin
+ "CompilerPlugin": {
+ "DscPath": "" # Don't support this test
+ },
+
+ ## options defined .pytool/Plugin/HostUnitTestCompilerPlugin
+ "HostUnitTestCompilerPlugin": {
+ "DscPath": "" # Don't support this test
+ },
+
+ ## options defined .pytool/Plugin/CharEncodingCheck
+ "CharEncodingCheck": {
+ "IgnoreFiles": []
+ },
+
+ ## options defined .pytool/Plugin/DependencyCheck
+ "DependencyCheck": {
+ "AcceptableDependencies": [
+ "MdePkg/MdePkg.dec",
+ "MdeModulePkg/MdeModulePkg.dec",
+ "ArmVirtPkg/ArmVirtPkg.dec",
+ "NetworkPkg/NetworkPkg.dec",
+ "ArmPkg/ArmPkg.dec",
+ "OvmfPkg/OvmfPkg.dec",
+ "EmbeddedPkg/EmbeddedPkg.dec",
+ "ArmPlatformPkg/ArmPlatformPkg.dec",
+ "SecurityPkg/SecurityPkg.dec",
+ "ShellPkg/ShellPkg.dec" #Is this ok?
+ ],
+ # For host based unit tests
+ "AcceptableDependencies-HOST_APPLICATION":[
+ "UnitTestFrameworkPkg/UnitTestFrameworkPkg.dec"
+ ],
+ # For UEFI shell based apps
+ "AcceptableDependencies-UEFI_APPLICATION":[
+
+ ],
+ "IgnoreInf": []
+ },
+
+ ## options defined .pytool/Plugin/DscCompleteCheck
+ "DscCompleteCheck": {
+ "IgnoreInf": [""],
+ "DscPath": "" # Don't support this test
+ },
+
+ ## options defined .pytool/Plugin/HostUnitTestDscCompleteCheck
+ "HostUnitTestDscCompleteCheck": {
+ "IgnoreInf": [""],
+ "DscPath": "" # Don't support this test
+ },
+
+ ## options defined .pytool/Plugin/GuidCheck
+ "GuidCheck": {
+ "IgnoreGuidName": [],
+ "IgnoreGuidValue": [],
+ "IgnoreFoldersAndFiles": [],
+ "IgnoreDuplicates": [],
+ },
+
+ ## options defined .pytool/Plugin/LibraryClassCheck
+ "LibraryClassCheck": {
+ "IgnoreHeaderFile": []
+ },
+
+ ## options defined .pytool/Plugin/SpellCheck
+ "SpellCheck": {
+ "AuditOnly": False, # Fails right now with over 270 errors
+ "IgnoreFiles": [], # use gitignore syntax to ignore errors in matching files
+ "ExtendWords": [
+ "setjump",
+ "plong",
+ "lparam",
+ "lpdword",
+ "lpthread",
+ "lresult",
+ "bootable",
+ "bsymbolic",
+ "endiannness",
+ "fvmain",
+ "multiboot",
+ "qemu's",
+ "ramdisk",
+ "ramfb",
+ "unbootable",
+ "virt's",
+ "werror",
+ "xenio"
+ ], # words to extend to the dictionary for this package
+ "IgnoreStandardPaths": [], # Standard Plugin defined paths that should be ignore
+ "AdditionalIncludePaths": [] # Additional paths to spell check (wildcards supported)
+ }
+}
diff --git a/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml b/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
new file mode 100644
index 000000000000..b9452ec5bfaf
--- /dev/null
+++ b/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
@@ -0,0 +1,89 @@
+## @file
+# Azure Pipeline build file for building a platform.
+#
+# Platform: ArmVirtQemu
+# OS: Ubuntu
+# Toolchain: GCC5
+#
+# Copyright (c) Microsoft Corporation.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+trigger:
+ - master
+pr:
+ - master
+
+jobs:
+ - job: Platform_CI
+ variables:
+ package: 'ArmVirtPkg'
+ vm_image: 'ubuntu-latest'
+ should_run: true
+ run_flags: "MAKE_STARTUP_NSH=TRUE QEMU_HEADLESS=TRUE"
+
+ #Use matrix to speed up the build process
+ strategy:
+ matrix:
+ QEMU_AARCH64_DEBUG:
+ Build.File: "$(package)/PlatformCI/PlatformBuild.py"
+ Build.Arch: "AARCH64"
+ Build.Flags: ""
+ Build.Target: "DEBUG"
+ Run.Flags: $(run_flags)
+ Run: $(should_run)
+ QEMU_AARCH64_RELEASE:
+ Build.File: "$(package)/PlatformCI/PlatformBuild.py"
+ Build.Arch: "AARCH64"
+ Build.Flags: ""
+ Build.Target: "RELEASE"
+ Run.Flags: $(run_flags)
+ Run: $(should_run)
+ QEMU_AARCH64_NOOPT:
+ Build.File: "$(package)/PlatformCI/PlatformBuild.py"
+ Build.Arch: "AARCH64"
+ Build.Flags: ""
+ Build.Target: "NOOPT"
+ Run.Flags: $(run_flags)
+ Run: $(should_run)
+ QEMU_ARM_DEBUG:
+ Build.File: "$(package)/PlatformCI/PlatformBuild.py"
+ Build.Arch: "ARM"
+ Build.Flags: ""
+ Build.Target: "DEBUG"
+ Run.Flags: $(run_flags)
+ Run: $(should_run)
+ QEMU_ARM_RELEASE:
+ Build.File: "$(package)/PlatformCI/PlatformBuild.py"
+ Build.Arch: "ARM"
+ Build.Flags: ""
+ Build.Target: "RELEASE"
+ Run.Flags: $(run_flags)
+ Run: $(should_run)
+ QEMU_ARM_NOOPT:
+ Build.File: "$(package)/PlatformCI/PlatformBuild.py"
+ Build.Arch: "ARM"
+ Build.Flags: ""
+ Build.Target: "NOOPT"
+ Run.Flags: $(run_flags)
+ Run: $(should_run)
+
+ workspace:
+ clean: all
+
+ pool:
+ vmImage: $(vm_image)
+
+ steps:
+ - template: ../../../.azurepipelines/templates/platform-build-run-steps.yml
+ parameters:
+ tool_chain_tag: GCC5
+ build_pkg: $(package)
+ build_target: $(Build.Target)
+ build_arch: $(Build.Arch)
+ build_file: $(Build.File)
+ build_flags: $(Build.Flags)
+ run_flags: $(Run.Flags)
+ extra_install_step:
+ - bash: sudo apt-get install qemu
+ displayName: Install qemu
+ condition: and(gt(variables.pkg_count, 0), succeeded())
diff --git a/ArmVirtPkg/PlatformCI/PlatformBuild.py b/ArmVirtPkg/PlatformCI/PlatformBuild.py
new file mode 100644
index 000000000000..dff653e919eb
--- /dev/null
+++ b/ArmVirtPkg/PlatformCI/PlatformBuild.py
@@ -0,0 +1,276 @@
+# @file
+# Script to Build ArmVirtPkg UEFI firmware
+#
+# Copyright (c) Microsoft Corporation.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+import os
+import logging
+import io
+
+from edk2toolext.environment import shell_environment
+from edk2toolext.environment.uefi_build import UefiBuilder
+from edk2toolext.invocables.edk2_platform_build import BuildSettingsManager
+from edk2toolext.invocables.edk2_setup import SetupSettingsManager, RequiredSubmodule
+from edk2toolext.invocables.edk2_update import UpdateSettingsManager
+from edk2toolext.invocables.edk2_pr_eval import PrEvalSettingsManager
+from edk2toollib.utility_functions import RunCmd
+from edk2toollib.utility_functions import GetHostInfo
+
+# ####################################################################################### #
+# Common Configuration #
+# ####################################################################################### #
+
+
+class CommonPlatform():
+ ''' Common settings for this platform. Define static data here and use
+ for the different parts of stuart
+ '''
+ PackagesSupported = ("ArmVirtPkg",)
+ ArchSupported = ("AARCH64", "ARM")
+ TargetsSupported = ("DEBUG", "RELEASE", "NOOPT")
+ Scopes = ('armvirt', 'edk2-build')
+ WorkspaceRoot = os.path.realpath(os.path.join(
+ os.path.dirname(os.path.abspath(__file__)), "..", ".."))
+
+ # ####################################################################################### #
+ # Configuration for Update & Setup #
+ # ####################################################################################### #
+
+
+class SettingsManager(UpdateSettingsManager, SetupSettingsManager, PrEvalSettingsManager):
+
+ def GetPackagesSupported(self):
+ ''' return iterable of edk2 packages supported by this build.
+ These should be edk2 workspace relative paths '''
+ return CommonPlatform.PackagesSupported
+
+ def GetArchitecturesSupported(self):
+ ''' return iterable of edk2 architectures supported by this build '''
+ return CommonPlatform.ArchSupported
+
+ def GetTargetsSupported(self):
+ ''' return iterable of edk2 target tags supported by this build '''
+ return CommonPlatform.TargetsSupported
+
+ def GetRequiredSubmodules(self):
+ ''' return iterable containing RequiredSubmodule objects.
+ If no RequiredSubmodules return an empty iterable
+ '''
+ rs = []
+
+ # intentionally declare this one with recursive false to avoid overhead
+ rs.append(RequiredSubmodule(
+ "CryptoPkg/Library/OpensslLib/openssl", False))
+
+ # To avoid maintenance of this file for every new submodule
+ # lets just parse the .gitmodules and add each if not already in list.
+ # The GetRequiredSubmodules is designed to allow a build to optimize
+ # the desired submodules but it isn't necessary for this repository.
+ result = io.StringIO()
+ ret = RunCmd("git", "config --file .gitmodules --get-regexp path", workingdir=self.GetWorkspaceRoot(), outstream=result)
+ # Cmd output is expected to look like:
+ # submodule.CryptoPkg/Library/OpensslLib/openssl.path CryptoPkg/Library/OpensslLib/openssl
+ # submodule.SoftFloat.path ArmPkg/Library/ArmSoftFloatLib/berkeley-softfloat-3
+ if ret == 0:
+ for line in result.getvalue().splitlines():
+ _, _, path = line.partition(" ")
+ if path is not None:
+ if path not in [x.path for x in rs]:
+ rs.append(RequiredSubmodule(path, True)) # add it with recursive since we don't know
+ return rs
+
+ def SetArchitectures(self, list_of_requested_architectures):
+ ''' Confirm the requests architecture list is valid and configure SettingsManager
+ to run only the requested architectures.
+
+ Raise Exception if a list_of_requested_architectures is not supported
+ '''
+ unsupported = set(list_of_requested_architectures) - \
+ set(self.GetArchitecturesSupported())
+ if(len(unsupported) > 0):
+ errorString = (
+ "Unsupported Architecture Requested: " + " ".join(unsupported))
+ logging.critical(errorString)
+ raise Exception(errorString)
+ self.ActualArchitectures = list_of_requested_architectures
+
+ def GetWorkspaceRoot(self):
+ ''' get WorkspacePath '''
+ return CommonPlatform.WorkspaceRoot
+
+ def GetActiveScopes(self):
+ ''' return tuple containing scopes that should be active for this process '''
+
+ scopes = CommonPlatform.Scopes
+ ActualToolChainTag = shell_environment.GetBuildVars().GetValue("TOOL_CHAIN_TAG", "")
+
+ if GetHostInfo().os.upper() == "LINUX" and ActualToolChainTag.upper().startswith("GCC"):
+ if "AARCH64" in self.ActualArchitectures:
+ scopes += ("gcc_aarch64_linux",)
+ if "ARM" in self.ActualArchitectures:
+ scopes += ("gcc_arm_linux",)
+ return scopes
+
+ def FilterPackagesToTest(self, changedFilesList: list, potentialPackagesList: list) -> list:
+ ''' Filter other cases that this package should be built
+ based on changed files. This should cover things that can't
+ be detected as dependencies. '''
+ build_these_packages = []
+ possible_packages = potentialPackagesList.copy()
+ for f in changedFilesList:
+ # BaseTools files that might change the build
+ if "BaseTools" in f:
+ if os.path.splitext(f) not in [".txt", ".md"]:
+ build_these_packages = possible_packages
+ break
+
+ # if the azure pipeline platform template file changed
+ if "platform-build-run-steps.yml" in f:
+ build_these_packages = possible_packages
+ break
+
+
+ return build_these_packages
+
+ def GetPlatformDscAndConfig(self) -> tuple:
+ ''' If a platform desires to provide its DSC then Policy 4 will evaluate if
+ any of the changes will be built in the dsc.
+
+ The tuple should be (<workspace relative path to dsc file>, <input dictionary of dsc key value pairs>)
+ '''
+ return (os.path.join("ArmVirtPkg", "ArmVirtQemu.dsc"), {})
+
+
+ # ####################################################################################### #
+ # Actual Configuration for Platform Build #
+ # ####################################################################################### #
+
+
+class PlatformBuilder(UefiBuilder, BuildSettingsManager):
+ def __init__(self):
+ UefiBuilder.__init__(self)
+
+ def AddCommandLineOptions(self, parserObj):
+ ''' Add command line options to the argparser '''
+ parserObj.add_argument('-a', "--arch", dest="build_arch", type=str, default="AARCH64",
+ help="Optional - Architecture to build. Default = AARCH64")
+
+ def RetrieveCommandLineOptions(self, args):
+ ''' Retrieve command line options from the argparser '''
+
+ shell_environment.GetBuildVars().SetValue(
+ "TARGET_ARCH", args.build_arch.upper(), "From CmdLine")
+
+ shell_environment.GetBuildVars().SetValue(
+ "ACTIVE_PLATFORM", "ArmVirtPkg/ArmVirtQemu.dsc", "From CmdLine")
+
+ def GetWorkspaceRoot(self):
+ ''' get WorkspacePath '''
+ return CommonPlatform.WorkspaceRoot
+
+ def GetPackagesPath(self):
+ ''' Return a list of workspace relative paths that should be mapped as edk2 PackagesPath '''
+ return ()
+
+ def GetActiveScopes(self):
+ ''' return tuple containing scopes that should be active for this process '''
+ scopes = CommonPlatform.Scopes
+ ActualToolChainTag = shell_environment.GetBuildVars().GetValue("TOOL_CHAIN_TAG", "")
+ Arch = shell_environment.GetBuildVars().GetValue("TARGET_ARCH", "")
+
+ if GetHostInfo().os.upper() == "LINUX" and ActualToolChainTag.upper().startswith("GCC"):
+ if "AARCH64" == Arch:
+ scopes += ("gcc_aarch64_linux",)
+ elif "ARM" == Arch:
+ scopes += ("gcc_arm_linux",)
+ return scopes
+
+ def GetName(self):
+ ''' Get the name of the repo, platform, or product being build '''
+ ''' Used for naming the log file, among others '''
+ # check the startup nsh flag and if set then rename the log file.
+ # this helps in CI so we don't overwrite the build log since running
+ # uses the stuart_build command.
+ if(shell_environment.GetBuildVars().GetValue("MAKE_STARTUP_NSH", "FALSE") == "TRUE"):
+ return "ArmVirtPkg_With_Run"
+ return "ArmVirtPkg"
+
+ def GetLoggingLevel(self, loggerType):
+ ''' Get the logging level for a given type
+ base == lowest logging level supported
+ con == Screen logging
+ txt == plain text file logging
+ md == markdown file logging
+ '''
+ return logging.DEBUG
+
+ def SetPlatformEnv(self):
+ logging.debug("PlatformBuilder SetPlatformEnv")
+ self.env.SetValue("PRODUCT_NAME", "ArmVirtQemu", "Platform Hardcoded")
+ self.env.SetValue("MAKE_STARTUP_NSH", "FALSE", "Default to false")
+ self.env.SetValue("QEMU_HEADLESS", "FALSE", "Default to false")
+ return 0
+
+ def PlatformPreBuild(self):
+ return 0
+
+ def PlatformPostBuild(self):
+ return 0
+
+ def FlashRomImage(self):
+ VirtualDrive = os.path.join(self.env.GetValue(
+ "BUILD_OUTPUT_BASE"), "VirtualDrive")
+ os.makedirs(VirtualDrive, exist_ok=True)
+ OutputPath_FV = os.path.join(
+ self.env.GetValue("BUILD_OUTPUT_BASE"), "FV")
+ Built_FV = os.path.join(OutputPath_FV, "QEMU_EFI.fd")
+
+ # pad fd to 64mb
+ with open(Built_FV, "ab") as fvfile:
+ fvfile.seek(0, os.SEEK_END)
+ additional = b'\0' * ((64 * 1024 * 1024)-fvfile.tell())
+ fvfile.write(additional)
+
+ # QEMU must be on that path
+
+ # Unique Command and Args parameters per ARCH
+ if (self.env.GetValue("TARGET_ARCH").upper() == "AARCH64"):
+ cmd = "qemu-system-aarch64"
+ args = "-M virt"
+ args += " -cpu cortex-a57" # emulate cpu
+ elif(self.env.GetValue("TARGET_ARCH").upper() == "ARM"):
+ cmd = "qemu-system-arm"
+ args = "-M virt"
+ args += " -cpu cortex-a15" # emulate cpu
+ else:
+ raise NotImplementedError()
+
+ # Common Args
+ args += " -pflash " + Built_FV # path to fw
+ args += " -m 1024" # 1gb memory
+ # turn off network
+ args += " -net none"
+ # Serial messages out
+ args += " -serial stdio"
+ # Mount disk with startup.nsh
+ args += f" -drive file=fat:rw:{VirtualDrive},format=raw,media=disk"
+
+ # Conditional Args
+ if (self.env.GetValue("QEMU_HEADLESS").upper() == "TRUE"):
+ args += " -display none" # no graphics
+
+ if (self.env.GetValue("MAKE_STARTUP_NSH").upper() == "TRUE"):
+ f = open(os.path.join(VirtualDrive, "startup.nsh"), "w")
+ f.write("BOOT SUCCESS !!! \n")
+ # add commands here
+ f.write("reset -s\n")
+ f.close()
+
+ ret = RunCmd(cmd, args)
+
+ if ret == 0xc0000005:
+ # for some reason getting a c0000005 on successful return
+ return 0
+
+ return ret
diff --git a/ArmVirtPkg/PlatformCI/ReadMe.md b/ArmVirtPkg/PlatformCI/ReadMe.md
new file mode 100644
index 000000000000..7c11d925f59e
--- /dev/null
+++ b/ArmVirtPkg/PlatformCI/ReadMe.md
@@ -0,0 +1,125 @@
+# ArmVirtPkg - Platform CI
+
+This Readme.md describes the Azure DevOps based Platform CI for ArmVirtPkg and how
+to use the same Pytools based build infrastructure locally.
+
+## Supported Configuration Details
+
+This solution for building and running ArmVirtPkg has only been validated with Ubuntu
+18.04 and the GCC5 toolchain. Two different firmware builds are supported and are
+described below.
+
+| Configuration name | Architecture | DSC File |Additional Flags |
+| :---------- | :----- | :----- | :---- |
+| AARCH64 | AARCH64 | ArmVirtQemu.dsc | None |
+| ARM | ARM | ArmVirtQemu.dsc | None |
+
+## EDK2 Developer environment
+
+- [Python 3.8.x - Download & Install](https://www.python.org/downloads/)
+- [GIT - Download & Install](https://git-scm.com/download/)
+- [QEMU - Download, Install, and add to your path](https://www.qemu.org/download/)
+- [Edk2 Source](https://github.com/tianocore/edk2)
+- Additional packages found necessary for Ubuntu 18.04
+ - apt-get install gcc g++ make uuid-dev
+
+Note: edksetup, Submodule initialization and manual installation of NASM, iASL, or
+the required cross-compiler toolchains are **not** required, this is handled by the
+Pytools build system.
+
+## Building with Pytools for ArmVirtPkg
+
+1. [Optional] Create a Python Virtual Environment - generally once per workspace
+
+ ``` bash
+ python -m venv <name of virtual environment>
+ ```
+
+2. [Optional] Activate Virtual Environment - each time new shell opened
+ - Windows
+
+ ``` bash
+ <name of virtual environment>/Scripts/activate.bat
+ ```
+
+ - Linux
+
+ ```bash
+ source <name of virtual environment>/bin/activate
+ ```
+
+3. Install Pytools - generally once per virtual env or whenever pip-requirements.txt changes
+
+ ``` bash
+ pip install --upgrade -r pip-requirements.txt
+ ```
+
+4. Initialize & Update Submodules - only when submodules updated
+
+ ``` bash
+ stuart_setup -c ArmVirtPkg/PlatformCI/PlatformBuild.py TOOL_CHAIN_TAG=<TOOL_CHAIN_TAG> -a <TARGET_ARCH>
+ ```
+
+5. Initialize & Update Dependencies - only as needed when ext_deps change
+
+ ``` bash
+ stuart_update -c ArmVirtPkg/PlatformCI/PlatformBuild.py TOOL_CHAIN_TAG=<TOOL_CHAIN_TAG> -a <TARGET_ARCH>
+ ```
+
+6. Compile the basetools if necessary - only when basetools C source files change
+
+ ``` bash
+ python BaseTools/Edk2ToolsBuild.py -t <ToolChainTag>
+ ```
+
+7. Compile Firmware
+
+ ``` bash
+ stuart_build -c ArmVirtPkg/PlatformCI/PlatformBuild.py TOOL_CHAIN_TAG=<TOOL_CHAIN_TAG> -a <TARGET_ARCH>
+ ```
+
+ - use `stuart_build -c ArmVirtPkg/PlatformCI/PlatformBuild.py -h` option to see additional
+ options like `--clean`
+
+8. Running Emulator
+ - You can add `--FlashRom` to the end of your build command and the emulator will run after the
+ build is complete.
+ - or use the `--FlashOnly` feature to just run the emulator.
+
+ ``` bash
+ stuart_build -c ArmVirtPkg/PlatformCI/PlatformBuild.py TOOL_CHAIN_TAG=<TOOL_CHAIN_TAG> -a <TARGET_ARCH> --FlashOnly
+ ```
+
+### Notes
+
+1. Including the expected build architecture and toolchain to the _stuart_update_ command is critical.
+ This is because there are extra scopes and tools that will be resolved during the update step that
+ need to match your build step.
+2. Configuring *ACTIVE_PLATFORM* and *TARGET_ARCH* in Conf/target.txt is **not** required. This
+ environment is set by PlatformBuild.py based upon the `[-a <TARGET_ARCH>]` parameter.
+3. QEMU must be on your path. On Windows this is a manual process and not part of the QEMU installer.
+
+**NOTE:** Logging the execution output will be in the normal stuart log as well as to your console.
+
+### Custom Build Options
+
+**MAKE_STARTUP_NSH=TRUE** will output a *startup.nsh* file to the location mapped as fs0. This is
+used in CI in combination with the `--FlashOnly` feature to run QEMU to the UEFI shell and then execute
+the contents of *startup.nsh*.
+
+**QEMU_HEADLESS=TRUE** Since CI servers run headless QEMU must be told to run with no display otherwise
+an error occurs. Locally you don't need to set this.
+
+### Passing Build Defines
+
+To pass build defines through _stuart_build_, prepend `BLD_*_`to the define name and pass it on the
+command-line. _stuart_build_ currently requires values to be assigned, so add an`=1` suffix for bare defines.
+For example, to enable the TPM2 support, instead of the traditional "-D TPM2_ENABLE=TRUE", the stuart_build
+command-line would be:
+
+`stuart_build -c ArmVirtPkg/PlatformCI/PlatformBuild.py BLD_*_TPM2_ENABLE=TRUE`
+
+## References
+
+- [Installing and using Pytools](https://github.com/tianocore/edk2-pytool-extensions/blob/master/docs/using.md#installing)
+- More on [python virtual environments](https://docs.python.org/3/library/venv.html)
diff --git a/ArmVirtPkg/PlatformCI/iasl_ext_dep.yaml b/ArmVirtPkg/PlatformCI/iasl_ext_dep.yaml
new file mode 100644
index 000000000000..8869ed3ecef1
--- /dev/null
+++ b/ArmVirtPkg/PlatformCI/iasl_ext_dep.yaml
@@ -0,0 +1,21 @@
+## @file
+# Download iasl executable tool from a nuget.org package
+# - package contains different binaries based on host
+# Add the folder with the tool to the path
+#
+# This is only downloaded for scope armvirt thus
+# should have no impact on the asl compiler used by any
+# other platform build
+#
+# Copyright (c) Microsoft Corporation.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "id": "iasl-armvirt-1",
+ "scope": "armvirt",
+ "type": "nuget",
+ "name": "iasl",
+ "source": "https://api.nuget.org/v3/index.json",
+ "version": "20190215.0.0",
+ "flags": ["set_path", "host_specific"],
+}
--
2.16.3.windows.1
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH v3 3/7] EmulatorPkg: Add Platform CI and configuration for Core CI
[not found] <20200424213108.19888-1-michael.kubacki@outlook.com>
2020-04-24 21:31 ` [PATCH v3 1/7] .azurepipelines: Add Platform CI template Michael Kubacki
2020-04-24 21:31 ` [PATCH v3 2/7] ArmVirtPkg: Add Platform CI and configuration for Core CI Michael Kubacki
@ 2020-04-24 21:31 ` Michael Kubacki
2020-04-28 11:22 ` [edk2-devel] " Liming Gao
2020-04-24 21:31 ` [PATCH v3 4/7] OvmfPkg: " Michael Kubacki
` (3 subsequent siblings)
6 siblings, 1 reply; 13+ messages in thread
From: Michael Kubacki @ 2020-04-24 21:31 UTC (permalink / raw)
To: devel; +Cc: Jordan Justen, Andrew Fish, Ray Ni
From: Sean Brogan <sean.brogan@microsoft.com>
REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2570
Add new Azure Pipeline definitions to build and run EmulatorPkg with:
* Ubuntu GCC5
* Windows VS2019
Add PyTool based build of EmulatorPkg
Add EmulatorPkg.ci.yaml for Core CI
Add ReadMe.md for details and instructions
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Andrew Fish <afish@apple.com>
Cc: Ray Ni <ray.ni@intel.com>
Signed-off-by: Sean Brogan <sean.brogan@microsoft.com>
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
Reviewed-by: Shenglei Zhang <shenglei.zhang@intel.com>
Reviewed-by: Bret Barkelew <bret.barkelew@microsoft.com>
---
EmulatorPkg/EmulatorPkg.ci.yaml | 85 ++++++
EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml | 95 +++++++
EmulatorPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml | 85 ++++++
EmulatorPkg/PlatformCI/PlatformBuild.py | 272 ++++++++++++++++++++
EmulatorPkg/PlatformCI/ReadMe.md | 128 +++++++++
5 files changed, 665 insertions(+)
diff --git a/EmulatorPkg/EmulatorPkg.ci.yaml b/EmulatorPkg/EmulatorPkg.ci.yaml
new file mode 100644
index 000000000000..81f81780ec76
--- /dev/null
+++ b/EmulatorPkg/EmulatorPkg.ci.yaml
@@ -0,0 +1,85 @@
+## @file
+# Core CI configuration for EmulatorPkg
+#
+# EmulatorPkg is part of Platform Ci for builds so this is only
+# used for code analysis.
+#
+# Copyright (c) Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ ## options defined .pytool/Plugin/CompilerPlugin
+ "CompilerPlugin": {
+ "DscPath": "" # Don't support this test
+ },
+
+ ## options defined .pytool/Plugin/HostUnitTestCompilerPlugin
+ "HostUnitTestCompilerPlugin": {
+ "DscPath": "" # Don't support this test
+ },
+
+ ## options defined .pytool/Plugin/CharEncodingCheck
+ "CharEncodingCheck": {
+ "IgnoreFiles": []
+ },
+
+ ## options defined .pytool/Plugin/DependencyCheck
+ "DependencyCheck": {
+ "AcceptableDependencies": [
+ # For this platform all packages are allowed???
+ "MdePkg/MdePkg.dec",
+ "MdeModulePkg/MdeModulePkg.dec",
+ "EmulatorPkg/EmulatorPkg.dec",
+ "NetworkPkg/NetworkPkg.dec",
+ "EmbeddedPkg/EmbeddedPkg.dec", ## is this one OK??
+ ],
+ # For host based unit tests
+ "AcceptableDependencies-HOST_APPLICATION":[
+ "UnitTestFrameworkPkg/UnitTestFrameworkPkg.dec"
+ ],
+ # For UEFI shell based apps
+ "AcceptableDependencies-UEFI_APPLICATION":[],
+ "IgnoreInf": []
+ },
+
+ ## options defined .pytool/Plugin/DscCompleteCheck
+ "DscCompleteCheck": {
+ "IgnoreInf": [""],
+ "DscPath": "" # Don't support this test
+ },
+
+ ## options defined .pytool/Plugin/HostUnitTestDscCompleteCheck
+ "HostUnitTestDscCompleteCheck": {
+ "IgnoreInf": [""],
+ "DscPath": "" # Don't support this test
+ },
+
+ ## options defined .pytool/Plugin/GuidCheck
+ "GuidCheck": {
+ "IgnoreGuidName": [],
+ "IgnoreGuidValue": [],
+ "IgnoreFoldersAndFiles": [],
+ "IgnoreDuplicates": [],
+ },
+
+ ## options defined .pytool/Plugin/LibraryClassCheck
+ "LibraryClassCheck": {
+ "IgnoreHeaderFile": []
+ },
+
+ ## options defined .pytool/Plugin/SpellCheck
+ "SpellCheck": {
+ "AuditOnly": True, # Fails right now with over 270 errors
+ "IgnoreFiles": [], # use gitignore syntax to ignore errors in matching files
+ "ExtendWords": [
+ "setjump",
+ "plong",
+ "lparam",
+ "lpdword",
+ "lpthread",
+ "lresult",
+ ], # words to extend to the dictionary for this package
+ "IgnoreStandardPaths": [], # Standard Plugin defined paths that should be ignore
+ "AdditionalIncludePaths": [] # Additional paths to spell check (wildcards supported)
+ }
+}
diff --git a/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml b/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
new file mode 100644
index 000000000000..12ef8226ff54
--- /dev/null
+++ b/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
@@ -0,0 +1,95 @@
+## @file
+# Azure Pipeline build file for building a platform.
+#
+# Platform: EmulatorPkg
+# OS: Ubuntu
+# Toolchain: GCC5
+#
+# Copyright (c) Microsoft Corporation.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+trigger:
+ - master
+pr:
+ - master
+jobs:
+ - job: Platform_CI
+ variables:
+ package: 'EmulatorPkg'
+ vm_image: 'ubuntu-latest'
+ should_run: false
+ run_flags: "MAKE_STARTUP_NSH=TRUE"
+
+ #Use matrix to speed up the build process
+ strategy:
+ matrix:
+ EmulatorPkg_X64_DEBUG:
+ Build.File: "$(package)/PlatformCI/PlatformBuild.py"
+ Build.Arch: "X64"
+ Build.Flags: ""
+ Build.Target: "DEBUG"
+ Run.Flags: $(run_flags)
+ Run: $(should_run)
+ EmulatorPkg_X64_RELEASE:
+ Build.File: "$(package)/PlatformCI/PlatformBuild.py"
+ Build.Arch: "X64"
+ Build.Flags: ""
+ Build.Target: "RELEASE"
+ Run.Flags: $(run_flags)
+ Run: $(should_run)
+ EmulatorPkg_X64_NOOPT:
+ Build.File: "$(package)/PlatformCI/PlatformBuild.py"
+ Build.Arch: "X64"
+ Build.Flags: ""
+ Build.Target: "NOOPT"
+ Run.Flags: $(run_flags)
+ Run: $(should_run)
+ EmulatorPkg_IA32_DEBUG:
+ Build.File: "$(package)/PlatformCI/PlatformBuild.py"
+ Build.Arch: "IA32"
+ Build.Flags: ""
+ Build.Target: "DEBUG"
+ Run.Flags: $(run_flags)
+ Run: $(should_run)
+ EmulatorPkg_IA32_RELEASE:
+ Build.File: "$(package)/PlatformCI/PlatformBuild.py"
+ Build.Arch: "IA32"
+ Build.Flags: ""
+ Build.Target: "RELEASE"
+ Run.Flags: $(run_flags)
+ Run: $(should_run)
+ EmulatorPkg_IA32_NOOPT:
+ Build.File: "$(package)/PlatformCI/PlatformBuild.py"
+ Build.Arch: "IA32"
+ Build.Flags: ""
+ Build.Target: "NOOPT"
+ Run.Flags: $(run_flags)
+ Run: $(should_run)
+
+ workspace:
+ clean: all
+
+ pool:
+ vmImage: $(vm_image)
+
+ steps:
+ - template: ../../../.azurepipelines/templates/platform-build-run-steps.yml
+ parameters:
+ tool_chain_tag: GCC5
+ build_pkg: $(package)
+ build_target: $(Build.Target)
+ build_arch: $(Build.Arch)
+ build_file: $(Build.File)
+ build_flags: $(Build.Flags)
+ run_flags: $(Run.Flags)
+ # Add steps to install some IA32 only dependencies
+ extra_install_step:
+ - bash: sudo dpkg --add-architecture i386
+ displayName: Add i386 to dpkg
+ condition: and(gt(variables.pkg_count, 0), eq(variables['Build.Arch'], 'IA32'), succeeded())
+ - bash: sudo apt-get update
+ displayName: do apt-get update
+ condition: and(gt(variables.pkg_count, 0), eq(variables['Build.Arch'], 'IA32'), succeeded())
+ - bash: sudo apt-get install libc6-dev:i386 libx11-dev:i386 libxext-dev:i386 lib32gcc-7-dev
+ displayName: Add additional i386 packages
+ condition: and(gt(variables.pkg_count, 0), eq(variables['Build.Arch'], 'IA32'), succeeded())
diff --git a/EmulatorPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml b/EmulatorPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml
new file mode 100644
index 000000000000..a5baf4b6064b
--- /dev/null
+++ b/EmulatorPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml
@@ -0,0 +1,85 @@
+## @file
+# Azure Pipeline build file for building a platform.
+#
+# Platform: EMULATORPKG
+# OS: Windows
+# Toolchain: VS2019
+#
+# Copyright (c) Microsoft Corporation.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+trigger:
+ - master
+pr:
+ - master
+
+jobs:
+ - job: Platform_CI
+ variables:
+ package: 'EmulatorPkg'
+ vm_image: 'windows-latest'
+ should_run: true
+ run_flags: "MAKE_STARTUP_NSH=TRUE"
+
+ #Use matrix to speed up the build process
+ strategy:
+ matrix:
+ EmulatorPkg_X64_DEBUG:
+ Build.File: "$(package)/PlatformCI/PlatformBuild.py"
+ Build.Arch: "X64"
+ Build.Flags: ""
+ Build.Target: "DEBUG"
+ Run.Flags: $(run_flags)
+ Run: $(should_run)
+ EmulatorPkg_X64_RELEASE:
+ Build.File: "$(package)/PlatformCI/PlatformBuild.py"
+ Build.Arch: "X64"
+ Build.Flags: ""
+ Build.Target: "RELEASE"
+ Run.Flags: $(run_flags)
+ Run: $(should_run)
+ EmulatorPkg_X64_NOOPT:
+ Build.File: "$(package)/PlatformCI/PlatformBuild.py"
+ Build.Arch: "X64"
+ Build.Flags: ""
+ Build.Target: "NOOPT"
+ Run.Flags: $(run_flags)
+ Run: $(should_run)
+ EmulatorPkg_IA32_DEBUG:
+ Build.File: "$(package)/PlatformCI/PlatformBuild.py"
+ Build.Arch: "IA32 "
+ Build.Flags: ""
+ Build.Target: "DEBUG"
+ Run.Flags: $(run_flags)
+ Run: $(should_run)
+ EmulatorPkg_IA32_RELEASE:
+ Build.File: "$(package)/PlatformCI/PlatformBuild.py"
+ Build.Arch: "IA32 "
+ Build.Flags: ""
+ Build.Target: "RELEASE"
+ Run.Flags: $(run_flags)
+ Run: $(should_run)
+ EmulatorPkg_IA32_NOOPT:
+ Build.File: "$(package)/PlatformCI/PlatformBuild.py"
+ Build.Arch: "IA32 "
+ Build.Flags: ""
+ Build.Target: "NOOPT"
+ Run.Flags: $(run_flags)
+ Run: $(should_run)
+
+ workspace:
+ clean: all
+
+ pool:
+ vmImage: $(vm_image)
+
+ steps:
+ - template: ../../../.azurepipelines/templates/platform-build-run-steps.yml
+ parameters:
+ tool_chain_tag: VS2019
+ build_pkg: $(package)
+ build_target: $(Build.Target)
+ build_arch: $(Build.Arch)
+ build_file: $(Build.File)
+ build_flags: $(Build.Flags)
+ run_flags: $(Run.Flags)
diff --git a/EmulatorPkg/PlatformCI/PlatformBuild.py b/EmulatorPkg/PlatformCI/PlatformBuild.py
new file mode 100644
index 000000000000..6defbf6ef148
--- /dev/null
+++ b/EmulatorPkg/PlatformCI/PlatformBuild.py
@@ -0,0 +1,272 @@
+# @file
+# Script to Build EmulatorPkg UEFI firmware
+#
+# Copyright (c) Microsoft Corporation.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+import os
+import logging
+import io
+
+from edk2toolext.environment import shell_environment
+from edk2toolext.environment.uefi_build import UefiBuilder
+from edk2toolext.invocables.edk2_platform_build import BuildSettingsManager
+from edk2toolext.invocables.edk2_setup import SetupSettingsManager, RequiredSubmodule
+from edk2toolext.invocables.edk2_update import UpdateSettingsManager
+from edk2toolext.invocables.edk2_pr_eval import PrEvalSettingsManager
+from edk2toollib.utility_functions import RunCmd
+from edk2toollib.utility_functions import GetHostInfo
+
+# ####################################################################################### #
+# Common Configuration #
+# ####################################################################################### #
+
+
+class CommonPlatform():
+ ''' Common settings for this platform. Define static data here and use
+ for the different parts of stuart
+ '''
+ PackagesSupported = ("EmulatorPkg",)
+ ArchSupported = ("X64", "IA32")
+ TargetsSupported = ("DEBUG", "RELEASE", "NOOPT")
+ Scopes = ('emulatorpkg', 'edk2-build')
+ WorkspaceRoot = os.path.realpath(os.path.join(
+ os.path.dirname(os.path.abspath(__file__)), "..", ".."))
+
+ # ####################################################################################### #
+ # Configuration for Update & Setup #
+ # ####################################################################################### #
+
+
+class SettingsManager(UpdateSettingsManager, SetupSettingsManager, PrEvalSettingsManager):
+
+ def GetPackagesSupported(self):
+ ''' return iterable of edk2 packages supported by this build.
+ These should be edk2 workspace relative paths '''
+ return CommonPlatform.PackagesSupported
+
+ def GetArchitecturesSupported(self):
+ ''' return iterable of edk2 architectures supported by this build '''
+ return CommonPlatform.ArchSupported
+
+ def GetTargetsSupported(self):
+ ''' return iterable of edk2 target tags supported by this build '''
+ return CommonPlatform.TargetsSupported
+
+ def GetRequiredSubmodules(self):
+ ''' return iterable containing RequiredSubmodule objects.
+ If no RequiredSubmodules return an empty iterable
+ '''
+ rs = []
+ # intentionally declare this one with recursive false to avoid overhead
+ rs.append(RequiredSubmodule(
+ "CryptoPkg/Library/OpensslLib/openssl", False))
+
+ # To avoid maintenance of this file for every new submodule
+ # lets just parse the .gitmodules and add each if not already in list.
+ # The GetRequiredSubmodules is designed to allow a build to optimize
+ # the desired submodules but it isn't necessary for this repository.
+ result = io.StringIO()
+ ret = RunCmd("git", "config --file .gitmodules --get-regexp path", workingdir=self.GetWorkspaceRoot(), outstream=result)
+ # Cmd output is expected to look like:
+ # submodule.CryptoPkg/Library/OpensslLib/openssl.path CryptoPkg/Library/OpensslLib/openssl
+ # submodule.SoftFloat.path ArmPkg/Library/ArmSoftFloatLib/berkeley-softfloat-3
+ if ret == 0:
+ for line in result.getvalue().splitlines():
+ _, _, path = line.partition(" ")
+ if path is not None:
+ if path not in [x.path for x in rs]:
+ rs.append(RequiredSubmodule(path, True)) # add it with recursive since we don't know
+ return rs
+
+ def SetArchitectures(self, list_of_requested_architectures):
+ ''' Confirm the requests architecture list is valid and configure SettingsManager
+ to run only the requested architectures.
+
+ Raise Exception if a list_of_requested_architectures is not supported
+ '''
+ unsupported = set(list_of_requested_architectures) - \
+ set(self.GetArchitecturesSupported())
+ if(len(unsupported) > 0):
+ errorString = (
+ "Unsupported Architecture Requested: " + " ".join(unsupported))
+ logging.critical(errorString)
+ raise Exception(errorString)
+ self.ActualArchitectures = list_of_requested_architectures
+
+ def GetWorkspaceRoot(self):
+ ''' get WorkspacePath '''
+ return CommonPlatform.WorkspaceRoot
+
+ def GetActiveScopes(self):
+ ''' return tuple containing scopes that should be active for this process '''
+ return CommonPlatform.Scopes
+
+ def FilterPackagesToTest(self, changedFilesList: list, potentialPackagesList: list) -> list:
+ ''' Filter other cases that this package should be built
+ based on changed files. This should cover things that can't
+ be detected as dependencies. '''
+ build_these_packages = []
+ possible_packages = potentialPackagesList.copy()
+ for f in changedFilesList:
+ # BaseTools files that might change the build
+ if "BaseTools" in f:
+ if os.path.splitext(f) not in [".txt", ".md"]:
+ build_these_packages = possible_packages
+ break
+ # if the azure pipeline platform template file changed
+ if "platform-build-run-steps.yml" in f:
+ build_these_packages = possible_packages
+ break
+ return build_these_packages
+
+ def GetPlatformDscAndConfig(self) -> tuple:
+ ''' If a platform desires to provide its DSC then Policy 4 will evaluate if
+ any of the changes will be built in the dsc.
+
+ The tuple should be (<workspace relative path to dsc file>, <input dictionary of dsc key value pairs>)
+ '''
+ return (os.path.join("EmulatorPkg", "EmulatorPkg.dsc"), {})
+
+ # ####################################################################################### #
+ # Actual Configuration for Platform Build #
+ # ####################################################################################### #
+
+
+class PlatformBuilder(UefiBuilder, BuildSettingsManager):
+ def __init__(self):
+ UefiBuilder.__init__(self)
+
+ def AddCommandLineOptions(self, parserObj):
+ ''' Add command line options to the argparser '''
+ parserObj.add_argument('-a', "--arch", dest="build_arch", type=str, default="X64",
+ help="Optional - architecture to build. IA32 will use IA32 for Pei & Dxe. "
+ "X64 will use X64 for both PEI and DXE.")
+
+ def RetrieveCommandLineOptions(self, args):
+ ''' Retrieve command line options from the argparser '''
+
+ shell_environment.GetBuildVars().SetValue(
+ "TARGET_ARCH", args.build_arch.upper(), "From CmdLine")
+ shell_environment.GetBuildVars().SetValue(
+ "ACTIVE_PLATFORM", "EmulatorPkg/EmulatorPkg.dsc", "From CmdLine")
+
+ def GetWorkspaceRoot(self):
+ ''' get WorkspacePath '''
+ return CommonPlatform.WorkspaceRoot
+
+ def GetPackagesPath(self):
+ ''' Return a list of workspace relative paths that should be mapped as edk2 PackagesPath '''
+ return ()
+
+ def GetActiveScopes(self):
+ ''' return tuple containing scopes that should be active for this process '''
+ return CommonPlatform.Scopes
+
+ def GetName(self):
+ ''' Get the name of the repo, platform, or product being build '''
+ ''' Used for naming the log file, among others '''
+
+ # check the startup nsh flag and if set then rename the log file.
+ # this helps in CI so we don't overwrite the build log since running
+ # uses the stuart_build command.
+ if(shell_environment.GetBuildVars().GetValue("MAKE_STARTUP_NSH", "FALSE") == "TRUE"):
+ return "EmulatorPkg_With_Run"
+ return "EmulatorPkg"
+
+ def GetLoggingLevel(self, loggerType):
+ ''' Get the logging level for a given type
+ base == lowest logging level supported
+ con == Screen logging
+ txt == plain text file logging
+ md == markdown file logging
+ '''
+ return logging.DEBUG
+
+ def SetPlatformEnv(self):
+ logging.debug("PlatformBuilder SetPlatformEnv")
+ self.env.SetValue("PRODUCT_NAME", "EmulatorPkg", "Platform Hardcoded")
+ self.env.SetValue("TOOL_CHAIN_TAG", "VS2019", "Default Toolchain")
+
+ # Add support for using the correct Platform Headers, tools, and Libs based on emulator architecture
+ # requested to be built when building VS2019 or VS2017
+ if self.env.GetValue("TOOL_CHAIN_TAG") == "VS2019" or self.env.GetValue("TOOL_CHAIN_TAG") == "VS2017":
+ key = self.env.GetValue("TOOL_CHAIN_TAG") + "_HOST"
+ if self.env.GetValue("TARGET_ARCH") == "IA32":
+ shell_environment.ShellEnvironment().set_shell_var(key, "x86")
+ elif self.env.GetValue("TARGET_ARCH") == "X64":
+ shell_environment.ShellEnvironment().set_shell_var(key, "x64")
+
+ # Add support for using the correct Platform Headers, tools, and Libs based on emulator architecture
+ # requested to be built when building on linux.
+ if GetHostInfo().os.upper() == "LINUX":
+ self.ConfigureLinuxDLinkPath()
+
+ if GetHostInfo().os.upper() == "WINDOWS":
+ self.env.SetValue("BLD_*_WIN_HOST_BUILD", "TRUE",
+ "Trigger Windows host build")
+
+ self.env.SetValue("MAKE_STARTUP_NSH", "FALSE", "Default to false")
+
+ # I don't see what this does but it is in build.sh
+ key = "BLD_*_BUILD_" + self.env.GetValue("TARGET_ARCH")
+ self.env.SetValue(key, "TRUE", "match script in build.sh")
+ return 0
+
+ def PlatformPreBuild(self):
+ return 0
+
+ def PlatformPostBuild(self):
+ return 0
+
+ def FlashRomImage(self):
+ ''' Use the FlashRom Function to run the emulator. This gives an easy stuart command line to
+ activate the emulator. '''
+
+ OutputPath = os.path.join(self.env.GetValue(
+ "BUILD_OUTPUT_BASE"), self.env.GetValue("TARGET_ARCH"))
+
+ if (self.env.GetValue("MAKE_STARTUP_NSH") == "TRUE"):
+ f = open(os.path.join(OutputPath, "startup.nsh"), "w")
+ f.write("BOOT SUCCESS !!! \n")
+ # add commands here
+ f.write("reset\n")
+ f.close()
+
+ if GetHostInfo().os.upper() == "WINDOWS":
+ cmd = "WinHost.exe"
+ elif GetHostInfo().os.upper() == "LINUX":
+ cmd = "./Host"
+ else:
+ logging.critical("Unsupported Host")
+ return -1
+ return RunCmd(cmd, "", workingdir=OutputPath)
+
+ def ConfigureLinuxDLinkPath(self):
+ '''
+ logic copied from build.sh to setup the correct libraries
+ '''
+ if self.env.GetValue("TARGET_ARCH") == "IA32":
+ LIB_NAMES = ["ld-linux.so.2", "libdl.so.2 crt1.o", "crti.o crtn.o"]
+ LIB_SEARCH_PATHS = ["/usr/lib/i386-linux-gnu",
+ "/usr/lib32", "/lib32", "/usr/lib", "/lib"]
+ elif self.env.GetValue("TARGET_ARCH") == "X64":
+ LIB_NAMES = ["ld-linux-x86-64.so.2",
+ "libdl.so.2", "crt1.o", "crti.o", "crtn.o"]
+ LIB_SEARCH_PATHS = ["/usr/lib/x86_64-linux-gnu",
+ "/usr/lib64", "/lib64", "/usr/lib", "/lib"]
+
+ HOST_DLINK_PATHS = ""
+ for lname in LIB_NAMES:
+ logging.debug(f"Looking for {lname}")
+ for dname in LIB_SEARCH_PATHS:
+ logging.debug(f"In {dname}")
+ if os.path.isfile(os.path.join(dname, lname)):
+ logging.debug(f"Found {lname} in {dname}")
+ HOST_DLINK_PATHS += os.path.join(
+ os.path.join(dname, lname)) + os.pathsep
+ break
+ HOST_DLINK_PATHS = HOST_DLINK_PATHS.rstrip(os.pathsep)
+ logging.critical(f"Setting HOST_DLINK_PATHS to {HOST_DLINK_PATHS}")
+ shell_environment.ShellEnvironment().set_shell_var(
+ "HOST_DLINK_PATHS", HOST_DLINK_PATHS)
diff --git a/EmulatorPkg/PlatformCI/ReadMe.md b/EmulatorPkg/PlatformCI/ReadMe.md
new file mode 100644
index 000000000000..c51838ec1aa7
--- /dev/null
+++ b/EmulatorPkg/PlatformCI/ReadMe.md
@@ -0,0 +1,128 @@
+# EmulatorPkg - Platform CI
+
+This ReadMe.md describes the Azure DevOps based Platform CI for EmulatorPkg and how
+to use the same Pytools based build infrastructure locally.
+
+## Supported Configuration Details
+
+This solution for building and running EmulatorPkg has only been validated with Windows 10
+with VS2019 and Ubuntu 18.04 with GCC5 toolchain. Two different firmware builds are
+supported and are described below.
+
+| Configuration name | Architectures | DSC File |Additional Flags |
+| :---- | :----- | :---- | :---- |
+| IA32 | IA32 | EmulatorPkg.dsc | None |
+| X64 | X64 | EmulatorPkg.dsc | None |
+
+## EDK2 Developer environment
+
+- [Python 3.8.x - Download & Install](https://www.python.org/downloads/)
+- [GIT - Download & Install](https://git-scm.com/download/)
+- [Edk2 Source](https://github.com/tianocore/edk2)
+- For building Basetools and other host applications
+
+ ``` bash
+ sudo apt-get update
+ sudo apt-get install gcc g++ make uuid-dev
+ ```
+
+- For building ARCH IA32 on X64 Ubuntu 18.04 LTS these steps where needed.
+
+ ``` bash
+ sudo dpkg --add-architecture i386
+ sudo apt-get update
+ sudo apt-get install libc6-dev:i386 libx11-dev:i386 libxext-dev:i386 lib32gcc-7-dev
+ ```
+
+Note: edksetup, Submodule initialization and manual installation of NASM, iASL, or
+the required cross-compiler toolchains are **not** required, this is handled by the
+Pytools build system.
+
+## Building with Pytools for EmulatorPkg
+
+1. [Optional] Create a Python Virtual Environment - generally once per workspace
+
+ ``` bash
+ python -m venv <name of virtual environment>
+ ```
+
+2. [Optional] Activate Virtual Environment - each time new shell opened
+ - Linux
+
+ ```bash
+ source <name of virtual environment>/bin/activate
+ ```
+
+ - Windows
+
+ ``` bash
+ <name of virtual environment>/Scripts/activate.bat
+ ```
+
+3. Install Pytools - generally once per virtual env or whenever pip-requirements.txt changes
+
+ ``` bash
+ pip install --upgrade -r pip-requirements.txt
+ ```
+
+4. Initialize & Update Submodules - only when submodules updated
+
+ ``` bash
+ stuart_setup -c EmulatorPkg/PlatformCI/PlatformBuild.py TOOL_CHAIN_TAG=<TOOL_CHAIN_TAG> -a <TARGET_ARCH>
+ ```
+
+5. Initialize & Update Dependencies - only as needed when ext_deps change
+
+ ``` bash
+ stuart_update -c EmulatorPkg/PlatformCI/PlatformBuild.py TOOL_CHAIN_TAG=<TOOL_CHAIN_TAG> -a <TARGET_ARCH>
+ ```
+
+6. Compile the basetools if necessary - only when basetools C source files change
+
+ ``` bash
+ python BaseTools/Edk2ToolsBuild.py -t <ToolChainTag>
+ ```
+
+7. Compile Firmware
+
+ ``` bash
+ stuart_build -c EmulatorPkg/PlatformCI/PlatformBuild.py TOOL_CHAIN_TAG=<TOOL_CHAIN_TAG> -a <TARGET_ARCH>
+ ```
+
+ - use `stuart_build -c EmulatorPkg/PlatformCI/PlatformBuild.py -h` option to see additional
+ options like `--clean`
+
+8. Running Emulator
+ - You can add `--FlashRom` to the end of your build command and the emulator will run after the
+ build is complete.
+ - or use the `--FlashOnly` feature to just run the emulator.
+
+ ``` bash
+ stuart_build -c EmulatorPkg/PlatformCI/PlatformBuild.py TOOL_CHAIN_TAG=<TOOL_CHAIN_TAG> -a <TARGET_ARCH> --FlashOnly
+ ```
+
+### Notes
+
+1. Configuring *ACTIVE_PLATFORM* and *TARGET_ARCH* in Conf/target.txt is **not** required. This
+ environment is set by PlatformBuild.py based upon the `[-a <TARGET_ARCH>]` parameter.
+
+**NOTE:** Logging the execution output will be in the normal stuart log as well as to your console.
+
+### Custom Build Options
+
+**MAKE_STARTUP_NSH=TRUE** will output a *startup.nsh* file to the location mapped as fs0. This is
+used in CI in combination with the `--FlashOnly` feature to run the Emulator to the UEFI shell and then execute
+the contents of *startup.nsh*.
+
+### Passing Build Defines
+
+To pass build defines through _stuart_build_, prepend `BLD_*_`to the define name and pass it on the
+command-line. _stuart_build_ currently requires values to be assigned, so add an`=1` suffix for bare defines.
+For example, to enable the IP6 Network Stack, the stuart_build command-line would be:
+
+`stuart_build -c EmulatorPkg/PlatformCI/PlatformBuild.py BLD_*_NETWORK_IP6_ENABLE=1`
+
+## References
+
+- [Installing and using Pytools](https://github.com/tianocore/edk2-pytool-extensions/blob/master/docs/using.md#installing)
+- More on [python virtual environments](https://docs.python.org/3/library/venv.html)
--
2.16.3.windows.1
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH v3 4/7] OvmfPkg: Add Platform CI and configuration for Core CI
[not found] <20200424213108.19888-1-michael.kubacki@outlook.com>
` (2 preceding siblings ...)
2020-04-24 21:31 ` [PATCH v3 3/7] EmulatorPkg: " Michael Kubacki
@ 2020-04-24 21:31 ` Michael Kubacki
2020-04-24 21:31 ` [PATCH v3 5/7] .pytool: Update CI Settings to support Emulator, ArmVirt, and Ovmf packages Michael Kubacki
` (2 subsequent siblings)
6 siblings, 0 replies; 13+ messages in thread
From: Michael Kubacki @ 2020-04-24 21:31 UTC (permalink / raw)
To: devel; +Cc: Jordan Justen, Laszlo Ersek, Ard Biesheuvel
From: Sean Brogan <sean.brogan@microsoft.com>
REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2570
Add new Azure Pipeline definitions to build and run OvmfPkg with:
* Ubuntu GCC5
* Windows VS2019
Add PyTool based build of OvmfPkg
Add extdep for managing the iasl dependency
Add OvmfPkg.ci.yaml for Core CI
Add ReadMe.md for details and instructions
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
Signed-off-by: Sean Brogan <sean.brogan@microsoft.com>
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
Acked-by: Laszlo Ersek <lersek@redhat.com>
Acked-by: Ard Biesheuvel <ard.biesheuvel@arm.com>
Reviewed-by: Shenglei Zhang <shenglei.zhang@intel.com>
Reviewed-by: Bret Barkelew <bret.barkelew@microsoft.com>
---
OvmfPkg/OvmfPkg.ci.yaml | 83 +++++++
OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml | 133 ++++++++++
OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml | 138 +++++++++++
OvmfPkg/PlatformCI/PlatformBuild.py | 254 ++++++++++++++++++++
OvmfPkg/PlatformCI/ReadMe.md | 137 +++++++++++
OvmfPkg/PlatformCI/iasl_ext_dep.yaml | 21 ++
6 files changed, 766 insertions(+)
diff --git a/OvmfPkg/OvmfPkg.ci.yaml b/OvmfPkg/OvmfPkg.ci.yaml
new file mode 100644
index 000000000000..98992f0429ff
--- /dev/null
+++ b/OvmfPkg/OvmfPkg.ci.yaml
@@ -0,0 +1,83 @@
+## @file
+# Core CI configuration for OvmfPkg
+#
+# OvmfPkg is part of Platform Ci for builds so this is only
+# used for code analysis.
+#
+# Copyright (c) Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ ## options defined .pytool/Plugin/CompilerPlugin
+ "CompilerPlugin": {
+ "DscPath": "" # Don't support this test
+ },
+
+ ## options defined .pytool/Plugin/HostUnitTestCompilerPlugin
+ "HostUnitTestCompilerPlugin": {
+ "DscPath": "" # Don't support this test
+ },
+
+ ## options defined .pytool/Plugin/CharEncodingCheck
+ "CharEncodingCheck": {
+ "IgnoreFiles": []
+ },
+
+ ## options defined .pytool/Plugin/DependencyCheck
+ "DependencyCheck": {
+ "AcceptableDependencies": [
+ "MdePkg/MdePkg.dec",
+ "MdeModulePkg/MdeModulePkg.dec",
+ "OvmfPkg/OvmfPkg.dec",
+ "NetworkPkg/NetworkPkg.dec",
+ "SecurityPkg/SecurityPkg.dec",
+ "UefiCpuPkg/UefiCpuPkg.dec",
+ "ShellPkg/ShellPkg.dec",
+ "EmbeddedPkg/EmbeddedPkg.dec",
+ "SourceLevelDebugPkg/SourceLevelDebugPkg.dec"
+ ],
+ # For host based unit tests
+ "AcceptableDependencies-HOST_APPLICATION":[
+ "UnitTestFrameworkPkg/UnitTestFrameworkPkg.dec"
+ ],
+ # For UEFI shell based apps
+ "AcceptableDependencies-UEFI_APPLICATION":[],
+ "IgnoreInf": []
+ },
+
+ ## options defined .pytool/Plugin/DscCompleteCheck
+ "DscCompleteCheck": {
+ "IgnoreInf": [""],
+ "DscPath": "" # Don't support this test
+ },
+
+ ## options defined .pytool/Plugin/HostUnitTestDscCompleteCheck
+ "HostUnitTestDscCompleteCheck": {
+ "IgnoreInf": [""],
+ "DscPath": "" # Don't support this test
+ },
+
+ ## options defined .pytool/Plugin/GuidCheck
+ "GuidCheck": {
+ "IgnoreGuidName": ["ResetVector", "XenResetVector"], # Expected duplication for gEfiFirmwareVolumeTopFileGuid
+ "IgnoreGuidValue": [],
+ "IgnoreFoldersAndFiles": [],
+ "IgnoreDuplicates": [],
+ },
+
+ ## options defined .pytool/Plugin/LibraryClassCheck
+ "LibraryClassCheck": {
+ "IgnoreHeaderFile": []
+ },
+
+ ## options defined .pytool/Plugin/SpellCheck
+ "SpellCheck": {
+ "AuditOnly": True, # Fails right now with over 270 errors
+ "IgnoreFiles": [], # use gitignore syntax to ignore errors in matching files
+ "ExtendWords": [
+
+ ], # words to extend to the dictionary for this package
+ "IgnoreStandardPaths": [], # Standard Plugin defined paths that should be ignore
+ "AdditionalIncludePaths": [] # Additional paths to spell check (wildcards supported)
+ }
+}
diff --git a/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml b/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
new file mode 100644
index 000000000000..a47d273217ab
--- /dev/null
+++ b/OvmfPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
@@ -0,0 +1,133 @@
+## @file
+# Azure Pipeline build file for building a platform.
+#
+# Platform: OVMF
+# OS: Ubuntu
+# Toolchain: GCC5
+#
+# Copyright (c) Microsoft Corporation.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+trigger:
+ - master
+pr:
+ - master
+
+jobs:
+ - job: Platform_CI
+ variables:
+ package: 'OvmfPkg'
+ vm_image: 'ubuntu-latest'
+ should_run: true
+ run_flags: "MAKE_STARTUP_NSH=TRUE QEMU_HEADLESS=TRUE"
+
+ #Use matrix to speed up the build process
+ strategy:
+ matrix:
+ OVMF_IA32_DEBUG:
+ Build.File: "$(package)/PlatformCI/PlatformBuild.py"
+ Build.Arch: "IA32"
+ Build.Flags: ""
+ Build.Target: "DEBUG"
+ Run.Flags: $(run_flags)
+ Run: $(should_run)
+ OVMF_IA32_RELEASE:
+ Build.File: "$(package)/PlatformCI/PlatformBuild.py"
+ Build.Arch: "IA32"
+ Build.Flags: ""
+ Build.Target: "RELEASE"
+ Run.Flags: $(run_flags)
+ Run: $(should_run)
+ OVMF_IA32_NOOPT:
+ Build.File: "$(package)/PlatformCI/PlatformBuild.py"
+ Build.Arch: "IA32"
+ Build.Flags: ""
+ Build.Target: "NOOPT"
+ Run.Flags: $(run_flags)
+ Run: $(should_run)
+
+ OVMF_X64_DEBUG:
+ Build.File: "$(package)/PlatformCI/PlatformBuild.py"
+ Build.Arch: "X64"
+ Build.Flags: ""
+ Build.Target: "DEBUG"
+ Run.Flags: $(run_flags)
+ Run: $(should_run)
+ OVMF_X64_RELEASE:
+ Build.File: "$(package)/PlatformCI/PlatformBuild.py"
+ Build.Arch: "X64"
+ Build.Flags: ""
+ Build.Target: "RELEASE"
+ Run.Flags: $(run_flags)
+ Run: $(should_run)
+ OVMF_X64_NOOPT:
+ Build.File: "$(package)/PlatformCI/PlatformBuild.py"
+ Build.Arch: "X64"
+ Build.Flags: ""
+ Build.Target: "NOOPT"
+ Run.Flags: $(run_flags)
+ Run: $(should_run)
+
+ OVMF_IA32X64_DEBUG:
+ Build.File: "$(package)/PlatformCI/PlatformBuild.py"
+ Build.Arch: "IA32,X64"
+ Build.Flags: ""
+ Build.Target: "DEBUG"
+ Run.Flags: $(run_flags)
+ Run: $(should_run)
+ OVMF_IA32X64_RELEASE:
+ Build.File: "$(package)/PlatformCI/PlatformBuild.py"
+ Build.Arch: "IA32,X64"
+ Build.Flags: ""
+ Build.Target: "RELEASE"
+ Run.Flags: $(run_flags)
+ Run: $(should_run)
+ OVMF_IA32X64_NOOPT:
+ Build.File: "$(package)/PlatformCI/PlatformBuild.py"
+ Build.Arch: "IA32,X64"
+ Build.Flags: ""
+ Build.Target: "NOOPT"
+ Run.Flags: $(run_flags)
+ Run: $(should_run)
+
+ OVMF_IA32X64_FULL_DEBUG:
+ Build.File: "$(package)/PlatformCI/PlatformBuild.py"
+ Build.Arch: "IA32,X64"
+ Build.Flags: "BLD_*_SECURE_BOOT_ENABLE=1 BLD_*_SMM_REQUIRE=1 BLD_*_TPM_ENABLE=1 BLD_*_TPM_CONFIG_ENABLE=1 BLD_*_NETWORK_TLS_ENABLE=1 BLD_*_NETWORK_IP6_ENABLE=1 BLD_*_NETWORK_HTTP_BOOT_ENABLE=1"
+ Build.Target: "DEBUG"
+ Run.Flags: $(run_flags)
+ Run: $(should_run)
+ OVMF_IA32X64_FULL_RELEASE:
+ Build.File: "$(package)/PlatformCI/PlatformBuild.py"
+ Build.Arch: "IA32,X64"
+ Build.Flags: "BLD_*_SECURE_BOOT_ENABLE=1 BLD_*_SMM_REQUIRE=1 BLD_*_TPM_ENABLE=1 BLD_*_TPM_CONFIG_ENABLE=1 BLD_*_NETWORK_TLS_ENABLE=1 BLD_*_NETWORK_IP6_ENABLE=1 BLD_*_NETWORK_HTTP_BOOT_ENABLE=1"
+ Build.Target: "RELEASE"
+ Run.Flags: $(run_flags)
+ Run: $(should_run)
+ OVMF_IA32X64_FULL_NOOPT:
+ Build.File: "$(package)/PlatformCI/PlatformBuild.py"
+ Build.Arch: "IA32,X64"
+ Build.Flags: "BLD_*_SECURE_BOOT_ENABLE=1 BLD_*_SMM_REQUIRE=1 BLD_*_TPM_ENABLE=1 BLD_*_TPM_CONFIG_ENABLE=1 BLD_*_NETWORK_TLS_ENABLE=1 BLD_*_NETWORK_IP6_ENABLE=1 BLD_*_NETWORK_HTTP_BOOT_ENABLE=1"
+ Build.Target: "NOOPT"
+ Run.Flags: $(run_flags)
+ Run: $(should_run)
+ workspace:
+ clean: all
+
+ pool:
+ vmImage: $(vm_image)
+
+ steps:
+ - template: ../../../.azurepipelines/templates/platform-build-run-steps.yml
+ parameters:
+ tool_chain_tag: GCC5
+ build_pkg: $(package)
+ build_target: $(Build.Target)
+ build_arch: $(Build.Arch)
+ build_file: $(Build.File)
+ build_flags: $(Build.Flags)
+ run_flags: $(Run.Flags)
+ extra_install_step:
+ - bash: sudo apt-get install qemu
+ displayName: Install qemu
+ condition: and(gt(variables.pkg_count, 0), succeeded())
diff --git a/OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml b/OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml
new file mode 100644
index 000000000000..02ed233fdb60
--- /dev/null
+++ b/OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml
@@ -0,0 +1,138 @@
+## @file
+# Azure Pipeline build file for building a platform.
+#
+# Platform: OVMF
+# OS: Windows
+# Toolchain: VS2019
+#
+# Copyright (c) Microsoft Corporation.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+trigger:
+ - master
+pr:
+ - master
+jobs:
+ - job: Platform_CI
+ variables:
+ package: 'OvmfPkg'
+ vm_image: 'windows-latest'
+ should_run: true
+ run_flags: "MAKE_STARTUP_NSH=TRUE QEMU_HEADLESS=TRUE"
+
+ #Use matrix to speed up the build process
+ strategy:
+ matrix:
+ OVMF_IA32_DEBUG:
+ Build.File: "$(package)/PlatformCI/PlatformBuild.py"
+ Build.Arch: "IA32"
+ Build.Flags: ""
+ Build.Target: "DEBUG"
+ Run.Flags: $(run_flags)
+ Run: $(should_run)
+ OVMF_IA32_RELEASE:
+ Build.File: "$(package)/PlatformCI/PlatformBuild.py"
+ Build.Arch: "IA32"
+ Build.Flags: ""
+ Build.Target: "RELEASE"
+ Run.Flags: $(run_flags)
+ Run: $(should_run)
+ OVMF_IA32_NOOPT:
+ Build.File: "$(package)/PlatformCI/PlatformBuild.py"
+ Build.Arch: "IA32"
+ Build.Flags: ""
+ Build.Target: "NOOPT"
+ Run.Flags: $(run_flags)
+ Run: $(should_run)
+
+ OVMF_X64_DEBUG:
+ Build.File: "$(package)/PlatformCI/PlatformBuild.py"
+ Build.Arch: "X64"
+ Build.Flags: ""
+ Build.Target: "DEBUG"
+ Run.Flags: $(run_flags)
+ Run: $(should_run)
+ OVMF_X64_RELEASE:
+ Build.File: "$(package)/PlatformCI/PlatformBuild.py"
+ Build.Arch: "X64"
+ Build.Flags: ""
+ Build.Target: "RELEASE"
+ Run.Flags: $(run_flags)
+ Run: $(should_run)
+ OVMF_X64_NOOPT:
+ Build.File: "$(package)/PlatformCI/PlatformBuild.py"
+ Build.Arch: "X64"
+ Build.Flags: ""
+ Build.Target: "NOOPT"
+ Run.Flags: $(run_flags)
+ Run: $(should_run)
+
+ OVMF_IA32X64_DEBUG:
+ Build.File: "$(package)/PlatformCI/PlatformBuild.py"
+ Build.Arch: "IA32,X64"
+ Build.Flags: ""
+ Build.Target: "DEBUG"
+ Run.Flags: $(run_flags)
+ Run: $(should_run)
+ OVMF_IA32X64_RELEASE:
+ Build.File: "$(package)/PlatformCI/PlatformBuild.py"
+ Build.Arch: "IA32,X64"
+ Build.Flags: ""
+ Build.Target: "RELEASE"
+ Run.Flags: $(run_flags)
+ Run: $(should_run)
+ OVMF_IA32X64_NOOPT:
+ Build.File: "$(package)/PlatformCI/PlatformBuild.py"
+ Build.Arch: "IA32,X64"
+ Build.Flags: ""
+ Build.Target: "NOOPT"
+ Run.Flags: $(run_flags)
+ Run: $(should_run)
+
+ OVMF_IA32X64_FULL_DEBUG:
+ Build.File: "$(package)/PlatformCI/PlatformBuild.py"
+ Build.Arch: "IA32,X64"
+ Build.Flags: "BLD_*_SECURE_BOOT_ENABLE=1 BLD_*_SMM_REQUIRE=1 BLD_*_TPM_ENABLE=1 BLD_*_TPM_CONFIG_ENABLE=1 BLD_*_NETWORK_TLS_ENABLE=1 BLD_*_NETWORK_IP6_ENABLE=1 BLD_*_NETWORK_HTTP_BOOT_ENABLE=1"
+ Build.Target: "DEBUG"
+ Run.Flags: $(run_flags)
+ Run: $(should_run)
+ OVMF_IA32X64_FULL_RELEASE:
+ Build.File: "$(package)/PlatformCI/PlatformBuild.py"
+ Build.Arch: "IA32,X64"
+ Build.Flags: "BLD_*_SECURE_BOOT_ENABLE=1 BLD_*_SMM_REQUIRE=1 BLD_*_TPM_ENABLE=1 BLD_*_TPM_CONFIG_ENABLE=1 BLD_*_NETWORK_TLS_ENABLE=1 BLD_*_NETWORK_IP6_ENABLE=1 BLD_*_NETWORK_HTTP_BOOT_ENABLE=1"
+ Build.Target: "RELEASE"
+ Run.Flags: $(run_flags)
+ Run: $(should_run)
+
+ # This currently creates a very large image which is too big for the FDF declared range
+ # Ovmf maintainers suggest to skip this build for now.
+ #
+ # OVMF_IA32X64_FULL_NOOPT:
+ # Build.File: "$(package)/PlatformCI/PlatformBuild.py"
+ # Build.Arch: "IA32,X64"
+ # Build.Flags: "BLD_*_SECURE_BOOT_ENABLE=1 BLD_*_SMM_REQUIRE=1 BLD_*_TPM_ENABLE=1 BLD_*_TPM_CONFIG_ENABLE=1 BLD_*_NETWORK_TLS_ENABLE=1 BLD_*_NETWORK_IP6_ENABLE=1 BLD_*_NETWORK_HTTP_BOOT_ENABLE=1"
+ # Build.Target: "NOOPT"
+ # Run.Flags: $(run_flags)
+ # Run: $(should_run)
+
+ workspace:
+ clean: all
+
+ pool:
+ vmImage: $(vm_image)
+
+ steps:
+ - template: ../../../.azurepipelines/templates/platform-build-run-steps.yml
+ parameters:
+ tool_chain_tag: VS2019
+ build_pkg: $(package)
+ build_target: $(Build.Target)
+ build_arch: $(Build.Arch)
+ build_file: $(Build.File)
+ build_flags: $(Build.Flags)
+ run_flags: $(Run.Flags)
+ extra_install_step:
+ - powershell: choco install qemu; Write-Host "##vso[task.prependpath]c:\Program Files\qemu"
+ displayName: Install QEMU and Set QEMU on path # friendly name displayed in the UI
+ condition: and(gt(variables.pkg_count, 0), succeeded())
+
diff --git a/OvmfPkg/PlatformCI/PlatformBuild.py b/OvmfPkg/PlatformCI/PlatformBuild.py
new file mode 100644
index 000000000000..627bb7b992db
--- /dev/null
+++ b/OvmfPkg/PlatformCI/PlatformBuild.py
@@ -0,0 +1,254 @@
+# @file
+# Script to Build OVMF UEFI firmware
+#
+# Copyright (c) Microsoft Corporation.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+import os
+import logging
+import io
+
+from edk2toolext.environment import shell_environment
+from edk2toolext.environment.uefi_build import UefiBuilder
+from edk2toolext.invocables.edk2_platform_build import BuildSettingsManager
+from edk2toolext.invocables.edk2_setup import SetupSettingsManager, RequiredSubmodule
+from edk2toolext.invocables.edk2_update import UpdateSettingsManager
+from edk2toolext.invocables.edk2_pr_eval import PrEvalSettingsManager
+from edk2toollib.utility_functions import RunCmd
+
+
+ # ####################################################################################### #
+ # Common Configuration #
+ # ####################################################################################### #
+class CommonPlatform():
+ ''' Common settings for this platform. Define static data here and use
+ for the different parts of stuart
+ '''
+ PackagesSupported = ("OvmfPkg",)
+ ArchSupported = ("IA32", "X64")
+ TargetsSupported = ("DEBUG", "RELEASE", "NOOPT")
+ Scopes = ('ovmf', 'edk2-build')
+ WorkspaceRoot = os.path.realpath(os.path.join(
+ os.path.dirname(os.path.abspath(__file__)), "..", ".."))
+
+ @classmethod
+ def GetDscName(cls, ArchCsv: str) -> str:
+ ''' return the DSC given the architectures requested.
+
+ ArchCsv: csv string containing all architectures to build
+ '''
+ dsc = "OvmfPkg"
+ if "IA32" in ArchCsv.upper().split(","):
+ dsc += "Ia32"
+ if "X64" in ArchCsv.upper().split(","):
+ dsc += "X64"
+ dsc += ".dsc"
+ return dsc
+
+
+ # ####################################################################################### #
+ # Configuration for Update & Setup #
+ # ####################################################################################### #
+class SettingsManager(UpdateSettingsManager, SetupSettingsManager, PrEvalSettingsManager):
+
+ def GetPackagesSupported(self):
+ ''' return iterable of edk2 packages supported by this build.
+ These should be edk2 workspace relative paths '''
+ return CommonPlatform.PackagesSupported
+
+ def GetArchitecturesSupported(self):
+ ''' return iterable of edk2 architectures supported by this build '''
+ return CommonPlatform.ArchSupported
+
+ def GetTargetsSupported(self):
+ ''' return iterable of edk2 target tags supported by this build '''
+ return CommonPlatform.TargetsSupported
+
+ def GetRequiredSubmodules(self):
+ ''' return iterable containing RequiredSubmodule objects.
+ If no RequiredSubmodules return an empty iterable
+ '''
+ rs = []
+
+ # intentionally declare this one with recursive false to avoid overhead
+ rs.append(RequiredSubmodule(
+ "CryptoPkg/Library/OpensslLib/openssl", False))
+
+ # To avoid maintenance of this file for every new submodule
+ # lets just parse the .gitmodules and add each if not already in list.
+ # The GetRequiredSubmodules is designed to allow a build to optimize
+ # the desired submodules but it isn't necessary for this repository.
+ result = io.StringIO()
+ ret = RunCmd("git", "config --file .gitmodules --get-regexp path", workingdir=self.GetWorkspaceRoot(), outstream=result)
+ # Cmd output is expected to look like:
+ # submodule.CryptoPkg/Library/OpensslLib/openssl.path CryptoPkg/Library/OpensslLib/openssl
+ # submodule.SoftFloat.path ArmPkg/Library/ArmSoftFloatLib/berkeley-softfloat-3
+ if ret == 0:
+ for line in result.getvalue().splitlines():
+ _, _, path = line.partition(" ")
+ if path is not None:
+ if path not in [x.path for x in rs]:
+ rs.append(RequiredSubmodule(path, True)) # add it with recursive since we don't know
+ return rs
+
+ def SetArchitectures(self, list_of_requested_architectures):
+ ''' Confirm the requests architecture list is valid and configure SettingsManager
+ to run only the requested architectures.
+
+ Raise Exception if a list_of_requested_architectures is not supported
+ '''
+ unsupported = set(list_of_requested_architectures) - set(self.GetArchitecturesSupported())
+ if(len(unsupported) > 0):
+ errorString = ( "Unsupported Architecture Requested: " + " ".join(unsupported))
+ logging.critical( errorString )
+ raise Exception( errorString )
+ self.ActualArchitectures = list_of_requested_architectures
+
+ def GetWorkspaceRoot(self):
+ ''' get WorkspacePath '''
+ return CommonPlatform.WorkspaceRoot
+
+ def GetActiveScopes(self):
+ ''' return tuple containing scopes that should be active for this process '''
+ return CommonPlatform.Scopes
+
+ def FilterPackagesToTest(self, changedFilesList: list, potentialPackagesList: list) -> list:
+ ''' Filter other cases that this package should be built
+ based on changed files. This should cover things that can't
+ be detected as dependencies. '''
+ build_these_packages = []
+ possible_packages = potentialPackagesList.copy()
+ for f in changedFilesList:
+ # BaseTools files that might change the build
+ if "BaseTools" in f:
+ if os.path.splitext(f) not in [".txt", ".md"]:
+ build_these_packages = possible_packages
+ break
+
+ # if the azure pipeline platform template file changed
+ if "platform-build-run-steps.yml" in f:
+ build_these_packages = possible_packages
+ break
+
+ return build_these_packages
+
+ def GetPlatformDscAndConfig(self) -> tuple:
+ ''' If a platform desires to provide its DSC then Policy 4 will evaluate if
+ any of the changes will be built in the dsc.
+
+ The tuple should be (<workspace relative path to dsc file>, <input dictionary of dsc key value pairs>)
+ '''
+ dsc = CommonPlatform.GetDscName(",".join(self.ActualArchitectures))
+ return (f"OvmfPkg/{dsc}", {})
+
+
+ # ####################################################################################### #
+ # Actual Configuration for Platform Build #
+ # ####################################################################################### #
+class PlatformBuilder( UefiBuilder, BuildSettingsManager):
+ def __init__(self):
+ UefiBuilder.__init__(self)
+
+ def AddCommandLineOptions(self, parserObj):
+ ''' Add command line options to the argparser '''
+ parserObj.add_argument('-a', "--arch", dest="build_arch", type=str, default="IA32,X64",
+ help="Optional - CSV of architecture to build. IA32 will use IA32 for Pei & Dxe. "
+ "X64 will use X64 for both PEI and DXE. IA32,X64 will use IA32 for PEI and "
+ "X64 for DXE. default is IA32,X64")
+
+ def RetrieveCommandLineOptions(self, args):
+ ''' Retrieve command line options from the argparser '''
+
+ shell_environment.GetBuildVars().SetValue("TARGET_ARCH"," ".join(args.build_arch.upper().split(",")), "From CmdLine")
+ dsc = CommonPlatform.GetDscName(args.build_arch)
+ shell_environment.GetBuildVars().SetValue("ACTIVE_PLATFORM", f"OvmfPkg/{dsc}", "From CmdLine")
+
+ def GetWorkspaceRoot(self):
+ ''' get WorkspacePath '''
+ return CommonPlatform.WorkspaceRoot
+
+ def GetPackagesPath(self):
+ ''' Return a list of workspace relative paths that should be mapped as edk2 PackagesPath '''
+ return ()
+
+ def GetActiveScopes(self):
+ ''' return tuple containing scopes that should be active for this process '''
+ return CommonPlatform.Scopes
+
+ def GetName(self):
+ ''' Get the name of the repo, platform, or product being build '''
+ ''' Used for naming the log file, among others '''
+ # check the startup nsh flag and if set then rename the log file.
+ # this helps in CI so we don't overwrite the build log since running
+ # uses the stuart_build command.
+ if(shell_environment.GetBuildVars().GetValue("MAKE_STARTUP_NSH", "FALSE") == "TRUE"):
+ return "OvmfPkg_With_Run"
+ return "OvmfPkg"
+
+ def GetLoggingLevel(self, loggerType):
+ ''' Get the logging level for a given type
+ base == lowest logging level supported
+ con == Screen logging
+ txt == plain text file logging
+ md == markdown file logging
+ '''
+ return logging.DEBUG
+
+ def SetPlatformEnv(self):
+ logging.debug("PlatformBuilder SetPlatformEnv")
+ self.env.SetValue("PRODUCT_NAME", "OVMF", "Platform Hardcoded")
+ self.env.SetValue("MAKE_STARTUP_NSH", "FALSE", "Default to false")
+ self.env.SetValue("QEMU_HEADLESS", "FALSE", "Default to false")
+ return 0
+
+ def PlatformPreBuild(self):
+ return 0
+
+ def PlatformPostBuild(self):
+ return 0
+
+ def FlashRomImage(self):
+ VirtualDrive = os.path.join(self.env.GetValue("BUILD_OUTPUT_BASE"), "VirtualDrive")
+ os.makedirs(VirtualDrive, exist_ok=True)
+ OutputPath_FV = os.path.join(self.env.GetValue("BUILD_OUTPUT_BASE"), "FV")
+
+ #
+ # QEMU must be on the path
+ #
+ cmd = "qemu-system-x86_64"
+ args = "-debugcon stdio" # write messages to stdio
+ args += " -global isa-debugcon.iobase=0x402" # debug messages out thru virtual io port
+ args += " -net none" # turn off network
+ args += f" -drive file=fat:rw:{VirtualDrive},format=raw,media=disk" # Mount disk with startup.nsh
+
+ if (self.env.GetValue("QEMU_HEADLESS").upper() == "TRUE"):
+ args += " -display none" # no graphics
+
+ if (self.env.GetBuildValue("SMM_REQUIRE") == "1"):
+ args += " -machine q35,smm=on" #,accel=(tcg|kvm)"
+ #args += " -m ..."
+ #args += " -smp ..."
+ args += " -global driver=cfi.pflash01,property=secure,value=on"
+ args += " -drive if=pflash,format=raw,unit=0,file=" + os.path.join(OutputPath_FV, "OVMF_CODE.fd") + ",readonly=on"
+ args += " -drive if=pflash,format=raw,unit=1,file=" + os.path.join(OutputPath_FV, "OVMF_VARS.fd")
+ else:
+ args += " -pflash " + os.path.join(OutputPath_FV, "OVMF.fd") # path to firmware
+
+
+ if (self.env.GetValue("MAKE_STARTUP_NSH").upper() == "TRUE"):
+ f = open(os.path.join(VirtualDrive, "startup.nsh"), "w")
+ f.write("BOOT SUCCESS !!! \n")
+ ## add commands here
+ f.write("reset -s\n")
+ f.close()
+
+ ret = RunCmd(cmd, args)
+
+ if ret == 0xc0000005:
+ #for some reason getting a c0000005 on successful return
+ return 0
+
+ return ret
+
+
+
diff --git a/OvmfPkg/PlatformCI/ReadMe.md b/OvmfPkg/PlatformCI/ReadMe.md
new file mode 100644
index 000000000000..2ce9007dbeaa
--- /dev/null
+++ b/OvmfPkg/PlatformCI/ReadMe.md
@@ -0,0 +1,137 @@
+# OvmfPkg - Platform CI
+
+This ReadMe.md describes the Azure DevOps based Platform CI for OvmfPkg and how
+to use the same Pytools based build infrastructure locally.
+
+## Supported Configuration Details
+
+This solution for building and running OvmfPkg has only been validated with Windows 10
+with VS2019 and Ubuntu 18.04 with GCC5 toolchain. Four different firmware builds are
+supported and are described below.
+
+| Configuration name | Architectures | DSC File |Additional Flags |
+| :---- | :----- | :---- | :---- |
+| IA32 | IA32 | OvmfPkgIa32.dsc | None |
+| X64 | X64 | OvmfPkgIa64.dsc | None |
+| IA32 X64 | PEI-IA32 DXE-X64 | OvmfPkgIa32X64.dsc | None |
+| IA32 X64 Full | PEI-IA32 DXE-X64 | OvmfPkgIa32X64.dsc | SECURE_BOOT_ENABLE=1 SMM_REQUIRE=1 TPM_ENABLE=1 TPM_CONFIG_ENABLE=1 NETWORK_TLS_ENABLE=1 NETWORK_IP6_ENABLE=1 NETWORK_HTTP_BOOT_ENABLE=1 |
+
+## EDK2 Developer environment
+
+- [Python 3.8.x - Download & Install](https://www.python.org/downloads/)
+- [GIT - Download & Install](https://git-scm.com/download/)
+- [QEMU - Download, Install, and add to your path](https://www.qemu.org/download/)
+- [Edk2 Source](https://github.com/tianocore/edk2)
+- Additional packages found necessary for Ubuntu 18.04
+ - apt-get install gcc g++ make uuid-dev
+
+Note: edksetup, Submodule initialization and manual installation of NASM, iASL, or
+the required cross-compiler toolchains are **not** required, this is handled by the
+Pytools build system.
+
+## Building with Pytools for OvmfPkg
+
+1. [Optional] Create a Python Virtual Environment - generally once per workspace
+
+ ``` bash
+ python -m venv <name of virtual environment>
+ ```
+
+2. [Optional] Activate Virtual Environment - each time new shell opened
+ - Linux
+
+ ```bash
+ source <name of virtual environment>/bin/activate
+ ```
+
+ - Windows
+
+ ``` bash
+ <name of virtual environment>/Scripts/activate.bat
+ ```
+
+3. Install Pytools - generally once per virtual env or whenever pip-requirements.txt changes
+
+ ``` bash
+ pip install --upgrade -r pip-requirements.txt
+ ```
+
+4. Initialize & Update Submodules - only when submodules updated
+
+ ``` bash
+ stuart_setup -c OvmfPkg/PlatformCI/PlatformBuild.py TOOL_CHAIN_TAG=<TOOL_CHAIN_TAG> -a <TARGET_ARCH>
+ ```
+
+5. Initialize & Update Dependencies - only as needed when ext_deps change
+
+ ``` bash
+ stuart_update -c OvmfPkg/PlatformCI/PlatformBuild.py TOOL_CHAIN_TAG=<TOOL_CHAIN_TAG> -a <TARGET_ARCH>
+ ```
+
+6. Compile the basetools if necessary - only when basetools C source files change
+
+ ``` bash
+ python BaseTools/Edk2ToolsBuild.py -t <ToolChainTag>
+ ```
+
+7. Compile Firmware
+ - To build IA32
+
+ ``` bash
+ stuart_build -c OvmfPkg/PlatformCI/PlatformBuild.py -a IA32 TOOL_CHAIN_TAG=<TOOL_CHAIN_TAG>
+ ```
+
+ - To build X64
+
+ ``` bash
+ stuart_build -c OvmfPkg/PlatformCI/PlatformBuild.py -a X64 TOOL_CHAIN_TAG=<TOOL_CHAIN_TAG>
+ ```
+
+ - To build IA32 X64
+
+ ``` bash
+ stuart_build -c OvmfPkg/PlatformCI/PlatformBuild.py -a IA32,X64 TOOL_CHAIN_TAG=<TOOL_CHAIN_TAG>
+ ```
+
+ - use `stuart_build -c OvmfPkg/PlatformCI/PlatformBuild.py -h` option to see additional
+ options like `--clean`
+
+8. Running Emulator
+ - You can add `--FlashRom` to the end of your build command and the emulator will run after the
+ build is complete.
+ - or use the `--FlashOnly` feature to just run the emulator.
+
+ ``` bash
+ stuart_build -c OvmfPkg/PlatformCI/PlatformBuild.py TOOL_CHAIN_TAG=<TOOL_CHAIN_TAG> -a <TARGET_ARCH> --FlashOnly
+ ```
+
+### Notes
+
+1. Configuring *ACTIVE_PLATFORM* and *TARGET_ARCH* in Conf/target.txt is **not** required. This
+ environment is set by PlatformBuild.py based upon the `[-a <TARGET_ARCH>]` parameter.
+2. QEMU must be on your path. On Windows this is a manual process and not part of the QEMU installer.
+
+**NOTE:** Logging the execution output will be in the normal stuart log as well as to your console.
+
+### Custom Build Options
+
+**MAKE_STARTUP_NSH=TRUE** will output a *startup.nsh* file to the location mapped as fs0. This is
+used in CI in combination with the `--FlashOnly` feature to run QEMU to the UEFI shell and then execute
+the contents of *startup.nsh*.
+
+**QEMU_HEADLESS=TRUE** Since CI servers run headless QEMU must be told to run with no display otherwise
+an error occurs. Locally you don't need to set this.
+
+### Passing Build Defines
+
+To pass build defines through _stuart_build_, prepend `BLD_*_`to the define name and pass it on the
+command-line. _stuart_build_ currently requires values to be assigned, so add an`=1` suffix for bare defines.
+For example, to enable the TPM2 support, instead of the traditional "-D E1000_ENABLE", the stuart_build
+command-line would be:
+
+`stuart_build -c OvmfPkg/PlatformCI/PlatformBuild.py BLD_*_E1000_ENABLE=1`
+
+## References
+
+- [Installing and using Pytools](https://github.com/tianocore/edk2-pytool-extensions/blob/master/docs/using.md#installing)
+- More on [python virtual environments](https://docs.python.org/3/library/venv.html)
diff --git a/OvmfPkg/PlatformCI/iasl_ext_dep.yaml b/OvmfPkg/PlatformCI/iasl_ext_dep.yaml
new file mode 100644
index 000000000000..cbee0e5a5a92
--- /dev/null
+++ b/OvmfPkg/PlatformCI/iasl_ext_dep.yaml
@@ -0,0 +1,21 @@
+## @file
+# Download iasl executable tool from a nuget.org package
+# - package contains different binaries based on host
+# Add the folder with the tool to the path
+#
+# This is only downloaded for scope ovmf thus
+# should have no impact on the asl compiler used by any
+# other given platform to build.
+#
+# Copyright (c) Microsoft Corporation.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "id": "iasl-ovmf-1",
+ "scope": "ovmf",
+ "type": "nuget",
+ "name": "iasl",
+ "source": "https://api.nuget.org/v3/index.json",
+ "version": "20190215.0.0",
+ "flags": ["set_path", "host_specific"],
+}
--
2.16.3.windows.1
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH v3 5/7] .pytool: Update CI Settings to support Emulator, ArmVirt, and Ovmf packages
[not found] <20200424213108.19888-1-michael.kubacki@outlook.com>
` (3 preceding siblings ...)
2020-04-24 21:31 ` [PATCH v3 4/7] OvmfPkg: " Michael Kubacki
@ 2020-04-24 21:31 ` Michael Kubacki
2020-04-24 21:31 ` [PATCH v3 6/7] .azurepipelines: Update Core CI build matrix to include platforms Michael Kubacki
2020-04-24 21:31 ` [PATCH v3 7/7] ReadMe: Convert to rst and add Platform CI Status Michael Kubacki
6 siblings, 0 replies; 13+ messages in thread
From: Michael Kubacki @ 2020-04-24 21:31 UTC (permalink / raw)
To: devel; +Cc: Sean Brogan, Bret Barkelew, Michael D Kinney, Liming Gao
From: Sean Brogan <sean.brogan@microsoft.com>
REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2570
Update pytools configuration to enable EmulatorPkg, ArmVirtPkg,
and OvmfPkg.
Update documentation for the status of those packages.
Remove future work items that are now complete.
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Sean Brogan <sean.brogan@microsoft.com>
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
Reviewed-by: Shenglei Zhang <shenglei.zhang@intel.com>
Reviewed-by: Bret Barkelew <bret.barkelew@microsoft.com>
---
.pytool/CISettings.py | 7 +++++--
.pytool/Plugin/SpellCheck/cspell.base.yaml | 14 ++++++++++++++
.pytool/Readme.md | 10 ++++------
3 files changed, 23 insertions(+), 8 deletions(-)
diff --git a/.pytool/CISettings.py b/.pytool/CISettings.py
index e373d17a6c34..7a8bd3380383 100644
--- a/.pytool/CISettings.py
+++ b/.pytool/CISettings.py
@@ -39,7 +39,9 @@ class Settings(CiBuildSettingsManager, UpdateSettingsManager, SetupSettingsManag
''' return iterable of edk2 packages supported by this build.
These should be edk2 workspace relative paths '''
- return ("MdePkg",
+ return ("ArmVirtPkg",
+ "EmulatorPkg",
+ "MdePkg",
"MdeModulePkg",
"NetworkPkg",
"PcAtChipsetPkg",
@@ -49,7 +51,8 @@ class Settings(CiBuildSettingsManager, UpdateSettingsManager, SetupSettingsManag
"ShellPkg",
"FatPkg",
"CryptoPkg",
- "UnitTestFrameworkPkg"
+ "UnitTestFrameworkPkg",
+ "OvmfPkg"
)
def GetArchitecturesSupported(self):
diff --git a/.pytool/Plugin/SpellCheck/cspell.base.yaml b/.pytool/Plugin/SpellCheck/cspell.base.yaml
index aa15170c013c..1ce5f6991104 100644
--- a/.pytool/Plugin/SpellCheck/cspell.base.yaml
+++ b/.pytool/Plugin/SpellCheck/cspell.base.yaml
@@ -164,5 +164,19 @@
"Sdhci",
"inmodule",
"RISCV",
+ "edksetup",
+ "iscsi",
+ "nvdata",
+ "pytools",
+ "NTDDI",
+ "Wnonportable",
+ "CLANGPDB",
+ "nologo",
+ "lldmap",
+ "ASMLINK",
+ "NODEFAULTLIB",
+ "vcruntimed",
+ "ucrtd",
+ "msvcrtd",
]
}
diff --git a/.pytool/Readme.md b/.pytool/Readme.md
index 135d283fe223..c7dce3b64ca0 100644
--- a/.pytool/Readme.md
+++ b/.pytool/Readme.md
@@ -6,11 +6,11 @@
| :---- | :----- | :---- | :--- |
| ArmPkg |
| ArmPlatformPkg |
-| ArmVirtPkg |
+| ArmVirtPkg | SEE PACKAGE README | SEE PACKAGE README |
| CryptoPkg | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode
| DynamicTablesPkg |
| EmbeddedPkg |
-| EmulatorPkg |
+| EmulatorPkg | SEE PACKAGE README | SEE PACKAGE README | Spell checking in audit mode
| FatPkg | :heavy_check_mark: | :heavy_check_mark: |
| FmpDevicePkg | :heavy_check_mark: | :heavy_check_mark: |
| IntelFsp2Pkg |
@@ -18,7 +18,7 @@
| MdeModulePkg | :heavy_check_mark: | :heavy_check_mark: | DxeIpl dependency on ArmPkg, Depends on StandaloneMmPkg, Spell checking in audit mode
| MdePkg | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode
| NetworkPkg | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode
-| OvmfPkg |
+| OvmfPkg | SEE PACKAGE README | SEE PACKAGE README | Spell checking in audit mode
| PcAtChipsetPkg | :heavy_check_mark: | :heavy_check_mark: |
| SecurityPkg | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode
| ShellPkg | :heavy_check_mark: | :heavy_check_mark: | Spell checking in audit mode, 3 modules are not being built by DSC
@@ -77,7 +77,7 @@ per package configuration which comes from this file.
## Running CI locally
The EDKII Tools environment (and by extension the ci) is designed to support
-easily and consistantly running locally and in a cloud ci environment. To do
+easily and consistently running locally and in a cloud ci environment. To do
that a few steps should be followed. Details of EDKII Tools can be found in the
[docs folder here](https://github.com/tianocore/edk2-pytool-extensions/tree/master/docs)
@@ -216,8 +216,6 @@ few standard scopes.
* Visual Studio AARCH64 and ARM support
* BaseTools C tools CI/PR and binary release process
* BaseTools Python tools CI/PR process
-* Host based unit testing
* Extensible private/closed source platform reporting
-* Platform builds, validation
* UEFI SCTs
* Other automation
--
2.16.3.windows.1
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH v3 6/7] .azurepipelines: Update Core CI build matrix to include platforms
[not found] <20200424213108.19888-1-michael.kubacki@outlook.com>
` (4 preceding siblings ...)
2020-04-24 21:31 ` [PATCH v3 5/7] .pytool: Update CI Settings to support Emulator, ArmVirt, and Ovmf packages Michael Kubacki
@ 2020-04-24 21:31 ` Michael Kubacki
2020-04-24 21:31 ` [PATCH v3 7/7] ReadMe: Convert to rst and add Platform CI Status Michael Kubacki
6 siblings, 0 replies; 13+ messages in thread
From: Michael Kubacki @ 2020-04-24 21:31 UTC (permalink / raw)
To: devel; +Cc: Sean Brogan, Bret Barkelew, Michael D Kinney, Liming Gao
From: Sean Brogan <sean.brogan@microsoft.com>
REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2570
Add ArmVirtPkg to Core CI matrix
Add EmulatorPkg to Core CI matrix
Add OvmfPkg to Core CI matrix
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Sean Brogan <sean.brogan@microsoft.com>
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
Reviewed-by: Shenglei Zhang <shenglei.zhang@intel.com>
Reviewed-by: Bret Barkelew <bret.barkelew@microsoft.com>
---
.azurepipelines/templates/pr-gate-build-job.yml | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/.azurepipelines/templates/pr-gate-build-job.yml b/.azurepipelines/templates/pr-gate-build-job.yml
index 61868554d43c..a9f89aa68451 100644
--- a/.azurepipelines/templates/pr-gate-build-job.yml
+++ b/.azurepipelines/templates/pr-gate-build-job.yml
@@ -44,6 +44,11 @@ jobs:
TARGET_SECURITY:
Build.Pkgs: 'SecurityPkg'
Build.Targets: 'DEBUG,RELEASE,NO-TARGET'
+ TARGET_PLATFORMS:
+ # For Platforms only check code. Leave it to Platform CI
+ # to build them.
+ Build.Pkgs: 'ArmVirtPkg,EmulatorPkg,OvmfPkg'
+ Build.Targets: 'NO-TARGET'
workspace:
clean: all
--
2.16.3.windows.1
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH v3 7/7] ReadMe: Convert to rst and add Platform CI Status
[not found] <20200424213108.19888-1-michael.kubacki@outlook.com>
` (5 preceding siblings ...)
2020-04-24 21:31 ` [PATCH v3 6/7] .azurepipelines: Update Core CI build matrix to include platforms Michael Kubacki
@ 2020-04-24 21:31 ` Michael Kubacki
2020-04-27 12:30 ` Leif Lindholm
6 siblings, 1 reply; 13+ messages in thread
From: Michael Kubacki @ 2020-04-24 21:31 UTC (permalink / raw)
To: devel; +Cc: Andrew Fish, Laszlo Ersek, Leif Lindholm, Michael D Kinney
From: Sean Brogan <sean.brogan@microsoft.com>
REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2570
Convert workspace root Readme.md to reStructuredText
ReStructuredText supports building tables with syntax that is easier
to read in plain-text.
Rename as ReadMe.rst.
Add Platform CI Status Tables
Cc: Andrew Fish <afish@apple.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Leif Lindholm <leif@nuviainc.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Signed-off-by: Sean Brogan <sean.brogan@microsoft.com>
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Bret Barkelew <bret.barkelew@microsoft.com>
---
ReadMe.rst | 354 ++++++++++++++++++++
Readme.md | 235 -------------
2 files changed, 354 insertions(+), 235 deletions(-)
diff --git a/ReadMe.rst b/ReadMe.rst
new file mode 100644
index 000000000000..c4a29b2809ba
--- /dev/null
+++ b/ReadMe.rst
@@ -0,0 +1,354 @@
+==============
+EDK II Project
+==============
+
+A modern, feature-rich, cross-platform firmware development
+environment for the UEFI and PI specifications from www.uefi.org.
+
+Core CI Build Status
+--------------------
+
+============================= ================= =============== ===================
+ Host Type & Toolchain Build Status Test Status Code Coverage
+============================= ================= =============== ===================
+Windows_VS2019_ |WindowsCiBuild| |WindowsCiTest| |WindowsCiCoverage|
+Ubuntu_GCC5_ |UbuntuCiBuild| |UbuntuCiTest| |UbuntuCiCoverage|
+============================= ================= =============== ===================
+
+`More CI Build information <.pytool/Readme.md>`__
+
+Platform CI Build Status
+------------------------
+
+Microsoft Windows VS2019
+````````````````````````
+
+============================= ================= ============= ============= ==============
+ Toolchain CONFIG DEBUG RELEASE NOOPT
+============================= ================= ============= ============= ==============
+EmulatorPkg_Win_VS2019_ | IA32 |em32d| |em32r| |em32n|
+| | X64 |em64d| |em64r| |em64n|
+OvmfPkg_Win_VS2019_ | IA32 |op32d| |op32r| |op32n|
+| | X64 |op64d| |op64r| |op64n|
+| | IA32 X64 |op3264d| |op3264r| |op3264n|
+| | IA32 X64 FULL |op3264fd| |op3264fr| |op3264fn|
+============================= ================= ============= ============= ==============
+
+Ubuntu 18.04 GCC5
+`````````````````
+
+============================= ================= ============= ============= ==============
+ Toolchain CONFIG DEBUG RELEASE NOOPT
+============================= ================= ============= ============= ==============
+ArmVirtPkg_Ubuntu_GCC5_ | AARCH64 |avAArch64du| |avAArch64ru| |avAArch64nu|
+| | ARM |avArmdu| |avArmru| |avArmnu|
+EmulatorPkg_Ubuntu_GCC5_ | IA32 |em32du| |em32ru| |em32nu|
+| | X64 |em64du| |em64ru| |em64nu|
+OvmfPkg_Ubuntu_GCC5_ | IA32 |op32du| |op32ru| |op32nu|
+| | X64 |op64du| |op64ru| |op64nu|
+| | IA32 X64 |op3264du| |op3264ru| |op3264nu|
+| | IA32 X64 FULL |op3264fdu| |op3264fru| |op3264fru|
+============================= ================= ============= ============= ==============
+
+|TCBZ_2668|_ - EmulatorPkg Ubuntu GCC5 Segfaults during execution.
+
+|TCBZ_2639|_ - EmulatorPkg Ubuntu GCC5 Segfaults during execution.
+
+`More ArmVirtPkg CI Build Information <ArmVirtPkg/PlatformCI/ReadMe.md>`__
+
+`More EmulatorPkg CI Build Information <EmulatorPkg/PlatformCI/ReadMe.md>`__
+
+`More OvmfPkg CI Build Information <OvmfPkg/PlatformCI/ReadMe.md>`__
+
+
+License Details
+---------------
+
+The majority of the content in the EDK II open source project uses a
+`BSD-2-Clause Plus Patent License <License.txt>`__. The EDK II open
+source project contains the following components that are covered by additional
+licenses:
+
+- `BaseTools/Source/C/BrotliCompress/brotli <https://github.com/google/brotli/blob/master/LICENSE>`__
+- `MdeModulePkg/Library/BrotliCustomDecompressLib/brotli <https://github.com/google/brotli/blob/master/LICENSE>`__
+- `BaseTools/Source/C/LzmaCompress <BaseTools/Source/C/LzmaCompress/LZMA-SDK-README.txt>`__
+- `MdeModulePkg/Library/LzmaCustomDecompressLib <MdeModulePkg/Library/LzmaCustomDecompressLib/LZMA-SDK-README.txt>`__
+- `IntelFrameworkModulePkg/Library/LzmaCustomDecompressLib/Sdk <IntelFrameworkModulePkg/Library/LzmaCustomDecompressLib/LZMA-SDK-README.txt>`__
+- `BaseTools/Source/C/VfrCompile/Pccts <BaseTools/Source/C/VfrCompile/Pccts/RIGHTS>`__
+- `MdeModulePkg/Universal/RegularExpressionDxe/oniguruma <https://github.com/kkos/oniguruma/blob/master/README.md>`__
+- `OvmfPkg <OvmfPkg/License.txt>`__
+- `CryptoPkg/Library/OpensslLib/openssl <https://github.com/openssl/openssl/blob/50eaac9f3337667259de725451f201e784599687/LICENSE>`__
+- `ArmPkg/Library/ArmSoftFloatLib/berkeley-softfloat-3 <https://github.com/ucb-bar/berkeley-softfloat-3/blob/b64af41c3276f97f0e181920400ee056b9c88037/COPYING.txt>`__
+
+The EDK II Project is composed of packages. The maintainers for each package
+are listed in `Maintainers.txt <Maintainers.txt>`__.
+
+Resources
+---------
+
+- `TianoCore <http://www.tianocore.org>`__
+- `EDK
+ II <https://github.com/tianocore/tianocore.github.io/wiki/EDK-II>`__
+- `Getting Started with EDK
+ II <https://github.com/tianocore/tianocore.github.io/wiki/Getting-Started-with-EDK-II>`__
+- `Mailing
+ Lists <https://github.com/tianocore/tianocore.github.io/wiki/Mailing-Lists>`__
+- `TianoCore Bugzilla <https://bugzilla.tianocore.org>`__
+- `How To
+ Contribute <https://github.com/tianocore/tianocore.github.io/wiki/How-To-Contribute>`__
+- `Release
+ Planning <https://github.com/tianocore/tianocore.github.io/wiki/EDK-II-Release-Planning>`__
+
+Code Contributions
+------------------
+
+To make a contribution to a TianoCore project, follow these steps.
+
+#. Create a change description in the format specified below to
+ use in the source control commit log.
+#. Your commit message must include your ``Signed-off-by`` signature
+#. Submit your code to the TianoCore project using the process
+ that the project documents on its web page. If the process is
+ not documented, then submit the code on development email list
+ for the project.
+#. It is preferred that contributions are submitted using the same
+ copyright license as the base project. When that is not possible,
+ then contributions using the following licenses can be accepted:
+
+- BSD (2-clause): http://opensource.org/licenses/BSD-2-Clause
+- BSD (3-clause): http://opensource.org/licenses/BSD-3-Clause
+- MIT: http://opensource.org/licenses/MIT
+- Python-2.0: http://opensource.org/licenses/Python-2.0
+- Zlib: http://opensource.org/licenses/Zlib
+
+For documentation:
+
+- FreeBSD Documentation License
+ https://www.freebsd.org/copyright/freebsd-doc-license.html
+
+Contributions of code put into the public domain can also be accepted.
+
+Contributions using other licenses might be accepted, but further
+review will be required.
+
+Developer Certificate of Origin
+-------------------------------
+
+Your change description should use the standard format for a
+commit message, and must include your ``Signed-off-by`` signature.
+
+In order to keep track of who did what, all patches contributed must
+include a statement that to the best of the contributor's knowledge
+they have the right to contribute it under the specified license.
+
+The test for this is as specified in the `Developer's Certificate of
+Origin (DCO) 1.1 <https://developercertificate.org/>`__. The contributor
+certifies compliance by adding a line saying
+
+Signed-off-by: Developer Name developer@example.org
+
+where ``Developer Name`` is the contributor's real name, and the email
+address is one the developer is reachable through at the time of
+contributing.
+
+::
+
+ Developer's Certificate of Origin 1.1
+
+ By making a contribution to this project, I certify that:
+
+ (a) The contribution was created in whole or in part by me and I
+ have the right to submit it under the open source license
+ indicated in the file; or
+
+ (b) The contribution is based upon previous work that, to the best
+ of my knowledge, is covered under an appropriate open source
+ license and I have the right under that license to submit that
+ work with modifications, whether created in whole or in part
+ by me, under the same open source license (unless I am
+ permitted to submit under a different license), as indicated
+ in the file; or
+
+ (c) The contribution was provided directly to me by some other
+ person who certified (a), (b) or (c) and I have not modified
+ it.
+
+ (d) I understand and agree that this project and the contribution
+ are public and that a record of the contribution (including all
+ personal information I submit with it, including my sign-off) is
+ maintained indefinitely and may be redistributed consistent with
+ this project or the open source license(s) involved.
+
+Sample Change Description / Commit Message
+------------------------------------------
+
+::
+
+ From: Contributor Name <contributor@example.com>
+ Subject: [Repository/Branch PATCH] Pkg-Module: Brief-single-line-summary
+
+ Full-commit-message
+
+ Signed-off-by: Contributor Name <contributor@example.com>
+
+Notes for sample patch email
+````````````````````````````
+
+- The first line of commit message is taken from the email's subject
+ line following ``[Repository/Branch PATCH]``. The remaining portion
+ of the commit message is the email's content.
+- ``git format-patch`` is one way to create this format
+
+Definitions for sample patch email
+``````````````````````````````````
+
+- ``Repository`` is the identifier of the repository the patch applies.
+ This identifier should only be provided for repositories other than
+ ``edk2``. For example ``edk2-BuildSpecification`` or ``staging``.
+- ``Branch`` is the identifier of the branch the patch applies. This
+ identifier should only be provided for branches other than
+ ``edk2/master``.
+ For example ``edk2/UDK2015``,
+ ``edk2-BuildSpecification/release/1.27``, or
+ ``staging/edk2-test``.
+- ``Module`` is a short identifier for the affected code or
+ documentation. For example ``MdePkg``, ``MdeModulePkg/UsbBusDxe``, ``Introduction``, or
+ ``EDK II INF File Format``.
+- ``Brief-single-line-summary`` is a short summary of the change.
+- The entire first line should be less than ~70 characters.
+- ``Full-commit-message`` a verbose multiple line comment describing
+ the change. Each line should be less than ~70 characters.
+- ``Signed-off-by`` is the contributor's signature identifying them
+ by their real/legal name and their email address.
+
+Submodules
+----------
+
+Submodule in EDK II is allowed but submodule chain should be avoided
+as possible as we can. Currently EDK II contains the following submodules
+
+- CryptoPkg/Library/OpensslLib/openssl
+- ArmPkg/Library/ArmSoftFloatLib/berkeley-softfloat-3
+- MdeModulePkg/Universal/RegularExpressionDxe/oniguruma
+- MdeModulePkg/Library/BrotliCustomDecompressLib/brotli
+- BaseTools/Source/C/BrotliCompress/brotli
+
+ArmSoftFloatLib is actually required by OpensslLib. It's inevitable
+in openssl-1.1.1 (since stable201905) for floating point parameter
+conversion, but should be dropped once there's no such need in future
+release of openssl.
+
+To get a full, buildable EDK II repository, use following steps of git
+command
+
+.. code-block:: bash
+
+ git clone https://github.com/tianocore/edk2.git
+ cd edk2
+ git submodule update --init
+ cd ..
+
+If there's update for submodules, use following git commands to get
+the latest submodules code.
+
+.. code-block:: bash
+
+ cd edk2
+ git pull
+ git submodule update
+
+Note: When cloning submodule repos, '--recursive' option is not
+recommended. EDK II itself will not use any code/feature from
+submodules in above submodules. So using '--recursive' adds a
+dependency on being able to reach servers we do not actually want
+any code from, as well as needlessly downloading code we will not
+use.
+
+.. ===================================================================
+.. This is a bunch of directives to make the README file more readable
+.. ===================================================================
+
+.. CoreCI
+
+.. _Windows_VS2019: https://dev.azure.com/tianocore/edk2-ci/_build/latest?definitionId=32&branchName=master
+.. |WindowsCiBuild| image:: https://dev.azure.com/tianocore/edk2-ci/_apis/build/status/Windows%20VS2019%20CI?branchName=master
+.. |WindowsCiTest| image:: https://img.shields.io/azure-devops/tests/tianocore/edk2-ci/32.svg
+.. |WindowsCiCoverage| image:: https://img.shields.io/badge/coverage-coming_soon-blue
+
+.. _Ubuntu_GCC5: https://dev.azure.com/tianocore/edk2-ci/_build/latest?definitionId=31&branchName=master
+.. |UbuntuCiBuild| image:: https://dev.azure.com/tianocore/edk2-ci/_apis/build/status/Ubuntu%20GCC5%20CI?branchName=master
+.. |UbuntuCiTest| image:: https://img.shields.io/azure-devops/tests/tianocore/edk2-ci/31.svg
+.. |UbuntuCiCoverage| image:: https://img.shields.io/badge/coverage-coming_soon-blue
+
+.. ArmVirtPkg
+
+.. _ArmVirtPkg_Ubuntu_GCC5: https://dev.azure.com/tianocore/edk2-ci-play/_build/latest?definitionId=41&branchName=master
+.. |avAArch64du| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/ArmVirtPkg/ArmVirtQemu%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20QEMU_AARCH64_DEBUG
+.. |avAArch64ru| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/ArmVirtPkg/ArmVirtQemu%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20QEMU_AARCH64_RELEASE
+.. |avAArch64nu| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/ArmVirtPkg/ArmVirtQemu%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20QEMU_AARCH64_NOOPT
+
+.. |avArmdu| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/ArmVirtPkg/ArmVirtQemu%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20QEMU_ARM_DEBUG
+.. |avArmru| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/ArmVirtPkg/ArmVirtQemu%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20QEMU_ARM_RELEASE
+.. |avArmnu| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/ArmVirtPkg/ArmVirtQemu%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20QEMU_ARM_NOOPT
+
+.. EmulatorPkg
+
+.. |TCBZ_2668| image:: https://img.shields.io/bugzilla/2668?baseUrl=https%3A%2F%2Fbugzilla.tianocore.org
+.. _TCBZ_2668: https://bugzilla.tianocore.org/show_bug.cgi?id=2668
+
+.. |TCBZ_2639| image:: https://img.shields.io/bugzilla/2639?baseUrl=https%3A%2F%2Fbugzilla.tianocore.org
+.. _TCBZ_2639: https://bugzilla.tianocore.org/show_bug.cgi?id=2639
+
+.. _EmulatorPkg_Win_VS2019: https://dev.azure.com/tianocore/edk2-ci-play/_build/latest?definitionId=40&branchName=master
+.. _EmulatorPkg_Ubuntu_GCC5: https://dev.azure.com/tianocore/edk2-ci-play/_build/latest?definitionId=39&branchName=master
+
+.. |em32d| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/EmulatorPkg/EmulatorPkg%20Windows%20VS2019?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20EmulatorPkg_IA32_DEBUG
+.. |em32du| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/EmulatorPkg/EmulatorPkg%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20EmulatorPkg_IA32_DEBUG
+.. |em32r| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/EmulatorPkg/EmulatorPkg%20Windows%20VS2019?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20EmulatorPkg_IA32_RELEASE
+.. |em32ru| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/EmulatorPkg/EmulatorPkg%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20EmulatorPkg_IA32_RELEASE
+.. |em32n| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/EmulatorPkg/EmulatorPkg%20Windows%20VS2019?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20EmulatorPkg_IA32_NOOPT
+.. |em32nu| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/EmulatorPkg/EmulatorPkg%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20EmulatorPkg_IA32_NOOPT
+
+.. |em64d| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/EmulatorPkg/EmulatorPkg%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20EmulatorPkg_X64_DEBUG
+.. |em64du| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/EmulatorPkg/EmulatorPkg%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20EmulatorPkg_X64_DEBUG
+.. |em64r| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/EmulatorPkg/EmulatorPkg%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20EmulatorPkg_X64_RELEASE
+.. |em64ru| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/EmulatorPkg/EmulatorPkg%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20EmulatorPkg_X64_RELEASE
+.. |em64n| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/EmulatorPkg/EmulatorPkg%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20EmulatorPkg_X64_NOOPT
+.. |em64nu| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/EmulatorPkg/EmulatorPkg%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20EmulatorPkg_X64_NOOPT
+
+.. OvmfPkg
+
+.. |TCBZ_2661| image:: https://img.shields.io/bugzilla/2661?baseUrl=https%3A%2F%2Fbugzilla.tianocore.org
+.. _TCBZ_2661: https://bugzilla.tianocore.org/show_bug.cgi?id=2661
+
+.. _OvmfPkg_Win_VS2019: https://dev.azure.com/tianocore/edk2-ci-play/_build/latest?definitionId=38&branchName=master/
+.. _OvmfPkg_Ubuntu_GCC5: https://dev.azure.com/tianocore/edk2-ci-play/_build/latest?definitionId=37&branchName=master
+
+.. |op32d| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/OVMF/OVMF%20Windows%20VS2019?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20OVMF_IA32_DEBUG
+.. |op32du| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/OVMF/OVMF%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20OVMF_IA32_DEBUG
+.. |op32r| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/OVMF/OVMF%20Windows%20VS2019?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20OVMF_IA32_RELEASE
+.. |op32ru| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/OVMF/OVMF%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20OVMF_IA32_RELEASE
+.. |op32n| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/OVMF/OVMF%20Windows%20VS2019?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20OVMF_IA32_NOOPT
+.. |op32nu| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/OVMF/OVMF%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20OVMF_IA32_NOOPT
+
+.. |op64d| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/OVMF/OVMF%20Windows%20VS2019?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20OVMF_X64_DEBUG
+.. |op64du| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/OVMF/OVMF%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20OVMF_X64_DEBUG
+.. |op64r| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/OVMF/OVMF%20Windows%20VS2019?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20OVMF_X64_RELEASE
+.. |op64ru| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/OVMF/OVMF%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20OVMF_X64_RELEASE
+.. |op64n| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/OVMF/OVMF%20Windows%20VS2019?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20OVMF_X64_NOOPT
+.. |op64nu| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/OVMF/OVMF%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20OVMF_X64_NOOPT
+
+
+.. |op3264d| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/OVMF/OVMF%20Windows%20VS2019?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20OVMF_IA32X64_DEBUG
+.. |op3264du| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/OVMF/OVMF%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20OVMF_IA32X64_DEBUG
+.. |op3264r| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/OVMF/OVMF%20Windows%20VS2019?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20OVMF_IA32X64_RELEASE
+.. |op3264ru| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/OVMF/OVMF%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20OVMF_IA32X64_RELEASE
+.. |op3264n| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/OVMF/OVMF%20Windows%20VS2019?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20OVMF_IA32X64_NOOPT
+.. |op3264nu| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/OVMF/OVMF%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20OVMF_IA32X64_NOOPT
+
+.. |op3264fd| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/OVMF/OVMF%20Windows%20VS2019?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20OVMF_IA32X64_FULL_DEBUG
+.. |op3264fdu| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/OVMF/OVMF%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20OVMF_IA32X64_FULL_DEBUG
+.. |op3264fr| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/OVMF/OVMF%20Windows%20VS2019?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20OVMF_IA32X64_FULL_RELEASE
+.. |op3264fru| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/OVMF/OVMF%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20OVMF_IA32X64_FULL_RELEASE
+.. |op3264fn| replace:: |TCBZ_2661|_
+.. |op3264fnu| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/OVMF/OVMF%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20OVMF_IA32X64_FULL_NOOPT
diff --git a/Readme.md b/Readme.md
deleted file mode 100644
index 843046365425..000000000000
--- a/Readme.md
+++ /dev/null
@@ -1,235 +0,0 @@
-# EDK II Project
-
-A modern, feature-rich, cross-platform firmware development environment
-for the UEFI and PI specifications from www.uefi.org.
-
-# Build Status
-
-<table>
- <tr>
- <th>Host Type</th>
- <th>Toolchain</th>
- <th>Branch</th>
- <th>Build Status</th>
- <th>Test Status</th>
- <th>Code Coverage</th>
- </tr>
- <tr>
- <td>Windows</td>
- <td>VS2019</td>
- <td>master</td>
- <td>
- <a href="https://dev.azure.com/tianocore/edk2-ci/_build/latest?definitionId=32&branchName=master">
- <img src="https://dev.azure.com/tianocore/edk2-ci/_apis/build/status/Windows%20VS2019%20CI?branchName=master"/></a>
- </td>
- <td>
- <a href="https://dev.azure.com/tianocore/edk2-ci/_build/latest?definitionId=32&branchName=master">
- <img src="https://img.shields.io/azure-devops/tests/tianocore/edk2-ci/32.svg"/></a>
- </td>
- <td>
- <a href="https://dev.azure.com/tianocore/edk2-ci/_build/latest?definitionId=32&branchName=master">
- <img src="https://img.shields.io/badge/coverage-coming_soon-blue"/></a>
- </td>
- </tr>
- <tr>
- <td>Ubuntu</td>
- <td>GCC</td>
- <td>master</td>
- <td>
- <a href="https://dev.azure.com/tianocore/edk2-ci/_build/latest?definitionId=31&branchName=master">
- <img src="https://dev.azure.com/tianocore/edk2-ci/_apis/build/status/Ubuntu%20GCC5%20CI?branchName=master"/></a>
- </td>
- <td>
- <a href="https://dev.azure.com/tianocore/edk2-ci/_build/latest?definitionId=31&branchName=master">
- <img src="https://img.shields.io/azure-devops/tests/tianocore/edk2-ci/31.svg"/></a>
- </td>
- <td>
- <a href="https://dev.azure.com/tianocore/edk2-ci/_build/latest?definitionId=31&branchName=master">
- <img src="https://img.shields.io/badge/coverage-coming_soon-blue"/></a>
- </td>
- </tr>
-</table>
-
-[More CI Build information](.pytool/Readme.md)
-
-# License Details
-
-The majority of the content in the EDK II open source project uses a
-[BSD-2-Clause Plus Patent License](License.txt). The EDK II open source project
-contains the following components that are covered by additional licenses:
-* [BaseTools/Source/C/BrotliCompress/brotli](https://github.com/google/brotli/blob/master/LICENSE)
-* [MdeModulePkg/Library/BrotliCustomDecompressLib/brotli](https://github.com/google/brotli/blob/master/LICENSE)
-* [BaseTools/Source/C/LzmaCompress](BaseTools/Source/C/LzmaCompress/LZMA-SDK-README.txt)
-* [MdeModulePkg/Library/LzmaCustomDecompressLib](MdeModulePkg/Library/LzmaCustomDecompressLib/LZMA-SDK-README.txt)
-* [IntelFrameworkModulePkg/Library/LzmaCustomDecompressLib/Sdk](IntelFrameworkModulePkg/Library/LzmaCustomDecompressLib/LZMA-SDK-README.txt)
-* [BaseTools/Source/C/VfrCompile/Pccts](BaseTools/Source/C/VfrCompile/Pccts/RIGHTS)
-* [MdeModulePkg/Universal/RegularExpressionDxe/oniguruma](https://github.com/kkos/oniguruma/blob/master/README.md)
-* [OvmfPkg](OvmfPkg/License.txt)
-* [CryptoPkg/Library/OpensslLib/openssl](https://github.com/openssl/openssl/blob/50eaac9f3337667259de725451f201e784599687/LICENSE)
-* [ArmPkg/Library/ArmSoftFloatLib/berkeley-softfloat-3](https://github.com/ucb-bar/berkeley-softfloat-3/blob/b64af41c3276f97f0e181920400ee056b9c88037/COPYING.txt)
-
-The EDK II Project is composed of packages. The maintainers for each package
-are listed in [Maintainers.txt](Maintainers.txt).
-
-# Resources
-* [TianoCore](http://www.tianocore.org)
-* [EDK II](https://github.com/tianocore/tianocore.github.io/wiki/EDK-II)
-* [Getting Started with EDK II](https://github.com/tianocore/tianocore.github.io/wiki/Getting-Started-with-EDK-II)
-* [Mailing Lists](https://github.com/tianocore/tianocore.github.io/wiki/Mailing-Lists)
-* [TianoCore Bugzilla](https://bugzilla.tianocore.org)
-* [How To Contribute](https://github.com/tianocore/tianocore.github.io/wiki/How-To-Contribute)
-* [Release Planning](https://github.com/tianocore/tianocore.github.io/wiki/EDK-II-Release-Planning)
-
-# Code Contributions
-To make a contribution to a TianoCore project, follow these steps.
-1. Create a change description in the format specified below to
- use in the source control commit log.
-2. Your commit message must include your `Signed-off-by` signature
-3. Submit your code to the TianoCore project using the process
- that the project documents on its web page. If the process is
- not documented, then submit the code on development email list
- for the project.
-4. It is preferred that contributions are submitted using the same
- copyright license as the base project. When that is not possible,
- then contributions using the following licenses can be accepted:
- * BSD (2-clause): http://opensource.org/licenses/BSD-2-Clause
- * BSD (3-clause): http://opensource.org/licenses/BSD-3-Clause
- * MIT: http://opensource.org/licenses/MIT
- * Python-2.0: http://opensource.org/licenses/Python-2.0
- * Zlib: http://opensource.org/licenses/Zlib
-
- For documentation:
- * FreeBSD Documentation License
- https://www.freebsd.org/copyright/freebsd-doc-license.html
-
- Contributions of code put into the public domain can also be
- accepted.
-
- Contributions using other licenses might be accepted, but further
- review will be required.
-
-# Developer Certificate of Origin
-
-Your change description should use the standard format for a
-commit message, and must include your `Signed-off-by` signature.
-
-In order to keep track of who did what, all patches contributed must
-include a statement that to the best of the contributor's knowledge
-they have the right to contribute it under the specified license.
-
-The test for this is as specified in the [Developer's Certificate of
-Origin (DCO) 1.1](https://developercertificate.org/). The contributor
-certifies compliance by adding a line saying
-
- Signed-off-by: Developer Name <developer@example.org>
-
-where `Developer Name` is the contributor's real name, and the email
-address is one the developer is reachable through at the time of
-contributing.
-
-```
-Developer's Certificate of Origin 1.1
-
-By making a contribution to this project, I certify that:
-
-(a) The contribution was created in whole or in part by me and I
- have the right to submit it under the open source license
- indicated in the file; or
-
-(b) The contribution is based upon previous work that, to the best
- of my knowledge, is covered under an appropriate open source
- license and I have the right under that license to submit that
- work with modifications, whether created in whole or in part
- by me, under the same open source license (unless I am
- permitted to submit under a different license), as indicated
- in the file; or
-
-(c) The contribution was provided directly to me by some other
- person who certified (a), (b) or (c) and I have not modified
- it.
-
-(d) I understand and agree that this project and the contribution
- are public and that a record of the contribution (including all
- personal information I submit with it, including my sign-off) is
- maintained indefinitely and may be redistributed consistent with
- this project or the open source license(s) involved.
-```
-
-# Sample Change Description / Commit Message
-
-```
-From: Contributor Name <contributor@example.com>
-Subject: [Repository/Branch PATCH] Pkg-Module: Brief-single-line-summary
-
-Full-commit-message
-
-Signed-off-by: Contributor Name <contributor@example.com>
-```
-
-## Notes for sample patch email
-
-* The first line of commit message is taken from the email's subject
- line following `[Repository/Branch PATCH]`. The remaining portion of the
- commit message is the email's content.
-* `git format-patch` is one way to create this format
-
-## Definitions for sample patch email
-
-* `Repository` is the identifier of the repository the patch applies.
- This identifier should only be provided for repositories other than
- `edk2`. For example `edk2-BuildSpecification` or `staging`.
-* `Branch` is the identifier of the branch the patch applies. This
- identifier should only be provided for branches other than `edk2/master`.
- For example `edk2/UDK2015`, `edk2-BuildSpecification/release/1.27`, or
- `staging/edk2-test`.
-* `Module` is a short identifier for the affected code or documentation. For
- example `MdePkg`, `MdeModulePkg/UsbBusDxe`, `Introduction`, or
- `EDK II INF File Format`.
-* `Brief-single-line-summary` is a short summary of the change.
-* The entire first line should be less than ~70 characters.
-* `Full-commit-message` a verbose multiple line comment describing
- the change. Each line should be less than ~70 characters.
-* `Signed-off-by` is the contributor's signature identifying them
- by their real/legal name and their email address.
-
-# Submodules
-
-Submodule in EDK II is allowed but submodule chain should be avoided
-as possible as we can. Currently EDK II contains the following submodules
-
-- CryptoPkg/Library/OpensslLib/openssl
-- ArmPkg/Library/ArmSoftFloatLib/berkeley-softfloat-3
-- MdeModulePkg/Universal/RegularExpressionDxe/oniguruma
-- MdeModulePkg/Library/BrotliCustomDecompressLib/brotli
-- BaseTools/Source/C/BrotliCompress/brotli
-
-ArmSoftFloatLib is actually required by OpensslLib. It's inevitable
-in openssl-1.1.1 (since stable201905) for floating point parameter
-conversion, but should be dropped once there's no such need in future
-release of openssl.
-
-To get a full, buildable EDK II repository, use following steps of git
-command
-
-```
-$ git clone https://github.com/tianocore/edk2.git
-$ cd edk2
-$ git submodule update --init
-$ cd ..
-```
-
-If there's update for submodules, use following git commands to get the
-latest submodules code.
-
-```
-$ cd edk2
-$ git pull
-$ git submodule update
-```
-
-Note: When cloning submodule repos, '--recursive' option is not
-recommended. EDK II itself will not use any code/feature from
-submodules in above submodules. So using '--recursive' adds a
-dependency on being able to reach servers we do not actually want
-any code from, as well as needlessly downloading code we will not
-use.
--
2.16.3.windows.1
^ permalink raw reply related [flat|nested] 13+ messages in thread
* Re: [PATCH v3 2/7] ArmVirtPkg: Add Platform CI and configuration for Core CI
2020-04-24 21:31 ` [PATCH v3 2/7] ArmVirtPkg: Add Platform CI and configuration for Core CI Michael Kubacki
@ 2020-04-27 12:22 ` Leif Lindholm
2020-04-27 21:21 ` Michael Kubacki
0 siblings, 1 reply; 13+ messages in thread
From: Leif Lindholm @ 2020-04-27 12:22 UTC (permalink / raw)
To: michael.kubacki; +Cc: devel, Laszlo Ersek, Ard Biesheuvel
On Fri, Apr 24, 2020 at 14:31:03 -0700, michael.kubacki@outlook.com wrote:
> From: Sean Brogan <sean.brogan@microsoft.com>
>
> REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2570
>
> Add new Azure Pipeline definitions to build and run ArmVirtPkg with:
> * Ubuntu GCC5
> Add PyTool based build of ArmVirtPkg
> Add extdep for managing the iasl dependency
> Add ArmVirtPkg.ci.yaml for Core CI
> Add ReadMe.md for details and instructions
>
> Cc: Laszlo Ersek <lersek@redhat.com>
> Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
> Cc: Leif Lindholm <leif@nuviainc.com>
> Signed-off-by: Sean Brogan <sean.brogan@microsoft.com>
> Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
The contributor attests the compliance of the submission with
https://developercertificate.org/ at the point of contribution by
using Signed-off-by.
Statements by others (including original authors) should not be copied
when copying code between projects. The From: tag preserves authorship
credit.
/
Leif
> Reviewed-by: Shenglei Zhang <shenglei.zhang@intel.com>
> Acked-by: Laszlo Ersek <lersek@redhat.com>
> Acked-by: Ard Biesheuvel <ard.biesheuvel@arm.com>
> Reviewed-by: Bret Barkelew <bret.barkelew@microsoft.com>
> ---
> ArmVirtPkg/ArmVirtPkg.ci.yaml | 103 ++++++++
> ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml | 89 +++++++
> ArmVirtPkg/PlatformCI/PlatformBuild.py | 276 ++++++++++++++++++++
> ArmVirtPkg/PlatformCI/ReadMe.md | 125 +++++++++
> ArmVirtPkg/PlatformCI/iasl_ext_dep.yaml | 21 ++
> 5 files changed, 614 insertions(+)
>
> diff --git a/ArmVirtPkg/ArmVirtPkg.ci.yaml b/ArmVirtPkg/ArmVirtPkg.ci.yaml
> new file mode 100644
> index 000000000000..4553725ee528
> --- /dev/null
> +++ b/ArmVirtPkg/ArmVirtPkg.ci.yaml
> @@ -0,0 +1,103 @@
> +## @file
> +# Core CI configuration for ArmVirtPkg
> +#
> +# ArmVirtPkg is part of Platform Ci for builds so this is only
> +# used for code analysis.
> +#
> +# Copyright (c) Microsoft Corporation
> +# SPDX-License-Identifier: BSD-2-Clause-Patent
> +##
> +{
> + ## options defined .pytool/Plugin/CompilerPlugin
> + "CompilerPlugin": {
> + "DscPath": "" # Don't support this test
> + },
> +
> + ## options defined .pytool/Plugin/HostUnitTestCompilerPlugin
> + "HostUnitTestCompilerPlugin": {
> + "DscPath": "" # Don't support this test
> + },
> +
> + ## options defined .pytool/Plugin/CharEncodingCheck
> + "CharEncodingCheck": {
> + "IgnoreFiles": []
> + },
> +
> + ## options defined .pytool/Plugin/DependencyCheck
> + "DependencyCheck": {
> + "AcceptableDependencies": [
> + "MdePkg/MdePkg.dec",
> + "MdeModulePkg/MdeModulePkg.dec",
> + "ArmVirtPkg/ArmVirtPkg.dec",
> + "NetworkPkg/NetworkPkg.dec",
> + "ArmPkg/ArmPkg.dec",
> + "OvmfPkg/OvmfPkg.dec",
> + "EmbeddedPkg/EmbeddedPkg.dec",
> + "ArmPlatformPkg/ArmPlatformPkg.dec",
> + "SecurityPkg/SecurityPkg.dec",
> + "ShellPkg/ShellPkg.dec" #Is this ok?
> + ],
> + # For host based unit tests
> + "AcceptableDependencies-HOST_APPLICATION":[
> + "UnitTestFrameworkPkg/UnitTestFrameworkPkg.dec"
> + ],
> + # For UEFI shell based apps
> + "AcceptableDependencies-UEFI_APPLICATION":[
> +
> + ],
> + "IgnoreInf": []
> + },
> +
> + ## options defined .pytool/Plugin/DscCompleteCheck
> + "DscCompleteCheck": {
> + "IgnoreInf": [""],
> + "DscPath": "" # Don't support this test
> + },
> +
> + ## options defined .pytool/Plugin/HostUnitTestDscCompleteCheck
> + "HostUnitTestDscCompleteCheck": {
> + "IgnoreInf": [""],
> + "DscPath": "" # Don't support this test
> + },
> +
> + ## options defined .pytool/Plugin/GuidCheck
> + "GuidCheck": {
> + "IgnoreGuidName": [],
> + "IgnoreGuidValue": [],
> + "IgnoreFoldersAndFiles": [],
> + "IgnoreDuplicates": [],
> + },
> +
> + ## options defined .pytool/Plugin/LibraryClassCheck
> + "LibraryClassCheck": {
> + "IgnoreHeaderFile": []
> + },
> +
> + ## options defined .pytool/Plugin/SpellCheck
> + "SpellCheck": {
> + "AuditOnly": False, # Fails right now with over 270 errors
> + "IgnoreFiles": [], # use gitignore syntax to ignore errors in matching files
> + "ExtendWords": [
> + "setjump",
> + "plong",
> + "lparam",
> + "lpdword",
> + "lpthread",
> + "lresult",
> + "bootable",
> + "bsymbolic",
> + "endiannness",
> + "fvmain",
> + "multiboot",
> + "qemu's",
> + "ramdisk",
> + "ramfb",
> + "unbootable",
> + "virt's",
> + "werror",
> + "xenio"
> + ], # words to extend to the dictionary for this package
> + "IgnoreStandardPaths": [], # Standard Plugin defined paths that should be ignore
> + "AdditionalIncludePaths": [] # Additional paths to spell check (wildcards supported)
> + }
> +}
> diff --git a/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml b/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> new file mode 100644
> index 000000000000..b9452ec5bfaf
> --- /dev/null
> +++ b/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> @@ -0,0 +1,89 @@
> +## @file
> +# Azure Pipeline build file for building a platform.
> +#
> +# Platform: ArmVirtQemu
> +# OS: Ubuntu
> +# Toolchain: GCC5
> +#
> +# Copyright (c) Microsoft Corporation.
> +# SPDX-License-Identifier: BSD-2-Clause-Patent
> +##
> +trigger:
> + - master
> +pr:
> + - master
> +
> +jobs:
> + - job: Platform_CI
> + variables:
> + package: 'ArmVirtPkg'
> + vm_image: 'ubuntu-latest'
> + should_run: true
> + run_flags: "MAKE_STARTUP_NSH=TRUE QEMU_HEADLESS=TRUE"
> +
> + #Use matrix to speed up the build process
> + strategy:
> + matrix:
> + QEMU_AARCH64_DEBUG:
> + Build.File: "$(package)/PlatformCI/PlatformBuild.py"
> + Build.Arch: "AARCH64"
> + Build.Flags: ""
> + Build.Target: "DEBUG"
> + Run.Flags: $(run_flags)
> + Run: $(should_run)
> + QEMU_AARCH64_RELEASE:
> + Build.File: "$(package)/PlatformCI/PlatformBuild.py"
> + Build.Arch: "AARCH64"
> + Build.Flags: ""
> + Build.Target: "RELEASE"
> + Run.Flags: $(run_flags)
> + Run: $(should_run)
> + QEMU_AARCH64_NOOPT:
> + Build.File: "$(package)/PlatformCI/PlatformBuild.py"
> + Build.Arch: "AARCH64"
> + Build.Flags: ""
> + Build.Target: "NOOPT"
> + Run.Flags: $(run_flags)
> + Run: $(should_run)
> + QEMU_ARM_DEBUG:
> + Build.File: "$(package)/PlatformCI/PlatformBuild.py"
> + Build.Arch: "ARM"
> + Build.Flags: ""
> + Build.Target: "DEBUG"
> + Run.Flags: $(run_flags)
> + Run: $(should_run)
> + QEMU_ARM_RELEASE:
> + Build.File: "$(package)/PlatformCI/PlatformBuild.py"
> + Build.Arch: "ARM"
> + Build.Flags: ""
> + Build.Target: "RELEASE"
> + Run.Flags: $(run_flags)
> + Run: $(should_run)
> + QEMU_ARM_NOOPT:
> + Build.File: "$(package)/PlatformCI/PlatformBuild.py"
> + Build.Arch: "ARM"
> + Build.Flags: ""
> + Build.Target: "NOOPT"
> + Run.Flags: $(run_flags)
> + Run: $(should_run)
> +
> + workspace:
> + clean: all
> +
> + pool:
> + vmImage: $(vm_image)
> +
> + steps:
> + - template: ../../../.azurepipelines/templates/platform-build-run-steps.yml
> + parameters:
> + tool_chain_tag: GCC5
> + build_pkg: $(package)
> + build_target: $(Build.Target)
> + build_arch: $(Build.Arch)
> + build_file: $(Build.File)
> + build_flags: $(Build.Flags)
> + run_flags: $(Run.Flags)
> + extra_install_step:
> + - bash: sudo apt-get install qemu
> + displayName: Install qemu
> + condition: and(gt(variables.pkg_count, 0), succeeded())
> diff --git a/ArmVirtPkg/PlatformCI/PlatformBuild.py b/ArmVirtPkg/PlatformCI/PlatformBuild.py
> new file mode 100644
> index 000000000000..dff653e919eb
> --- /dev/null
> +++ b/ArmVirtPkg/PlatformCI/PlatformBuild.py
> @@ -0,0 +1,276 @@
> +# @file
> +# Script to Build ArmVirtPkg UEFI firmware
> +#
> +# Copyright (c) Microsoft Corporation.
> +# SPDX-License-Identifier: BSD-2-Clause-Patent
> +##
> +import os
> +import logging
> +import io
> +
> +from edk2toolext.environment import shell_environment
> +from edk2toolext.environment.uefi_build import UefiBuilder
> +from edk2toolext.invocables.edk2_platform_build import BuildSettingsManager
> +from edk2toolext.invocables.edk2_setup import SetupSettingsManager, RequiredSubmodule
> +from edk2toolext.invocables.edk2_update import UpdateSettingsManager
> +from edk2toolext.invocables.edk2_pr_eval import PrEvalSettingsManager
> +from edk2toollib.utility_functions import RunCmd
> +from edk2toollib.utility_functions import GetHostInfo
> +
> +# ####################################################################################### #
> +# Common Configuration #
> +# ####################################################################################### #
> +
> +
> +class CommonPlatform():
> + ''' Common settings for this platform. Define static data here and use
> + for the different parts of stuart
> + '''
> + PackagesSupported = ("ArmVirtPkg",)
> + ArchSupported = ("AARCH64", "ARM")
> + TargetsSupported = ("DEBUG", "RELEASE", "NOOPT")
> + Scopes = ('armvirt', 'edk2-build')
> + WorkspaceRoot = os.path.realpath(os.path.join(
> + os.path.dirname(os.path.abspath(__file__)), "..", ".."))
> +
> + # ####################################################################################### #
> + # Configuration for Update & Setup #
> + # ####################################################################################### #
> +
> +
> +class SettingsManager(UpdateSettingsManager, SetupSettingsManager, PrEvalSettingsManager):
> +
> + def GetPackagesSupported(self):
> + ''' return iterable of edk2 packages supported by this build.
> + These should be edk2 workspace relative paths '''
> + return CommonPlatform.PackagesSupported
> +
> + def GetArchitecturesSupported(self):
> + ''' return iterable of edk2 architectures supported by this build '''
> + return CommonPlatform.ArchSupported
> +
> + def GetTargetsSupported(self):
> + ''' return iterable of edk2 target tags supported by this build '''
> + return CommonPlatform.TargetsSupported
> +
> + def GetRequiredSubmodules(self):
> + ''' return iterable containing RequiredSubmodule objects.
> + If no RequiredSubmodules return an empty iterable
> + '''
> + rs = []
> +
> + # intentionally declare this one with recursive false to avoid overhead
> + rs.append(RequiredSubmodule(
> + "CryptoPkg/Library/OpensslLib/openssl", False))
> +
> + # To avoid maintenance of this file for every new submodule
> + # lets just parse the .gitmodules and add each if not already in list.
> + # The GetRequiredSubmodules is designed to allow a build to optimize
> + # the desired submodules but it isn't necessary for this repository.
> + result = io.StringIO()
> + ret = RunCmd("git", "config --file .gitmodules --get-regexp path", workingdir=self.GetWorkspaceRoot(), outstream=result)
> + # Cmd output is expected to look like:
> + # submodule.CryptoPkg/Library/OpensslLib/openssl.path CryptoPkg/Library/OpensslLib/openssl
> + # submodule.SoftFloat.path ArmPkg/Library/ArmSoftFloatLib/berkeley-softfloat-3
> + if ret == 0:
> + for line in result.getvalue().splitlines():
> + _, _, path = line.partition(" ")
> + if path is not None:
> + if path not in [x.path for x in rs]:
> + rs.append(RequiredSubmodule(path, True)) # add it with recursive since we don't know
> + return rs
> +
> + def SetArchitectures(self, list_of_requested_architectures):
> + ''' Confirm the requests architecture list is valid and configure SettingsManager
> + to run only the requested architectures.
> +
> + Raise Exception if a list_of_requested_architectures is not supported
> + '''
> + unsupported = set(list_of_requested_architectures) - \
> + set(self.GetArchitecturesSupported())
> + if(len(unsupported) > 0):
> + errorString = (
> + "Unsupported Architecture Requested: " + " ".join(unsupported))
> + logging.critical(errorString)
> + raise Exception(errorString)
> + self.ActualArchitectures = list_of_requested_architectures
> +
> + def GetWorkspaceRoot(self):
> + ''' get WorkspacePath '''
> + return CommonPlatform.WorkspaceRoot
> +
> + def GetActiveScopes(self):
> + ''' return tuple containing scopes that should be active for this process '''
> +
> + scopes = CommonPlatform.Scopes
> + ActualToolChainTag = shell_environment.GetBuildVars().GetValue("TOOL_CHAIN_TAG", "")
> +
> + if GetHostInfo().os.upper() == "LINUX" and ActualToolChainTag.upper().startswith("GCC"):
> + if "AARCH64" in self.ActualArchitectures:
> + scopes += ("gcc_aarch64_linux",)
> + if "ARM" in self.ActualArchitectures:
> + scopes += ("gcc_arm_linux",)
> + return scopes
> +
> + def FilterPackagesToTest(self, changedFilesList: list, potentialPackagesList: list) -> list:
> + ''' Filter other cases that this package should be built
> + based on changed files. This should cover things that can't
> + be detected as dependencies. '''
> + build_these_packages = []
> + possible_packages = potentialPackagesList.copy()
> + for f in changedFilesList:
> + # BaseTools files that might change the build
> + if "BaseTools" in f:
> + if os.path.splitext(f) not in [".txt", ".md"]:
> + build_these_packages = possible_packages
> + break
> +
> + # if the azure pipeline platform template file changed
> + if "platform-build-run-steps.yml" in f:
> + build_these_packages = possible_packages
> + break
> +
> +
> + return build_these_packages
> +
> + def GetPlatformDscAndConfig(self) -> tuple:
> + ''' If a platform desires to provide its DSC then Policy 4 will evaluate if
> + any of the changes will be built in the dsc.
> +
> + The tuple should be (<workspace relative path to dsc file>, <input dictionary of dsc key value pairs>)
> + '''
> + return (os.path.join("ArmVirtPkg", "ArmVirtQemu.dsc"), {})
> +
> +
> + # ####################################################################################### #
> + # Actual Configuration for Platform Build #
> + # ####################################################################################### #
> +
> +
> +class PlatformBuilder(UefiBuilder, BuildSettingsManager):
> + def __init__(self):
> + UefiBuilder.__init__(self)
> +
> + def AddCommandLineOptions(self, parserObj):
> + ''' Add command line options to the argparser '''
> + parserObj.add_argument('-a', "--arch", dest="build_arch", type=str, default="AARCH64",
> + help="Optional - Architecture to build. Default = AARCH64")
> +
> + def RetrieveCommandLineOptions(self, args):
> + ''' Retrieve command line options from the argparser '''
> +
> + shell_environment.GetBuildVars().SetValue(
> + "TARGET_ARCH", args.build_arch.upper(), "From CmdLine")
> +
> + shell_environment.GetBuildVars().SetValue(
> + "ACTIVE_PLATFORM", "ArmVirtPkg/ArmVirtQemu.dsc", "From CmdLine")
> +
> + def GetWorkspaceRoot(self):
> + ''' get WorkspacePath '''
> + return CommonPlatform.WorkspaceRoot
> +
> + def GetPackagesPath(self):
> + ''' Return a list of workspace relative paths that should be mapped as edk2 PackagesPath '''
> + return ()
> +
> + def GetActiveScopes(self):
> + ''' return tuple containing scopes that should be active for this process '''
> + scopes = CommonPlatform.Scopes
> + ActualToolChainTag = shell_environment.GetBuildVars().GetValue("TOOL_CHAIN_TAG", "")
> + Arch = shell_environment.GetBuildVars().GetValue("TARGET_ARCH", "")
> +
> + if GetHostInfo().os.upper() == "LINUX" and ActualToolChainTag.upper().startswith("GCC"):
> + if "AARCH64" == Arch:
> + scopes += ("gcc_aarch64_linux",)
> + elif "ARM" == Arch:
> + scopes += ("gcc_arm_linux",)
> + return scopes
> +
> + def GetName(self):
> + ''' Get the name of the repo, platform, or product being build '''
> + ''' Used for naming the log file, among others '''
> + # check the startup nsh flag and if set then rename the log file.
> + # this helps in CI so we don't overwrite the build log since running
> + # uses the stuart_build command.
> + if(shell_environment.GetBuildVars().GetValue("MAKE_STARTUP_NSH", "FALSE") == "TRUE"):
> + return "ArmVirtPkg_With_Run"
> + return "ArmVirtPkg"
> +
> + def GetLoggingLevel(self, loggerType):
> + ''' Get the logging level for a given type
> + base == lowest logging level supported
> + con == Screen logging
> + txt == plain text file logging
> + md == markdown file logging
> + '''
> + return logging.DEBUG
> +
> + def SetPlatformEnv(self):
> + logging.debug("PlatformBuilder SetPlatformEnv")
> + self.env.SetValue("PRODUCT_NAME", "ArmVirtQemu", "Platform Hardcoded")
> + self.env.SetValue("MAKE_STARTUP_NSH", "FALSE", "Default to false")
> + self.env.SetValue("QEMU_HEADLESS", "FALSE", "Default to false")
> + return 0
> +
> + def PlatformPreBuild(self):
> + return 0
> +
> + def PlatformPostBuild(self):
> + return 0
> +
> + def FlashRomImage(self):
> + VirtualDrive = os.path.join(self.env.GetValue(
> + "BUILD_OUTPUT_BASE"), "VirtualDrive")
> + os.makedirs(VirtualDrive, exist_ok=True)
> + OutputPath_FV = os.path.join(
> + self.env.GetValue("BUILD_OUTPUT_BASE"), "FV")
> + Built_FV = os.path.join(OutputPath_FV, "QEMU_EFI.fd")
> +
> + # pad fd to 64mb
> + with open(Built_FV, "ab") as fvfile:
> + fvfile.seek(0, os.SEEK_END)
> + additional = b'\0' * ((64 * 1024 * 1024)-fvfile.tell())
> + fvfile.write(additional)
> +
> + # QEMU must be on that path
> +
> + # Unique Command and Args parameters per ARCH
> + if (self.env.GetValue("TARGET_ARCH").upper() == "AARCH64"):
> + cmd = "qemu-system-aarch64"
> + args = "-M virt"
> + args += " -cpu cortex-a57" # emulate cpu
> + elif(self.env.GetValue("TARGET_ARCH").upper() == "ARM"):
> + cmd = "qemu-system-arm"
> + args = "-M virt"
> + args += " -cpu cortex-a15" # emulate cpu
> + else:
> + raise NotImplementedError()
> +
> + # Common Args
> + args += " -pflash " + Built_FV # path to fw
> + args += " -m 1024" # 1gb memory
> + # turn off network
> + args += " -net none"
> + # Serial messages out
> + args += " -serial stdio"
> + # Mount disk with startup.nsh
> + args += f" -drive file=fat:rw:{VirtualDrive},format=raw,media=disk"
> +
> + # Conditional Args
> + if (self.env.GetValue("QEMU_HEADLESS").upper() == "TRUE"):
> + args += " -display none" # no graphics
> +
> + if (self.env.GetValue("MAKE_STARTUP_NSH").upper() == "TRUE"):
> + f = open(os.path.join(VirtualDrive, "startup.nsh"), "w")
> + f.write("BOOT SUCCESS !!! \n")
> + # add commands here
> + f.write("reset -s\n")
> + f.close()
> +
> + ret = RunCmd(cmd, args)
> +
> + if ret == 0xc0000005:
> + # for some reason getting a c0000005 on successful return
> + return 0
> +
> + return ret
> diff --git a/ArmVirtPkg/PlatformCI/ReadMe.md b/ArmVirtPkg/PlatformCI/ReadMe.md
> new file mode 100644
> index 000000000000..7c11d925f59e
> --- /dev/null
> +++ b/ArmVirtPkg/PlatformCI/ReadMe.md
> @@ -0,0 +1,125 @@
> +# ArmVirtPkg - Platform CI
> +
> +This Readme.md describes the Azure DevOps based Platform CI for ArmVirtPkg and how
> +to use the same Pytools based build infrastructure locally.
> +
> +## Supported Configuration Details
> +
> +This solution for building and running ArmVirtPkg has only been validated with Ubuntu
> +18.04 and the GCC5 toolchain. Two different firmware builds are supported and are
> +described below.
> +
> +| Configuration name | Architecture | DSC File |Additional Flags |
> +| :---------- | :----- | :----- | :---- |
> +| AARCH64 | AARCH64 | ArmVirtQemu.dsc | None |
> +| ARM | ARM | ArmVirtQemu.dsc | None |
> +
> +## EDK2 Developer environment
> +
> +- [Python 3.8.x - Download & Install](https://www.python.org/downloads/)
> +- [GIT - Download & Install](https://git-scm.com/download/)
> +- [QEMU - Download, Install, and add to your path](https://www.qemu.org/download/)
> +- [Edk2 Source](https://github.com/tianocore/edk2)
> +- Additional packages found necessary for Ubuntu 18.04
> + - apt-get install gcc g++ make uuid-dev
> +
> +Note: edksetup, Submodule initialization and manual installation of NASM, iASL, or
> +the required cross-compiler toolchains are **not** required, this is handled by the
> +Pytools build system.
> +
> +## Building with Pytools for ArmVirtPkg
> +
> +1. [Optional] Create a Python Virtual Environment - generally once per workspace
> +
> + ``` bash
> + python -m venv <name of virtual environment>
> + ```
> +
> +2. [Optional] Activate Virtual Environment - each time new shell opened
> + - Windows
> +
> + ``` bash
> + <name of virtual environment>/Scripts/activate.bat
> + ```
> +
> + - Linux
> +
> + ```bash
> + source <name of virtual environment>/bin/activate
> + ```
> +
> +3. Install Pytools - generally once per virtual env or whenever pip-requirements.txt changes
> +
> + ``` bash
> + pip install --upgrade -r pip-requirements.txt
> + ```
> +
> +4. Initialize & Update Submodules - only when submodules updated
> +
> + ``` bash
> + stuart_setup -c ArmVirtPkg/PlatformCI/PlatformBuild.py TOOL_CHAIN_TAG=<TOOL_CHAIN_TAG> -a <TARGET_ARCH>
> + ```
> +
> +5. Initialize & Update Dependencies - only as needed when ext_deps change
> +
> + ``` bash
> + stuart_update -c ArmVirtPkg/PlatformCI/PlatformBuild.py TOOL_CHAIN_TAG=<TOOL_CHAIN_TAG> -a <TARGET_ARCH>
> + ```
> +
> +6. Compile the basetools if necessary - only when basetools C source files change
> +
> + ``` bash
> + python BaseTools/Edk2ToolsBuild.py -t <ToolChainTag>
> + ```
> +
> +7. Compile Firmware
> +
> + ``` bash
> + stuart_build -c ArmVirtPkg/PlatformCI/PlatformBuild.py TOOL_CHAIN_TAG=<TOOL_CHAIN_TAG> -a <TARGET_ARCH>
> + ```
> +
> + - use `stuart_build -c ArmVirtPkg/PlatformCI/PlatformBuild.py -h` option to see additional
> + options like `--clean`
> +
> +8. Running Emulator
> + - You can add `--FlashRom` to the end of your build command and the emulator will run after the
> + build is complete.
> + - or use the `--FlashOnly` feature to just run the emulator.
> +
> + ``` bash
> + stuart_build -c ArmVirtPkg/PlatformCI/PlatformBuild.py TOOL_CHAIN_TAG=<TOOL_CHAIN_TAG> -a <TARGET_ARCH> --FlashOnly
> + ```
> +
> +### Notes
> +
> +1. Including the expected build architecture and toolchain to the _stuart_update_ command is critical.
> + This is because there are extra scopes and tools that will be resolved during the update step that
> + need to match your build step.
> +2. Configuring *ACTIVE_PLATFORM* and *TARGET_ARCH* in Conf/target.txt is **not** required. This
> + environment is set by PlatformBuild.py based upon the `[-a <TARGET_ARCH>]` parameter.
> +3. QEMU must be on your path. On Windows this is a manual process and not part of the QEMU installer.
> +
> +**NOTE:** Logging the execution output will be in the normal stuart log as well as to your console.
> +
> +### Custom Build Options
> +
> +**MAKE_STARTUP_NSH=TRUE** will output a *startup.nsh* file to the location mapped as fs0. This is
> +used in CI in combination with the `--FlashOnly` feature to run QEMU to the UEFI shell and then execute
> +the contents of *startup.nsh*.
> +
> +**QEMU_HEADLESS=TRUE** Since CI servers run headless QEMU must be told to run with no display otherwise
> +an error occurs. Locally you don't need to set this.
> +
> +### Passing Build Defines
> +
> +To pass build defines through _stuart_build_, prepend `BLD_*_`to the define name and pass it on the
> +command-line. _stuart_build_ currently requires values to be assigned, so add an`=1` suffix for bare defines.
> +For example, to enable the TPM2 support, instead of the traditional "-D TPM2_ENABLE=TRUE", the stuart_build
> +command-line would be:
> +
> +`stuart_build -c ArmVirtPkg/PlatformCI/PlatformBuild.py BLD_*_TPM2_ENABLE=TRUE`
> +
> +## References
> +
> +- [Installing and using Pytools](https://github.com/tianocore/edk2-pytool-extensions/blob/master/docs/using.md#installing)
> +- More on [python virtual environments](https://docs.python.org/3/library/venv.html)
> diff --git a/ArmVirtPkg/PlatformCI/iasl_ext_dep.yaml b/ArmVirtPkg/PlatformCI/iasl_ext_dep.yaml
> new file mode 100644
> index 000000000000..8869ed3ecef1
> --- /dev/null
> +++ b/ArmVirtPkg/PlatformCI/iasl_ext_dep.yaml
> @@ -0,0 +1,21 @@
> +## @file
> +# Download iasl executable tool from a nuget.org package
> +# - package contains different binaries based on host
> +# Add the folder with the tool to the path
> +#
> +# This is only downloaded for scope armvirt thus
> +# should have no impact on the asl compiler used by any
> +# other platform build
> +#
> +# Copyright (c) Microsoft Corporation.
> +# SPDX-License-Identifier: BSD-2-Clause-Patent
> +##
> +{
> + "id": "iasl-armvirt-1",
> + "scope": "armvirt",
> + "type": "nuget",
> + "name": "iasl",
> + "source": "https://api.nuget.org/v3/index.json",
> + "version": "20190215.0.0",
> + "flags": ["set_path", "host_specific"],
> +}
> --
> 2.16.3.windows.1
>
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH v3 7/7] ReadMe: Convert to rst and add Platform CI Status
2020-04-24 21:31 ` [PATCH v3 7/7] ReadMe: Convert to rst and add Platform CI Status Michael Kubacki
@ 2020-04-27 12:30 ` Leif Lindholm
0 siblings, 0 replies; 13+ messages in thread
From: Leif Lindholm @ 2020-04-27 12:30 UTC (permalink / raw)
To: michael.kubacki; +Cc: devel, Andrew Fish, Laszlo Ersek, Michael D Kinney
On Fri, Apr 24, 2020 at 14:31:08 -0700, michael.kubacki@outlook.com wrote:
> From: Sean Brogan <sean.brogan@microsoft.com>
>
> REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2570
>
> Convert workspace root Readme.md to reStructuredText
> ReStructuredText supports building tables with syntax that is easier
> to read in plain-text.
> Rename as ReadMe.rst.
> Add Platform CI Status Tables
>
> Cc: Andrew Fish <afish@apple.com>
> Cc: Laszlo Ersek <lersek@redhat.com>
> Cc: Leif Lindholm <leif@nuviainc.com>
> Cc: Michael D Kinney <michael.d.kinney@intel.com>
> Signed-off-by: Sean Brogan <sean.brogan@microsoft.com>
With the above line dropped:
Reviewed-by: Leif Lindholm <leif@nuviainc.com>
Thanks Sean, this is a big improvement!
> Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
> Reviewed-by: Laszlo Ersek <lersek@redhat.com>
> Reviewed-by: Bret Barkelew <bret.barkelew@microsoft.com>
> ---
> ReadMe.rst | 354 ++++++++++++++++++++
> Readme.md | 235 -------------
> 2 files changed, 354 insertions(+), 235 deletions(-)
>
> diff --git a/ReadMe.rst b/ReadMe.rst
> new file mode 100644
> index 000000000000..c4a29b2809ba
> --- /dev/null
> +++ b/ReadMe.rst
> @@ -0,0 +1,354 @@
> +==============
> +EDK II Project
> +==============
> +
> +A modern, feature-rich, cross-platform firmware development
> +environment for the UEFI and PI specifications from www.uefi.org.
> +
> +Core CI Build Status
> +--------------------
> +
> +============================= ================= =============== ===================
> + Host Type & Toolchain Build Status Test Status Code Coverage
> +============================= ================= =============== ===================
> +Windows_VS2019_ |WindowsCiBuild| |WindowsCiTest| |WindowsCiCoverage|
> +Ubuntu_GCC5_ |UbuntuCiBuild| |UbuntuCiTest| |UbuntuCiCoverage|
> +============================= ================= =============== ===================
> +
> +`More CI Build information <.pytool/Readme.md>`__
> +
> +Platform CI Build Status
> +------------------------
> +
> +Microsoft Windows VS2019
> +````````````````````````
> +
> +============================= ================= ============= ============= ==============
> + Toolchain CONFIG DEBUG RELEASE NOOPT
> +============================= ================= ============= ============= ==============
> +EmulatorPkg_Win_VS2019_ | IA32 |em32d| |em32r| |em32n|
> +| | X64 |em64d| |em64r| |em64n|
> +OvmfPkg_Win_VS2019_ | IA32 |op32d| |op32r| |op32n|
> +| | X64 |op64d| |op64r| |op64n|
> +| | IA32 X64 |op3264d| |op3264r| |op3264n|
> +| | IA32 X64 FULL |op3264fd| |op3264fr| |op3264fn|
> +============================= ================= ============= ============= ==============
> +
> +Ubuntu 18.04 GCC5
> +`````````````````
> +
> +============================= ================= ============= ============= ==============
> + Toolchain CONFIG DEBUG RELEASE NOOPT
> +============================= ================= ============= ============= ==============
> +ArmVirtPkg_Ubuntu_GCC5_ | AARCH64 |avAArch64du| |avAArch64ru| |avAArch64nu|
> +| | ARM |avArmdu| |avArmru| |avArmnu|
> +EmulatorPkg_Ubuntu_GCC5_ | IA32 |em32du| |em32ru| |em32nu|
> +| | X64 |em64du| |em64ru| |em64nu|
> +OvmfPkg_Ubuntu_GCC5_ | IA32 |op32du| |op32ru| |op32nu|
> +| | X64 |op64du| |op64ru| |op64nu|
> +| | IA32 X64 |op3264du| |op3264ru| |op3264nu|
> +| | IA32 X64 FULL |op3264fdu| |op3264fru| |op3264fru|
> +============================= ================= ============= ============= ==============
> +
> +|TCBZ_2668|_ - EmulatorPkg Ubuntu GCC5 Segfaults during execution.
> +
> +|TCBZ_2639|_ - EmulatorPkg Ubuntu GCC5 Segfaults during execution.
> +
> +`More ArmVirtPkg CI Build Information <ArmVirtPkg/PlatformCI/ReadMe.md>`__
> +
> +`More EmulatorPkg CI Build Information <EmulatorPkg/PlatformCI/ReadMe.md>`__
> +
> +`More OvmfPkg CI Build Information <OvmfPkg/PlatformCI/ReadMe.md>`__
> +
> +
> +License Details
> +---------------
> +
> +The majority of the content in the EDK II open source project uses a
> +`BSD-2-Clause Plus Patent License <License.txt>`__. The EDK II open
> +source project contains the following components that are covered by additional
> +licenses:
> +
> +- `BaseTools/Source/C/BrotliCompress/brotli <https://github.com/google/brotli/blob/master/LICENSE>`__
> +- `MdeModulePkg/Library/BrotliCustomDecompressLib/brotli <https://github.com/google/brotli/blob/master/LICENSE>`__
> +- `BaseTools/Source/C/LzmaCompress <BaseTools/Source/C/LzmaCompress/LZMA-SDK-README.txt>`__
> +- `MdeModulePkg/Library/LzmaCustomDecompressLib <MdeModulePkg/Library/LzmaCustomDecompressLib/LZMA-SDK-README.txt>`__
> +- `IntelFrameworkModulePkg/Library/LzmaCustomDecompressLib/Sdk <IntelFrameworkModulePkg/Library/LzmaCustomDecompressLib/LZMA-SDK-README.txt>`__
> +- `BaseTools/Source/C/VfrCompile/Pccts <BaseTools/Source/C/VfrCompile/Pccts/RIGHTS>`__
> +- `MdeModulePkg/Universal/RegularExpressionDxe/oniguruma <https://github.com/kkos/oniguruma/blob/master/README.md>`__
> +- `OvmfPkg <OvmfPkg/License.txt>`__
> +- `CryptoPkg/Library/OpensslLib/openssl <https://github.com/openssl/openssl/blob/50eaac9f3337667259de725451f201e784599687/LICENSE>`__
> +- `ArmPkg/Library/ArmSoftFloatLib/berkeley-softfloat-3 <https://github.com/ucb-bar/berkeley-softfloat-3/blob/b64af41c3276f97f0e181920400ee056b9c88037/COPYING.txt>`__
> +
> +The EDK II Project is composed of packages. The maintainers for each package
> +are listed in `Maintainers.txt <Maintainers.txt>`__.
> +
> +Resources
> +---------
> +
> +- `TianoCore <http://www.tianocore.org>`__
> +- `EDK
> + II <https://github.com/tianocore/tianocore.github.io/wiki/EDK-II>`__
> +- `Getting Started with EDK
> + II <https://github.com/tianocore/tianocore.github.io/wiki/Getting-Started-with-EDK-II>`__
> +- `Mailing
> + Lists <https://github.com/tianocore/tianocore.github.io/wiki/Mailing-Lists>`__
> +- `TianoCore Bugzilla <https://bugzilla.tianocore.org>`__
> +- `How To
> + Contribute <https://github.com/tianocore/tianocore.github.io/wiki/How-To-Contribute>`__
> +- `Release
> + Planning <https://github.com/tianocore/tianocore.github.io/wiki/EDK-II-Release-Planning>`__
> +
> +Code Contributions
> +------------------
> +
> +To make a contribution to a TianoCore project, follow these steps.
> +
> +#. Create a change description in the format specified below to
> + use in the source control commit log.
> +#. Your commit message must include your ``Signed-off-by`` signature
> +#. Submit your code to the TianoCore project using the process
> + that the project documents on its web page. If the process is
> + not documented, then submit the code on development email list
> + for the project.
> +#. It is preferred that contributions are submitted using the same
> + copyright license as the base project. When that is not possible,
> + then contributions using the following licenses can be accepted:
> +
> +- BSD (2-clause): http://opensource.org/licenses/BSD-2-Clause
> +- BSD (3-clause): http://opensource.org/licenses/BSD-3-Clause
> +- MIT: http://opensource.org/licenses/MIT
> +- Python-2.0: http://opensource.org/licenses/Python-2.0
> +- Zlib: http://opensource.org/licenses/Zlib
> +
> +For documentation:
> +
> +- FreeBSD Documentation License
> + https://www.freebsd.org/copyright/freebsd-doc-license.html
> +
> +Contributions of code put into the public domain can also be accepted.
> +
> +Contributions using other licenses might be accepted, but further
> +review will be required.
> +
> +Developer Certificate of Origin
> +-------------------------------
> +
> +Your change description should use the standard format for a
> +commit message, and must include your ``Signed-off-by`` signature.
> +
> +In order to keep track of who did what, all patches contributed must
> +include a statement that to the best of the contributor's knowledge
> +they have the right to contribute it under the specified license.
> +
> +The test for this is as specified in the `Developer's Certificate of
> +Origin (DCO) 1.1 <https://developercertificate.org/>`__. The contributor
> +certifies compliance by adding a line saying
> +
> +Signed-off-by: Developer Name developer@example.org
> +
> +where ``Developer Name`` is the contributor's real name, and the email
> +address is one the developer is reachable through at the time of
> +contributing.
> +
> +::
> +
> + Developer's Certificate of Origin 1.1
> +
> + By making a contribution to this project, I certify that:
> +
> + (a) The contribution was created in whole or in part by me and I
> + have the right to submit it under the open source license
> + indicated in the file; or
> +
> + (b) The contribution is based upon previous work that, to the best
> + of my knowledge, is covered under an appropriate open source
> + license and I have the right under that license to submit that
> + work with modifications, whether created in whole or in part
> + by me, under the same open source license (unless I am
> + permitted to submit under a different license), as indicated
> + in the file; or
> +
> + (c) The contribution was provided directly to me by some other
> + person who certified (a), (b) or (c) and I have not modified
> + it.
> +
> + (d) I understand and agree that this project and the contribution
> + are public and that a record of the contribution (including all
> + personal information I submit with it, including my sign-off) is
> + maintained indefinitely and may be redistributed consistent with
> + this project or the open source license(s) involved.
> +
> +Sample Change Description / Commit Message
> +------------------------------------------
> +
> +::
> +
> + From: Contributor Name <contributor@example.com>
> + Subject: [Repository/Branch PATCH] Pkg-Module: Brief-single-line-summary
> +
> + Full-commit-message
> +
> + Signed-off-by: Contributor Name <contributor@example.com>
> +
> +Notes for sample patch email
> +````````````````````````````
> +
> +- The first line of commit message is taken from the email's subject
> + line following ``[Repository/Branch PATCH]``. The remaining portion
> + of the commit message is the email's content.
> +- ``git format-patch`` is one way to create this format
> +
> +Definitions for sample patch email
> +``````````````````````````````````
> +
> +- ``Repository`` is the identifier of the repository the patch applies.
> + This identifier should only be provided for repositories other than
> + ``edk2``. For example ``edk2-BuildSpecification`` or ``staging``.
> +- ``Branch`` is the identifier of the branch the patch applies. This
> + identifier should only be provided for branches other than
> + ``edk2/master``.
> + For example ``edk2/UDK2015``,
> + ``edk2-BuildSpecification/release/1.27``, or
> + ``staging/edk2-test``.
> +- ``Module`` is a short identifier for the affected code or
> + documentation. For example ``MdePkg``, ``MdeModulePkg/UsbBusDxe``, ``Introduction``, or
> + ``EDK II INF File Format``.
> +- ``Brief-single-line-summary`` is a short summary of the change.
> +- The entire first line should be less than ~70 characters.
> +- ``Full-commit-message`` a verbose multiple line comment describing
> + the change. Each line should be less than ~70 characters.
> +- ``Signed-off-by`` is the contributor's signature identifying them
> + by their real/legal name and their email address.
> +
> +Submodules
> +----------
> +
> +Submodule in EDK II is allowed but submodule chain should be avoided
> +as possible as we can. Currently EDK II contains the following submodules
> +
> +- CryptoPkg/Library/OpensslLib/openssl
> +- ArmPkg/Library/ArmSoftFloatLib/berkeley-softfloat-3
> +- MdeModulePkg/Universal/RegularExpressionDxe/oniguruma
> +- MdeModulePkg/Library/BrotliCustomDecompressLib/brotli
> +- BaseTools/Source/C/BrotliCompress/brotli
> +
> +ArmSoftFloatLib is actually required by OpensslLib. It's inevitable
> +in openssl-1.1.1 (since stable201905) for floating point parameter
> +conversion, but should be dropped once there's no such need in future
> +release of openssl.
> +
> +To get a full, buildable EDK II repository, use following steps of git
> +command
> +
> +.. code-block:: bash
> +
> + git clone https://github.com/tianocore/edk2.git
> + cd edk2
> + git submodule update --init
> + cd ..
> +
> +If there's update for submodules, use following git commands to get
> +the latest submodules code.
> +
> +.. code-block:: bash
> +
> + cd edk2
> + git pull
> + git submodule update
> +
> +Note: When cloning submodule repos, '--recursive' option is not
> +recommended. EDK II itself will not use any code/feature from
> +submodules in above submodules. So using '--recursive' adds a
> +dependency on being able to reach servers we do not actually want
> +any code from, as well as needlessly downloading code we will not
> +use.
> +
> +.. ===================================================================
> +.. This is a bunch of directives to make the README file more readable
> +.. ===================================================================
> +
> +.. CoreCI
> +
> +.. _Windows_VS2019: https://dev.azure.com/tianocore/edk2-ci/_build/latest?definitionId=32&branchName=master
> +.. |WindowsCiBuild| image:: https://dev.azure.com/tianocore/edk2-ci/_apis/build/status/Windows%20VS2019%20CI?branchName=master
> +.. |WindowsCiTest| image:: https://img.shields.io/azure-devops/tests/tianocore/edk2-ci/32.svg
> +.. |WindowsCiCoverage| image:: https://img.shields.io/badge/coverage-coming_soon-blue
> +
> +.. _Ubuntu_GCC5: https://dev.azure.com/tianocore/edk2-ci/_build/latest?definitionId=31&branchName=master
> +.. |UbuntuCiBuild| image:: https://dev.azure.com/tianocore/edk2-ci/_apis/build/status/Ubuntu%20GCC5%20CI?branchName=master
> +.. |UbuntuCiTest| image:: https://img.shields.io/azure-devops/tests/tianocore/edk2-ci/31.svg
> +.. |UbuntuCiCoverage| image:: https://img.shields.io/badge/coverage-coming_soon-blue
> +
> +.. ArmVirtPkg
> +
> +.. _ArmVirtPkg_Ubuntu_GCC5: https://dev.azure.com/tianocore/edk2-ci-play/_build/latest?definitionId=41&branchName=master
> +.. |avAArch64du| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/ArmVirtPkg/ArmVirtQemu%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20QEMU_AARCH64_DEBUG
> +.. |avAArch64ru| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/ArmVirtPkg/ArmVirtQemu%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20QEMU_AARCH64_RELEASE
> +.. |avAArch64nu| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/ArmVirtPkg/ArmVirtQemu%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20QEMU_AARCH64_NOOPT
> +
> +.. |avArmdu| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/ArmVirtPkg/ArmVirtQemu%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20QEMU_ARM_DEBUG
> +.. |avArmru| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/ArmVirtPkg/ArmVirtQemu%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20QEMU_ARM_RELEASE
> +.. |avArmnu| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/ArmVirtPkg/ArmVirtQemu%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20QEMU_ARM_NOOPT
> +
> +.. EmulatorPkg
> +
> +.. |TCBZ_2668| image:: https://img.shields.io/bugzilla/2668?baseUrl=https%3A%2F%2Fbugzilla.tianocore.org
> +.. _TCBZ_2668: https://bugzilla.tianocore.org/show_bug.cgi?id=2668
> +
> +.. |TCBZ_2639| image:: https://img.shields.io/bugzilla/2639?baseUrl=https%3A%2F%2Fbugzilla.tianocore.org
> +.. _TCBZ_2639: https://bugzilla.tianocore.org/show_bug.cgi?id=2639
> +
> +.. _EmulatorPkg_Win_VS2019: https://dev.azure.com/tianocore/edk2-ci-play/_build/latest?definitionId=40&branchName=master
> +.. _EmulatorPkg_Ubuntu_GCC5: https://dev.azure.com/tianocore/edk2-ci-play/_build/latest?definitionId=39&branchName=master
> +
> +.. |em32d| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/EmulatorPkg/EmulatorPkg%20Windows%20VS2019?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20EmulatorPkg_IA32_DEBUG
> +.. |em32du| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/EmulatorPkg/EmulatorPkg%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20EmulatorPkg_IA32_DEBUG
> +.. |em32r| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/EmulatorPkg/EmulatorPkg%20Windows%20VS2019?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20EmulatorPkg_IA32_RELEASE
> +.. |em32ru| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/EmulatorPkg/EmulatorPkg%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20EmulatorPkg_IA32_RELEASE
> +.. |em32n| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/EmulatorPkg/EmulatorPkg%20Windows%20VS2019?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20EmulatorPkg_IA32_NOOPT
> +.. |em32nu| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/EmulatorPkg/EmulatorPkg%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20EmulatorPkg_IA32_NOOPT
> +
> +.. |em64d| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/EmulatorPkg/EmulatorPkg%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20EmulatorPkg_X64_DEBUG
> +.. |em64du| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/EmulatorPkg/EmulatorPkg%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20EmulatorPkg_X64_DEBUG
> +.. |em64r| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/EmulatorPkg/EmulatorPkg%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20EmulatorPkg_X64_RELEASE
> +.. |em64ru| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/EmulatorPkg/EmulatorPkg%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20EmulatorPkg_X64_RELEASE
> +.. |em64n| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/EmulatorPkg/EmulatorPkg%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20EmulatorPkg_X64_NOOPT
> +.. |em64nu| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/EmulatorPkg/EmulatorPkg%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20EmulatorPkg_X64_NOOPT
> +
> +.. OvmfPkg
> +
> +.. |TCBZ_2661| image:: https://img.shields.io/bugzilla/2661?baseUrl=https%3A%2F%2Fbugzilla.tianocore.org
> +.. _TCBZ_2661: https://bugzilla.tianocore.org/show_bug.cgi?id=2661
> +
> +.. _OvmfPkg_Win_VS2019: https://dev.azure.com/tianocore/edk2-ci-play/_build/latest?definitionId=38&branchName=master/
> +.. _OvmfPkg_Ubuntu_GCC5: https://dev.azure.com/tianocore/edk2-ci-play/_build/latest?definitionId=37&branchName=master
> +
> +.. |op32d| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/OVMF/OVMF%20Windows%20VS2019?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20OVMF_IA32_DEBUG
> +.. |op32du| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/OVMF/OVMF%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20OVMF_IA32_DEBUG
> +.. |op32r| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/OVMF/OVMF%20Windows%20VS2019?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20OVMF_IA32_RELEASE
> +.. |op32ru| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/OVMF/OVMF%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20OVMF_IA32_RELEASE
> +.. |op32n| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/OVMF/OVMF%20Windows%20VS2019?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20OVMF_IA32_NOOPT
> +.. |op32nu| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/OVMF/OVMF%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20OVMF_IA32_NOOPT
> +
> +.. |op64d| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/OVMF/OVMF%20Windows%20VS2019?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20OVMF_X64_DEBUG
> +.. |op64du| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/OVMF/OVMF%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20OVMF_X64_DEBUG
> +.. |op64r| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/OVMF/OVMF%20Windows%20VS2019?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20OVMF_X64_RELEASE
> +.. |op64ru| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/OVMF/OVMF%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20OVMF_X64_RELEASE
> +.. |op64n| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/OVMF/OVMF%20Windows%20VS2019?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20OVMF_X64_NOOPT
> +.. |op64nu| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/OVMF/OVMF%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20OVMF_X64_NOOPT
> +
> +
> +.. |op3264d| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/OVMF/OVMF%20Windows%20VS2019?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20OVMF_IA32X64_DEBUG
> +.. |op3264du| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/OVMF/OVMF%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20OVMF_IA32X64_DEBUG
> +.. |op3264r| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/OVMF/OVMF%20Windows%20VS2019?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20OVMF_IA32X64_RELEASE
> +.. |op3264ru| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/OVMF/OVMF%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20OVMF_IA32X64_RELEASE
> +.. |op3264n| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/OVMF/OVMF%20Windows%20VS2019?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20OVMF_IA32X64_NOOPT
> +.. |op3264nu| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/OVMF/OVMF%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20OVMF_IA32X64_NOOPT
> +
> +.. |op3264fd| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/OVMF/OVMF%20Windows%20VS2019?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20OVMF_IA32X64_FULL_DEBUG
> +.. |op3264fdu| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/OVMF/OVMF%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20OVMF_IA32X64_FULL_DEBUG
> +.. |op3264fr| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/OVMF/OVMF%20Windows%20VS2019?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20OVMF_IA32X64_FULL_RELEASE
> +.. |op3264fru| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/OVMF/OVMF%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20OVMF_IA32X64_FULL_RELEASE
> +.. |op3264fn| replace:: |TCBZ_2661|_
> +.. |op3264fnu| image:: https://dev.azure.com/tianocore/edk2-ci-play/_apis/build/status/OVMF/OVMF%20Ubuntu%20GCC5?branchName=master&jobName=Platform_CI&configuration=Platform_CI%20OVMF_IA32X64_FULL_NOOPT
> diff --git a/Readme.md b/Readme.md
> deleted file mode 100644
> index 843046365425..000000000000
> --- a/Readme.md
> +++ /dev/null
> @@ -1,235 +0,0 @@
> -# EDK II Project
> -
> -A modern, feature-rich, cross-platform firmware development environment
> -for the UEFI and PI specifications from www.uefi.org.
> -
> -# Build Status
> -
> -<table>
> - <tr>
> - <th>Host Type</th>
> - <th>Toolchain</th>
> - <th>Branch</th>
> - <th>Build Status</th>
> - <th>Test Status</th>
> - <th>Code Coverage</th>
> - </tr>
> - <tr>
> - <td>Windows</td>
> - <td>VS2019</td>
> - <td>master</td>
> - <td>
> - <a href="https://dev.azure.com/tianocore/edk2-ci/_build/latest?definitionId=32&branchName=master">
> - <img src="https://dev.azure.com/tianocore/edk2-ci/_apis/build/status/Windows%20VS2019%20CI?branchName=master"/></a>
> - </td>
> - <td>
> - <a href="https://dev.azure.com/tianocore/edk2-ci/_build/latest?definitionId=32&branchName=master">
> - <img src="https://img.shields.io/azure-devops/tests/tianocore/edk2-ci/32.svg"/></a>
> - </td>
> - <td>
> - <a href="https://dev.azure.com/tianocore/edk2-ci/_build/latest?definitionId=32&branchName=master">
> - <img src="https://img.shields.io/badge/coverage-coming_soon-blue"/></a>
> - </td>
> - </tr>
> - <tr>
> - <td>Ubuntu</td>
> - <td>GCC</td>
> - <td>master</td>
> - <td>
> - <a href="https://dev.azure.com/tianocore/edk2-ci/_build/latest?definitionId=31&branchName=master">
> - <img src="https://dev.azure.com/tianocore/edk2-ci/_apis/build/status/Ubuntu%20GCC5%20CI?branchName=master"/></a>
> - </td>
> - <td>
> - <a href="https://dev.azure.com/tianocore/edk2-ci/_build/latest?definitionId=31&branchName=master">
> - <img src="https://img.shields.io/azure-devops/tests/tianocore/edk2-ci/31.svg"/></a>
> - </td>
> - <td>
> - <a href="https://dev.azure.com/tianocore/edk2-ci/_build/latest?definitionId=31&branchName=master">
> - <img src="https://img.shields.io/badge/coverage-coming_soon-blue"/></a>
> - </td>
> - </tr>
> -</table>
> -
> -[More CI Build information](.pytool/Readme.md)
> -
> -# License Details
> -
> -The majority of the content in the EDK II open source project uses a
> -[BSD-2-Clause Plus Patent License](License.txt). The EDK II open source project
> -contains the following components that are covered by additional licenses:
> -* [BaseTools/Source/C/BrotliCompress/brotli](https://github.com/google/brotli/blob/master/LICENSE)
> -* [MdeModulePkg/Library/BrotliCustomDecompressLib/brotli](https://github.com/google/brotli/blob/master/LICENSE)
> -* [BaseTools/Source/C/LzmaCompress](BaseTools/Source/C/LzmaCompress/LZMA-SDK-README.txt)
> -* [MdeModulePkg/Library/LzmaCustomDecompressLib](MdeModulePkg/Library/LzmaCustomDecompressLib/LZMA-SDK-README.txt)
> -* [IntelFrameworkModulePkg/Library/LzmaCustomDecompressLib/Sdk](IntelFrameworkModulePkg/Library/LzmaCustomDecompressLib/LZMA-SDK-README.txt)
> -* [BaseTools/Source/C/VfrCompile/Pccts](BaseTools/Source/C/VfrCompile/Pccts/RIGHTS)
> -* [MdeModulePkg/Universal/RegularExpressionDxe/oniguruma](https://github.com/kkos/oniguruma/blob/master/README.md)
> -* [OvmfPkg](OvmfPkg/License.txt)
> -* [CryptoPkg/Library/OpensslLib/openssl](https://github.com/openssl/openssl/blob/50eaac9f3337667259de725451f201e784599687/LICENSE)
> -* [ArmPkg/Library/ArmSoftFloatLib/berkeley-softfloat-3](https://github.com/ucb-bar/berkeley-softfloat-3/blob/b64af41c3276f97f0e181920400ee056b9c88037/COPYING.txt)
> -
> -The EDK II Project is composed of packages. The maintainers for each package
> -are listed in [Maintainers.txt](Maintainers.txt).
> -
> -# Resources
> -* [TianoCore](http://www.tianocore.org)
> -* [EDK II](https://github.com/tianocore/tianocore.github.io/wiki/EDK-II)
> -* [Getting Started with EDK II](https://github.com/tianocore/tianocore.github.io/wiki/Getting-Started-with-EDK-II)
> -* [Mailing Lists](https://github.com/tianocore/tianocore.github.io/wiki/Mailing-Lists)
> -* [TianoCore Bugzilla](https://bugzilla.tianocore.org)
> -* [How To Contribute](https://github.com/tianocore/tianocore.github.io/wiki/How-To-Contribute)
> -* [Release Planning](https://github.com/tianocore/tianocore.github.io/wiki/EDK-II-Release-Planning)
> -
> -# Code Contributions
> -To make a contribution to a TianoCore project, follow these steps.
> -1. Create a change description in the format specified below to
> - use in the source control commit log.
> -2. Your commit message must include your `Signed-off-by` signature
> -3. Submit your code to the TianoCore project using the process
> - that the project documents on its web page. If the process is
> - not documented, then submit the code on development email list
> - for the project.
> -4. It is preferred that contributions are submitted using the same
> - copyright license as the base project. When that is not possible,
> - then contributions using the following licenses can be accepted:
> - * BSD (2-clause): http://opensource.org/licenses/BSD-2-Clause
> - * BSD (3-clause): http://opensource.org/licenses/BSD-3-Clause
> - * MIT: http://opensource.org/licenses/MIT
> - * Python-2.0: http://opensource.org/licenses/Python-2.0
> - * Zlib: http://opensource.org/licenses/Zlib
> -
> - For documentation:
> - * FreeBSD Documentation License
> - https://www.freebsd.org/copyright/freebsd-doc-license.html
> -
> - Contributions of code put into the public domain can also be
> - accepted.
> -
> - Contributions using other licenses might be accepted, but further
> - review will be required.
> -
> -# Developer Certificate of Origin
> -
> -Your change description should use the standard format for a
> -commit message, and must include your `Signed-off-by` signature.
> -
> -In order to keep track of who did what, all patches contributed must
> -include a statement that to the best of the contributor's knowledge
> -they have the right to contribute it under the specified license.
> -
> -The test for this is as specified in the [Developer's Certificate of
> -Origin (DCO) 1.1](https://developercertificate.org/). The contributor
> -certifies compliance by adding a line saying
> -
> - Signed-off-by: Developer Name <developer@example.org>
> -
> -where `Developer Name` is the contributor's real name, and the email
> -address is one the developer is reachable through at the time of
> -contributing.
> -
> -```
> -Developer's Certificate of Origin 1.1
> -
> -By making a contribution to this project, I certify that:
> -
> -(a) The contribution was created in whole or in part by me and I
> - have the right to submit it under the open source license
> - indicated in the file; or
> -
> -(b) The contribution is based upon previous work that, to the best
> - of my knowledge, is covered under an appropriate open source
> - license and I have the right under that license to submit that
> - work with modifications, whether created in whole or in part
> - by me, under the same open source license (unless I am
> - permitted to submit under a different license), as indicated
> - in the file; or
> -
> -(c) The contribution was provided directly to me by some other
> - person who certified (a), (b) or (c) and I have not modified
> - it.
> -
> -(d) I understand and agree that this project and the contribution
> - are public and that a record of the contribution (including all
> - personal information I submit with it, including my sign-off) is
> - maintained indefinitely and may be redistributed consistent with
> - this project or the open source license(s) involved.
> -```
> -
> -# Sample Change Description / Commit Message
> -
> -```
> -From: Contributor Name <contributor@example.com>
> -Subject: [Repository/Branch PATCH] Pkg-Module: Brief-single-line-summary
> -
> -Full-commit-message
> -
> -Signed-off-by: Contributor Name <contributor@example.com>
> -```
> -
> -## Notes for sample patch email
> -
> -* The first line of commit message is taken from the email's subject
> - line following `[Repository/Branch PATCH]`. The remaining portion of the
> - commit message is the email's content.
> -* `git format-patch` is one way to create this format
> -
> -## Definitions for sample patch email
> -
> -* `Repository` is the identifier of the repository the patch applies.
> - This identifier should only be provided for repositories other than
> - `edk2`. For example `edk2-BuildSpecification` or `staging`.
> -* `Branch` is the identifier of the branch the patch applies. This
> - identifier should only be provided for branches other than `edk2/master`.
> - For example `edk2/UDK2015`, `edk2-BuildSpecification/release/1.27`, or
> - `staging/edk2-test`.
> -* `Module` is a short identifier for the affected code or documentation. For
> - example `MdePkg`, `MdeModulePkg/UsbBusDxe`, `Introduction`, or
> - `EDK II INF File Format`.
> -* `Brief-single-line-summary` is a short summary of the change.
> -* The entire first line should be less than ~70 characters.
> -* `Full-commit-message` a verbose multiple line comment describing
> - the change. Each line should be less than ~70 characters.
> -* `Signed-off-by` is the contributor's signature identifying them
> - by their real/legal name and their email address.
> -
> -# Submodules
> -
> -Submodule in EDK II is allowed but submodule chain should be avoided
> -as possible as we can. Currently EDK II contains the following submodules
> -
> -- CryptoPkg/Library/OpensslLib/openssl
> -- ArmPkg/Library/ArmSoftFloatLib/berkeley-softfloat-3
> -- MdeModulePkg/Universal/RegularExpressionDxe/oniguruma
> -- MdeModulePkg/Library/BrotliCustomDecompressLib/brotli
> -- BaseTools/Source/C/BrotliCompress/brotli
> -
> -ArmSoftFloatLib is actually required by OpensslLib. It's inevitable
> -in openssl-1.1.1 (since stable201905) for floating point parameter
> -conversion, but should be dropped once there's no such need in future
> -release of openssl.
> -
> -To get a full, buildable EDK II repository, use following steps of git
> -command
> -
> -```
> -$ git clone https://github.com/tianocore/edk2.git
> -$ cd edk2
> -$ git submodule update --init
> -$ cd ..
> -```
> -
> -If there's update for submodules, use following git commands to get the
> -latest submodules code.
> -
> -```
> -$ cd edk2
> -$ git pull
> -$ git submodule update
> -```
> -
> -Note: When cloning submodule repos, '--recursive' option is not
> -recommended. EDK II itself will not use any code/feature from
> -submodules in above submodules. So using '--recursive' adds a
> -dependency on being able to reach servers we do not actually want
> -any code from, as well as needlessly downloading code we will not
> -use.
> --
> 2.16.3.windows.1
>
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH v3 2/7] ArmVirtPkg: Add Platform CI and configuration for Core CI
2020-04-27 12:22 ` Leif Lindholm
@ 2020-04-27 21:21 ` Michael Kubacki
2020-04-28 10:23 ` Leif Lindholm
0 siblings, 1 reply; 13+ messages in thread
From: Michael Kubacki @ 2020-04-27 21:21 UTC (permalink / raw)
To: Leif Lindholm; +Cc: devel, Laszlo Ersek, Ard Biesheuvel
I believe I misunderstood the conclusion of which Signed-off-by to
include in the v1 series discussion -
https://edk2.groups.io/g/devel/message/57129.
These patches are taken from a branch on Sean's fork of edk2 as noted in
the cover letter - https://edk2.groups.io/g/devel/message/58114. Sean
has given a Signed-off-by on these patches in that branch.
Is the guidance in this case to only include my Signed-off-by?
Thanks,
Michael
On 4/27/2020 5:22 AM, Leif Lindholm wrote:
> On Fri, Apr 24, 2020 at 14:31:03 -0700, michael.kubacki@outlook.com wrote:
>> From: Sean Brogan <sean.brogan@microsoft.com>
>>
>> REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2570
>>
>> Add new Azure Pipeline definitions to build and run ArmVirtPkg with:
>> * Ubuntu GCC5
>> Add PyTool based build of ArmVirtPkg
>> Add extdep for managing the iasl dependency
>> Add ArmVirtPkg.ci.yaml for Core CI
>> Add ReadMe.md for details and instructions
>>
>> Cc: Laszlo Ersek <lersek@redhat.com>
>> Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
>> Cc: Leif Lindholm <leif@nuviainc.com>
>> Signed-off-by: Sean Brogan <sean.brogan@microsoft.com>
>> Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
>
> The contributor attests the compliance of the submission with
> https://developercertificate.org/ at the point of contribution by
> using Signed-off-by.
>
> Statements by others (including original authors) should not be copied
> when copying code between projects. The From: tag preserves authorship
> credit.
>
> /
> Leif
>
>> Reviewed-by: Shenglei Zhang <shenglei.zhang@intel.com>
>> Acked-by: Laszlo Ersek <lersek@redhat.com>
>> Acked-by: Ard Biesheuvel <ard.biesheuvel@arm.com>
>> Reviewed-by: Bret Barkelew <bret.barkelew@microsoft.com>
>> ---
>> ArmVirtPkg/ArmVirtPkg.ci.yaml | 103 ++++++++
>> ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml | 89 +++++++
>> ArmVirtPkg/PlatformCI/PlatformBuild.py | 276 ++++++++++++++++++++
>> ArmVirtPkg/PlatformCI/ReadMe.md | 125 +++++++++
>> ArmVirtPkg/PlatformCI/iasl_ext_dep.yaml | 21 ++
>> 5 files changed, 614 insertions(+)
>>
>> diff --git a/ArmVirtPkg/ArmVirtPkg.ci.yaml b/ArmVirtPkg/ArmVirtPkg.ci.yaml
>> new file mode 100644
>> index 000000000000..4553725ee528
>> --- /dev/null
>> +++ b/ArmVirtPkg/ArmVirtPkg.ci.yaml
>> @@ -0,0 +1,103 @@
>> +## @file
>> +# Core CI configuration for ArmVirtPkg
>> +#
>> +# ArmVirtPkg is part of Platform Ci for builds so this is only
>> +# used for code analysis.
>> +#
>> +# Copyright (c) Microsoft Corporation
>> +# SPDX-License-Identifier: BSD-2-Clause-Patent
>> +##
>> +{
>> + ## options defined .pytool/Plugin/CompilerPlugin
>> + "CompilerPlugin": {
>> + "DscPath": "" # Don't support this test
>> + },
>> +
>> + ## options defined .pytool/Plugin/HostUnitTestCompilerPlugin
>> + "HostUnitTestCompilerPlugin": {
>> + "DscPath": "" # Don't support this test
>> + },
>> +
>> + ## options defined .pytool/Plugin/CharEncodingCheck
>> + "CharEncodingCheck": {
>> + "IgnoreFiles": []
>> + },
>> +
>> + ## options defined .pytool/Plugin/DependencyCheck
>> + "DependencyCheck": {
>> + "AcceptableDependencies": [
>> + "MdePkg/MdePkg.dec",
>> + "MdeModulePkg/MdeModulePkg.dec",
>> + "ArmVirtPkg/ArmVirtPkg.dec",
>> + "NetworkPkg/NetworkPkg.dec",
>> + "ArmPkg/ArmPkg.dec",
>> + "OvmfPkg/OvmfPkg.dec",
>> + "EmbeddedPkg/EmbeddedPkg.dec",
>> + "ArmPlatformPkg/ArmPlatformPkg.dec",
>> + "SecurityPkg/SecurityPkg.dec",
>> + "ShellPkg/ShellPkg.dec" #Is this ok?
>> + ],
>> + # For host based unit tests
>> + "AcceptableDependencies-HOST_APPLICATION":[
>> + "UnitTestFrameworkPkg/UnitTestFrameworkPkg.dec"
>> + ],
>> + # For UEFI shell based apps
>> + "AcceptableDependencies-UEFI_APPLICATION":[
>> +
>> + ],
>> + "IgnoreInf": []
>> + },
>> +
>> + ## options defined .pytool/Plugin/DscCompleteCheck
>> + "DscCompleteCheck": {
>> + "IgnoreInf": [""],
>> + "DscPath": "" # Don't support this test
>> + },
>> +
>> + ## options defined .pytool/Plugin/HostUnitTestDscCompleteCheck
>> + "HostUnitTestDscCompleteCheck": {
>> + "IgnoreInf": [""],
>> + "DscPath": "" # Don't support this test
>> + },
>> +
>> + ## options defined .pytool/Plugin/GuidCheck
>> + "GuidCheck": {
>> + "IgnoreGuidName": [],
>> + "IgnoreGuidValue": [],
>> + "IgnoreFoldersAndFiles": [],
>> + "IgnoreDuplicates": [],
>> + },
>> +
>> + ## options defined .pytool/Plugin/LibraryClassCheck
>> + "LibraryClassCheck": {
>> + "IgnoreHeaderFile": []
>> + },
>> +
>> + ## options defined .pytool/Plugin/SpellCheck
>> + "SpellCheck": {
>> + "AuditOnly": False, # Fails right now with over 270 errors
>> + "IgnoreFiles": [], # use gitignore syntax to ignore errors in matching files
>> + "ExtendWords": [
>> + "setjump",
>> + "plong",
>> + "lparam",
>> + "lpdword",
>> + "lpthread",
>> + "lresult",
>> + "bootable",
>> + "bsymbolic",
>> + "endiannness",
>> + "fvmain",
>> + "multiboot",
>> + "qemu's",
>> + "ramdisk",
>> + "ramfb",
>> + "unbootable",
>> + "virt's",
>> + "werror",
>> + "xenio"
>> + ], # words to extend to the dictionary for this package
>> + "IgnoreStandardPaths": [], # Standard Plugin defined paths that should be ignore
>> + "AdditionalIncludePaths": [] # Additional paths to spell check (wildcards supported)
>> + }
>> +}
>> diff --git a/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml b/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
>> new file mode 100644
>> index 000000000000..b9452ec5bfaf
>> --- /dev/null
>> +++ b/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
>> @@ -0,0 +1,89 @@
>> +## @file
>> +# Azure Pipeline build file for building a platform.
>> +#
>> +# Platform: ArmVirtQemu
>> +# OS: Ubuntu
>> +# Toolchain: GCC5
>> +#
>> +# Copyright (c) Microsoft Corporation.
>> +# SPDX-License-Identifier: BSD-2-Clause-Patent
>> +##
>> +trigger:
>> + - master
>> +pr:
>> + - master
>> +
>> +jobs:
>> + - job: Platform_CI
>> + variables:
>> + package: 'ArmVirtPkg'
>> + vm_image: 'ubuntu-latest'
>> + should_run: true
>> + run_flags: "MAKE_STARTUP_NSH=TRUE QEMU_HEADLESS=TRUE"
>> +
>> + #Use matrix to speed up the build process
>> + strategy:
>> + matrix:
>> + QEMU_AARCH64_DEBUG:
>> + Build.File: "$(package)/PlatformCI/PlatformBuild.py"
>> + Build.Arch: "AARCH64"
>> + Build.Flags: ""
>> + Build.Target: "DEBUG"
>> + Run.Flags: $(run_flags)
>> + Run: $(should_run)
>> + QEMU_AARCH64_RELEASE:
>> + Build.File: "$(package)/PlatformCI/PlatformBuild.py"
>> + Build.Arch: "AARCH64"
>> + Build.Flags: ""
>> + Build.Target: "RELEASE"
>> + Run.Flags: $(run_flags)
>> + Run: $(should_run)
>> + QEMU_AARCH64_NOOPT:
>> + Build.File: "$(package)/PlatformCI/PlatformBuild.py"
>> + Build.Arch: "AARCH64"
>> + Build.Flags: ""
>> + Build.Target: "NOOPT"
>> + Run.Flags: $(run_flags)
>> + Run: $(should_run)
>> + QEMU_ARM_DEBUG:
>> + Build.File: "$(package)/PlatformCI/PlatformBuild.py"
>> + Build.Arch: "ARM"
>> + Build.Flags: ""
>> + Build.Target: "DEBUG"
>> + Run.Flags: $(run_flags)
>> + Run: $(should_run)
>> + QEMU_ARM_RELEASE:
>> + Build.File: "$(package)/PlatformCI/PlatformBuild.py"
>> + Build.Arch: "ARM"
>> + Build.Flags: ""
>> + Build.Target: "RELEASE"
>> + Run.Flags: $(run_flags)
>> + Run: $(should_run)
>> + QEMU_ARM_NOOPT:
>> + Build.File: "$(package)/PlatformCI/PlatformBuild.py"
>> + Build.Arch: "ARM"
>> + Build.Flags: ""
>> + Build.Target: "NOOPT"
>> + Run.Flags: $(run_flags)
>> + Run: $(should_run)
>> +
>> + workspace:
>> + clean: all
>> +
>> + pool:
>> + vmImage: $(vm_image)
>> +
>> + steps:
>> + - template: ../../../.azurepipelines/templates/platform-build-run-steps.yml
>> + parameters:
>> + tool_chain_tag: GCC5
>> + build_pkg: $(package)
>> + build_target: $(Build.Target)
>> + build_arch: $(Build.Arch)
>> + build_file: $(Build.File)
>> + build_flags: $(Build.Flags)
>> + run_flags: $(Run.Flags)
>> + extra_install_step:
>> + - bash: sudo apt-get install qemu
>> + displayName: Install qemu
>> + condition: and(gt(variables.pkg_count, 0), succeeded())
>> diff --git a/ArmVirtPkg/PlatformCI/PlatformBuild.py b/ArmVirtPkg/PlatformCI/PlatformBuild.py
>> new file mode 100644
>> index 000000000000..dff653e919eb
>> --- /dev/null
>> +++ b/ArmVirtPkg/PlatformCI/PlatformBuild.py
>> @@ -0,0 +1,276 @@
>> +# @file
>> +# Script to Build ArmVirtPkg UEFI firmware
>> +#
>> +# Copyright (c) Microsoft Corporation.
>> +# SPDX-License-Identifier: BSD-2-Clause-Patent
>> +##
>> +import os
>> +import logging
>> +import io
>> +
>> +from edk2toolext.environment import shell_environment
>> +from edk2toolext.environment.uefi_build import UefiBuilder
>> +from edk2toolext.invocables.edk2_platform_build import BuildSettingsManager
>> +from edk2toolext.invocables.edk2_setup import SetupSettingsManager, RequiredSubmodule
>> +from edk2toolext.invocables.edk2_update import UpdateSettingsManager
>> +from edk2toolext.invocables.edk2_pr_eval import PrEvalSettingsManager
>> +from edk2toollib.utility_functions import RunCmd
>> +from edk2toollib.utility_functions import GetHostInfo
>> +
>> +# ####################################################################################### #
>> +# Common Configuration #
>> +# ####################################################################################### #
>> +
>> +
>> +class CommonPlatform():
>> + ''' Common settings for this platform. Define static data here and use
>> + for the different parts of stuart
>> + '''
>> + PackagesSupported = ("ArmVirtPkg",)
>> + ArchSupported = ("AARCH64", "ARM")
>> + TargetsSupported = ("DEBUG", "RELEASE", "NOOPT")
>> + Scopes = ('armvirt', 'edk2-build')
>> + WorkspaceRoot = os.path.realpath(os.path.join(
>> + os.path.dirname(os.path.abspath(__file__)), "..", ".."))
>> +
>> + # ####################################################################################### #
>> + # Configuration for Update & Setup #
>> + # ####################################################################################### #
>> +
>> +
>> +class SettingsManager(UpdateSettingsManager, SetupSettingsManager, PrEvalSettingsManager):
>> +
>> + def GetPackagesSupported(self):
>> + ''' return iterable of edk2 packages supported by this build.
>> + These should be edk2 workspace relative paths '''
>> + return CommonPlatform.PackagesSupported
>> +
>> + def GetArchitecturesSupported(self):
>> + ''' return iterable of edk2 architectures supported by this build '''
>> + return CommonPlatform.ArchSupported
>> +
>> + def GetTargetsSupported(self):
>> + ''' return iterable of edk2 target tags supported by this build '''
>> + return CommonPlatform.TargetsSupported
>> +
>> + def GetRequiredSubmodules(self):
>> + ''' return iterable containing RequiredSubmodule objects.
>> + If no RequiredSubmodules return an empty iterable
>> + '''
>> + rs = []
>> +
>> + # intentionally declare this one with recursive false to avoid overhead
>> + rs.append(RequiredSubmodule(
>> + "CryptoPkg/Library/OpensslLib/openssl", False))
>> +
>> + # To avoid maintenance of this file for every new submodule
>> + # lets just parse the .gitmodules and add each if not already in list.
>> + # The GetRequiredSubmodules is designed to allow a build to optimize
>> + # the desired submodules but it isn't necessary for this repository.
>> + result = io.StringIO()
>> + ret = RunCmd("git", "config --file .gitmodules --get-regexp path", workingdir=self.GetWorkspaceRoot(), outstream=result)
>> + # Cmd output is expected to look like:
>> + # submodule.CryptoPkg/Library/OpensslLib/openssl.path CryptoPkg/Library/OpensslLib/openssl
>> + # submodule.SoftFloat.path ArmPkg/Library/ArmSoftFloatLib/berkeley-softfloat-3
>> + if ret == 0:
>> + for line in result.getvalue().splitlines():
>> + _, _, path = line.partition(" ")
>> + if path is not None:
>> + if path not in [x.path for x in rs]:
>> + rs.append(RequiredSubmodule(path, True)) # add it with recursive since we don't know
>> + return rs
>> +
>> + def SetArchitectures(self, list_of_requested_architectures):
>> + ''' Confirm the requests architecture list is valid and configure SettingsManager
>> + to run only the requested architectures.
>> +
>> + Raise Exception if a list_of_requested_architectures is not supported
>> + '''
>> + unsupported = set(list_of_requested_architectures) - \
>> + set(self.GetArchitecturesSupported())
>> + if(len(unsupported) > 0):
>> + errorString = (
>> + "Unsupported Architecture Requested: " + " ".join(unsupported))
>> + logging.critical(errorString)
>> + raise Exception(errorString)
>> + self.ActualArchitectures = list_of_requested_architectures
>> +
>> + def GetWorkspaceRoot(self):
>> + ''' get WorkspacePath '''
>> + return CommonPlatform.WorkspaceRoot
>> +
>> + def GetActiveScopes(self):
>> + ''' return tuple containing scopes that should be active for this process '''
>> +
>> + scopes = CommonPlatform.Scopes
>> + ActualToolChainTag = shell_environment.GetBuildVars().GetValue("TOOL_CHAIN_TAG", "")
>> +
>> + if GetHostInfo().os.upper() == "LINUX" and ActualToolChainTag.upper().startswith("GCC"):
>> + if "AARCH64" in self.ActualArchitectures:
>> + scopes += ("gcc_aarch64_linux",)
>> + if "ARM" in self.ActualArchitectures:
>> + scopes += ("gcc_arm_linux",)
>> + return scopes
>> +
>> + def FilterPackagesToTest(self, changedFilesList: list, potentialPackagesList: list) -> list:
>> + ''' Filter other cases that this package should be built
>> + based on changed files. This should cover things that can't
>> + be detected as dependencies. '''
>> + build_these_packages = []
>> + possible_packages = potentialPackagesList.copy()
>> + for f in changedFilesList:
>> + # BaseTools files that might change the build
>> + if "BaseTools" in f:
>> + if os.path.splitext(f) not in [".txt", ".md"]:
>> + build_these_packages = possible_packages
>> + break
>> +
>> + # if the azure pipeline platform template file changed
>> + if "platform-build-run-steps.yml" in f:
>> + build_these_packages = possible_packages
>> + break
>> +
>> +
>> + return build_these_packages
>> +
>> + def GetPlatformDscAndConfig(self) -> tuple:
>> + ''' If a platform desires to provide its DSC then Policy 4 will evaluate if
>> + any of the changes will be built in the dsc.
>> +
>> + The tuple should be (<workspace relative path to dsc file>, <input dictionary of dsc key value pairs>)
>> + '''
>> + return (os.path.join("ArmVirtPkg", "ArmVirtQemu.dsc"), {})
>> +
>> +
>> + # ####################################################################################### #
>> + # Actual Configuration for Platform Build #
>> + # ####################################################################################### #
>> +
>> +
>> +class PlatformBuilder(UefiBuilder, BuildSettingsManager):
>> + def __init__(self):
>> + UefiBuilder.__init__(self)
>> +
>> + def AddCommandLineOptions(self, parserObj):
>> + ''' Add command line options to the argparser '''
>> + parserObj.add_argument('-a', "--arch", dest="build_arch", type=str, default="AARCH64",
>> + help="Optional - Architecture to build. Default = AARCH64")
>> +
>> + def RetrieveCommandLineOptions(self, args):
>> + ''' Retrieve command line options from the argparser '''
>> +
>> + shell_environment.GetBuildVars().SetValue(
>> + "TARGET_ARCH", args.build_arch.upper(), "From CmdLine")
>> +
>> + shell_environment.GetBuildVars().SetValue(
>> + "ACTIVE_PLATFORM", "ArmVirtPkg/ArmVirtQemu.dsc", "From CmdLine")
>> +
>> + def GetWorkspaceRoot(self):
>> + ''' get WorkspacePath '''
>> + return CommonPlatform.WorkspaceRoot
>> +
>> + def GetPackagesPath(self):
>> + ''' Return a list of workspace relative paths that should be mapped as edk2 PackagesPath '''
>> + return ()
>> +
>> + def GetActiveScopes(self):
>> + ''' return tuple containing scopes that should be active for this process '''
>> + scopes = CommonPlatform.Scopes
>> + ActualToolChainTag = shell_environment.GetBuildVars().GetValue("TOOL_CHAIN_TAG", "")
>> + Arch = shell_environment.GetBuildVars().GetValue("TARGET_ARCH", "")
>> +
>> + if GetHostInfo().os.upper() == "LINUX" and ActualToolChainTag.upper().startswith("GCC"):
>> + if "AARCH64" == Arch:
>> + scopes += ("gcc_aarch64_linux",)
>> + elif "ARM" == Arch:
>> + scopes += ("gcc_arm_linux",)
>> + return scopes
>> +
>> + def GetName(self):
>> + ''' Get the name of the repo, platform, or product being build '''
>> + ''' Used for naming the log file, among others '''
>> + # check the startup nsh flag and if set then rename the log file.
>> + # this helps in CI so we don't overwrite the build log since running
>> + # uses the stuart_build command.
>> + if(shell_environment.GetBuildVars().GetValue("MAKE_STARTUP_NSH", "FALSE") == "TRUE"):
>> + return "ArmVirtPkg_With_Run"
>> + return "ArmVirtPkg"
>> +
>> + def GetLoggingLevel(self, loggerType):
>> + ''' Get the logging level for a given type
>> + base == lowest logging level supported
>> + con == Screen logging
>> + txt == plain text file logging
>> + md == markdown file logging
>> + '''
>> + return logging.DEBUG
>> +
>> + def SetPlatformEnv(self):
>> + logging.debug("PlatformBuilder SetPlatformEnv")
>> + self.env.SetValue("PRODUCT_NAME", "ArmVirtQemu", "Platform Hardcoded")
>> + self.env.SetValue("MAKE_STARTUP_NSH", "FALSE", "Default to false")
>> + self.env.SetValue("QEMU_HEADLESS", "FALSE", "Default to false")
>> + return 0
>> +
>> + def PlatformPreBuild(self):
>> + return 0
>> +
>> + def PlatformPostBuild(self):
>> + return 0
>> +
>> + def FlashRomImage(self):
>> + VirtualDrive = os.path.join(self.env.GetValue(
>> + "BUILD_OUTPUT_BASE"), "VirtualDrive")
>> + os.makedirs(VirtualDrive, exist_ok=True)
>> + OutputPath_FV = os.path.join(
>> + self.env.GetValue("BUILD_OUTPUT_BASE"), "FV")
>> + Built_FV = os.path.join(OutputPath_FV, "QEMU_EFI.fd")
>> +
>> + # pad fd to 64mb
>> + with open(Built_FV, "ab") as fvfile:
>> + fvfile.seek(0, os.SEEK_END)
>> + additional = b'\0' * ((64 * 1024 * 1024)-fvfile.tell())
>> + fvfile.write(additional)
>> +
>> + # QEMU must be on that path
>> +
>> + # Unique Command and Args parameters per ARCH
>> + if (self.env.GetValue("TARGET_ARCH").upper() == "AARCH64"):
>> + cmd = "qemu-system-aarch64"
>> + args = "-M virt"
>> + args += " -cpu cortex-a57" # emulate cpu
>> + elif(self.env.GetValue("TARGET_ARCH").upper() == "ARM"):
>> + cmd = "qemu-system-arm"
>> + args = "-M virt"
>> + args += " -cpu cortex-a15" # emulate cpu
>> + else:
>> + raise NotImplementedError()
>> +
>> + # Common Args
>> + args += " -pflash " + Built_FV # path to fw
>> + args += " -m 1024" # 1gb memory
>> + # turn off network
>> + args += " -net none"
>> + # Serial messages out
>> + args += " -serial stdio"
>> + # Mount disk with startup.nsh
>> + args += f" -drive file=fat:rw:{VirtualDrive},format=raw,media=disk"
>> +
>> + # Conditional Args
>> + if (self.env.GetValue("QEMU_HEADLESS").upper() == "TRUE"):
>> + args += " -display none" # no graphics
>> +
>> + if (self.env.GetValue("MAKE_STARTUP_NSH").upper() == "TRUE"):
>> + f = open(os.path.join(VirtualDrive, "startup.nsh"), "w")
>> + f.write("BOOT SUCCESS !!! \n")
>> + # add commands here
>> + f.write("reset -s\n")
>> + f.close()
>> +
>> + ret = RunCmd(cmd, args)
>> +
>> + if ret == 0xc0000005:
>> + # for some reason getting a c0000005 on successful return
>> + return 0
>> +
>> + return ret
>> diff --git a/ArmVirtPkg/PlatformCI/ReadMe.md b/ArmVirtPkg/PlatformCI/ReadMe.md
>> new file mode 100644
>> index 000000000000..7c11d925f59e
>> --- /dev/null
>> +++ b/ArmVirtPkg/PlatformCI/ReadMe.md
>> @@ -0,0 +1,125 @@
>> +# ArmVirtPkg - Platform CI
>> +
>> +This Readme.md describes the Azure DevOps based Platform CI for ArmVirtPkg and how
>> +to use the same Pytools based build infrastructure locally.
>> +
>> +## Supported Configuration Details
>> +
>> +This solution for building and running ArmVirtPkg has only been validated with Ubuntu
>> +18.04 and the GCC5 toolchain. Two different firmware builds are supported and are
>> +described below.
>> +
>> +| Configuration name | Architecture | DSC File |Additional Flags |
>> +| :---------- | :----- | :----- | :---- |
>> +| AARCH64 | AARCH64 | ArmVirtQemu.dsc | None |
>> +| ARM | ARM | ArmVirtQemu.dsc | None |
>> +
>> +## EDK2 Developer environment
>> +
>> +- [Python 3.8.x - Download & Install](https://www.python.org/downloads/)
>> +- [GIT - Download & Install](https://git-scm.com/download/)
>> +- [QEMU - Download, Install, and add to your path](https://www.qemu.org/download/)
>> +- [Edk2 Source](https://github.com/tianocore/edk2)
>> +- Additional packages found necessary for Ubuntu 18.04
>> + - apt-get install gcc g++ make uuid-dev
>> +
>> +Note: edksetup, Submodule initialization and manual installation of NASM, iASL, or
>> +the required cross-compiler toolchains are **not** required, this is handled by the
>> +Pytools build system.
>> +
>> +## Building with Pytools for ArmVirtPkg
>> +
>> +1. [Optional] Create a Python Virtual Environment - generally once per workspace
>> +
>> + ``` bash
>> + python -m venv <name of virtual environment>
>> + ```
>> +
>> +2. [Optional] Activate Virtual Environment - each time new shell opened
>> + - Windows
>> +
>> + ``` bash
>> + <name of virtual environment>/Scripts/activate.bat
>> + ```
>> +
>> + - Linux
>> +
>> + ```bash
>> + source <name of virtual environment>/bin/activate
>> + ```
>> +
>> +3. Install Pytools - generally once per virtual env or whenever pip-requirements.txt changes
>> +
>> + ``` bash
>> + pip install --upgrade -r pip-requirements.txt
>> + ```
>> +
>> +4. Initialize & Update Submodules - only when submodules updated
>> +
>> + ``` bash
>> + stuart_setup -c ArmVirtPkg/PlatformCI/PlatformBuild.py TOOL_CHAIN_TAG=<TOOL_CHAIN_TAG> -a <TARGET_ARCH>
>> + ```
>> +
>> +5. Initialize & Update Dependencies - only as needed when ext_deps change
>> +
>> + ``` bash
>> + stuart_update -c ArmVirtPkg/PlatformCI/PlatformBuild.py TOOL_CHAIN_TAG=<TOOL_CHAIN_TAG> -a <TARGET_ARCH>
>> + ```
>> +
>> +6. Compile the basetools if necessary - only when basetools C source files change
>> +
>> + ``` bash
>> + python BaseTools/Edk2ToolsBuild.py -t <ToolChainTag>
>> + ```
>> +
>> +7. Compile Firmware
>> +
>> + ``` bash
>> + stuart_build -c ArmVirtPkg/PlatformCI/PlatformBuild.py TOOL_CHAIN_TAG=<TOOL_CHAIN_TAG> -a <TARGET_ARCH>
>> + ```
>> +
>> + - use `stuart_build -c ArmVirtPkg/PlatformCI/PlatformBuild.py -h` option to see additional
>> + options like `--clean`
>> +
>> +8. Running Emulator
>> + - You can add `--FlashRom` to the end of your build command and the emulator will run after the
>> + build is complete.
>> + - or use the `--FlashOnly` feature to just run the emulator.
>> +
>> + ``` bash
>> + stuart_build -c ArmVirtPkg/PlatformCI/PlatformBuild.py TOOL_CHAIN_TAG=<TOOL_CHAIN_TAG> -a <TARGET_ARCH> --FlashOnly
>> + ```
>> +
>> +### Notes
>> +
>> +1. Including the expected build architecture and toolchain to the _stuart_update_ command is critical.
>> + This is because there are extra scopes and tools that will be resolved during the update step that
>> + need to match your build step.
>> +2. Configuring *ACTIVE_PLATFORM* and *TARGET_ARCH* in Conf/target.txt is **not** required. This
>> + environment is set by PlatformBuild.py based upon the `[-a <TARGET_ARCH>]` parameter.
>> +3. QEMU must be on your path. On Windows this is a manual process and not part of the QEMU installer.
>> +
>> +**NOTE:** Logging the execution output will be in the normal stuart log as well as to your console.
>> +
>> +### Custom Build Options
>> +
>> +**MAKE_STARTUP_NSH=TRUE** will output a *startup.nsh* file to the location mapped as fs0. This is
>> +used in CI in combination with the `--FlashOnly` feature to run QEMU to the UEFI shell and then execute
>> +the contents of *startup.nsh*.
>> +
>> +**QEMU_HEADLESS=TRUE** Since CI servers run headless QEMU must be told to run with no display otherwise
>> +an error occurs. Locally you don't need to set this.
>> +
>> +### Passing Build Defines
>> +
>> +To pass build defines through _stuart_build_, prepend `BLD_*_`to the define name and pass it on the
>> +command-line. _stuart_build_ currently requires values to be assigned, so add an`=1` suffix for bare defines.
>> +For example, to enable the TPM2 support, instead of the traditional "-D TPM2_ENABLE=TRUE", the stuart_build
>> +command-line would be:
>> +
>> +`stuart_build -c ArmVirtPkg/PlatformCI/PlatformBuild.py BLD_*_TPM2_ENABLE=TRUE`
>> +
>> +## References
>> +
>> +- [Installing and using Pytools](https://github.com/tianocore/edk2-pytool-extensions/blob/master/docs/using.md#installing)
>> +- More on [python virtual environments](https://docs.python.org/3/library/venv.html)
>> diff --git a/ArmVirtPkg/PlatformCI/iasl_ext_dep.yaml b/ArmVirtPkg/PlatformCI/iasl_ext_dep.yaml
>> new file mode 100644
>> index 000000000000..8869ed3ecef1
>> --- /dev/null
>> +++ b/ArmVirtPkg/PlatformCI/iasl_ext_dep.yaml
>> @@ -0,0 +1,21 @@
>> +## @file
>> +# Download iasl executable tool from a nuget.org package
>> +# - package contains different binaries based on host
>> +# Add the folder with the tool to the path
>> +#
>> +# This is only downloaded for scope armvirt thus
>> +# should have no impact on the asl compiler used by any
>> +# other platform build
>> +#
>> +# Copyright (c) Microsoft Corporation.
>> +# SPDX-License-Identifier: BSD-2-Clause-Patent
>> +##
>> +{
>> + "id": "iasl-armvirt-1",
>> + "scope": "armvirt",
>> + "type": "nuget",
>> + "name": "iasl",
>> + "source": "https://api.nuget.org/v3/index.json",
>> + "version": "20190215.0.0",
>> + "flags": ["set_path", "host_specific"],
>> +}
>> --
>> 2.16.3.windows.1
>>
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH v3 2/7] ArmVirtPkg: Add Platform CI and configuration for Core CI
2020-04-27 21:21 ` Michael Kubacki
@ 2020-04-28 10:23 ` Leif Lindholm
2020-04-28 12:47 ` Laszlo Ersek
0 siblings, 1 reply; 13+ messages in thread
From: Leif Lindholm @ 2020-04-28 10:23 UTC (permalink / raw)
To: Michael Kubacki; +Cc: devel, Laszlo Ersek, Ard Biesheuvel
On Mon, Apr 27, 2020 at 14:21:07 -0700, Michael Kubacki wrote:
> I believe I misunderstood the conclusion of which Signed-off-by to include
> in the v1 series discussion - https://edk2.groups.io/g/devel/message/57129.
>
> These patches are taken from a branch on Sean's fork of edk2 as noted in the
> cover letter - https://edk2.groups.io/g/devel/message/58114. Sean has given
> a Signed-off-by on these patches in that branch.
>
> Is the guidance in this case to only include my Signed-off-by?
Yes. You can only attest to your own legal understanding of a given
situation, not to that of others - so "passing on" attestations given
by someone else in a different context ends up being misleading.
But this isn't the first time this has been misunderstood - perhaps we
should add an explicit statement to ReadMe.rst (as it will be after
this set) that "When importing patches from external sources, existing
Signed-off-by tags must be stripped out.".
Regards,
Leif
> Thanks,
> Michael
>
> On 4/27/2020 5:22 AM, Leif Lindholm wrote:
> > On Fri, Apr 24, 2020 at 14:31:03 -0700, michael.kubacki@outlook.com wrote:
> > > From: Sean Brogan <sean.brogan@microsoft.com>
> > >
> > > REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2570
> > >
> > > Add new Azure Pipeline definitions to build and run ArmVirtPkg with:
> > > * Ubuntu GCC5
> > > Add PyTool based build of ArmVirtPkg
> > > Add extdep for managing the iasl dependency
> > > Add ArmVirtPkg.ci.yaml for Core CI
> > > Add ReadMe.md for details and instructions
> > >
> > > Cc: Laszlo Ersek <lersek@redhat.com>
> > > Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
> > > Cc: Leif Lindholm <leif@nuviainc.com>
> > > Signed-off-by: Sean Brogan <sean.brogan@microsoft.com>
> > > Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
> >
> > The contributor attests the compliance of the submission with
> > https://developercertificate.org/ at the point of contribution by
> > using Signed-off-by.
> >
> > Statements by others (including original authors) should not be copied
> > when copying code between projects. The From: tag preserves authorship
> > credit.
> >
> > /
> > Leif
> >
> > > Reviewed-by: Shenglei Zhang <shenglei.zhang@intel.com>
> > > Acked-by: Laszlo Ersek <lersek@redhat.com>
> > > Acked-by: Ard Biesheuvel <ard.biesheuvel@arm.com>
> > > Reviewed-by: Bret Barkelew <bret.barkelew@microsoft.com>
> > > ---
> > > ArmVirtPkg/ArmVirtPkg.ci.yaml | 103 ++++++++
> > > ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml | 89 +++++++
> > > ArmVirtPkg/PlatformCI/PlatformBuild.py | 276 ++++++++++++++++++++
> > > ArmVirtPkg/PlatformCI/ReadMe.md | 125 +++++++++
> > > ArmVirtPkg/PlatformCI/iasl_ext_dep.yaml | 21 ++
> > > 5 files changed, 614 insertions(+)
> > >
> > > diff --git a/ArmVirtPkg/ArmVirtPkg.ci.yaml b/ArmVirtPkg/ArmVirtPkg.ci.yaml
> > > new file mode 100644
> > > index 000000000000..4553725ee528
> > > --- /dev/null
> > > +++ b/ArmVirtPkg/ArmVirtPkg.ci.yaml
> > > @@ -0,0 +1,103 @@
> > > +## @file
> > > +# Core CI configuration for ArmVirtPkg
> > > +#
> > > +# ArmVirtPkg is part of Platform Ci for builds so this is only
> > > +# used for code analysis.
> > > +#
> > > +# Copyright (c) Microsoft Corporation
> > > +# SPDX-License-Identifier: BSD-2-Clause-Patent
> > > +##
> > > +{
> > > + ## options defined .pytool/Plugin/CompilerPlugin
> > > + "CompilerPlugin": {
> > > + "DscPath": "" # Don't support this test
> > > + },
> > > +
> > > + ## options defined .pytool/Plugin/HostUnitTestCompilerPlugin
> > > + "HostUnitTestCompilerPlugin": {
> > > + "DscPath": "" # Don't support this test
> > > + },
> > > +
> > > + ## options defined .pytool/Plugin/CharEncodingCheck
> > > + "CharEncodingCheck": {
> > > + "IgnoreFiles": []
> > > + },
> > > +
> > > + ## options defined .pytool/Plugin/DependencyCheck
> > > + "DependencyCheck": {
> > > + "AcceptableDependencies": [
> > > + "MdePkg/MdePkg.dec",
> > > + "MdeModulePkg/MdeModulePkg.dec",
> > > + "ArmVirtPkg/ArmVirtPkg.dec",
> > > + "NetworkPkg/NetworkPkg.dec",
> > > + "ArmPkg/ArmPkg.dec",
> > > + "OvmfPkg/OvmfPkg.dec",
> > > + "EmbeddedPkg/EmbeddedPkg.dec",
> > > + "ArmPlatformPkg/ArmPlatformPkg.dec",
> > > + "SecurityPkg/SecurityPkg.dec",
> > > + "ShellPkg/ShellPkg.dec" #Is this ok?
> > > + ],
> > > + # For host based unit tests
> > > + "AcceptableDependencies-HOST_APPLICATION":[
> > > + "UnitTestFrameworkPkg/UnitTestFrameworkPkg.dec"
> > > + ],
> > > + # For UEFI shell based apps
> > > + "AcceptableDependencies-UEFI_APPLICATION":[
> > > +
> > > + ],
> > > + "IgnoreInf": []
> > > + },
> > > +
> > > + ## options defined .pytool/Plugin/DscCompleteCheck
> > > + "DscCompleteCheck": {
> > > + "IgnoreInf": [""],
> > > + "DscPath": "" # Don't support this test
> > > + },
> > > +
> > > + ## options defined .pytool/Plugin/HostUnitTestDscCompleteCheck
> > > + "HostUnitTestDscCompleteCheck": {
> > > + "IgnoreInf": [""],
> > > + "DscPath": "" # Don't support this test
> > > + },
> > > +
> > > + ## options defined .pytool/Plugin/GuidCheck
> > > + "GuidCheck": {
> > > + "IgnoreGuidName": [],
> > > + "IgnoreGuidValue": [],
> > > + "IgnoreFoldersAndFiles": [],
> > > + "IgnoreDuplicates": [],
> > > + },
> > > +
> > > + ## options defined .pytool/Plugin/LibraryClassCheck
> > > + "LibraryClassCheck": {
> > > + "IgnoreHeaderFile": []
> > > + },
> > > +
> > > + ## options defined .pytool/Plugin/SpellCheck
> > > + "SpellCheck": {
> > > + "AuditOnly": False, # Fails right now with over 270 errors
> > > + "IgnoreFiles": [], # use gitignore syntax to ignore errors in matching files
> > > + "ExtendWords": [
> > > + "setjump",
> > > + "plong",
> > > + "lparam",
> > > + "lpdword",
> > > + "lpthread",
> > > + "lresult",
> > > + "bootable",
> > > + "bsymbolic",
> > > + "endiannness",
> > > + "fvmain",
> > > + "multiboot",
> > > + "qemu's",
> > > + "ramdisk",
> > > + "ramfb",
> > > + "unbootable",
> > > + "virt's",
> > > + "werror",
> > > + "xenio"
> > > + ], # words to extend to the dictionary for this package
> > > + "IgnoreStandardPaths": [], # Standard Plugin defined paths that should be ignore
> > > + "AdditionalIncludePaths": [] # Additional paths to spell check (wildcards supported)
> > > + }
> > > +}
> > > diff --git a/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml b/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> > > new file mode 100644
> > > index 000000000000..b9452ec5bfaf
> > > --- /dev/null
> > > +++ b/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> > > @@ -0,0 +1,89 @@
> > > +## @file
> > > +# Azure Pipeline build file for building a platform.
> > > +#
> > > +# Platform: ArmVirtQemu
> > > +# OS: Ubuntu
> > > +# Toolchain: GCC5
> > > +#
> > > +# Copyright (c) Microsoft Corporation.
> > > +# SPDX-License-Identifier: BSD-2-Clause-Patent
> > > +##
> > > +trigger:
> > > + - master
> > > +pr:
> > > + - master
> > > +
> > > +jobs:
> > > + - job: Platform_CI
> > > + variables:
> > > + package: 'ArmVirtPkg'
> > > + vm_image: 'ubuntu-latest'
> > > + should_run: true
> > > + run_flags: "MAKE_STARTUP_NSH=TRUE QEMU_HEADLESS=TRUE"
> > > +
> > > + #Use matrix to speed up the build process
> > > + strategy:
> > > + matrix:
> > > + QEMU_AARCH64_DEBUG:
> > > + Build.File: "$(package)/PlatformCI/PlatformBuild.py"
> > > + Build.Arch: "AARCH64"
> > > + Build.Flags: ""
> > > + Build.Target: "DEBUG"
> > > + Run.Flags: $(run_flags)
> > > + Run: $(should_run)
> > > + QEMU_AARCH64_RELEASE:
> > > + Build.File: "$(package)/PlatformCI/PlatformBuild.py"
> > > + Build.Arch: "AARCH64"
> > > + Build.Flags: ""
> > > + Build.Target: "RELEASE"
> > > + Run.Flags: $(run_flags)
> > > + Run: $(should_run)
> > > + QEMU_AARCH64_NOOPT:
> > > + Build.File: "$(package)/PlatformCI/PlatformBuild.py"
> > > + Build.Arch: "AARCH64"
> > > + Build.Flags: ""
> > > + Build.Target: "NOOPT"
> > > + Run.Flags: $(run_flags)
> > > + Run: $(should_run)
> > > + QEMU_ARM_DEBUG:
> > > + Build.File: "$(package)/PlatformCI/PlatformBuild.py"
> > > + Build.Arch: "ARM"
> > > + Build.Flags: ""
> > > + Build.Target: "DEBUG"
> > > + Run.Flags: $(run_flags)
> > > + Run: $(should_run)
> > > + QEMU_ARM_RELEASE:
> > > + Build.File: "$(package)/PlatformCI/PlatformBuild.py"
> > > + Build.Arch: "ARM"
> > > + Build.Flags: ""
> > > + Build.Target: "RELEASE"
> > > + Run.Flags: $(run_flags)
> > > + Run: $(should_run)
> > > + QEMU_ARM_NOOPT:
> > > + Build.File: "$(package)/PlatformCI/PlatformBuild.py"
> > > + Build.Arch: "ARM"
> > > + Build.Flags: ""
> > > + Build.Target: "NOOPT"
> > > + Run.Flags: $(run_flags)
> > > + Run: $(should_run)
> > > +
> > > + workspace:
> > > + clean: all
> > > +
> > > + pool:
> > > + vmImage: $(vm_image)
> > > +
> > > + steps:
> > > + - template: ../../../.azurepipelines/templates/platform-build-run-steps.yml
> > > + parameters:
> > > + tool_chain_tag: GCC5
> > > + build_pkg: $(package)
> > > + build_target: $(Build.Target)
> > > + build_arch: $(Build.Arch)
> > > + build_file: $(Build.File)
> > > + build_flags: $(Build.Flags)
> > > + run_flags: $(Run.Flags)
> > > + extra_install_step:
> > > + - bash: sudo apt-get install qemu
> > > + displayName: Install qemu
> > > + condition: and(gt(variables.pkg_count, 0), succeeded())
> > > diff --git a/ArmVirtPkg/PlatformCI/PlatformBuild.py b/ArmVirtPkg/PlatformCI/PlatformBuild.py
> > > new file mode 100644
> > > index 000000000000..dff653e919eb
> > > --- /dev/null
> > > +++ b/ArmVirtPkg/PlatformCI/PlatformBuild.py
> > > @@ -0,0 +1,276 @@
> > > +# @file
> > > +# Script to Build ArmVirtPkg UEFI firmware
> > > +#
> > > +# Copyright (c) Microsoft Corporation.
> > > +# SPDX-License-Identifier: BSD-2-Clause-Patent
> > > +##
> > > +import os
> > > +import logging
> > > +import io
> > > +
> > > +from edk2toolext.environment import shell_environment
> > > +from edk2toolext.environment.uefi_build import UefiBuilder
> > > +from edk2toolext.invocables.edk2_platform_build import BuildSettingsManager
> > > +from edk2toolext.invocables.edk2_setup import SetupSettingsManager, RequiredSubmodule
> > > +from edk2toolext.invocables.edk2_update import UpdateSettingsManager
> > > +from edk2toolext.invocables.edk2_pr_eval import PrEvalSettingsManager
> > > +from edk2toollib.utility_functions import RunCmd
> > > +from edk2toollib.utility_functions import GetHostInfo
> > > +
> > > +# ####################################################################################### #
> > > +# Common Configuration #
> > > +# ####################################################################################### #
> > > +
> > > +
> > > +class CommonPlatform():
> > > + ''' Common settings for this platform. Define static data here and use
> > > + for the different parts of stuart
> > > + '''
> > > + PackagesSupported = ("ArmVirtPkg",)
> > > + ArchSupported = ("AARCH64", "ARM")
> > > + TargetsSupported = ("DEBUG", "RELEASE", "NOOPT")
> > > + Scopes = ('armvirt', 'edk2-build')
> > > + WorkspaceRoot = os.path.realpath(os.path.join(
> > > + os.path.dirname(os.path.abspath(__file__)), "..", ".."))
> > > +
> > > + # ####################################################################################### #
> > > + # Configuration for Update & Setup #
> > > + # ####################################################################################### #
> > > +
> > > +
> > > +class SettingsManager(UpdateSettingsManager, SetupSettingsManager, PrEvalSettingsManager):
> > > +
> > > + def GetPackagesSupported(self):
> > > + ''' return iterable of edk2 packages supported by this build.
> > > + These should be edk2 workspace relative paths '''
> > > + return CommonPlatform.PackagesSupported
> > > +
> > > + def GetArchitecturesSupported(self):
> > > + ''' return iterable of edk2 architectures supported by this build '''
> > > + return CommonPlatform.ArchSupported
> > > +
> > > + def GetTargetsSupported(self):
> > > + ''' return iterable of edk2 target tags supported by this build '''
> > > + return CommonPlatform.TargetsSupported
> > > +
> > > + def GetRequiredSubmodules(self):
> > > + ''' return iterable containing RequiredSubmodule objects.
> > > + If no RequiredSubmodules return an empty iterable
> > > + '''
> > > + rs = []
> > > +
> > > + # intentionally declare this one with recursive false to avoid overhead
> > > + rs.append(RequiredSubmodule(
> > > + "CryptoPkg/Library/OpensslLib/openssl", False))
> > > +
> > > + # To avoid maintenance of this file for every new submodule
> > > + # lets just parse the .gitmodules and add each if not already in list.
> > > + # The GetRequiredSubmodules is designed to allow a build to optimize
> > > + # the desired submodules but it isn't necessary for this repository.
> > > + result = io.StringIO()
> > > + ret = RunCmd("git", "config --file .gitmodules --get-regexp path", workingdir=self.GetWorkspaceRoot(), outstream=result)
> > > + # Cmd output is expected to look like:
> > > + # submodule.CryptoPkg/Library/OpensslLib/openssl.path CryptoPkg/Library/OpensslLib/openssl
> > > + # submodule.SoftFloat.path ArmPkg/Library/ArmSoftFloatLib/berkeley-softfloat-3
> > > + if ret == 0:
> > > + for line in result.getvalue().splitlines():
> > > + _, _, path = line.partition(" ")
> > > + if path is not None:
> > > + if path not in [x.path for x in rs]:
> > > + rs.append(RequiredSubmodule(path, True)) # add it with recursive since we don't know
> > > + return rs
> > > +
> > > + def SetArchitectures(self, list_of_requested_architectures):
> > > + ''' Confirm the requests architecture list is valid and configure SettingsManager
> > > + to run only the requested architectures.
> > > +
> > > + Raise Exception if a list_of_requested_architectures is not supported
> > > + '''
> > > + unsupported = set(list_of_requested_architectures) - \
> > > + set(self.GetArchitecturesSupported())
> > > + if(len(unsupported) > 0):
> > > + errorString = (
> > > + "Unsupported Architecture Requested: " + " ".join(unsupported))
> > > + logging.critical(errorString)
> > > + raise Exception(errorString)
> > > + self.ActualArchitectures = list_of_requested_architectures
> > > +
> > > + def GetWorkspaceRoot(self):
> > > + ''' get WorkspacePath '''
> > > + return CommonPlatform.WorkspaceRoot
> > > +
> > > + def GetActiveScopes(self):
> > > + ''' return tuple containing scopes that should be active for this process '''
> > > +
> > > + scopes = CommonPlatform.Scopes
> > > + ActualToolChainTag = shell_environment.GetBuildVars().GetValue("TOOL_CHAIN_TAG", "")
> > > +
> > > + if GetHostInfo().os.upper() == "LINUX" and ActualToolChainTag.upper().startswith("GCC"):
> > > + if "AARCH64" in self.ActualArchitectures:
> > > + scopes += ("gcc_aarch64_linux",)
> > > + if "ARM" in self.ActualArchitectures:
> > > + scopes += ("gcc_arm_linux",)
> > > + return scopes
> > > +
> > > + def FilterPackagesToTest(self, changedFilesList: list, potentialPackagesList: list) -> list:
> > > + ''' Filter other cases that this package should be built
> > > + based on changed files. This should cover things that can't
> > > + be detected as dependencies. '''
> > > + build_these_packages = []
> > > + possible_packages = potentialPackagesList.copy()
> > > + for f in changedFilesList:
> > > + # BaseTools files that might change the build
> > > + if "BaseTools" in f:
> > > + if os.path.splitext(f) not in [".txt", ".md"]:
> > > + build_these_packages = possible_packages
> > > + break
> > > +
> > > + # if the azure pipeline platform template file changed
> > > + if "platform-build-run-steps.yml" in f:
> > > + build_these_packages = possible_packages
> > > + break
> > > +
> > > +
> > > + return build_these_packages
> > > +
> > > + def GetPlatformDscAndConfig(self) -> tuple:
> > > + ''' If a platform desires to provide its DSC then Policy 4 will evaluate if
> > > + any of the changes will be built in the dsc.
> > > +
> > > + The tuple should be (<workspace relative path to dsc file>, <input dictionary of dsc key value pairs>)
> > > + '''
> > > + return (os.path.join("ArmVirtPkg", "ArmVirtQemu.dsc"), {})
> > > +
> > > +
> > > + # ####################################################################################### #
> > > + # Actual Configuration for Platform Build #
> > > + # ####################################################################################### #
> > > +
> > > +
> > > +class PlatformBuilder(UefiBuilder, BuildSettingsManager):
> > > + def __init__(self):
> > > + UefiBuilder.__init__(self)
> > > +
> > > + def AddCommandLineOptions(self, parserObj):
> > > + ''' Add command line options to the argparser '''
> > > + parserObj.add_argument('-a', "--arch", dest="build_arch", type=str, default="AARCH64",
> > > + help="Optional - Architecture to build. Default = AARCH64")
> > > +
> > > + def RetrieveCommandLineOptions(self, args):
> > > + ''' Retrieve command line options from the argparser '''
> > > +
> > > + shell_environment.GetBuildVars().SetValue(
> > > + "TARGET_ARCH", args.build_arch.upper(), "From CmdLine")
> > > +
> > > + shell_environment.GetBuildVars().SetValue(
> > > + "ACTIVE_PLATFORM", "ArmVirtPkg/ArmVirtQemu.dsc", "From CmdLine")
> > > +
> > > + def GetWorkspaceRoot(self):
> > > + ''' get WorkspacePath '''
> > > + return CommonPlatform.WorkspaceRoot
> > > +
> > > + def GetPackagesPath(self):
> > > + ''' Return a list of workspace relative paths that should be mapped as edk2 PackagesPath '''
> > > + return ()
> > > +
> > > + def GetActiveScopes(self):
> > > + ''' return tuple containing scopes that should be active for this process '''
> > > + scopes = CommonPlatform.Scopes
> > > + ActualToolChainTag = shell_environment.GetBuildVars().GetValue("TOOL_CHAIN_TAG", "")
> > > + Arch = shell_environment.GetBuildVars().GetValue("TARGET_ARCH", "")
> > > +
> > > + if GetHostInfo().os.upper() == "LINUX" and ActualToolChainTag.upper().startswith("GCC"):
> > > + if "AARCH64" == Arch:
> > > + scopes += ("gcc_aarch64_linux",)
> > > + elif "ARM" == Arch:
> > > + scopes += ("gcc_arm_linux",)
> > > + return scopes
> > > +
> > > + def GetName(self):
> > > + ''' Get the name of the repo, platform, or product being build '''
> > > + ''' Used for naming the log file, among others '''
> > > + # check the startup nsh flag and if set then rename the log file.
> > > + # this helps in CI so we don't overwrite the build log since running
> > > + # uses the stuart_build command.
> > > + if(shell_environment.GetBuildVars().GetValue("MAKE_STARTUP_NSH", "FALSE") == "TRUE"):
> > > + return "ArmVirtPkg_With_Run"
> > > + return "ArmVirtPkg"
> > > +
> > > + def GetLoggingLevel(self, loggerType):
> > > + ''' Get the logging level for a given type
> > > + base == lowest logging level supported
> > > + con == Screen logging
> > > + txt == plain text file logging
> > > + md == markdown file logging
> > > + '''
> > > + return logging.DEBUG
> > > +
> > > + def SetPlatformEnv(self):
> > > + logging.debug("PlatformBuilder SetPlatformEnv")
> > > + self.env.SetValue("PRODUCT_NAME", "ArmVirtQemu", "Platform Hardcoded")
> > > + self.env.SetValue("MAKE_STARTUP_NSH", "FALSE", "Default to false")
> > > + self.env.SetValue("QEMU_HEADLESS", "FALSE", "Default to false")
> > > + return 0
> > > +
> > > + def PlatformPreBuild(self):
> > > + return 0
> > > +
> > > + def PlatformPostBuild(self):
> > > + return 0
> > > +
> > > + def FlashRomImage(self):
> > > + VirtualDrive = os.path.join(self.env.GetValue(
> > > + "BUILD_OUTPUT_BASE"), "VirtualDrive")
> > > + os.makedirs(VirtualDrive, exist_ok=True)
> > > + OutputPath_FV = os.path.join(
> > > + self.env.GetValue("BUILD_OUTPUT_BASE"), "FV")
> > > + Built_FV = os.path.join(OutputPath_FV, "QEMU_EFI.fd")
> > > +
> > > + # pad fd to 64mb
> > > + with open(Built_FV, "ab") as fvfile:
> > > + fvfile.seek(0, os.SEEK_END)
> > > + additional = b'\0' * ((64 * 1024 * 1024)-fvfile.tell())
> > > + fvfile.write(additional)
> > > +
> > > + # QEMU must be on that path
> > > +
> > > + # Unique Command and Args parameters per ARCH
> > > + if (self.env.GetValue("TARGET_ARCH").upper() == "AARCH64"):
> > > + cmd = "qemu-system-aarch64"
> > > + args = "-M virt"
> > > + args += " -cpu cortex-a57" # emulate cpu
> > > + elif(self.env.GetValue("TARGET_ARCH").upper() == "ARM"):
> > > + cmd = "qemu-system-arm"
> > > + args = "-M virt"
> > > + args += " -cpu cortex-a15" # emulate cpu
> > > + else:
> > > + raise NotImplementedError()
> > > +
> > > + # Common Args
> > > + args += " -pflash " + Built_FV # path to fw
> > > + args += " -m 1024" # 1gb memory
> > > + # turn off network
> > > + args += " -net none"
> > > + # Serial messages out
> > > + args += " -serial stdio"
> > > + # Mount disk with startup.nsh
> > > + args += f" -drive file=fat:rw:{VirtualDrive},format=raw,media=disk"
> > > +
> > > + # Conditional Args
> > > + if (self.env.GetValue("QEMU_HEADLESS").upper() == "TRUE"):
> > > + args += " -display none" # no graphics
> > > +
> > > + if (self.env.GetValue("MAKE_STARTUP_NSH").upper() == "TRUE"):
> > > + f = open(os.path.join(VirtualDrive, "startup.nsh"), "w")
> > > + f.write("BOOT SUCCESS !!! \n")
> > > + # add commands here
> > > + f.write("reset -s\n")
> > > + f.close()
> > > +
> > > + ret = RunCmd(cmd, args)
> > > +
> > > + if ret == 0xc0000005:
> > > + # for some reason getting a c0000005 on successful return
> > > + return 0
> > > +
> > > + return ret
> > > diff --git a/ArmVirtPkg/PlatformCI/ReadMe.md b/ArmVirtPkg/PlatformCI/ReadMe.md
> > > new file mode 100644
> > > index 000000000000..7c11d925f59e
> > > --- /dev/null
> > > +++ b/ArmVirtPkg/PlatformCI/ReadMe.md
> > > @@ -0,0 +1,125 @@
> > > +# ArmVirtPkg - Platform CI
> > > +
> > > +This Readme.md describes the Azure DevOps based Platform CI for ArmVirtPkg and how
> > > +to use the same Pytools based build infrastructure locally.
> > > +
> > > +## Supported Configuration Details
> > > +
> > > +This solution for building and running ArmVirtPkg has only been validated with Ubuntu
> > > +18.04 and the GCC5 toolchain. Two different firmware builds are supported and are
> > > +described below.
> > > +
> > > +| Configuration name | Architecture | DSC File |Additional Flags |
> > > +| :---------- | :----- | :----- | :---- |
> > > +| AARCH64 | AARCH64 | ArmVirtQemu.dsc | None |
> > > +| ARM | ARM | ArmVirtQemu.dsc | None |
> > > +
> > > +## EDK2 Developer environment
> > > +
> > > +- [Python 3.8.x - Download & Install](https://www.python.org/downloads/)
> > > +- [GIT - Download & Install](https://git-scm.com/download/)
> > > +- [QEMU - Download, Install, and add to your path](https://www.qemu.org/download/)
> > > +- [Edk2 Source](https://github.com/tianocore/edk2)
> > > +- Additional packages found necessary for Ubuntu 18.04
> > > + - apt-get install gcc g++ make uuid-dev
> > > +
> > > +Note: edksetup, Submodule initialization and manual installation of NASM, iASL, or
> > > +the required cross-compiler toolchains are **not** required, this is handled by the
> > > +Pytools build system.
> > > +
> > > +## Building with Pytools for ArmVirtPkg
> > > +
> > > +1. [Optional] Create a Python Virtual Environment - generally once per workspace
> > > +
> > > + ``` bash
> > > + python -m venv <name of virtual environment>
> > > + ```
> > > +
> > > +2. [Optional] Activate Virtual Environment - each time new shell opened
> > > + - Windows
> > > +
> > > + ``` bash
> > > + <name of virtual environment>/Scripts/activate.bat
> > > + ```
> > > +
> > > + - Linux
> > > +
> > > + ```bash
> > > + source <name of virtual environment>/bin/activate
> > > + ```
> > > +
> > > +3. Install Pytools - generally once per virtual env or whenever pip-requirements.txt changes
> > > +
> > > + ``` bash
> > > + pip install --upgrade -r pip-requirements.txt
> > > + ```
> > > +
> > > +4. Initialize & Update Submodules - only when submodules updated
> > > +
> > > + ``` bash
> > > + stuart_setup -c ArmVirtPkg/PlatformCI/PlatformBuild.py TOOL_CHAIN_TAG=<TOOL_CHAIN_TAG> -a <TARGET_ARCH>
> > > + ```
> > > +
> > > +5. Initialize & Update Dependencies - only as needed when ext_deps change
> > > +
> > > + ``` bash
> > > + stuart_update -c ArmVirtPkg/PlatformCI/PlatformBuild.py TOOL_CHAIN_TAG=<TOOL_CHAIN_TAG> -a <TARGET_ARCH>
> > > + ```
> > > +
> > > +6. Compile the basetools if necessary - only when basetools C source files change
> > > +
> > > + ``` bash
> > > + python BaseTools/Edk2ToolsBuild.py -t <ToolChainTag>
> > > + ```
> > > +
> > > +7. Compile Firmware
> > > +
> > > + ``` bash
> > > + stuart_build -c ArmVirtPkg/PlatformCI/PlatformBuild.py TOOL_CHAIN_TAG=<TOOL_CHAIN_TAG> -a <TARGET_ARCH>
> > > + ```
> > > +
> > > + - use `stuart_build -c ArmVirtPkg/PlatformCI/PlatformBuild.py -h` option to see additional
> > > + options like `--clean`
> > > +
> > > +8. Running Emulator
> > > + - You can add `--FlashRom` to the end of your build command and the emulator will run after the
> > > + build is complete.
> > > + - or use the `--FlashOnly` feature to just run the emulator.
> > > +
> > > + ``` bash
> > > + stuart_build -c ArmVirtPkg/PlatformCI/PlatformBuild.py TOOL_CHAIN_TAG=<TOOL_CHAIN_TAG> -a <TARGET_ARCH> --FlashOnly
> > > + ```
> > > +
> > > +### Notes
> > > +
> > > +1. Including the expected build architecture and toolchain to the _stuart_update_ command is critical.
> > > + This is because there are extra scopes and tools that will be resolved during the update step that
> > > + need to match your build step.
> > > +2. Configuring *ACTIVE_PLATFORM* and *TARGET_ARCH* in Conf/target.txt is **not** required. This
> > > + environment is set by PlatformBuild.py based upon the `[-a <TARGET_ARCH>]` parameter.
> > > +3. QEMU must be on your path. On Windows this is a manual process and not part of the QEMU installer.
> > > +
> > > +**NOTE:** Logging the execution output will be in the normal stuart log as well as to your console.
> > > +
> > > +### Custom Build Options
> > > +
> > > +**MAKE_STARTUP_NSH=TRUE** will output a *startup.nsh* file to the location mapped as fs0. This is
> > > +used in CI in combination with the `--FlashOnly` feature to run QEMU to the UEFI shell and then execute
> > > +the contents of *startup.nsh*.
> > > +
> > > +**QEMU_HEADLESS=TRUE** Since CI servers run headless QEMU must be told to run with no display otherwise
> > > +an error occurs. Locally you don't need to set this.
> > > +
> > > +### Passing Build Defines
> > > +
> > > +To pass build defines through _stuart_build_, prepend `BLD_*_`to the define name and pass it on the
> > > +command-line. _stuart_build_ currently requires values to be assigned, so add an`=1` suffix for bare defines.
> > > +For example, to enable the TPM2 support, instead of the traditional "-D TPM2_ENABLE=TRUE", the stuart_build
> > > +command-line would be:
> > > +
> > > +`stuart_build -c ArmVirtPkg/PlatformCI/PlatformBuild.py BLD_*_TPM2_ENABLE=TRUE`
> > > +
> > > +## References
> > > +
> > > +- [Installing and using Pytools](https://github.com/tianocore/edk2-pytool-extensions/blob/master/docs/using.md#installing)
> > > +- More on [python virtual environments](https://docs.python.org/3/library/venv.html)
> > > diff --git a/ArmVirtPkg/PlatformCI/iasl_ext_dep.yaml b/ArmVirtPkg/PlatformCI/iasl_ext_dep.yaml
> > > new file mode 100644
> > > index 000000000000..8869ed3ecef1
> > > --- /dev/null
> > > +++ b/ArmVirtPkg/PlatformCI/iasl_ext_dep.yaml
> > > @@ -0,0 +1,21 @@
> > > +## @file
> > > +# Download iasl executable tool from a nuget.org package
> > > +# - package contains different binaries based on host
> > > +# Add the folder with the tool to the path
> > > +#
> > > +# This is only downloaded for scope armvirt thus
> > > +# should have no impact on the asl compiler used by any
> > > +# other platform build
> > > +#
> > > +# Copyright (c) Microsoft Corporation.
> > > +# SPDX-License-Identifier: BSD-2-Clause-Patent
> > > +##
> > > +{
> > > + "id": "iasl-armvirt-1",
> > > + "scope": "armvirt",
> > > + "type": "nuget",
> > > + "name": "iasl",
> > > + "source": "https://api.nuget.org/v3/index.json",
> > > + "version": "20190215.0.0",
> > > + "flags": ["set_path", "host_specific"],
> > > +}
> > > --
> > > 2.16.3.windows.1
> > >
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [edk2-devel] [PATCH v3 3/7] EmulatorPkg: Add Platform CI and configuration for Core CI
2020-04-24 21:31 ` [PATCH v3 3/7] EmulatorPkg: " Michael Kubacki
@ 2020-04-28 11:22 ` Liming Gao
0 siblings, 0 replies; 13+ messages in thread
From: Liming Gao @ 2020-04-28 11:22 UTC (permalink / raw)
To: devel@edk2.groups.io, michael.kubacki@outlook.com
Cc: Justen, Jordan L, Andrew Fish, Ni, Ray, Gao, Liming
I and Ray have replied this patch on version 2. Seemly, this version miss it.
Please add Acked-by: Ray Ni <ray.ni@intel.com>, Reviewed-by: Liming Gao <liming.gao@intel.com>
https://edk2.groups.io/g/devel/message/57743
Thanks
Liming
> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Michael Kubacki
> Sent: Saturday, April 25, 2020 5:31 AM
> To: devel@edk2.groups.io
> Cc: Justen, Jordan L <jordan.l.justen@intel.com>; Andrew Fish <afish@apple.com>; Ni, Ray <ray.ni@intel.com>
> Subject: [edk2-devel] [PATCH v3 3/7] EmulatorPkg: Add Platform CI and configuration for Core CI
>
> From: Sean Brogan <sean.brogan@microsoft.com>
>
> REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2570
>
> Add new Azure Pipeline definitions to build and run EmulatorPkg with:
> * Ubuntu GCC5
> * Windows VS2019
> Add PyTool based build of EmulatorPkg
> Add EmulatorPkg.ci.yaml for Core CI
> Add ReadMe.md for details and instructions
>
> Cc: Jordan Justen <jordan.l.justen@intel.com>
> Cc: Andrew Fish <afish@apple.com>
> Cc: Ray Ni <ray.ni@intel.com>
> Signed-off-by: Sean Brogan <sean.brogan@microsoft.com>
> Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
> Reviewed-by: Shenglei Zhang <shenglei.zhang@intel.com>
> Reviewed-by: Bret Barkelew <bret.barkelew@microsoft.com>
> ---
> EmulatorPkg/EmulatorPkg.ci.yaml | 85 ++++++
> EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml | 95 +++++++
> EmulatorPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml | 85 ++++++
> EmulatorPkg/PlatformCI/PlatformBuild.py | 272 ++++++++++++++++++++
> EmulatorPkg/PlatformCI/ReadMe.md | 128 +++++++++
> 5 files changed, 665 insertions(+)
>
> diff --git a/EmulatorPkg/EmulatorPkg.ci.yaml b/EmulatorPkg/EmulatorPkg.ci.yaml
> new file mode 100644
> index 000000000000..81f81780ec76
> --- /dev/null
> +++ b/EmulatorPkg/EmulatorPkg.ci.yaml
> @@ -0,0 +1,85 @@
> +## @file
> +# Core CI configuration for EmulatorPkg
> +#
> +# EmulatorPkg is part of Platform Ci for builds so this is only
> +# used for code analysis.
> +#
> +# Copyright (c) Microsoft Corporation
> +# SPDX-License-Identifier: BSD-2-Clause-Patent
> +##
> +{
> + ## options defined .pytool/Plugin/CompilerPlugin
> + "CompilerPlugin": {
> + "DscPath": "" # Don't support this test
> + },
> +
> + ## options defined .pytool/Plugin/HostUnitTestCompilerPlugin
> + "HostUnitTestCompilerPlugin": {
> + "DscPath": "" # Don't support this test
> + },
> +
> + ## options defined .pytool/Plugin/CharEncodingCheck
> + "CharEncodingCheck": {
> + "IgnoreFiles": []
> + },
> +
> + ## options defined .pytool/Plugin/DependencyCheck
> + "DependencyCheck": {
> + "AcceptableDependencies": [
> + # For this platform all packages are allowed???
> + "MdePkg/MdePkg.dec",
> + "MdeModulePkg/MdeModulePkg.dec",
> + "EmulatorPkg/EmulatorPkg.dec",
> + "NetworkPkg/NetworkPkg.dec",
> + "EmbeddedPkg/EmbeddedPkg.dec", ## is this one OK??
> + ],
> + # For host based unit tests
> + "AcceptableDependencies-HOST_APPLICATION":[
> + "UnitTestFrameworkPkg/UnitTestFrameworkPkg.dec"
> + ],
> + # For UEFI shell based apps
> + "AcceptableDependencies-UEFI_APPLICATION":[],
> + "IgnoreInf": []
> + },
> +
> + ## options defined .pytool/Plugin/DscCompleteCheck
> + "DscCompleteCheck": {
> + "IgnoreInf": [""],
> + "DscPath": "" # Don't support this test
> + },
> +
> + ## options defined .pytool/Plugin/HostUnitTestDscCompleteCheck
> + "HostUnitTestDscCompleteCheck": {
> + "IgnoreInf": [""],
> + "DscPath": "" # Don't support this test
> + },
> +
> + ## options defined .pytool/Plugin/GuidCheck
> + "GuidCheck": {
> + "IgnoreGuidName": [],
> + "IgnoreGuidValue": [],
> + "IgnoreFoldersAndFiles": [],
> + "IgnoreDuplicates": [],
> + },
> +
> + ## options defined .pytool/Plugin/LibraryClassCheck
> + "LibraryClassCheck": {
> + "IgnoreHeaderFile": []
> + },
> +
> + ## options defined .pytool/Plugin/SpellCheck
> + "SpellCheck": {
> + "AuditOnly": True, # Fails right now with over 270 errors
> + "IgnoreFiles": [], # use gitignore syntax to ignore errors in matching files
> + "ExtendWords": [
> + "setjump",
> + "plong",
> + "lparam",
> + "lpdword",
> + "lpthread",
> + "lresult",
> + ], # words to extend to the dictionary for this package
> + "IgnoreStandardPaths": [], # Standard Plugin defined paths that should be ignore
> + "AdditionalIncludePaths": [] # Additional paths to spell check (wildcards supported)
> + }
> +}
> diff --git a/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml b/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> new file mode 100644
> index 000000000000..12ef8226ff54
> --- /dev/null
> +++ b/EmulatorPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml
> @@ -0,0 +1,95 @@
> +## @file
> +# Azure Pipeline build file for building a platform.
> +#
> +# Platform: EmulatorPkg
> +# OS: Ubuntu
> +# Toolchain: GCC5
> +#
> +# Copyright (c) Microsoft Corporation.
> +# SPDX-License-Identifier: BSD-2-Clause-Patent
> +##
> +trigger:
> + - master
> +pr:
> + - master
> +jobs:
> + - job: Platform_CI
> + variables:
> + package: 'EmulatorPkg'
> + vm_image: 'ubuntu-latest'
> + should_run: false
> + run_flags: "MAKE_STARTUP_NSH=TRUE"
> +
> + #Use matrix to speed up the build process
> + strategy:
> + matrix:
> + EmulatorPkg_X64_DEBUG:
> + Build.File: "$(package)/PlatformCI/PlatformBuild.py"
> + Build.Arch: "X64"
> + Build.Flags: ""
> + Build.Target: "DEBUG"
> + Run.Flags: $(run_flags)
> + Run: $(should_run)
> + EmulatorPkg_X64_RELEASE:
> + Build.File: "$(package)/PlatformCI/PlatformBuild.py"
> + Build.Arch: "X64"
> + Build.Flags: ""
> + Build.Target: "RELEASE"
> + Run.Flags: $(run_flags)
> + Run: $(should_run)
> + EmulatorPkg_X64_NOOPT:
> + Build.File: "$(package)/PlatformCI/PlatformBuild.py"
> + Build.Arch: "X64"
> + Build.Flags: ""
> + Build.Target: "NOOPT"
> + Run.Flags: $(run_flags)
> + Run: $(should_run)
> + EmulatorPkg_IA32_DEBUG:
> + Build.File: "$(package)/PlatformCI/PlatformBuild.py"
> + Build.Arch: "IA32"
> + Build.Flags: ""
> + Build.Target: "DEBUG"
> + Run.Flags: $(run_flags)
> + Run: $(should_run)
> + EmulatorPkg_IA32_RELEASE:
> + Build.File: "$(package)/PlatformCI/PlatformBuild.py"
> + Build.Arch: "IA32"
> + Build.Flags: ""
> + Build.Target: "RELEASE"
> + Run.Flags: $(run_flags)
> + Run: $(should_run)
> + EmulatorPkg_IA32_NOOPT:
> + Build.File: "$(package)/PlatformCI/PlatformBuild.py"
> + Build.Arch: "IA32"
> + Build.Flags: ""
> + Build.Target: "NOOPT"
> + Run.Flags: $(run_flags)
> + Run: $(should_run)
> +
> + workspace:
> + clean: all
> +
> + pool:
> + vmImage: $(vm_image)
> +
> + steps:
> + - template: ../../../.azurepipelines/templates/platform-build-run-steps.yml
> + parameters:
> + tool_chain_tag: GCC5
> + build_pkg: $(package)
> + build_target: $(Build.Target)
> + build_arch: $(Build.Arch)
> + build_file: $(Build.File)
> + build_flags: $(Build.Flags)
> + run_flags: $(Run.Flags)
> + # Add steps to install some IA32 only dependencies
> + extra_install_step:
> + - bash: sudo dpkg --add-architecture i386
> + displayName: Add i386 to dpkg
> + condition: and(gt(variables.pkg_count, 0), eq(variables['Build.Arch'], 'IA32'), succeeded())
> + - bash: sudo apt-get update
> + displayName: do apt-get update
> + condition: and(gt(variables.pkg_count, 0), eq(variables['Build.Arch'], 'IA32'), succeeded())
> + - bash: sudo apt-get install libc6-dev:i386 libx11-dev:i386 libxext-dev:i386 lib32gcc-7-dev
> + displayName: Add additional i386 packages
> + condition: and(gt(variables.pkg_count, 0), eq(variables['Build.Arch'], 'IA32'), succeeded())
> diff --git a/EmulatorPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml b/EmulatorPkg/PlatformCI/.azurepipelines/Windows-
> VS2019.yml
> new file mode 100644
> index 000000000000..a5baf4b6064b
> --- /dev/null
> +++ b/EmulatorPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml
> @@ -0,0 +1,85 @@
> +## @file
> +# Azure Pipeline build file for building a platform.
> +#
> +# Platform: EMULATORPKG
> +# OS: Windows
> +# Toolchain: VS2019
> +#
> +# Copyright (c) Microsoft Corporation.
> +# SPDX-License-Identifier: BSD-2-Clause-Patent
> +##
> +trigger:
> + - master
> +pr:
> + - master
> +
> +jobs:
> + - job: Platform_CI
> + variables:
> + package: 'EmulatorPkg'
> + vm_image: 'windows-latest'
> + should_run: true
> + run_flags: "MAKE_STARTUP_NSH=TRUE"
> +
> + #Use matrix to speed up the build process
> + strategy:
> + matrix:
> + EmulatorPkg_X64_DEBUG:
> + Build.File: "$(package)/PlatformCI/PlatformBuild.py"
> + Build.Arch: "X64"
> + Build.Flags: ""
> + Build.Target: "DEBUG"
> + Run.Flags: $(run_flags)
> + Run: $(should_run)
> + EmulatorPkg_X64_RELEASE:
> + Build.File: "$(package)/PlatformCI/PlatformBuild.py"
> + Build.Arch: "X64"
> + Build.Flags: ""
> + Build.Target: "RELEASE"
> + Run.Flags: $(run_flags)
> + Run: $(should_run)
> + EmulatorPkg_X64_NOOPT:
> + Build.File: "$(package)/PlatformCI/PlatformBuild.py"
> + Build.Arch: "X64"
> + Build.Flags: ""
> + Build.Target: "NOOPT"
> + Run.Flags: $(run_flags)
> + Run: $(should_run)
> + EmulatorPkg_IA32_DEBUG:
> + Build.File: "$(package)/PlatformCI/PlatformBuild.py"
> + Build.Arch: "IA32 "
> + Build.Flags: ""
> + Build.Target: "DEBUG"
> + Run.Flags: $(run_flags)
> + Run: $(should_run)
> + EmulatorPkg_IA32_RELEASE:
> + Build.File: "$(package)/PlatformCI/PlatformBuild.py"
> + Build.Arch: "IA32 "
> + Build.Flags: ""
> + Build.Target: "RELEASE"
> + Run.Flags: $(run_flags)
> + Run: $(should_run)
> + EmulatorPkg_IA32_NOOPT:
> + Build.File: "$(package)/PlatformCI/PlatformBuild.py"
> + Build.Arch: "IA32 "
> + Build.Flags: ""
> + Build.Target: "NOOPT"
> + Run.Flags: $(run_flags)
> + Run: $(should_run)
> +
> + workspace:
> + clean: all
> +
> + pool:
> + vmImage: $(vm_image)
> +
> + steps:
> + - template: ../../../.azurepipelines/templates/platform-build-run-steps.yml
> + parameters:
> + tool_chain_tag: VS2019
> + build_pkg: $(package)
> + build_target: $(Build.Target)
> + build_arch: $(Build.Arch)
> + build_file: $(Build.File)
> + build_flags: $(Build.Flags)
> + run_flags: $(Run.Flags)
> diff --git a/EmulatorPkg/PlatformCI/PlatformBuild.py b/EmulatorPkg/PlatformCI/PlatformBuild.py
> new file mode 100644
> index 000000000000..6defbf6ef148
> --- /dev/null
> +++ b/EmulatorPkg/PlatformCI/PlatformBuild.py
> @@ -0,0 +1,272 @@
> +# @file
> +# Script to Build EmulatorPkg UEFI firmware
> +#
> +# Copyright (c) Microsoft Corporation.
> +# SPDX-License-Identifier: BSD-2-Clause-Patent
> +##
> +import os
> +import logging
> +import io
> +
> +from edk2toolext.environment import shell_environment
> +from edk2toolext.environment.uefi_build import UefiBuilder
> +from edk2toolext.invocables.edk2_platform_build import BuildSettingsManager
> +from edk2toolext.invocables.edk2_setup import SetupSettingsManager, RequiredSubmodule
> +from edk2toolext.invocables.edk2_update import UpdateSettingsManager
> +from edk2toolext.invocables.edk2_pr_eval import PrEvalSettingsManager
> +from edk2toollib.utility_functions import RunCmd
> +from edk2toollib.utility_functions import GetHostInfo
> +
> +# ####################################################################################### #
> +# Common Configuration #
> +# ####################################################################################### #
> +
> +
> +class CommonPlatform():
> + ''' Common settings for this platform. Define static data here and use
> + for the different parts of stuart
> + '''
> + PackagesSupported = ("EmulatorPkg",)
> + ArchSupported = ("X64", "IA32")
> + TargetsSupported = ("DEBUG", "RELEASE", "NOOPT")
> + Scopes = ('emulatorpkg', 'edk2-build')
> + WorkspaceRoot = os.path.realpath(os.path.join(
> + os.path.dirname(os.path.abspath(__file__)), "..", ".."))
> +
> + # ####################################################################################### #
> + # Configuration for Update & Setup #
> + # ####################################################################################### #
> +
> +
> +class SettingsManager(UpdateSettingsManager, SetupSettingsManager, PrEvalSettingsManager):
> +
> + def GetPackagesSupported(self):
> + ''' return iterable of edk2 packages supported by this build.
> + These should be edk2 workspace relative paths '''
> + return CommonPlatform.PackagesSupported
> +
> + def GetArchitecturesSupported(self):
> + ''' return iterable of edk2 architectures supported by this build '''
> + return CommonPlatform.ArchSupported
> +
> + def GetTargetsSupported(self):
> + ''' return iterable of edk2 target tags supported by this build '''
> + return CommonPlatform.TargetsSupported
> +
> + def GetRequiredSubmodules(self):
> + ''' return iterable containing RequiredSubmodule objects.
> + If no RequiredSubmodules return an empty iterable
> + '''
> + rs = []
> + # intentionally declare this one with recursive false to avoid overhead
> + rs.append(RequiredSubmodule(
> + "CryptoPkg/Library/OpensslLib/openssl", False))
> +
> + # To avoid maintenance of this file for every new submodule
> + # lets just parse the .gitmodules and add each if not already in list.
> + # The GetRequiredSubmodules is designed to allow a build to optimize
> + # the desired submodules but it isn't necessary for this repository.
> + result = io.StringIO()
> + ret = RunCmd("git", "config --file .gitmodules --get-regexp path", workingdir=self.GetWorkspaceRoot(), outstream=result)
> + # Cmd output is expected to look like:
> + # submodule.CryptoPkg/Library/OpensslLib/openssl.path CryptoPkg/Library/OpensslLib/openssl
> + # submodule.SoftFloat.path ArmPkg/Library/ArmSoftFloatLib/berkeley-softfloat-3
> + if ret == 0:
> + for line in result.getvalue().splitlines():
> + _, _, path = line.partition(" ")
> + if path is not None:
> + if path not in [x.path for x in rs]:
> + rs.append(RequiredSubmodule(path, True)) # add it with recursive since we don't know
> + return rs
> +
> + def SetArchitectures(self, list_of_requested_architectures):
> + ''' Confirm the requests architecture list is valid and configure SettingsManager
> + to run only the requested architectures.
> +
> + Raise Exception if a list_of_requested_architectures is not supported
> + '''
> + unsupported = set(list_of_requested_architectures) - \
> + set(self.GetArchitecturesSupported())
> + if(len(unsupported) > 0):
> + errorString = (
> + "Unsupported Architecture Requested: " + " ".join(unsupported))
> + logging.critical(errorString)
> + raise Exception(errorString)
> + self.ActualArchitectures = list_of_requested_architectures
> +
> + def GetWorkspaceRoot(self):
> + ''' get WorkspacePath '''
> + return CommonPlatform.WorkspaceRoot
> +
> + def GetActiveScopes(self):
> + ''' return tuple containing scopes that should be active for this process '''
> + return CommonPlatform.Scopes
> +
> + def FilterPackagesToTest(self, changedFilesList: list, potentialPackagesList: list) -> list:
> + ''' Filter other cases that this package should be built
> + based on changed files. This should cover things that can't
> + be detected as dependencies. '''
> + build_these_packages = []
> + possible_packages = potentialPackagesList.copy()
> + for f in changedFilesList:
> + # BaseTools files that might change the build
> + if "BaseTools" in f:
> + if os.path.splitext(f) not in [".txt", ".md"]:
> + build_these_packages = possible_packages
> + break
> + # if the azure pipeline platform template file changed
> + if "platform-build-run-steps.yml" in f:
> + build_these_packages = possible_packages
> + break
> + return build_these_packages
> +
> + def GetPlatformDscAndConfig(self) -> tuple:
> + ''' If a platform desires to provide its DSC then Policy 4 will evaluate if
> + any of the changes will be built in the dsc.
> +
> + The tuple should be (<workspace relative path to dsc file>, <input dictionary of dsc key value pairs>)
> + '''
> + return (os.path.join("EmulatorPkg", "EmulatorPkg.dsc"), {})
> +
> + # ####################################################################################### #
> + # Actual Configuration for Platform Build #
> + # ####################################################################################### #
> +
> +
> +class PlatformBuilder(UefiBuilder, BuildSettingsManager):
> + def __init__(self):
> + UefiBuilder.__init__(self)
> +
> + def AddCommandLineOptions(self, parserObj):
> + ''' Add command line options to the argparser '''
> + parserObj.add_argument('-a', "--arch", dest="build_arch", type=str, default="X64",
> + help="Optional - architecture to build. IA32 will use IA32 for Pei & Dxe. "
> + "X64 will use X64 for both PEI and DXE.")
> +
> + def RetrieveCommandLineOptions(self, args):
> + ''' Retrieve command line options from the argparser '''
> +
> + shell_environment.GetBuildVars().SetValue(
> + "TARGET_ARCH", args.build_arch.upper(), "From CmdLine")
> + shell_environment.GetBuildVars().SetValue(
> + "ACTIVE_PLATFORM", "EmulatorPkg/EmulatorPkg.dsc", "From CmdLine")
> +
> + def GetWorkspaceRoot(self):
> + ''' get WorkspacePath '''
> + return CommonPlatform.WorkspaceRoot
> +
> + def GetPackagesPath(self):
> + ''' Return a list of workspace relative paths that should be mapped as edk2 PackagesPath '''
> + return ()
> +
> + def GetActiveScopes(self):
> + ''' return tuple containing scopes that should be active for this process '''
> + return CommonPlatform.Scopes
> +
> + def GetName(self):
> + ''' Get the name of the repo, platform, or product being build '''
> + ''' Used for naming the log file, among others '''
> +
> + # check the startup nsh flag and if set then rename the log file.
> + # this helps in CI so we don't overwrite the build log since running
> + # uses the stuart_build command.
> + if(shell_environment.GetBuildVars().GetValue("MAKE_STARTUP_NSH", "FALSE") == "TRUE"):
> + return "EmulatorPkg_With_Run"
> + return "EmulatorPkg"
> +
> + def GetLoggingLevel(self, loggerType):
> + ''' Get the logging level for a given type
> + base == lowest logging level supported
> + con == Screen logging
> + txt == plain text file logging
> + md == markdown file logging
> + '''
> + return logging.DEBUG
> +
> + def SetPlatformEnv(self):
> + logging.debug("PlatformBuilder SetPlatformEnv")
> + self.env.SetValue("PRODUCT_NAME", "EmulatorPkg", "Platform Hardcoded")
> + self.env.SetValue("TOOL_CHAIN_TAG", "VS2019", "Default Toolchain")
> +
> + # Add support for using the correct Platform Headers, tools, and Libs based on emulator architecture
> + # requested to be built when building VS2019 or VS2017
> + if self.env.GetValue("TOOL_CHAIN_TAG") == "VS2019" or self.env.GetValue("TOOL_CHAIN_TAG") == "VS2017":
> + key = self.env.GetValue("TOOL_CHAIN_TAG") + "_HOST"
> + if self.env.GetValue("TARGET_ARCH") == "IA32":
> + shell_environment.ShellEnvironment().set_shell_var(key, "x86")
> + elif self.env.GetValue("TARGET_ARCH") == "X64":
> + shell_environment.ShellEnvironment().set_shell_var(key, "x64")
> +
> + # Add support for using the correct Platform Headers, tools, and Libs based on emulator architecture
> + # requested to be built when building on linux.
> + if GetHostInfo().os.upper() == "LINUX":
> + self.ConfigureLinuxDLinkPath()
> +
> + if GetHostInfo().os.upper() == "WINDOWS":
> + self.env.SetValue("BLD_*_WIN_HOST_BUILD", "TRUE",
> + "Trigger Windows host build")
> +
> + self.env.SetValue("MAKE_STARTUP_NSH", "FALSE", "Default to false")
> +
> + # I don't see what this does but it is in build.sh
> + key = "BLD_*_BUILD_" + self.env.GetValue("TARGET_ARCH")
> + self.env.SetValue(key, "TRUE", "match script in build.sh")
> + return 0
> +
> + def PlatformPreBuild(self):
> + return 0
> +
> + def PlatformPostBuild(self):
> + return 0
> +
> + def FlashRomImage(self):
> + ''' Use the FlashRom Function to run the emulator. This gives an easy stuart command line to
> + activate the emulator. '''
> +
> + OutputPath = os.path.join(self.env.GetValue(
> + "BUILD_OUTPUT_BASE"), self.env.GetValue("TARGET_ARCH"))
> +
> + if (self.env.GetValue("MAKE_STARTUP_NSH") == "TRUE"):
> + f = open(os.path.join(OutputPath, "startup.nsh"), "w")
> + f.write("BOOT SUCCESS !!! \n")
> + # add commands here
> + f.write("reset\n")
> + f.close()
> +
> + if GetHostInfo().os.upper() == "WINDOWS":
> + cmd = "WinHost.exe"
> + elif GetHostInfo().os.upper() == "LINUX":
> + cmd = "./Host"
> + else:
> + logging.critical("Unsupported Host")
> + return -1
> + return RunCmd(cmd, "", workingdir=OutputPath)
> +
> + def ConfigureLinuxDLinkPath(self):
> + '''
> + logic copied from build.sh to setup the correct libraries
> + '''
> + if self.env.GetValue("TARGET_ARCH") == "IA32":
> + LIB_NAMES = ["ld-linux.so.2", "libdl.so.2 crt1.o", "crti.o crtn.o"]
> + LIB_SEARCH_PATHS = ["/usr/lib/i386-linux-gnu",
> + "/usr/lib32", "/lib32", "/usr/lib", "/lib"]
> + elif self.env.GetValue("TARGET_ARCH") == "X64":
> + LIB_NAMES = ["ld-linux-x86-64.so.2",
> + "libdl.so.2", "crt1.o", "crti.o", "crtn.o"]
> + LIB_SEARCH_PATHS = ["/usr/lib/x86_64-linux-gnu",
> + "/usr/lib64", "/lib64", "/usr/lib", "/lib"]
> +
> + HOST_DLINK_PATHS = ""
> + for lname in LIB_NAMES:
> + logging.debug(f"Looking for {lname}")
> + for dname in LIB_SEARCH_PATHS:
> + logging.debug(f"In {dname}")
> + if os.path.isfile(os.path.join(dname, lname)):
> + logging.debug(f"Found {lname} in {dname}")
> + HOST_DLINK_PATHS += os.path.join(
> + os.path.join(dname, lname)) + os.pathsep
> + break
> + HOST_DLINK_PATHS = HOST_DLINK_PATHS.rstrip(os.pathsep)
> + logging.critical(f"Setting HOST_DLINK_PATHS to {HOST_DLINK_PATHS}")
> + shell_environment.ShellEnvironment().set_shell_var(
> + "HOST_DLINK_PATHS", HOST_DLINK_PATHS)
> diff --git a/EmulatorPkg/PlatformCI/ReadMe.md b/EmulatorPkg/PlatformCI/ReadMe.md
> new file mode 100644
> index 000000000000..c51838ec1aa7
> --- /dev/null
> +++ b/EmulatorPkg/PlatformCI/ReadMe.md
> @@ -0,0 +1,128 @@
> +# EmulatorPkg - Platform CI
> +
> +This ReadMe.md describes the Azure DevOps based Platform CI for EmulatorPkg and how
> +to use the same Pytools based build infrastructure locally.
> +
> +## Supported Configuration Details
> +
> +This solution for building and running EmulatorPkg has only been validated with Windows 10
> +with VS2019 and Ubuntu 18.04 with GCC5 toolchain. Two different firmware builds are
> +supported and are described below.
> +
> +| Configuration name | Architectures | DSC File |Additional Flags |
> +| :---- | :----- | :---- | :---- |
> +| IA32 | IA32 | EmulatorPkg.dsc | None |
> +| X64 | X64 | EmulatorPkg.dsc | None |
> +
> +## EDK2 Developer environment
> +
> +- [Python 3.8.x - Download & Install](https://www.python.org/downloads/)
> +- [GIT - Download & Install](https://git-scm.com/download/)
> +- [Edk2 Source](https://github.com/tianocore/edk2)
> +- For building Basetools and other host applications
> +
> + ``` bash
> + sudo apt-get update
> + sudo apt-get install gcc g++ make uuid-dev
> + ```
> +
> +- For building ARCH IA32 on X64 Ubuntu 18.04 LTS these steps where needed.
> +
> + ``` bash
> + sudo dpkg --add-architecture i386
> + sudo apt-get update
> + sudo apt-get install libc6-dev:i386 libx11-dev:i386 libxext-dev:i386 lib32gcc-7-dev
> + ```
> +
> +Note: edksetup, Submodule initialization and manual installation of NASM, iASL, or
> +the required cross-compiler toolchains are **not** required, this is handled by the
> +Pytools build system.
> +
> +## Building with Pytools for EmulatorPkg
> +
> +1. [Optional] Create a Python Virtual Environment - generally once per workspace
> +
> + ``` bash
> + python -m venv <name of virtual environment>
> + ```
> +
> +2. [Optional] Activate Virtual Environment - each time new shell opened
> + - Linux
> +
> + ```bash
> + source <name of virtual environment>/bin/activate
> + ```
> +
> + - Windows
> +
> + ``` bash
> + <name of virtual environment>/Scripts/activate.bat
> + ```
> +
> +3. Install Pytools - generally once per virtual env or whenever pip-requirements.txt changes
> +
> + ``` bash
> + pip install --upgrade -r pip-requirements.txt
> + ```
> +
> +4. Initialize & Update Submodules - only when submodules updated
> +
> + ``` bash
> + stuart_setup -c EmulatorPkg/PlatformCI/PlatformBuild.py TOOL_CHAIN_TAG=<TOOL_CHAIN_TAG> -a <TARGET_ARCH>
> + ```
> +
> +5. Initialize & Update Dependencies - only as needed when ext_deps change
> +
> + ``` bash
> + stuart_update -c EmulatorPkg/PlatformCI/PlatformBuild.py TOOL_CHAIN_TAG=<TOOL_CHAIN_TAG> -a <TARGET_ARCH>
> + ```
> +
> +6. Compile the basetools if necessary - only when basetools C source files change
> +
> + ``` bash
> + python BaseTools/Edk2ToolsBuild.py -t <ToolChainTag>
> + ```
> +
> +7. Compile Firmware
> +
> + ``` bash
> + stuart_build -c EmulatorPkg/PlatformCI/PlatformBuild.py TOOL_CHAIN_TAG=<TOOL_CHAIN_TAG> -a <TARGET_ARCH>
> + ```
> +
> + - use `stuart_build -c EmulatorPkg/PlatformCI/PlatformBuild.py -h` option to see additional
> + options like `--clean`
> +
> +8. Running Emulator
> + - You can add `--FlashRom` to the end of your build command and the emulator will run after the
> + build is complete.
> + - or use the `--FlashOnly` feature to just run the emulator.
> +
> + ``` bash
> + stuart_build -c EmulatorPkg/PlatformCI/PlatformBuild.py TOOL_CHAIN_TAG=<TOOL_CHAIN_TAG> -a <TARGET_ARCH> --
> FlashOnly
> + ```
> +
> +### Notes
> +
> +1. Configuring *ACTIVE_PLATFORM* and *TARGET_ARCH* in Conf/target.txt is **not** required. This
> + environment is set by PlatformBuild.py based upon the `[-a <TARGET_ARCH>]` parameter.
> +
> +**NOTE:** Logging the execution output will be in the normal stuart log as well as to your console.
> +
> +### Custom Build Options
> +
> +**MAKE_STARTUP_NSH=TRUE** will output a *startup.nsh* file to the location mapped as fs0. This is
> +used in CI in combination with the `--FlashOnly` feature to run the Emulator to the UEFI shell and then execute
> +the contents of *startup.nsh*.
> +
> +### Passing Build Defines
> +
> +To pass build defines through _stuart_build_, prepend `BLD_*_`to the define name and pass it on the
> +command-line. _stuart_build_ currently requires values to be assigned, so add an`=1` suffix for bare defines.
> +For example, to enable the IP6 Network Stack, the stuart_build command-line would be:
> +
> +`stuart_build -c EmulatorPkg/PlatformCI/PlatformBuild.py BLD_*_NETWORK_IP6_ENABLE=1`
> +
> +## References
> +
> +- [Installing and using Pytools](https://github.com/tianocore/edk2-pytool-extensions/blob/master/docs/using.md#installing)
> +- More on [python virtual environments](https://docs.python.org/3/library/venv.html)
> --
> 2.16.3.windows.1
>
>
>
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH v3 2/7] ArmVirtPkg: Add Platform CI and configuration for Core CI
2020-04-28 10:23 ` Leif Lindholm
@ 2020-04-28 12:47 ` Laszlo Ersek
0 siblings, 0 replies; 13+ messages in thread
From: Laszlo Ersek @ 2020-04-28 12:47 UTC (permalink / raw)
To: Leif Lindholm, Michael Kubacki; +Cc: devel, Ard Biesheuvel
On 04/28/20 12:23, Leif Lindholm wrote:
> On Mon, Apr 27, 2020 at 14:21:07 -0700, Michael Kubacki wrote:
>> I believe I misunderstood the conclusion of which Signed-off-by to include
>> in the v1 series discussion - https://edk2.groups.io/g/devel/message/57129.
>>
>> These patches are taken from a branch on Sean's fork of edk2 as noted in the
>> cover letter - https://edk2.groups.io/g/devel/message/58114. Sean has given
>> a Signed-off-by on these patches in that branch.
>>
>> Is the guidance in this case to only include my Signed-off-by?
>
> Yes. You can only attest to your own legal understanding of a given
> situation, not to that of others - so "passing on" attestations given
> by someone else in a different context ends up being misleading.
>
> But this isn't the first time this has been misunderstood - perhaps we
> should add an explicit statement to ReadMe.rst (as it will be after
> this set) that "When importing patches from external sources, existing
> Signed-off-by tags must be stripped out.".
That would be nice. I know I need such a reminder.
Thanks
Laszlo
^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2020-04-28 12:48 UTC | newest]
Thread overview: 13+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <20200424213108.19888-1-michael.kubacki@outlook.com>
2020-04-24 21:31 ` [PATCH v3 1/7] .azurepipelines: Add Platform CI template Michael Kubacki
2020-04-24 21:31 ` [PATCH v3 2/7] ArmVirtPkg: Add Platform CI and configuration for Core CI Michael Kubacki
2020-04-27 12:22 ` Leif Lindholm
2020-04-27 21:21 ` Michael Kubacki
2020-04-28 10:23 ` Leif Lindholm
2020-04-28 12:47 ` Laszlo Ersek
2020-04-24 21:31 ` [PATCH v3 3/7] EmulatorPkg: " Michael Kubacki
2020-04-28 11:22 ` [edk2-devel] " Liming Gao
2020-04-24 21:31 ` [PATCH v3 4/7] OvmfPkg: " Michael Kubacki
2020-04-24 21:31 ` [PATCH v3 5/7] .pytool: Update CI Settings to support Emulator, ArmVirt, and Ovmf packages Michael Kubacki
2020-04-24 21:31 ` [PATCH v3 6/7] .azurepipelines: Update Core CI build matrix to include platforms Michael Kubacki
2020-04-24 21:31 ` [PATCH v3 7/7] ReadMe: Convert to rst and add Platform CI Status Michael Kubacki
2020-04-27 12:30 ` Leif Lindholm
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox