From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-wm1-f52.google.com (mail-wm1-f52.google.com [209.85.128.52]) by mx.groups.io with SMTP id smtpd.web12.4495.1587990176726071242 for ; Mon, 27 Apr 2020 05:22:57 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@nuviainc-com.20150623.gappssmtp.com header.s=20150623 header.b=lxt2/70i; spf=pass (domain: nuviainc.com, ip: 209.85.128.52, mailfrom: leif@nuviainc.com) Received: by mail-wm1-f52.google.com with SMTP id x4so19302720wmj.1 for ; Mon, 27 Apr 2020 05:22:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nuviainc-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=2lkkUmbTR+V59D6hhIJHg26BJLLR3YtDV99Mk5KUQb8=; b=lxt2/70iAKHqfWqdmOVZQUree6IrnRynOgo6FZ8C+YYCTCiMFa3D5RlzYqSUUogm2t AIuumE4nR10sIF4vkmm/XxYkHeRO6tEjf+sO6jPJ8fh5pPmvhTKh4UE1cDOo+1dauJuy Fm+tXMfagz8Ru+BskkcXRw2rdDLj0AcSqJ+2C7LNKy2VEA38pcAAtYCH6439I8iTi0v6 4qK2zZgedXE9UYQI6pb6k4mnyvDpVNH+ki34AWGZDAibgkjOdbK7tzT+jv0cqcx7wrOl ESen4Z/Cn8Dy1IXudes3KXqpzk6yeWMRkHQQey0ZtdMxS3vODih5LFt/0BLhy0V1nDC3 023Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=2lkkUmbTR+V59D6hhIJHg26BJLLR3YtDV99Mk5KUQb8=; b=EcnOt8P9H2ZMG+q8bOyFCl/Yy3Bi9whl8otmE0xR6/revhwSsoJbTx5F1Wu3o7TeLA OM+d7raD+EQhrK52A0f0mjV3mita2B/O1W4d3RxNqhtG4/TKElYasGN3n9+ETwNqhuz5 HfSorUDpeNYH8Ew/RitajZ3oF64xlB1mTyjZsnHxO16uByGn/xthZxAy8Wud9rL4gwAP x+5YHF+u8jAFKOXX8c8Ju93VVSr2s0pSP5iVK/oGs8Q1jqEDjOvNw6BdzVDesO3gAkZM cZbYd4L/K2x2WY+f8r7BjJ50EC/NOULLVz4ZuzQzvna4X6EgUEJ5j1Zc3vb6H05fYg+x MQuQ== X-Gm-Message-State: AGi0PubIir3sL5cGOptRo7p23Jl0M5I34CT2WHiEzSOy8+a8QWSzt+2d mzd7JZSYGz0gJ4z529ecz3kUAQ== X-Google-Smtp-Source: APiQypKZFdb8cxOsNHj4+Th8Dg7T6SFUjfimNWQEst0iz1xZ1Gx+hB5OA/OOvSU/bvONm5j7dJP21w== X-Received: by 2002:a7b:c1d4:: with SMTP id a20mr26836519wmj.111.1587990175113; Mon, 27 Apr 2020 05:22:55 -0700 (PDT) Return-Path: Received: from vanye ([2001:470:1f09:12f0:b26e:bfff:fea9:f1b8]) by smtp.gmail.com with ESMTPSA id 92sm1147728wrm.71.2020.04.27.05.22.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Apr 2020 05:22:54 -0700 (PDT) Date: Mon, 27 Apr 2020 13:22:52 +0100 From: "Leif Lindholm" To: michael.kubacki@outlook.com Cc: devel@edk2.groups.io, Laszlo Ersek , Ard Biesheuvel Subject: Re: [PATCH v3 2/7] ArmVirtPkg: Add Platform CI and configuration for Core CI Message-ID: <20200427122252.GQ14075@vanye> References: <20200424213108.19888-1-michael.kubacki@outlook.com> MIME-Version: 1.0 In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Fri, Apr 24, 2020 at 14:31:03 -0700, michael.kubacki@outlook.com wrote: > From: Sean Brogan > > REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2570 > > Add new Azure Pipeline definitions to build and run ArmVirtPkg with: > * Ubuntu GCC5 > Add PyTool based build of ArmVirtPkg > Add extdep for managing the iasl dependency > Add ArmVirtPkg.ci.yaml for Core CI > Add ReadMe.md for details and instructions > > Cc: Laszlo Ersek > Cc: Ard Biesheuvel > Cc: Leif Lindholm > Signed-off-by: Sean Brogan > Signed-off-by: Michael Kubacki The contributor attests the compliance of the submission with https://developercertificate.org/ at the point of contribution by using Signed-off-by. Statements by others (including original authors) should not be copied when copying code between projects. The From: tag preserves authorship credit. / Leif > Reviewed-by: Shenglei Zhang > Acked-by: Laszlo Ersek > Acked-by: Ard Biesheuvel > Reviewed-by: Bret Barkelew > --- > ArmVirtPkg/ArmVirtPkg.ci.yaml | 103 ++++++++ > ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml | 89 +++++++ > ArmVirtPkg/PlatformCI/PlatformBuild.py | 276 ++++++++++++++++++++ > ArmVirtPkg/PlatformCI/ReadMe.md | 125 +++++++++ > ArmVirtPkg/PlatformCI/iasl_ext_dep.yaml | 21 ++ > 5 files changed, 614 insertions(+) > > diff --git a/ArmVirtPkg/ArmVirtPkg.ci.yaml b/ArmVirtPkg/ArmVirtPkg.ci.yaml > new file mode 100644 > index 000000000000..4553725ee528 > --- /dev/null > +++ b/ArmVirtPkg/ArmVirtPkg.ci.yaml > @@ -0,0 +1,103 @@ > +## @file > +# Core CI configuration for ArmVirtPkg > +# > +# ArmVirtPkg is part of Platform Ci for builds so this is only > +# used for code analysis. > +# > +# Copyright (c) Microsoft Corporation > +# SPDX-License-Identifier: BSD-2-Clause-Patent > +## > +{ > + ## options defined .pytool/Plugin/CompilerPlugin > + "CompilerPlugin": { > + "DscPath": "" # Don't support this test > + }, > + > + ## options defined .pytool/Plugin/HostUnitTestCompilerPlugin > + "HostUnitTestCompilerPlugin": { > + "DscPath": "" # Don't support this test > + }, > + > + ## options defined .pytool/Plugin/CharEncodingCheck > + "CharEncodingCheck": { > + "IgnoreFiles": [] > + }, > + > + ## options defined .pytool/Plugin/DependencyCheck > + "DependencyCheck": { > + "AcceptableDependencies": [ > + "MdePkg/MdePkg.dec", > + "MdeModulePkg/MdeModulePkg.dec", > + "ArmVirtPkg/ArmVirtPkg.dec", > + "NetworkPkg/NetworkPkg.dec", > + "ArmPkg/ArmPkg.dec", > + "OvmfPkg/OvmfPkg.dec", > + "EmbeddedPkg/EmbeddedPkg.dec", > + "ArmPlatformPkg/ArmPlatformPkg.dec", > + "SecurityPkg/SecurityPkg.dec", > + "ShellPkg/ShellPkg.dec" #Is this ok? > + ], > + # For host based unit tests > + "AcceptableDependencies-HOST_APPLICATION":[ > + "UnitTestFrameworkPkg/UnitTestFrameworkPkg.dec" > + ], > + # For UEFI shell based apps > + "AcceptableDependencies-UEFI_APPLICATION":[ > + > + ], > + "IgnoreInf": [] > + }, > + > + ## options defined .pytool/Plugin/DscCompleteCheck > + "DscCompleteCheck": { > + "IgnoreInf": [""], > + "DscPath": "" # Don't support this test > + }, > + > + ## options defined .pytool/Plugin/HostUnitTestDscCompleteCheck > + "HostUnitTestDscCompleteCheck": { > + "IgnoreInf": [""], > + "DscPath": "" # Don't support this test > + }, > + > + ## options defined .pytool/Plugin/GuidCheck > + "GuidCheck": { > + "IgnoreGuidName": [], > + "IgnoreGuidValue": [], > + "IgnoreFoldersAndFiles": [], > + "IgnoreDuplicates": [], > + }, > + > + ## options defined .pytool/Plugin/LibraryClassCheck > + "LibraryClassCheck": { > + "IgnoreHeaderFile": [] > + }, > + > + ## options defined .pytool/Plugin/SpellCheck > + "SpellCheck": { > + "AuditOnly": False, # Fails right now with over 270 errors > + "IgnoreFiles": [], # use gitignore syntax to ignore errors in matching files > + "ExtendWords": [ > + "setjump", > + "plong", > + "lparam", > + "lpdword", > + "lpthread", > + "lresult", > + "bootable", > + "bsymbolic", > + "endiannness", > + "fvmain", > + "multiboot", > + "qemu's", > + "ramdisk", > + "ramfb", > + "unbootable", > + "virt's", > + "werror", > + "xenio" > + ], # words to extend to the dictionary for this package > + "IgnoreStandardPaths": [], # Standard Plugin defined paths that should be ignore > + "AdditionalIncludePaths": [] # Additional paths to spell check (wildcards supported) > + } > +} > diff --git a/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml b/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml > new file mode 100644 > index 000000000000..b9452ec5bfaf > --- /dev/null > +++ b/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml > @@ -0,0 +1,89 @@ > +## @file > +# Azure Pipeline build file for building a platform. > +# > +# Platform: ArmVirtQemu > +# OS: Ubuntu > +# Toolchain: GCC5 > +# > +# Copyright (c) Microsoft Corporation. > +# SPDX-License-Identifier: BSD-2-Clause-Patent > +## > +trigger: > + - master > +pr: > + - master > + > +jobs: > + - job: Platform_CI > + variables: > + package: 'ArmVirtPkg' > + vm_image: 'ubuntu-latest' > + should_run: true > + run_flags: "MAKE_STARTUP_NSH=TRUE QEMU_HEADLESS=TRUE" > + > + #Use matrix to speed up the build process > + strategy: > + matrix: > + QEMU_AARCH64_DEBUG: > + Build.File: "$(package)/PlatformCI/PlatformBuild.py" > + Build.Arch: "AARCH64" > + Build.Flags: "" > + Build.Target: "DEBUG" > + Run.Flags: $(run_flags) > + Run: $(should_run) > + QEMU_AARCH64_RELEASE: > + Build.File: "$(package)/PlatformCI/PlatformBuild.py" > + Build.Arch: "AARCH64" > + Build.Flags: "" > + Build.Target: "RELEASE" > + Run.Flags: $(run_flags) > + Run: $(should_run) > + QEMU_AARCH64_NOOPT: > + Build.File: "$(package)/PlatformCI/PlatformBuild.py" > + Build.Arch: "AARCH64" > + Build.Flags: "" > + Build.Target: "NOOPT" > + Run.Flags: $(run_flags) > + Run: $(should_run) > + QEMU_ARM_DEBUG: > + Build.File: "$(package)/PlatformCI/PlatformBuild.py" > + Build.Arch: "ARM" > + Build.Flags: "" > + Build.Target: "DEBUG" > + Run.Flags: $(run_flags) > + Run: $(should_run) > + QEMU_ARM_RELEASE: > + Build.File: "$(package)/PlatformCI/PlatformBuild.py" > + Build.Arch: "ARM" > + Build.Flags: "" > + Build.Target: "RELEASE" > + Run.Flags: $(run_flags) > + Run: $(should_run) > + QEMU_ARM_NOOPT: > + Build.File: "$(package)/PlatformCI/PlatformBuild.py" > + Build.Arch: "ARM" > + Build.Flags: "" > + Build.Target: "NOOPT" > + Run.Flags: $(run_flags) > + Run: $(should_run) > + > + workspace: > + clean: all > + > + pool: > + vmImage: $(vm_image) > + > + steps: > + - template: ../../../.azurepipelines/templates/platform-build-run-steps.yml > + parameters: > + tool_chain_tag: GCC5 > + build_pkg: $(package) > + build_target: $(Build.Target) > + build_arch: $(Build.Arch) > + build_file: $(Build.File) > + build_flags: $(Build.Flags) > + run_flags: $(Run.Flags) > + extra_install_step: > + - bash: sudo apt-get install qemu > + displayName: Install qemu > + condition: and(gt(variables.pkg_count, 0), succeeded()) > diff --git a/ArmVirtPkg/PlatformCI/PlatformBuild.py b/ArmVirtPkg/PlatformCI/PlatformBuild.py > new file mode 100644 > index 000000000000..dff653e919eb > --- /dev/null > +++ b/ArmVirtPkg/PlatformCI/PlatformBuild.py > @@ -0,0 +1,276 @@ > +# @file > +# Script to Build ArmVirtPkg UEFI firmware > +# > +# Copyright (c) Microsoft Corporation. > +# SPDX-License-Identifier: BSD-2-Clause-Patent > +## > +import os > +import logging > +import io > + > +from edk2toolext.environment import shell_environment > +from edk2toolext.environment.uefi_build import UefiBuilder > +from edk2toolext.invocables.edk2_platform_build import BuildSettingsManager > +from edk2toolext.invocables.edk2_setup import SetupSettingsManager, RequiredSubmodule > +from edk2toolext.invocables.edk2_update import UpdateSettingsManager > +from edk2toolext.invocables.edk2_pr_eval import PrEvalSettingsManager > +from edk2toollib.utility_functions import RunCmd > +from edk2toollib.utility_functions import GetHostInfo > + > +# ####################################################################################### # > +# Common Configuration # > +# ####################################################################################### # > + > + > +class CommonPlatform(): > + ''' Common settings for this platform. Define static data here and use > + for the different parts of stuart > + ''' > + PackagesSupported = ("ArmVirtPkg",) > + ArchSupported = ("AARCH64", "ARM") > + TargetsSupported = ("DEBUG", "RELEASE", "NOOPT") > + Scopes = ('armvirt', 'edk2-build') > + WorkspaceRoot = os.path.realpath(os.path.join( > + os.path.dirname(os.path.abspath(__file__)), "..", "..")) > + > + # ####################################################################################### # > + # Configuration for Update & Setup # > + # ####################################################################################### # > + > + > +class SettingsManager(UpdateSettingsManager, SetupSettingsManager, PrEvalSettingsManager): > + > + def GetPackagesSupported(self): > + ''' return iterable of edk2 packages supported by this build. > + These should be edk2 workspace relative paths ''' > + return CommonPlatform.PackagesSupported > + > + def GetArchitecturesSupported(self): > + ''' return iterable of edk2 architectures supported by this build ''' > + return CommonPlatform.ArchSupported > + > + def GetTargetsSupported(self): > + ''' return iterable of edk2 target tags supported by this build ''' > + return CommonPlatform.TargetsSupported > + > + def GetRequiredSubmodules(self): > + ''' return iterable containing RequiredSubmodule objects. > + If no RequiredSubmodules return an empty iterable > + ''' > + rs = [] > + > + # intentionally declare this one with recursive false to avoid overhead > + rs.append(RequiredSubmodule( > + "CryptoPkg/Library/OpensslLib/openssl", False)) > + > + # To avoid maintenance of this file for every new submodule > + # lets just parse the .gitmodules and add each if not already in list. > + # The GetRequiredSubmodules is designed to allow a build to optimize > + # the desired submodules but it isn't necessary for this repository. > + result = io.StringIO() > + ret = RunCmd("git", "config --file .gitmodules --get-regexp path", workingdir=self.GetWorkspaceRoot(), outstream=result) > + # Cmd output is expected to look like: > + # submodule.CryptoPkg/Library/OpensslLib/openssl.path CryptoPkg/Library/OpensslLib/openssl > + # submodule.SoftFloat.path ArmPkg/Library/ArmSoftFloatLib/berkeley-softfloat-3 > + if ret == 0: > + for line in result.getvalue().splitlines(): > + _, _, path = line.partition(" ") > + if path is not None: > + if path not in [x.path for x in rs]: > + rs.append(RequiredSubmodule(path, True)) # add it with recursive since we don't know > + return rs > + > + def SetArchitectures(self, list_of_requested_architectures): > + ''' Confirm the requests architecture list is valid and configure SettingsManager > + to run only the requested architectures. > + > + Raise Exception if a list_of_requested_architectures is not supported > + ''' > + unsupported = set(list_of_requested_architectures) - \ > + set(self.GetArchitecturesSupported()) > + if(len(unsupported) > 0): > + errorString = ( > + "Unsupported Architecture Requested: " + " ".join(unsupported)) > + logging.critical(errorString) > + raise Exception(errorString) > + self.ActualArchitectures = list_of_requested_architectures > + > + def GetWorkspaceRoot(self): > + ''' get WorkspacePath ''' > + return CommonPlatform.WorkspaceRoot > + > + def GetActiveScopes(self): > + ''' return tuple containing scopes that should be active for this process ''' > + > + scopes = CommonPlatform.Scopes > + ActualToolChainTag = shell_environment.GetBuildVars().GetValue("TOOL_CHAIN_TAG", "") > + > + if GetHostInfo().os.upper() == "LINUX" and ActualToolChainTag.upper().startswith("GCC"): > + if "AARCH64" in self.ActualArchitectures: > + scopes += ("gcc_aarch64_linux",) > + if "ARM" in self.ActualArchitectures: > + scopes += ("gcc_arm_linux",) > + return scopes > + > + def FilterPackagesToTest(self, changedFilesList: list, potentialPackagesList: list) -> list: > + ''' Filter other cases that this package should be built > + based on changed files. This should cover things that can't > + be detected as dependencies. ''' > + build_these_packages = [] > + possible_packages = potentialPackagesList.copy() > + for f in changedFilesList: > + # BaseTools files that might change the build > + if "BaseTools" in f: > + if os.path.splitext(f) not in [".txt", ".md"]: > + build_these_packages = possible_packages > + break > + > + # if the azure pipeline platform template file changed > + if "platform-build-run-steps.yml" in f: > + build_these_packages = possible_packages > + break > + > + > + return build_these_packages > + > + def GetPlatformDscAndConfig(self) -> tuple: > + ''' If a platform desires to provide its DSC then Policy 4 will evaluate if > + any of the changes will be built in the dsc. > + > + The tuple should be (, ) > + ''' > + return (os.path.join("ArmVirtPkg", "ArmVirtQemu.dsc"), {}) > + > + > + # ####################################################################################### # > + # Actual Configuration for Platform Build # > + # ####################################################################################### # > + > + > +class PlatformBuilder(UefiBuilder, BuildSettingsManager): > + def __init__(self): > + UefiBuilder.__init__(self) > + > + def AddCommandLineOptions(self, parserObj): > + ''' Add command line options to the argparser ''' > + parserObj.add_argument('-a', "--arch", dest="build_arch", type=str, default="AARCH64", > + help="Optional - Architecture to build. Default = AARCH64") > + > + def RetrieveCommandLineOptions(self, args): > + ''' Retrieve command line options from the argparser ''' > + > + shell_environment.GetBuildVars().SetValue( > + "TARGET_ARCH", args.build_arch.upper(), "From CmdLine") > + > + shell_environment.GetBuildVars().SetValue( > + "ACTIVE_PLATFORM", "ArmVirtPkg/ArmVirtQemu.dsc", "From CmdLine") > + > + def GetWorkspaceRoot(self): > + ''' get WorkspacePath ''' > + return CommonPlatform.WorkspaceRoot > + > + def GetPackagesPath(self): > + ''' Return a list of workspace relative paths that should be mapped as edk2 PackagesPath ''' > + return () > + > + def GetActiveScopes(self): > + ''' return tuple containing scopes that should be active for this process ''' > + scopes = CommonPlatform.Scopes > + ActualToolChainTag = shell_environment.GetBuildVars().GetValue("TOOL_CHAIN_TAG", "") > + Arch = shell_environment.GetBuildVars().GetValue("TARGET_ARCH", "") > + > + if GetHostInfo().os.upper() == "LINUX" and ActualToolChainTag.upper().startswith("GCC"): > + if "AARCH64" == Arch: > + scopes += ("gcc_aarch64_linux",) > + elif "ARM" == Arch: > + scopes += ("gcc_arm_linux",) > + return scopes > + > + def GetName(self): > + ''' Get the name of the repo, platform, or product being build ''' > + ''' Used for naming the log file, among others ''' > + # check the startup nsh flag and if set then rename the log file. > + # this helps in CI so we don't overwrite the build log since running > + # uses the stuart_build command. > + if(shell_environment.GetBuildVars().GetValue("MAKE_STARTUP_NSH", "FALSE") == "TRUE"): > + return "ArmVirtPkg_With_Run" > + return "ArmVirtPkg" > + > + def GetLoggingLevel(self, loggerType): > + ''' Get the logging level for a given type > + base == lowest logging level supported > + con == Screen logging > + txt == plain text file logging > + md == markdown file logging > + ''' > + return logging.DEBUG > + > + def SetPlatformEnv(self): > + logging.debug("PlatformBuilder SetPlatformEnv") > + self.env.SetValue("PRODUCT_NAME", "ArmVirtQemu", "Platform Hardcoded") > + self.env.SetValue("MAKE_STARTUP_NSH", "FALSE", "Default to false") > + self.env.SetValue("QEMU_HEADLESS", "FALSE", "Default to false") > + return 0 > + > + def PlatformPreBuild(self): > + return 0 > + > + def PlatformPostBuild(self): > + return 0 > + > + def FlashRomImage(self): > + VirtualDrive = os.path.join(self.env.GetValue( > + "BUILD_OUTPUT_BASE"), "VirtualDrive") > + os.makedirs(VirtualDrive, exist_ok=True) > + OutputPath_FV = os.path.join( > + self.env.GetValue("BUILD_OUTPUT_BASE"), "FV") > + Built_FV = os.path.join(OutputPath_FV, "QEMU_EFI.fd") > + > + # pad fd to 64mb > + with open(Built_FV, "ab") as fvfile: > + fvfile.seek(0, os.SEEK_END) > + additional = b'\0' * ((64 * 1024 * 1024)-fvfile.tell()) > + fvfile.write(additional) > + > + # QEMU must be on that path > + > + # Unique Command and Args parameters per ARCH > + if (self.env.GetValue("TARGET_ARCH").upper() == "AARCH64"): > + cmd = "qemu-system-aarch64" > + args = "-M virt" > + args += " -cpu cortex-a57" # emulate cpu > + elif(self.env.GetValue("TARGET_ARCH").upper() == "ARM"): > + cmd = "qemu-system-arm" > + args = "-M virt" > + args += " -cpu cortex-a15" # emulate cpu > + else: > + raise NotImplementedError() > + > + # Common Args > + args += " -pflash " + Built_FV # path to fw > + args += " -m 1024" # 1gb memory > + # turn off network > + args += " -net none" > + # Serial messages out > + args += " -serial stdio" > + # Mount disk with startup.nsh > + args += f" -drive file=fat:rw:{VirtualDrive},format=raw,media=disk" > + > + # Conditional Args > + if (self.env.GetValue("QEMU_HEADLESS").upper() == "TRUE"): > + args += " -display none" # no graphics > + > + if (self.env.GetValue("MAKE_STARTUP_NSH").upper() == "TRUE"): > + f = open(os.path.join(VirtualDrive, "startup.nsh"), "w") > + f.write("BOOT SUCCESS !!! \n") > + # add commands here > + f.write("reset -s\n") > + f.close() > + > + ret = RunCmd(cmd, args) > + > + if ret == 0xc0000005: > + # for some reason getting a c0000005 on successful return > + return 0 > + > + return ret > diff --git a/ArmVirtPkg/PlatformCI/ReadMe.md b/ArmVirtPkg/PlatformCI/ReadMe.md > new file mode 100644 > index 000000000000..7c11d925f59e > --- /dev/null > +++ b/ArmVirtPkg/PlatformCI/ReadMe.md > @@ -0,0 +1,125 @@ > +# ArmVirtPkg - Platform CI > + > +This Readme.md describes the Azure DevOps based Platform CI for ArmVirtPkg and how > +to use the same Pytools based build infrastructure locally. > + > +## Supported Configuration Details > + > +This solution for building and running ArmVirtPkg has only been validated with Ubuntu > +18.04 and the GCC5 toolchain. Two different firmware builds are supported and are > +described below. > + > +| Configuration name | Architecture | DSC File |Additional Flags | > +| :---------- | :----- | :----- | :---- | > +| AARCH64 | AARCH64 | ArmVirtQemu.dsc | None | > +| ARM | ARM | ArmVirtQemu.dsc | None | > + > +## EDK2 Developer environment > + > +- [Python 3.8.x - Download & Install](https://www.python.org/downloads/) > +- [GIT - Download & Install](https://git-scm.com/download/) > +- [QEMU - Download, Install, and add to your path](https://www.qemu.org/download/) > +- [Edk2 Source](https://github.com/tianocore/edk2) > +- Additional packages found necessary for Ubuntu 18.04 > + - apt-get install gcc g++ make uuid-dev > + > +Note: edksetup, Submodule initialization and manual installation of NASM, iASL, or > +the required cross-compiler toolchains are **not** required, this is handled by the > +Pytools build system. > + > +## Building with Pytools for ArmVirtPkg > + > +1. [Optional] Create a Python Virtual Environment - generally once per workspace > + > + ``` bash > + python -m venv > + ``` > + > +2. [Optional] Activate Virtual Environment - each time new shell opened > + - Windows > + > + ``` bash > + /Scripts/activate.bat > + ``` > + > + - Linux > + > + ```bash > + source /bin/activate > + ``` > + > +3. Install Pytools - generally once per virtual env or whenever pip-requirements.txt changes > + > + ``` bash > + pip install --upgrade -r pip-requirements.txt > + ``` > + > +4. Initialize & Update Submodules - only when submodules updated > + > + ``` bash > + stuart_setup -c ArmVirtPkg/PlatformCI/PlatformBuild.py TOOL_CHAIN_TAG= -a > + ``` > + > +5. Initialize & Update Dependencies - only as needed when ext_deps change > + > + ``` bash > + stuart_update -c ArmVirtPkg/PlatformCI/PlatformBuild.py TOOL_CHAIN_TAG= -a > + ``` > + > +6. Compile the basetools if necessary - only when basetools C source files change > + > + ``` bash > + python BaseTools/Edk2ToolsBuild.py -t > + ``` > + > +7. Compile Firmware > + > + ``` bash > + stuart_build -c ArmVirtPkg/PlatformCI/PlatformBuild.py TOOL_CHAIN_TAG= -a > + ``` > + > + - use `stuart_build -c ArmVirtPkg/PlatformCI/PlatformBuild.py -h` option to see additional > + options like `--clean` > + > +8. Running Emulator > + - You can add `--FlashRom` to the end of your build command and the emulator will run after the > + build is complete. > + - or use the `--FlashOnly` feature to just run the emulator. > + > + ``` bash > + stuart_build -c ArmVirtPkg/PlatformCI/PlatformBuild.py TOOL_CHAIN_TAG= -a --FlashOnly > + ``` > + > +### Notes > + > +1. Including the expected build architecture and toolchain to the _stuart_update_ command is critical. > + This is because there are extra scopes and tools that will be resolved during the update step that > + need to match your build step. > +2. Configuring *ACTIVE_PLATFORM* and *TARGET_ARCH* in Conf/target.txt is **not** required. This > + environment is set by PlatformBuild.py based upon the `[-a ]` parameter. > +3. QEMU must be on your path. On Windows this is a manual process and not part of the QEMU installer. > + > +**NOTE:** Logging the execution output will be in the normal stuart log as well as to your console. > + > +### Custom Build Options > + > +**MAKE_STARTUP_NSH=TRUE** will output a *startup.nsh* file to the location mapped as fs0. This is > +used in CI in combination with the `--FlashOnly` feature to run QEMU to the UEFI shell and then execute > +the contents of *startup.nsh*. > + > +**QEMU_HEADLESS=TRUE** Since CI servers run headless QEMU must be told to run with no display otherwise > +an error occurs. Locally you don't need to set this. > + > +### Passing Build Defines > + > +To pass build defines through _stuart_build_, prepend `BLD_*_`to the define name and pass it on the > +command-line. _stuart_build_ currently requires values to be assigned, so add an`=1` suffix for bare defines. > +For example, to enable the TPM2 support, instead of the traditional "-D TPM2_ENABLE=TRUE", the stuart_build > +command-line would be: > + > +`stuart_build -c ArmVirtPkg/PlatformCI/PlatformBuild.py BLD_*_TPM2_ENABLE=TRUE` > + > +## References > + > +- [Installing and using Pytools](https://github.com/tianocore/edk2-pytool-extensions/blob/master/docs/using.md#installing) > +- More on [python virtual environments](https://docs.python.org/3/library/venv.html) > diff --git a/ArmVirtPkg/PlatformCI/iasl_ext_dep.yaml b/ArmVirtPkg/PlatformCI/iasl_ext_dep.yaml > new file mode 100644 > index 000000000000..8869ed3ecef1 > --- /dev/null > +++ b/ArmVirtPkg/PlatformCI/iasl_ext_dep.yaml > @@ -0,0 +1,21 @@ > +## @file > +# Download iasl executable tool from a nuget.org package > +# - package contains different binaries based on host > +# Add the folder with the tool to the path > +# > +# This is only downloaded for scope armvirt thus > +# should have no impact on the asl compiler used by any > +# other platform build > +# > +# Copyright (c) Microsoft Corporation. > +# SPDX-License-Identifier: BSD-2-Clause-Patent > +## > +{ > + "id": "iasl-armvirt-1", > + "scope": "armvirt", > + "type": "nuget", > + "name": "iasl", > + "source": "https://api.nuget.org/v3/index.json", > + "version": "20190215.0.0", > + "flags": ["set_path", "host_specific"], > +} > -- > 2.16.3.windows.1 >