From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-wr1-f53.google.com (mail-wr1-f53.google.com [209.85.221.53]) by mx.groups.io with SMTP id smtpd.web10.4676.1588069413699112688 for ; Tue, 28 Apr 2020 03:23:34 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@nuviainc-com.20150623.gappssmtp.com header.s=20150623 header.b=K46yqWMx; spf=pass (domain: nuviainc.com, ip: 209.85.221.53, mailfrom: leif@nuviainc.com) Received: by mail-wr1-f53.google.com with SMTP id x18so23972873wrq.2 for ; Tue, 28 Apr 2020 03:23:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nuviainc-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=Wm+PL64ZPWlwSQl/mJTNS/7eJ5hmX/gFKmqcdsALdJE=; b=K46yqWMxb0HkManBqKBpSV3hrqAcKlNVCUjnV++ONv2jB78F5O3OGfh0cExlE0QXWA QSpT1iuJOsPNzZcI5UbuqKPcqo5wwIP/TaNvSwBzk1a0cIKIM9TEj8ioBHtnSCZuyNFt NYRfR8dm6lBYsw8p3cIwnZAl9bvRPOvxiZ9zvjjVbekkaTTWQdPrEQg5dEt230HfJfuc L3WStIodH+mFY3QtjYFG5oNuzGpMEg/kSoOk/SwJFota9FUvhX3Np/Ha6TO3Vnqy/Qpn /80AL5Pmg/u/fi5zbYvY7YUrQkNMWbexgdkdhKFdhlLCzfrSo4buApnxRMwMEaGzQLbw cG8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=Wm+PL64ZPWlwSQl/mJTNS/7eJ5hmX/gFKmqcdsALdJE=; b=AEr58/d67AI/iPU45FnUVKSGaUTTgXh3brCUChgl22tRaam6d06fQ3hJdLq+93PtqU qPmzDQ/4qclH3H9gzrBg3OEWf9SWQ4RIdbOLn0fbeXgZZbEWuPRRdy5cjfMTKOBl8XKA wE4+sBiifSCw1QWBLrouDpkgabPDAgxMX4vylE7qKwuhZYIsAg4hznMLhy7VVIpUe9S0 Sxsiuzkgnif+7RANFv8NuimWRPUlCk7wE0zEve+lKddgPSyhtc4wogYzyn9pr0Mifqyx B7fZNAv2WWTiODnheXTQtPRUrsKzlBhAQKsJjD8S3RkX9P4+ny54Hh2tN6/UyMvjuS1U mKOA== X-Gm-Message-State: AGi0PuajwWIgsrlwyyu8hsPKpqJIO68LaXNnSF/aHopBI4z5x2Ramocb dKrI0IG4NJ8O+vrUwz/xLZMiiA== X-Google-Smtp-Source: APiQypJl7RB7JKEtUQpfw/Tl2MfatKdFrtICLpzfOCQZ1iX2CTvKQo7dRi4iczZUjItK0RpIDBTrvQ== X-Received: by 2002:a05:6000:1287:: with SMTP id f7mr33420437wrx.345.1588069412101; Tue, 28 Apr 2020 03:23:32 -0700 (PDT) Return-Path: Received: from vanye ([2001:470:1f09:12f0:b26e:bfff:fea9:f1b8]) by smtp.gmail.com with ESMTPSA id o18sm24417342wrp.23.2020.04.28.03.23.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Apr 2020 03:23:31 -0700 (PDT) Date: Tue, 28 Apr 2020 11:23:29 +0100 From: "Leif Lindholm" To: Michael Kubacki Cc: devel@edk2.groups.io, Laszlo Ersek , Ard Biesheuvel Subject: Re: [PATCH v3 2/7] ArmVirtPkg: Add Platform CI and configuration for Core CI Message-ID: <20200428102329.GC21486@vanye> References: <20200424213108.19888-1-michael.kubacki@outlook.com> <20200427122252.GQ14075@vanye> MIME-Version: 1.0 In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Mon, Apr 27, 2020 at 14:21:07 -0700, Michael Kubacki wrote: > I believe I misunderstood the conclusion of which Signed-off-by to include > in the v1 series discussion - https://edk2.groups.io/g/devel/message/57129. > > These patches are taken from a branch on Sean's fork of edk2 as noted in the > cover letter - https://edk2.groups.io/g/devel/message/58114. Sean has given > a Signed-off-by on these patches in that branch. > > Is the guidance in this case to only include my Signed-off-by? Yes. You can only attest to your own legal understanding of a given situation, not to that of others - so "passing on" attestations given by someone else in a different context ends up being misleading. But this isn't the first time this has been misunderstood - perhaps we should add an explicit statement to ReadMe.rst (as it will be after this set) that "When importing patches from external sources, existing Signed-off-by tags must be stripped out.". Regards, Leif > Thanks, > Michael > > On 4/27/2020 5:22 AM, Leif Lindholm wrote: > > On Fri, Apr 24, 2020 at 14:31:03 -0700, michael.kubacki@outlook.com wrote: > > > From: Sean Brogan > > > > > > REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2570 > > > > > > Add new Azure Pipeline definitions to build and run ArmVirtPkg with: > > > * Ubuntu GCC5 > > > Add PyTool based build of ArmVirtPkg > > > Add extdep for managing the iasl dependency > > > Add ArmVirtPkg.ci.yaml for Core CI > > > Add ReadMe.md for details and instructions > > > > > > Cc: Laszlo Ersek > > > Cc: Ard Biesheuvel > > > Cc: Leif Lindholm > > > Signed-off-by: Sean Brogan > > > Signed-off-by: Michael Kubacki > > > > The contributor attests the compliance of the submission with > > https://developercertificate.org/ at the point of contribution by > > using Signed-off-by. > > > > Statements by others (including original authors) should not be copied > > when copying code between projects. The From: tag preserves authorship > > credit. > > > > / > > Leif > > > > > Reviewed-by: Shenglei Zhang > > > Acked-by: Laszlo Ersek > > > Acked-by: Ard Biesheuvel > > > Reviewed-by: Bret Barkelew > > > --- > > > ArmVirtPkg/ArmVirtPkg.ci.yaml | 103 ++++++++ > > > ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml | 89 +++++++ > > > ArmVirtPkg/PlatformCI/PlatformBuild.py | 276 ++++++++++++++++++++ > > > ArmVirtPkg/PlatformCI/ReadMe.md | 125 +++++++++ > > > ArmVirtPkg/PlatformCI/iasl_ext_dep.yaml | 21 ++ > > > 5 files changed, 614 insertions(+) > > > > > > diff --git a/ArmVirtPkg/ArmVirtPkg.ci.yaml b/ArmVirtPkg/ArmVirtPkg.ci.yaml > > > new file mode 100644 > > > index 000000000000..4553725ee528 > > > --- /dev/null > > > +++ b/ArmVirtPkg/ArmVirtPkg.ci.yaml > > > @@ -0,0 +1,103 @@ > > > +## @file > > > +# Core CI configuration for ArmVirtPkg > > > +# > > > +# ArmVirtPkg is part of Platform Ci for builds so this is only > > > +# used for code analysis. > > > +# > > > +# Copyright (c) Microsoft Corporation > > > +# SPDX-License-Identifier: BSD-2-Clause-Patent > > > +## > > > +{ > > > + ## options defined .pytool/Plugin/CompilerPlugin > > > + "CompilerPlugin": { > > > + "DscPath": "" # Don't support this test > > > + }, > > > + > > > + ## options defined .pytool/Plugin/HostUnitTestCompilerPlugin > > > + "HostUnitTestCompilerPlugin": { > > > + "DscPath": "" # Don't support this test > > > + }, > > > + > > > + ## options defined .pytool/Plugin/CharEncodingCheck > > > + "CharEncodingCheck": { > > > + "IgnoreFiles": [] > > > + }, > > > + > > > + ## options defined .pytool/Plugin/DependencyCheck > > > + "DependencyCheck": { > > > + "AcceptableDependencies": [ > > > + "MdePkg/MdePkg.dec", > > > + "MdeModulePkg/MdeModulePkg.dec", > > > + "ArmVirtPkg/ArmVirtPkg.dec", > > > + "NetworkPkg/NetworkPkg.dec", > > > + "ArmPkg/ArmPkg.dec", > > > + "OvmfPkg/OvmfPkg.dec", > > > + "EmbeddedPkg/EmbeddedPkg.dec", > > > + "ArmPlatformPkg/ArmPlatformPkg.dec", > > > + "SecurityPkg/SecurityPkg.dec", > > > + "ShellPkg/ShellPkg.dec" #Is this ok? > > > + ], > > > + # For host based unit tests > > > + "AcceptableDependencies-HOST_APPLICATION":[ > > > + "UnitTestFrameworkPkg/UnitTestFrameworkPkg.dec" > > > + ], > > > + # For UEFI shell based apps > > > + "AcceptableDependencies-UEFI_APPLICATION":[ > > > + > > > + ], > > > + "IgnoreInf": [] > > > + }, > > > + > > > + ## options defined .pytool/Plugin/DscCompleteCheck > > > + "DscCompleteCheck": { > > > + "IgnoreInf": [""], > > > + "DscPath": "" # Don't support this test > > > + }, > > > + > > > + ## options defined .pytool/Plugin/HostUnitTestDscCompleteCheck > > > + "HostUnitTestDscCompleteCheck": { > > > + "IgnoreInf": [""], > > > + "DscPath": "" # Don't support this test > > > + }, > > > + > > > + ## options defined .pytool/Plugin/GuidCheck > > > + "GuidCheck": { > > > + "IgnoreGuidName": [], > > > + "IgnoreGuidValue": [], > > > + "IgnoreFoldersAndFiles": [], > > > + "IgnoreDuplicates": [], > > > + }, > > > + > > > + ## options defined .pytool/Plugin/LibraryClassCheck > > > + "LibraryClassCheck": { > > > + "IgnoreHeaderFile": [] > > > + }, > > > + > > > + ## options defined .pytool/Plugin/SpellCheck > > > + "SpellCheck": { > > > + "AuditOnly": False, # Fails right now with over 270 errors > > > + "IgnoreFiles": [], # use gitignore syntax to ignore errors in matching files > > > + "ExtendWords": [ > > > + "setjump", > > > + "plong", > > > + "lparam", > > > + "lpdword", > > > + "lpthread", > > > + "lresult", > > > + "bootable", > > > + "bsymbolic", > > > + "endiannness", > > > + "fvmain", > > > + "multiboot", > > > + "qemu's", > > > + "ramdisk", > > > + "ramfb", > > > + "unbootable", > > > + "virt's", > > > + "werror", > > > + "xenio" > > > + ], # words to extend to the dictionary for this package > > > + "IgnoreStandardPaths": [], # Standard Plugin defined paths that should be ignore > > > + "AdditionalIncludePaths": [] # Additional paths to spell check (wildcards supported) > > > + } > > > +} > > > diff --git a/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml b/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml > > > new file mode 100644 > > > index 000000000000..b9452ec5bfaf > > > --- /dev/null > > > +++ b/ArmVirtPkg/PlatformCI/.azurepipelines/Ubuntu-GCC5.yml > > > @@ -0,0 +1,89 @@ > > > +## @file > > > +# Azure Pipeline build file for building a platform. > > > +# > > > +# Platform: ArmVirtQemu > > > +# OS: Ubuntu > > > +# Toolchain: GCC5 > > > +# > > > +# Copyright (c) Microsoft Corporation. > > > +# SPDX-License-Identifier: BSD-2-Clause-Patent > > > +## > > > +trigger: > > > + - master > > > +pr: > > > + - master > > > + > > > +jobs: > > > + - job: Platform_CI > > > + variables: > > > + package: 'ArmVirtPkg' > > > + vm_image: 'ubuntu-latest' > > > + should_run: true > > > + run_flags: "MAKE_STARTUP_NSH=TRUE QEMU_HEADLESS=TRUE" > > > + > > > + #Use matrix to speed up the build process > > > + strategy: > > > + matrix: > > > + QEMU_AARCH64_DEBUG: > > > + Build.File: "$(package)/PlatformCI/PlatformBuild.py" > > > + Build.Arch: "AARCH64" > > > + Build.Flags: "" > > > + Build.Target: "DEBUG" > > > + Run.Flags: $(run_flags) > > > + Run: $(should_run) > > > + QEMU_AARCH64_RELEASE: > > > + Build.File: "$(package)/PlatformCI/PlatformBuild.py" > > > + Build.Arch: "AARCH64" > > > + Build.Flags: "" > > > + Build.Target: "RELEASE" > > > + Run.Flags: $(run_flags) > > > + Run: $(should_run) > > > + QEMU_AARCH64_NOOPT: > > > + Build.File: "$(package)/PlatformCI/PlatformBuild.py" > > > + Build.Arch: "AARCH64" > > > + Build.Flags: "" > > > + Build.Target: "NOOPT" > > > + Run.Flags: $(run_flags) > > > + Run: $(should_run) > > > + QEMU_ARM_DEBUG: > > > + Build.File: "$(package)/PlatformCI/PlatformBuild.py" > > > + Build.Arch: "ARM" > > > + Build.Flags: "" > > > + Build.Target: "DEBUG" > > > + Run.Flags: $(run_flags) > > > + Run: $(should_run) > > > + QEMU_ARM_RELEASE: > > > + Build.File: "$(package)/PlatformCI/PlatformBuild.py" > > > + Build.Arch: "ARM" > > > + Build.Flags: "" > > > + Build.Target: "RELEASE" > > > + Run.Flags: $(run_flags) > > > + Run: $(should_run) > > > + QEMU_ARM_NOOPT: > > > + Build.File: "$(package)/PlatformCI/PlatformBuild.py" > > > + Build.Arch: "ARM" > > > + Build.Flags: "" > > > + Build.Target: "NOOPT" > > > + Run.Flags: $(run_flags) > > > + Run: $(should_run) > > > + > > > + workspace: > > > + clean: all > > > + > > > + pool: > > > + vmImage: $(vm_image) > > > + > > > + steps: > > > + - template: ../../../.azurepipelines/templates/platform-build-run-steps.yml > > > + parameters: > > > + tool_chain_tag: GCC5 > > > + build_pkg: $(package) > > > + build_target: $(Build.Target) > > > + build_arch: $(Build.Arch) > > > + build_file: $(Build.File) > > > + build_flags: $(Build.Flags) > > > + run_flags: $(Run.Flags) > > > + extra_install_step: > > > + - bash: sudo apt-get install qemu > > > + displayName: Install qemu > > > + condition: and(gt(variables.pkg_count, 0), succeeded()) > > > diff --git a/ArmVirtPkg/PlatformCI/PlatformBuild.py b/ArmVirtPkg/PlatformCI/PlatformBuild.py > > > new file mode 100644 > > > index 000000000000..dff653e919eb > > > --- /dev/null > > > +++ b/ArmVirtPkg/PlatformCI/PlatformBuild.py > > > @@ -0,0 +1,276 @@ > > > +# @file > > > +# Script to Build ArmVirtPkg UEFI firmware > > > +# > > > +# Copyright (c) Microsoft Corporation. > > > +# SPDX-License-Identifier: BSD-2-Clause-Patent > > > +## > > > +import os > > > +import logging > > > +import io > > > + > > > +from edk2toolext.environment import shell_environment > > > +from edk2toolext.environment.uefi_build import UefiBuilder > > > +from edk2toolext.invocables.edk2_platform_build import BuildSettingsManager > > > +from edk2toolext.invocables.edk2_setup import SetupSettingsManager, RequiredSubmodule > > > +from edk2toolext.invocables.edk2_update import UpdateSettingsManager > > > +from edk2toolext.invocables.edk2_pr_eval import PrEvalSettingsManager > > > +from edk2toollib.utility_functions import RunCmd > > > +from edk2toollib.utility_functions import GetHostInfo > > > + > > > +# ####################################################################################### # > > > +# Common Configuration # > > > +# ####################################################################################### # > > > + > > > + > > > +class CommonPlatform(): > > > + ''' Common settings for this platform. Define static data here and use > > > + for the different parts of stuart > > > + ''' > > > + PackagesSupported = ("ArmVirtPkg",) > > > + ArchSupported = ("AARCH64", "ARM") > > > + TargetsSupported = ("DEBUG", "RELEASE", "NOOPT") > > > + Scopes = ('armvirt', 'edk2-build') > > > + WorkspaceRoot = os.path.realpath(os.path.join( > > > + os.path.dirname(os.path.abspath(__file__)), "..", "..")) > > > + > > > + # ####################################################################################### # > > > + # Configuration for Update & Setup # > > > + # ####################################################################################### # > > > + > > > + > > > +class SettingsManager(UpdateSettingsManager, SetupSettingsManager, PrEvalSettingsManager): > > > + > > > + def GetPackagesSupported(self): > > > + ''' return iterable of edk2 packages supported by this build. > > > + These should be edk2 workspace relative paths ''' > > > + return CommonPlatform.PackagesSupported > > > + > > > + def GetArchitecturesSupported(self): > > > + ''' return iterable of edk2 architectures supported by this build ''' > > > + return CommonPlatform.ArchSupported > > > + > > > + def GetTargetsSupported(self): > > > + ''' return iterable of edk2 target tags supported by this build ''' > > > + return CommonPlatform.TargetsSupported > > > + > > > + def GetRequiredSubmodules(self): > > > + ''' return iterable containing RequiredSubmodule objects. > > > + If no RequiredSubmodules return an empty iterable > > > + ''' > > > + rs = [] > > > + > > > + # intentionally declare this one with recursive false to avoid overhead > > > + rs.append(RequiredSubmodule( > > > + "CryptoPkg/Library/OpensslLib/openssl", False)) > > > + > > > + # To avoid maintenance of this file for every new submodule > > > + # lets just parse the .gitmodules and add each if not already in list. > > > + # The GetRequiredSubmodules is designed to allow a build to optimize > > > + # the desired submodules but it isn't necessary for this repository. > > > + result = io.StringIO() > > > + ret = RunCmd("git", "config --file .gitmodules --get-regexp path", workingdir=self.GetWorkspaceRoot(), outstream=result) > > > + # Cmd output is expected to look like: > > > + # submodule.CryptoPkg/Library/OpensslLib/openssl.path CryptoPkg/Library/OpensslLib/openssl > > > + # submodule.SoftFloat.path ArmPkg/Library/ArmSoftFloatLib/berkeley-softfloat-3 > > > + if ret == 0: > > > + for line in result.getvalue().splitlines(): > > > + _, _, path = line.partition(" ") > > > + if path is not None: > > > + if path not in [x.path for x in rs]: > > > + rs.append(RequiredSubmodule(path, True)) # add it with recursive since we don't know > > > + return rs > > > + > > > + def SetArchitectures(self, list_of_requested_architectures): > > > + ''' Confirm the requests architecture list is valid and configure SettingsManager > > > + to run only the requested architectures. > > > + > > > + Raise Exception if a list_of_requested_architectures is not supported > > > + ''' > > > + unsupported = set(list_of_requested_architectures) - \ > > > + set(self.GetArchitecturesSupported()) > > > + if(len(unsupported) > 0): > > > + errorString = ( > > > + "Unsupported Architecture Requested: " + " ".join(unsupported)) > > > + logging.critical(errorString) > > > + raise Exception(errorString) > > > + self.ActualArchitectures = list_of_requested_architectures > > > + > > > + def GetWorkspaceRoot(self): > > > + ''' get WorkspacePath ''' > > > + return CommonPlatform.WorkspaceRoot > > > + > > > + def GetActiveScopes(self): > > > + ''' return tuple containing scopes that should be active for this process ''' > > > + > > > + scopes = CommonPlatform.Scopes > > > + ActualToolChainTag = shell_environment.GetBuildVars().GetValue("TOOL_CHAIN_TAG", "") > > > + > > > + if GetHostInfo().os.upper() == "LINUX" and ActualToolChainTag.upper().startswith("GCC"): > > > + if "AARCH64" in self.ActualArchitectures: > > > + scopes += ("gcc_aarch64_linux",) > > > + if "ARM" in self.ActualArchitectures: > > > + scopes += ("gcc_arm_linux",) > > > + return scopes > > > + > > > + def FilterPackagesToTest(self, changedFilesList: list, potentialPackagesList: list) -> list: > > > + ''' Filter other cases that this package should be built > > > + based on changed files. This should cover things that can't > > > + be detected as dependencies. ''' > > > + build_these_packages = [] > > > + possible_packages = potentialPackagesList.copy() > > > + for f in changedFilesList: > > > + # BaseTools files that might change the build > > > + if "BaseTools" in f: > > > + if os.path.splitext(f) not in [".txt", ".md"]: > > > + build_these_packages = possible_packages > > > + break > > > + > > > + # if the azure pipeline platform template file changed > > > + if "platform-build-run-steps.yml" in f: > > > + build_these_packages = possible_packages > > > + break > > > + > > > + > > > + return build_these_packages > > > + > > > + def GetPlatformDscAndConfig(self) -> tuple: > > > + ''' If a platform desires to provide its DSC then Policy 4 will evaluate if > > > + any of the changes will be built in the dsc. > > > + > > > + The tuple should be (, ) > > > + ''' > > > + return (os.path.join("ArmVirtPkg", "ArmVirtQemu.dsc"), {}) > > > + > > > + > > > + # ####################################################################################### # > > > + # Actual Configuration for Platform Build # > > > + # ####################################################################################### # > > > + > > > + > > > +class PlatformBuilder(UefiBuilder, BuildSettingsManager): > > > + def __init__(self): > > > + UefiBuilder.__init__(self) > > > + > > > + def AddCommandLineOptions(self, parserObj): > > > + ''' Add command line options to the argparser ''' > > > + parserObj.add_argument('-a', "--arch", dest="build_arch", type=str, default="AARCH64", > > > + help="Optional - Architecture to build. Default = AARCH64") > > > + > > > + def RetrieveCommandLineOptions(self, args): > > > + ''' Retrieve command line options from the argparser ''' > > > + > > > + shell_environment.GetBuildVars().SetValue( > > > + "TARGET_ARCH", args.build_arch.upper(), "From CmdLine") > > > + > > > + shell_environment.GetBuildVars().SetValue( > > > + "ACTIVE_PLATFORM", "ArmVirtPkg/ArmVirtQemu.dsc", "From CmdLine") > > > + > > > + def GetWorkspaceRoot(self): > > > + ''' get WorkspacePath ''' > > > + return CommonPlatform.WorkspaceRoot > > > + > > > + def GetPackagesPath(self): > > > + ''' Return a list of workspace relative paths that should be mapped as edk2 PackagesPath ''' > > > + return () > > > + > > > + def GetActiveScopes(self): > > > + ''' return tuple containing scopes that should be active for this process ''' > > > + scopes = CommonPlatform.Scopes > > > + ActualToolChainTag = shell_environment.GetBuildVars().GetValue("TOOL_CHAIN_TAG", "") > > > + Arch = shell_environment.GetBuildVars().GetValue("TARGET_ARCH", "") > > > + > > > + if GetHostInfo().os.upper() == "LINUX" and ActualToolChainTag.upper().startswith("GCC"): > > > + if "AARCH64" == Arch: > > > + scopes += ("gcc_aarch64_linux",) > > > + elif "ARM" == Arch: > > > + scopes += ("gcc_arm_linux",) > > > + return scopes > > > + > > > + def GetName(self): > > > + ''' Get the name of the repo, platform, or product being build ''' > > > + ''' Used for naming the log file, among others ''' > > > + # check the startup nsh flag and if set then rename the log file. > > > + # this helps in CI so we don't overwrite the build log since running > > > + # uses the stuart_build command. > > > + if(shell_environment.GetBuildVars().GetValue("MAKE_STARTUP_NSH", "FALSE") == "TRUE"): > > > + return "ArmVirtPkg_With_Run" > > > + return "ArmVirtPkg" > > > + > > > + def GetLoggingLevel(self, loggerType): > > > + ''' Get the logging level for a given type > > > + base == lowest logging level supported > > > + con == Screen logging > > > + txt == plain text file logging > > > + md == markdown file logging > > > + ''' > > > + return logging.DEBUG > > > + > > > + def SetPlatformEnv(self): > > > + logging.debug("PlatformBuilder SetPlatformEnv") > > > + self.env.SetValue("PRODUCT_NAME", "ArmVirtQemu", "Platform Hardcoded") > > > + self.env.SetValue("MAKE_STARTUP_NSH", "FALSE", "Default to false") > > > + self.env.SetValue("QEMU_HEADLESS", "FALSE", "Default to false") > > > + return 0 > > > + > > > + def PlatformPreBuild(self): > > > + return 0 > > > + > > > + def PlatformPostBuild(self): > > > + return 0 > > > + > > > + def FlashRomImage(self): > > > + VirtualDrive = os.path.join(self.env.GetValue( > > > + "BUILD_OUTPUT_BASE"), "VirtualDrive") > > > + os.makedirs(VirtualDrive, exist_ok=True) > > > + OutputPath_FV = os.path.join( > > > + self.env.GetValue("BUILD_OUTPUT_BASE"), "FV") > > > + Built_FV = os.path.join(OutputPath_FV, "QEMU_EFI.fd") > > > + > > > + # pad fd to 64mb > > > + with open(Built_FV, "ab") as fvfile: > > > + fvfile.seek(0, os.SEEK_END) > > > + additional = b'\0' * ((64 * 1024 * 1024)-fvfile.tell()) > > > + fvfile.write(additional) > > > + > > > + # QEMU must be on that path > > > + > > > + # Unique Command and Args parameters per ARCH > > > + if (self.env.GetValue("TARGET_ARCH").upper() == "AARCH64"): > > > + cmd = "qemu-system-aarch64" > > > + args = "-M virt" > > > + args += " -cpu cortex-a57" # emulate cpu > > > + elif(self.env.GetValue("TARGET_ARCH").upper() == "ARM"): > > > + cmd = "qemu-system-arm" > > > + args = "-M virt" > > > + args += " -cpu cortex-a15" # emulate cpu > > > + else: > > > + raise NotImplementedError() > > > + > > > + # Common Args > > > + args += " -pflash " + Built_FV # path to fw > > > + args += " -m 1024" # 1gb memory > > > + # turn off network > > > + args += " -net none" > > > + # Serial messages out > > > + args += " -serial stdio" > > > + # Mount disk with startup.nsh > > > + args += f" -drive file=fat:rw:{VirtualDrive},format=raw,media=disk" > > > + > > > + # Conditional Args > > > + if (self.env.GetValue("QEMU_HEADLESS").upper() == "TRUE"): > > > + args += " -display none" # no graphics > > > + > > > + if (self.env.GetValue("MAKE_STARTUP_NSH").upper() == "TRUE"): > > > + f = open(os.path.join(VirtualDrive, "startup.nsh"), "w") > > > + f.write("BOOT SUCCESS !!! \n") > > > + # add commands here > > > + f.write("reset -s\n") > > > + f.close() > > > + > > > + ret = RunCmd(cmd, args) > > > + > > > + if ret == 0xc0000005: > > > + # for some reason getting a c0000005 on successful return > > > + return 0 > > > + > > > + return ret > > > diff --git a/ArmVirtPkg/PlatformCI/ReadMe.md b/ArmVirtPkg/PlatformCI/ReadMe.md > > > new file mode 100644 > > > index 000000000000..7c11d925f59e > > > --- /dev/null > > > +++ b/ArmVirtPkg/PlatformCI/ReadMe.md > > > @@ -0,0 +1,125 @@ > > > +# ArmVirtPkg - Platform CI > > > + > > > +This Readme.md describes the Azure DevOps based Platform CI for ArmVirtPkg and how > > > +to use the same Pytools based build infrastructure locally. > > > + > > > +## Supported Configuration Details > > > + > > > +This solution for building and running ArmVirtPkg has only been validated with Ubuntu > > > +18.04 and the GCC5 toolchain. Two different firmware builds are supported and are > > > +described below. > > > + > > > +| Configuration name | Architecture | DSC File |Additional Flags | > > > +| :---------- | :----- | :----- | :---- | > > > +| AARCH64 | AARCH64 | ArmVirtQemu.dsc | None | > > > +| ARM | ARM | ArmVirtQemu.dsc | None | > > > + > > > +## EDK2 Developer environment > > > + > > > +- [Python 3.8.x - Download & Install](https://www.python.org/downloads/) > > > +- [GIT - Download & Install](https://git-scm.com/download/) > > > +- [QEMU - Download, Install, and add to your path](https://www.qemu.org/download/) > > > +- [Edk2 Source](https://github.com/tianocore/edk2) > > > +- Additional packages found necessary for Ubuntu 18.04 > > > + - apt-get install gcc g++ make uuid-dev > > > + > > > +Note: edksetup, Submodule initialization and manual installation of NASM, iASL, or > > > +the required cross-compiler toolchains are **not** required, this is handled by the > > > +Pytools build system. > > > + > > > +## Building with Pytools for ArmVirtPkg > > > + > > > +1. [Optional] Create a Python Virtual Environment - generally once per workspace > > > + > > > + ``` bash > > > + python -m venv > > > + ``` > > > + > > > +2. [Optional] Activate Virtual Environment - each time new shell opened > > > + - Windows > > > + > > > + ``` bash > > > + /Scripts/activate.bat > > > + ``` > > > + > > > + - Linux > > > + > > > + ```bash > > > + source /bin/activate > > > + ``` > > > + > > > +3. Install Pytools - generally once per virtual env or whenever pip-requirements.txt changes > > > + > > > + ``` bash > > > + pip install --upgrade -r pip-requirements.txt > > > + ``` > > > + > > > +4. Initialize & Update Submodules - only when submodules updated > > > + > > > + ``` bash > > > + stuart_setup -c ArmVirtPkg/PlatformCI/PlatformBuild.py TOOL_CHAIN_TAG= -a > > > + ``` > > > + > > > +5. Initialize & Update Dependencies - only as needed when ext_deps change > > > + > > > + ``` bash > > > + stuart_update -c ArmVirtPkg/PlatformCI/PlatformBuild.py TOOL_CHAIN_TAG= -a > > > + ``` > > > + > > > +6. Compile the basetools if necessary - only when basetools C source files change > > > + > > > + ``` bash > > > + python BaseTools/Edk2ToolsBuild.py -t > > > + ``` > > > + > > > +7. Compile Firmware > > > + > > > + ``` bash > > > + stuart_build -c ArmVirtPkg/PlatformCI/PlatformBuild.py TOOL_CHAIN_TAG= -a > > > + ``` > > > + > > > + - use `stuart_build -c ArmVirtPkg/PlatformCI/PlatformBuild.py -h` option to see additional > > > + options like `--clean` > > > + > > > +8. Running Emulator > > > + - You can add `--FlashRom` to the end of your build command and the emulator will run after the > > > + build is complete. > > > + - or use the `--FlashOnly` feature to just run the emulator. > > > + > > > + ``` bash > > > + stuart_build -c ArmVirtPkg/PlatformCI/PlatformBuild.py TOOL_CHAIN_TAG= -a --FlashOnly > > > + ``` > > > + > > > +### Notes > > > + > > > +1. Including the expected build architecture and toolchain to the _stuart_update_ command is critical. > > > + This is because there are extra scopes and tools that will be resolved during the update step that > > > + need to match your build step. > > > +2. Configuring *ACTIVE_PLATFORM* and *TARGET_ARCH* in Conf/target.txt is **not** required. This > > > + environment is set by PlatformBuild.py based upon the `[-a ]` parameter. > > > +3. QEMU must be on your path. On Windows this is a manual process and not part of the QEMU installer. > > > + > > > +**NOTE:** Logging the execution output will be in the normal stuart log as well as to your console. > > > + > > > +### Custom Build Options > > > + > > > +**MAKE_STARTUP_NSH=TRUE** will output a *startup.nsh* file to the location mapped as fs0. This is > > > +used in CI in combination with the `--FlashOnly` feature to run QEMU to the UEFI shell and then execute > > > +the contents of *startup.nsh*. > > > + > > > +**QEMU_HEADLESS=TRUE** Since CI servers run headless QEMU must be told to run with no display otherwise > > > +an error occurs. Locally you don't need to set this. > > > + > > > +### Passing Build Defines > > > + > > > +To pass build defines through _stuart_build_, prepend `BLD_*_`to the define name and pass it on the > > > +command-line. _stuart_build_ currently requires values to be assigned, so add an`=1` suffix for bare defines. > > > +For example, to enable the TPM2 support, instead of the traditional "-D TPM2_ENABLE=TRUE", the stuart_build > > > +command-line would be: > > > + > > > +`stuart_build -c ArmVirtPkg/PlatformCI/PlatformBuild.py BLD_*_TPM2_ENABLE=TRUE` > > > + > > > +## References > > > + > > > +- [Installing and using Pytools](https://github.com/tianocore/edk2-pytool-extensions/blob/master/docs/using.md#installing) > > > +- More on [python virtual environments](https://docs.python.org/3/library/venv.html) > > > diff --git a/ArmVirtPkg/PlatformCI/iasl_ext_dep.yaml b/ArmVirtPkg/PlatformCI/iasl_ext_dep.yaml > > > new file mode 100644 > > > index 000000000000..8869ed3ecef1 > > > --- /dev/null > > > +++ b/ArmVirtPkg/PlatformCI/iasl_ext_dep.yaml > > > @@ -0,0 +1,21 @@ > > > +## @file > > > +# Download iasl executable tool from a nuget.org package > > > +# - package contains different binaries based on host > > > +# Add the folder with the tool to the path > > > +# > > > +# This is only downloaded for scope armvirt thus > > > +# should have no impact on the asl compiler used by any > > > +# other platform build > > > +# > > > +# Copyright (c) Microsoft Corporation. > > > +# SPDX-License-Identifier: BSD-2-Clause-Patent > > > +## > > > +{ > > > + "id": "iasl-armvirt-1", > > > + "scope": "armvirt", > > > + "type": "nuget", > > > + "name": "iasl", > > > + "source": "https://api.nuget.org/v3/index.json", > > > + "version": "20190215.0.0", > > > + "flags": ["set_path", "host_specific"], > > > +} > > > -- > > > 2.16.3.windows.1 > > >