Re: commit "BaseTools: Improve the cache hit in the edk2 build cache" causes build.py unknown fatal error

["Steven Shi" <steven.shi@intel.com>]

[-- Attachment #1: Type: text/plain, Size: 6768 bytes --]

Hi Vladimir,
Below is an example to do clean build with the cache. The build cache is enabled through --hash --binary-destination and --binary-source options.


·         Produce the build cache to a folder (e.g. BinCache):
$ build -p OvmfPkg/OvmfPkgIa32.dsc -a IA32 -t GCC5 -b DEBUG --hash --binary-destination=BinCache

·         Remove the Build folder
$ rm Build/ -r –f

·         Consume the build cache:
$ build -p OvmfPkg/OvmfPkgIa32.dsc -a IA32 -t GCC5 -b DEBUG --hash --binary-source=BinCache

Thanks

Steven Shi
Intel\SSG\FID\Firmware Infrastructure

From: Vladimir Olovyannikov [mailto:vladimir.olovyannikov@broadcom.com]
Sent: Tuesday, September 10, 2019 4:15 AM
To: Feng, Bob C <bob.c.feng@intel.com>; Shi, Steven <steven.shi@intel.com>; devel@edk2.groups.io
Cc: Gao, Liming <liming.gao@intel.com>; Rodriguez, Christian <christian.rodriguez@intel.com>; Johnson, Michael <michael.johnson@intel.com>
Subject: RE: commit "BaseTools: Improve the cache hit in the edk2 build cache" causes build.py unknown fatal error

Hi Bob, Steven,

Thank you for update.
Yes, it fails with “build clean”.
Example:
. ./edksetup.sh
build -a AARCH64 -p BroadcomPlatformPkg/StingrayPkg/StingrayPkg.dsc -t GCC5 -b RELEASE -D PLAT=Stingray -D TARGET=RELEASE -D EDK2_STAMP="dev/bcm958742k 59b754c9f697" cleanall

Bob, with your patch it builds fine.
Steven, in case I am missing something, how do you enable the build cache?

Thank you,
Vladimir
From: Feng, Bob C [mailto:bob.c.feng@intel.com<mailto:bob.c.feng@intel.com>]
Sent: Sunday, September 8, 2019 6:23 PM
To: Shi, Steven <steven.shi@intel.com<mailto:steven.shi@intel.com>>; Vladimir Olovyannikov <vladimir.olovyannikov@broadcom.com<mailto:vladimir.olovyannikov@broadcom.com>>; devel@edk2.groups.io<mailto:devel@edk2.groups.io>
Cc: Gao, Liming <liming.gao@intel.com<mailto:liming.gao@intel.com>>; Rodriguez, Christian <christian.rodriguez@intel.com<mailto:christian.rodriguez@intel.com>>; Johnson, Michael <michael.johnson@intel.com<mailto:michael.johnson@intel.com>>
Subject: RE: commit "BaseTools: Improve the cache hit in the edk2 build cache" causes build.py unknown fatal error

Hi Vladimir and Steven,

I checked code and think when the build target are 'clean', 'cleanlib', 'cleanall', 'run' and 'fds', build will fail with the error like you mentioned.
I filed a Bugzilla https://bugzilla.tianocore.org/show_bug.cgi?id=2166
And provide a fix in https://edk2.groups.io/g/devel/message/47014

Thanks,
Bob

From: Shi, Steven
Sent: Saturday, September 7, 2019 10:16 AM
To: Vladimir Olovyannikov <vladimir.olovyannikov@broadcom.com<mailto:vladimir.olovyannikov@broadcom.com>>; devel@edk2.groups.io<mailto:devel@edk2.groups.io>
Cc: Gao, Liming <liming.gao@intel.com<mailto:liming.gao@intel.com>>; Feng, Bob C <bob.c.feng@intel.com<mailto:bob.c.feng@intel.com>>; Rodriguez, Christian <christian.rodriguez@intel.com<mailto:christian.rodriguez@intel.com>>; Johnson, Michael <michael.johnson@intel.com<mailto:michael.johnson@intel.com>>
Subject: RE: commit "BaseTools: Improve the cache hit in the edk2 build cache" causes build.py unknown fatal error

Hi Vladimir,
Did you enable the build cache? Could you let me know your build steps and commands?


Thanks
Steven

From: Vladimir Olovyannikov [mailto:vladimir.olovyannikov@broadcom.com]
Sent: Saturday, September 7, 2019 4:45 AM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Shi, Steven <steven.shi@intel.com<mailto:steven.shi@intel.com>>
Cc: Gao, Liming <liming.gao@intel.com<mailto:liming.gao@intel.com>>; Feng, Bob C <bob.c.feng@intel.com<mailto:bob.c.feng@intel.com>>; Rodriguez, Christian <christian.rodriguez@intel.com<mailto:christian.rodriguez@intel.com>>; Johnson, Michael <michael.johnson@intel.com<mailto:michael.johnson@intel.com>>
Subject: commit "BaseTools: Improve the cache hit in the edk2 build cache" causes build.py unknown fatal error

Hi Steven,

The commit
0e7e7a264cd80ab71ea0f9e9da2d0617d4b539c4
BaseTools: Improve the cache hit in the edk2 build cache
Author: Shi, Steven <steven.shi@intel.com<mailto:steven.shi@intel.com>>  2019-08-15 07:26:17
Committer: Feng, Bob C <bob.c.feng@intel.com<mailto:bob.c.feng@intel.com>>  2019-08-19 23:06:05
Parent: 811328172841ce3f7ec1fe7033c130726ad189b7 (ShellPkg/UefiShellDriver1CommandsLib: Make array big enough)
Child:  56c786b0a9fd4f08b1f0bc10b7718a7577cb0146 (BaseTools: Print first cache missing file for build cachle)
Branches: master, remotes/origin/master
Follows: edk2-stable201905
Precedes: edk2-stable201908

causes "unknown fatal error" in build.py when building our platform.

Here is the log:
...
build.py...
 : error C0DE: Unknown fatal error when processing [/home/volovyan/otherwork/sandbox/uefi-related/uefi-orig/ArmPkg/Library/ArmGicArchSecLib/ArmGicArchSecLib.inf]

(Please send email to devel@edk2.groups.io<mailto:devel@edk2.groups.io> for help, attaching following call stack trace!)

(Python 3.6.8 on linux) Traceback (most recent call last):
  File "/home/volovyan/otherwork/sandbox/uefi-related/uefi-orig/BaseTools/BinWrappers/PosixLike/../../Source/Python/build/build.py", line 2493, in Main
    MyBuild.Launch()
  File "/home/volovyan/otherwork/sandbox/uefi-related/uefi-orig/BaseTools/BinWrappers/PosixLike/../../Source/Python/build/build.py", line 2287, in Launch
    self._BuildPlatform()
  File "/home/volovyan/otherwork/sandbox/uefi-related/uefi-orig/BaseTools/BinWrappers/PosixLike/../../Source/Python/build/build.py", line 1742, in _BuildPlatform
    self._BuildPa(self.Target, Pa, FfsCommand=CmdListDict,PcdMaList=PcdMaList)
  File "/home/volovyan/otherwork/sandbox/uefi-related/uefi-orig/BaseTools/BinWrappers/PosixLike/../../Source/Python/build/build.py", line 1236, in _BuildPa
    AutoGenObject.CreateCodeFile(True)
  File "/home/volovyan/otherwork/sandbox/uefi-related/uefi-orig/BaseTools/Source/Python/Common/caching.py", line 18, in CallMeHere
    Value = self._function(obj, *args,**kwargs)
  File "/home/volovyan/otherwork/sandbox/uefi-related/uefi-orig/BaseTools/Source/Python/AutoGen/PlatformAutoGen.py", line 171, in CreateCodeFile
    Ma.CreateCodeFile(CreateModuleCodeFile)
  File "/home/volovyan/otherwork/sandbox/uefi-related/uefi-orig/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py", line 1841, in CreateCodeFile
    LibraryAutoGen.CreateCodeFile()
  File "/home/volovyan/otherwork/sandbox/uefi-related/uefi-orig/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py", line 1886, in CreateCodeFile
    with GlobalData.cache_lock:
AttributeError: __enter__


- Failed -
Build end time: 13:31:30, Sep.06 2019
Build total time: 00:00:04

Builds before the above mentioned commit was pushed, were fine.

Thank you,
Vladimir

[-- Attachment #2: Type: text/html, Size: 21837 bytes --]