Building current OVMF master with multiple workspaces on Windows/VS2017 fails due to missing includes?

Building current OVMF master with multiple workspaces on Windows/VS2017 fails due to missing includes?

[Ryszard Knop]

I have a bit odd issues with building OVMF from the current master 
branch (commit 0eccea3...), with the following setup using multiple 
workspaces (this is important):

- Windows 10, 64-bit
- Visual Studio 2017 build tools (conf'd with VS2017_PREFIX envvar)
- NASM 2.14.02 (conf'd with NASM_PREFIX envvar)
- Python 3.7.3 running the latest BaseTools
- WORKSPACE envvar points at a dir with extra modules to be included 
with OVMF (C:\Env\OvmfBuild)
- PACKAGES_PATH points at the EDK2 repo (C:\Env\Workspace\edk2)

All envvars are set before edksetup. Trying to build -a X64 -p 
OvmfPkg/OvmfPkgX64.dsc -b RELEASE -t VS2017 fails with the following:

"C:\Env\BuildTools\MSVS2017_SDK_WDK_Windows10_16299\Program 
Files\Microsoft Visual 
Studio\2017\BuildTools\VC\Tools\MSVC\14.10.25017\bin\Hostx86\x64\cl.exe" 
/nologo /E /TC /FIAutoGen.h 
/IC:\Env\Workspace\edk2\MdePkg\Library\BaseLib\X64 
/IC:\Env\Workspace\edk2\MdePkg\Library\BaseLib 
/Ic:\Env\OvmfBuild\Build\OvmfX64\RELEASE_VS2017\X64\MdePkg\Library\BaseLib\BaseLib\DEBUG 
/IC:\Env\Workspace\edk2\MdePkg /IC:\Env\Workspace\edk2\MdePkg\Include 
/IC:\Env\Workspace\edk2\MdePkg\Include\X64 
C:\Env\Workspace\edk2\MdePkg\Library\BaseLib\X64\LongJump.nasm > 
c:\Env\OvmfBuild\Build\OvmfX64\RELEASE_VS2017\X64\MdePkg\Library\BaseLib\BaseLib\OUTPUT\X64\LongJump.i
LongJump.nasm
         Trim --trim-long --source-code -o 
c:\Env\OvmfBuild\Build\OvmfX64\RELEASE_VS2017\X64\MdePkg\Library\BaseLib\BaseLib\OUTPUT\X64\LongJump.iii 
c:\Env\OvmfBuild\Build\OvmfX64\RELEASE_VS2017\X64\MdePkg\Library\BaseLib\BaseLib\OUTPUT\X64\LongJump.i
         "C:\Env\BuildTools\NASM\2.14.02\nasm" 
-IC:\Env\Workspace\edk2\MdePkg\Library\BaseLib\X64\ -Ox -f win64 -o 
c:\Env\OvmfBuild\Build\OvmfX64\RELEASE_VS2017\X64\MdePkg\Library\BaseLib\BaseLib\OUTPUT\X64\LongJump.obj 
c:\Env\OvmfBuild\Build\OvmfX64\RELEASE_VS2017\X64\MdePkg\Library\BaseLib\BaseLib\OUTPUT\X64\LongJump.iii
c:\Env\OvmfBuild\Build\OvmfX64\RELEASE_VS2017\X64\MdePkg\Library\BaseLib\BaseLib\OUTPUT\X64\LongJump.iii:16: 
fatal: unable to open include file `Nasm.inc'
NMAKE : fatal error U1077: 'C:\Env\BuildTools\NASM\2.14.02\nasm.EXE' : 
return code '0x1'

Missing Nasm.inc, which is present in MdePkg/Include/X64/Nasm.inc - but 
the weird thing is, adding extra NASM include paths was added recently 
(in BaseTools/Source/Python/AutoGen/GenMake.py there's a block that 
replaces NASM_INC with appropriate directories, build_rule.template 
should generate NASM calls with these, as of commit 7c3a1ef). After 
adding a few prints to the GenMake I can see it does find the proper 
include paths, but none of them land in the final NASM call in the 
Makefile, even though it knows about the Nasm.inc dependency - the 
NASM_INC macro seems to be completely missing from the generated Makefile:

$(OUTPUT_DIR)\X64\LongJump.obj : $(MAKE_FILE)
$(OUTPUT_DIR)\X64\LongJump.obj : C:\Env\Workspace\edk2\MdePkg\Include\Base.h
$(OUTPUT_DIR)\X64\LongJump.obj : 
C:\Env\Workspace\edk2\MdePkg\Include\X64\Nasm.inc
$(OUTPUT_DIR)\X64\LongJump.obj : $(DEBUG_DIR)\AutoGen.h
$(OUTPUT_DIR)\X64\LongJump.obj : 
C:\Env\Workspace\edk2\MdePkg\Include\X64\ProcessorBind.h
$(OUTPUT_DIR)\X64\LongJump.obj : 
C:\Env\Workspace\edk2\MdePkg\Include\Library\PcdLib.h
$(OUTPUT_DIR)\X64\LongJump.obj : 
C:\Env\Workspace\edk2\MdePkg\Library\BaseLib\X64\LongJump.nasm
     "$(PP)" $(PP_FLAGS) $(INC) 
C:\Env\Workspace\edk2\MdePkg\Library\BaseLib\X64\LongJump.nasm > 
c:\Env\OvmfBuild\Build\OvmfX64\RELEASE_VS2017\X64\MdePkg\Library\BaseLib\BaseLib\OUTPUT\X64\LongJump.i
     Trim --trim-long --source-code -o 
c:\Env\OvmfBuild\Build\OvmfX64\RELEASE_VS2017\X64\MdePkg\Library\BaseLib\BaseLib\OUTPUT\X64\LongJump.iii 
c:\Env\OvmfBuild\Build\OvmfX64\RELEASE_VS2017\X64\MdePkg\Library\BaseLib\BaseLib\OUTPUT\X64\LongJump.i
     "$(NASM)" -IC:\Env\Workspace\edk2\MdePkg\Library\BaseLib\X64\ 
$(NASM_FLAGS) -o 
c:\Env\OvmfBuild\Build\OvmfX64\RELEASE_VS2017\X64\MdePkg\Library\BaseLib\BaseLib\OUTPUT\X64\LongJump.obj 
c:\Env\OvmfBuild\Build\OvmfX64\RELEASE_VS2017\X64\MdePkg\Library\BaseLib\BaseLib\OUTPUT\X64\LongJump.iii

The nasty workaround is to copy Nasm.inc into MdePkg/Library/BaseLib/X64 
and everything builds just fine after that.

Another interesting thing is that if I don't use WORKSPACE and 
PACKAGES_PATH (both contain no spaces and are on local file systems) but 
rather build everything directly in the EDK2 directory, OVMF also builds 
fine (Makefile contains the NASM_INC definition which is included in 
NASM calls). Have anyone also hit this issue?

Thanks, Richard

Re: [edk2-devel] Building current OVMF master with multiple workspaces on Windows/VS2017 fails due to missing includes?

[Laszlo Ersek]

On 04/16/19 16:56, Ryszard Knop wrote:
> I have a bit odd issues with building OVMF from the current master
> branch (commit 0eccea3...), with the following setup using multiple
> workspaces (this is important):
> 
> - Windows 10, 64-bit
> - Visual Studio 2017 build tools (conf'd with VS2017_PREFIX envvar)
> - NASM 2.14.02 (conf'd with NASM_PREFIX envvar)
> - Python 3.7.3 running the latest BaseTools
> - WORKSPACE envvar points at a dir with extra modules to be included
> with OVMF (C:\Env\OvmfBuild)
> - PACKAGES_PATH points at the EDK2 repo (C:\Env\Workspace\edk2)
> 
> All envvars are set before edksetup.

On Linux anyway, the above has worked for me, with two finer points:

- In $WORKSPACE, the Conf/ directory is created manually (as an empty
directory), just before sourcing edksetup.sh

- (not sure if this matters, but) the "--platform" option of "build"
refers to a standalone DSC file under $WORKSPACE, not to one of the OVMF
DSC files under $PACKAGES_PATH.

The use case for the above is different from yours though -- it's not
for building OVMF with extra modules included, but for building
standalone UEFI applications with various libraries from edk2.

... It's also possible, I guess, that the fix for
<https://bugzilla.tianocore.org/show_bug.cgi?id=1085> simply forgot
about PACKAGES_PATH.

If you can provide a stripped-down reproducer (with basically "empty"
additional driver modules etc), I'd suggest packaging it all up in a zip
file, and reporting a new TianoCore BZ for BaseTools, with the zip file
attached. (The zip file is particularly helpful because it can preserve
the directory structure too, which individual BZ attachments couldn't.)

Thanks
Laszlo


> Trying to build -a X64 -p
> OvmfPkg/OvmfPkgX64.dsc -b RELEASE -t VS2017 fails with the following:
> 
> "C:\Env\BuildTools\MSVS2017_SDK_WDK_Windows10_16299\Program
> Files\Microsoft Visual
> Studio\2017\BuildTools\VC\Tools\MSVC\14.10.25017\bin\Hostx86\x64\cl.exe"
> /nologo /E /TC /FIAutoGen.h
> /IC:\Env\Workspace\edk2\MdePkg\Library\BaseLib\X64
> /IC:\Env\Workspace\edk2\MdePkg\Library\BaseLib
> /Ic:\Env\OvmfBuild\Build\OvmfX64\RELEASE_VS2017\X64\MdePkg\Library\BaseLib\BaseLib\DEBUG
> /IC:\Env\Workspace\edk2\MdePkg /IC:\Env\Workspace\edk2\MdePkg\Include
> /IC:\Env\Workspace\edk2\MdePkg\Include\X64
> C:\Env\Workspace\edk2\MdePkg\Library\BaseLib\X64\LongJump.nasm >
> c:\Env\OvmfBuild\Build\OvmfX64\RELEASE_VS2017\X64\MdePkg\Library\BaseLib\BaseLib\OUTPUT\X64\LongJump.i
> 
> LongJump.nasm
>         Trim --trim-long --source-code -o
> c:\Env\OvmfBuild\Build\OvmfX64\RELEASE_VS2017\X64\MdePkg\Library\BaseLib\BaseLib\OUTPUT\X64\LongJump.iii
> c:\Env\OvmfBuild\Build\OvmfX64\RELEASE_VS2017\X64\MdePkg\Library\BaseLib\BaseLib\OUTPUT\X64\LongJump.i
> 
>         "C:\Env\BuildTools\NASM\2.14.02\nasm"
> -IC:\Env\Workspace\edk2\MdePkg\Library\BaseLib\X64\ -Ox -f win64 -o
> c:\Env\OvmfBuild\Build\OvmfX64\RELEASE_VS2017\X64\MdePkg\Library\BaseLib\BaseLib\OUTPUT\X64\LongJump.obj
> c:\Env\OvmfBuild\Build\OvmfX64\RELEASE_VS2017\X64\MdePkg\Library\BaseLib\BaseLib\OUTPUT\X64\LongJump.iii
> 
> c:\Env\OvmfBuild\Build\OvmfX64\RELEASE_VS2017\X64\MdePkg\Library\BaseLib\BaseLib\OUTPUT\X64\LongJump.iii:16:
> fatal: unable to open include file `Nasm.inc'
> NMAKE : fatal error U1077: 'C:\Env\BuildTools\NASM\2.14.02\nasm.EXE' :
> return code '0x1'
> 
> Missing Nasm.inc, which is present in MdePkg/Include/X64/Nasm.inc - but
> the weird thing is, adding extra NASM include paths was added recently
> (in BaseTools/Source/Python/AutoGen/GenMake.py there's a block that
> replaces NASM_INC with appropriate directories, build_rule.template
> should generate NASM calls with these, as of commit 7c3a1ef). After
> adding a few prints to the GenMake I can see it does find the proper
> include paths, but none of them land in the final NASM call in the
> Makefile, even though it knows about the Nasm.inc dependency - the
> NASM_INC macro seems to be completely missing from the generated Makefile:
> 
> $(OUTPUT_DIR)\X64\LongJump.obj : $(MAKE_FILE)
> $(OUTPUT_DIR)\X64\LongJump.obj :
> C:\Env\Workspace\edk2\MdePkg\Include\Base.h
> $(OUTPUT_DIR)\X64\LongJump.obj :
> C:\Env\Workspace\edk2\MdePkg\Include\X64\Nasm.inc
> $(OUTPUT_DIR)\X64\LongJump.obj : $(DEBUG_DIR)\AutoGen.h
> $(OUTPUT_DIR)\X64\LongJump.obj :
> C:\Env\Workspace\edk2\MdePkg\Include\X64\ProcessorBind.h
> $(OUTPUT_DIR)\X64\LongJump.obj :
> C:\Env\Workspace\edk2\MdePkg\Include\Library\PcdLib.h
> $(OUTPUT_DIR)\X64\LongJump.obj :
> C:\Env\Workspace\edk2\MdePkg\Library\BaseLib\X64\LongJump.nasm
>     "$(PP)" $(PP_FLAGS) $(INC)
> C:\Env\Workspace\edk2\MdePkg\Library\BaseLib\X64\LongJump.nasm >
> c:\Env\OvmfBuild\Build\OvmfX64\RELEASE_VS2017\X64\MdePkg\Library\BaseLib\BaseLib\OUTPUT\X64\LongJump.i
> 
>     Trim --trim-long --source-code -o
> c:\Env\OvmfBuild\Build\OvmfX64\RELEASE_VS2017\X64\MdePkg\Library\BaseLib\BaseLib\OUTPUT\X64\LongJump.iii
> c:\Env\OvmfBuild\Build\OvmfX64\RELEASE_VS2017\X64\MdePkg\Library\BaseLib\BaseLib\OUTPUT\X64\LongJump.i
> 
>     "$(NASM)" -IC:\Env\Workspace\edk2\MdePkg\Library\BaseLib\X64\
> $(NASM_FLAGS) -o
> c:\Env\OvmfBuild\Build\OvmfX64\RELEASE_VS2017\X64\MdePkg\Library\BaseLib\BaseLib\OUTPUT\X64\LongJump.obj
> c:\Env\OvmfBuild\Build\OvmfX64\RELEASE_VS2017\X64\MdePkg\Library\BaseLib\BaseLib\OUTPUT\X64\LongJump.iii
> 
> 
> The nasty workaround is to copy Nasm.inc into MdePkg/Library/BaseLib/X64
> and everything builds just fine after that.
> 
> Another interesting thing is that if I don't use WORKSPACE and
> PACKAGES_PATH (both contain no spaces and are on local file systems) but
> rather build everything directly in the EDK2 directory, OVMF also builds
> fine (Makefile contains the NASM_INC definition which is included in
> NASM calls). Have anyone also hit this issue?
> 
> Thanks, Richard
> 
> 
> 
> 

Re: [edk2-devel] Building current OVMF master with multiple workspaces on Windows/VS2017 fails due to missing includes?

[Liming Gao]

This change needs to use new build_rule.txt. When call edksetup.bat, please try 'edksetup.bat Reconfig'. This command will copy build_rule.txt into Conf directory. 

Thanks
Liming
> -----Original Message-----
> From: devel@edk2.groups.io [mailto:devel@edk2.groups.io] On Behalf Of Ryszard Knop
> Sent: Tuesday, April 16, 2019 10:56 PM
> To: devel@edk2.groups.io
> Subject: [edk2-devel] Building current OVMF master with multiple workspaces on Windows/VS2017 fails due to missing includes?
> 
> I have a bit odd issues with building OVMF from the current master
> branch (commit 0eccea3...), with the following setup using multiple
> workspaces (this is important):
> 
> - Windows 10, 64-bit
> - Visual Studio 2017 build tools (conf'd with VS2017_PREFIX envvar)
> - NASM 2.14.02 (conf'd with NASM_PREFIX envvar)
> - Python 3.7.3 running the latest BaseTools
> - WORKSPACE envvar points at a dir with extra modules to be included
> with OVMF (C:\Env\OvmfBuild)
> - PACKAGES_PATH points at the EDK2 repo (C:\Env\Workspace\edk2)
> 
> All envvars are set before edksetup. Trying to build -a X64 -p
> OvmfPkg/OvmfPkgX64.dsc -b RELEASE -t VS2017 fails with the following:
> 
> "C:\Env\BuildTools\MSVS2017_SDK_WDK_Windows10_16299\Program
> Files\Microsoft Visual
> Studio\2017\BuildTools\VC\Tools\MSVC\14.10.25017\bin\Hostx86\x64\cl.exe"
> /nologo /E /TC /FIAutoGen.h
> /IC:\Env\Workspace\edk2\MdePkg\Library\BaseLib\X64
> /IC:\Env\Workspace\edk2\MdePkg\Library\BaseLib
> /Ic:\Env\OvmfBuild\Build\OvmfX64\RELEASE_VS2017\X64\MdePkg\Library\BaseLib\BaseLib\DEBUG
> /IC:\Env\Workspace\edk2\MdePkg /IC:\Env\Workspace\edk2\MdePkg\Include
> /IC:\Env\Workspace\edk2\MdePkg\Include\X64
> C:\Env\Workspace\edk2\MdePkg\Library\BaseLib\X64\LongJump.nasm >
> c:\Env\OvmfBuild\Build\OvmfX64\RELEASE_VS2017\X64\MdePkg\Library\BaseLib\BaseLib\OUTPUT\X64\LongJump.i
> LongJump.nasm
>          Trim --trim-long --source-code -o
> c:\Env\OvmfBuild\Build\OvmfX64\RELEASE_VS2017\X64\MdePkg\Library\BaseLib\BaseLib\OUTPUT\X64\LongJump.iii
> c:\Env\OvmfBuild\Build\OvmfX64\RELEASE_VS2017\X64\MdePkg\Library\BaseLib\BaseLib\OUTPUT\X64\LongJump.i
>          "C:\Env\BuildTools\NASM\2.14.02\nasm"
> -IC:\Env\Workspace\edk2\MdePkg\Library\BaseLib\X64\ -Ox -f win64 -o
> c:\Env\OvmfBuild\Build\OvmfX64\RELEASE_VS2017\X64\MdePkg\Library\BaseLib\BaseLib\OUTPUT\X64\LongJump.obj
> c:\Env\OvmfBuild\Build\OvmfX64\RELEASE_VS2017\X64\MdePkg\Library\BaseLib\BaseLib\OUTPUT\X64\LongJump.iii
> c:\Env\OvmfBuild\Build\OvmfX64\RELEASE_VS2017\X64\MdePkg\Library\BaseLib\BaseLib\OUTPUT\X64\LongJump.iii:16:
> fatal: unable to open include file `Nasm.inc'
> NMAKE : fatal error U1077: 'C:\Env\BuildTools\NASM\2.14.02\nasm.EXE' :
> return code '0x1'
> 
> Missing Nasm.inc, which is present in MdePkg/Include/X64/Nasm.inc - but
> the weird thing is, adding extra NASM include paths was added recently
> (in BaseTools/Source/Python/AutoGen/GenMake.py there's a block that
> replaces NASM_INC with appropriate directories, build_rule.template
> should generate NASM calls with these, as of commit 7c3a1ef). After
> adding a few prints to the GenMake I can see it does find the proper
> include paths, but none of them land in the final NASM call in the
> Makefile, even though it knows about the Nasm.inc dependency - the
> NASM_INC macro seems to be completely missing from the generated Makefile:
> 
> $(OUTPUT_DIR)\X64\LongJump.obj : $(MAKE_FILE)
> $(OUTPUT_DIR)\X64\LongJump.obj : C:\Env\Workspace\edk2\MdePkg\Include\Base.h
> $(OUTPUT_DIR)\X64\LongJump.obj :
> C:\Env\Workspace\edk2\MdePkg\Include\X64\Nasm.inc
> $(OUTPUT_DIR)\X64\LongJump.obj : $(DEBUG_DIR)\AutoGen.h
> $(OUTPUT_DIR)\X64\LongJump.obj :
> C:\Env\Workspace\edk2\MdePkg\Include\X64\ProcessorBind.h
> $(OUTPUT_DIR)\X64\LongJump.obj :
> C:\Env\Workspace\edk2\MdePkg\Include\Library\PcdLib.h
> $(OUTPUT_DIR)\X64\LongJump.obj :
> C:\Env\Workspace\edk2\MdePkg\Library\BaseLib\X64\LongJump.nasm
>      "$(PP)" $(PP_FLAGS) $(INC)
> C:\Env\Workspace\edk2\MdePkg\Library\BaseLib\X64\LongJump.nasm >
> c:\Env\OvmfBuild\Build\OvmfX64\RELEASE_VS2017\X64\MdePkg\Library\BaseLib\BaseLib\OUTPUT\X64\LongJump.i
>      Trim --trim-long --source-code -o
> c:\Env\OvmfBuild\Build\OvmfX64\RELEASE_VS2017\X64\MdePkg\Library\BaseLib\BaseLib\OUTPUT\X64\LongJump.iii
> c:\Env\OvmfBuild\Build\OvmfX64\RELEASE_VS2017\X64\MdePkg\Library\BaseLib\BaseLib\OUTPUT\X64\LongJump.i
>      "$(NASM)" -IC:\Env\Workspace\edk2\MdePkg\Library\BaseLib\X64\
> $(NASM_FLAGS) -o
> c:\Env\OvmfBuild\Build\OvmfX64\RELEASE_VS2017\X64\MdePkg\Library\BaseLib\BaseLib\OUTPUT\X64\LongJump.obj
> c:\Env\OvmfBuild\Build\OvmfX64\RELEASE_VS2017\X64\MdePkg\Library\BaseLib\BaseLib\OUTPUT\X64\LongJump.iii
> 
> The nasty workaround is to copy Nasm.inc into MdePkg/Library/BaseLib/X64
> and everything builds just fine after that.
> 
> Another interesting thing is that if I don't use WORKSPACE and
> PACKAGES_PATH (both contain no spaces and are on local file systems) but
> rather build everything directly in the EDK2 directory, OVMF also builds
> fine (Makefile contains the NASM_INC definition which is included in
> NASM calls). Have anyone also hit this issue?
> 
> Thanks, Richard
> 
> 
>