From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=40.92.71.33; helo=eur03-db5-obe.outbound.protection.outlook.com; envelope-from=minnowware@outlook.com; receiver=edk2-devel@lists.01.org Received: from EUR03-DB5-obe.outbound.protection.outlook.com (mail-oln040092071033.outbound.protection.outlook.com [40.92.71.33]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 2DDD9211EC690 for ; Sat, 30 Mar 2019 01:14:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=nOCXuygH7e0bFiSMHFco1joD0F4WAJV6T8Mix5Gx3EM=; b=kXXCvUuo6FXip3RuMnnKs/uF8EWNvBNa/XwjsjTFzp+s0exL8Xi8VHpz3dQp+Lmwfuc9cElkDbpVw+5vlQvKshbt4OivbYUfNQSb5n3K3HWcJLGCnQPAKbLe7g+7N/fEDv/YgJo2pIMZZGHCC1oFwf9kIov4GkE6ogDhYkudMkyKYzeknacUVoBhPQ3815Pk9MZn+uWLJQgH1Ut4asLBiYp42HyRkJ6oBMx3lfEcBzt4m/SjRsFKdh3i3d2Uz74DvNSDH8J06lEzsGjpGmsTkWKpGGqlJEmitQ4uF7mkIA4L7/LUZqFQsPTmG9ThSCQVsKeecfNZLwb1X2kUiCpfFg== Received: from AM5EUR03FT038.eop-EUR03.prod.protection.outlook.com (10.152.16.57) by AM5EUR03HT200.eop-EUR03.prod.protection.outlook.com (10.152.16.224) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.1750.16; Sat, 30 Mar 2019 08:14:49 +0000 Received: from AM5PR04MB2993.eurprd04.prod.outlook.com (10.152.16.54) by AM5EUR03FT038.mail.protection.outlook.com (10.152.17.118) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.1750.16 via Frontend Transport; Sat, 30 Mar 2019 08:14:49 +0000 Received: from AM5PR04MB2993.eurprd04.prod.outlook.com ([fe80::6db5:ab5e:e97a:149c]) by AM5PR04MB2993.eurprd04.prod.outlook.com ([fe80::6db5:ab5e:e97a:149c%4]) with mapi id 15.20.1730.019; Sat, 30 Mar 2019 08:14:49 +0000 From: Minnow Ware To: "Ni, Ray" , "Feng, Bob C" , "edk2-devel@lists.01.org" CC: "Gao, Liming" Thread-Topic: [edk2] [Patch] BaseTools:Enable the /MP option of MSVC compiler Thread-Index: AQHU5is+4AmKiE/Yy0iTuo+U02jIIKYjZ0uAgABl280= Date: Sat, 30 Mar 2019 08:14:49 +0000 Message-ID: References: <20190329123007.52756-1-bob.c.feng@intel.com>, <734D49CCEBEEF84792F5B80ED585239D5C0A498B@SHSMSX104.ccr.corp.intel.com> In-Reply-To: <734D49CCEBEEF84792F5B80ED585239D5C0A498B@SHSMSX104.ccr.corp.intel.com> Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-incomingtopheadermarker: OriginalChecksum:9E969C970A4D518450A71213F127636E15E0F6B4D78E4D0136D85F1A29889DC5; UpperCasedChecksum:ECE3940C133109B5F2B585AEB00D0C0863F373E046D14EECF6EAE89C4340E7F0; SizeAsReceived:7030; Count:45 x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [yLj3bGA7P8V8ee5/ID3+bNDcLYhw5rT0] x-ms-publictraffictype: Email x-incomingheadercount: 45 x-eopattributedmessage: 0 x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(20181119110)(201702061078)(5061506573)(5061507331)(1603103135)(2017031320274)(9123020)(9118020)(2017031322404)(2017031324274)(2017031323274)(1601125500)(1603101475)(1701031045); SRVR:AM5EUR03HT200; x-ms-traffictypediagnostic: AM5EUR03HT200: x-microsoft-antispam-message-info: 4itnfb6qaiCAYTCiZRJpF0vzcUNxc6LYNefy78sARM/VPX745VhsryAeN0KIWpWJ MIME-Version: 1.0 X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: e5188a83-453b-49ae-91a7-08d6b4e7c78f X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-originalarrivaltime: 30 Mar 2019 08:14:49.7123 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Internet X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5EUR03HT200 X-Content-Filtered-By: Mailman/MimeDel 2.1.29 Subject: Re: [Patch] BaseTools:Enable the /MP option of MSVC compiler X-BeenThere: edk2-devel@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: EDK II Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Mar 2019 08:14:52 -0000 Content-Language: en-US Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable It would also be good, to add a REBUILD target to the makefiles, that forces the compiler to translate all C-files from a component at once, w/o dependency check e.g.: REBUILD: cl /c [additional switches] file1.c file2.c file3.c file4.c so that the compiler itself is invoked only once per component. This is possible for Microsoft, Intel and GNU C-compiler. Best regards, Kilian https://github.com/MinnowWare ________________________________ From: edk2-devel on behalf of Ni, Ray Sent: Saturday, March 30, 2019 2:42:21 AM To: Feng, Bob C; edk2-devel@lists.01.org Cc: Gao, Liming Subject: Re: [edk2] [Patch] BaseTools:Enable the /MP option of MSVC compile= r https://devblogs.microsoft.com/cppblog/recommendations-to-speed-c-builds-in= -visual-studio/#_PCH Bob, Per above article, It seems /Gm needs to be removed when /MP is used. > -----Original Message----- > From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of Fe= ng, > Bob C > Sent: Friday, March 29, 2019 8:30 PM > To: edk2-devel@lists.01.org > Cc: Gao, Liming > Subject: [edk2] [Patch] BaseTools:Enable the /MP option of MSVC compiler > > From: Zhiju Fan > > https://bugzilla.tianocore.org/show_bug.cgi?id=3D1672 > The /MP option of MSVC compiler can reduce the total time to compile the > source files on the command line. > > This patch is going to enable this MSVC option in BaseTools. > > Cc: Bob Feng > Cc: Liming Gao > Contributed-under: TianoCore Contribution Agreement 1.1 > Signed-off-by: Zhiju.Fan > --- > BaseTools/Conf/build_rule.template | 2 +- > BaseTools/Source/Python/AutoGen/GenMake.py | 81 ++++++++++++++++++++- > - > 2 files changed, 75 insertions(+), 8 deletions(-) > > diff --git a/BaseTools/Conf/build_rule.template > b/BaseTools/Conf/build_rule.template > index e56b1d9c59..e7d736740f 100755 > --- a/BaseTools/Conf/build_rule.template > +++ b/BaseTools/Conf/build_rule.template > @@ -127,11 +127,11 @@ > > > $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.obj > > > - "$(CC)" /Fo${dst} $(CC_FLAGS) $(INC) ${src} > + "$(CC)" /MP7 /Fo${d_path}\ $(CC_FLAGS) $(INC) ${src} > > > # For RVCTCYGWIN CC_FLAGS must be first to work around pathing i= ssues > "$(CC)" $(CC_FLAGS) -c -o ${dst} $(INC) ${src} > > diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py > b/BaseTools/Source/Python/AutoGen/GenMake.py > index b441817b52..04951346ad 100644 > --- a/BaseTools/Source/Python/AutoGen/GenMake.py > +++ b/BaseTools/Source/Python/AutoGen/GenMake.py > @@ -433,11 +433,11 @@ cleanlib: > self.BuildTargetList =3D [] # [target string] > self.PendingBuildTargetList =3D [] # [FileBuildRule objects] > self.CommonFileDependency =3D [] > self.FileListMacros =3D {} > self.ListFileMacros =3D {} > - > + self.ObjTargetDict =3D {} > self.FileCache =3D {} > self.LibraryBuildCommandList =3D [] > self.LibraryFileList =3D [] > self.LibraryMakefileList =3D [] > self.LibraryBuildDirectoryList =3D [] @@ -616,10 +616,15 @@ clea= nlib: > ListFileName, > "\n".join(self.ListFileMacros[ListFileMacro]), > False > ) > > + # Generate objlist used to create .obj file > + for Type in self.ObjTargetDict: > + NewLine =3D ' '.join(list(self.ObjTargetDict[Type])) > + FileMacroList.append("OBJLIST_%s =3D %s" % > + (list(self.ObjTargetDict.keys()).index(Type), NewLine)) > + > BcTargetList =3D [] > > MakefileName =3D self._FILE_NAME_[self._FileType] > LibraryMakeCommandList =3D [] > for D in self.LibraryBuildDirectoryList: > @@ -925,17 +930,22 @@ cleanlib: > # Extract common files list in the dependency files > # > for File in DepSet: > self.CommonFileDependency.append(self.PlaceMacro(File.Path, > self.Macros)) > > + CmdSumDict =3D {} > + CmdTargetDict =3D {} > + CmdCppDict =3D {} > + DependencyDict =3D FileDependencyDict.copy() > for File in FileDependencyDict: > # skip non-C files > if File.Ext not in [".c", ".C"] or File.Name =3D=3D "AutoGen= .c": > continue > NewDepSet =3D set(FileDependencyDict[File]) > NewDepSet -=3D DepSet > FileDependencyDict[File] =3D ["$(COMMON_DEPS)"] + list(NewDe= pSet) > + DependencyDict[File] =3D list(NewDepSet) > > # Convert target description object to target string in makefile > for Type in self._AutoGenObject.Targets: > for T in self._AutoGenObject.Targets[Type]: > # Generate related macros if needed @@ -943,15 +953,25 @= @ > cleanlib: > self.FileListMacros[T.FileListMacro] =3D [] > if T.GenListFile and T.ListFileMacro not in self.ListFil= eMacros: > self.ListFileMacros[T.ListFileMacro] =3D [] > if T.GenIncListFile and T.IncListFileMacro not in self.L= istFileMacros: > self.ListFileMacros[T.IncListFileMacro] =3D [] > + if self._AutoGenObject.BuildRuleFamily =3D=3D TAB_COMPIL= ER_MSFT and > Type =3D=3D TAB_C_CODE_FILE: > + NewFile =3D self.PlaceMacro(str(T), self.Macros) > + if self.ObjTargetDict.get(T.Target.SubDir): > + self.ObjTargetDict[T.Target.SubDir].add(NewFile) > + else: > + self.ObjTargetDict[T.Target.SubDir] =3D set() > + > + self.ObjTargetDict[T.Target.SubDir].add(NewFile) > > Deps =3D [] > + CCodeDeps =3D [] > # Add force-dependencies > for Dep in T.Dependencies: > Deps.append(self.PlaceMacro(str(Dep), self.Macros)) > + if Dep !=3D '$(MAKE_FILE)': > + CCodeDeps.append(self.PlaceMacro(str(Dep), > + self.Macros)) > # Add inclusion-dependencies > if len(T.Inputs) =3D=3D 1 and T.Inputs[0] in FileDepende= ncyDict: > for F in FileDependencyDict[T.Inputs[0]]: > Deps.append(self.PlaceMacro(str(F), self.Macros)= ) > # Add source-dependencies @@ -971,16 +991,63 @@ cleanlib= : > if T.GenFileListMacro: > Deps.append("$(%s)" % T.FileListMacro) > if Type in [TAB_OBJECT_FILE, TAB_STATIC_LIBRARY]: > Deps.append("$(%s)" % T.ListFileMacro) > > - TargetDict =3D { > - "target" : self.PlaceMacro(T.Target.Path, self.= Macros), > - "cmd" : "\n\t".join(T.Commands), > - "deps" : Deps > - } > - > self.BuildTargetList.append(self._BUILD_TARGET_TEMPLATE.Replace(TargetDic > t)) > + if self._AutoGenObject.BuildRuleFamily =3D=3D TAB_COMPIL= ER_MSFT and > Type =3D=3D TAB_C_CODE_FILE: > + T, CmdTarget, CmdTargetDict, CmdCppDict =3D self.Par= serCCodeFile(T, > Type, CmdSumDict, CmdTargetDict, CmdCppDict, DependencyDict) > + TargetDict =3D {"target": self.PlaceMacro(T.Target.P= ath, self.Macros), > "cmd": "\n\t".join(T.Commands),"deps": CCodeDeps} > + CmdLine =3D > self._BUILD_TARGET_TEMPLATE.Replace(TargetDict).rstrip().replace('\t$(OBJ= LIS > T', '$(OBJLIST') > + if T.Commands: > + CmdLine =3D '%s%s' %(CmdLine, TAB_LINE_BREAK) > + if CCodeDeps or CmdLine: > + self.BuildTargetList.append(CmdLine) > + else: > + TargetDict =3D {"target": self.PlaceMacro(T.Target.P= ath, self.Macros), > "cmd": "\n\t".join(T.Commands),"deps": Deps} > + > + self.BuildTargetList.append(self._BUILD_TARGET_TEMPLATE.Replace(Target > + Dict)) > + > + def ParserCCodeFile(self, T, Type, CmdSumDict, CmdTargetDict, CmdCpp= Dict, > DependencyDict): > + if not CmdSumDict: > + for item in self._AutoGenObject.Targets[Type]: > + CmdSumDict[item.Target.SubDir] =3D item.Target.BaseName > + for CppPath in item.Inputs: > + Path =3D self.PlaceMacro(CppPath.Path, self.Macros) > + if CmdCppDict.get(item.Target.SubDir): > + CmdCppDict[item.Target.SubDir].append(Path) > + else: > + CmdCppDict[item.Target.SubDir] =3D ['$(MAKE_FILE= )', Path] > + if CppPath.Path in DependencyDict: > + for Temp in DependencyDict[CppPath.Path]: > + Path =3D self.PlaceMacro(Temp.Path, self.Mac= ros) > + if Path not in (self.CommonFileDependency + > CmdCppDict[item.Target.SubDir]): > + CmdCppDict[item.Target.SubDir].append(Pa= th) > + if T.Commands: > + CommandList =3D T.Commands[:] > + for Item in CommandList[:]: > + SingleCommandList =3D Item.split() > + if len(SingleCommandList) > 0 and '$(CC)' in SingleComma= ndList[0]: > + for Temp in SingleCommandList: > + if Temp.startswith('/Fo'): > + CmdSign =3D '%s%s' % (Temp.rsplit(TAB_SLASH,= 1)[0], TAB_SLASH) > + break > + else: continue > + if CmdSign not in list(CmdTargetDict.keys()): > + CmdLine =3D Item.replace(Temp, CmdSign) > + if GlobalData.gOptions.ThreadNumber: > + CmdLine =3D CmdLine.replace('/MP7', '%s%d' %= ('/MP', > GlobalData.gOptions.ThreadNumber)) > + CmdTargetDict[CmdSign] =3D CmdLine > + else: > + CmdTargetDict[CmdSign] =3D "%s %s" % (CmdTargetD= ict[CmdSign], > SingleCommandList[-1]) > + Index =3D CommandList.index(Item) > + CommandList.pop(Index) > + if SingleCommandList[-1].endswith("%s%s.c" % (TAB_SL= ASH, > CmdSumDict[CmdSign.lstrip('/Fo').rsplit(TAB_SLASH, 1)[0]])): > + Cpplist =3D CmdCppDict[T.Target.SubDir] > + Cpplist.insert(0, '$(OBJLIST_%d): $(COMMON_DEPS)= ' % > list(self.ObjTargetDict.keys()).index(T.Target.SubDir)) > + T.Commands[Index] =3D '%s\n\t%s' % (' \\\n\t'.jo= in(Cpplist), > CmdTargetDict[CmdSign]) > + else: > + T.Commands.pop(Index) > + return T, CmdSumDict, CmdTargetDict, CmdCppDict > > ## For creating makefile targets for dependent libraries > def ProcessDependentLibrary(self): > for LibraryAutoGen in self._AutoGenObject.LibraryAutoGenList: > if not LibraryAutoGen.IsBinaryModule: > -- > 2.20.1.windows.1 > > _______________________________________________ > edk2-devel mailing list > edk2-devel@lists.01.org > https://lists.01.org/mailman/listinfo/edk2-devel _______________________________________________ edk2-devel mailing list edk2-devel@lists.01.org https://lists.01.org/mailman/listinfo/edk2-devel