[edk2-devel] [PATCH v2 0/2] Update BaseTools NULL Include and Library Matching

[edk2-devel] [PATCH v2 0/2] Update BaseTools NULL Include and Library Matching

[Taylor Beebe]

v1:
  - Initial patch series
v2:
  - Simplified the check for if the currently evaluated inf is a module or library.
  - Added a commit to use stronger matching of NULL includes (check for pattern "NULL<n>")

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>

Taylor Beebe (2):
  BaseTools: Don't Recurse NULL Includes Not Linked to Module
  BaseTools: Use Stronger Matching for NULL Linked Libraries

 BaseTools/Source/Python/GenFds/FfsInfStatement.py    | 4 ++--
 BaseTools/Source/Python/Workspace/WorkspaceCommon.py | 8 +++++---
 2 files changed, 7 insertions(+), 5 deletions(-)

-- 
2.40.1.vfs.0.0



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#117061): https://edk2.groups.io/g/devel/message/117061
Mute This Topic: https://groups.io/mt/105092033/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

[edk2-devel] [PATCH v2 1/2] BaseTools: Don't Recurse NULL Includes Not Linked to Module

[Taylor Beebe]

When collecting the required library instances for modules and
libraries, included libraries will be recursed to ensure the module is
built with all the libraries directly linked to it and indirectly
linked to it via included libraries.

Using the following scenario as an example:

[LibraryClasses.common.DXE_CORE]
NULL|Path/To/Library1.inf // Includes DebugLib

[LibraryClasses.common.DXE_DRIVER]
NULL|Path/To/Library2.inf // Includes DebugLib

[LibraryClasses.common.DXE_CORE, LibraryClasses.common.DXE_DRIVER]
DebugLib|MdePkg/Library/BaseDebugLibSerialPort/BaseDebugLibSerialPort.inf

[Components]
MdeModulePkg/Core/Dxe/DxeMain.inf // Includes DebugLib

The DXE_CORE NULL library will be assigned a fake library class like
NULL1 and the DXE_DRIVER will be assigned NULL2. The recursion logic
will see NULL1 as a directly linked and will add an instance of it to
the list of libraries which need to be included in the module. When
DebugLib is evaluated, the recursion logic will add the libraries
DebugLib depends on to the queue which includes both NULL1 and NULL2.
When NULL2 is unqueued, an instance of it will also be added to the
list of libraries needed to build DxeMain which now means that both
NULL1 and NULL2 have been linked.

NULL includes outside of module overrides are not supported according
to the spec, but we do it anyways so this seems like a case which
should be fixed. This change updates the recursion logic to skip
evaluating NULL libraries unless they are linked directly to the
module/library being evaluated.

Signed-off-by: Taylor Beebe <taylor.d.beebe@gmail.com>
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
---
 BaseTools/Source/Python/Workspace/WorkspaceCommon.py | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/BaseTools/Source/Python/Workspace/WorkspaceCommon.py b/BaseTools/Source/Python/Workspace/WorkspaceCommon.py
index 9e506fc646..8bb6553c6f 100644
--- a/BaseTools/Source/Python/Workspace/WorkspaceCommon.py
+++ b/BaseTools/Source/Python/Workspace/WorkspaceCommon.py
@@ -123,6 +123,8 @@ def GetModuleLibInstances(Module, Platform, BuildDatabase, Arch, Target, Toolcha
     while len(LibraryConsumerList) > 0:
         M = LibraryConsumerList.pop()
         for LibraryClassName in M.LibraryClasses:
+            if LibraryClassName.startswith("NULL") and bool(M.LibraryClass):
+                continue
             if LibraryClassName not in LibraryInstance:
                 # override library instance for this module
                 LibraryPath = Platform.Modules[str(Module)].LibraryClasses.get(LibraryClassName,Platform.LibraryClasses[LibraryClassName, ModuleType])
-- 
2.40.1.vfs.0.0



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#117062): https://edk2.groups.io/g/devel/message/117062
Mute This Topic: https://groups.io/mt/105092034/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

[edk2-devel] [PATCH v2 2/2] BaseTools: Use Stronger Matching for NULL Linked Libraries

[Taylor Beebe]

To prevent the possibility that a library with a name like
NULLTestLib is interpreted as a NULL linked library, use
more explicit pattern matching to ensure that the library
name follows the pattern NULL%d.

Signed-off-by: Taylor Beebe <taylor.d.beebe@gmail.com>
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
---
 BaseTools/Source/Python/GenFds/FfsInfStatement.py    | 4 ++--
 BaseTools/Source/Python/Workspace/WorkspaceCommon.py | 8 ++++----
 2 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/BaseTools/Source/Python/GenFds/FfsInfStatement.py b/BaseTools/Source/Python/GenFds/FfsInfStatement.py
index 6550d939d4..ec9713484e 100644
--- a/BaseTools/Source/Python/GenFds/FfsInfStatement.py
+++ b/BaseTools/Source/Python/GenFds/FfsInfStatement.py
@@ -93,7 +93,7 @@ class FfsInfStatement(FfsInfStatementClassObject):
 
                 if ModuleType != SUP_MODULE_USER_DEFINED and ModuleType != SUP_MODULE_HOST_APPLICATION:
                     for LibraryClass in PlatformDataBase.LibraryClasses.GetKeys():
-                        if LibraryClass.startswith("NULL") and PlatformDataBase.LibraryClasses[LibraryClass, ModuleType]:
+                        if LibraryClass.startswith("NULL") and LibraryClass[4:].isdigit() and PlatformDataBase.LibraryClasses[LibraryClass, ModuleType]:
                             self.InfModule.LibraryClasses[LibraryClass] = PlatformDataBase.LibraryClasses[LibraryClass, ModuleType]
 
                 StrModule = str(self.InfModule)
@@ -101,7 +101,7 @@ class FfsInfStatement(FfsInfStatementClassObject):
                 if StrModule in PlatformDataBase.Modules:
                     PlatformModule = PlatformDataBase.Modules[StrModule]
                     for LibraryClass in PlatformModule.LibraryClasses:
-                        if LibraryClass.startswith("NULL"):
+                        if LibraryClass.startswith("NULL") and LibraryClass[4:].isdigit():
                             self.InfModule.LibraryClasses[LibraryClass] = PlatformModule.LibraryClasses[LibraryClass]
 
                 DependencyList = [self.InfModule]
diff --git a/BaseTools/Source/Python/Workspace/WorkspaceCommon.py b/BaseTools/Source/Python/Workspace/WorkspaceCommon.py
index 8bb6553c6f..c3b26b370a 100644
--- a/BaseTools/Source/Python/Workspace/WorkspaceCommon.py
+++ b/BaseTools/Source/Python/Workspace/WorkspaceCommon.py
@@ -102,12 +102,12 @@ def GetModuleLibInstances(Module, Platform, BuildDatabase, Arch, Target, Toolcha
     #
     if Module.ModuleType != SUP_MODULE_USER_DEFINED:
         for LibraryClass in Platform.LibraryClasses.GetKeys():
-            if LibraryClass.startswith("NULL") and Platform.LibraryClasses[LibraryClass, Module.ModuleType]:
+            if LibraryClass.startswith("NULL") and LibraryClass[4:].isdigit() and Platform.LibraryClasses[LibraryClass, Module.ModuleType]:
                 Module.LibraryClasses[LibraryClass] = Platform.LibraryClasses[LibraryClass, Module.ModuleType]
 
     # add forced library instances (specified in module overrides)
     for LibraryClass in Platform.Modules[str(Module)].LibraryClasses:
-        if LibraryClass.startswith("NULL"):
+        if LibraryClass.startswith("NULL") and LibraryClass[4:].isdigit():
             Module.LibraryClasses[LibraryClass] = Platform.Modules[str(Module)].LibraryClasses[LibraryClass]
 
     # EdkII module
@@ -123,7 +123,7 @@ def GetModuleLibInstances(Module, Platform, BuildDatabase, Arch, Target, Toolcha
     while len(LibraryConsumerList) > 0:
         M = LibraryConsumerList.pop()
         for LibraryClassName in M.LibraryClasses:
-            if LibraryClassName.startswith("NULL") and bool(M.LibraryClass):
+            if LibraryClassName.startswith("NULL") and LibraryClass[4:].isdigit() and bool(M.LibraryClass):
                 continue
             if LibraryClassName not in LibraryInstance:
                 # override library instance for this module
@@ -141,7 +141,7 @@ def GetModuleLibInstances(Module, Platform, BuildDatabase, Arch, Target, Toolcha
 
                 LibraryModule = BuildDatabase[LibraryPath, Arch, Target, Toolchain]
                 # for those forced library instance (NULL library), add a fake library class
-                if LibraryClassName.startswith("NULL"):
+                if LibraryClassName.startswith("NULL") and LibraryClass[4:].isdigit():
                     LibraryModule.LibraryClass.append(LibraryClassObject(LibraryClassName, [ModuleType]))
                 elif LibraryModule.LibraryClass is None \
                      or len(LibraryModule.LibraryClass) == 0 \
-- 
2.40.1.vfs.0.0



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#117063): https://edk2.groups.io/g/devel/message/117063
Mute This Topic: https://groups.io/mt/105092035/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

回复: [edk2-devel] [PATCH v2 0/2] Update BaseTools NULL Include and Library Matching

[gaoliming via groups.io]

Can you submit a Bugzilla for this problem? 

Can you give one example to explain the incorrect usage?

Thanks
Liming
> -----邮件原件-----
> 发件人: devel@edk2.groups.io <devel@edk2.groups.io> 代表 Taylor Beebe
> 发送时间: 2024年3月23日 3:19
> 收件人: devel@edk2.groups.io
> 抄送: Rebecca Cran <rebecca@bsdio.com>; Liming Gao
> <gaoliming@byosoft.com.cn>; Bob Feng <bob.c.feng@intel.com>; Yuwei Chen
> <yuwei.chen@intel.com>
> 主题: [edk2-devel] [PATCH v2 0/2] Update BaseTools NULL Include and
> Library Matching
> 
> v1:
>   - Initial patch series
> v2:
>   - Simplified the check for if the currently evaluated inf is a module or
> library.
>   - Added a commit to use stronger matching of NULL includes (check for
> pattern "NULL<n>")
> 
> Cc: Rebecca Cran <rebecca@bsdio.com>
> Cc: Liming Gao <gaoliming@byosoft.com.cn>
> Cc: Bob Feng <bob.c.feng@intel.com>
> Cc: Yuwei Chen <yuwei.chen@intel.com>
> 
> Taylor Beebe (2):
>   BaseTools: Don't Recurse NULL Includes Not Linked to Module
>   BaseTools: Use Stronger Matching for NULL Linked Libraries
> 
>  BaseTools/Source/Python/GenFds/FfsInfStatement.py    | 4 ++--
>  BaseTools/Source/Python/Workspace/WorkspaceCommon.py | 8 +++++---
>  2 files changed, 7 insertions(+), 5 deletions(-)
> 
> --
> 2.40.1.vfs.0.0
> 
> 
> 
> 
> 





-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#117184): https://edk2.groups.io/g/devel/message/117184
Mute This Topic: https://groups.io/mt/105189358/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: 回复: [edk2-devel] [PATCH v2 0/2] Update BaseTools NULL Include and Library Matching

[Taylor Beebe]

Hi Liming,

Each patch describes the issue being fixed with an example.

I created a bugzilla and assigned it to me for Patch 1: 
https://bugzilla.tianocore.org/show_bug.cgi?id=4744

GitHub PR: https://github.com/tianocore/edk2/pull/5365

Thanks :)

-Taylor

On 3/27/24 6:00 PM, gaoliming wrote:
> Can you submit a Bugzilla for this problem?
>
> Can you give one example to explain the incorrect usage?
>
> Thanks
> Liming
>> -----邮件原件-----
>> 发件人: devel@edk2.groups.io <devel@edk2.groups.io> 代表 Taylor Beebe
>> 发送时间: 2024年3月23日 3:19
>> 收件人: devel@edk2.groups.io
>> 抄送: Rebecca Cran <rebecca@bsdio.com>; Liming Gao
>> <gaoliming@byosoft.com.cn>; Bob Feng <bob.c.feng@intel.com>; Yuwei Chen
>> <yuwei.chen@intel.com>
>> 主题: [edk2-devel] [PATCH v2 0/2] Update BaseTools NULL Include and
>> Library Matching
>>
>> v1:
>>    - Initial patch series
>> v2:
>>    - Simplified the check for if the currently evaluated inf is a module or
>> library.
>>    - Added a commit to use stronger matching of NULL includes (check for
>> pattern "NULL<n>")
>>
>> Cc: Rebecca Cran <rebecca@bsdio.com>
>> Cc: Liming Gao <gaoliming@byosoft.com.cn>
>> Cc: Bob Feng <bob.c.feng@intel.com>
>> Cc: Yuwei Chen <yuwei.chen@intel.com>
>>
>> Taylor Beebe (2):
>>    BaseTools: Don't Recurse NULL Includes Not Linked to Module
>>    BaseTools: Use Stronger Matching for NULL Linked Libraries
>>
>>   BaseTools/Source/Python/GenFds/FfsInfStatement.py    | 4 ++--
>>   BaseTools/Source/Python/Workspace/WorkspaceCommon.py | 8 +++++---
>>   2 files changed, 7 insertions(+), 5 deletions(-)
>>
>> --
>> 2.40.1.vfs.0.0
>>
>>
>>
>> 
>>
>
>


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#117218): https://edk2.groups.io/g/devel/message/117218
Mute This Topic: https://groups.io/mt/105189358/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

回复: 回复: [edk2-devel] [PATCH v2 0/2] Update BaseTools NULL Include and Library Matching

[gaoliming via groups.io]

Taylor:
  Thanks for you detail information. I understand this problem. I agree your fix. Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>

Thanks
Liming
> -----邮件原件-----
> 发件人: devel@edk2.groups.io <devel@edk2.groups.io> 代表 Taylor Beebe
> 发送时间: 2024年3月29日 8:28
> 收件人: gaoliming <gaoliming@byosoft.com.cn>; devel@edk2.groups.io
> 抄送: 'Rebecca Cran' <rebecca@bsdio.com>; 'Bob Feng'
> <bob.c.feng@intel.com>; 'Yuwei Chen' <yuwei.chen@intel.com>
> 主题: Re: 回复: [edk2-devel] [PATCH v2 0/2] Update BaseTools NULL Include
> and Library Matching
> 
> Hi Liming,
> 
> Each patch describes the issue being fixed with an example.
> 
> I created a bugzilla and assigned it to me for Patch 1:
> https://bugzilla.tianocore.org/show_bug.cgi?id=4744
> 
> GitHub PR: https://github.com/tianocore/edk2/pull/5365
> 
> Thanks :)
> 
> -Taylor
> 
> On 3/27/24 6:00 PM, gaoliming wrote:
> > Can you submit a Bugzilla for this problem?
> >
> > Can you give one example to explain the incorrect usage?
> >
> > Thanks
> > Liming
> >> -----邮件原件-----
> >> 发件人: devel@edk2.groups.io <devel@edk2.groups.io> 代表 Taylor
> Beebe
> >> 发送时间: 2024年3月23日 3:19
> >> 收件人: devel@edk2.groups.io
> >> 抄送: Rebecca Cran <rebecca@bsdio.com>; Liming Gao
> >> <gaoliming@byosoft.com.cn>; Bob Feng <bob.c.feng@intel.com>; Yuwei
> Chen
> >> <yuwei.chen@intel.com>
> >> 主题: [edk2-devel] [PATCH v2 0/2] Update BaseTools NULL Include and
> >> Library Matching
> >>
> >> v1:
> >>    - Initial patch series
> >> v2:
> >>    - Simplified the check for if the currently evaluated inf is a module or
> >> library.
> >>    - Added a commit to use stronger matching of NULL includes (check
> for
> >> pattern "NULL<n>")
> >>
> >> Cc: Rebecca Cran <rebecca@bsdio.com>
> >> Cc: Liming Gao <gaoliming@byosoft.com.cn>
> >> Cc: Bob Feng <bob.c.feng@intel.com>
> >> Cc: Yuwei Chen <yuwei.chen@intel.com>
> >>
> >> Taylor Beebe (2):
> >>    BaseTools: Don't Recurse NULL Includes Not Linked to Module
> >>    BaseTools: Use Stronger Matching for NULL Linked Libraries
> >>
> >>   BaseTools/Source/Python/GenFds/FfsInfStatement.py    | 4 ++--
> >>   BaseTools/Source/Python/Workspace/WorkspaceCommon.py | 8
> +++++---
> >>   2 files changed, 7 insertions(+), 5 deletions(-)
> >>
> >> --
> >> 2.40.1.vfs.0.0
> >>
> >>
> >>
> >>
> >>
> >
> >
> 
> 
> 
> 





-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#117304): https://edk2.groups.io/g/devel/message/117304
Mute This Topic: https://groups.io/mt/105280288/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: 回复: 回复: [edk2-devel] [PATCH v2 0/2] Update BaseTools NULL Include and Library Matching

[Taylor Beebe]

Hi Liming,

I made a mistake in patch 2 of the v2 series. In v3, lines 126 and 144 
of WorkspaceCommon.py update the check to:

`LibraryClassName[4:].isdigit()`

instead of

`LibraryClass[4:].isdigit()`

Can you re-review with this change?

-Taylor

On 4/1/2024 6:37 PM, gaoliming via groups.io wrote:
> Taylor:
>    Thanks for you detail information. I understand this problem. I agree your fix. Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
>
> Thanks
> Liming
>


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#117558): https://edk2.groups.io/g/devel/message/117558
Mute This Topic: https://groups.io/mt/105280288/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

回复: 回复: 回复: [edk2-devel] [PATCH v2 0/2] Update BaseTools NULL Include and Library Matching

[gaoliming via groups.io]

Thanks for your update. Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>

> -----邮件原件-----
> 发件人: devel@edk2.groups.io <devel@edk2.groups.io> 代表 Taylor Beebe
> 发送时间: 2024年4月10日 3:43
> 收件人: devel@edk2.groups.io; gaoliming@byosoft.com.cn
> 抄送: 'Rebecca Cran' <rebecca@bsdio.com>; 'Bob Feng'
> <bob.c.feng@intel.com>; 'Yuwei Chen' <yuwei.chen@intel.com>
> 主题: Re: 回复: 回复: [edk2-devel] [PATCH v2 0/2] Update BaseTools NULL
> Include and Library Matching
> 
> Hi Liming,
> 
> I made a mistake in patch 2 of the v2 series. In v3, lines 126 and 144
> of WorkspaceCommon.py update the check to:
> 
> `LibraryClassName[4:].isdigit()`
> 
> instead of
> 
> `LibraryClass[4:].isdigit()`
> 
> Can you re-review with this change?
> 
> -Taylor
> 
> On 4/1/2024 6:37 PM, gaoliming via groups.io wrote:
> > Taylor:
> >    Thanks for you detail information. I understand this problem. I agree
> your fix. Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
> >
> > Thanks
> > Liming
> >
> 
> 
> 
> 





-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#117747): https://edk2.groups.io/g/devel/message/117747
Mute This Topic: https://groups.io/mt/105529073/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-