Re: [PATCH] [MdePkg ] New Debug Message Levels

["Kinney, Michael D" <michael.d.kinney@intel.com>]

Felix,

It is the use case for DEBUG_NET|DEBUG_INFO that I am concerned about.

I agree that setting either or both of these bits activates the message.

We already support setting the mask at the module level, so we have control
over verbosity with use of ERROR, WARN, INFO, VERBOSE in each module.  However, 
setting the mask for every module can be time consuming, so it is easier 
to set the global mask value at the top of the DSC file.  In this example, 
if only NET messages are needed, then the NET bit is set in the mask.  
However, you will get all messages that are ORed with NET, so there is no 
ability to enable/disable ERROR, WARN, INFO, VERBOSE within the NET subsystem.

I am concerned that adding more subsystems without the ability to adjust 
these other levels will become an issue because we are running out of bits 
in the mask.  What if there is a request for NET_1, NET_2, ... NET_n for 
verbose levels within a specific subsystem.

I agree that one solution is to have a different DebugLib implementation
that interprets the bits differently, but that could make implementing
DEBUG() statements more difficult because developers will see multiple
interpretations of the mask.

I would prefer to look into ways we can enhance the current DebugLib 
to provide control over verbosity within a subsystem.

One proposal is to treat some of the bits as a verbosity mask bits and 
others as subsystem mask bits and have well defined rules for the behavior
when bits are mixed together.

We could view the following bits as verbosity settings:
* DEBUG_INIT
* DEBUG_INFO
* DEBUG_VERBOSE

The following bits are critical error/warn messages that are
displayed no matter what verbosity bits are mixed with them
* DEBUG_ERROR
* DEBUG_WARN 

And the rest of the bits are subsystem specific messages that
support being mixed with a verbosity bits:

UEFI/PI Core Subsystems
========================
DEBUG_LOAD
DEBUG_POOL
DEBUG_PAGE
DEBUG_GCD
DEBUG_DISPATCH
DEBUG_EVENT
DEBUG_VARIABLE
DEBUG_BM

CPU Subsystems
==============
DEBUG_CACHE

I/O Subsystems
==============
DEBUG_BLKIO     
DEBUG_FS        
DEBUG_NET       
DEBUG_UNDI      
DEBUG_LOADFILE  

Would be a small change to the behavior of current DebugLib.

Then add your new subsystem bits.

And consider adding a couple extra verbosity levels such as
VERBOSE2, VERBOSE3, VERBOSE4, so all subsystems can have
multiple levels of verbosity,

Best regards,

Mike

> -----Original Message-----
> From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of Felix Poludov
> Sent: Monday, September 26, 2016 2:06 PM
> To: Kinney, Michael D <michael.d.kinney@intel.com>; Gao, Liming
> <liming.gao@intel.com>; edk2-devel@lists.01.org
> Subject: Re: [edk2] [PATCH] [MdePkg ] New Debug Message Levels
> 
> Mike,
> 
> The goal is to provide a higher level of granularity in controlling debug messages.
> Or to put it in different terms, improve developer's ability to reduce amount of
> "noise" in the debug log.
> For example, if I'm debugging a chipset-specific issue, I'm not necessarily
> interested in debug messages from the generic modules and vice versa.
> 
> As far as combing the definitions, I think we can leave it up to individual
> developers.
> I know that with the current implementation DebugPrint statement is printed if it
> least one of the used levels is enabled.
> For example, DEBUG_NET|DEBUG_INFO is printed when either DEBUG_NET, or DEBUG_INFO, or
> both are enabled.
> I think this is fine.
> Moreover, it's pretty easy to customize the default policy by using a custom DebugLib
> and/or DebugPrintErrorLevel instances.
> 
> 
> -----Original Message-----
> From: Kinney, Michael D [mailto:michael.d.kinney@intel.com]
> Sent: Wednesday, September 21, 2016 2:10 AM
> To: Felix Poludov; Gao, Liming; edk2-devel@lists.01.org
> Subject: RE: [PATCH] [MdePkg ] New Debug Message Levels
> 
> Felix,
> 
> Thanks for the quick update.
> 
> Quick question on usage model.  Do you expect to be able to mix
> an I/O subsystem bit with ERROR, WARN, INFO, or VERBOSE bits?
> 
> Unfortunately, I do not think the current definition and implementation
> of the DebugLib instances support this type mixing with a result a
> developer may expect.
> 
> Can you summarize or provide some example use cases for these new bits?
> 
> Thanks,
> 
> Mike
> 
> 
> > -----Original Message-----
> > From: Felix Poludov [mailto:Felixp@ami.com]
> > Sent: Tuesday, September 20, 2016 12:29 PM
> > To: Kinney, Michael D <michael.d.kinney@intel.com>; Gao, Liming
> > <liming.gao@intel.com>; edk2-devel@lists.01.org
> > Subject: RE: [PATCH] [MdePkg ] New Debug Message Levels
> >
> > Mike and Liming,
> >
> > Thanks for reviewing the patch.
> > I implemented all the requested modifications:
> >  - EFI_D_ definitions are removed
> >  - DEBUG_CSM is removed
> >  - Validation expression in MdePkg.dec is updated
> >  - MdePkg.uni is updated
> >
> > Here is the updated patch
> > Contributed-under: TianoCore Contribution Agreement 1.0
> > Signed-off-by: Felix Polyudov <felixp@ami.com>
> > ==
> > diff --git a/MdePkg/Include/Library/DebugLib.h b/MdePkg/Include/Library/DebugLib.h
> > index 803872c..34ea29f 100644
> > --- a/MdePkg/Include/Library/DebugLib.h
> > +++ b/MdePkg/Include/Library/DebugLib.h
> > @@ -54,6 +54,11 @@ WITHOUT WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER
> EXPRESS
> > OR IMPLIED.
> >  #define DEBUG_CACHE     0x00200000  // Memory range cachability changes
> >  #define DEBUG_VERBOSE   0x00400000  // Detailed debug messages that may
> >                                      // significantly impact boot performance
> > +#define DEBUG_USB       0x00000200  // USB Driver
> > +#define DEBUG_PCI       0x00000800  // PCI Bus Driver
> > +#define DEBUG_CPU       0x00002000  // Processor Driver
> > +#define DEBUG_CHIPSET   0x00008000  // Chipset/SoC Drivers
> > +#define DEBUG_PLATFORM  0x00040000  // Platform/Board Drivers
> >  #define DEBUG_ERROR     0x80000000  // Error
> >
> >  //
> > diff --git a/MdePkg/MdePkg.dec b/MdePkg/MdePkg.dec
> > index 606e2f1..1747bb1 100644
> > --- a/MdePkg/MdePkg.dec
> > +++ b/MdePkg/MdePkg.dec
> > @@ -1883,11 +1883,16 @@
> >    #  BIT6  - Information message.<BR>
> >    #  BIT7  - Dispatcher message.<BR>
> >    #  BIT8  - Variable message.<BR>
> > +  #  BIT9  - USB Driver message.<BR>
> >    #  BIT10 - Boot Manager message.<BR>
> > +  #  BIT11 - PCI Bus Driver message.<BR>
> >    #  BIT12 - BlockIo Driver message.<BR>
> > +  #  BIT13 - Processor Driver message.<BR>
> >    #  BIT14 - Network Driver message.<BR>
> > +  #  BIT15 - Chipset/SoC Driver message.<BR>
> >    #  BIT16 - UNDI Driver message.<BR>
> >    #  BIT17 - LoadFile message.<BR>
> > +  #  BIT18 - Platform/Board Driver message.<BR>
> >    #  BIT19 - Event message.<BR>
> >    #  BIT20 - Global Coherency Database changes message.<BR>
> >    #  BIT21 - Memory range cachability changes message.<BR>
> > @@ -1951,18 +1956,23 @@
> >    #  BIT6  - Information message.<BR>
> >    #  BIT7  - Dispatcher message.<BR>
> >    #  BIT8  - Variable message.<BR>
> > +  #  BIT9  - USB Driver message.<BR>
> >    #  BIT10 - Boot Manager message.<BR>
> > +  #  BIT11 - PCI Bus Driver message.<BR>
> >    #  BIT12 - BlockIo Driver message.<BR>
> > +  #  BIT13 - Processor Driver message.<BR>
> >    #  BIT14 - Network Driver message.<BR>
> > +  #  BIT15 - Chipset/SoC Driver message.<BR>
> >    #  BIT16 - UNDI Driver message.<BR>
> >    #  BIT17 - LoadFile message.<BR>
> > +  #  BIT18 - Platform/Board Driver message.<BR>
> >    #  BIT19 - Event message.<BR>
> >    #  BIT20 - Global Coherency Database changes message.<BR>
> >    #  BIT21 - Memory range cachability changes message.<BR>
> >    #  BIT22 - Detailed debug message.<BR>
> >    #  BIT31 - Error message.<BR>
> >    # @Prompt Debug Message Print Level.
> > -  # @Expression  0x80000002 | (gEfiMdePkgTokenSpaceGuid.PcdDebugPrintErrorLevel &
> > 0x7F84AA00) == 0
> > +  # @Expression  0x80000002 | (gEfiMdePkgTokenSpaceGuid.PcdDebugPrintErrorLevel &
> > 0x7F800000) == 0
> >    gEfiMdePkgTokenSpaceGuid.PcdDebugPrintErrorLevel|0x80000000|UINT32|0x00000006
> >
> >    ## The mask is used to control ReportStatusCodeLib behavior.<BR><BR>
> > diff --git a/MdePkg/MdePkg.uni b/MdePkg/MdePkg.uni
> > index a110e45..4f3999d 100644
> > --- a/MdePkg/MdePkg.uni
> > +++ b/MdePkg/MdePkg.uni
> > @@ -196,11 +196,16 @@
> >
> > "BIT6  - Information message.<BR>\n"
> >
> > "BIT7  - Dispatcher message.<BR>\n"
> >
> > "BIT8  - Variable message.<BR>\n"
> > +
> > "BIT9  - USB Driver message.<BR>\n"
> >
> > "BIT10 - Boot Manager message.<BR>\n"
> > +
> > "BIT11 - PCI Bus Driver message.<BR>\n"
> >
> > "BIT12 - BlockIo Driver message.<BR>\n"
> > +
> > "BIT13 - Processor Driver message.<BR>\n"
> >
> > "BIT14 - Network Driver message.<BR>\n"
> > +
> > "BIT15 - Chipset/SoC Driver message.<BR>\n"
> >
> > "BIT16 - UNDI Driver message.<BR>\n"
> >
> > "BIT17 - LoadFile message.<BR>\n"
> > +
> > "BIT18 - Platform/Board Driver message.<BR>\n"
> >
> > "BIT19 - Event message.<BR>\n"
> >
> > "BIT20 - Global Coherency Database changes message.<BR>\n"
> >
> > "BIT21 - Memory range cacheability changes message.<BR>\n"
> > @@ -219,11 +224,16 @@
> >
> > "BIT6  - Information message.<BR>\n"
> >
> > "BIT7  - Dispatcher message.<BR>\n"
> >
> > "BIT8  - Variable message.<BR>\n"
> > +
> > "BIT9  - USB Driver message.<BR>\n"
> >
> > "BIT10 - Boot Manager message.<BR>\n"
> > +
> > "BIT11 - PCI Bus Driver message.<BR>\n"
> >
> > "BIT12 - BlockIo Driver message.<BR>\n"
> > +
> > "BIT13 - Processor Driver message.<BR>\n"
> >
> > "BIT14 - Network Driver message.<BR>\n"
> > +
> > "BIT15 - Chipset/SoC Driver message.<BR>\n"
> >
> > "BIT16 - UNDI Driver message.<BR>\n"
> >
> > "BIT17 - LoadFile message.<BR>\n"
> > +
> > "BIT18 - Platform/Board Driver message.<BR>\n"
> >
> > "BIT19 - Event message.<BR>\n"
> >
> > "BIT20 - Global Coherency Database changes message.<BR>\n"
> >
> > "BIT21 - Memory range cacheability changes message.<BR>\n"
> > ==
> >
> > -----Original Message-----
> > From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of Kinney,
> > Michael D
> > Sent: Tuesday, September 20, 2016 12:26 PM
> > To: Gao, Liming; Felix Poludov; edk2-devel@lists.01.org; Kinney, Michael D
> > Subject: Re: [edk2] [PATCH] [MdePkg ] New Debug Message Levels
> >
> > Felix,
> >
> > We should only add the DEBUG_ defines for new bit values.  We would like to retire
> > the use of EFI_D_ defines and encourage use of DEBUG_ defines.  The reason is that
> > DEBUG() macros apply to all execution phases and module types.
> >
> > Also, what is the use case for DEBUG_CSM.  I would image this would only be used
> for
> > CSM wrapper modules, and the exiting define values should be sufficient for that
> > specific use case.  That bit mask is starting to get full.
> >
> > Thanks,
> >
> > Mike
> >
> > > -----Original Message-----
> > > From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of Gao,
> Liming
> > > Sent: Tuesday, September 20, 2016 7:01 AM
> > > To: Felix Poludov <Felixp@ami.com>; edk2-devel@lists.01.org
> > > Subject: Re: [edk2] [PATCH] [MdePkg ] New Debug Message Levels
> > >
> > > Felix:
> > >   I have two comments.
> > > 1) Update @Expression  0x80000002 |
> > (gEfiMdePkgTokenSpaceGuid.PcdDebugPrintErrorLevel
> > > & 0x7F84AA00) == 0. 0x7F84AA00 should be updated to match the valid debug error
> > level
> > > bit mask.
> > > 2) PCD help message in MdePkg.uni is also required to be updated.
> > >
> > > Thanks
> > > Liming
> > > -----Original Message-----
> > > From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of Felix
> > Poludov
> > > Sent: Thursday, September 15, 2016 3:27 AM
> > > To: edk2-devel@lists.01.org
> > > Subject: Re: [edk2] [PATCH] [MdePkg ] New Debug Message Levels
> > >
> > > Resending with the patch (the patch was missing in the original e-mail).
> > >
> > > -----Original Message-----
> > > From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of Felix
> > Poludov
> > > Sent: Wednesday, September 14, 2016 12:05 PM
> > > To: edk2-devel@lists.01.org
> > > Subject: [edk2] [PATCH] [MdePkg ] New Debug Message Levels
> > >
> > > Dear MdePkg maintainer,
> > >
> > > DebugLib.h defines debug message levels. Some values such as DEBUG_ERROR and
> > > DEBUG_INFO define messages category, whereas other values such as DEBUG_BLKIO and
> > > DEBUG_VARIABLE define message domain.
> > > This patch adds definitions for several additional message domains.
> > >
> > > Contributed-under: TianoCore Contribution Agreement 1.0
> > > Signed-off-by: Felix Polyudov <felixp@ami.com<mailto:felixp@ami.com>>
> > >
> > > Thanks
> > > Felix
> > >
> > > ==
> > > diff --git a/MdePkg/Include/Library/DebugLib.h
> b/MdePkg/Include/Library/DebugLib.h
> > > index 803872c..47549b7 100644
> > > --- a/MdePkg/Include/Library/DebugLib.h
> > > +++ b/MdePkg/Include/Library/DebugLib.h
> > > @@ -54,6 +54,12 @@ WITHOUT WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER
> > EXPRESS
> > > OR IMPLIED.
> > >  #define DEBUG_CACHE     0x00200000  // Memory range cachability changes
> > >  #define DEBUG_VERBOSE   0x00400000  // Detailed debug messages that may
> > >                                      // significantly impact boot performance
> > > +#define DEBUG_USB       0x00000200  // USB Driver
> > > +#define DEBUG_PCI       0x00000800  // PCI Bus Driver
> > > +#define DEBUG_CSM       0x00002000  // Compatibility Support Module
> > > +#define DEBUG_CPU       0x00008000  // Processor Driver
> > > +#define DEBUG_CHIPSET   0x00040000  // Chipset/SoC Drivers
> > > +#define DEBUG_PLATFORM  0x00800000  // Platform/Board Drivers
> > >  #define DEBUG_ERROR     0x80000000  // Error
> > >
> > >  //
> > > @@ -75,6 +81,12 @@ WITHOUT WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER
> > EXPRESS
> > > OR IMPLIED.
> > >  #define EFI_D_LOADFILE  DEBUG_LOADFILE
> > >  #define EFI_D_EVENT     DEBUG_EVENT
> > >  #define EFI_D_VERBOSE   DEBUG_VERBOSE
> > > +#define EFI_D_USB       DEBUG_USB
> > > +#define EFI_D_PCI       DEBUG_PCI
> > > +#define EFI_D_CSM       DEBUG_CSM
> > > +#define EFI_D_CPU       DEBUG_CPU
> > > +#define EFI_D_CHIPSET   DEBUG_CHIPSET
> > > +#define EFI_D_PLATFORM  DEBUG_PLATFORM
> > >  #define EFI_D_ERROR     DEBUG_ERROR
> > >
> > >  /**
> > > diff --git a/MdePkg/MdePkg.dec b/MdePkg/MdePkg.dec
> > > index 606e2f1..eaae076 100644
> > > --- a/MdePkg/MdePkg.dec
> > > +++ b/MdePkg/MdePkg.dec
> > > @@ -1883,15 +1883,21 @@
> > >    #  BIT6  - Information message.<BR>
> > >    #  BIT7  - Dispatcher message.<BR>
> > >    #  BIT8  - Variable message.<BR>
> > > +  #  BIT9  - USB Driver message.<BR>
> > >    #  BIT10 - Boot Manager message.<BR>
> > > +  #  BIT11 - PCI Bus Driver message.<BR>
> > >    #  BIT12 - BlockIo Driver message.<BR>
> > > +  #  BIT13 - Compatibility Support Module message.<BR>
> > >    #  BIT14 - Network Driver message.<BR>
> > > +  #  BIT15 - Processor Driver message.<BR>
> > >    #  BIT16 - UNDI Driver message.<BR>
> > >    #  BIT17 - LoadFile message.<BR>
> > > +  #  BIT18 - Chipset/SoC Driver message.<BR>
> > >    #  BIT19 - Event message.<BR>
> > >    #  BIT20 - Global Coherency Database changes message.<BR>
> > >    #  BIT21 - Memory range cachability changes message.<BR>
> > >    #  BIT22 - Detailed debug message.<BR>
> > > +  #  BIT23 - Platform/Board Driver message.<BR>
> > >    #  BIT31 - Error message.<BR>
> > >    # @Prompt Fixed Debug Message Print Level.
> > >
> > gEfiMdePkgTokenSpaceGuid.PcdFixedDebugPrintErrorLevel|0xFFFFFFFF|UINT32|0x30001016
> > > @@ -1951,15 +1957,21 @@
> > >    #  BIT6  - Information message.<BR>
> > >    #  BIT7  - Dispatcher message.<BR>
> > >    #  BIT8  - Variable message.<BR>
> > > +  #  BIT9  - USB Driver message.<BR>
> > >    #  BIT10 - Boot Manager message.<BR>
> > > +  #  BIT11 - PCI Bus Driver message.<BR>
> > >    #  BIT12 - BlockIo Driver message.<BR>
> > > +  #  BIT13 - Compatibility Support Module message.<BR>
> > >    #  BIT14 - Network Driver message.<BR>
> > > +  #  BIT15 - Processor Driver message.<BR>
> > >    #  BIT16 - UNDI Driver message.<BR>
> > >    #  BIT17 - LoadFile message.<BR>
> > > +  #  BIT18 - Chipset/SoC Driver message.<BR>
> > >    #  BIT19 - Event message.<BR>
> > >    #  BIT20 - Global Coherency Database changes message.<BR>
> > >    #  BIT21 - Memory range cachability changes message.<BR>
> > >    #  BIT22 - Detailed debug message.<BR>
> > > +  #  BIT23 - Platform/Board Driver message.<BR>
> > >    #  BIT31 - Error message.<BR>
> > >    # @Prompt Debug Message Print Level.
> > >    # @Expression  0x80000002 | (gEfiMdePkgTokenSpaceGuid.PcdDebugPrintErrorLevel
> &
> > > 0x7F84AA00) == 0
> > >
> > > Please consider the environment before printing this email.
> > >
> > > The information contained in this message may be confidential and proprietary to
> > > American Megatrends, Inc.  This communication is intended to be read only by the
> > > individual or entity to whom it is addressed or by their designee. If the reader
> of
> > > this message is not the intended recipient, you are on notice that any
> distribution
> > > of this message, in any form, is strictly prohibited.  Please promptly notify the
> > > sender by reply e-mail or by telephone at 770-246-8600, and then delete or
> destroy
> > > all copies of the transmission.
> > > _______________________________________________
> > > edk2-devel mailing list
> > > edk2-devel@lists.01.org
> > > https://lists.01.org/mailman/listinfo/edk2-devel
> > > _______________________________________________
> > > edk2-devel mailing list
> > > edk2-devel@lists.01.org
> > > https://lists.01.org/mailman/listinfo/edk2-devel
> > _______________________________________________
> > edk2-devel mailing list
> > edk2-devel@lists.01.org
> > https://lists.01.org/mailman/listinfo/edk2-devel
> >
> > Please consider the environment before printing this email.
> >
> > The information contained in this message may be confidential and proprietary to
> > American Megatrends, Inc.  This communication is intended to be read only by the
> > individual or entity to whom it is addressed or by their designee. If the reader of
> > this message is not the intended recipient, you are on notice that any distribution
> > of this message, in any form, is strictly prohibited.  Please promptly notify the
> > sender by reply e-mail or by telephone at 770-246-8600, and then delete or destroy
> > all copies of the transmission.
> 
> Please consider the environment before printing this email.
> 
> The information contained in this message may be confidential and proprietary to
> American Megatrends, Inc.  This communication is intended to be read only by the
> individual or entity to whom it is addressed or by their designee. If the reader of
> this message is not the intended recipient, you are on notice that any distribution
> of this message, in any form, is strictly prohibited.  Please promptly notify the
> sender by reply e-mail or by telephone at 770-246-8600, and then delete or destroy
> all copies of the transmission.
> _______________________________________________
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel