[RFCv2] code-first process for UEFI-forum specifications

[RFCv2] code-first process for UEFI-forum specifications

[Leif Lindholm]

Changes to v2 of this proposal:
- Feedback from Laszlo[a] and Mike[b] incorporated.
  - I opted to view Mike's responses to Laszlo's questions as
    accepted, as no follow-up was made.

Feedback from Felix[c] *not* incorporated, as while I agree with all
of it, I am not convinced that information should go here, but should
instead reside with the UEFI Forum. (This text documents the public
part of the process - it would cause me slight impedance mismatch to
have it also document the non-public part.)

[a] https://edk2.groups.io/g/devel/message/53422
[b] https://edk2.groups.io/g/devel/message/53738
[c] https://edk2.groups.io/g/devel/message/54453

/
    Leif

---

This is a proposal for a process by which new features can be added to UEFI
forum specifications after first having been designed and prototyped in the
open.

This process lets changes and the development of new features happen in the
open, without violating the UEFI forum bylaws which prevent publication of
code for in-draft features/changes.

The process does not in fact change the UEFI bylaws - the change is that the
development (of both specification and code) happens in the open. The resulting
specification update is then submitted to the appropriate working goup as an
Engineering Change Request (ECR), and voted on. For the UEFI Forum, this is a
change in workflow, not a change in process.

ECRs are tracked in a UEFI Forum Mantis instance, access restricted to UEFI
Forum Members. TianoCore will enable this new process by providing areas on
https://bugzilla.tianocore.org/ to track both specification updates and
reference implementations and new repositories under
https://github.com/tianocore/ dedicated to hold "code first".


## Bugzilla

bugzilla.tianocore.org will have a product category each for
  * ACPI Specification
  * PI Specification
  * UEFI Specification

Each product category will have a separate components for
  * Specification
  * Reference implementation


## Github
New repositories will be added for holding the text changes and the source code.

Specification text changes will be held within the affected source repository,
in the Github flavour of markdown, in a file (or split across several files)
with .md suffix.
(This one may break down where we have a specification change affecting multiple
specifications, but at that point we can track it with multiple BZ entries)

Reference implementations targeting EDK2 will be held in branches on
edk2-staging.
Additional repositories for implementing reference features in
additional open source projects can be added in the future, as required.


## Intended workflow
The entity initiating a specifiation update raises a Bugzilla in the appropriate
area in bugzilla.tianocore.org. This entry contains the outline of the change,
and the full initial draft text is attached.

If multiple specification updates are interdependent, especially if between
different specifications, then multiple bugzilla entries should be created.
These bugzilla entries *must* be linked together with dependencies.

After the BZs have been created, new branches should be created in the relevant
repositories for each bugzilla - the branch names should be BZ####, where ####
describes the bugzilla ID assigned, optionally followed by a '-' and something
more descriptive. If multiple bugzilla entries must coexist on a single branch,
one of them is designated the 'top-level', with dependencies properly tracked.
That BZ will be the one naming the branch.


## Source code
In order to ensure draft code does not accidentally leak into production use,
and to signify when the changeover from draft to final happens, *all* new or
modified[1] identifiers need to be prefixed with the relevant BZ####.

[1] Modified in a non-backwards-compatible way. If, for example, a statically
    sized array is grown - this does not need to be prefixed. But a tag in a
	comment would be *highly* recommended.

### File names
New public header files need the prefix. I.e. `Bz1234MyNewProtocol.h`
Private header files do not need the prefix.

### Contents

The tagging must follow the coding style used by each affected codebase.
Examples:

| Released in spec      | Draft version in tree       | Comment                |
| ---                   | ---                         | ---                    |
| `FunctionName`        | `Bz1234FunctionName`        |                        |
| `HEADER_MACRO`        | `BZ1234_HEADER_MACRO`       |                        |

For data structures or enums, any new or non-backwards-compatible structs or
fields require a prefix. As above, growing an existing array in an existing
struct requires no prefix.

| `typedef SOME_STRUCT` | `BZ1234_SOME_STRUCT`        | Typedef only [2]       |
| `StructField`         | `Bz1234StructField`         | In existing struct[3]  |
| `typedef SOME_ENUM`   | `BZ1234_SOME_ENUM`          | Typedef only [2]       |

[2] If the struct or enum definition is separate from the typedef in the public
    header, the definition does not need the prefix.
[3] Individual fields in newly added typedefd struct do not need prefix, the
    struct already carried the prefix.

Variable prefixes indicating global scope ('g' or 'm') go before the BZ prefix.

| `gSomeGuid`           | `gBz1234SomeGuid`           |                        |

Local identifiers, including module-global ones (m-prefixed) do not require a
BZ prefix.

Re: [edk2-rfc] [RFCv2] code-first process for UEFI-forum specifications

[Ni, Ray]

> 
> ## Github
> New repositories will be added for holding the text changes and the source code.
> 
> Specification text changes will be held within the affected source repository,
> in the Github flavour of markdown, in a file (or split across several files)
> with .md suffix.

What's the case when multiple .MD files are needed?

> (This one may break down where we have a specification change affecting multiple
> specifications, but at that point we can track it with multiple BZ entries)

> 
> 
> ## Source code
> In order to ensure draft code does not accidentally leak into production use,
> and to signify when the changeover from draft to final happens, *all* new or
> modified[1] identifiers need to be prefixed with the relevant BZ####.
> 
> [1] Modified in a non-backwards-compatible way. If, for example, a statically
>     sized array is grown - this does not need to be prefixed. But a tag in a
> 	comment would be *highly* recommended.

If a protocol is enhanced to provide more interfaces with increased revision number,
would you like the protocol name to be prefixed with BZ####?
Or just the new interfaces added to the protocol are prefixed the BZ####?
I think just prefixing the new interfaces can meet the purpose.

But the protocol definition is changed, it also needs to be prefixed according to this flow.
Can you clarify a bit more?

> 
> ### File names
> New public header files need the prefix. I.e. `Bz1234MyNewProtocol.h`
> Private header files do not need the prefix.
> 
> ### Contents
> 
> The tagging must follow the coding style used by each affected codebase.
> Examples:
> 
> | Released in spec      | Draft version in tree       | Comment                |
> | ---                   | ---                         | ---                    |
> | `FunctionName`        | `Bz1234FunctionName`        |                        |
> | `HEADER_MACRO`        | `BZ1234_HEADER_MACRO`       |                        |

If FunctionName or HEADER_MACRO is defined in non-public header files, I don't
think they require the prefix. Do you agree?
 
> For data structures or enums, any new or non-backwards-compatible structs or
> fields require a prefix. As above, growing an existing array in an existing
> struct requires no prefix.
> 
> | `typedef SOME_STRUCT` | `BZ1234_SOME_STRUCT`        | Typedef only [2]       |
> | `StructField`         | `Bz1234StructField`         | In existing struct[3]  |
> | `typedef SOME_ENUM`   | `BZ1234_SOME_ENUM`          | Typedef only [2]       |
> 
> [2] If the struct or enum definition is separate from the typedef in the public
>     header, the definition does not need the prefix.

What does "separate" mean?
Does it mean "struct or enum in the public header BzXXX.h don't need the prefix"?
If yes, then I think macros defined in BzXXX.h also don't need the prefix.

> [3] Individual fields in newly added typedefd struct do not need prefix, the
>     struct already carried the prefix.
> 
> Variable prefixes indicating global scope ('g' or 'm') go before the BZ prefix.
> 
> | `gSomeGuid`           | `gBz1234SomeGuid`           |                        |
> 
> Local identifiers, including module-global ones (m-prefixed) do not require a
> BZ prefix.

I think only the names (struct type name, enum type name, interface name, protocol/ppi name)
defined in public header files need the BZ prefix when the public header doesn't have prefix.
Right?

Re: [edk2-devel] [RFCv2] code-first process for UEFI-forum specifications

[Laszlo Ersek]

On 03/23/20 20:05, Leif Lindholm wrote:
> Changes to v2 of this proposal:
> - Feedback from Laszlo[a] and Mike[b] incorporated.
>   - I opted to view Mike's responses to Laszlo's questions as
>     accepted, as no follow-up was made.
> 
> Feedback from Felix[c] *not* incorporated, as while I agree with all
> of it, I am not convinced that information should go here, but should
> instead reside with the UEFI Forum. (This text documents the public
> part of the process - it would cause me slight impedance mismatch to
> have it also document the non-public part.)
> 
> [a] https://edk2.groups.io/g/devel/message/53422
> [b] https://edk2.groups.io/g/devel/message/53738
> [c] https://edk2.groups.io/g/devel/message/54453
> 
> /
>     Leif
> 
> ---
> 
> This is a proposal for a process by which new features can be added to UEFI
> forum specifications after first having been designed and prototyped in the
> open.
> 
> This process lets changes and the development of new features happen in the
> open, without violating the UEFI forum bylaws which prevent publication of
> code for in-draft features/changes.
> 
> The process does not in fact change the UEFI bylaws - the change is that the
> development (of both specification and code) happens in the open. The resulting
> specification update is then submitted to the appropriate working goup as an
> Engineering Change Request (ECR), and voted on. For the UEFI Forum, this is a
> change in workflow, not a change in process.
> 
> ECRs are tracked in a UEFI Forum Mantis instance, access restricted to UEFI
> Forum Members. TianoCore will enable this new process by providing areas on
> https://bugzilla.tianocore.org/ to track both specification updates and
> reference implementations and new repositories under
> https://github.com/tianocore/ dedicated to hold "code first".
> 
> 
> ## Bugzilla
> 
> bugzilla.tianocore.org will have a product category each for
>   * ACPI Specification
>   * PI Specification
>   * UEFI Specification
> 
> Each product category will have a separate components for
>   * Specification
>   * Reference implementation
> 
> 
> ## Github
> New repositories will be added for holding the text changes and the source code.
> 
> Specification text changes will be held within the affected source repository,
> in the Github flavour of markdown, in a file (or split across several files)
> with .md suffix.
> (This one may break down where we have a specification change affecting multiple
> specifications, but at that point we can track it with multiple BZ entries)
> 
> Reference implementations targeting EDK2 will be held in branches on
> edk2-staging.
> Additional repositories for implementing reference features in
> additional open source projects can be added in the future, as required.
> 
> 
> ## Intended workflow
> The entity initiating a specifiation update raises a Bugzilla in the appropriate
> area in bugzilla.tianocore.org. This entry contains the outline of the change,
> and the full initial draft text is attached.
> 
> If multiple specification updates are interdependent, especially if between
> different specifications, then multiple bugzilla entries should be created.
> These bugzilla entries *must* be linked together with dependencies.
> 
> After the BZs have been created, new branches should be created in the relevant
> repositories for each bugzilla - the branch names should be BZ####, where ####
> describes the bugzilla ID assigned, optionally followed by a '-' and something
> more descriptive. If multiple bugzilla entries must coexist on a single branch,
> one of them is designated the 'top-level', with dependencies properly tracked.
> That BZ will be the one naming the branch.
> 
> 
> ## Source code
> In order to ensure draft code does not accidentally leak into production use,
> and to signify when the changeover from draft to final happens, *all* new or
> modified[1] identifiers need to be prefixed with the relevant BZ####.
> 
> [1] Modified in a non-backwards-compatible way. If, for example, a statically
>     sized array is grown - this does not need to be prefixed. But a tag in a
> 	comment would be *highly* recommended.
> 
> ### File names
> New public header files need the prefix. I.e. `Bz1234MyNewProtocol.h`
> Private header files do not need the prefix.
> 
> ### Contents
> 
> The tagging must follow the coding style used by each affected codebase.
> Examples:
> 
> | Released in spec      | Draft version in tree       | Comment                |
> | ---                   | ---                         | ---                    |
> | `FunctionName`        | `Bz1234FunctionName`        |                        |
> | `HEADER_MACRO`        | `BZ1234_HEADER_MACRO`       |                        |
> 
> For data structures or enums, any new or non-backwards-compatible structs or
> fields require a prefix. As above, growing an existing array in an existing
> struct requires no prefix.
> 
> | `typedef SOME_STRUCT` | `BZ1234_SOME_STRUCT`        | Typedef only [2]       |
> | `StructField`         | `Bz1234StructField`         | In existing struct[3]  |
> | `typedef SOME_ENUM`   | `BZ1234_SOME_ENUM`          | Typedef only [2]       |
> 
> [2] If the struct or enum definition is separate from the typedef in the public
>     header, the definition does not need the prefix.
> [3] Individual fields in newly added typedefd struct do not need prefix, the
>     struct already carried the prefix.
> 
> Variable prefixes indicating global scope ('g' or 'm') go before the BZ prefix.
> 
> | `gSomeGuid`           | `gBz1234SomeGuid`           |                        |
> 
> Local identifiers, including module-global ones (m-prefixed) do not require a
> BZ prefix.

Acked-by: Laszlo Ersek <lersek@redhat.com>

Re: [edk2-rfc] [RFCv2] code-first process for UEFI-forum specifications

[Samer El-Haj-Mahmoud]

Acked-by: Samer El-Haj-Mahmoud <Samer.El-Haj-Mahmoud@arm.com>



> -----Original Message-----
> From: rfc@edk2.groups.io <rfc@edk2.groups.io> On Behalf Of Leif Lindholm
> via Groups.Io
> Sent: Monday, March 23, 2020 3:06 PM
> To: devel@edk2.groups.io; rfc@edk2.groups.io
> Cc: Felixp@ami.com; Mark Doran <mark.doran@intel.com>; Andrew Fish
> <afish@apple.com>; Laszlo Ersek <lersek@redhat.com>; Michael D Kinney
> <michael.d.kinney@intel.com>
> Subject: [edk2-rfc] [RFCv2] code-first process for UEFI-forum specifications
>
> Changes to v2 of this proposal:
> - Feedback from Laszlo[a] and Mike[b] incorporated.
>   - I opted to view Mike's responses to Laszlo's questions as
>     accepted, as no follow-up was made.
>
> Feedback from Felix[c] *not* incorporated, as while I agree with all of it, I am
> not convinced that information should go here, but should instead reside with
> the UEFI Forum. (This text documents the public part of the process - it would
> cause me slight impedance mismatch to have it also document the non-public
> part.)
>
> [a] https://edk2.groups.io/g/devel/message/53422
> [b] https://edk2.groups.io/g/devel/message/53738
> [c] https://edk2.groups.io/g/devel/message/54453
>
> /
>     Leif
>
> ---
>
> This is a proposal for a process by which new features can be added to UEFI
> forum specifications after first having been designed and prototyped in the
> open.
>
> This process lets changes and the development of new features happen in the
> open, without violating the UEFI forum bylaws which prevent publication of
> code for in-draft features/changes.
>
> The process does not in fact change the UEFI bylaws - the change is that the
> development (of both specification and code) happens in the open. The
> resulting specification update is then submitted to the appropriate working
> goup as an Engineering Change Request (ECR), and voted on. For the UEFI
> Forum, this is a change in workflow, not a change in process.
>
> ECRs are tracked in a UEFI Forum Mantis instance, access restricted to UEFI
> Forum Members. TianoCore will enable this new process by providing areas
> on https://bugzilla.tianocore.org/ to track both specification updates and
> reference implementations and new repositories under
> https://github.com/tianocore/ dedicated to hold "code first".
>
>
> ## Bugzilla
>
> bugzilla.tianocore.org will have a product category each for
>   * ACPI Specification
>   * PI Specification
>   * UEFI Specification
>
> Each product category will have a separate components for
>   * Specification
>   * Reference implementation
>
>
> ## Github
> New repositories will be added for holding the text changes and the source
> code.
>
> Specification text changes will be held within the affected source repository,
> in the Github flavour of markdown, in a file (or split across several files) with
> .md suffix.
> (This one may break down where we have a specification change affecting
> multiple specifications, but at that point we can track it with multiple BZ
> entries)
>
> Reference implementations targeting EDK2 will be held in branches on edk2-
> staging.
> Additional repositories for implementing reference features in additional
> open source projects can be added in the future, as required.
>
>
> ## Intended workflow
> The entity initiating a specifiation update raises a Bugzilla in the appropriate
> area in bugzilla.tianocore.org. This entry contains the outline of the change,
> and the full initial draft text is attached.
>
> If multiple specification updates are interdependent, especially if between
> different specifications, then multiple bugzilla entries should be created.
> These bugzilla entries *must* be linked together with dependencies.
>
> After the BZs have been created, new branches should be created in the
> relevant repositories for each bugzilla - the branch names should be BZ####,
> where #### describes the bugzilla ID assigned, optionally followed by a '-' and
> something more descriptive. If multiple bugzilla entries must coexist on a
> single branch, one of them is designated the 'top-level', with dependencies
> properly tracked.
> That BZ will be the one naming the branch.
>
>
> ## Source code
> In order to ensure draft code does not accidentally leak into production use,
> and to signify when the changeover from draft to final happens, *all* new or
> modified[1] identifiers need to be prefixed with the relevant BZ####.
>
> [1] Modified in a non-backwards-compatible way. If, for example, a statically
>     sized array is grown - this does not need to be prefixed. But a tag in a
> comment would be *highly* recommended.
>
> ### File names
> New public header files need the prefix. I.e. `Bz1234MyNewProtocol.h`
> Private header files do not need the prefix.
>
> ### Contents
>
> The tagging must follow the coding style used by each affected codebase.
> Examples:
>
> | Released in spec      | Draft version in tree       | Comment                |
> | ---                   | ---                         | ---                    |
> | `FunctionName`        | `Bz1234FunctionName`        |                        |
> | `HEADER_MACRO`        | `BZ1234_HEADER_MACRO`       |                        |
>
> For data structures or enums, any new or non-backwards-compatible structs
> or fields require a prefix. As above, growing an existing array in an existing
> struct requires no prefix.
>
> | `typedef SOME_STRUCT` | `BZ1234_SOME_STRUCT`        | Typedef only [2]
> |
> | `StructField`         | `Bz1234StructField`         | In existing struct[3]  |
> | `typedef SOME_ENUM`   | `BZ1234_SOME_ENUM`          | Typedef only [2]
> |
>
> [2] If the struct or enum definition is separate from the typedef in the public
>     header, the definition does not need the prefix.
> [3] Individual fields in newly added typedefd struct do not need prefix, the
>     struct already carried the prefix.
>
> Variable prefixes indicating global scope ('g' or 'm') go before the BZ prefix.
>
> | `gSomeGuid`           | `gBz1234SomeGuid`           |                        |
>
> Local identifiers, including module-global ones (m-prefixed) do not require a
> BZ prefix.
>
> 

IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

Re: [edk2-rfc] [RFCv2] code-first process for UEFI-forum specifications

[Leif Lindholm]

On Wed, Mar 25, 2020 at 05:14:59 +0000, Ni, Ray wrote:
> > 
> > ## Github
> > New repositories will be added for holding the text changes and the source code.
> > 
> > Specification text changes will be held within the affected source repository,
> > in the Github flavour of markdown, in a file (or split across several files)
> > with .md suffix.
> 
> What's the case when multiple .MD files are needed?

For example if a branch covers changes to multiple specifications, as
described elsewhere. Or if it simply makes sense due to content size.
It is possible, now we've migrated to .rst for edk2, that we should
change the format recommentded in this proposal too.

> > (This one may break down where we have a specification change affecting multiple
> > specifications, but at that point we can track it with multiple BZ entries)
> 
> > 
> > 
> > ## Source code
> > In order to ensure draft code does not accidentally leak into production use,
> > and to signify when the changeover from draft to final happens, *all* new or
> > modified[1] identifiers need to be prefixed with the relevant BZ####.
> > 
> > [1] Modified in a non-backwards-compatible way. If, for example, a statically
> >     sized array is grown - this does not need to be prefixed. But a tag in a
> > 	comment would be *highly* recommended.
> 
> If a protocol is enhanced to provide more interfaces with increased revision number,
> would you like the protocol name to be prefixed with BZ####?
> Or just the new interfaces added to the protocol are prefixed the BZ####?
> I think just prefixing the new interfaces can meet the purpose.

Adding new interfaces to a protocol does not affect its
backwards-compatibility, which was what I was trying to cover above.
If you can think of a better way of describing it. I am very open to
suggestions.

> But the protocol definition is changed, it also needs to be prefixed according to this flow.
> Can you clarify a bit more?

In that instance, only the new interfaces would need the prefix.

> > 
> > ### File names
> > New public header files need the prefix. I.e. `Bz1234MyNewProtocol.h`
> > Private header files do not need the prefix.
> > 
> > ### Contents
> > 
> > The tagging must follow the coding style used by each affected codebase.
> > Examples:
> > 
> > | Released in spec      | Draft version in tree       | Comment                |
> > | ---                   | ---                         | ---                    |
> > | `FunctionName`        | `Bz1234FunctionName`        |                        |
> > | `HEADER_MACRO`        | `BZ1234_HEADER_MACRO`       |                        |
> 
> If FunctionName or HEADER_MACRO is defined in non-public header files, I don't
> think they require the prefix. Do you agree?

Only public interfaces need prefix. This also means that non-public
interfaces should be STATIC where possible.

> > For data structures or enums, any new or non-backwards-compatible structs or
> > fields require a prefix. As above, growing an existing array in an existing
> > struct requires no prefix.
> > 
> > | `typedef SOME_STRUCT` | `BZ1234_SOME_STRUCT`        | Typedef only [2]       |
> > | `StructField`         | `Bz1234StructField`         | In existing struct[3]  |
> > | `typedef SOME_ENUM`   | `BZ1234_SOME_ENUM`          | Typedef only [2]       |
> > 
> > [2] If the struct or enum definition is separate from the typedef in the public
> >     header, the definition does not need the prefix.
> 
> What does "separate" mean?
> Does it mean "struct or enum in the public header BzXXX.h don't need the prefix"?
> If yes, then I think macros defined in BzXXX.h also don't need the prefix.

Struct or enum definitions in the public header BzXXX.h don't need the
prefix *when they have a public-facing typedef with the prefix*.
Everything new or not-backwards-compatible needs to be referred to via
the prefixed names in external modules.

I.e. we can have
struct _SomeNewThing {
};

typedef struct _SomeNewThing BzXXX_PROTOCOL;

(This is meant simply as shorthand, reducing the amount of changes
required for the published version.)

> > [3] Individual fields in newly added typedefd struct do not need prefix, the
> >     struct already carried the prefix.
> > 
> > Variable prefixes indicating global scope ('g' or 'm') go before the BZ prefix.
> > 
> > | `gSomeGuid`           | `gBz1234SomeGuid`           |                        |
> > 
> > Local identifiers, including module-global ones (m-prefixed) do not require a
> > BZ prefix.
> 
> I think only the names (struct type name, enum type name, interface name, protocol/ppi name)
> defined in public header files need the BZ prefix when the public header doesn't have prefix.
> Right?

That is one way we *could* do it. It is not one I am proposing.
My idea is that it should be very clear from looking at code whether
it includes non-ratified proposal code.

/
    Leif