Re: [edk2-devel] [PATCH v1 2/6] ArmVirtPkg: Add Platform CI and configuration for Core CI

Re: [edk2-devel] [PATCH v1 2/6] ArmVirtPkg: Add Platform CI and configuration for Core CI

[Sean]

[-- Attachment #1: Type: text/plain, Size: 1406 bytes --]

Ard/Laszlo,

On the topic of Package files.
1. The *.ci.yaml is designed to be at the root of the package and this follows all other Core ci usage.  The Core CI will only look for this file at the root as of now.
2.  The PlatformBuild.py file is exactly like a build.bat or build.sh file found in many platform packages.  This file is relevant for local dev as it will build the package and should support the feature set of the platform.  I would hope that the package maintainers would continue to update this file to match the features of their package otherwise CI will become less and less relevant overtime.  Thus I think it belongs in the same place as build.sh.  The only other pattern we could follow is a ".pytools" folder like for Core CI.  But again unless you strongly disagree I would prefer the first.
3. The readme file...You tell me where?  Or it could be combined with the existing readme.md file?
4. The iasl ext_dep file.  I could see this moving deeper into the package.  Where would you like it? This file is an external dependency to control the version of iasl used on this platform.  It should be kept current with what the platform expects for iasl usage.

Anyway, I need more actionable feedback if you don't agree with the above.

On the forward slashes and typo.  I agree and will get them in the next version.   Thanks for the review.

Thanks
Sean

[-- Attachment #2: Type: text/html, Size: 1568 bytes --]

Re: [edk2-devel] [PATCH v1 2/6] ArmVirtPkg: Add Platform CI and configuration for Core CI

[Sean]

[-- Attachment #1: Type: text/plain, Size: 297 bytes --]

Any thoughts?  I would like to get this in before any more failures get checked in causing more dependencies.

I have a branch here: https://github.com/spbrogan/edk2/tree/PlatformAndCoreCIForOvmfArmVirtEmulatorPackages_v7
but am waiting for more feedback to make V2 patchset.

Thanks
Sean

[-- Attachment #2: Type: text/html, Size: 451 bytes --]

Re: [edk2-devel] [PATCH v1 2/6] ArmVirtPkg: Add Platform CI and configuration for Core CI

[Laszlo Ersek]

Hi Sean,

On 04/15/20 08:55, Sean via groups.io wrote:
> Any thoughts?

I have not ignored your email. Instead, I have not seen it.

The reason is that you messaged the list only -- you didn't put me in
the To: or Cc: headers, despite addressing me by name in the message body.

While I do follow the entire list, my personal inbox enjoys a much
higher priority than my edk2-devel list folder -- the latter is (almost)
just one of my many list folders. Therefore, if a query is sent only to
the list, and not copied to me, I could come to it much later, or (in
the worst case) I might even totally miss it. (E.g. if I were
overloaded, and the message were deeply nested in an existent thread
and/or the subject line didn't obviously call for my attention.)

General mailing list etiquette (not specific to edk2-devel) is that,
when asking a public question to a specific subscriber, then both the
list and that subscriber be copied on the message. This way all other
interested subscribers (and the list archive) can follow the discussion,
*and* the specific subscriber will not only find the query in their list
folder, but also in their inbox.

What's even worse is that, your email was sent in response to Ard's (the
one that said "I agree with Laszlo's point here"), and I *was* CC'd on
Ard's message. Which means that your MUA explicitly *dropped* me from
your message. I'm not sure what MUA does that -- I can even imagine it
was the groups.io WebUI! --, but whatever MUA it was, it was very wrong.
Dropping CC's *breaks* mailing list discussions.

Anyway, I'll now read your previous email and attempt an answer.

(To be clear, I've now seen your *current* message too only because I'm
at the moment a bit less tired, and happened to catch it in my list folder.)

Thanks
Laszlo

Re: [edk2-devel] [PATCH v1 2/6] ArmVirtPkg: Add Platform CI and configuration for Core CI

[Laszlo Ersek]

On 04/09/20 21:57, Sean via groups.io wrote:
> Ard/Laszlo,
> 
> On the topic of Package files.
> 1. The *.ci.yaml is designed to be at the root of the package and this follows all other Core ci usage.  The Core CI will only look for this file at the root as of now.

OK. This makes sense to me. Even if we wanted to put the files away
under a subdirectory, we'd have to inform the Core CI system somehow --
and that would likely take the form of having to name the subdirectory
in a particular way. Which is not different from having a fixed name
regular file in the package root dir.

Thankfully, this file does have ".ci." in the name.

> 2.  The PlatformBuild.py file is exactly like a build.bat or build.sh file found in many platform packages.  This file is relevant for local dev as it will build the package and should support the feature set of the platform.  I would hope that the package maintainers would continue to update this file to match the features of their package otherwise CI will become less and less relevant overtime.  Thus I think it belongs in the same place as build.sh.

Thank you, this explanation is very helpful!

ArmVirtPkg does not have a "build.sh" of the kind that you mention --
and I strongly prefer it that way.

OvmfPkg happens to have a "build.sh" -- and if it were up to me, I would
remove it. I have stated this before, perhaps multiple times. I strongly
prefer people invoking "build" manually, or writing their *own* scripts
for wrapping the "build" utility. I never ever use "OvmfPkg/build.sh"
myself -- it doesn't do what I want (and what I want, in this aspect,
should not be in the upstream edk2 tree).

But, again, OVMF's "build.sh" was added in commit 66325870afaf
("OvmfPkg/build.sh: Add features and replace build32/64.sh",
2011-01-09), and by the time I "arrived" and had grown a distaste for
it, other users had become dependent on it. So I can not propose its
removal. I can certainly argue against spreading the practice to ArmVirtPkg.

I *do* agree that "PlatformBuild.py" is necessary for CI, and keeping it
up-to-date is also necessary for good coverage by CI. I'm just
requesting to move it out of the package root dir.

> The only other pattern we could follow is a ".pytools" folder like for Core CI.  But again unless you strongly disagree I would prefer the first.

The project root already has a .pytool subdir, so (I think?) there would
be a precedent.

But, the leading dot (for "hiding" the subdirectory) is totally not my
point. I don't wish to hide the CI metafiles to *that* extent. I'd just
like to clearly associate them with the CI purpose -- it's fine if they
are visible when the directory is listed with default switches, but the
names should be specific.

> 3. The readme file...You tell me where?  Or it could be combined with the existing readme.md file?
> 4. The iasl ext_dep file.  I could see this moving deeper into the package.  Where would you like it? This file is an external dependency to control the version of iasl used on this platform.  It should be kept current with what the platform expects for iasl usage.
> 
> Anyway, I need more actionable feedback if you don't agree with the above.

How do you feel about the following directory structure:

  ArmVirtPkg/ArmVirtPkg.ci.yaml
  ArmVirtPkg/PlatformCI/Ubuntu-GCC5.yml
  ArmVirtPkg/PlatformCI/PlatformBuild.py
  ArmVirtPkg/PlatformCI/README-pytools.md
  ArmVirtPkg/PlatformCI/iasl_ext_dep.yaml

(Note that I also removed the ".azurepipelines/" prefix from
"Ubuntu-GCC5.yml", in addition to pushing it under "PlatformCI/". If
".azurepipelines/" is required, then I'm fine with it, as long as we can
also keep it under "ArmVirtPkg/PlatformCI/".)

The result would be that someone running "ls" or "dir" in ArmVirtPkg
would see two CI-related entries: the "ArmVirtPkg.ci.yaml" control file,
and the "PlatformCI" directory. Both entries clearly state "CI" in the
name, and separate CI objects from the rest of the package.

Thanks!
Laszlo

Re: [edk2-devel] [PATCH v1 2/6] ArmVirtPkg: Add Platform CI and configuration for Core CI

[Ard Biesheuvel]

On 4/15/20 7:18 PM, Laszlo Ersek wrote:
> On 04/09/20 21:57, Sean via groups.io wrote:
>> Ard/Laszlo,
>>
>> On the topic of Package files.
>> 1. The *.ci.yaml is designed to be at the root of the package and this follows all other Core ci usage.  The Core CI will only look for this file at the root as of now.
> 
> OK. This makes sense to me. Even if we wanted to put the files away
> under a subdirectory, we'd have to inform the Core CI system somehow --
> and that would likely take the form of having to name the subdirectory
> in a particular way. Which is not different from having a fixed name
> regular file in the package root dir.
> 
> Thankfully, this file does have ".ci." in the name.
> 
>> 2.  The PlatformBuild.py file is exactly like a build.bat or build.sh file found in many platform packages.  This file is relevant for local dev as it will build the package and should support the feature set of the platform.  I would hope that the package maintainers would continue to update this file to match the features of their package otherwise CI will become less and less relevant overtime.  Thus I think it belongs in the same place as build.sh.
> 
> Thank you, this explanation is very helpful!
> 
> ArmVirtPkg does not have a "build.sh" of the kind that you mention --
> and I strongly prefer it that way.
> 

Agreed. I never added one because I never felt the need for it. 'build' 
has a standard interface across platforms and works in a uniform way. 
build.sh are bespoke helpers, and so I don't think they should have been 
added in the first place.

> OvmfPkg happens to have a "build.sh" -- and if it were up to me, I would
> remove it. I have stated this before, perhaps multiple times. I strongly
> prefer people invoking "build" manually, or writing their *own* scripts
> for wrapping the "build" utility. I never ever use "OvmfPkg/build.sh"
> myself -- it doesn't do what I want (and what I want, in this aspect,
> should not be in the upstream edk2 tree).
> 
> But, again, OVMF's "build.sh" was added in commit 66325870afaf
> ("OvmfPkg/build.sh: Add features and replace build32/64.sh",
> 2011-01-09), and by the time I "arrived" and had grown a distaste for
> it, other users had become dependent on it. So I can not propose its
> removal. I can certainly argue against spreading the practice to ArmVirtPkg.
> 
> I *do* agree that "PlatformBuild.py" is necessary for CI, and keeping it
> up-to-date is also necessary for good coverage by CI. I'm just
> requesting to move it out of the package root dir.
> 

Agreed. And if it has a uniform interface across packages, even better.

>> The only other pattern we could follow is a ".pytools" folder like for Core CI.  But again unless you strongly disagree I would prefer the first.
> 
> The project root already has a .pytool subdir, so (I think?) there would
> be a precedent.
> 
> But, the leading dot (for "hiding" the subdirectory) is totally not my
> point. I don't wish to hide the CI metafiles to *that* extent. I'd just
> like to clearly associate them with the CI purpose -- it's fine if they
> are visible when the directory is listed with default switches, but the
> names should be specific.
> 
>> 3. The readme file...You tell me where?  Or it could be combined with the existing readme.md file?
>> 4. The iasl ext_dep file.  I could see this moving deeper into the package.  Where would you like it? This file is an external dependency to control the version of iasl used on this platform.  It should be kept current with what the platform expects for iasl usage.
>>
>> Anyway, I need more actionable feedback if you don't agree with the above.
> 
> How do you feel about the following directory structure:
> 
>    ArmVirtPkg/ArmVirtPkg.ci.yaml
>    ArmVirtPkg/PlatformCI/Ubuntu-GCC5.yml
>    ArmVirtPkg/PlatformCI/PlatformBuild.py
>    ArmVirtPkg/PlatformCI/README-pytools.md
>    ArmVirtPkg/PlatformCI/iasl_ext_dep.yaml
> 

This looks good to me, and could be used for other packages as well, afaict.

-- 
Ard.

Re: [edk2-devel] [PATCH v1 2/6] ArmVirtPkg: Add Platform CI and configuration for Core CI

[Sean]

[-- Attachment #1: Type: text/plain, Size: 965 bytes --]

On Wed, Apr 15, 2020 at 09:58 AM, Laszlo Ersek wrote:

> 
> I have not ignored your email. Instead, I have not seen it.
> 
> The reason is that you messaged the list only -- you didn't put me in
> the To: or Cc: headers, despite addressing me by name in the message body.

I use the groups.io web client for all mailing list activities.  Did i tell you how much I love the mailing list.  :) My corporate mail server changes the subject line which also destroys threading so I figured this was better.  Maybe not as i have no control over the CC/To lines of the email that is sent.   Do Bret's responses which add "[External]" (example here: https://edk2.groups.io/g/devel/message/57377) ( https://edk2.groups.io/g/devel/message/57377 ) to every reply show up better (in groups.io web ui this is harder to follow since it starts a new thread)?

Thanks for continuing to check the full list for messages as I appreciate your feedback.

Thanks
Sean

[-- Attachment #2: Type: text/html, Size: 1082 bytes --]

Re: [edk2-devel] [PATCH v1 2/6] ArmVirtPkg: Add Platform CI and configuration for Core CI

[Sean]

[-- Attachment #1: Type: text/plain, Size: 1295 bytes --]

On Wed, Apr 15, 2020 at 10:18 AM, Laszlo Ersek wrote:

> 
> ArmVirtPkg/ArmVirtPkg.ci.yaml
> ArmVirtPkg/PlatformCI/Ubuntu-GCC5.yml
> ArmVirtPkg/PlatformCI/PlatformBuild.py
> ArmVirtPkg/PlatformCI/README-pytools.md
> ArmVirtPkg/PlatformCI/iasl_ext_dep.yaml

I am ok with the above except one thought on the readme.  One nice thing about the markdown readme files are the badge shows up in github when you view the package.  This is a quick and easy way to see the current status.

* Ovmf has a pretty stale readme and does not take advantage of markdown.    We could convert it to MD, clean up, and then merge in the content from the pytools.md.  I would need help or a package maintainer to do the cleanup of the readme to make sure it contained the content you desired.
* ArmVirtPkg doesn't have a readme and this is definitely a barrier to entry for the package.  I would suggest creating one and then merging in the content from the pytools.md.
* EmulatorPkg has one.  I would just suggest a merge but i am yet to get any feedback from those maintainers.

If that isn't desirable i would at least suggest we change the title to just ReadMe.md so that GitHub shows it by default when the PaltformCI folder is viewed form the web or in editor like vscode.

Thanks
Sean

[-- Attachment #2: Type: text/html, Size: 1460 bytes --]

Re: [edk2-devel] [PATCH v1 2/6] ArmVirtPkg: Add Platform CI and configuration for Core CI

[Laszlo Ersek]

On 04/15/20 22:38, sean.brogan via [] wrote:
> On Wed, Apr 15, 2020 at 10:18 AM, Laszlo Ersek wrote:
>
>>
>> ArmVirtPkg/ArmVirtPkg.ci.yaml
>> ArmVirtPkg/PlatformCI/Ubuntu-GCC5.yml
>> ArmVirtPkg/PlatformCI/PlatformBuild.py
>> ArmVirtPkg/PlatformCI/README-pytools.md
>> ArmVirtPkg/PlatformCI/iasl_ext_dep.yaml
>
> I am ok with the above except one thought on the readme.  One nice
> thing about the markdown readme files are the badge shows up in github
> when you view the package.  This is a quick and easy way to see the
> current status.

I agree this is very useful.

What I dislike is that, when I open "Readme.md" (e.g. in the project
root) in a normal terminal, I'm greeted by a HTML tag soup under the
heading "# Build Status".

Markdown is supposed to be readable as plain text. Embedding the
page-ful of build status HTML in "Readme.md" defeats that purpose.
github should either consume a different file (too) for displaying
status badges, or else the "Readme.md" file should reference the HTML
snippet in question with some kind of link or directive, rather than
directly containing it.

Perhaps github already offers this feature -- that would be awesome. I
would be happy with the following variant, for example:

  ArmVirtPkg/ArmVirtPkg.ci.yaml
  ArmVirtPkg/PlatformCI/PlatformBuild.py
  ArmVirtPkg/PlatformCI/Ubuntu-GCC5.yml
  ArmVirtPkg/PlatformCI/iasl_ext_dep.yaml
  ArmVirtPkg/README-ci.md
  ArmVirtPkg/README.md

"README.md" would contain the package description that read nice in a
terminal too. Then, "README.md" would either (somehow?) include
"README-ci.md" by reference, or else github would render both
"README-ci.md" and "README.md".

>
> * Ovmf has a pretty stale readme

Hm, I'd say "somewhat" stale. We do keep it up-to-date with "very
important" stuff.

My excuse for not polishing it more -- which I honestly do believe is a
*valid* excuse -- is that users have shown repeatedly that they don't
read the README at all. I've explained basic stuff like "how to capture
OVMF's debug log" umpteen times on the list, despite it being spelled
out in the README. The fact is that effort put towards careful
documentation is almost entirely lost effort -- this was also clearly
proved by the (non-)reaction that I got to my OVMF white paper that I
wrote a few years back (~60 A4 pages, if I recall correctly).

Documentation is just not *worth* polishing, considering the user base
as a whole. I for one go to the available documentation *before*
starting to use new software, or when questions pop up, but it seems
like I belong to a vanishingly small camp with that. People just flock
to social media (or, in the least wrong case: they come to this mailing
list), and ask questions they could already find the answers to in
existent documentation.

This is a  bitter realization for me, especially having written
relatively substantial articles for the edk2 wiki:

  https://github.com/tianocore/tianocore.github.io/wiki/Laszlo's-unkempt-git-guide-for-edk2-contributors-and-maintainers
  https://github.com/tianocore/tianocore.github.io/wiki/Testing-SMM-with-QEMU,-KVM-and-libvirt

but it is what I now believe.

> and does not take advantage of markdown.

Correct. That's not intentional; I think the README just predates the
usefulness of markdown (i.e. it predates moving the project to
github.com). IIRC.

> We could convert it to MD, clean up, and then merge in the content
> from the pytools.md.  I would need help or a package maintainer to do
> the cleanup of the readme to make sure it contained the content you
> desired.

So my problem with this is two-fold. First, regarding just the markdown
conversion, I agree it would be nice, but I don't wish to sink any work
into it (see my opinion above, about polishing documentation). If
someone wanted to spend time on just a structural conversion to markdown
(not modifying content), I'd be OK to review that.

Second, merging (i.e., flattening) the tag soup from "README-pytools.md"
into the main package "Readme.md" is something that I'm opposed to, as
it interferes with consuming the readme from a plain terminal or text
editor.

> * ArmVirtPkg doesn't have a readme and this is definitely a barrier to
> entry for the package.  I would suggest creating one and then merging
> in the content from the pytools.md.

Creating a readme in MD format: would be nice if someone contributed
that ("patches welcome" :) ).

*Merging* the HTML tag soup: please let's not do that. (I'm totally fine
if it is introduced in a separate, appropriately named or located file.)

> * EmulatorPkg has one.  I would just suggest a merge but i am yet to
> get any feedback from those maintainers.
>
> If that isn't desirable i would at least suggest we change the title
> to just ReadMe.md so that GitHub shows it by default when the
> PaltformCI folder is viewed form the web or in editor like vscode.

This sounds 100% viable and great to me. I didn't expect this could
work! (I'm generally unaware of the readme filename patterns, and
locations, that github.com recognizes; sorry about that.) Having an
"unadorned" ReadMe.md file under PlatformCI is just perfect.

So if I understand correctly, we could choose:

  ArmVirtPkg/ArmVirtPkg.ci.yaml
  ArmVirtPkg/PlatformCI/PlatformBuild.py
  ArmVirtPkg/PlatformCI/ReadMe.md
  ArmVirtPkg/PlatformCI/Ubuntu-GCC5.yml
  ArmVirtPkg/PlatformCI/iasl_ext_dep.yaml

Do I understand right?

Because, I'd find this great!

Thank you!
Laszlo

Re: [edk2-devel] [PATCH v1 2/6] ArmVirtPkg: Add Platform CI and configuration for Core CI

[Laszlo Ersek]

Hi Sean,

On 04/15/20 21:31, sean.brogan via [] wrote:

> My corporate mail server changes the subject line which also destroys
> threading so I figured this was better.  Maybe not as i have no
> control over the CC/To lines of the email that is sent.   Do Bret's
> responses which add "[External]" (example here:
> https://edk2.groups.io/g/devel/message/57377) (
> https://edk2.groups.io/g/devel/message/57377 ) to every reply show up
> better (in groups.io web ui this is harder to follow since it starts
> a new thread)?

Thank you very much for providing Bret's email as an example!

Bret's answers in that thread, namely:

  https://edk2.groups.io/g/devel/message/57377
  https://edk2.groups.io/g/devel/message/57381

are well-threaded. Both in my personal list folder, as presented by my
MUA (Thunderbird), and also in e.g. the following (secondary) mailing
list archive:

  http://mid.mail-archive.com/CY4PR21MB074349715A82905EB420D7DBEFDA0@CY4PR21MB0743.namprd21.prod.outlook.com
  http://mid.mail-archive.com/CY4PR21MB07431322730AF342BF898A74EFDA0@CY4PR21MB0743.namprd21.prod.outlook.com

If you open each one of those "mail-archive.com" links in separate
browser windows or tabs, and scroll to the bottom of both pages, you'll
see that Bret's answer's are correctly threaded against the rest of the
messages in that discussion.

The subject line is not directly relevant for threading. Some MUAs use
the subject as a fallback option for reconstructing threads, but the
primary linkage for threading is the In-Reply-To header of the response
email -- which is supposed to reference the Message-Id header of the
original email.

So, for example, Rebecca's thread starter carried the header

  Message-ID: <62933ee8-7fb9-7cd3-08e5-6d6e7dac31e9@bsdio.com>

and Bret's reply correctly carried the header

  In-Reply-To: <62933ee8-7fb9-7cd3-08e5-6d6e7dac31e9@bsdio.com>

It doesn't matter (for threading) that Bret's infrastructure changed the
subject. Sometimes people intentionally change the subject line, while
staying in the same thread -- that signals that they go off on a
tangent, but still want to stay on topic a little (or else they want to
preserve the linkage to the original discussion). For example, the
original subject could be

  foo bar
    Re: foo bar
      Re: foo bar

and then someone could go off on a tangent (in the same thread!) like
this:

        quux [was: foo bar]
          Re: quux [was: foo bar]

So while the "[External]" stuff is not particularly useful or welcome,
regarding the list traffic, it is not a deal breaker. The threading does
remain correct. If the groups.io WebUI loses track of threading just
because of subject line changes, then that's a groups.io WebUI bug.

Bret's MUA also seems to correctly preserve CC's -- Rebecca's thread
starter CC'd Bob and Liming, and Bret's responses carry those CC's too.

If you could adopt Bret's way of interacting with the list, I think that
would be great!

Thanks!
Laszlo

Re: [edk2-devel] [PATCH v1 2/6] ArmVirtPkg: Add Platform CI and configuration for Core CI

[Sean]

Laszlo,

Regarding your comments about disliking the verbosity of the markdown table/html table for build status both in Core Ci and now these Platform CI readme files.  

As a learning experience I updated the OvmfPkg readme to use reStructuredText instead of markdown.  Not sure if I like RST but it does allow the links to not be in html and supports directives so you can push all that text to end of the file.  

Do you like this enough that I should rework all three readmes and we should discuss if RST should be used instead of MD for the edk2 project? 

RST version: https://github.com/spbrogan/edk2/blob/PlatformAndCoreCIForOvmfArmVirtEmulatorPackages_v8/OvmfPkg/README.rst 
MD version: https://github.com/spbrogan/edk2/blob/PlatformAndCoreCIForOvmfArmVirtEmulatorPackages_v7/OvmfPkg/README-pytools.md

This is really the last remaining issue for the PlatformCI patchset.

Thanks
Sean



-----Original Message-----
From: Laszlo Ersek <lersek@redhat.com> 
Sent: Thursday, April 16, 2020 7:52 AM
To: Sean Brogan <sean.brogan@microsoft.com>
Cc: devel@edk2.groups.io; Ard Biesheuvel <ard.biesheuvel@linaro.org>
Subject: [EXTERNAL] Re: [edk2-devel] [PATCH v1 2/6] ArmVirtPkg: Add Platform CI and configuration for Core CI

On 04/15/20 22:38, sean.brogan via [] wrote:
> On Wed, Apr 15, 2020 at 10:18 AM, Laszlo Ersek wrote:
>
>>
>> ArmVirtPkg/ArmVirtPkg.ci.yaml
>> ArmVirtPkg/PlatformCI/Ubuntu-GCC5.yml
>> ArmVirtPkg/PlatformCI/PlatformBuild.py
>> ArmVirtPkg/PlatformCI/README-pytools.md
>> ArmVirtPkg/PlatformCI/iasl_ext_dep.yaml
>
> I am ok with the above except one thought on the readme.  One nice 
> thing about the markdown readme files are the badge shows up in github 
> when you view the package.  This is a quick and easy way to see the 
> current status.

I agree this is very useful.

What I dislike is that, when I open "Readme.md" (e.g. in the project
root) in a normal terminal, I'm greeted by a HTML tag soup under the heading "# Build Status".

Markdown is supposed to be readable as plain text. Embedding the page-ful of build status HTML in "Readme.md" defeats that purpose.
github should either consume a different file (too) for displaying status badges, or else the "Readme.md" file should reference the HTML snippet in question with some kind of link or directive, rather than directly containing it.

Perhaps github already offers this feature -- that would be awesome. I would be happy with the following variant, for example:

  ArmVirtPkg/ArmVirtPkg.ci.yaml
  ArmVirtPkg/PlatformCI/PlatformBuild.py
  ArmVirtPkg/PlatformCI/Ubuntu-GCC5.yml
  ArmVirtPkg/PlatformCI/iasl_ext_dep.yaml
  ArmVirtPkg/README-ci.md
  ArmVirtPkg/README.md

"README.md" would contain the package description that read nice in a terminal too. Then, "README.md" would either (somehow?) include "README-ci.md" by reference, or else github would render both "README-ci.md" and "README.md".

>
> * Ovmf has a pretty stale readme

Hm, I'd say "somewhat" stale. We do keep it up-to-date with "very important" stuff.

My excuse for not polishing it more -- which I honestly do believe is a
*valid* excuse -- is that users have shown repeatedly that they don't read the README at all. I've explained basic stuff like "how to capture OVMF's debug log" umpteen times on the list, despite it being spelled out in the README. The fact is that effort put towards careful documentation is almost entirely lost effort -- this was also clearly proved by the (non-)reaction that I got to my OVMF white paper that I wrote a few years back (~60 A4 pages, if I recall correctly).

Documentation is just not *worth* polishing, considering the user base as a whole. I for one go to the available documentation *before* starting to use new software, or when questions pop up, but it seems like I belong to a vanishingly small camp with that. People just flock to social media (or, in the least wrong case: they come to this mailing list), and ask questions they could already find the answers to in existent documentation.

This is a  bitter realization for me, especially having written relatively substantial articles for the edk2 wiki:

  https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FLaszlo%27s-unkempt-git-guide-for-edk2-contributors-and-maintainers&amp;data=02%7C01%7Csean.brogan%40microsoft.com%7Cd94de877b9e74e06f08208d7e215bca4%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637226455298322408&amp;sdata=afDNhjdLt%2B9G4idYYGARh0RiTUnxCBx4fyBA5xT8k9Q%3D&amp;reserved=0
  https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FTesting-SMM-with-QEMU%2C-KVM-and-libvirt&amp;data=02%7C01%7Csean.brogan%40microsoft.com%7Cd94de877b9e74e06f08208d7e215bca4%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637226455298322408&amp;sdata=nUtyttk3BdcGKyTepkUy0OnILT7%2FcBnHCMyt5eAO%2BG8%3D&amp;reserved=0

but it is what I now believe.

> and does not take advantage of markdown.

Correct. That's not intentional; I think the README just predates the usefulness of markdown (i.e. it predates moving the project to github.com). IIRC.

> We could convert it to MD, clean up, and then merge in the content 
> from the pytools.md.  I would need help or a package maintainer to do 
> the cleanup of the readme to make sure it contained the content you 
> desired.

So my problem with this is two-fold. First, regarding just the markdown conversion, I agree it would be nice, but I don't wish to sink any work into it (see my opinion above, about polishing documentation). If someone wanted to spend time on just a structural conversion to markdown (not modifying content), I'd be OK to review that.

Second, merging (i.e., flattening) the tag soup from "README-pytools.md"
into the main package "Readme.md" is something that I'm opposed to, as it interferes with consuming the readme from a plain terminal or text editor.

> * ArmVirtPkg doesn't have a readme and this is definitely a barrier to 
> entry for the package.  I would suggest creating one and then merging 
> in the content from the pytools.md.

Creating a readme in MD format: would be nice if someone contributed that ("patches welcome" :) ).

*Merging* the HTML tag soup: please let's not do that. (I'm totally fine if it is introduced in a separate, appropriately named or located file.)

> * EmulatorPkg has one.  I would just suggest a merge but i am yet to 
> get any feedback from those maintainers.
>
> If that isn't desirable i would at least suggest we change the title 
> to just ReadMe.md so that GitHub shows it by default when the 
> PaltformCI folder is viewed form the web or in editor like vscode.

This sounds 100% viable and great to me. I didn't expect this could work! (I'm generally unaware of the readme filename patterns, and locations, that github.com recognizes; sorry about that.) Having an "unadorned" ReadMe.md file under PlatformCI is just perfect.

So if I understand correctly, we could choose:

  ArmVirtPkg/ArmVirtPkg.ci.yaml
  ArmVirtPkg/PlatformCI/PlatformBuild.py
  ArmVirtPkg/PlatformCI/ReadMe.md
  ArmVirtPkg/PlatformCI/Ubuntu-GCC5.yml
  ArmVirtPkg/PlatformCI/iasl_ext_dep.yaml

Do I understand right?

Because, I'd find this great!

Thank you!
Laszlo

Re: [edk2-devel] [PATCH v1 2/6] ArmVirtPkg: Add Platform CI and configuration for Core CI

[Ard Biesheuvel]

On 4/19/20 10:29 AM, Sean via groups.io wrote:
> Laszlo,
> 
> Regarding your comments about disliking the verbosity of the markdown table/html table for build status both in Core Ci and now these Platform CI readme files.
> 
> As a learning experience I updated the OvmfPkg readme to use reStructuredText instead of markdown.  Not sure if I like RST but it does allow the links to not be in html and supports directives so you can push all that text to end of the file.
> 
> Do you like this enough that I should rework all three readmes and we should discuss if RST should be used instead of MD for the edk2 project?
> 
> RST version: https://github.com/spbrogan/edk2/blob/PlatformAndCoreCIForOvmfArmVirtEmulatorPackages_v8/OvmfPkg/README.rst
> MD version: https://github.com/spbrogan/edk2/blob/PlatformAndCoreCIForOvmfArmVirtEmulatorPackages_v7/OvmfPkg/README-pytools.md
> 

I think the rendered RST looks really good, and is perfectly usable as 
plaintext too. AFAICT, we are not losing out on anything the v7 version 
provided, right? If not, I am in favor of going with RST in this case, 
which I suppose implies that I am in favor of permitting it in EDK2 for 
purposes such as this one.

That does not necessarily imply that all markdown should be replaced 
with RST. I'm perfectly happy letting them co-exist, and I don't have an 
opinion on whether one should be recommended over the other, or which 
that should have to be.

> This is really the last remaining issue for the PlatformCI patchset.
> 

Agreed. This is converging really nicely.

-- 
Ard.

Re: [edk2-devel] [PATCH v1 2/6] ArmVirtPkg: Add Platform CI and configuration for Core CI

[Rebecca Cran]

> -----Original Message-----
> From: Laszlo Ersek <lersek@redhat.com>
> Sent: Thursday, April 16, 2020 7:52 AM
> To: Sean Brogan <sean.brogan@microsoft.com>
> Cc: devel@edk2.groups.io; Ard Biesheuvel <ard.biesheuvel@linaro.org>
> Subject: [EXTERNAL] Re: [edk2-devel] [PATCH v1 2/6] ArmVirtPkg: Add Platform CI and configuration for Core CI
>
>
> My excuse for not polishing it more -- which I honestly do believe is a
> *valid* excuse -- is that users have shown repeatedly that they don't read the README at all. I've explained basic stuff like "how to capture OVMF's debug log" umpteen times on the list, despite it being spelled out in the README. The fact is that effort put towards careful documentation is almost entirely lost effort -- this was also clearly proved by the (non-)reaction that I got to my OVMF white paper that I wrote a few years back (~60 A4 pages, if I recall correctly).
>
> Documentation is just not *worth* polishing, considering the user base as a whole. I for one go to the available documentation *before* starting to use new software, or when questions pop up, but it seems like I belong to a vanishingly small camp with that. People just flock to social media (or, in the least wrong case: they come to this mailing list), and ask questions they could already find the answers to in existent documentation.
>
> This is a  bitter realization for me, especially having written relatively substantial articles for the edk2 wiki:
>
>    https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FLaszlo%27s-unkempt-git-guide-for-edk2-contributors-and-maintainers&amp;data=02%7C01%7Csean.brogan%40microsoft.com%7Cd94de877b9e74e06f08208d7e215bca4%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637226455298322408&amp;sdata=afDNhjdLt%2B9G4idYYGARh0RiTUnxCBx4fyBA5xT8k9Q%3D&amp;reserved=0
>    https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FTesting-SMM-with-QEMU%2C-KVM-and-libvirt&amp;data=02%7C01%7Csean.brogan%40microsoft.com%7Cd94de877b9e74e06f08208d7e215bca4%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637226455298322408&amp;sdata=nUtyttk3BdcGKyTepkUy0OnILT7%2FcBnHCMyt5eAO%2BG8%3D&amp;reserved=0
>
> but it is what I now believe.

Where are those pages linked from (i.e. how would people find them)? The 
Github wiki just confuses me: I don't see how it's useful for a project 
as large/complex as TianoCore.

Compare it to https://wiki.freebsd.org/, where there's a large table of 
contents right on the front page.  And then clicking on 'UEFI', you go 
to https://wiki.freebsd.org/UEFI where there's another table of contents 
- and has the breadcrumb navigation. It feels much easier to use to me 
at least.


And having looked through the OVMF README file in the past, I recall it 
being pretty difficult to find information in. It's a little long, and 
with it all being plain text it's not very easy to navigate.


-- 

Rebecca Cran

Re: [edk2-devel] [PATCH v1 2/6] ArmVirtPkg: Add Platform CI and configuration for Core CI

[Laszlo Ersek]

On 04/19/20 11:35, Ard Biesheuvel wrote:
> On 4/19/20 10:29 AM, Sean via groups.io wrote:
>> Laszlo,
>>
>> Regarding your comments about disliking the verbosity of the markdown
>> table/html table for build status both in Core Ci and now these
>> Platform CI readme files.
>>
>> As a learning experience I updated the OvmfPkg readme to use
>> reStructuredText instead of markdown.  Not sure if I like RST but it
>> does allow the links to not be in html and supports directives so you
>> can push all that text to end of the file.
>>
>> Do you like this enough that I should rework all three readmes and we
>> should discuss if RST should be used instead of MD for the edk2 project?
>>
>> RST version:
>> https://github.com/spbrogan/edk2/blob/PlatformAndCoreCIForOvmfArmVirtEmulatorPackages_v8/OvmfPkg/README.rst
>>
>> MD version:
>> https://github.com/spbrogan/edk2/blob/PlatformAndCoreCIForOvmfArmVirtEmulatorPackages_v7/OvmfPkg/README-pytools.md
>>
>>
> 
> I think the rendered RST looks really good, and is perfectly usable as
> plaintext too. AFAICT, we are not losing out on anything the v7 version
> provided, right? If not, I am in favor of going with RST in this case,
> which I suppose implies that I am in favor of permitting it in EDK2 for
> purposes such as this one.
> 
> That does not necessarily imply that all markdown should be replaced
> with RST. I'm perfectly happy letting them co-exist, and I don't have an
> opinion on whether one should be recommended over the other, or which
> that should have to be.
> 
>> This is really the last remaining issue for the PlatformCI patchset.
>>
> 
> Agreed. This is converging really nicely.
> 

Agreed 100% on all counts.

Thanks
Laszlo

Re: [edk2-devel] [PATCH v1 2/6] ArmVirtPkg: Add Platform CI and configuration for Core CI

[Laszlo Ersek]

On 04/19/20 22:56, Rebecca Cran wrote:

> Where are those pages linked from (i.e. how would people find them)?

I've used the following three methods to find content in the wiki:

(1) web search

(2) if you open <https://github.com/tianocore/tianocore.github.io/wiki>
and click on the "Pages" box to the right, you get a small search field.

(3) grep a local clone of the wiki.

Not implying that these are convenient or efficient.

Thanks
Laszlo

Re: [EXTERNAL] Re: [edk2-devel] [PATCH v1 2/6] ArmVirtPkg: Add Platform CI and configuration for Core CI

[Bret Barkelew]

[-- Attachment #1: Type: text/plain, Size: 1359 bytes --]

Reviewed-by: Bret Barkelew <bret.barkelew@microsoft.com>

- Bret

________________________________
From: devel@edk2.groups.io <devel@edk2.groups.io> on behalf of Laszlo Ersek via groups.io <lersek=redhat.com@groups.io>
Sent: Monday, April 20, 2020 4:08:53 AM
To: Rebecca Cran <rebecca@bsdio.com>; devel@edk2.groups.io <devel@edk2.groups.io>; Sean Brogan <sean.brogan@microsoft.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>; Kinney, Michael D <michael.d.kinney@intel.com>
Subject: [EXTERNAL] Re: [edk2-devel] [PATCH v1 2/6] ArmVirtPkg: Add Platform CI and configuration for Core CI

On 04/19/20 22:56, Rebecca Cran wrote:

> Where are those pages linked from (i.e. how would people find them)?

I've used the following three methods to find content in the wiki:

(1) web search

(2) if you open <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki&amp;data=02%7C01%7Cbret.barkelew%40microsoft.com%7C32def7f7f4f0408ce52908d7e51b3ca9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637229777459482694&amp;sdata=%2BEw9TO%2F3VQATva2nbPNRMZDq7I3fBHx9YqOJvGIpX7k%3D&amp;reserved=0>
and click on the "Pages" box to the right, you get a small search field.

(3) grep a local clone of the wiki.

Not implying that these are convenient or efficient.

Thanks
Laszlo





[-- Attachment #2: Type: text/html, Size: 3502 bytes --]