* Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
@ 2021-11-30 22:34 Michael D Kinney
2021-12-01 1:42 ` Wu, Hao A
` (9 more replies)
0 siblings, 10 replies; 39+ messages in thread
From: Michael D Kinney @ 2021-11-30 22:34 UTC (permalink / raw)
To: devel@edk2.groups.io, Kinney, Michael D, Michael Kubacki,
Andrew Fish (afish@apple.com), Leif Lindholm
Hello,
Thank you for your patience during this extended hard freeze.
Just one more step to go. There has been a delay in the review of
the patch series with the uncrustify source changes. PR(6). This
patch series was not sent out as patch review email because of its
very large size. It only contains source style changes and the
CompareBuild tool and GitHub action has shown there are no binary
differences introduced with these source style changes.
If you are a package maintainer, then please review the following
branch/PR for your package contents and review the EDK II CI results
and BuildCompare results. I do not expect a line by line review
because we already had time to provide feedback on the source style
performed by uncrustify. Instead, a Reviewed-by for your package
indicates that you have reviewed the EDK II CI results and CompareBuild
tool functionality and results and you accept the source style
changes to your package.
* https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
* https://github.com/tianocore/edk2/pull/2229
* https://github.com/mdkinney/edk2/actions/runs/1521618836
Additional details on this update below.
Thank you,
Mike
Changes from Update #3
----------------------------------------------------------------------------
* Pushed PR (5)
* Added link to PR(6). EDK II CI Status is PASS. Build Compare PASS.
* Waiting for review of PR (6)
* Review of PR (7) completed and waiting for review of PR (6)
----------------------------------------------------------------------------
Changes from Update #2
----------------------------------------------------------------------------
* Changed order of PRs swapping (4) and (5). The PR that activates
increases the max CI agent job time is independent of all the other
PRs and its review is complete, so it can be committed now.
* Pushed PRs (1), (2), (3), (4).
* Waiting for review to complete for PRs (5) and (6)
* Reviews complete for PR (7)
* Identifies steps using git filter-branch to apply uncrustify changes to a
code review patch series that was generated before the uncrustify changes
avoiding manual merge.
* Identified steps using git filter-repo to generate an alternate history of
the edk2 repo with uncrustify changes applied on every commit. This may
be useful when evaluating changes to files using tools like git blame
without the large uncrustify patch series.
---------------------------------------------------------------------------
Changes from Update #1
----------------------------------------------------------------------------
* Changed order of PRs swapping (6) and (7). The PR that activates
EDK II CI check UncrustifyCheck has to be last because it unconditionally
checks all C/H files in all packages. Not just files that have been
modified like some of the other checkers.
* Updated link to the branch with the UncrustifyCheck plugin that has been
updated with a one line change and Reviewed-by and Tested-by tags.
https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v6
* Reviews complete for (1), (2), (3), (5), and (7)
---------------------------------------------------------------------------
Michael Kubacki and I have prepared the patches required to apply the
uncrustify changes and enable EDK II CI to check all submitted
patches have been run through uncrustify.
We have verified through the CompareBuild GitHub Action that the
format changes performed by uncrustify have no functional changes.
All of the OBJ, LIB, DLL, EFI, FFS, FV, and FD files match 100%
across 70 VS2019/GCC5 builds of all package/platform DSC files in
the edk2 repo.
The hard freeze will be extended after the edk2-stable202111 tag until
all uncrustify related changes are committed. We do not expect this
to take more than a few days. Do not push any PRs until the hard
freeze is lifted.
The changes are broken up into 7 patch series/PRs. The PRs are ordered
so they can be submitted using the normal submission process and EDK II
CI will pass for each one. Details are listed below.
Uncrustify 73.0.3 for EDK II
=============================
* Sources: https://dev.azure.com/projectmu/_git/Uncrustify
* Documentation: https://dev.azure.com/projectmu/Uncrustify/_wiki/wikis/Uncrustify.wiki/1/Project-Mu-(EDK-II)-Fork-Readme
* Download: https://dev.azure.com/projectmu/Uncrustify/_packaging?_a=package&feed=mu_uncrustify&package=mu-uncrustify-release&protocolType=NuGet&version=73.0.3
Installing Uncrustify
======================
The Uncrustify tool is installed automatically when the Pytools
environment is used and the stuart* commands are run to complete the
environment setup. Please see:
https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally
Uncrustify can also be installed from the download page listed above
or built from sources from the source link above.
The Documentation link provides instruction on how to run uncrustify from
the command line or install as a Visual Studio Code plugin. The main
uncrustify documentation also describes how to integrate with a few other
editors.
We have also discussed a client side githook. That effort has not started.
Let us know if that is a feature you would find useful.
Developer impact for new code reviews
======================================
Once the uncrustify checker is active in EDK II CI, developers must
make sure their patches are run through the uncrustify tool before
sending the patches for review.
Developers must install and run uncrustify against changes files before
sending patch review emails or submitting PR for EDK II CI. If EDK II CI
detects and differences in source formatting, then EDK II CI will fail
and the developer must run uncrustify and resubmit the patches.
Developer impact to patch series/PRs reviewed during edk2-stable201121 soft/hard freeze
=======================================================================================
Developers must rebase their changes after the uncrustify source changes are
committed. The branch with a preview of the uncrustify changes can be used
to start this rebase work.
https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
The following steps can be used to update an existing branch with the
required uncrustify format. This is the Windows version. I will add
the Linux version soon.
1) Fetch and checkout and rebase to latest edk2/master
git fetch origin
git checkout master
git rebase origin/master
2) Make a backup copy of plugin UncrustifyCheck outside WORKSPACE.
(e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable and
EDK II specific uncrustify configuration file available when working
with a branch that does not have those tools in its scope.
xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
3) Check out the patch series branch (e.g. MyBranch)
git checkout MyBranch
4) Rebase patch series against edk2-stable202111
git rebase edk2-stable202111
5) Create new branch for the uncrustifed version (e.g. MyBranch_Uncrustified)
git checkout -b MyBranch_Uncrustified
6) Use git filter-branch to uncrustify all the commits in the series
between the rebase target from (2) and HEAD of the branch. A filter
can be used to scope the uncrustify operations to only the C/H files
in the specific package the patch series is against. (e.g. DynamicTablesPkg).
BaseTools should always be excluded. If the package scoped filter is
not used, it will still work, but will take longer to run because
uncrustify will rescan every C/H files in the whole repo.
git filter-branch --tree-filter "git ls-files DynamicTablesPkg*.c DynamicTablesPkg*.h :!BaseTools/* | c:\\Temp\\UncrustifyCheck\\mu-uncrustify-release_extdep\\Windows-x86\\uncrustify.exe -c c:\\Temp\\UncrustifyCheck\\uncrustify.cfg -F - --replace --no-backup --if-changed" edk2-stable202111..HEAD
7) Now that all the individual patches in the branch are uncrustified,
rebase against latest edk2/master that is already uncrustified.
git rebase master
8) Verify the patches in this new branch.
Impacts to tracing history across the uncrusity changes
=======================================================
Tools the view file and line history do work with the large uncrustify
patch series. One impact is that the operations can be very slow due
to the large uncrustify patches.
One option to provide a faster experience is to provide an alternate
version of the edk2 repository as "documentation" that has the
entire history re-written with uncrustify run on every commit.
The tool called git-filter-repo can be used to perform this
transformation and runs in a reasonable period of time (a few hours)
https://github.com/newren/git-filter-repo
https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-demos/lint-history
The following steps can be used to perform this transformation.
This is the Windows version. I will add the Linux version soon.
** WARNING ** This operation modifies(rewrites) all the commits
in the local copy of the repo. Do not perform
these steps on a local repo you are using for
active development.
1) Clone edk2 into a new directory (see **WARNING**)
git clone https://github.com/tianocore/edk2.git edk2-uncrustified
cd edk2-uncrustified
2) Setup python virtual env, install pytools, and run stuart commands
to setup build environment which includes installing uncrustify tools.
https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally
3) Make a backup copy of plugin UncrustifyCheck outside WORKSPACE.
(e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable and
EDK II specific uncrustify configuration file available when working
with a branch that does not have those tools in its scope.
xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
4) Use lint-history.py from git-filter-repo examples
https://github.com/newren/git-filter-repo
https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-demos/lint-history
Line #127 - Add try except around subprocess.check_call() with except
being pass. This is required because there are a few commits of C
files in the edk2 repo that have incorrect C syntax and do not
build with a C compiler and break the uncrustify parser. Skip reformat
of C files that can not be parsed by uncrustify. These rare instances
are addressed in the commit that fixes the C syntax error.
Run this slightly modified version of lint-history. Include only
C/H files and exclude directories that start with 'Tools' or 'BaseTools'.
This step took about 2.2 hours on a laptop.
lint-history.py
--relevant "return (not filename.startswith(b'Tools') and not filename.startswith(b'BaseTools') and (filename.endswith(b'.c') or filename.endswith(b'.h')))"
c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\mu-uncrustify-release_extdep\\Windows-x86\\uncrustify.exe -c c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\uncrustify.cfg --replace --no-backup --if-changed
Order of PRs to apply during extended hard freeze
==================================================
1) Update EmulatorPkg Win Host [BuildOptions] MSFT CC_FLAGS to not force debug information
* https://bugzilla.tianocore.org/show_bug.cgi?id=3747
* https://github.com/mdkinney/edk2/tree/Bug_3747_EmulatorPkg_WinHost_ReproducibleBuild
* https://github.com/tianocore/edk2/pull/2215
* Required for EmulatorPkg to pass CompareBuild for VS2019 IA32/X64 builds.
* Status: Review complete. PR pushed.
2) EccCheck should not revert staged and local changes
* https://bugzilla.tianocore.org/show_bug.cgi?id=2986
* https://github.com/mdkinney/edk2/tree/Bug_2986_EccCheckRemoveGitRevert_V2
* https://github.com/tianocore/edk2/pull/2216
* Required for EDK II CI to complete in a reasonable period of time when
processing the 4000+ source file style changes made by uncrustify.
* Also fixes critical bugs that can potentially corrupt git state when
EccCheck is run locally.
* Status: Review complete. PR pushed.
3) Update pytool LicenseCheck plugin to use temp directory for diff output file
* https://bugzilla.tianocore.org/show_bug.cgi?id=3746
* https://github.com/mdkinney/edk2/tree/Bug_3746_LicenseCheckUseDiffOutputFile_V2
* https://github.com/tianocore/edk2/pull/2217
* Required to reduce EDK II CI build times.
* Status: Review complete. PR pushed.
4) Update max job time from 60 min to 120 minutes in .azurepipelines/templates
* https://bugzilla.tianocore.org/show_bug.cgi?id=3750
* https://github.com/mdkinney/edk2/tree/Bug_3750_IncreaseAzurePipelinesTimeout
* https://github.com/tianocore/edk2/pull/2219
* Required to allow EccCheck of uncrustify changes to complete on Azure
Pipelines CI agents without timing out.
* Status: Review complete. PR pushed.
5) Update Package YAML to ignore specific ECC files/errors
* https://bugzilla.tianocore.org/show_bug.cgi?id=3749
* https://github.com/mdkinney/edk2/tree/Bug_3749_EccCheckIgnoreFilesErrors
* https://github.com/tianocore/edk2/pull/2218
* Required to pass EccCheck
* Status: Review complete. PR pushed
6) Uncrustify Source Changes
* https://bugzilla.tianocore.org/show_bug.cgi?id=3737
* https://bugzilla.tianocore.org/show_bug.cgi?id=3739
* https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
* https://github.com/tianocore/edk2/pull/2229
* Build comparison result PASS: https://github.com/mdkinney/edk2/actions/runs/1521618836
* EFI_D_ -> DEBUG changes required to pass PatchCheck
* Uncrustify format changes required to pass UncrustifyCheck
* Status:
Waiting for review
7) UncrustifyCheck EDK II CI Plugin
* https://bugzilla.tianocore.org/show_bug.cgi?id=3748
* https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v6
* Required to enforce all PRs submitted to EDK II CI match uncrustify format.
* Unconditionally checks all packages. Can not be committed until all C/H
source files have been updated.
* Status: Review complete
Combined Branch/PR for Review/Test
==================================
* Build Comparison results must pass 100% across the full set of PRs before
the individual PRs can be pushed in the order listed above.
* Branch: https://github.com/mdkinney/edk2/tree/TestOnly_Uncrustify_PR_Series
* PR: https://github.com/tianocore/edk2/pull/2229
Status = PASS
* CompareBuild:
Branch: https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
--ref1: ef9a059cdb15844fe52a49af2bf7d86b9dd3e9bf
--ref2: Bug_3737_3739_ApplyUncrustifyChanges_V5
Extra Options: -n 4 --quiet
Results: https://github.com/mdkinney/edk2/actions/runs/1521618836
30 VS2019 build comparisons PASS
40 GCC5 build comparisons PASS
100% PASS
The following git log shows the set of patches from --ref1 to --ref 2across
which there are no differences in any of the OBJ/LIB/DLL/EFI/FFS/FV/FD files.
--ref2
b7d4bf0675b7 (HEAD -> Bug_3737_3739_ApplyUncrustifyChanges_V5) UnitTestFrameworkPkg: Apply uncrusitify changes
7f03d25f60e7 UefiPayloadPkg: Apply uncrusitify changes
0bfd8d9b5ac9 UefiCpuPkg: Apply uncrusitify changes
e1cd9bfb9dea StandaloneMmPkg: Apply uncrusitify changes
5da2f65be378 SourceLevelDebugPkg: Apply uncrusitify changes
95b86de07e5d SignedCapsulePkg: Apply uncrusitify changes
fe71d97246c4 ShellPkg: Apply uncrusitify changes
54c21c952992 SecurityPkg: Apply uncrusitify changes
187a3785f12b RedfishPkg: Apply uncrusitify changes
810100002a46 PcAtChipsetPkg: Apply uncrusitify changes
276a695c0cf2 OvmfPkg: Apply uncrusitify changes
303c0a91ab07 NetworkPkg: Apply uncrusitify changes
bc80792cd1b1 MdePkg: Apply uncrusitify changes
3ea86be17a2a MdeModulePkg: Apply uncrusitify changes
c70ef11ed0cd IntelFsp2WrapperPkg: Apply uncrusitify changes
c0291221f252 IntelFsp2Pkg: Apply uncrusitify changes
6a479952a690 FmpDevicePkg: Apply uncrusitify changes
3a7c05b7070d FatPkg: Apply uncrusitify changes
b789f98c8959 EmulatorPkg: Apply uncrusitify changes
952d7a1c9220 EmbeddedPkg: Apply uncrusitify changes
a1cc9881bab6 DynamicTablesPkg: Apply uncrusitify changes
50654dfe5785 CryptoPkg: Apply uncrusitify changes
ed965a02dfa1 ArmVirtPkg: Apply uncrusitify changes
9744023fbc46 ArmPlatformPkg: Apply uncrusitify changes
7a1cde5f5bba ArmPkg: Apply uncrusitify changes
19d17e0913e8 UefiCpuPkg: Change use of EFI_D_* to DEBUG_*
ffa718b4f994 SourceLevelDebugPkg: Change use of EFI_D_* to DEBUG_*
b86cb3c5e5b4 ShellPkg: Change use of EFI_D_* to DEBUG_*
c7c42204dc07 SecurityPkg: Change use of EFI_D_* to DEBUG_*
16b8e6f958e4 PcAtChipsetPkg: Change use of EFI_D_* to DEBUG_*
0ac3f8b2dac5 OvmfPkg: Change use of EFI_D_* to DEBUG_*
bc5004b8d294 NetworkPkg: Change use of EFI_D_* to DEBUG_*
6f671a8e2377 MdePkg: Change use of EFI_D_* to DEBUG_*
a10c610ff9a3 MdeModulePkg: Change use of EFI_D_* to DEBUG_*
09a3bddba390 FatPkg: Change use of EFI_D_* to DEBUG_*
59c61318246a EmulatorPkg: Change use of EFI_D_* to DEBUG_*
3a80367dda3b EmbeddedPkg: Change use of EFI_D_* to DEBUG_*
23eb1aaf80ca ArmVirtPkg: Change use of EFI_D_* to DEBUG_*
875914b45c54 ArmPlatformPkg: Change use of EFI_D_* to DEBUG_*
eb2eca82b451 ArmPkg: Change use of EFI_D_* to DEBUG_*
f0f3f5aae7c4 (origin/master, origin/HEAD, master) UnitTestFrameworkPkg: Update YAML to ignore specific ECC files/errors
c05734797790 UefiPayloadPkg: Update YAML to ignore specific ECC files/errors
c30c40d6c63d StandaloneMmPkg: Update YAML to ignore specific ECC files/errors
9944508e85f1 ShellPkg: Update YAML to ignore specific ECC files/errors
60fa40be458d SecurityPkg: Update YAML to ignore specific ECC files/errors
df790cd6b37e MdePkg: Update YAML to ignore specific ECC files/errors
9deb9370766e MdeModulePkg: Update YAML to ignore specific ECC files/errors
d7d30e8f219f EmulatorPkg: Update YAML to ignore specific ECC files/errors
d5744ecba813 CryptoPkg: Update YAML to ignore specific ECC files/errors
c97fee87f0f9 ArmVirtPkg: Update YAML to ignore specific ECC files/errors
1939fc9569f2 ArmPlatformPkg: Update YAML to ignore specific ECC files/errors
365dced2c37a ArmPkg: Update YAML to ignore specific ECC files/errors
76a1ce4d5fec .azurepipelines/templates: Update max pipeline job time to 2 hours
99f84ff47390 .pytools/Plugin/LicenseCheck: Use temp directory for git diff output
3019f1bbabf1 .pytool/Plugin/EccCheck: Add performance optimizations
854462bd3479 .pytool/Plugin/EccCheck: Remove temp directory on exception
69877614fdee .pytool/Plugin/EccCheck: Remove RevertCode()
--ref1
ef9a059cdb15 EmulatorPkg/Win/Host: Update CC_FLAGS
bb1bba3d7767 (tag: edk2-stable202111) NetworkPkg: Fix invalid pointer for DNS response token on error
Best regards,
Mike
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
2021-11-30 22:34 Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4 Michael D Kinney
@ 2021-12-01 1:42 ` Wu, Hao A
2021-12-01 2:10 ` 回复: [edk2-devel] " gaoliming
` (8 subsequent siblings)
9 siblings, 0 replies; 39+ messages in thread
From: Wu, Hao A @ 2021-12-01 1:42 UTC (permalink / raw)
To: devel@edk2.groups.io, Kinney, Michael D, Michael Kubacki,
Andrew Fish (afish@apple.com), Leif Lindholm
> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Michael
> D Kinney
> Sent: Wednesday, December 1, 2021 6:34 AM
> To: devel@edk2.groups.io; Kinney, Michael D <michael.d.kinney@intel.com>;
> Michael Kubacki <michael.kubacki@microsoft.com>; Andrew Fish
> (afish@apple.com) <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>
> Subject: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended
> Hard Freeze Update #4
>
> Hello,
>
> Thank you for your patience during this extended hard freeze.
>
> Just one more step to go. There has been a delay in the review of the patch
> series with the uncrustify source changes. PR(6). This patch series was not
> sent out as patch review email because of its very large size. It only contains
> source style changes and the CompareBuild tool and GitHub action has
> shown there are no binary differences introduced with these source style
> changes.
>
> If you are a package maintainer, then please review the following branch/PR
> for your package contents and review the EDK II CI results and BuildCompare
> results. I do not expect a line by line review because we already had time to
> provide feedback on the source style performed by uncrustify. Instead, a
> Reviewed-by for your package indicates that you have reviewed the EDK II CI
> results and CompareBuild tool functionality and results and you accept the
> source style changes to your package.
For SourceLevelDebugPkg:
Reviewed-by: Hao A Wu <hao.a.wu@intel.com>
Best Regards,
Hao Wu
>
> *
> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyC
> hanges_V5
> * https://github.com/tianocore/edk2/pull/2229
> * https://github.com/mdkinney/edk2/actions/runs/1521618836
>
> Additional details on this update below.
>
> Thank you,
>
> Mike
>
>
> Changes from Update #3
> ----------------------------------------------------------------------------
> * Pushed PR (5)
> * Added link to PR(6). EDK II CI Status is PASS. Build Compare PASS.
> * Waiting for review of PR (6)
> * Review of PR (7) completed and waiting for review of PR (6)
> ----------------------------------------------------------------------------
>
> Changes from Update #2
> ----------------------------------------------------------------------------
> * Changed order of PRs swapping (4) and (5). The PR that activates
> increases the max CI agent job time is independent of all the other
> PRs and its review is complete, so it can be committed now.
> * Pushed PRs (1), (2), (3), (4).
> * Waiting for review to complete for PRs (5) and (6)
> * Reviews complete for PR (7)
> * Identifies steps using git filter-branch to apply uncrustify changes to a
> code review patch series that was generated before the uncrustify changes
> avoiding manual merge.
> * Identified steps using git filter-repo to generate an alternate history of
> the edk2 repo with uncrustify changes applied on every commit. This may
> be useful when evaluating changes to files using tools like git blame
> without the large uncrustify patch series.
> ---------------------------------------------------------------------------
>
> Changes from Update #1
> ----------------------------------------------------------------------------
> * Changed order of PRs swapping (6) and (7). The PR that activates
> EDK II CI check UncrustifyCheck has to be last because it unconditionally
> checks all C/H files in all packages. Not just files that have been
> modified like some of the other checkers.
> * Updated link to the branch with the UncrustifyCheck plugin that has been
> updated with a one line change and Reviewed-by and Tested-by tags.
>
> https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plug
> in_v6
> * Reviews complete for (1), (2), (3), (5), and (7)
> ---------------------------------------------------------------------------
>
> Michael Kubacki and I have prepared the patches required to apply the
> uncrustify changes and enable EDK II CI to check all submitted patches have
> been run through uncrustify.
>
> We have verified through the CompareBuild GitHub Action that the format
> changes performed by uncrustify have no functional changes.
> All of the OBJ, LIB, DLL, EFI, FFS, FV, and FD files match 100% across 70
> VS2019/GCC5 builds of all package/platform DSC files in the edk2 repo.
>
> The hard freeze will be extended after the edk2-stable202111 tag until all
> uncrustify related changes are committed. We do not expect this to take
> more than a few days. Do not push any PRs until the hard freeze is lifted.
>
> The changes are broken up into 7 patch series/PRs. The PRs are ordered so
> they can be submitted using the normal submission process and EDK II CI will
> pass for each one. Details are listed below.
>
> Uncrustify 73.0.3 for EDK II
> =============================
> * Sources: https://dev.azure.com/projectmu/_git/Uncrustify
> * Documentation:
> https://dev.azure.com/projectmu/Uncrustify/_wiki/wikis/Uncrustify.wiki/1/
> Project-Mu-(EDK-II)-Fork-Readme
> * Download:
> https://dev.azure.com/projectmu/Uncrustify/_packaging?_a=package&fee
> d=mu_uncrustify&package=mu-uncrustify-
> release&protocolType=NuGet&version=73.0.3
>
> Installing Uncrustify
> ======================
> The Uncrustify tool is installed automatically when the Pytools environment
> is used and the stuart* commands are run to complete the environment
> setup. Please see:
>
> https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally
>
> Uncrustify can also be installed from the download page listed above or built
> from sources from the source link above.
>
> The Documentation link provides instruction on how to run uncrustify from
> the command line or install as a Visual Studio Code plugin. The main
> uncrustify documentation also describes how to integrate with a few other
> editors.
>
> We have also discussed a client side githook. That effort has not started.
> Let us know if that is a feature you would find useful.
>
> Developer impact for new code reviews
> ======================================
> Once the uncrustify checker is active in EDK II CI, developers must make sure
> their patches are run through the uncrustify tool before sending the patches
> for review.
>
> Developers must install and run uncrustify against changes files before
> sending patch review emails or submitting PR for EDK II CI. If EDK II CI detects
> and differences in source formatting, then EDK II CI will fail and the
> developer must run uncrustify and resubmit the patches.
>
> Developer impact to patch series/PRs reviewed during edk2-stable201121
> soft/hard freeze
> ==========================================================
> =============================
> Developers must rebase their changes after the uncrustify source changes
> are committed. The branch with a preview of the uncrustify changes can be
> used to start this rebase work.
>
>
> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyC
> hanges_V5
>
> The following steps can be used to update an existing branch with the
> required uncrustify format. This is the Windows version. I will add the Linux
> version soon.
>
> 1) Fetch and checkout and rebase to latest edk2/master
>
> git fetch origin
> git checkout master
> git rebase origin/master
>
> 2) Make a backup copy of plugin UncrustifyCheck outside WORKSPACE.
> (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable and
> EDK II specific uncrustify configuration file available when working
> with a branch that does not have those tools in its scope.
>
> xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
>
> 3) Check out the patch series branch (e.g. MyBranch)
>
> git checkout MyBranch
>
> 4) Rebase patch series against edk2-stable202111
>
> git rebase edk2-stable202111
>
> 5) Create new branch for the uncrustifed version (e.g.
> MyBranch_Uncrustified)
>
> git checkout -b MyBranch_Uncrustified
>
> 6) Use git filter-branch to uncrustify all the commits in the series
> between the rebase target from (2) and HEAD of the branch. A filter
> can be used to scope the uncrustify operations to only the C/H files
> in the specific package the patch series is against. (e.g.
> DynamicTablesPkg).
> BaseTools should always be excluded. If the package scoped filter is
> not used, it will still work, but will take longer to run because
> uncrustify will rescan every C/H files in the whole repo.
>
> git filter-branch --tree-filter "git ls-files DynamicTablesPkg*.c
> DynamicTablesPkg*.h :!BaseTools/* | c:\\Temp\\UncrustifyCheck\\mu-
> uncrustify-release_extdep\\Windows-x86\\uncrustify.exe -c
> c:\\Temp\\UncrustifyCheck\\uncrustify.cfg -F - --replace --no-backup --if-
> changed" edk2-stable202111..HEAD
>
> 7) Now that all the individual patches in the branch are uncrustified,
> rebase against latest edk2/master that is already uncrustified.
>
> git rebase master
>
> 8) Verify the patches in this new branch.
>
> Impacts to tracing history across the uncrusity changes
> =======================================================
> Tools the view file and line history do work with the large uncrustify patch
> series. One impact is that the operations can be very slow due to the large
> uncrustify patches.
>
> One option to provide a faster experience is to provide an alternate version
> of the edk2 repository as "documentation" that has the entire history re-
> written with uncrustify run on every commit.
> The tool called git-filter-repo can be used to perform this transformation and
> runs in a reasonable period of time (a few hours)
>
> https://github.com/newren/git-filter-repo
> https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-
> demos/lint-history
>
> The following steps can be used to perform this transformation.
> This is the Windows version. I will add the Linux version soon.
>
> ** WARNING ** This operation modifies(rewrites) all the commits
> in the local copy of the repo. Do not perform
> these steps on a local repo you are using for
> active development.
>
> 1) Clone edk2 into a new directory (see **WARNING**)
>
> git clone https://github.com/tianocore/edk2.git edk2-uncrustified
> cd edk2-uncrustified
>
> 2) Setup python virtual env, install pytools, and run stuart commands
> to setup build environment which includes installing uncrustify tools.
>
> https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-
> locally
>
> 3) Make a backup copy of plugin UncrustifyCheck outside WORKSPACE.
> (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable and
> EDK II specific uncrustify configuration file available when working
> with a branch that does not have those tools in its scope.
>
> xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
>
> 4) Use lint-history.py from git-filter-repo examples
>
> https://github.com/newren/git-filter-repo
> https://github.com/newren/git-filter-repo/blob/main/contrib/filter-
> repo-demos/lint-history
>
> Line #127 - Add try except around subprocess.check_call() with except
> being pass. This is required because there are a few commits of C
> files in the edk2 repo that have incorrect C syntax and do not
> build with a C compiler and break the uncrustify parser. Skip reformat
> of C files that can not be parsed by uncrustify. These rare instances
> are addressed in the commit that fixes the C syntax error.
>
> Run this slightly modified version of lint-history. Include only
> C/H files and exclude directories that start with 'Tools' or 'BaseTools'.
> This step took about 2.2 hours on a laptop.
>
> lint-history.py
> --relevant "return (not filename.startswith(b'Tools') and not
> filename.startswith(b'BaseTools') and (filename.endswith(b'.c') or
> filename.endswith(b'.h')))"
> c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\mu-
> uncrustify-release_extdep\\Windows-x86\\uncrustify.exe -c
> c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\uncrustify.cfg --
> replace --no-backup --if-changed
>
> Order of PRs to apply during extended hard freeze
> ==================================================
> 1) Update EmulatorPkg Win Host [BuildOptions] MSFT CC_FLAGS to not force
> debug information
> * https://bugzilla.tianocore.org/show_bug.cgi?id=3747
> *
> https://github.com/mdkinney/edk2/tree/Bug_3747_EmulatorPkg_WinHost_
> ReproducibleBuild
> * https://github.com/tianocore/edk2/pull/2215
> * Required for EmulatorPkg to pass CompareBuild for VS2019 IA32/X64
> builds.
> * Status: Review complete. PR pushed.
>
> 2) EccCheck should not revert staged and local changes
> * https://bugzilla.tianocore.org/show_bug.cgi?id=2986
> *
> https://github.com/mdkinney/edk2/tree/Bug_2986_EccCheckRemoveGitRe
> vert_V2
> * https://github.com/tianocore/edk2/pull/2216
> * Required for EDK II CI to complete in a reasonable period of time when
> processing the 4000+ source file style changes made by uncrustify.
> * Also fixes critical bugs that can potentially corrupt git state when
> EccCheck is run locally.
> * Status: Review complete. PR pushed.
>
> 3) Update pytool LicenseCheck plugin to use temp directory for diff output
> file
> * https://bugzilla.tianocore.org/show_bug.cgi?id=3746
> *
> https://github.com/mdkinney/edk2/tree/Bug_3746_LicenseCheckUseDiffO
> utputFile_V2
> * https://github.com/tianocore/edk2/pull/2217
> * Required to reduce EDK II CI build times.
> * Status: Review complete. PR pushed.
>
> 4) Update max job time from 60 min to 120 minutes
> in .azurepipelines/templates
> * https://bugzilla.tianocore.org/show_bug.cgi?id=3750
> *
> https://github.com/mdkinney/edk2/tree/Bug_3750_IncreaseAzurePipelines
> Timeout
> * https://github.com/tianocore/edk2/pull/2219
> * Required to allow EccCheck of uncrustify changes to complete on Azure
> Pipelines CI agents without timing out.
> * Status: Review complete. PR pushed.
>
> 5) Update Package YAML to ignore specific ECC files/errors
> * https://bugzilla.tianocore.org/show_bug.cgi?id=3749
> *
> https://github.com/mdkinney/edk2/tree/Bug_3749_EccCheckIgnoreFilesErr
> ors
> * https://github.com/tianocore/edk2/pull/2218
> * Required to pass EccCheck
> * Status: Review complete. PR pushed
>
> 6) Uncrustify Source Changes
> * https://bugzilla.tianocore.org/show_bug.cgi?id=3737
> * https://bugzilla.tianocore.org/show_bug.cgi?id=3739
> *
> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyC
> hanges_V5
> * https://github.com/tianocore/edk2/pull/2229
> * Build comparison result PASS:
> https://github.com/mdkinney/edk2/actions/runs/1521618836
> * EFI_D_ -> DEBUG changes required to pass PatchCheck
> * Uncrustify format changes required to pass UncrustifyCheck
> * Status:
> Waiting for review
>
> 7) UncrustifyCheck EDK II CI Plugin
> * https://bugzilla.tianocore.org/show_bug.cgi?id=3748
> *
> https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plug
> in_v6
> * Required to enforce all PRs submitted to EDK II CI match uncrustify format.
> * Unconditionally checks all packages. Can not be committed until all C/H
> source files have been updated.
> * Status: Review complete
>
> Combined Branch/PR for Review/Test
> ==================================
> * Build Comparison results must pass 100% across the full set of PRs before
> the individual PRs can be pushed in the order listed above.
> * Branch:
> https://github.com/mdkinney/edk2/tree/TestOnly_Uncrustify_PR_Series
> * PR: https://github.com/tianocore/edk2/pull/2229
> Status = PASS
> * CompareBuild:
> Branch:
> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyC
> hanges_V5
> --ref1: ef9a059cdb15844fe52a49af2bf7d86b9dd3e9bf
> --ref2: Bug_3737_3739_ApplyUncrustifyChanges_V5
> Extra Options: -n 4 --quiet
> Results: https://github.com/mdkinney/edk2/actions/runs/1521618836
> 30 VS2019 build comparisons PASS
> 40 GCC5 build comparisons PASS
> 100% PASS
>
> The following git log shows the set of patches from --ref1 to --ref 2across
> which there are no differences in any of the OBJ/LIB/DLL/EFI/FFS/FV/FD files.
>
> --ref2
> b7d4bf0675b7 (HEAD -> Bug_3737_3739_ApplyUncrustifyChanges_V5)
> UnitTestFrameworkPkg: Apply uncrusitify changes
> 7f03d25f60e7 UefiPayloadPkg: Apply uncrusitify changes
> 0bfd8d9b5ac9 UefiCpuPkg: Apply uncrusitify changes
> e1cd9bfb9dea StandaloneMmPkg: Apply uncrusitify changes
> 5da2f65be378 SourceLevelDebugPkg: Apply uncrusitify changes
> 95b86de07e5d SignedCapsulePkg: Apply uncrusitify changes
> fe71d97246c4 ShellPkg: Apply uncrusitify changes
> 54c21c952992 SecurityPkg: Apply uncrusitify changes
> 187a3785f12b RedfishPkg: Apply uncrusitify changes
> 810100002a46 PcAtChipsetPkg: Apply uncrusitify changes
> 276a695c0cf2 OvmfPkg: Apply uncrusitify changes
> 303c0a91ab07 NetworkPkg: Apply uncrusitify changes
> bc80792cd1b1 MdePkg: Apply uncrusitify changes
> 3ea86be17a2a MdeModulePkg: Apply uncrusitify changes
> c70ef11ed0cd IntelFsp2WrapperPkg: Apply uncrusitify changes
> c0291221f252 IntelFsp2Pkg: Apply uncrusitify changes
> 6a479952a690 FmpDevicePkg: Apply uncrusitify changes
> 3a7c05b7070d FatPkg: Apply uncrusitify changes
> b789f98c8959 EmulatorPkg: Apply uncrusitify changes
> 952d7a1c9220 EmbeddedPkg: Apply uncrusitify changes
> a1cc9881bab6 DynamicTablesPkg: Apply uncrusitify changes
> 50654dfe5785 CryptoPkg: Apply uncrusitify changes
> ed965a02dfa1 ArmVirtPkg: Apply uncrusitify changes
> 9744023fbc46 ArmPlatformPkg: Apply uncrusitify changes
> 7a1cde5f5bba ArmPkg: Apply uncrusitify changes
> 19d17e0913e8 UefiCpuPkg: Change use of EFI_D_* to DEBUG_*
> ffa718b4f994 SourceLevelDebugPkg: Change use of EFI_D_* to DEBUG_*
> b86cb3c5e5b4 ShellPkg: Change use of EFI_D_* to DEBUG_*
> c7c42204dc07 SecurityPkg: Change use of EFI_D_* to DEBUG_*
> 16b8e6f958e4 PcAtChipsetPkg: Change use of EFI_D_* to DEBUG_*
> 0ac3f8b2dac5 OvmfPkg: Change use of EFI_D_* to DEBUG_*
> bc5004b8d294 NetworkPkg: Change use of EFI_D_* to DEBUG_*
> 6f671a8e2377 MdePkg: Change use of EFI_D_* to DEBUG_*
> a10c610ff9a3 MdeModulePkg: Change use of EFI_D_* to DEBUG_*
> 09a3bddba390 FatPkg: Change use of EFI_D_* to DEBUG_*
> 59c61318246a EmulatorPkg: Change use of EFI_D_* to DEBUG_*
> 3a80367dda3b EmbeddedPkg: Change use of EFI_D_* to DEBUG_*
> 23eb1aaf80ca ArmVirtPkg: Change use of EFI_D_* to DEBUG_*
> 875914b45c54 ArmPlatformPkg: Change use of EFI_D_* to DEBUG_*
> eb2eca82b451 ArmPkg: Change use of EFI_D_* to DEBUG_*
> f0f3f5aae7c4 (origin/master, origin/HEAD, master) UnitTestFrameworkPkg:
> Update YAML to ignore specific ECC files/errors
> c05734797790 UefiPayloadPkg: Update YAML to ignore specific ECC
> files/errors
> c30c40d6c63d StandaloneMmPkg: Update YAML to ignore specific ECC
> files/errors
> 9944508e85f1 ShellPkg: Update YAML to ignore specific ECC files/errors
> 60fa40be458d SecurityPkg: Update YAML to ignore specific ECC files/errors
> df790cd6b37e MdePkg: Update YAML to ignore specific ECC files/errors
> 9deb9370766e MdeModulePkg: Update YAML to ignore specific ECC
> files/errors
> d7d30e8f219f EmulatorPkg: Update YAML to ignore specific ECC files/errors
> d5744ecba813 CryptoPkg: Update YAML to ignore specific ECC files/errors
> c97fee87f0f9 ArmVirtPkg: Update YAML to ignore specific ECC files/errors
> 1939fc9569f2 ArmPlatformPkg: Update YAML to ignore specific ECC
> files/errors
> 365dced2c37a ArmPkg: Update YAML to ignore specific ECC files/errors
> 76a1ce4d5fec .azurepipelines/templates: Update max pipeline job time to 2
> hours
> 99f84ff47390 .pytools/Plugin/LicenseCheck: Use temp directory for git diff
> output
> 3019f1bbabf1 .pytool/Plugin/EccCheck: Add performance optimizations
> 854462bd3479 .pytool/Plugin/EccCheck: Remove temp directory on
> exception
> 69877614fdee .pytool/Plugin/EccCheck: Remove RevertCode()
> --ref1
> ef9a059cdb15 EmulatorPkg/Win/Host: Update CC_FLAGS
> bb1bba3d7767 (tag: edk2-stable202111) NetworkPkg: Fix invalid pointer for
> DNS response token on error
>
> Best regards,
>
> Mike
>
>
>
>
>
>
>
>
>
^ permalink raw reply [flat|nested] 39+ messages in thread
* 回复: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
2021-11-30 22:34 Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4 Michael D Kinney
2021-12-01 1:42 ` Wu, Hao A
@ 2021-12-01 2:10 ` gaoliming
2021-12-01 2:26 ` Ni, Ray
` (7 subsequent siblings)
9 siblings, 0 replies; 39+ messages in thread
From: gaoliming @ 2021-12-01 2:10 UTC (permalink / raw)
To: devel, michael.d.kinney, 'Michael Kubacki',
'Andrew Fish', 'Leif Lindholm'
For MdePkg, MdeModulePkg, FmpDevicePkg
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
> -----邮件原件-----
> 发件人: devel@edk2.groups.io <devel@edk2.groups.io> 代表 Michael D
> Kinney
> 发送时间: 2021年12月1日 6:34
> 收件人: devel@edk2.groups.io; Kinney, Michael D
> <michael.d.kinney@intel.com>; Michael Kubacki
> <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com)
> <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>
> 主题: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard
> Freeze Update #4
>
> Hello,
>
> Thank you for your patience during this extended hard freeze.
>
> Just one more step to go. There has been a delay in the review of
> the patch series with the uncrustify source changes. PR(6). This
> patch series was not sent out as patch review email because of its
> very large size. It only contains source style changes and the
> CompareBuild tool and GitHub action has shown there are no binary
> differences introduced with these source style changes.
>
> If you are a package maintainer, then please review the following
> branch/PR for your package contents and review the EDK II CI results
> and BuildCompare results. I do not expect a line by line review
> because we already had time to provide feedback on the source style
> performed by uncrustify. Instead, a Reviewed-by for your package
> indicates that you have reviewed the EDK II CI results and CompareBuild
> tool functionality and results and you accept the source style
> changes to your package.
>
> *
> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyCh
> anges_V5
> * https://github.com/tianocore/edk2/pull/2229
> * https://github.com/mdkinney/edk2/actions/runs/1521618836
>
> Additional details on this update below.
>
> Thank you,
>
> Mike
>
>
> Changes from Update #3
> ----------------------------------------------------------------------------
> * Pushed PR (5)
> * Added link to PR(6). EDK II CI Status is PASS. Build Compare PASS.
> * Waiting for review of PR (6)
> * Review of PR (7) completed and waiting for review of PR (6)
> ----------------------------------------------------------------------------
>
> Changes from Update #2
> ----------------------------------------------------------------------------
> * Changed order of PRs swapping (4) and (5). The PR that activates
> increases the max CI agent job time is independent of all the other
> PRs and its review is complete, so it can be committed now.
> * Pushed PRs (1), (2), (3), (4).
> * Waiting for review to complete for PRs (5) and (6)
> * Reviews complete for PR (7)
> * Identifies steps using git filter-branch to apply uncrustify changes to a
> code review patch series that was generated before the uncrustify changes
> avoiding manual merge.
> * Identified steps using git filter-repo to generate an alternate history of
> the edk2 repo with uncrustify changes applied on every commit. This may
> be useful when evaluating changes to files using tools like git blame
> without the large uncrustify patch series.
> ---------------------------------------------------------------------------
>
> Changes from Update #1
> ----------------------------------------------------------------------------
> * Changed order of PRs swapping (6) and (7). The PR that activates
> EDK II CI check UncrustifyCheck has to be last because it unconditionally
> checks all C/H files in all packages. Not just files that have been
> modified like some of the other checkers.
> * Updated link to the branch with the UncrustifyCheck plugin that has been
> updated with a one line change and Reviewed-by and Tested-by tags.
>
> https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin
> _v6
> * Reviews complete for (1), (2), (3), (5), and (7)
> ---------------------------------------------------------------------------
>
> Michael Kubacki and I have prepared the patches required to apply the
> uncrustify changes and enable EDK II CI to check all submitted
> patches have been run through uncrustify.
>
> We have verified through the CompareBuild GitHub Action that the
> format changes performed by uncrustify have no functional changes.
> All of the OBJ, LIB, DLL, EFI, FFS, FV, and FD files match 100%
> across 70 VS2019/GCC5 builds of all package/platform DSC files in
> the edk2 repo.
>
> The hard freeze will be extended after the edk2-stable202111 tag until
> all uncrustify related changes are committed. We do not expect this
> to take more than a few days. Do not push any PRs until the hard
> freeze is lifted.
>
> The changes are broken up into 7 patch series/PRs. The PRs are ordered
> so they can be submitted using the normal submission process and EDK II
> CI will pass for each one. Details are listed below.
>
> Uncrustify 73.0.3 for EDK II
> =============================
> * Sources: https://dev.azure.com/projectmu/_git/Uncrustify
> * Documentation:
> https://dev.azure.com/projectmu/Uncrustify/_wiki/wikis/Uncrustify.wiki/1/P
> roject-Mu-(EDK-II)-Fork-Readme
> * Download:
> https://dev.azure.com/projectmu/Uncrustify/_packaging?_a=package&feed=
> mu_uncrustify&package=mu-uncrustify-release&protocolType=NuGet&versio
> n=73.0.3
>
> Installing Uncrustify
> ======================
> The Uncrustify tool is installed automatically when the Pytools
> environment is used and the stuart* commands are run to complete the
> environment setup. Please see:
>
>
> https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally
>
> Uncrustify can also be installed from the download page listed above
> or built from sources from the source link above.
>
> The Documentation link provides instruction on how to run uncrustify from
> the command line or install as a Visual Studio Code plugin. The main
> uncrustify documentation also describes how to integrate with a few other
> editors.
>
> We have also discussed a client side githook. That effort has not started.
> Let us know if that is a feature you would find useful.
>
> Developer impact for new code reviews
> ======================================
> Once the uncrustify checker is active in EDK II CI, developers must
> make sure their patches are run through the uncrustify tool before
> sending the patches for review.
>
> Developers must install and run uncrustify against changes files before
> sending patch review emails or submitting PR for EDK II CI. If EDK II CI
> detects and differences in source formatting, then EDK II CI will fail
> and the developer must run uncrustify and resubmit the patches.
>
> Developer impact to patch series/PRs reviewed during edk2-stable201121
> soft/hard freeze
> ==============================================================
> =========================
> Developers must rebase their changes after the uncrustify source changes are
> committed. The branch with a preview of the uncrustify changes can be
> used
> to start this rebase work.
>
>
> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyCh
> anges_V5
>
> The following steps can be used to update an existing branch with the
> required uncrustify format. This is the Windows version. I will add
> the Linux version soon.
>
> 1) Fetch and checkout and rebase to latest edk2/master
>
> git fetch origin
> git checkout master
> git rebase origin/master
>
> 2) Make a backup copy of plugin UncrustifyCheck outside WORKSPACE.
> (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable and
> EDK II specific uncrustify configuration file available when working
> with a branch that does not have those tools in its scope.
>
> xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
>
> 3) Check out the patch series branch (e.g. MyBranch)
>
> git checkout MyBranch
>
> 4) Rebase patch series against edk2-stable202111
>
> git rebase edk2-stable202111
>
> 5) Create new branch for the uncrustifed version (e.g.
> MyBranch_Uncrustified)
>
> git checkout -b MyBranch_Uncrustified
>
> 6) Use git filter-branch to uncrustify all the commits in the series
> between the rebase target from (2) and HEAD of the branch. A
> filter
> can be used to scope the uncrustify operations to only the C/H files
> in the specific package the patch series is against. (e.g.
> DynamicTablesPkg).
> BaseTools should always be excluded. If the package scoped filter is
> not used, it will still work, but will take longer to run because
> uncrustify will rescan every C/H files in the whole repo.
>
> git filter-branch --tree-filter "git ls-files DynamicTablesPkg*.c
> DynamicTablesPkg*.h :!BaseTools/* |
> c:\\Temp\\UncrustifyCheck\\mu-uncrustify-release_extdep\\Windows-x86\\
> uncrustify.exe -c c:\\Temp\\UncrustifyCheck\\uncrustify.cfg -F - --replace
> --no-backup --if-changed" edk2-stable202111..HEAD
>
> 7) Now that all the individual patches in the branch are uncrustified,
> rebase against latest edk2/master that is already uncrustified.
>
> git rebase master
>
> 8) Verify the patches in this new branch.
>
> Impacts to tracing history across the uncrusity changes
> =======================================================
> Tools the view file and line history do work with the large uncrustify
> patch series. One impact is that the operations can be very slow due
> to the large uncrustify patches.
>
> One option to provide a faster experience is to provide an alternate
> version of the edk2 repository as "documentation" that has the
> entire history re-written with uncrustify run on every commit.
> The tool called git-filter-repo can be used to perform this
> transformation and runs in a reasonable period of time (a few hours)
>
> https://github.com/newren/git-filter-repo
>
> https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-dem
> os/lint-history
>
> The following steps can be used to perform this transformation.
> This is the Windows version. I will add the Linux version soon.
>
> ** WARNING ** This operation modifies(rewrites) all the commits
> in the local copy of the repo. Do not perform
> these steps on a local repo you are using for
> active development.
>
> 1) Clone edk2 into a new directory (see **WARNING**)
>
> git clone https://github.com/tianocore/edk2.git
> edk2-uncrustified
> cd edk2-uncrustified
>
> 2) Setup python virtual env, install pytools, and run stuart commands
> to setup build environment which includes installing uncrustify tools.
>
>
> https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally
>
> 3) Make a backup copy of plugin UncrustifyCheck outside WORKSPACE.
> (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable and
> EDK II specific uncrustify configuration file available when working
> with a branch that does not have those tools in its scope.
>
> xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
>
> 4) Use lint-history.py from git-filter-repo examples
>
> https://github.com/newren/git-filter-repo
>
> https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-dem
> os/lint-history
>
> Line #127 - Add try except around subprocess.check_call() with
> except
> being pass. This is required because there are a few commits of C
> files in the edk2 repo that have incorrect C syntax and do not
> build with a C compiler and break the uncrustify parser. Skip
> reformat
> of C files that can not be parsed by uncrustify. These rare instances
> are addressed in the commit that fixes the C syntax error.
>
> Run this slightly modified version of lint-history. Include only
> C/H files and exclude directories that start with 'Tools' or 'BaseTools'.
> This step took about 2.2 hours on a laptop.
>
> lint-history.py
> --relevant "return (not filename.startswith(b'Tools') and
> not filename.startswith(b'BaseTools') and (filename.endswith(b'.c') or
> filename.endswith(b'.h')))"
>
> c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\mu-uncrustify-release
> _extdep\\Windows-x86\\uncrustify.exe -c
> c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\uncrustify.cfg --replace
> --no-backup --if-changed
>
> Order of PRs to apply during extended hard freeze
> ==================================================
> 1) Update EmulatorPkg Win Host [BuildOptions] MSFT CC_FLAGS to not force
> debug information
> * https://bugzilla.tianocore.org/show_bug.cgi?id=3747
> *
> https://github.com/mdkinney/edk2/tree/Bug_3747_EmulatorPkg_WinHost_R
> eproducibleBuild
> * https://github.com/tianocore/edk2/pull/2215
> * Required for EmulatorPkg to pass CompareBuild for VS2019 IA32/X64
> builds.
> * Status: Review complete. PR pushed.
>
> 2) EccCheck should not revert staged and local changes
> * https://bugzilla.tianocore.org/show_bug.cgi?id=2986
> *
> https://github.com/mdkinney/edk2/tree/Bug_2986_EccCheckRemoveGitRev
> ert_V2
> * https://github.com/tianocore/edk2/pull/2216
> * Required for EDK II CI to complete in a reasonable period of time when
> processing the 4000+ source file style changes made by uncrustify.
> * Also fixes critical bugs that can potentially corrupt git state when
> EccCheck is run locally.
> * Status: Review complete. PR pushed.
>
> 3) Update pytool LicenseCheck plugin to use temp directory for diff output file
> * https://bugzilla.tianocore.org/show_bug.cgi?id=3746
> *
> https://github.com/mdkinney/edk2/tree/Bug_3746_LicenseCheckUseDiffOut
> putFile_V2
> * https://github.com/tianocore/edk2/pull/2217
> * Required to reduce EDK II CI build times.
> * Status: Review complete. PR pushed.
>
> 4) Update max job time from 60 min to 120 minutes
> in .azurepipelines/templates
> * https://bugzilla.tianocore.org/show_bug.cgi?id=3750
> *
> https://github.com/mdkinney/edk2/tree/Bug_3750_IncreaseAzurePipelinesTi
> meout
> * https://github.com/tianocore/edk2/pull/2219
> * Required to allow EccCheck of uncrustify changes to complete on Azure
> Pipelines CI agents without timing out.
> * Status: Review complete. PR pushed.
>
> 5) Update Package YAML to ignore specific ECC files/errors
> * https://bugzilla.tianocore.org/show_bug.cgi?id=3749
> *
> https://github.com/mdkinney/edk2/tree/Bug_3749_EccCheckIgnoreFilesErro
> rs
> * https://github.com/tianocore/edk2/pull/2218
> * Required to pass EccCheck
> * Status: Review complete. PR pushed
>
> 6) Uncrustify Source Changes
> * https://bugzilla.tianocore.org/show_bug.cgi?id=3737
> * https://bugzilla.tianocore.org/show_bug.cgi?id=3739
> *
> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyCh
> anges_V5
> * https://github.com/tianocore/edk2/pull/2229
> * Build comparison result PASS:
> https://github.com/mdkinney/edk2/actions/runs/1521618836
> * EFI_D_ -> DEBUG changes required to pass PatchCheck
> * Uncrustify format changes required to pass UncrustifyCheck
> * Status:
> Waiting for review
>
> 7) UncrustifyCheck EDK II CI Plugin
> * https://bugzilla.tianocore.org/show_bug.cgi?id=3748
> *
> https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin
> _v6
> * Required to enforce all PRs submitted to EDK II CI match uncrustify
> format.
> * Unconditionally checks all packages. Can not be committed until all
> C/H
> source files have been updated.
> * Status: Review complete
>
> Combined Branch/PR for Review/Test
> ==================================
> * Build Comparison results must pass 100% across the full set of PRs before
> the individual PRs can be pushed in the order listed above.
> * Branch:
> https://github.com/mdkinney/edk2/tree/TestOnly_Uncrustify_PR_Series
> * PR: https://github.com/tianocore/edk2/pull/2229
> Status = PASS
> * CompareBuild:
> Branch:
> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyCh
> anges_V5
> --ref1: ef9a059cdb15844fe52a49af2bf7d86b9dd3e9bf
> --ref2: Bug_3737_3739_ApplyUncrustifyChanges_V5
> Extra Options: -n 4 --quiet
> Results: https://github.com/mdkinney/edk2/actions/runs/1521618836
> 30 VS2019 build comparisons PASS
> 40 GCC5 build comparisons PASS
> 100% PASS
>
> The following git log shows the set of patches from --ref1 to --ref 2across
> which there are no differences in any of the OBJ/LIB/DLL/EFI/FFS/FV/FD files.
>
> --ref2
> b7d4bf0675b7 (HEAD -> Bug_3737_3739_ApplyUncrustifyChanges_V5)
> UnitTestFrameworkPkg: Apply uncrusitify changes
> 7f03d25f60e7 UefiPayloadPkg: Apply uncrusitify changes
> 0bfd8d9b5ac9 UefiCpuPkg: Apply uncrusitify changes
> e1cd9bfb9dea StandaloneMmPkg: Apply uncrusitify changes
> 5da2f65be378 SourceLevelDebugPkg: Apply uncrusitify changes
> 95b86de07e5d SignedCapsulePkg: Apply uncrusitify changes
> fe71d97246c4 ShellPkg: Apply uncrusitify changes
> 54c21c952992 SecurityPkg: Apply uncrusitify changes
> 187a3785f12b RedfishPkg: Apply uncrusitify changes
> 810100002a46 PcAtChipsetPkg: Apply uncrusitify changes
> 276a695c0cf2 OvmfPkg: Apply uncrusitify changes
> 303c0a91ab07 NetworkPkg: Apply uncrusitify changes
> bc80792cd1b1 MdePkg: Apply uncrusitify changes
> 3ea86be17a2a MdeModulePkg: Apply uncrusitify changes
> c70ef11ed0cd IntelFsp2WrapperPkg: Apply uncrusitify changes
> c0291221f252 IntelFsp2Pkg: Apply uncrusitify changes
> 6a479952a690 FmpDevicePkg: Apply uncrusitify changes
> 3a7c05b7070d FatPkg: Apply uncrusitify changes
> b789f98c8959 EmulatorPkg: Apply uncrusitify changes
> 952d7a1c9220 EmbeddedPkg: Apply uncrusitify changes
> a1cc9881bab6 DynamicTablesPkg: Apply uncrusitify changes
> 50654dfe5785 CryptoPkg: Apply uncrusitify changes
> ed965a02dfa1 ArmVirtPkg: Apply uncrusitify changes
> 9744023fbc46 ArmPlatformPkg: Apply uncrusitify changes
> 7a1cde5f5bba ArmPkg: Apply uncrusitify changes
> 19d17e0913e8 UefiCpuPkg: Change use of EFI_D_* to DEBUG_*
> ffa718b4f994 SourceLevelDebugPkg: Change use of EFI_D_* to DEBUG_*
> b86cb3c5e5b4 ShellPkg: Change use of EFI_D_* to DEBUG_*
> c7c42204dc07 SecurityPkg: Change use of EFI_D_* to DEBUG_*
> 16b8e6f958e4 PcAtChipsetPkg: Change use of EFI_D_* to DEBUG_*
> 0ac3f8b2dac5 OvmfPkg: Change use of EFI_D_* to DEBUG_*
> bc5004b8d294 NetworkPkg: Change use of EFI_D_* to DEBUG_*
> 6f671a8e2377 MdePkg: Change use of EFI_D_* to DEBUG_*
> a10c610ff9a3 MdeModulePkg: Change use of EFI_D_* to DEBUG_*
> 09a3bddba390 FatPkg: Change use of EFI_D_* to DEBUG_*
> 59c61318246a EmulatorPkg: Change use of EFI_D_* to DEBUG_*
> 3a80367dda3b EmbeddedPkg: Change use of EFI_D_* to DEBUG_*
> 23eb1aaf80ca ArmVirtPkg: Change use of EFI_D_* to DEBUG_*
> 875914b45c54 ArmPlatformPkg: Change use of EFI_D_* to DEBUG_*
> eb2eca82b451 ArmPkg: Change use of EFI_D_* to DEBUG_*
> f0f3f5aae7c4 (origin/master, origin/HEAD, master) UnitTestFrameworkPkg:
> Update YAML to ignore specific ECC files/errors
> c05734797790 UefiPayloadPkg: Update YAML to ignore specific ECC
> files/errors
> c30c40d6c63d StandaloneMmPkg: Update YAML to ignore specific ECC
> files/errors
> 9944508e85f1 ShellPkg: Update YAML to ignore specific ECC files/errors
> 60fa40be458d SecurityPkg: Update YAML to ignore specific ECC files/errors
> df790cd6b37e MdePkg: Update YAML to ignore specific ECC files/errors
> 9deb9370766e MdeModulePkg: Update YAML to ignore specific ECC
> files/errors
> d7d30e8f219f EmulatorPkg: Update YAML to ignore specific ECC files/errors
> d5744ecba813 CryptoPkg: Update YAML to ignore specific ECC files/errors
> c97fee87f0f9 ArmVirtPkg: Update YAML to ignore specific ECC files/errors
> 1939fc9569f2 ArmPlatformPkg: Update YAML to ignore specific ECC
> files/errors
> 365dced2c37a ArmPkg: Update YAML to ignore specific ECC files/errors
> 76a1ce4d5fec .azurepipelines/templates: Update max pipeline job time to 2
> hours
> 99f84ff47390 .pytools/Plugin/LicenseCheck: Use temp directory for git diff
> output
> 3019f1bbabf1 .pytool/Plugin/EccCheck: Add performance optimizations
> 854462bd3479 .pytool/Plugin/EccCheck: Remove temp directory on
> exception
> 69877614fdee .pytool/Plugin/EccCheck: Remove RevertCode()
> --ref1
> ef9a059cdb15 EmulatorPkg/Win/Host: Update CC_FLAGS
> bb1bba3d7767 (tag: edk2-stable202111) NetworkPkg: Fix invalid pointer for
> DNS response token on error
>
> Best regards,
>
> Mike
>
>
>
>
>
>
>
>
>
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
2021-11-30 22:34 Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4 Michael D Kinney
2021-12-01 1:42 ` Wu, Hao A
2021-12-01 2:10 ` 回复: [edk2-devel] " gaoliming
@ 2021-12-01 2:26 ` Ni, Ray
2021-12-01 2:45 ` Chiu, Chasel
` (6 subsequent siblings)
9 siblings, 0 replies; 39+ messages in thread
From: Ni, Ray @ 2021-12-01 2:26 UTC (permalink / raw)
To: devel@edk2.groups.io, Kinney, Michael D, Michael Kubacki,
Andrew Fish (afish@apple.com), Leif Lindholm
For below packages, Reviewed-by: Ray Ni <ray.ni@intel.com>
* EmulatorPkg
* FatPkg
* PcAtChipsetPkg
* ShellPkg
* UefiCpuPkg
* UefiPayloadPkg
-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Michael D Kinney
Sent: Wednesday, December 1, 2021 6:34 AM
To: devel@edk2.groups.io; Kinney, Michael D <michael.d.kinney@intel.com>; Michael Kubacki <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com) <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>
Subject: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
Hello,
Thank you for your patience during this extended hard freeze.
Just one more step to go. There has been a delay in the review of the patch series with the uncrustify source changes. PR(6). This patch series was not sent out as patch review email because of its very large size. It only contains source style changes and the CompareBuild tool and GitHub action has shown there are no binary differences introduced with these source style changes.
If you are a package maintainer, then please review the following branch/PR for your package contents and review the EDK II CI results and BuildCompare results. I do not expect a line by line review because we already had time to provide feedback on the source style performed by uncrustify. Instead, a Reviewed-by for your package indicates that you have reviewed the EDK II CI results and CompareBuild tool functionality and results and you accept the source style changes to your package.
* https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
* https://github.com/tianocore/edk2/pull/2229
* https://github.com/mdkinney/edk2/actions/runs/1521618836
Additional details on this update below.
Thank you,
Mike
Changes from Update #3
----------------------------------------------------------------------------
* Pushed PR (5)
* Added link to PR(6). EDK II CI Status is PASS. Build Compare PASS.
* Waiting for review of PR (6)
* Review of PR (7) completed and waiting for review of PR (6)
----------------------------------------------------------------------------
Changes from Update #2
----------------------------------------------------------------------------
* Changed order of PRs swapping (4) and (5). The PR that activates
increases the max CI agent job time is independent of all the other
PRs and its review is complete, so it can be committed now.
* Pushed PRs (1), (2), (3), (4).
* Waiting for review to complete for PRs (5) and (6)
* Reviews complete for PR (7)
* Identifies steps using git filter-branch to apply uncrustify changes to a
code review patch series that was generated before the uncrustify changes
avoiding manual merge.
* Identified steps using git filter-repo to generate an alternate history of
the edk2 repo with uncrustify changes applied on every commit. This may
be useful when evaluating changes to files using tools like git blame
without the large uncrustify patch series.
---------------------------------------------------------------------------
Changes from Update #1
----------------------------------------------------------------------------
* Changed order of PRs swapping (6) and (7). The PR that activates
EDK II CI check UncrustifyCheck has to be last because it unconditionally
checks all C/H files in all packages. Not just files that have been
modified like some of the other checkers.
* Updated link to the branch with the UncrustifyCheck plugin that has been
updated with a one line change and Reviewed-by and Tested-by tags.
https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v6
* Reviews complete for (1), (2), (3), (5), and (7)
---------------------------------------------------------------------------
Michael Kubacki and I have prepared the patches required to apply the uncrustify changes and enable EDK II CI to check all submitted patches have been run through uncrustify.
We have verified through the CompareBuild GitHub Action that the format changes performed by uncrustify have no functional changes.
All of the OBJ, LIB, DLL, EFI, FFS, FV, and FD files match 100% across 70 VS2019/GCC5 builds of all package/platform DSC files in the edk2 repo.
The hard freeze will be extended after the edk2-stable202111 tag until all uncrustify related changes are committed. We do not expect this to take more than a few days. Do not push any PRs until the hard freeze is lifted.
The changes are broken up into 7 patch series/PRs. The PRs are ordered so they can be submitted using the normal submission process and EDK II CI will pass for each one. Details are listed below.
Uncrustify 73.0.3 for EDK II
=============================
* Sources: https://dev.azure.com/projectmu/_git/Uncrustify
* Documentation: https://dev.azure.com/projectmu/Uncrustify/_wiki/wikis/Uncrustify.wiki/1/Project-Mu-(EDK-II)-Fork-Readme
* Download: https://dev.azure.com/projectmu/Uncrustify/_packaging?_a=package&feed=mu_uncrustify&package=mu-uncrustify-release&protocolType=NuGet&version=73.0.3
Installing Uncrustify
======================
The Uncrustify tool is installed automatically when the Pytools environment is used and the stuart* commands are run to complete the environment setup. Please see:
https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally
Uncrustify can also be installed from the download page listed above or built from sources from the source link above.
The Documentation link provides instruction on how to run uncrustify from the command line or install as a Visual Studio Code plugin. The main uncrustify documentation also describes how to integrate with a few other editors.
We have also discussed a client side githook. That effort has not started.
Let us know if that is a feature you would find useful.
Developer impact for new code reviews
======================================
Once the uncrustify checker is active in EDK II CI, developers must make sure their patches are run through the uncrustify tool before sending the patches for review.
Developers must install and run uncrustify against changes files before sending patch review emails or submitting PR for EDK II CI. If EDK II CI detects and differences in source formatting, then EDK II CI will fail and the developer must run uncrustify and resubmit the patches.
Developer impact to patch series/PRs reviewed during edk2-stable201121 soft/hard freeze =======================================================================================
Developers must rebase their changes after the uncrustify source changes are committed. The branch with a preview of the uncrustify changes can be used to start this rebase work.
https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
The following steps can be used to update an existing branch with the required uncrustify format. This is the Windows version. I will add the Linux version soon.
1) Fetch and checkout and rebase to latest edk2/master
git fetch origin
git checkout master
git rebase origin/master
2) Make a backup copy of plugin UncrustifyCheck outside WORKSPACE.
(e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable and
EDK II specific uncrustify configuration file available when working
with a branch that does not have those tools in its scope.
xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
3) Check out the patch series branch (e.g. MyBranch)
git checkout MyBranch
4) Rebase patch series against edk2-stable202111
git rebase edk2-stable202111
5) Create new branch for the uncrustifed version (e.g. MyBranch_Uncrustified)
git checkout -b MyBranch_Uncrustified
6) Use git filter-branch to uncrustify all the commits in the series
between the rebase target from (2) and HEAD of the branch. A filter
can be used to scope the uncrustify operations to only the C/H files
in the specific package the patch series is against. (e.g. DynamicTablesPkg).
BaseTools should always be excluded. If the package scoped filter is
not used, it will still work, but will take longer to run because
uncrustify will rescan every C/H files in the whole repo.
git filter-branch --tree-filter "git ls-files DynamicTablesPkg*.c DynamicTablesPkg*.h :!BaseTools/* | c:\\Temp\\UncrustifyCheck\\mu-uncrustify-release_extdep\\Windows-x86\\uncrustify.exe -c c:\\Temp\\UncrustifyCheck\\uncrustify.cfg -F - --replace --no-backup --if-changed" edk2-stable202111..HEAD
7) Now that all the individual patches in the branch are uncrustified,
rebase against latest edk2/master that is already uncrustified.
git rebase master
8) Verify the patches in this new branch.
Impacts to tracing history across the uncrusity changes =======================================================
Tools the view file and line history do work with the large uncrustify patch series. One impact is that the operations can be very slow due to the large uncrustify patches.
One option to provide a faster experience is to provide an alternate version of the edk2 repository as "documentation" that has the entire history re-written with uncrustify run on every commit.
The tool called git-filter-repo can be used to perform this transformation and runs in a reasonable period of time (a few hours)
https://github.com/newren/git-filter-repo
https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-demos/lint-history
The following steps can be used to perform this transformation.
This is the Windows version. I will add the Linux version soon.
** WARNING ** This operation modifies(rewrites) all the commits
in the local copy of the repo. Do not perform
these steps on a local repo you are using for
active development.
1) Clone edk2 into a new directory (see **WARNING**)
git clone https://github.com/tianocore/edk2.git edk2-uncrustified
cd edk2-uncrustified
2) Setup python virtual env, install pytools, and run stuart commands
to setup build environment which includes installing uncrustify tools.
https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally
3) Make a backup copy of plugin UncrustifyCheck outside WORKSPACE.
(e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable and
EDK II specific uncrustify configuration file available when working
with a branch that does not have those tools in its scope.
xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
4) Use lint-history.py from git-filter-repo examples
https://github.com/newren/git-filter-repo
https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-demos/lint-history
Line #127 - Add try except around subprocess.check_call() with except
being pass. This is required because there are a few commits of C
files in the edk2 repo that have incorrect C syntax and do not
build with a C compiler and break the uncrustify parser. Skip reformat
of C files that can not be parsed by uncrustify. These rare instances
are addressed in the commit that fixes the C syntax error.
Run this slightly modified version of lint-history. Include only
C/H files and exclude directories that start with 'Tools' or 'BaseTools'.
This step took about 2.2 hours on a laptop.
lint-history.py
--relevant "return (not filename.startswith(b'Tools') and not filename.startswith(b'BaseTools') and (filename.endswith(b'.c') or filename.endswith(b'.h')))"
c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\mu-uncrustify-release_extdep\\Windows-x86\\uncrustify.exe -c c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\uncrustify.cfg --replace --no-backup --if-changed
Order of PRs to apply during extended hard freeze ==================================================
1) Update EmulatorPkg Win Host [BuildOptions] MSFT CC_FLAGS to not force debug information
* https://bugzilla.tianocore.org/show_bug.cgi?id=3747
* https://github.com/mdkinney/edk2/tree/Bug_3747_EmulatorPkg_WinHost_ReproducibleBuild
* https://github.com/tianocore/edk2/pull/2215
* Required for EmulatorPkg to pass CompareBuild for VS2019 IA32/X64 builds.
* Status: Review complete. PR pushed.
2) EccCheck should not revert staged and local changes
* https://bugzilla.tianocore.org/show_bug.cgi?id=2986
* https://github.com/mdkinney/edk2/tree/Bug_2986_EccCheckRemoveGitRevert_V2
* https://github.com/tianocore/edk2/pull/2216
* Required for EDK II CI to complete in a reasonable period of time when
processing the 4000+ source file style changes made by uncrustify.
* Also fixes critical bugs that can potentially corrupt git state when
EccCheck is run locally.
* Status: Review complete. PR pushed.
3) Update pytool LicenseCheck plugin to use temp directory for diff output file
* https://bugzilla.tianocore.org/show_bug.cgi?id=3746
* https://github.com/mdkinney/edk2/tree/Bug_3746_LicenseCheckUseDiffOutputFile_V2
* https://github.com/tianocore/edk2/pull/2217
* Required to reduce EDK II CI build times.
* Status: Review complete. PR pushed.
4) Update max job time from 60 min to 120 minutes in .azurepipelines/templates
* https://bugzilla.tianocore.org/show_bug.cgi?id=3750
* https://github.com/mdkinney/edk2/tree/Bug_3750_IncreaseAzurePipelinesTimeout
* https://github.com/tianocore/edk2/pull/2219
* Required to allow EccCheck of uncrustify changes to complete on Azure
Pipelines CI agents without timing out.
* Status: Review complete. PR pushed.
5) Update Package YAML to ignore specific ECC files/errors
* https://bugzilla.tianocore.org/show_bug.cgi?id=3749
* https://github.com/mdkinney/edk2/tree/Bug_3749_EccCheckIgnoreFilesErrors
* https://github.com/tianocore/edk2/pull/2218
* Required to pass EccCheck
* Status: Review complete. PR pushed
6) Uncrustify Source Changes
* https://bugzilla.tianocore.org/show_bug.cgi?id=3737
* https://bugzilla.tianocore.org/show_bug.cgi?id=3739
* https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
* https://github.com/tianocore/edk2/pull/2229
* Build comparison result PASS: https://github.com/mdkinney/edk2/actions/runs/1521618836
* EFI_D_ -> DEBUG changes required to pass PatchCheck
* Uncrustify format changes required to pass UncrustifyCheck
* Status:
Waiting for review
7) UncrustifyCheck EDK II CI Plugin
* https://bugzilla.tianocore.org/show_bug.cgi?id=3748
* https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v6
* Required to enforce all PRs submitted to EDK II CI match uncrustify format.
* Unconditionally checks all packages. Can not be committed until all C/H
source files have been updated.
* Status: Review complete
Combined Branch/PR for Review/Test
==================================
* Build Comparison results must pass 100% across the full set of PRs before
the individual PRs can be pushed in the order listed above.
* Branch: https://github.com/mdkinney/edk2/tree/TestOnly_Uncrustify_PR_Series
* PR: https://github.com/tianocore/edk2/pull/2229
Status = PASS
* CompareBuild:
Branch: https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
--ref1: ef9a059cdb15844fe52a49af2bf7d86b9dd3e9bf
--ref2: Bug_3737_3739_ApplyUncrustifyChanges_V5
Extra Options: -n 4 --quiet
Results: https://github.com/mdkinney/edk2/actions/runs/1521618836
30 VS2019 build comparisons PASS
40 GCC5 build comparisons PASS
100% PASS
The following git log shows the set of patches from --ref1 to --ref 2across which there are no differences in any of the OBJ/LIB/DLL/EFI/FFS/FV/FD files.
--ref2
b7d4bf0675b7 (HEAD -> Bug_3737_3739_ApplyUncrustifyChanges_V5) UnitTestFrameworkPkg: Apply uncrusitify changes
7f03d25f60e7 UefiPayloadPkg: Apply uncrusitify changes
0bfd8d9b5ac9 UefiCpuPkg: Apply uncrusitify changes
e1cd9bfb9dea StandaloneMmPkg: Apply uncrusitify changes
5da2f65be378 SourceLevelDebugPkg: Apply uncrusitify changes
95b86de07e5d SignedCapsulePkg: Apply uncrusitify changes
fe71d97246c4 ShellPkg: Apply uncrusitify changes
54c21c952992 SecurityPkg: Apply uncrusitify changes
187a3785f12b RedfishPkg: Apply uncrusitify changes
810100002a46 PcAtChipsetPkg: Apply uncrusitify changes
276a695c0cf2 OvmfPkg: Apply uncrusitify changes
303c0a91ab07 NetworkPkg: Apply uncrusitify changes
bc80792cd1b1 MdePkg: Apply uncrusitify changes
3ea86be17a2a MdeModulePkg: Apply uncrusitify changes
c70ef11ed0cd IntelFsp2WrapperPkg: Apply uncrusitify changes
c0291221f252 IntelFsp2Pkg: Apply uncrusitify changes
6a479952a690 FmpDevicePkg: Apply uncrusitify changes
3a7c05b7070d FatPkg: Apply uncrusitify changes
b789f98c8959 EmulatorPkg: Apply uncrusitify changes
952d7a1c9220 EmbeddedPkg: Apply uncrusitify changes
a1cc9881bab6 DynamicTablesPkg: Apply uncrusitify changes
50654dfe5785 CryptoPkg: Apply uncrusitify changes
ed965a02dfa1 ArmVirtPkg: Apply uncrusitify changes
9744023fbc46 ArmPlatformPkg: Apply uncrusitify changes
7a1cde5f5bba ArmPkg: Apply uncrusitify changes
19d17e0913e8 UefiCpuPkg: Change use of EFI_D_* to DEBUG_*
ffa718b4f994 SourceLevelDebugPkg: Change use of EFI_D_* to DEBUG_*
b86cb3c5e5b4 ShellPkg: Change use of EFI_D_* to DEBUG_*
c7c42204dc07 SecurityPkg: Change use of EFI_D_* to DEBUG_*
16b8e6f958e4 PcAtChipsetPkg: Change use of EFI_D_* to DEBUG_*
0ac3f8b2dac5 OvmfPkg: Change use of EFI_D_* to DEBUG_*
bc5004b8d294 NetworkPkg: Change use of EFI_D_* to DEBUG_*
6f671a8e2377 MdePkg: Change use of EFI_D_* to DEBUG_*
a10c610ff9a3 MdeModulePkg: Change use of EFI_D_* to DEBUG_*
09a3bddba390 FatPkg: Change use of EFI_D_* to DEBUG_*
59c61318246a EmulatorPkg: Change use of EFI_D_* to DEBUG_*
3a80367dda3b EmbeddedPkg: Change use of EFI_D_* to DEBUG_*
23eb1aaf80ca ArmVirtPkg: Change use of EFI_D_* to DEBUG_*
875914b45c54 ArmPlatformPkg: Change use of EFI_D_* to DEBUG_*
eb2eca82b451 ArmPkg: Change use of EFI_D_* to DEBUG_*
f0f3f5aae7c4 (origin/master, origin/HEAD, master) UnitTestFrameworkPkg: Update YAML to ignore specific ECC files/errors
c05734797790 UefiPayloadPkg: Update YAML to ignore specific ECC files/errors
c30c40d6c63d StandaloneMmPkg: Update YAML to ignore specific ECC files/errors
9944508e85f1 ShellPkg: Update YAML to ignore specific ECC files/errors
60fa40be458d SecurityPkg: Update YAML to ignore specific ECC files/errors
df790cd6b37e MdePkg: Update YAML to ignore specific ECC files/errors
9deb9370766e MdeModulePkg: Update YAML to ignore specific ECC files/errors
d7d30e8f219f EmulatorPkg: Update YAML to ignore specific ECC files/errors
d5744ecba813 CryptoPkg: Update YAML to ignore specific ECC files/errors
c97fee87f0f9 ArmVirtPkg: Update YAML to ignore specific ECC files/errors
1939fc9569f2 ArmPlatformPkg: Update YAML to ignore specific ECC files/errors
365dced2c37a ArmPkg: Update YAML to ignore specific ECC files/errors
76a1ce4d5fec .azurepipelines/templates: Update max pipeline job time to 2 hours
99f84ff47390 .pytools/Plugin/LicenseCheck: Use temp directory for git diff output
3019f1bbabf1 .pytool/Plugin/EccCheck: Add performance optimizations
854462bd3479 .pytool/Plugin/EccCheck: Remove temp directory on exception
69877614fdee .pytool/Plugin/EccCheck: Remove RevertCode()
--ref1
ef9a059cdb15 EmulatorPkg/Win/Host: Update CC_FLAGS
bb1bba3d7767 (tag: edk2-stable202111) NetworkPkg: Fix invalid pointer for DNS response token on error
Best regards,
Mike
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
2021-11-30 22:34 Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4 Michael D Kinney
` (2 preceding siblings ...)
2021-12-01 2:26 ` Ni, Ray
@ 2021-12-01 2:45 ` Chiu, Chasel
2021-12-01 5:34 ` Wang, Jian J
` (5 subsequent siblings)
9 siblings, 0 replies; 39+ messages in thread
From: Chiu, Chasel @ 2021-12-01 2:45 UTC (permalink / raw)
To: devel@edk2.groups.io, Kinney, Michael D, Michael Kubacki,
Andrew Fish (afish@apple.com), Leif Lindholm
For below patches, Reviewed-by: Chasel Chiu <chasel.chiu@intel.com>
Thanks,
Chasel
IntelFsp2Pkg: Apply uncrusitify changes
IntelFsp2WrapperPkg: Apply uncrusitify changes
> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Michael D
> Kinney
> Sent: Wednesday, December 1, 2021 6:34 AM
> To: devel@edk2.groups.io; Kinney, Michael D <michael.d.kinney@intel.com>;
> Michael Kubacki <michael.kubacki@microsoft.com>; Andrew Fish
> (afish@apple.com) <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>
> Subject: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard
> Freeze Update #4
>
> Hello,
>
> Thank you for your patience during this extended hard freeze.
>
> Just one more step to go. There has been a delay in the review of the patch
> series with the uncrustify source changes. PR(6). This patch series was not sent
> out as patch review email because of its very large size. It only contains source
> style changes and the CompareBuild tool and GitHub action has shown there are
> no binary differences introduced with these source style changes.
>
> If you are a package maintainer, then please review the following branch/PR for
> your package contents and review the EDK II CI results and BuildCompare results.
> I do not expect a line by line review because we already had time to provide
> feedback on the source style performed by uncrustify. Instead, a Reviewed-by
> for your package indicates that you have reviewed the EDK II CI results and
> CompareBuild tool functionality and results and you accept the source style
> changes to your package.
>
> *
> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChan
> ges_V5
> * https://github.com/tianocore/edk2/pull/2229
> * https://github.com/mdkinney/edk2/actions/runs/1521618836
>
> Additional details on this update below.
>
> Thank you,
>
> Mike
>
>
> Changes from Update #3
> ----------------------------------------------------------------------------
> * Pushed PR (5)
> * Added link to PR(6). EDK II CI Status is PASS. Build Compare PASS.
> * Waiting for review of PR (6)
> * Review of PR (7) completed and waiting for review of PR (6)
> ----------------------------------------------------------------------------
>
> Changes from Update #2
> ----------------------------------------------------------------------------
> * Changed order of PRs swapping (4) and (5). The PR that activates
> increases the max CI agent job time is independent of all the other
> PRs and its review is complete, so it can be committed now.
> * Pushed PRs (1), (2), (3), (4).
> * Waiting for review to complete for PRs (5) and (6)
> * Reviews complete for PR (7)
> * Identifies steps using git filter-branch to apply uncrustify changes to a
> code review patch series that was generated before the uncrustify changes
> avoiding manual merge.
> * Identified steps using git filter-repo to generate an alternate history of
> the edk2 repo with uncrustify changes applied on every commit. This may
> be useful when evaluating changes to files using tools like git blame
> without the large uncrustify patch series.
> ---------------------------------------------------------------------------
>
> Changes from Update #1
> ----------------------------------------------------------------------------
> * Changed order of PRs swapping (6) and (7). The PR that activates
> EDK II CI check UncrustifyCheck has to be last because it unconditionally
> checks all C/H files in all packages. Not just files that have been
> modified like some of the other checkers.
> * Updated link to the branch with the UncrustifyCheck plugin that has been
> updated with a one line change and Reviewed-by and Tested-by tags.
>
> https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v
> 6
> * Reviews complete for (1), (2), (3), (5), and (7)
> ---------------------------------------------------------------------------
>
> Michael Kubacki and I have prepared the patches required to apply the uncrustify
> changes and enable EDK II CI to check all submitted patches have been run
> through uncrustify.
>
> We have verified through the CompareBuild GitHub Action that the format
> changes performed by uncrustify have no functional changes.
> All of the OBJ, LIB, DLL, EFI, FFS, FV, and FD files match 100% across 70
> VS2019/GCC5 builds of all package/platform DSC files in the edk2 repo.
>
> The hard freeze will be extended after the edk2-stable202111 tag until all
> uncrustify related changes are committed. We do not expect this to take more
> than a few days. Do not push any PRs until the hard freeze is lifted.
>
> The changes are broken up into 7 patch series/PRs. The PRs are ordered so they
> can be submitted using the normal submission process and EDK II CI will pass for
> each one. Details are listed below.
>
> Uncrustify 73.0.3 for EDK II
> =============================
> * Sources: https://dev.azure.com/projectmu/_git/Uncrustify
> * Documentation:
> https://dev.azure.com/projectmu/Uncrustify/_wiki/wikis/Uncrustify.wiki/1/Proj
> ect-Mu-(EDK-II)-Fork-Readme
> * Download:
> https://dev.azure.com/projectmu/Uncrustify/_packaging?_a=package&feed=m
> u_uncrustify&package=mu-uncrustify-
> release&protocolType=NuGet&version=73.0.3
>
> Installing Uncrustify
> ======================
> The Uncrustify tool is installed automatically when the Pytools environment is
> used and the stuart* commands are run to complete the environment setup.
> Please see:
>
> https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally
>
> Uncrustify can also be installed from the download page listed above or built
> from sources from the source link above.
>
> The Documentation link provides instruction on how to run uncrustify from the
> command line or install as a Visual Studio Code plugin. The main uncrustify
> documentation also describes how to integrate with a few other editors.
>
> We have also discussed a client side githook. That effort has not started.
> Let us know if that is a feature you would find useful.
>
> Developer impact for new code reviews
> ======================================
> Once the uncrustify checker is active in EDK II CI, developers must make sure
> their patches are run through the uncrustify tool before sending the patches for
> review.
>
> Developers must install and run uncrustify against changes files before sending
> patch review emails or submitting PR for EDK II CI. If EDK II CI detects and
> differences in source formatting, then EDK II CI will fail and the developer must
> run uncrustify and resubmit the patches.
>
> Developer impact to patch series/PRs reviewed during edk2-stable201121
> soft/hard freeze
> =================================================================
> ======================
> Developers must rebase their changes after the uncrustify source changes are
> committed. The branch with a preview of the uncrustify changes can be used to
> start this rebase work.
>
>
> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChan
> ges_V5
>
> The following steps can be used to update an existing branch with the required
> uncrustify format. This is the Windows version. I will add the Linux version soon.
>
> 1) Fetch and checkout and rebase to latest edk2/master
>
> git fetch origin
> git checkout master
> git rebase origin/master
>
> 2) Make a backup copy of plugin UncrustifyCheck outside WORKSPACE.
> (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable and
> EDK II specific uncrustify configuration file available when working
> with a branch that does not have those tools in its scope.
>
> xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
>
> 3) Check out the patch series branch (e.g. MyBranch)
>
> git checkout MyBranch
>
> 4) Rebase patch series against edk2-stable202111
>
> git rebase edk2-stable202111
>
> 5) Create new branch for the uncrustifed version (e.g. MyBranch_Uncrustified)
>
> git checkout -b MyBranch_Uncrustified
>
> 6) Use git filter-branch to uncrustify all the commits in the series
> between the rebase target from (2) and HEAD of the branch. A filter
> can be used to scope the uncrustify operations to only the C/H files
> in the specific package the patch series is against. (e.g. DynamicTablesPkg).
> BaseTools should always be excluded. If the package scoped filter is
> not used, it will still work, but will take longer to run because
> uncrustify will rescan every C/H files in the whole repo.
>
> git filter-branch --tree-filter "git ls-files DynamicTablesPkg*.c
> DynamicTablesPkg*.h :!BaseTools/* | c:\\Temp\\UncrustifyCheck\\mu-
> uncrustify-release_extdep\\Windows-x86\\uncrustify.exe -c
> c:\\Temp\\UncrustifyCheck\\uncrustify.cfg -F - --replace --no-backup --if-
> changed" edk2-stable202111..HEAD
>
> 7) Now that all the individual patches in the branch are uncrustified,
> rebase against latest edk2/master that is already uncrustified.
>
> git rebase master
>
> 8) Verify the patches in this new branch.
>
> Impacts to tracing history across the uncrusity changes
> =======================================================
> Tools the view file and line history do work with the large uncrustify patch series.
> One impact is that the operations can be very slow due to the large uncrustify
> patches.
>
> One option to provide a faster experience is to provide an alternate version of
> the edk2 repository as "documentation" that has the entire history re-written
> with uncrustify run on every commit.
> The tool called git-filter-repo can be used to perform this transformation and
> runs in a reasonable period of time (a few hours)
>
> https://github.com/newren/git-filter-repo
> https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-
> demos/lint-history
>
> The following steps can be used to perform this transformation.
> This is the Windows version. I will add the Linux version soon.
>
> ** WARNING ** This operation modifies(rewrites) all the commits
> in the local copy of the repo. Do not perform
> these steps on a local repo you are using for
> active development.
>
> 1) Clone edk2 into a new directory (see **WARNING**)
>
> git clone https://github.com/tianocore/edk2.git edk2-uncrustified
> cd edk2-uncrustified
>
> 2) Setup python virtual env, install pytools, and run stuart commands
> to setup build environment which includes installing uncrustify tools.
>
> https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-
> locally
>
> 3) Make a backup copy of plugin UncrustifyCheck outside WORKSPACE.
> (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable and
> EDK II specific uncrustify configuration file available when working
> with a branch that does not have those tools in its scope.
>
> xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
>
> 4) Use lint-history.py from git-filter-repo examples
>
> https://github.com/newren/git-filter-repo
> https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-
> demos/lint-history
>
> Line #127 - Add try except around subprocess.check_call() with except
> being pass. This is required because there are a few commits of C
> files in the edk2 repo that have incorrect C syntax and do not
> build with a C compiler and break the uncrustify parser. Skip reformat
> of C files that can not be parsed by uncrustify. These rare instances
> are addressed in the commit that fixes the C syntax error.
>
> Run this slightly modified version of lint-history. Include only
> C/H files and exclude directories that start with 'Tools' or 'BaseTools'.
> This step took about 2.2 hours on a laptop.
>
> lint-history.py
> --relevant "return (not filename.startswith(b'Tools') and not
> filename.startswith(b'BaseTools') and (filename.endswith(b'.c') or
> filename.endswith(b'.h')))"
> c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\mu-uncrustify-
> release_extdep\\Windows-x86\\uncrustify.exe -c
> c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\uncrustify.cfg --replace --
> no-backup --if-changed
>
> Order of PRs to apply during extended hard freeze
> ==================================================
> 1) Update EmulatorPkg Win Host [BuildOptions] MSFT CC_FLAGS to not force
> debug information
> * https://bugzilla.tianocore.org/show_bug.cgi?id=3747
> *
> https://github.com/mdkinney/edk2/tree/Bug_3747_EmulatorPkg_WinHost_Rep
> roducibleBuild
> * https://github.com/tianocore/edk2/pull/2215
> * Required for EmulatorPkg to pass CompareBuild for VS2019 IA32/X64 builds.
> * Status: Review complete. PR pushed.
>
> 2) EccCheck should not revert staged and local changes
> * https://bugzilla.tianocore.org/show_bug.cgi?id=2986
> *
> https://github.com/mdkinney/edk2/tree/Bug_2986_EccCheckRemoveGitRevert
> _V2
> * https://github.com/tianocore/edk2/pull/2216
> * Required for EDK II CI to complete in a reasonable period of time when
> processing the 4000+ source file style changes made by uncrustify.
> * Also fixes critical bugs that can potentially corrupt git state when
> EccCheck is run locally.
> * Status: Review complete. PR pushed.
>
> 3) Update pytool LicenseCheck plugin to use temp directory for diff output file
> * https://bugzilla.tianocore.org/show_bug.cgi?id=3746
> *
> https://github.com/mdkinney/edk2/tree/Bug_3746_LicenseCheckUseDiffOutput
> File_V2
> * https://github.com/tianocore/edk2/pull/2217
> * Required to reduce EDK II CI build times.
> * Status: Review complete. PR pushed.
>
> 4) Update max job time from 60 min to 120 minutes in .azurepipelines/templates
> * https://bugzilla.tianocore.org/show_bug.cgi?id=3750
> *
> https://github.com/mdkinney/edk2/tree/Bug_3750_IncreaseAzurePipelinesTime
> out
> * https://github.com/tianocore/edk2/pull/2219
> * Required to allow EccCheck of uncrustify changes to complete on Azure
> Pipelines CI agents without timing out.
> * Status: Review complete. PR pushed.
>
> 5) Update Package YAML to ignore specific ECC files/errors
> * https://bugzilla.tianocore.org/show_bug.cgi?id=3749
> *
> https://github.com/mdkinney/edk2/tree/Bug_3749_EccCheckIgnoreFilesErrors
> * https://github.com/tianocore/edk2/pull/2218
> * Required to pass EccCheck
> * Status: Review complete. PR pushed
>
> 6) Uncrustify Source Changes
> * https://bugzilla.tianocore.org/show_bug.cgi?id=3737
> * https://bugzilla.tianocore.org/show_bug.cgi?id=3739
> *
> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChan
> ges_V5
> * https://github.com/tianocore/edk2/pull/2229
> * Build comparison result PASS:
> https://github.com/mdkinney/edk2/actions/runs/1521618836
> * EFI_D_ -> DEBUG changes required to pass PatchCheck
> * Uncrustify format changes required to pass UncrustifyCheck
> * Status:
> Waiting for review
>
> 7) UncrustifyCheck EDK II CI Plugin
> * https://bugzilla.tianocore.org/show_bug.cgi?id=3748
> *
> https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v
> 6
> * Required to enforce all PRs submitted to EDK II CI match uncrustify format.
> * Unconditionally checks all packages. Can not be committed until all C/H
> source files have been updated.
> * Status: Review complete
>
> Combined Branch/PR for Review/Test
> ==================================
> * Build Comparison results must pass 100% across the full set of PRs before
> the individual PRs can be pushed in the order listed above.
> * Branch:
> https://github.com/mdkinney/edk2/tree/TestOnly_Uncrustify_PR_Series
> * PR: https://github.com/tianocore/edk2/pull/2229
> Status = PASS
> * CompareBuild:
> Branch:
> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChan
> ges_V5
> --ref1: ef9a059cdb15844fe52a49af2bf7d86b9dd3e9bf
> --ref2: Bug_3737_3739_ApplyUncrustifyChanges_V5
> Extra Options: -n 4 --quiet
> Results: https://github.com/mdkinney/edk2/actions/runs/1521618836
> 30 VS2019 build comparisons PASS
> 40 GCC5 build comparisons PASS
> 100% PASS
>
> The following git log shows the set of patches from --ref1 to --ref 2across which
> there are no differences in any of the OBJ/LIB/DLL/EFI/FFS/FV/FD files.
>
> --ref2
> b7d4bf0675b7 (HEAD -> Bug_3737_3739_ApplyUncrustifyChanges_V5)
> UnitTestFrameworkPkg: Apply uncrusitify changes
> 7f03d25f60e7 UefiPayloadPkg: Apply uncrusitify changes
> 0bfd8d9b5ac9 UefiCpuPkg: Apply uncrusitify changes
> e1cd9bfb9dea StandaloneMmPkg: Apply uncrusitify changes
> 5da2f65be378 SourceLevelDebugPkg: Apply uncrusitify changes
> 95b86de07e5d SignedCapsulePkg: Apply uncrusitify changes
> fe71d97246c4 ShellPkg: Apply uncrusitify changes
> 54c21c952992 SecurityPkg: Apply uncrusitify changes
> 187a3785f12b RedfishPkg: Apply uncrusitify changes
> 810100002a46 PcAtChipsetPkg: Apply uncrusitify changes
> 276a695c0cf2 OvmfPkg: Apply uncrusitify changes
> 303c0a91ab07 NetworkPkg: Apply uncrusitify changes
> bc80792cd1b1 MdePkg: Apply uncrusitify changes
> 3ea86be17a2a MdeModulePkg: Apply uncrusitify changes
> c70ef11ed0cd IntelFsp2WrapperPkg: Apply uncrusitify changes
> c0291221f252 IntelFsp2Pkg: Apply uncrusitify changes
> 6a479952a690 FmpDevicePkg: Apply uncrusitify changes
> 3a7c05b7070d FatPkg: Apply uncrusitify changes
> b789f98c8959 EmulatorPkg: Apply uncrusitify changes
> 952d7a1c9220 EmbeddedPkg: Apply uncrusitify changes
> a1cc9881bab6 DynamicTablesPkg: Apply uncrusitify changes
> 50654dfe5785 CryptoPkg: Apply uncrusitify changes
> ed965a02dfa1 ArmVirtPkg: Apply uncrusitify changes
> 9744023fbc46 ArmPlatformPkg: Apply uncrusitify changes
> 7a1cde5f5bba ArmPkg: Apply uncrusitify changes
> 19d17e0913e8 UefiCpuPkg: Change use of EFI_D_* to DEBUG_*
> ffa718b4f994 SourceLevelDebugPkg: Change use of EFI_D_* to DEBUG_*
> b86cb3c5e5b4 ShellPkg: Change use of EFI_D_* to DEBUG_*
> c7c42204dc07 SecurityPkg: Change use of EFI_D_* to DEBUG_*
> 16b8e6f958e4 PcAtChipsetPkg: Change use of EFI_D_* to DEBUG_*
> 0ac3f8b2dac5 OvmfPkg: Change use of EFI_D_* to DEBUG_*
> bc5004b8d294 NetworkPkg: Change use of EFI_D_* to DEBUG_*
> 6f671a8e2377 MdePkg: Change use of EFI_D_* to DEBUG_*
> a10c610ff9a3 MdeModulePkg: Change use of EFI_D_* to DEBUG_*
> 09a3bddba390 FatPkg: Change use of EFI_D_* to DEBUG_*
> 59c61318246a EmulatorPkg: Change use of EFI_D_* to DEBUG_*
> 3a80367dda3b EmbeddedPkg: Change use of EFI_D_* to DEBUG_*
> 23eb1aaf80ca ArmVirtPkg: Change use of EFI_D_* to DEBUG_*
> 875914b45c54 ArmPlatformPkg: Change use of EFI_D_* to DEBUG_*
> eb2eca82b451 ArmPkg: Change use of EFI_D_* to DEBUG_*
> f0f3f5aae7c4 (origin/master, origin/HEAD, master) UnitTestFrameworkPkg:
> Update YAML to ignore specific ECC files/errors
> c05734797790 UefiPayloadPkg: Update YAML to ignore specific ECC files/errors
> c30c40d6c63d StandaloneMmPkg: Update YAML to ignore specific ECC
> files/errors
> 9944508e85f1 ShellPkg: Update YAML to ignore specific ECC files/errors
> 60fa40be458d SecurityPkg: Update YAML to ignore specific ECC files/errors
> df790cd6b37e MdePkg: Update YAML to ignore specific ECC files/errors
> 9deb9370766e MdeModulePkg: Update YAML to ignore specific ECC files/errors
> d7d30e8f219f EmulatorPkg: Update YAML to ignore specific ECC files/errors
> d5744ecba813 CryptoPkg: Update YAML to ignore specific ECC files/errors
> c97fee87f0f9 ArmVirtPkg: Update YAML to ignore specific ECC files/errors
> 1939fc9569f2 ArmPlatformPkg: Update YAML to ignore specific ECC files/errors
> 365dced2c37a ArmPkg: Update YAML to ignore specific ECC files/errors
> 76a1ce4d5fec .azurepipelines/templates: Update max pipeline job time to 2
> hours
> 99f84ff47390 .pytools/Plugin/LicenseCheck: Use temp directory for git diff
> output
> 3019f1bbabf1 .pytool/Plugin/EccCheck: Add performance optimizations
> 854462bd3479 .pytool/Plugin/EccCheck: Remove temp directory on exception
> 69877614fdee .pytool/Plugin/EccCheck: Remove RevertCode()
> --ref1
> ef9a059cdb15 EmulatorPkg/Win/Host: Update CC_FLAGS
> bb1bba3d7767 (tag: edk2-stable202111) NetworkPkg: Fix invalid pointer for DNS
> response token on error
>
> Best regards,
>
> Mike
>
>
>
>
>
>
>
>
>
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
2021-11-30 22:34 Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4 Michael D Kinney
` (3 preceding siblings ...)
2021-12-01 2:45 ` Chiu, Chasel
@ 2021-12-01 5:34 ` Wang, Jian J
2021-12-01 6:38 ` Wang, Jian J
` (4 subsequent siblings)
9 siblings, 0 replies; 39+ messages in thread
From: Wang, Jian J @ 2021-12-01 5:34 UTC (permalink / raw)
To: devel@edk2.groups.io, Kinney, Michael D, Michael Kubacki,
Andrew Fish (afish@apple.com), Leif Lindholm
For SecurityPkg and CryptoPkg,
Reviewed-by: Jian J Wang <jian.j.wang@intel.com>
Regards,
Jian
> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Michael D
> Kinney
> Sent: Wednesday, December 01, 2021 6:34 AM
> To: devel@edk2.groups.io; Kinney, Michael D <michael.d.kinney@intel.com>;
> Michael Kubacki <michael.kubacki@microsoft.com>; Andrew Fish
> (afish@apple.com) <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>
> Subject: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard
> Freeze Update #4
>
> Hello,
>
> Thank you for your patience during this extended hard freeze.
>
> Just one more step to go. There has been a delay in the review of
> the patch series with the uncrustify source changes. PR(6). This
> patch series was not sent out as patch review email because of its
> very large size. It only contains source style changes and the
> CompareBuild tool and GitHub action has shown there are no binary
> differences introduced with these source style changes.
>
> If you are a package maintainer, then please review the following
> branch/PR for your package contents and review the EDK II CI results
> and BuildCompare results. I do not expect a line by line review
> because we already had time to provide feedback on the source style
> performed by uncrustify. Instead, a Reviewed-by for your package
> indicates that you have reviewed the EDK II CI results and CompareBuild
> tool functionality and results and you accept the source style
> changes to your package.
>
> *
> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChan
> ges_V5
> * https://github.com/tianocore/edk2/pull/2229
> * https://github.com/mdkinney/edk2/actions/runs/1521618836
>
> Additional details on this update below.
>
> Thank you,
>
> Mike
>
>
> Changes from Update #3
> ----------------------------------------------------------------------------
> * Pushed PR (5)
> * Added link to PR(6). EDK II CI Status is PASS. Build Compare PASS.
> * Waiting for review of PR (6)
> * Review of PR (7) completed and waiting for review of PR (6)
> ----------------------------------------------------------------------------
>
> Changes from Update #2
> ----------------------------------------------------------------------------
> * Changed order of PRs swapping (4) and (5). The PR that activates
> increases the max CI agent job time is independent of all the other
> PRs and its review is complete, so it can be committed now.
> * Pushed PRs (1), (2), (3), (4).
> * Waiting for review to complete for PRs (5) and (6)
> * Reviews complete for PR (7)
> * Identifies steps using git filter-branch to apply uncrustify changes to a
> code review patch series that was generated before the uncrustify changes
> avoiding manual merge.
> * Identified steps using git filter-repo to generate an alternate history of
> the edk2 repo with uncrustify changes applied on every commit. This may
> be useful when evaluating changes to files using tools like git blame
> without the large uncrustify patch series.
> ---------------------------------------------------------------------------
>
> Changes from Update #1
> ----------------------------------------------------------------------------
> * Changed order of PRs swapping (6) and (7). The PR that activates
> EDK II CI check UncrustifyCheck has to be last because it unconditionally
> checks all C/H files in all packages. Not just files that have been
> modified like some of the other checkers.
> * Updated link to the branch with the UncrustifyCheck plugin that has been
> updated with a one line change and Reviewed-by and Tested-by tags.
>
> https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v
> 6
> * Reviews complete for (1), (2), (3), (5), and (7)
> ---------------------------------------------------------------------------
>
> Michael Kubacki and I have prepared the patches required to apply the
> uncrustify changes and enable EDK II CI to check all submitted
> patches have been run through uncrustify.
>
> We have verified through the CompareBuild GitHub Action that the
> format changes performed by uncrustify have no functional changes.
> All of the OBJ, LIB, DLL, EFI, FFS, FV, and FD files match 100%
> across 70 VS2019/GCC5 builds of all package/platform DSC files in
> the edk2 repo.
>
> The hard freeze will be extended after the edk2-stable202111 tag until
> all uncrustify related changes are committed. We do not expect this
> to take more than a few days. Do not push any PRs until the hard
> freeze is lifted.
>
> The changes are broken up into 7 patch series/PRs. The PRs are ordered
> so they can be submitted using the normal submission process and EDK II
> CI will pass for each one. Details are listed below.
>
> Uncrustify 73.0.3 for EDK II
> =============================
> * Sources: https://dev.azure.com/projectmu/_git/Uncrustify
> * Documentation:
> https://dev.azure.com/projectmu/Uncrustify/_wiki/wikis/Uncrustify.wiki/1/Proj
> ect-Mu-(EDK-II)-Fork-Readme
> * Download:
> https://dev.azure.com/projectmu/Uncrustify/_packaging?_a=package&feed=m
> u_uncrustify&package=mu-uncrustify-
> release&protocolType=NuGet&version=73.0.3
>
> Installing Uncrustify
> ======================
> The Uncrustify tool is installed automatically when the Pytools
> environment is used and the stuart* commands are run to complete the
> environment setup. Please see:
>
> https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally
>
> Uncrustify can also be installed from the download page listed above
> or built from sources from the source link above.
>
> The Documentation link provides instruction on how to run uncrustify from
> the command line or install as a Visual Studio Code plugin. The main
> uncrustify documentation also describes how to integrate with a few other
> editors.
>
> We have also discussed a client side githook. That effort has not started.
> Let us know if that is a feature you would find useful.
>
> Developer impact for new code reviews
> ======================================
> Once the uncrustify checker is active in EDK II CI, developers must
> make sure their patches are run through the uncrustify tool before
> sending the patches for review.
>
> Developers must install and run uncrustify against changes files before
> sending patch review emails or submitting PR for EDK II CI. If EDK II CI
> detects and differences in source formatting, then EDK II CI will fail
> and the developer must run uncrustify and resubmit the patches.
>
> Developer impact to patch series/PRs reviewed during edk2-stable201121
> soft/hard freeze
> =================================================================
> ======================
> Developers must rebase their changes after the uncrustify source changes are
> committed. The branch with a preview of the uncrustify changes can be used
> to start this rebase work.
>
>
> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChan
> ges_V5
>
> The following steps can be used to update an existing branch with the
> required uncrustify format. This is the Windows version. I will add
> the Linux version soon.
>
> 1) Fetch and checkout and rebase to latest edk2/master
>
> git fetch origin
> git checkout master
> git rebase origin/master
>
> 2) Make a backup copy of plugin UncrustifyCheck outside WORKSPACE.
> (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable and
> EDK II specific uncrustify configuration file available when working
> with a branch that does not have those tools in its scope.
>
> xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
>
> 3) Check out the patch series branch (e.g. MyBranch)
>
> git checkout MyBranch
>
> 4) Rebase patch series against edk2-stable202111
>
> git rebase edk2-stable202111
>
> 5) Create new branch for the uncrustifed version (e.g. MyBranch_Uncrustified)
>
> git checkout -b MyBranch_Uncrustified
>
> 6) Use git filter-branch to uncrustify all the commits in the series
> between the rebase target from (2) and HEAD of the branch. A filter
> can be used to scope the uncrustify operations to only the C/H files
> in the specific package the patch series is against. (e.g. DynamicTablesPkg).
> BaseTools should always be excluded. If the package scoped filter is
> not used, it will still work, but will take longer to run because
> uncrustify will rescan every C/H files in the whole repo.
>
> git filter-branch --tree-filter "git ls-files DynamicTablesPkg*.c
> DynamicTablesPkg*.h :!BaseTools/* | c:\\Temp\\UncrustifyCheck\\mu-
> uncrustify-release_extdep\\Windows-x86\\uncrustify.exe -c
> c:\\Temp\\UncrustifyCheck\\uncrustify.cfg -F - --replace --no-backup --if-
> changed" edk2-stable202111..HEAD
>
> 7) Now that all the individual patches in the branch are uncrustified,
> rebase against latest edk2/master that is already uncrustified.
>
> git rebase master
>
> 8) Verify the patches in this new branch.
>
> Impacts to tracing history across the uncrusity changes
> =======================================================
> Tools the view file and line history do work with the large uncrustify
> patch series. One impact is that the operations can be very slow due
> to the large uncrustify patches.
>
> One option to provide a faster experience is to provide an alternate
> version of the edk2 repository as "documentation" that has the
> entire history re-written with uncrustify run on every commit.
> The tool called git-filter-repo can be used to perform this
> transformation and runs in a reasonable period of time (a few hours)
>
> https://github.com/newren/git-filter-repo
> https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-
> demos/lint-history
>
> The following steps can be used to perform this transformation.
> This is the Windows version. I will add the Linux version soon.
>
> ** WARNING ** This operation modifies(rewrites) all the commits
> in the local copy of the repo. Do not perform
> these steps on a local repo you are using for
> active development.
>
> 1) Clone edk2 into a new directory (see **WARNING**)
>
> git clone https://github.com/tianocore/edk2.git edk2-uncrustified
> cd edk2-uncrustified
>
> 2) Setup python virtual env, install pytools, and run stuart commands
> to setup build environment which includes installing uncrustify tools.
>
> https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-
> locally
>
> 3) Make a backup copy of plugin UncrustifyCheck outside WORKSPACE.
> (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable and
> EDK II specific uncrustify configuration file available when working
> with a branch that does not have those tools in its scope.
>
> xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
>
> 4) Use lint-history.py from git-filter-repo examples
>
> https://github.com/newren/git-filter-repo
> https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-
> demos/lint-history
>
> Line #127 - Add try except around subprocess.check_call() with except
> being pass. This is required because there are a few commits of C
> files in the edk2 repo that have incorrect C syntax and do not
> build with a C compiler and break the uncrustify parser. Skip reformat
> of C files that can not be parsed by uncrustify. These rare instances
> are addressed in the commit that fixes the C syntax error.
>
> Run this slightly modified version of lint-history. Include only
> C/H files and exclude directories that start with 'Tools' or 'BaseTools'.
> This step took about 2.2 hours on a laptop.
>
> lint-history.py
> --relevant "return (not filename.startswith(b'Tools') and not
> filename.startswith(b'BaseTools') and (filename.endswith(b'.c') or
> filename.endswith(b'.h')))"
> c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\mu-uncrustify-
> release_extdep\\Windows-x86\\uncrustify.exe -c
> c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\uncrustify.cfg --replace --
> no-backup --if-changed
>
> Order of PRs to apply during extended hard freeze
> ==================================================
> 1) Update EmulatorPkg Win Host [BuildOptions] MSFT CC_FLAGS to not force
> debug information
> * https://bugzilla.tianocore.org/show_bug.cgi?id=3747
> *
> https://github.com/mdkinney/edk2/tree/Bug_3747_EmulatorPkg_WinHost_Rep
> roducibleBuild
> * https://github.com/tianocore/edk2/pull/2215
> * Required for EmulatorPkg to pass CompareBuild for VS2019 IA32/X64 builds.
> * Status: Review complete. PR pushed.
>
> 2) EccCheck should not revert staged and local changes
> * https://bugzilla.tianocore.org/show_bug.cgi?id=2986
> *
> https://github.com/mdkinney/edk2/tree/Bug_2986_EccCheckRemoveGitRevert
> _V2
> * https://github.com/tianocore/edk2/pull/2216
> * Required for EDK II CI to complete in a reasonable period of time when
> processing the 4000+ source file style changes made by uncrustify.
> * Also fixes critical bugs that can potentially corrupt git state when
> EccCheck is run locally.
> * Status: Review complete. PR pushed.
>
> 3) Update pytool LicenseCheck plugin to use temp directory for diff output file
> * https://bugzilla.tianocore.org/show_bug.cgi?id=3746
> *
> https://github.com/mdkinney/edk2/tree/Bug_3746_LicenseCheckUseDiffOutput
> File_V2
> * https://github.com/tianocore/edk2/pull/2217
> * Required to reduce EDK II CI build times.
> * Status: Review complete. PR pushed.
>
> 4) Update max job time from 60 min to 120 minutes in .azurepipelines/templates
> * https://bugzilla.tianocore.org/show_bug.cgi?id=3750
> *
> https://github.com/mdkinney/edk2/tree/Bug_3750_IncreaseAzurePipelinesTime
> out
> * https://github.com/tianocore/edk2/pull/2219
> * Required to allow EccCheck of uncrustify changes to complete on Azure
> Pipelines CI agents without timing out.
> * Status: Review complete. PR pushed.
>
> 5) Update Package YAML to ignore specific ECC files/errors
> * https://bugzilla.tianocore.org/show_bug.cgi?id=3749
> *
> https://github.com/mdkinney/edk2/tree/Bug_3749_EccCheckIgnoreFilesErrors
> * https://github.com/tianocore/edk2/pull/2218
> * Required to pass EccCheck
> * Status: Review complete. PR pushed
>
> 6) Uncrustify Source Changes
> * https://bugzilla.tianocore.org/show_bug.cgi?id=3737
> * https://bugzilla.tianocore.org/show_bug.cgi?id=3739
> *
> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChan
> ges_V5
> * https://github.com/tianocore/edk2/pull/2229
> * Build comparison result PASS:
> https://github.com/mdkinney/edk2/actions/runs/1521618836
> * EFI_D_ -> DEBUG changes required to pass PatchCheck
> * Uncrustify format changes required to pass UncrustifyCheck
> * Status:
> Waiting for review
>
> 7) UncrustifyCheck EDK II CI Plugin
> * https://bugzilla.tianocore.org/show_bug.cgi?id=3748
> *
> https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v
> 6
> * Required to enforce all PRs submitted to EDK II CI match uncrustify format.
> * Unconditionally checks all packages. Can not be committed until all C/H
> source files have been updated.
> * Status: Review complete
>
> Combined Branch/PR for Review/Test
> ==================================
> * Build Comparison results must pass 100% across the full set of PRs before
> the individual PRs can be pushed in the order listed above.
> * Branch:
> https://github.com/mdkinney/edk2/tree/TestOnly_Uncrustify_PR_Series
> * PR: https://github.com/tianocore/edk2/pull/2229
> Status = PASS
> * CompareBuild:
> Branch:
> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChan
> ges_V5
> --ref1: ef9a059cdb15844fe52a49af2bf7d86b9dd3e9bf
> --ref2: Bug_3737_3739_ApplyUncrustifyChanges_V5
> Extra Options: -n 4 --quiet
> Results: https://github.com/mdkinney/edk2/actions/runs/1521618836
> 30 VS2019 build comparisons PASS
> 40 GCC5 build comparisons PASS
> 100% PASS
>
> The following git log shows the set of patches from --ref1 to --ref 2across
> which there are no differences in any of the OBJ/LIB/DLL/EFI/FFS/FV/FD files.
>
> --ref2
> b7d4bf0675b7 (HEAD -> Bug_3737_3739_ApplyUncrustifyChanges_V5)
> UnitTestFrameworkPkg: Apply uncrusitify changes
> 7f03d25f60e7 UefiPayloadPkg: Apply uncrusitify changes
> 0bfd8d9b5ac9 UefiCpuPkg: Apply uncrusitify changes
> e1cd9bfb9dea StandaloneMmPkg: Apply uncrusitify changes
> 5da2f65be378 SourceLevelDebugPkg: Apply uncrusitify changes
> 95b86de07e5d SignedCapsulePkg: Apply uncrusitify changes
> fe71d97246c4 ShellPkg: Apply uncrusitify changes
> 54c21c952992 SecurityPkg: Apply uncrusitify changes
> 187a3785f12b RedfishPkg: Apply uncrusitify changes
> 810100002a46 PcAtChipsetPkg: Apply uncrusitify changes
> 276a695c0cf2 OvmfPkg: Apply uncrusitify changes
> 303c0a91ab07 NetworkPkg: Apply uncrusitify changes
> bc80792cd1b1 MdePkg: Apply uncrusitify changes
> 3ea86be17a2a MdeModulePkg: Apply uncrusitify changes
> c70ef11ed0cd IntelFsp2WrapperPkg: Apply uncrusitify changes
> c0291221f252 IntelFsp2Pkg: Apply uncrusitify changes
> 6a479952a690 FmpDevicePkg: Apply uncrusitify changes
> 3a7c05b7070d FatPkg: Apply uncrusitify changes
> b789f98c8959 EmulatorPkg: Apply uncrusitify changes
> 952d7a1c9220 EmbeddedPkg: Apply uncrusitify changes
> a1cc9881bab6 DynamicTablesPkg: Apply uncrusitify changes
> 50654dfe5785 CryptoPkg: Apply uncrusitify changes
> ed965a02dfa1 ArmVirtPkg: Apply uncrusitify changes
> 9744023fbc46 ArmPlatformPkg: Apply uncrusitify changes
> 7a1cde5f5bba ArmPkg: Apply uncrusitify changes
> 19d17e0913e8 UefiCpuPkg: Change use of EFI_D_* to DEBUG_*
> ffa718b4f994 SourceLevelDebugPkg: Change use of EFI_D_* to DEBUG_*
> b86cb3c5e5b4 ShellPkg: Change use of EFI_D_* to DEBUG_*
> c7c42204dc07 SecurityPkg: Change use of EFI_D_* to DEBUG_*
> 16b8e6f958e4 PcAtChipsetPkg: Change use of EFI_D_* to DEBUG_*
> 0ac3f8b2dac5 OvmfPkg: Change use of EFI_D_* to DEBUG_*
> bc5004b8d294 NetworkPkg: Change use of EFI_D_* to DEBUG_*
> 6f671a8e2377 MdePkg: Change use of EFI_D_* to DEBUG_*
> a10c610ff9a3 MdeModulePkg: Change use of EFI_D_* to DEBUG_*
> 09a3bddba390 FatPkg: Change use of EFI_D_* to DEBUG_*
> 59c61318246a EmulatorPkg: Change use of EFI_D_* to DEBUG_*
> 3a80367dda3b EmbeddedPkg: Change use of EFI_D_* to DEBUG_*
> 23eb1aaf80ca ArmVirtPkg: Change use of EFI_D_* to DEBUG_*
> 875914b45c54 ArmPlatformPkg: Change use of EFI_D_* to DEBUG_*
> eb2eca82b451 ArmPkg: Change use of EFI_D_* to DEBUG_*
> f0f3f5aae7c4 (origin/master, origin/HEAD, master) UnitTestFrameworkPkg:
> Update YAML to ignore specific ECC files/errors
> c05734797790 UefiPayloadPkg: Update YAML to ignore specific ECC files/errors
> c30c40d6c63d StandaloneMmPkg: Update YAML to ignore specific ECC
> files/errors
> 9944508e85f1 ShellPkg: Update YAML to ignore specific ECC files/errors
> 60fa40be458d SecurityPkg: Update YAML to ignore specific ECC files/errors
> df790cd6b37e MdePkg: Update YAML to ignore specific ECC files/errors
> 9deb9370766e MdeModulePkg: Update YAML to ignore specific ECC files/errors
> d7d30e8f219f EmulatorPkg: Update YAML to ignore specific ECC files/errors
> d5744ecba813 CryptoPkg: Update YAML to ignore specific ECC files/errors
> c97fee87f0f9 ArmVirtPkg: Update YAML to ignore specific ECC files/errors
> 1939fc9569f2 ArmPlatformPkg: Update YAML to ignore specific ECC files/errors
> 365dced2c37a ArmPkg: Update YAML to ignore specific ECC files/errors
> 76a1ce4d5fec .azurepipelines/templates: Update max pipeline job time to 2
> hours
> 99f84ff47390 .pytools/Plugin/LicenseCheck: Use temp directory for git diff
> output
> 3019f1bbabf1 .pytool/Plugin/EccCheck: Add performance optimizations
> 854462bd3479 .pytool/Plugin/EccCheck: Remove temp directory on exception
> 69877614fdee .pytool/Plugin/EccCheck: Remove RevertCode()
> --ref1
> ef9a059cdb15 EmulatorPkg/Win/Host: Update CC_FLAGS
> bb1bba3d7767 (tag: edk2-stable202111) NetworkPkg: Fix invalid pointer for DNS
> response token on error
>
> Best regards,
>
> Mike
>
>
>
>
>
>
>
>
>
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
2021-11-30 22:34 Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4 Michael D Kinney
` (4 preceding siblings ...)
2021-12-01 5:34 ` Wang, Jian J
@ 2021-12-01 6:38 ` Wang, Jian J
2021-12-01 7:33 ` [edk2-devel] " Abner Chang
2021-12-01 12:59 ` Sami Mujawar
` (3 subsequent siblings)
9 siblings, 1 reply; 39+ messages in thread
From: Wang, Jian J @ 2021-12-01 6:38 UTC (permalink / raw)
To: devel@edk2.groups.io, Kinney, Michael D, Michael Kubacki,
Andrew Fish (afish@apple.com), Leif Lindholm
For SignedCapulePkg,
Reviewed-by: Jian J Wang <jian.j.wang@intel.com>
Regards,
Jian
> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Michael D
> Kinney
> Sent: Wednesday, December 01, 2021 6:34 AM
> To: devel@edk2.groups.io; Kinney, Michael D <michael.d.kinney@intel.com>;
> Michael Kubacki <michael.kubacki@microsoft.com>; Andrew Fish
> (afish@apple.com) <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>
> Subject: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard
> Freeze Update #4
>
> Hello,
>
> Thank you for your patience during this extended hard freeze.
>
> Just one more step to go. There has been a delay in the review of
> the patch series with the uncrustify source changes. PR(6). This
> patch series was not sent out as patch review email because of its
> very large size. It only contains source style changes and the
> CompareBuild tool and GitHub action has shown there are no binary
> differences introduced with these source style changes.
>
> If you are a package maintainer, then please review the following
> branch/PR for your package contents and review the EDK II CI results
> and BuildCompare results. I do not expect a line by line review
> because we already had time to provide feedback on the source style
> performed by uncrustify. Instead, a Reviewed-by for your package
> indicates that you have reviewed the EDK II CI results and CompareBuild
> tool functionality and results and you accept the source style
> changes to your package.
>
> *
> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChan
> ges_V5
> * https://github.com/tianocore/edk2/pull/2229
> * https://github.com/mdkinney/edk2/actions/runs/1521618836
>
> Additional details on this update below.
>
> Thank you,
>
> Mike
>
>
> Changes from Update #3
> ----------------------------------------------------------------------------
> * Pushed PR (5)
> * Added link to PR(6). EDK II CI Status is PASS. Build Compare PASS.
> * Waiting for review of PR (6)
> * Review of PR (7) completed and waiting for review of PR (6)
> ----------------------------------------------------------------------------
>
> Changes from Update #2
> ----------------------------------------------------------------------------
> * Changed order of PRs swapping (4) and (5). The PR that activates
> increases the max CI agent job time is independent of all the other
> PRs and its review is complete, so it can be committed now.
> * Pushed PRs (1), (2), (3), (4).
> * Waiting for review to complete for PRs (5) and (6)
> * Reviews complete for PR (7)
> * Identifies steps using git filter-branch to apply uncrustify changes to a
> code review patch series that was generated before the uncrustify changes
> avoiding manual merge.
> * Identified steps using git filter-repo to generate an alternate history of
> the edk2 repo with uncrustify changes applied on every commit. This may
> be useful when evaluating changes to files using tools like git blame
> without the large uncrustify patch series.
> ---------------------------------------------------------------------------
>
> Changes from Update #1
> ----------------------------------------------------------------------------
> * Changed order of PRs swapping (6) and (7). The PR that activates
> EDK II CI check UncrustifyCheck has to be last because it unconditionally
> checks all C/H files in all packages. Not just files that have been
> modified like some of the other checkers.
> * Updated link to the branch with the UncrustifyCheck plugin that has been
> updated with a one line change and Reviewed-by and Tested-by tags.
>
> https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v
> 6
> * Reviews complete for (1), (2), (3), (5), and (7)
> ---------------------------------------------------------------------------
>
> Michael Kubacki and I have prepared the patches required to apply the
> uncrustify changes and enable EDK II CI to check all submitted
> patches have been run through uncrustify.
>
> We have verified through the CompareBuild GitHub Action that the
> format changes performed by uncrustify have no functional changes.
> All of the OBJ, LIB, DLL, EFI, FFS, FV, and FD files match 100%
> across 70 VS2019/GCC5 builds of all package/platform DSC files in
> the edk2 repo.
>
> The hard freeze will be extended after the edk2-stable202111 tag until
> all uncrustify related changes are committed. We do not expect this
> to take more than a few days. Do not push any PRs until the hard
> freeze is lifted.
>
> The changes are broken up into 7 patch series/PRs. The PRs are ordered
> so they can be submitted using the normal submission process and EDK II
> CI will pass for each one. Details are listed below.
>
> Uncrustify 73.0.3 for EDK II
> =============================
> * Sources: https://dev.azure.com/projectmu/_git/Uncrustify
> * Documentation:
> https://dev.azure.com/projectmu/Uncrustify/_wiki/wikis/Uncrustify.wiki/1/Proj
> ect-Mu-(EDK-II)-Fork-Readme
> * Download:
> https://dev.azure.com/projectmu/Uncrustify/_packaging?_a=package&feed=m
> u_uncrustify&package=mu-uncrustify-
> release&protocolType=NuGet&version=73.0.3
>
> Installing Uncrustify
> ======================
> The Uncrustify tool is installed automatically when the Pytools
> environment is used and the stuart* commands are run to complete the
> environment setup. Please see:
>
> https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally
>
> Uncrustify can also be installed from the download page listed above
> or built from sources from the source link above.
>
> The Documentation link provides instruction on how to run uncrustify from
> the command line or install as a Visual Studio Code plugin. The main
> uncrustify documentation also describes how to integrate with a few other
> editors.
>
> We have also discussed a client side githook. That effort has not started.
> Let us know if that is a feature you would find useful.
>
> Developer impact for new code reviews
> ======================================
> Once the uncrustify checker is active in EDK II CI, developers must
> make sure their patches are run through the uncrustify tool before
> sending the patches for review.
>
> Developers must install and run uncrustify against changes files before
> sending patch review emails or submitting PR for EDK II CI. If EDK II CI
> detects and differences in source formatting, then EDK II CI will fail
> and the developer must run uncrustify and resubmit the patches.
>
> Developer impact to patch series/PRs reviewed during edk2-stable201121
> soft/hard freeze
> =================================================================
> ======================
> Developers must rebase their changes after the uncrustify source changes are
> committed. The branch with a preview of the uncrustify changes can be used
> to start this rebase work.
>
>
> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChan
> ges_V5
>
> The following steps can be used to update an existing branch with the
> required uncrustify format. This is the Windows version. I will add
> the Linux version soon.
>
> 1) Fetch and checkout and rebase to latest edk2/master
>
> git fetch origin
> git checkout master
> git rebase origin/master
>
> 2) Make a backup copy of plugin UncrustifyCheck outside WORKSPACE.
> (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable and
> EDK II specific uncrustify configuration file available when working
> with a branch that does not have those tools in its scope.
>
> xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
>
> 3) Check out the patch series branch (e.g. MyBranch)
>
> git checkout MyBranch
>
> 4) Rebase patch series against edk2-stable202111
>
> git rebase edk2-stable202111
>
> 5) Create new branch for the uncrustifed version (e.g. MyBranch_Uncrustified)
>
> git checkout -b MyBranch_Uncrustified
>
> 6) Use git filter-branch to uncrustify all the commits in the series
> between the rebase target from (2) and HEAD of the branch. A filter
> can be used to scope the uncrustify operations to only the C/H files
> in the specific package the patch series is against. (e.g. DynamicTablesPkg).
> BaseTools should always be excluded. If the package scoped filter is
> not used, it will still work, but will take longer to run because
> uncrustify will rescan every C/H files in the whole repo.
>
> git filter-branch --tree-filter "git ls-files DynamicTablesPkg*.c
> DynamicTablesPkg*.h :!BaseTools/* | c:\\Temp\\UncrustifyCheck\\mu-
> uncrustify-release_extdep\\Windows-x86\\uncrustify.exe -c
> c:\\Temp\\UncrustifyCheck\\uncrustify.cfg -F - --replace --no-backup --if-
> changed" edk2-stable202111..HEAD
>
> 7) Now that all the individual patches in the branch are uncrustified,
> rebase against latest edk2/master that is already uncrustified.
>
> git rebase master
>
> 8) Verify the patches in this new branch.
>
> Impacts to tracing history across the uncrusity changes
> =======================================================
> Tools the view file and line history do work with the large uncrustify
> patch series. One impact is that the operations can be very slow due
> to the large uncrustify patches.
>
> One option to provide a faster experience is to provide an alternate
> version of the edk2 repository as "documentation" that has the
> entire history re-written with uncrustify run on every commit.
> The tool called git-filter-repo can be used to perform this
> transformation and runs in a reasonable period of time (a few hours)
>
> https://github.com/newren/git-filter-repo
> https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-
> demos/lint-history
>
> The following steps can be used to perform this transformation.
> This is the Windows version. I will add the Linux version soon.
>
> ** WARNING ** This operation modifies(rewrites) all the commits
> in the local copy of the repo. Do not perform
> these steps on a local repo you are using for
> active development.
>
> 1) Clone edk2 into a new directory (see **WARNING**)
>
> git clone https://github.com/tianocore/edk2.git edk2-uncrustified
> cd edk2-uncrustified
>
> 2) Setup python virtual env, install pytools, and run stuart commands
> to setup build environment which includes installing uncrustify tools.
>
> https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-
> locally
>
> 3) Make a backup copy of plugin UncrustifyCheck outside WORKSPACE.
> (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable and
> EDK II specific uncrustify configuration file available when working
> with a branch that does not have those tools in its scope.
>
> xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
>
> 4) Use lint-history.py from git-filter-repo examples
>
> https://github.com/newren/git-filter-repo
> https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-
> demos/lint-history
>
> Line #127 - Add try except around subprocess.check_call() with except
> being pass. This is required because there are a few commits of C
> files in the edk2 repo that have incorrect C syntax and do not
> build with a C compiler and break the uncrustify parser. Skip reformat
> of C files that can not be parsed by uncrustify. These rare instances
> are addressed in the commit that fixes the C syntax error.
>
> Run this slightly modified version of lint-history. Include only
> C/H files and exclude directories that start with 'Tools' or 'BaseTools'.
> This step took about 2.2 hours on a laptop.
>
> lint-history.py
> --relevant "return (not filename.startswith(b'Tools') and not
> filename.startswith(b'BaseTools') and (filename.endswith(b'.c') or
> filename.endswith(b'.h')))"
> c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\mu-uncrustify-
> release_extdep\\Windows-x86\\uncrustify.exe -c
> c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\uncrustify.cfg --replace --
> no-backup --if-changed
>
> Order of PRs to apply during extended hard freeze
> ==================================================
> 1) Update EmulatorPkg Win Host [BuildOptions] MSFT CC_FLAGS to not force
> debug information
> * https://bugzilla.tianocore.org/show_bug.cgi?id=3747
> *
> https://github.com/mdkinney/edk2/tree/Bug_3747_EmulatorPkg_WinHost_Rep
> roducibleBuild
> * https://github.com/tianocore/edk2/pull/2215
> * Required for EmulatorPkg to pass CompareBuild for VS2019 IA32/X64 builds.
> * Status: Review complete. PR pushed.
>
> 2) EccCheck should not revert staged and local changes
> * https://bugzilla.tianocore.org/show_bug.cgi?id=2986
> *
> https://github.com/mdkinney/edk2/tree/Bug_2986_EccCheckRemoveGitRevert
> _V2
> * https://github.com/tianocore/edk2/pull/2216
> * Required for EDK II CI to complete in a reasonable period of time when
> processing the 4000+ source file style changes made by uncrustify.
> * Also fixes critical bugs that can potentially corrupt git state when
> EccCheck is run locally.
> * Status: Review complete. PR pushed.
>
> 3) Update pytool LicenseCheck plugin to use temp directory for diff output file
> * https://bugzilla.tianocore.org/show_bug.cgi?id=3746
> *
> https://github.com/mdkinney/edk2/tree/Bug_3746_LicenseCheckUseDiffOutput
> File_V2
> * https://github.com/tianocore/edk2/pull/2217
> * Required to reduce EDK II CI build times.
> * Status: Review complete. PR pushed.
>
> 4) Update max job time from 60 min to 120 minutes in .azurepipelines/templates
> * https://bugzilla.tianocore.org/show_bug.cgi?id=3750
> *
> https://github.com/mdkinney/edk2/tree/Bug_3750_IncreaseAzurePipelinesTime
> out
> * https://github.com/tianocore/edk2/pull/2219
> * Required to allow EccCheck of uncrustify changes to complete on Azure
> Pipelines CI agents without timing out.
> * Status: Review complete. PR pushed.
>
> 5) Update Package YAML to ignore specific ECC files/errors
> * https://bugzilla.tianocore.org/show_bug.cgi?id=3749
> *
> https://github.com/mdkinney/edk2/tree/Bug_3749_EccCheckIgnoreFilesErrors
> * https://github.com/tianocore/edk2/pull/2218
> * Required to pass EccCheck
> * Status: Review complete. PR pushed
>
> 6) Uncrustify Source Changes
> * https://bugzilla.tianocore.org/show_bug.cgi?id=3737
> * https://bugzilla.tianocore.org/show_bug.cgi?id=3739
> *
> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChan
> ges_V5
> * https://github.com/tianocore/edk2/pull/2229
> * Build comparison result PASS:
> https://github.com/mdkinney/edk2/actions/runs/1521618836
> * EFI_D_ -> DEBUG changes required to pass PatchCheck
> * Uncrustify format changes required to pass UncrustifyCheck
> * Status:
> Waiting for review
>
> 7) UncrustifyCheck EDK II CI Plugin
> * https://bugzilla.tianocore.org/show_bug.cgi?id=3748
> *
> https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v
> 6
> * Required to enforce all PRs submitted to EDK II CI match uncrustify format.
> * Unconditionally checks all packages. Can not be committed until all C/H
> source files have been updated.
> * Status: Review complete
>
> Combined Branch/PR for Review/Test
> ==================================
> * Build Comparison results must pass 100% across the full set of PRs before
> the individual PRs can be pushed in the order listed above.
> * Branch:
> https://github.com/mdkinney/edk2/tree/TestOnly_Uncrustify_PR_Series
> * PR: https://github.com/tianocore/edk2/pull/2229
> Status = PASS
> * CompareBuild:
> Branch:
> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChan
> ges_V5
> --ref1: ef9a059cdb15844fe52a49af2bf7d86b9dd3e9bf
> --ref2: Bug_3737_3739_ApplyUncrustifyChanges_V5
> Extra Options: -n 4 --quiet
> Results: https://github.com/mdkinney/edk2/actions/runs/1521618836
> 30 VS2019 build comparisons PASS
> 40 GCC5 build comparisons PASS
> 100% PASS
>
> The following git log shows the set of patches from --ref1 to --ref 2across
> which there are no differences in any of the OBJ/LIB/DLL/EFI/FFS/FV/FD files.
>
> --ref2
> b7d4bf0675b7 (HEAD -> Bug_3737_3739_ApplyUncrustifyChanges_V5)
> UnitTestFrameworkPkg: Apply uncrusitify changes
> 7f03d25f60e7 UefiPayloadPkg: Apply uncrusitify changes
> 0bfd8d9b5ac9 UefiCpuPkg: Apply uncrusitify changes
> e1cd9bfb9dea StandaloneMmPkg: Apply uncrusitify changes
> 5da2f65be378 SourceLevelDebugPkg: Apply uncrusitify changes
> 95b86de07e5d SignedCapsulePkg: Apply uncrusitify changes
> fe71d97246c4 ShellPkg: Apply uncrusitify changes
> 54c21c952992 SecurityPkg: Apply uncrusitify changes
> 187a3785f12b RedfishPkg: Apply uncrusitify changes
> 810100002a46 PcAtChipsetPkg: Apply uncrusitify changes
> 276a695c0cf2 OvmfPkg: Apply uncrusitify changes
> 303c0a91ab07 NetworkPkg: Apply uncrusitify changes
> bc80792cd1b1 MdePkg: Apply uncrusitify changes
> 3ea86be17a2a MdeModulePkg: Apply uncrusitify changes
> c70ef11ed0cd IntelFsp2WrapperPkg: Apply uncrusitify changes
> c0291221f252 IntelFsp2Pkg: Apply uncrusitify changes
> 6a479952a690 FmpDevicePkg: Apply uncrusitify changes
> 3a7c05b7070d FatPkg: Apply uncrusitify changes
> b789f98c8959 EmulatorPkg: Apply uncrusitify changes
> 952d7a1c9220 EmbeddedPkg: Apply uncrusitify changes
> a1cc9881bab6 DynamicTablesPkg: Apply uncrusitify changes
> 50654dfe5785 CryptoPkg: Apply uncrusitify changes
> ed965a02dfa1 ArmVirtPkg: Apply uncrusitify changes
> 9744023fbc46 ArmPlatformPkg: Apply uncrusitify changes
> 7a1cde5f5bba ArmPkg: Apply uncrusitify changes
> 19d17e0913e8 UefiCpuPkg: Change use of EFI_D_* to DEBUG_*
> ffa718b4f994 SourceLevelDebugPkg: Change use of EFI_D_* to DEBUG_*
> b86cb3c5e5b4 ShellPkg: Change use of EFI_D_* to DEBUG_*
> c7c42204dc07 SecurityPkg: Change use of EFI_D_* to DEBUG_*
> 16b8e6f958e4 PcAtChipsetPkg: Change use of EFI_D_* to DEBUG_*
> 0ac3f8b2dac5 OvmfPkg: Change use of EFI_D_* to DEBUG_*
> bc5004b8d294 NetworkPkg: Change use of EFI_D_* to DEBUG_*
> 6f671a8e2377 MdePkg: Change use of EFI_D_* to DEBUG_*
> a10c610ff9a3 MdeModulePkg: Change use of EFI_D_* to DEBUG_*
> 09a3bddba390 FatPkg: Change use of EFI_D_* to DEBUG_*
> 59c61318246a EmulatorPkg: Change use of EFI_D_* to DEBUG_*
> 3a80367dda3b EmbeddedPkg: Change use of EFI_D_* to DEBUG_*
> 23eb1aaf80ca ArmVirtPkg: Change use of EFI_D_* to DEBUG_*
> 875914b45c54 ArmPlatformPkg: Change use of EFI_D_* to DEBUG_*
> eb2eca82b451 ArmPkg: Change use of EFI_D_* to DEBUG_*
> f0f3f5aae7c4 (origin/master, origin/HEAD, master) UnitTestFrameworkPkg:
> Update YAML to ignore specific ECC files/errors
> c05734797790 UefiPayloadPkg: Update YAML to ignore specific ECC files/errors
> c30c40d6c63d StandaloneMmPkg: Update YAML to ignore specific ECC
> files/errors
> 9944508e85f1 ShellPkg: Update YAML to ignore specific ECC files/errors
> 60fa40be458d SecurityPkg: Update YAML to ignore specific ECC files/errors
> df790cd6b37e MdePkg: Update YAML to ignore specific ECC files/errors
> 9deb9370766e MdeModulePkg: Update YAML to ignore specific ECC files/errors
> d7d30e8f219f EmulatorPkg: Update YAML to ignore specific ECC files/errors
> d5744ecba813 CryptoPkg: Update YAML to ignore specific ECC files/errors
> c97fee87f0f9 ArmVirtPkg: Update YAML to ignore specific ECC files/errors
> 1939fc9569f2 ArmPlatformPkg: Update YAML to ignore specific ECC files/errors
> 365dced2c37a ArmPkg: Update YAML to ignore specific ECC files/errors
> 76a1ce4d5fec .azurepipelines/templates: Update max pipeline job time to 2
> hours
> 99f84ff47390 .pytools/Plugin/LicenseCheck: Use temp directory for git diff
> output
> 3019f1bbabf1 .pytool/Plugin/EccCheck: Add performance optimizations
> 854462bd3479 .pytool/Plugin/EccCheck: Remove temp directory on exception
> 69877614fdee .pytool/Plugin/EccCheck: Remove RevertCode()
> --ref1
> ef9a059cdb15 EmulatorPkg/Win/Host: Update CC_FLAGS
> bb1bba3d7767 (tag: edk2-stable202111) NetworkPkg: Fix invalid pointer for DNS
> response token on error
>
> Best regards,
>
> Mike
>
>
>
>
>
>
>
>
>
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
2021-12-01 6:38 ` Wang, Jian J
@ 2021-12-01 7:33 ` Abner Chang
2021-12-01 16:43 ` Michael D Kinney
0 siblings, 1 reply; 39+ messages in thread
From: Abner Chang @ 2021-12-01 7:33 UTC (permalink / raw)
To: devel@edk2.groups.io, jian.j.wang@intel.com, Kinney, Michael D,
Michael Kubacki, Andrew Fish (afish@apple.com), Leif Lindholm
Hi Mike,
Is that possible to not applying uncrusitify on the source files that cloned from other open source?
e.g. RedfishPkg/PrivateLibrary/RedfishLib/edk2libredfish/*.*
Thanks
Abner
> -----Original Message-----
> From: devel@edk2.groups.io [mailto:devel@edk2.groups.io] On Behalf Of
> Wang, Jian J
> Sent: Wednesday, December 1, 2021 2:39 PM
> To: devel@edk2.groups.io; Kinney, Michael D <michael.d.kinney@intel.com>;
> Michael Kubacki <michael.kubacki@microsoft.com>; Andrew Fish
> (afish@apple.com) <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>
> Subject: Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended
> Hard Freeze Update #4
>
> For SignedCapulePkg,
>
> Reviewed-by: Jian J Wang <jian.j.wang@intel.com>
>
> Regards,
> Jian
>
> > -----Original Message-----
> > From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of
> Michael D
> > Kinney
> > Sent: Wednesday, December 01, 2021 6:34 AM
> > To: devel@edk2.groups.io; Kinney, Michael D
> <michael.d.kinney@intel.com>;
> > Michael Kubacki <michael.kubacki@microsoft.com>; Andrew Fish
> > (afish@apple.com) <afish@apple.com>; Leif Lindholm
> <leif@nuviainc.com>
> > Subject: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended
> Hard
> > Freeze Update #4
> >
> > Hello,
> >
> > Thank you for your patience during this extended hard freeze.
> >
> > Just one more step to go. There has been a delay in the review of
> > the patch series with the uncrustify source changes. PR(6). This
> > patch series was not sent out as patch review email because of its
> > very large size. It only contains source style changes and the
> > CompareBuild tool and GitHub action has shown there are no binary
> > differences introduced with these source style changes.
> >
> > If you are a package maintainer, then please review the following
> > branch/PR for your package contents and review the EDK II CI results
> > and BuildCompare results. I do not expect a line by line review
> > because we already had time to provide feedback on the source style
> > performed by uncrustify. Instead, a Reviewed-by for your package
> > indicates that you have reviewed the EDK II CI results and CompareBuild
> > tool functionality and results and you accept the source style
> > changes to your package.
> >
> > *
> >
> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyC
> han
> > ges_V5
> > * https://github.com/tianocore/edk2/pull/2229
> > * https://github.com/mdkinney/edk2/actions/runs/1521618836
> >
> > Additional details on this update below.
> >
> > Thank you,
> >
> > Mike
> >
> >
> > Changes from Update #3
> > ----------------------------------------------------------------------------
> > * Pushed PR (5)
> > * Added link to PR(6). EDK II CI Status is PASS. Build Compare PASS.
> > * Waiting for review of PR (6)
> > * Review of PR (7) completed and waiting for review of PR (6)
> > ----------------------------------------------------------------------------
> >
> > Changes from Update #2
> > ----------------------------------------------------------------------------
> > * Changed order of PRs swapping (4) and (5). The PR that activates
> > increases the max CI agent job time is independent of all the other
> > PRs and its review is complete, so it can be committed now.
> > * Pushed PRs (1), (2), (3), (4).
> > * Waiting for review to complete for PRs (5) and (6)
> > * Reviews complete for PR (7)
> > * Identifies steps using git filter-branch to apply uncrustify changes to a
> > code review patch series that was generated before the uncrustify
> changes
> > avoiding manual merge.
> > * Identified steps using git filter-repo to generate an alternate history of
> > the edk2 repo with uncrustify changes applied on every commit. This may
> > be useful when evaluating changes to files using tools like git blame
> > without the large uncrustify patch series.
> > ---------------------------------------------------------------------------
> >
> > Changes from Update #1
> > ----------------------------------------------------------------------------
> > * Changed order of PRs swapping (6) and (7). The PR that activates
> > EDK II CI check UncrustifyCheck has to be last because it unconditionally
> > checks all C/H files in all packages. Not just files that have been
> > modified like some of the other checkers.
> > * Updated link to the branch with the UncrustifyCheck plugin that has been
> > updated with a one line change and Reviewed-by and Tested-by tags.
> >
> >
> https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plug
> in_v
> > 6
> > * Reviews complete for (1), (2), (3), (5), and (7)
> > ---------------------------------------------------------------------------
> >
> > Michael Kubacki and I have prepared the patches required to apply the
> > uncrustify changes and enable EDK II CI to check all submitted
> > patches have been run through uncrustify.
> >
> > We have verified through the CompareBuild GitHub Action that the
> > format changes performed by uncrustify have no functional changes.
> > All of the OBJ, LIB, DLL, EFI, FFS, FV, and FD files match 100%
> > across 70 VS2019/GCC5 builds of all package/platform DSC files in
> > the edk2 repo.
> >
> > The hard freeze will be extended after the edk2-stable202111 tag until
> > all uncrustify related changes are committed. We do not expect this
> > to take more than a few days. Do not push any PRs until the hard
> > freeze is lifted.
> >
> > The changes are broken up into 7 patch series/PRs. The PRs are ordered
> > so they can be submitted using the normal submission process and EDK II
> > CI will pass for each one. Details are listed below.
> >
> > Uncrustify 73.0.3 for EDK II
> > =============================
> > * Sources:
> INVALID URI REMOVED
> stify__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKkoslR8D0LLPZTWXL0OKtMZJOYp
> cSVHoHwMD2t8$
> > * Documentation:
> >
> INVALID URI REMOVED
> /_wiki/wikis/Uncrustify.wiki/1/Proj__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKk
> oslR8D0LLPZTWXL0OKtMZJOYpcSVHoP8OtlxM$
> > ect-Mu-(EDK-II)-Fork-Readme
> > * Download:
> >
> INVALID URI REMOVED
> /_packaging?_a=package&feed=m__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKk
> oslR8D0LLPZTWXL0OKtMZJOYpcSVHo4yTWeAM$
> > u_uncrustify&package=mu-uncrustify-
> > release&protocolType=NuGet&version=73.0.3
> >
> > Installing Uncrustify
> > ======================
> > The Uncrustify tool is installed automatically when the Pytools
> > environment is used and the stuart* commands are run to complete the
> > environment setup. Please see:
> >
> > https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-
> locally
> >
> > Uncrustify can also be installed from the download page listed above
> > or built from sources from the source link above.
> >
> > The Documentation link provides instruction on how to run uncrustify from
> > the command line or install as a Visual Studio Code plugin. The main
> > uncrustify documentation also describes how to integrate with a few other
> > editors.
> >
> > We have also discussed a client side githook. That effort has not started.
> > Let us know if that is a feature you would find useful.
> >
> > Developer impact for new code reviews
> > ======================================
> > Once the uncrustify checker is active in EDK II CI, developers must
> > make sure their patches are run through the uncrustify tool before
> > sending the patches for review.
> >
> > Developers must install and run uncrustify against changes files before
> > sending patch review emails or submitting PR for EDK II CI. If EDK II CI
> > detects and differences in source formatting, then EDK II CI will fail
> > and the developer must run uncrustify and resubmit the patches.
> >
> > Developer impact to patch series/PRs reviewed during edk2-stable201121
> > soft/hard freeze
> >
> ==========================================================
> =======
> > ======================
> > Developers must rebase their changes after the uncrustify source changes
> are
> > committed. The branch with a preview of the uncrustify changes can be
> used
> > to start this rebase work.
> >
> >
> >
> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyC
> han
> > ges_V5
> >
> > The following steps can be used to update an existing branch with the
> > required uncrustify format. This is the Windows version. I will add
> > the Linux version soon.
> >
> > 1) Fetch and checkout and rebase to latest edk2/master
> >
> > git fetch origin
> > git checkout master
> > git rebase origin/master
> >
> > 2) Make a backup copy of plugin UncrustifyCheck outside WORKSPACE.
> > (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable and
> > EDK II specific uncrustify configuration file available when working
> > with a branch that does not have those tools in its scope.
> >
> > xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
> >
> > 3) Check out the patch series branch (e.g. MyBranch)
> >
> > git checkout MyBranch
> >
> > 4) Rebase patch series against edk2-stable202111
> >
> > git rebase edk2-stable202111
> >
> > 5) Create new branch for the uncrustifed version (e.g.
> MyBranch_Uncrustified)
> >
> > git checkout -b MyBranch_Uncrustified
> >
> > 6) Use git filter-branch to uncrustify all the commits in the series
> > between the rebase target from (2) and HEAD of the branch. A filter
> > can be used to scope the uncrustify operations to only the C/H files
> > in the specific package the patch series is against. (e.g.
> DynamicTablesPkg).
> > BaseTools should always be excluded. If the package scoped filter is
> > not used, it will still work, but will take longer to run because
> > uncrustify will rescan every C/H files in the whole repo.
> >
> > git filter-branch --tree-filter "git ls-files DynamicTablesPkg*.c
> > DynamicTablesPkg*.h :!BaseTools/* | c:\\Temp\\UncrustifyCheck\\mu-
> > uncrustify-release_extdep\\Windows-x86\\uncrustify.exe -c
> > c:\\Temp\\UncrustifyCheck\\uncrustify.cfg -F - --replace --no-backup --if-
> > changed" edk2-stable202111..HEAD
> >
> > 7) Now that all the individual patches in the branch are uncrustified,
> > rebase against latest edk2/master that is already uncrustified.
> >
> > git rebase master
> >
> > 8) Verify the patches in this new branch.
> >
> > Impacts to tracing history across the uncrusity changes
> > =======================================================
> > Tools the view file and line history do work with the large uncrustify
> > patch series. One impact is that the operations can be very slow due
> > to the large uncrustify patches.
> >
> > One option to provide a faster experience is to provide an alternate
> > version of the edk2 repository as "documentation" that has the
> > entire history re-written with uncrustify run on every commit.
> > The tool called git-filter-repo can be used to perform this
> > transformation and runs in a reasonable period of time (a few hours)
> >
> > https://github.com/newren/git-filter-repo
> > https://github.com/newren/git-filter-repo/blob/main/contrib/filter-
> repo-
> > demos/lint-history
> >
> > The following steps can be used to perform this transformation.
> > This is the Windows version. I will add the Linux version soon.
> >
> > ** WARNING ** This operation modifies(rewrites) all the commits
> > in the local copy of the repo. Do not perform
> > these steps on a local repo you are using for
> > active development.
> >
> > 1) Clone edk2 into a new directory (see **WARNING**)
> >
> > git clone https://github.com/tianocore/edk2.git edk2-uncrustified
> > cd edk2-uncrustified
> >
> > 2) Setup python virtual env, install pytools, and run stuart commands
> > to setup build environment which includes installing uncrustify tools.
> >
> > https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-
> > locally
> >
> > 3) Make a backup copy of plugin UncrustifyCheck outside WORKSPACE.
> > (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable and
> > EDK II specific uncrustify configuration file available when working
> > with a branch that does not have those tools in its scope.
> >
> > xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
> >
> > 4) Use lint-history.py from git-filter-repo examples
> >
> > https://github.com/newren/git-filter-repo
> > https://github.com/newren/git-filter-repo/blob/main/contrib/filter-
> repo-
> > demos/lint-history
> >
> > Line #127 - Add try except around subprocess.check_call() with except
> > being pass. This is required because there are a few commits of C
> > files in the edk2 repo that have incorrect C syntax and do not
> > build with a C compiler and break the uncrustify parser. Skip reformat
> > of C files that can not be parsed by uncrustify. These rare instances
> > are addressed in the commit that fixes the C syntax error.
> >
> > Run this slightly modified version of lint-history. Include only
> > C/H files and exclude directories that start with 'Tools' or 'BaseTools'.
> > This step took about 2.2 hours on a laptop.
> >
> > lint-history.py
> > --relevant "return (not filename.startswith(b'Tools') and not
> > filename.startswith(b'BaseTools') and (filename.endswith(b'.c') or
> > filename.endswith(b'.h')))"
> > c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\mu-
> uncrustify-
> > release_extdep\\Windows-x86\\uncrustify.exe -c
> > c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\uncrustify.cfg --
> replace --
> > no-backup --if-changed
> >
> > Order of PRs to apply during extended hard freeze
> > ==================================================
> > 1) Update EmulatorPkg Win Host [BuildOptions] MSFT CC_FLAGS to not
> force
> > debug information
> > *
> INVALID URI REMOVED
> d=3747__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKkoslR8D0LLPZTWXL0OKtMZJ
> OYpcSVHo2BEG33c$
> > *
> >
> https://github.com/mdkinney/edk2/tree/Bug_3747_EmulatorPkg_WinHost_
> Rep
> > roducibleBuild
> > * https://github.com/tianocore/edk2/pull/2215
> > * Required for EmulatorPkg to pass CompareBuild for VS2019 IA32/X64
> builds.
> > * Status: Review complete. PR pushed.
> >
> > 2) EccCheck should not revert staged and local changes
> > *
> INVALID URI REMOVED
> d=2986__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKkoslR8D0LLPZTWXL0OKtMZJ
> OYpcSVHotvR0Unk$
> > *
> >
> https://github.com/mdkinney/edk2/tree/Bug_2986_EccCheckRemoveGitRe
> vert
> > _V2
> > * https://github.com/tianocore/edk2/pull/2216
> > * Required for EDK II CI to complete in a reasonable period of time when
> > processing the 4000+ source file style changes made by uncrustify.
> > * Also fixes critical bugs that can potentially corrupt git state when
> > EccCheck is run locally.
> > * Status: Review complete. PR pushed.
> >
> > 3) Update pytool LicenseCheck plugin to use temp directory for diff output
> file
> > *
> INVALID URI REMOVED
> d=3746__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKkoslR8D0LLPZTWXL0OKtMZJ
> OYpcSVHoWRuHaBw$
> > *
> >
> https://github.com/mdkinney/edk2/tree/Bug_3746_LicenseCheckUseDiffO
> utput
> > File_V2
> > * https://github.com/tianocore/edk2/pull/2217
> > * Required to reduce EDK II CI build times.
> > * Status: Review complete. PR pushed.
> >
> > 4) Update max job time from 60 min to 120 minutes
> in .azurepipelines/templates
> > *
> INVALID URI REMOVED
> d=3750__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKkoslR8D0LLPZTWXL0OKtMZJ
> OYpcSVHozaVshgc$
> > *
> >
> https://github.com/mdkinney/edk2/tree/Bug_3750_IncreaseAzurePipelines
> Time
> > out
> > * https://github.com/tianocore/edk2/pull/2219
> > * Required to allow EccCheck of uncrustify changes to complete on Azure
> > Pipelines CI agents without timing out.
> > * Status: Review complete. PR pushed.
> >
> > 5) Update Package YAML to ignore specific ECC files/errors
> > *
> INVALID URI REMOVED
> d=3749__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKkoslR8D0LLPZTWXL0OKtMZJ
> OYpcSVHo-fY3dMo$
> > *
> >
> https://github.com/mdkinney/edk2/tree/Bug_3749_EccCheckIgnoreFilesErr
> ors
> > * https://github.com/tianocore/edk2/pull/2218
> > * Required to pass EccCheck
> > * Status: Review complete. PR pushed
> >
> > 6) Uncrustify Source Changes
> > *
> INVALID URI REMOVED
> d=3737__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKkoslR8D0LLPZTWXL0OKtMZJ
> OYpcSVHoMRa47YU$
> > *
> INVALID URI REMOVED
> d=3739__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKkoslR8D0LLPZTWXL0OKtMZJ
> OYpcSVHoPAlckP8$
> > *
> >
> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyC
> han
> > ges_V5
> > * https://github.com/tianocore/edk2/pull/2229
> > * Build comparison result PASS:
> > https://github.com/mdkinney/edk2/actions/runs/1521618836
> > * EFI_D_ -> DEBUG changes required to pass PatchCheck
> > * Uncrustify format changes required to pass UncrustifyCheck
> > * Status:
> > Waiting for review
> >
> > 7) UncrustifyCheck EDK II CI Plugin
> > *
> INVALID URI REMOVED
> d=3748__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKkoslR8D0LLPZTWXL0OKtMZJ
> OYpcSVHoCzEWymQ$
> > *
> >
> https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plug
> in_v
> > 6
> > * Required to enforce all PRs submitted to EDK II CI match uncrustify
> format.
> > * Unconditionally checks all packages. Can not be committed until all C/H
> > source files have been updated.
> > * Status: Review complete
> >
> > Combined Branch/PR for Review/Test
> > ==================================
> > * Build Comparison results must pass 100% across the full set of PRs before
> > the individual PRs can be pushed in the order listed above.
> > * Branch:
> > https://github.com/mdkinney/edk2/tree/TestOnly_Uncrustify_PR_Series
> > * PR: https://github.com/tianocore/edk2/pull/2229
> > Status = PASS
> > * CompareBuild:
> > Branch:
> >
> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyC
> han
> > ges_V5
> > --ref1: ef9a059cdb15844fe52a49af2bf7d86b9dd3e9bf
> > --ref2: Bug_3737_3739_ApplyUncrustifyChanges_V5
> > Extra Options: -n 4 --quiet
> > Results: https://github.com/mdkinney/edk2/actions/runs/1521618836
> > 30 VS2019 build comparisons PASS
> > 40 GCC5 build comparisons PASS
> > 100% PASS
> >
> > The following git log shows the set of patches from --ref1 to --ref 2across
> > which there are no differences in any of the OBJ/LIB/DLL/EFI/FFS/FV/FD
> files.
> >
> > --ref2
> > b7d4bf0675b7 (HEAD -> Bug_3737_3739_ApplyUncrustifyChanges_V5)
> > UnitTestFrameworkPkg: Apply uncrusitify changes
> > 7f03d25f60e7 UefiPayloadPkg: Apply uncrusitify changes
> > 0bfd8d9b5ac9 UefiCpuPkg: Apply uncrusitify changes
> > e1cd9bfb9dea StandaloneMmPkg: Apply uncrusitify changes
> > 5da2f65be378 SourceLevelDebugPkg: Apply uncrusitify changes
> > 95b86de07e5d SignedCapsulePkg: Apply uncrusitify changes
> > fe71d97246c4 ShellPkg: Apply uncrusitify changes
> > 54c21c952992 SecurityPkg: Apply uncrusitify changes
> > 187a3785f12b RedfishPkg: Apply uncrusitify changes
> > 810100002a46 PcAtChipsetPkg: Apply uncrusitify changes
> > 276a695c0cf2 OvmfPkg: Apply uncrusitify changes
> > 303c0a91ab07 NetworkPkg: Apply uncrusitify changes
> > bc80792cd1b1 MdePkg: Apply uncrusitify changes
> > 3ea86be17a2a MdeModulePkg: Apply uncrusitify changes
> > c70ef11ed0cd IntelFsp2WrapperPkg: Apply uncrusitify changes
> > c0291221f252 IntelFsp2Pkg: Apply uncrusitify changes
> > 6a479952a690 FmpDevicePkg: Apply uncrusitify changes
> > 3a7c05b7070d FatPkg: Apply uncrusitify changes
> > b789f98c8959 EmulatorPkg: Apply uncrusitify changes
> > 952d7a1c9220 EmbeddedPkg: Apply uncrusitify changes
> > a1cc9881bab6 DynamicTablesPkg: Apply uncrusitify changes
> > 50654dfe5785 CryptoPkg: Apply uncrusitify changes
> > ed965a02dfa1 ArmVirtPkg: Apply uncrusitify changes
> > 9744023fbc46 ArmPlatformPkg: Apply uncrusitify changes
> > 7a1cde5f5bba ArmPkg: Apply uncrusitify changes
> > 19d17e0913e8 UefiCpuPkg: Change use of EFI_D_* to DEBUG_*
> > ffa718b4f994 SourceLevelDebugPkg: Change use of EFI_D_* to DEBUG_*
> > b86cb3c5e5b4 ShellPkg: Change use of EFI_D_* to DEBUG_*
> > c7c42204dc07 SecurityPkg: Change use of EFI_D_* to DEBUG_*
> > 16b8e6f958e4 PcAtChipsetPkg: Change use of EFI_D_* to DEBUG_*
> > 0ac3f8b2dac5 OvmfPkg: Change use of EFI_D_* to DEBUG_*
> > bc5004b8d294 NetworkPkg: Change use of EFI_D_* to DEBUG_*
> > 6f671a8e2377 MdePkg: Change use of EFI_D_* to DEBUG_*
> > a10c610ff9a3 MdeModulePkg: Change use of EFI_D_* to DEBUG_*
> > 09a3bddba390 FatPkg: Change use of EFI_D_* to DEBUG_*
> > 59c61318246a EmulatorPkg: Change use of EFI_D_* to DEBUG_*
> > 3a80367dda3b EmbeddedPkg: Change use of EFI_D_* to DEBUG_*
> > 23eb1aaf80ca ArmVirtPkg: Change use of EFI_D_* to DEBUG_*
> > 875914b45c54 ArmPlatformPkg: Change use of EFI_D_* to DEBUG_*
> > eb2eca82b451 ArmPkg: Change use of EFI_D_* to DEBUG_*
> > f0f3f5aae7c4 (origin/master, origin/HEAD, master) UnitTestFrameworkPkg:
> > Update YAML to ignore specific ECC files/errors
> > c05734797790 UefiPayloadPkg: Update YAML to ignore specific ECC
> files/errors
> > c30c40d6c63d StandaloneMmPkg: Update YAML to ignore specific ECC
> > files/errors
> > 9944508e85f1 ShellPkg: Update YAML to ignore specific ECC files/errors
> > 60fa40be458d SecurityPkg: Update YAML to ignore specific ECC files/errors
> > df790cd6b37e MdePkg: Update YAML to ignore specific ECC files/errors
> > 9deb9370766e MdeModulePkg: Update YAML to ignore specific ECC
> files/errors
> > d7d30e8f219f EmulatorPkg: Update YAML to ignore specific ECC files/errors
> > d5744ecba813 CryptoPkg: Update YAML to ignore specific ECC files/errors
> > c97fee87f0f9 ArmVirtPkg: Update YAML to ignore specific ECC files/errors
> > 1939fc9569f2 ArmPlatformPkg: Update YAML to ignore specific ECC
> files/errors
> > 365dced2c37a ArmPkg: Update YAML to ignore specific ECC files/errors
> > 76a1ce4d5fec .azurepipelines/templates: Update max pipeline job time to
> 2
> > hours
> > 99f84ff47390 .pytools/Plugin/LicenseCheck: Use temp directory for git diff
> > output
> > 3019f1bbabf1 .pytool/Plugin/EccCheck: Add performance optimizations
> > 854462bd3479 .pytool/Plugin/EccCheck: Remove temp directory on
> exception
> > 69877614fdee .pytool/Plugin/EccCheck: Remove RevertCode()
> > --ref1
> > ef9a059cdb15 EmulatorPkg/Win/Host: Update CC_FLAGS
> > bb1bba3d7767 (tag: edk2-stable202111) NetworkPkg: Fix invalid pointer for
> DNS
> > response token on error
> >
> > Best regards,
> >
> > Mike
> >
> >
> >
> >
> >
> >
> >
> >
> >
>
>
>
>
>
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
2021-11-30 22:34 Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4 Michael D Kinney
` (5 preceding siblings ...)
2021-12-01 6:38 ` Wang, Jian J
@ 2021-12-01 12:59 ` Sami Mujawar
2021-12-01 18:52 ` Bret Barkelew
` (2 subsequent siblings)
9 siblings, 0 replies; 39+ messages in thread
From: Sami Mujawar @ 2021-12-01 12:59 UTC (permalink / raw)
To: Michael D Kinney, devel
[-- Attachment #1: Type: text/plain, Size: 193 bytes --]
Hi Mike,
For DynamicTablesPkg, ArmPkg, ArmPlatformPkg, ArmVirtPkg & StandaloneMmPkg
Reviewed-by: Sami Mujawar <sami.mujawar@arm.com> ( sami.mujawar@arm.com )
Regards,
Sami Mujawar
[-- Attachment #2: Type: text/html, Size: 286 bytes --]
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
2021-12-01 7:33 ` [edk2-devel] " Abner Chang
@ 2021-12-01 16:43 ` Michael D Kinney
2021-12-01 17:05 ` Michael Kubacki
0 siblings, 1 reply; 39+ messages in thread
From: Michael D Kinney @ 2021-12-01 16:43 UTC (permalink / raw)
To: devel@edk2.groups.io, Chang, Abner, Wang, Jian J, Michael Kubacki,
Andrew Fish (afish@apple.com), Leif Lindholm, Kinney, Michael D
Hi Abner,
Yes. That is possible. We are already excluding BaseTools.
In order to get the file lists that apply to uncrustify, we would have to change from:
git ls-files *.c *.h :!BaseTools/*
To:
git ls-files *.c *.h :!BaseTools/* :!RedfishPkg/PrivateLibrary/RedfishLib/edk2libredfish/*
However, I do not see a feature in the UncrustifyCheck to check all c/h files in RedFishPkg and exclude one of its directories.
If we can specify an exclusion like this in the RedFishPkg YAML file, then I think that would work well.
Michael Kubacki would have to comment on how we support this in EDK II CI checks.
Best regards,
Mike
> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Abner Chang
> Sent: Tuesday, November 30, 2021 11:34 PM
> To: devel@edk2.groups.io; Wang, Jian J <jian.j.wang@intel.com>; Kinney, Michael D <michael.d.kinney@intel.com>; Michael
> Kubacki <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com) <afish@apple.com>; Leif Lindholm
> <leif@nuviainc.com>
> Subject: Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
>
> Hi Mike,
> Is that possible to not applying uncrusitify on the source files that cloned from other open source?
> e.g. RedfishPkg/PrivateLibrary/RedfishLib/edk2libredfish/*.*
>
> Thanks
> Abner
>
> > -----Original Message-----
> > From: devel@edk2.groups.io [mailto:devel@edk2.groups.io] On Behalf Of
> > Wang, Jian J
> > Sent: Wednesday, December 1, 2021 2:39 PM
> > To: devel@edk2.groups.io; Kinney, Michael D <michael.d.kinney@intel.com>;
> > Michael Kubacki <michael.kubacki@microsoft.com>; Andrew Fish
> > (afish@apple.com) <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>
> > Subject: Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended
> > Hard Freeze Update #4
> >
> > For SignedCapulePkg,
> >
> > Reviewed-by: Jian J Wang <jian.j.wang@intel.com>
> >
> > Regards,
> > Jian
> >
> > > -----Original Message-----
> > > From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of
> > Michael D
> > > Kinney
> > > Sent: Wednesday, December 01, 2021 6:34 AM
> > > To: devel@edk2.groups.io; Kinney, Michael D
> > <michael.d.kinney@intel.com>;
> > > Michael Kubacki <michael.kubacki@microsoft.com>; Andrew Fish
> > > (afish@apple.com) <afish@apple.com>; Leif Lindholm
> > <leif@nuviainc.com>
> > > Subject: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended
> > Hard
> > > Freeze Update #4
> > >
> > > Hello,
> > >
> > > Thank you for your patience during this extended hard freeze.
> > >
> > > Just one more step to go. There has been a delay in the review of
> > > the patch series with the uncrustify source changes. PR(6). This
> > > patch series was not sent out as patch review email because of its
> > > very large size. It only contains source style changes and the
> > > CompareBuild tool and GitHub action has shown there are no binary
> > > differences introduced with these source style changes.
> > >
> > > If you are a package maintainer, then please review the following
> > > branch/PR for your package contents and review the EDK II CI results
> > > and BuildCompare results. I do not expect a line by line review
> > > because we already had time to provide feedback on the source style
> > > performed by uncrustify. Instead, a Reviewed-by for your package
> > > indicates that you have reviewed the EDK II CI results and CompareBuild
> > > tool functionality and results and you accept the source style
> > > changes to your package.
> > >
> > > *
> > >
> > https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyC
> > han
> > > ges_V5
> > > * https://github.com/tianocore/edk2/pull/2229
> > > * https://github.com/mdkinney/edk2/actions/runs/1521618836
> > >
> > > Additional details on this update below.
> > >
> > > Thank you,
> > >
> > > Mike
> > >
> > >
> > > Changes from Update #3
> > > ----------------------------------------------------------------------------
> > > * Pushed PR (5)
> > > * Added link to PR(6). EDK II CI Status is PASS. Build Compare PASS.
> > > * Waiting for review of PR (6)
> > > * Review of PR (7) completed and waiting for review of PR (6)
> > > ----------------------------------------------------------------------------
> > >
> > > Changes from Update #2
> > > ----------------------------------------------------------------------------
> > > * Changed order of PRs swapping (4) and (5). The PR that activates
> > > increases the max CI agent job time is independent of all the other
> > > PRs and its review is complete, so it can be committed now.
> > > * Pushed PRs (1), (2), (3), (4).
> > > * Waiting for review to complete for PRs (5) and (6)
> > > * Reviews complete for PR (7)
> > > * Identifies steps using git filter-branch to apply uncrustify changes to a
> > > code review patch series that was generated before the uncrustify
> > changes
> > > avoiding manual merge.
> > > * Identified steps using git filter-repo to generate an alternate history of
> > > the edk2 repo with uncrustify changes applied on every commit. This may
> > > be useful when evaluating changes to files using tools like git blame
> > > without the large uncrustify patch series.
> > > ---------------------------------------------------------------------------
> > >
> > > Changes from Update #1
> > > ----------------------------------------------------------------------------
> > > * Changed order of PRs swapping (6) and (7). The PR that activates
> > > EDK II CI check UncrustifyCheck has to be last because it unconditionally
> > > checks all C/H files in all packages. Not just files that have been
> > > modified like some of the other checkers.
> > > * Updated link to the branch with the UncrustifyCheck plugin that has been
> > > updated with a one line change and Reviewed-by and Tested-by tags.
> > >
> > >
> > https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plug
> > in_v
> > > 6
> > > * Reviews complete for (1), (2), (3), (5), and (7)
> > > ---------------------------------------------------------------------------
> > >
> > > Michael Kubacki and I have prepared the patches required to apply the
> > > uncrustify changes and enable EDK II CI to check all submitted
> > > patches have been run through uncrustify.
> > >
> > > We have verified through the CompareBuild GitHub Action that the
> > > format changes performed by uncrustify have no functional changes.
> > > All of the OBJ, LIB, DLL, EFI, FFS, FV, and FD files match 100%
> > > across 70 VS2019/GCC5 builds of all package/platform DSC files in
> > > the edk2 repo.
> > >
> > > The hard freeze will be extended after the edk2-stable202111 tag until
> > > all uncrustify related changes are committed. We do not expect this
> > > to take more than a few days. Do not push any PRs until the hard
> > > freeze is lifted.
> > >
> > > The changes are broken up into 7 patch series/PRs. The PRs are ordered
> > > so they can be submitted using the normal submission process and EDK II
> > > CI will pass for each one. Details are listed below.
> > >
> > > Uncrustify 73.0.3 for EDK II
> > > =============================
> > > * Sources:
> > INVALID URI REMOVED
> > stify__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKkoslR8D0LLPZTWXL0OKtMZJOYp
> > cSVHoHwMD2t8$
> > > * Documentation:
> > >
> > INVALID URI REMOVED
> > /_wiki/wikis/Uncrustify.wiki/1/Proj__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKk
> > oslR8D0LLPZTWXL0OKtMZJOYpcSVHoP8OtlxM$
> > > ect-Mu-(EDK-II)-Fork-Readme
> > > * Download:
> > >
> > INVALID URI REMOVED
> > /_packaging?_a=package&feed=m__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKk
> > oslR8D0LLPZTWXL0OKtMZJOYpcSVHo4yTWeAM$
> > > u_uncrustify&package=mu-uncrustify-
> > > release&protocolType=NuGet&version=73.0.3
> > >
> > > Installing Uncrustify
> > > ======================
> > > The Uncrustify tool is installed automatically when the Pytools
> > > environment is used and the stuart* commands are run to complete the
> > > environment setup. Please see:
> > >
> > > https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-
> > locally
> > >
> > > Uncrustify can also be installed from the download page listed above
> > > or built from sources from the source link above.
> > >
> > > The Documentation link provides instruction on how to run uncrustify from
> > > the command line or install as a Visual Studio Code plugin. The main
> > > uncrustify documentation also describes how to integrate with a few other
> > > editors.
> > >
> > > We have also discussed a client side githook. That effort has not started.
> > > Let us know if that is a feature you would find useful.
> > >
> > > Developer impact for new code reviews
> > > ======================================
> > > Once the uncrustify checker is active in EDK II CI, developers must
> > > make sure their patches are run through the uncrustify tool before
> > > sending the patches for review.
> > >
> > > Developers must install and run uncrustify against changes files before
> > > sending patch review emails or submitting PR for EDK II CI. If EDK II CI
> > > detects and differences in source formatting, then EDK II CI will fail
> > > and the developer must run uncrustify and resubmit the patches.
> > >
> > > Developer impact to patch series/PRs reviewed during edk2-stable201121
> > > soft/hard freeze
> > >
> > ==========================================================
> > =======
> > > ======================
> > > Developers must rebase their changes after the uncrustify source changes
> > are
> > > committed. The branch with a preview of the uncrustify changes can be
> > used
> > > to start this rebase work.
> > >
> > >
> > >
> > https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyC
> > han
> > > ges_V5
> > >
> > > The following steps can be used to update an existing branch with the
> > > required uncrustify format. This is the Windows version. I will add
> > > the Linux version soon.
> > >
> > > 1) Fetch and checkout and rebase to latest edk2/master
> > >
> > > git fetch origin
> > > git checkout master
> > > git rebase origin/master
> > >
> > > 2) Make a backup copy of plugin UncrustifyCheck outside WORKSPACE.
> > > (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable and
> > > EDK II specific uncrustify configuration file available when working
> > > with a branch that does not have those tools in its scope.
> > >
> > > xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
> > >
> > > 3) Check out the patch series branch (e.g. MyBranch)
> > >
> > > git checkout MyBranch
> > >
> > > 4) Rebase patch series against edk2-stable202111
> > >
> > > git rebase edk2-stable202111
> > >
> > > 5) Create new branch for the uncrustifed version (e.g.
> > MyBranch_Uncrustified)
> > >
> > > git checkout -b MyBranch_Uncrustified
> > >
> > > 6) Use git filter-branch to uncrustify all the commits in the series
> > > between the rebase target from (2) and HEAD of the branch. A filter
> > > can be used to scope the uncrustify operations to only the C/H files
> > > in the specific package the patch series is against. (e.g.
> > DynamicTablesPkg).
> > > BaseTools should always be excluded. If the package scoped filter is
> > > not used, it will still work, but will take longer to run because
> > > uncrustify will rescan every C/H files in the whole repo.
> > >
> > > git filter-branch --tree-filter "git ls-files DynamicTablesPkg*.c
> > > DynamicTablesPkg*.h :!BaseTools/* | c:\\Temp\\UncrustifyCheck\\mu-
> > > uncrustify-release_extdep\\Windows-x86\\uncrustify.exe -c
> > > c:\\Temp\\UncrustifyCheck\\uncrustify.cfg -F - --replace --no-backup --if-
> > > changed" edk2-stable202111..HEAD
> > >
> > > 7) Now that all the individual patches in the branch are uncrustified,
> > > rebase against latest edk2/master that is already uncrustified.
> > >
> > > git rebase master
> > >
> > > 8) Verify the patches in this new branch.
> > >
> > > Impacts to tracing history across the uncrusity changes
> > > =======================================================
> > > Tools the view file and line history do work with the large uncrustify
> > > patch series. One impact is that the operations can be very slow due
> > > to the large uncrustify patches.
> > >
> > > One option to provide a faster experience is to provide an alternate
> > > version of the edk2 repository as "documentation" that has the
> > > entire history re-written with uncrustify run on every commit.
> > > The tool called git-filter-repo can be used to perform this
> > > transformation and runs in a reasonable period of time (a few hours)
> > >
> > > https://github.com/newren/git-filter-repo
> > > https://github.com/newren/git-filter-repo/blob/main/contrib/filter-
> > repo-
> > > demos/lint-history
> > >
> > > The following steps can be used to perform this transformation.
> > > This is the Windows version. I will add the Linux version soon.
> > >
> > > ** WARNING ** This operation modifies(rewrites) all the commits
> > > in the local copy of the repo. Do not perform
> > > these steps on a local repo you are using for
> > > active development.
> > >
> > > 1) Clone edk2 into a new directory (see **WARNING**)
> > >
> > > git clone https://github.com/tianocore/edk2.git edk2-uncrustified
> > > cd edk2-uncrustified
> > >
> > > 2) Setup python virtual env, install pytools, and run stuart commands
> > > to setup build environment which includes installing uncrustify tools.
> > >
> > > https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-
> > > locally
> > >
> > > 3) Make a backup copy of plugin UncrustifyCheck outside WORKSPACE.
> > > (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable and
> > > EDK II specific uncrustify configuration file available when working
> > > with a branch that does not have those tools in its scope.
> > >
> > > xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
> > >
> > > 4) Use lint-history.py from git-filter-repo examples
> > >
> > > https://github.com/newren/git-filter-repo
> > > https://github.com/newren/git-filter-repo/blob/main/contrib/filter-
> > repo-
> > > demos/lint-history
> > >
> > > Line #127 - Add try except around subprocess.check_call() with except
> > > being pass. This is required because there are a few commits of C
> > > files in the edk2 repo that have incorrect C syntax and do not
> > > build with a C compiler and break the uncrustify parser. Skip reformat
> > > of C files that can not be parsed by uncrustify. These rare instances
> > > are addressed in the commit that fixes the C syntax error.
> > >
> > > Run this slightly modified version of lint-history. Include only
> > > C/H files and exclude directories that start with 'Tools' or 'BaseTools'.
> > > This step took about 2.2 hours on a laptop.
> > >
> > > lint-history.py
> > > --relevant "return (not filename.startswith(b'Tools') and not
> > > filename.startswith(b'BaseTools') and (filename.endswith(b'.c') or
> > > filename.endswith(b'.h')))"
> > > c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\mu-
> > uncrustify-
> > > release_extdep\\Windows-x86\\uncrustify.exe -c
> > > c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\uncrustify.cfg --
> > replace --
> > > no-backup --if-changed
> > >
> > > Order of PRs to apply during extended hard freeze
> > > ==================================================
> > > 1) Update EmulatorPkg Win Host [BuildOptions] MSFT CC_FLAGS to not
> > force
> > > debug information
> > > *
> > INVALID URI REMOVED
> > d=3747__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKkoslR8D0LLPZTWXL0OKtMZJ
> > OYpcSVHo2BEG33c$
> > > *
> > >
> > https://github.com/mdkinney/edk2/tree/Bug_3747_EmulatorPkg_WinHost_
> > Rep
> > > roducibleBuild
> > > * https://github.com/tianocore/edk2/pull/2215
> > > * Required for EmulatorPkg to pass CompareBuild for VS2019 IA32/X64
> > builds.
> > > * Status: Review complete. PR pushed.
> > >
> > > 2) EccCheck should not revert staged and local changes
> > > *
> > INVALID URI REMOVED
> > d=2986__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKkoslR8D0LLPZTWXL0OKtMZJ
> > OYpcSVHotvR0Unk$
> > > *
> > >
> > https://github.com/mdkinney/edk2/tree/Bug_2986_EccCheckRemoveGitRe
> > vert
> > > _V2
> > > * https://github.com/tianocore/edk2/pull/2216
> > > * Required for EDK II CI to complete in a reasonable period of time when
> > > processing the 4000+ source file style changes made by uncrustify.
> > > * Also fixes critical bugs that can potentially corrupt git state when
> > > EccCheck is run locally.
> > > * Status: Review complete. PR pushed.
> > >
> > > 3) Update pytool LicenseCheck plugin to use temp directory for diff output
> > file
> > > *
> > INVALID URI REMOVED
> > d=3746__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKkoslR8D0LLPZTWXL0OKtMZJ
> > OYpcSVHoWRuHaBw$
> > > *
> > >
> > https://github.com/mdkinney/edk2/tree/Bug_3746_LicenseCheckUseDiffO
> > utput
> > > File_V2
> > > * https://github.com/tianocore/edk2/pull/2217
> > > * Required to reduce EDK II CI build times.
> > > * Status: Review complete. PR pushed.
> > >
> > > 4) Update max job time from 60 min to 120 minutes
> > in .azurepipelines/templates
> > > *
> > INVALID URI REMOVED
> > d=3750__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKkoslR8D0LLPZTWXL0OKtMZJ
> > OYpcSVHozaVshgc$
> > > *
> > >
> > https://github.com/mdkinney/edk2/tree/Bug_3750_IncreaseAzurePipelines
> > Time
> > > out
> > > * https://github.com/tianocore/edk2/pull/2219
> > > * Required to allow EccCheck of uncrustify changes to complete on Azure
> > > Pipelines CI agents without timing out.
> > > * Status: Review complete. PR pushed.
> > >
> > > 5) Update Package YAML to ignore specific ECC files/errors
> > > *
> > INVALID URI REMOVED
> > d=3749__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKkoslR8D0LLPZTWXL0OKtMZJ
> > OYpcSVHo-fY3dMo$
> > > *
> > >
> > https://github.com/mdkinney/edk2/tree/Bug_3749_EccCheckIgnoreFilesErr
> > ors
> > > * https://github.com/tianocore/edk2/pull/2218
> > > * Required to pass EccCheck
> > > * Status: Review complete. PR pushed
> > >
> > > 6) Uncrustify Source Changes
> > > *
> > INVALID URI REMOVED
> > d=3737__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKkoslR8D0LLPZTWXL0OKtMZJ
> > OYpcSVHoMRa47YU$
> > > *
> > INVALID URI REMOVED
> > d=3739__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKkoslR8D0LLPZTWXL0OKtMZJ
> > OYpcSVHoPAlckP8$
> > > *
> > >
> > https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyC
> > han
> > > ges_V5
> > > * https://github.com/tianocore/edk2/pull/2229
> > > * Build comparison result PASS:
> > > https://github.com/mdkinney/edk2/actions/runs/1521618836
> > > * EFI_D_ -> DEBUG changes required to pass PatchCheck
> > > * Uncrustify format changes required to pass UncrustifyCheck
> > > * Status:
> > > Waiting for review
> > >
> > > 7) UncrustifyCheck EDK II CI Plugin
> > > *
> > INVALID URI REMOVED
> > d=3748__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKkoslR8D0LLPZTWXL0OKtMZJ
> > OYpcSVHoCzEWymQ$
> > > *
> > >
> > https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plug
> > in_v
> > > 6
> > > * Required to enforce all PRs submitted to EDK II CI match uncrustify
> > format.
> > > * Unconditionally checks all packages. Can not be committed until all C/H
> > > source files have been updated.
> > > * Status: Review complete
> > >
> > > Combined Branch/PR for Review/Test
> > > ==================================
> > > * Build Comparison results must pass 100% across the full set of PRs before
> > > the individual PRs can be pushed in the order listed above.
> > > * Branch:
> > > https://github.com/mdkinney/edk2/tree/TestOnly_Uncrustify_PR_Series
> > > * PR: https://github.com/tianocore/edk2/pull/2229
> > > Status = PASS
> > > * CompareBuild:
> > > Branch:
> > >
> > https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyC
> > han
> > > ges_V5
> > > --ref1: ef9a059cdb15844fe52a49af2bf7d86b9dd3e9bf
> > > --ref2: Bug_3737_3739_ApplyUncrustifyChanges_V5
> > > Extra Options: -n 4 --quiet
> > > Results: https://github.com/mdkinney/edk2/actions/runs/1521618836
> > > 30 VS2019 build comparisons PASS
> > > 40 GCC5 build comparisons PASS
> > > 100% PASS
> > >
> > > The following git log shows the set of patches from --ref1 to --ref 2across
> > > which there are no differences in any of the OBJ/LIB/DLL/EFI/FFS/FV/FD
> > files.
> > >
> > > --ref2
> > > b7d4bf0675b7 (HEAD -> Bug_3737_3739_ApplyUncrustifyChanges_V5)
> > > UnitTestFrameworkPkg: Apply uncrusitify changes
> > > 7f03d25f60e7 UefiPayloadPkg: Apply uncrusitify changes
> > > 0bfd8d9b5ac9 UefiCpuPkg: Apply uncrusitify changes
> > > e1cd9bfb9dea StandaloneMmPkg: Apply uncrusitify changes
> > > 5da2f65be378 SourceLevelDebugPkg: Apply uncrusitify changes
> > > 95b86de07e5d SignedCapsulePkg: Apply uncrusitify changes
> > > fe71d97246c4 ShellPkg: Apply uncrusitify changes
> > > 54c21c952992 SecurityPkg: Apply uncrusitify changes
> > > 187a3785f12b RedfishPkg: Apply uncrusitify changes
> > > 810100002a46 PcAtChipsetPkg: Apply uncrusitify changes
> > > 276a695c0cf2 OvmfPkg: Apply uncrusitify changes
> > > 303c0a91ab07 NetworkPkg: Apply uncrusitify changes
> > > bc80792cd1b1 MdePkg: Apply uncrusitify changes
> > > 3ea86be17a2a MdeModulePkg: Apply uncrusitify changes
> > > c70ef11ed0cd IntelFsp2WrapperPkg: Apply uncrusitify changes
> > > c0291221f252 IntelFsp2Pkg: Apply uncrusitify changes
> > > 6a479952a690 FmpDevicePkg: Apply uncrusitify changes
> > > 3a7c05b7070d FatPkg: Apply uncrusitify changes
> > > b789f98c8959 EmulatorPkg: Apply uncrusitify changes
> > > 952d7a1c9220 EmbeddedPkg: Apply uncrusitify changes
> > > a1cc9881bab6 DynamicTablesPkg: Apply uncrusitify changes
> > > 50654dfe5785 CryptoPkg: Apply uncrusitify changes
> > > ed965a02dfa1 ArmVirtPkg: Apply uncrusitify changes
> > > 9744023fbc46 ArmPlatformPkg: Apply uncrusitify changes
> > > 7a1cde5f5bba ArmPkg: Apply uncrusitify changes
> > > 19d17e0913e8 UefiCpuPkg: Change use of EFI_D_* to DEBUG_*
> > > ffa718b4f994 SourceLevelDebugPkg: Change use of EFI_D_* to DEBUG_*
> > > b86cb3c5e5b4 ShellPkg: Change use of EFI_D_* to DEBUG_*
> > > c7c42204dc07 SecurityPkg: Change use of EFI_D_* to DEBUG_*
> > > 16b8e6f958e4 PcAtChipsetPkg: Change use of EFI_D_* to DEBUG_*
> > > 0ac3f8b2dac5 OvmfPkg: Change use of EFI_D_* to DEBUG_*
> > > bc5004b8d294 NetworkPkg: Change use of EFI_D_* to DEBUG_*
> > > 6f671a8e2377 MdePkg: Change use of EFI_D_* to DEBUG_*
> > > a10c610ff9a3 MdeModulePkg: Change use of EFI_D_* to DEBUG_*
> > > 09a3bddba390 FatPkg: Change use of EFI_D_* to DEBUG_*
> > > 59c61318246a EmulatorPkg: Change use of EFI_D_* to DEBUG_*
> > > 3a80367dda3b EmbeddedPkg: Change use of EFI_D_* to DEBUG_*
> > > 23eb1aaf80ca ArmVirtPkg: Change use of EFI_D_* to DEBUG_*
> > > 875914b45c54 ArmPlatformPkg: Change use of EFI_D_* to DEBUG_*
> > > eb2eca82b451 ArmPkg: Change use of EFI_D_* to DEBUG_*
> > > f0f3f5aae7c4 (origin/master, origin/HEAD, master) UnitTestFrameworkPkg:
> > > Update YAML to ignore specific ECC files/errors
> > > c05734797790 UefiPayloadPkg: Update YAML to ignore specific ECC
> > files/errors
> > > c30c40d6c63d StandaloneMmPkg: Update YAML to ignore specific ECC
> > > files/errors
> > > 9944508e85f1 ShellPkg: Update YAML to ignore specific ECC files/errors
> > > 60fa40be458d SecurityPkg: Update YAML to ignore specific ECC files/errors
> > > df790cd6b37e MdePkg: Update YAML to ignore specific ECC files/errors
> > > 9deb9370766e MdeModulePkg: Update YAML to ignore specific ECC
> > files/errors
> > > d7d30e8f219f EmulatorPkg: Update YAML to ignore specific ECC files/errors
> > > d5744ecba813 CryptoPkg: Update YAML to ignore specific ECC files/errors
> > > c97fee87f0f9 ArmVirtPkg: Update YAML to ignore specific ECC files/errors
> > > 1939fc9569f2 ArmPlatformPkg: Update YAML to ignore specific ECC
> > files/errors
> > > 365dced2c37a ArmPkg: Update YAML to ignore specific ECC files/errors
> > > 76a1ce4d5fec .azurepipelines/templates: Update max pipeline job time to
> > 2
> > > hours
> > > 99f84ff47390 .pytools/Plugin/LicenseCheck: Use temp directory for git diff
> > > output
> > > 3019f1bbabf1 .pytool/Plugin/EccCheck: Add performance optimizations
> > > 854462bd3479 .pytool/Plugin/EccCheck: Remove temp directory on
> > exception
> > > 69877614fdee .pytool/Plugin/EccCheck: Remove RevertCode()
> > > --ref1
> > > ef9a059cdb15 EmulatorPkg/Win/Host: Update CC_FLAGS
> > > bb1bba3d7767 (tag: edk2-stable202111) NetworkPkg: Fix invalid pointer for
> > DNS
> > > response token on error
> > >
> > > Best regards,
> > >
> > > Mike
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> >
> >
> >
> >
> >
>
>
>
>
>
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
2021-12-01 16:43 ` Michael D Kinney
@ 2021-12-01 17:05 ` Michael Kubacki
2021-12-01 17:39 ` Michael D Kinney
2021-12-02 11:00 ` Gerd Hoffmann
0 siblings, 2 replies; 39+ messages in thread
From: Michael Kubacki @ 2021-12-01 17:05 UTC (permalink / raw)
To: devel, michael.d.kinney, Chang, Abner, Wang, Jian J,
Michael Kubacki, Andrew Fish (afish@apple.com), Leif Lindholm
This can of course be done at a technical level but it smells of a code
cohesion problem.
The code is in the edk2 project so it would reason that it should be
uniform in style with other code and at least close to the EDK II C
Coding Standard Specification.
Currently, git submodules and ignored path are already excluded in
addition to the ability to mark packages in "audit mode" so they do not
fail the results. I would personally prefer not to allow arbitrary
subdirectories to diverge from the rest of the codebase / package but if
there's consensus this is a good idea, I can look into adding it.
Regards,
Michael
On 12/1/2021 11:43 AM, Michael D Kinney wrote:
> Hi Abner,
>
> Yes. That is possible. We are already excluding BaseTools.
>
> In order to get the file lists that apply to uncrustify, we would have to change from:
>
> git ls-files *.c *.h :!BaseTools/*
>
> To:
>
> git ls-files *.c *.h :!BaseTools/* :!RedfishPkg/PrivateLibrary/RedfishLib/edk2libredfish/*
>
> However, I do not see a feature in the UncrustifyCheck to check all c/h files in RedFishPkg and exclude one of its directories.
> If we can specify an exclusion like this in the RedFishPkg YAML file, then I think that would work well.
>
> Michael Kubacki would have to comment on how we support this in EDK II CI checks.
>
> Best regards,
>
> Mike
>
>
>> -----Original Message-----
>> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Abner Chang
>> Sent: Tuesday, November 30, 2021 11:34 PM
>> To: devel@edk2.groups.io; Wang, Jian J <jian.j.wang@intel.com>; Kinney, Michael D <michael.d.kinney@intel.com>; Michael
>> Kubacki <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com) <afish@apple.com>; Leif Lindholm
>> <leif@nuviainc.com>
>> Subject: Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
>>
>> Hi Mike,
>> Is that possible to not applying uncrusitify on the source files that cloned from other open source?
>> e.g. RedfishPkg/PrivateLibrary/RedfishLib/edk2libredfish/*.*
>>
>> Thanks
>> Abner
>>
>>> -----Original Message-----
>>> From: devel@edk2.groups.io [mailto:devel@edk2.groups.io] On Behalf Of
>>> Wang, Jian J
>>> Sent: Wednesday, December 1, 2021 2:39 PM
>>> To: devel@edk2.groups.io; Kinney, Michael D <michael.d.kinney@intel.com>;
>>> Michael Kubacki <michael.kubacki@microsoft.com>; Andrew Fish
>>> (afish@apple.com) <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>
>>> Subject: Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended
>>> Hard Freeze Update #4
>>>
>>> For SignedCapulePkg,
>>>
>>> Reviewed-by: Jian J Wang <jian.j.wang@intel.com>
>>>
>>> Regards,
>>> Jian
>>>
>>>> -----Original Message-----
>>>> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of
>>> Michael D
>>>> Kinney
>>>> Sent: Wednesday, December 01, 2021 6:34 AM
>>>> To: devel@edk2.groups.io; Kinney, Michael D
>>> <michael.d.kinney@intel.com>;
>>>> Michael Kubacki <michael.kubacki@microsoft.com>; Andrew Fish
>>>> (afish@apple.com) <afish@apple.com>; Leif Lindholm
>>> <leif@nuviainc.com>
>>>> Subject: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended
>>> Hard
>>>> Freeze Update #4
>>>>
>>>> Hello,
>>>>
>>>> Thank you for your patience during this extended hard freeze.
>>>>
>>>> Just one more step to go. There has been a delay in the review of
>>>> the patch series with the uncrustify source changes. PR(6). This
>>>> patch series was not sent out as patch review email because of its
>>>> very large size. It only contains source style changes and the
>>>> CompareBuild tool and GitHub action has shown there are no binary
>>>> differences introduced with these source style changes.
>>>>
>>>> If you are a package maintainer, then please review the following
>>>> branch/PR for your package contents and review the EDK II CI results
>>>> and BuildCompare results. I do not expect a line by line review
>>>> because we already had time to provide feedback on the source style
>>>> performed by uncrustify. Instead, a Reviewed-by for your package
>>>> indicates that you have reviewed the EDK II CI results and CompareBuild
>>>> tool functionality and results and you accept the source style
>>>> changes to your package.
>>>>
>>>> *
>>>>
>>> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyC
>>> han
>>>> ges_V5
>>>> * https://github.com/tianocore/edk2/pull/2229
>>>> * https://github.com/mdkinney/edk2/actions/runs/1521618836
>>>>
>>>> Additional details on this update below.
>>>>
>>>> Thank you,
>>>>
>>>> Mike
>>>>
>>>>
>>>> Changes from Update #3
>>>> ----------------------------------------------------------------------------
>>>> * Pushed PR (5)
>>>> * Added link to PR(6). EDK II CI Status is PASS. Build Compare PASS.
>>>> * Waiting for review of PR (6)
>>>> * Review of PR (7) completed and waiting for review of PR (6)
>>>> ----------------------------------------------------------------------------
>>>>
>>>> Changes from Update #2
>>>> ----------------------------------------------------------------------------
>>>> * Changed order of PRs swapping (4) and (5). The PR that activates
>>>> increases the max CI agent job time is independent of all the other
>>>> PRs and its review is complete, so it can be committed now.
>>>> * Pushed PRs (1), (2), (3), (4).
>>>> * Waiting for review to complete for PRs (5) and (6)
>>>> * Reviews complete for PR (7)
>>>> * Identifies steps using git filter-branch to apply uncrustify changes to a
>>>> code review patch series that was generated before the uncrustify
>>> changes
>>>> avoiding manual merge.
>>>> * Identified steps using git filter-repo to generate an alternate history of
>>>> the edk2 repo with uncrustify changes applied on every commit. This may
>>>> be useful when evaluating changes to files using tools like git blame
>>>> without the large uncrustify patch series.
>>>> ---------------------------------------------------------------------------
>>>>
>>>> Changes from Update #1
>>>> ----------------------------------------------------------------------------
>>>> * Changed order of PRs swapping (6) and (7). The PR that activates
>>>> EDK II CI check UncrustifyCheck has to be last because it unconditionally
>>>> checks all C/H files in all packages. Not just files that have been
>>>> modified like some of the other checkers.
>>>> * Updated link to the branch with the UncrustifyCheck plugin that has been
>>>> updated with a one line change and Reviewed-by and Tested-by tags.
>>>>
>>>>
>>> https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plug
>>> in_v
>>>> 6
>>>> * Reviews complete for (1), (2), (3), (5), and (7)
>>>> ---------------------------------------------------------------------------
>>>>
>>>> Michael Kubacki and I have prepared the patches required to apply the
>>>> uncrustify changes and enable EDK II CI to check all submitted
>>>> patches have been run through uncrustify.
>>>>
>>>> We have verified through the CompareBuild GitHub Action that the
>>>> format changes performed by uncrustify have no functional changes.
>>>> All of the OBJ, LIB, DLL, EFI, FFS, FV, and FD files match 100%
>>>> across 70 VS2019/GCC5 builds of all package/platform DSC files in
>>>> the edk2 repo.
>>>>
>>>> The hard freeze will be extended after the edk2-stable202111 tag until
>>>> all uncrustify related changes are committed. We do not expect this
>>>> to take more than a few days. Do not push any PRs until the hard
>>>> freeze is lifted.
>>>>
>>>> The changes are broken up into 7 patch series/PRs. The PRs are ordered
>>>> so they can be submitted using the normal submission process and EDK II
>>>> CI will pass for each one. Details are listed below.
>>>>
>>>> Uncrustify 73.0.3 for EDK II
>>>> =============================
>>>> * Sources:
>>> INVALID URI REMOVED
>>> stify__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKkoslR8D0LLPZTWXL0OKtMZJOYp
>>> cSVHoHwMD2t8$
>>>> * Documentation:
>>>>
>>> INVALID URI REMOVED
>>> /_wiki/wikis/Uncrustify.wiki/1/Proj__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKk
>>> oslR8D0LLPZTWXL0OKtMZJOYpcSVHoP8OtlxM$
>>>> ect-Mu-(EDK-II)-Fork-Readme
>>>> * Download:
>>>>
>>> INVALID URI REMOVED
>>> /_packaging?_a=package&feed=m__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKk
>>> oslR8D0LLPZTWXL0OKtMZJOYpcSVHo4yTWeAM$
>>>> u_uncrustify&package=mu-uncrustify-
>>>> release&protocolType=NuGet&version=73.0.3
>>>>
>>>> Installing Uncrustify
>>>> ======================
>>>> The Uncrustify tool is installed automatically when the Pytools
>>>> environment is used and the stuart* commands are run to complete the
>>>> environment setup. Please see:
>>>>
>>>> https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-
>>> locally
>>>>
>>>> Uncrustify can also be installed from the download page listed above
>>>> or built from sources from the source link above.
>>>>
>>>> The Documentation link provides instruction on how to run uncrustify from
>>>> the command line or install as a Visual Studio Code plugin. The main
>>>> uncrustify documentation also describes how to integrate with a few other
>>>> editors.
>>>>
>>>> We have also discussed a client side githook. That effort has not started.
>>>> Let us know if that is a feature you would find useful.
>>>>
>>>> Developer impact for new code reviews
>>>> ======================================
>>>> Once the uncrustify checker is active in EDK II CI, developers must
>>>> make sure their patches are run through the uncrustify tool before
>>>> sending the patches for review.
>>>>
>>>> Developers must install and run uncrustify against changes files before
>>>> sending patch review emails or submitting PR for EDK II CI. If EDK II CI
>>>> detects and differences in source formatting, then EDK II CI will fail
>>>> and the developer must run uncrustify and resubmit the patches.
>>>>
>>>> Developer impact to patch series/PRs reviewed during edk2-stable201121
>>>> soft/hard freeze
>>>>
>>> ==========================================================
>>> =======
>>>> ======================
>>>> Developers must rebase their changes after the uncrustify source changes
>>> are
>>>> committed. The branch with a preview of the uncrustify changes can be
>>> used
>>>> to start this rebase work.
>>>>
>>>>
>>>>
>>> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyC
>>> han
>>>> ges_V5
>>>>
>>>> The following steps can be used to update an existing branch with the
>>>> required uncrustify format. This is the Windows version. I will add
>>>> the Linux version soon.
>>>>
>>>> 1) Fetch and checkout and rebase to latest edk2/master
>>>>
>>>> git fetch origin
>>>> git checkout master
>>>> git rebase origin/master
>>>>
>>>> 2) Make a backup copy of plugin UncrustifyCheck outside WORKSPACE.
>>>> (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable and
>>>> EDK II specific uncrustify configuration file available when working
>>>> with a branch that does not have those tools in its scope.
>>>>
>>>> xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
>>>>
>>>> 3) Check out the patch series branch (e.g. MyBranch)
>>>>
>>>> git checkout MyBranch
>>>>
>>>> 4) Rebase patch series against edk2-stable202111
>>>>
>>>> git rebase edk2-stable202111
>>>>
>>>> 5) Create new branch for the uncrustifed version (e.g.
>>> MyBranch_Uncrustified)
>>>>
>>>> git checkout -b MyBranch_Uncrustified
>>>>
>>>> 6) Use git filter-branch to uncrustify all the commits in the series
>>>> between the rebase target from (2) and HEAD of the branch. A filter
>>>> can be used to scope the uncrustify operations to only the C/H files
>>>> in the specific package the patch series is against. (e.g.
>>> DynamicTablesPkg).
>>>> BaseTools should always be excluded. If the package scoped filter is
>>>> not used, it will still work, but will take longer to run because
>>>> uncrustify will rescan every C/H files in the whole repo.
>>>>
>>>> git filter-branch --tree-filter "git ls-files DynamicTablesPkg*.c
>>>> DynamicTablesPkg*.h :!BaseTools/* | c:\\Temp\\UncrustifyCheck\\mu-
>>>> uncrustify-release_extdep\\Windows-x86\\uncrustify.exe -c
>>>> c:\\Temp\\UncrustifyCheck\\uncrustify.cfg -F - --replace --no-backup --if-
>>>> changed" edk2-stable202111..HEAD
>>>>
>>>> 7) Now that all the individual patches in the branch are uncrustified,
>>>> rebase against latest edk2/master that is already uncrustified.
>>>>
>>>> git rebase master
>>>>
>>>> 8) Verify the patches in this new branch.
>>>>
>>>> Impacts to tracing history across the uncrusity changes
>>>> =======================================================
>>>> Tools the view file and line history do work with the large uncrustify
>>>> patch series. One impact is that the operations can be very slow due
>>>> to the large uncrustify patches.
>>>>
>>>> One option to provide a faster experience is to provide an alternate
>>>> version of the edk2 repository as "documentation" that has the
>>>> entire history re-written with uncrustify run on every commit.
>>>> The tool called git-filter-repo can be used to perform this
>>>> transformation and runs in a reasonable period of time (a few hours)
>>>>
>>>> https://github.com/newren/git-filter-repo
>>>> https://github.com/newren/git-filter-repo/blob/main/contrib/filter-
>>> repo-
>>>> demos/lint-history
>>>>
>>>> The following steps can be used to perform this transformation.
>>>> This is the Windows version. I will add the Linux version soon.
>>>>
>>>> ** WARNING ** This operation modifies(rewrites) all the commits
>>>> in the local copy of the repo. Do not perform
>>>> these steps on a local repo you are using for
>>>> active development.
>>>>
>>>> 1) Clone edk2 into a new directory (see **WARNING**)
>>>>
>>>> git clone https://github.com/tianocore/edk2.git edk2-uncrustified
>>>> cd edk2-uncrustified
>>>>
>>>> 2) Setup python virtual env, install pytools, and run stuart commands
>>>> to setup build environment which includes installing uncrustify tools.
>>>>
>>>> https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-
>>>> locally
>>>>
>>>> 3) Make a backup copy of plugin UncrustifyCheck outside WORKSPACE.
>>>> (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable and
>>>> EDK II specific uncrustify configuration file available when working
>>>> with a branch that does not have those tools in its scope.
>>>>
>>>> xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
>>>>
>>>> 4) Use lint-history.py from git-filter-repo examples
>>>>
>>>> https://github.com/newren/git-filter-repo
>>>> https://github.com/newren/git-filter-repo/blob/main/contrib/filter-
>>> repo-
>>>> demos/lint-history
>>>>
>>>> Line #127 - Add try except around subprocess.check_call() with except
>>>> being pass. This is required because there are a few commits of C
>>>> files in the edk2 repo that have incorrect C syntax and do not
>>>> build with a C compiler and break the uncrustify parser. Skip reformat
>>>> of C files that can not be parsed by uncrustify. These rare instances
>>>> are addressed in the commit that fixes the C syntax error.
>>>>
>>>> Run this slightly modified version of lint-history. Include only
>>>> C/H files and exclude directories that start with 'Tools' or 'BaseTools'.
>>>> This step took about 2.2 hours on a laptop.
>>>>
>>>> lint-history.py
>>>> --relevant "return (not filename.startswith(b'Tools') and not
>>>> filename.startswith(b'BaseTools') and (filename.endswith(b'.c') or
>>>> filename.endswith(b'.h')))"
>>>> c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\mu-
>>> uncrustify-
>>>> release_extdep\\Windows-x86\\uncrustify.exe -c
>>>> c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\uncrustify.cfg --
>>> replace --
>>>> no-backup --if-changed
>>>>
>>>> Order of PRs to apply during extended hard freeze
>>>> ==================================================
>>>> 1) Update EmulatorPkg Win Host [BuildOptions] MSFT CC_FLAGS to not
>>> force
>>>> debug information
>>>> *
>>> INVALID URI REMOVED
>>> d=3747__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKkoslR8D0LLPZTWXL0OKtMZJ
>>> OYpcSVHo2BEG33c$
>>>> *
>>>>
>>> https://github.com/mdkinney/edk2/tree/Bug_3747_EmulatorPkg_WinHost_
>>> Rep
>>>> roducibleBuild
>>>> * https://github.com/tianocore/edk2/pull/2215
>>>> * Required for EmulatorPkg to pass CompareBuild for VS2019 IA32/X64
>>> builds.
>>>> * Status: Review complete. PR pushed.
>>>>
>>>> 2) EccCheck should not revert staged and local changes
>>>> *
>>> INVALID URI REMOVED
>>> d=2986__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKkoslR8D0LLPZTWXL0OKtMZJ
>>> OYpcSVHotvR0Unk$
>>>> *
>>>>
>>> https://github.com/mdkinney/edk2/tree/Bug_2986_EccCheckRemoveGitRe
>>> vert
>>>> _V2
>>>> * https://github.com/tianocore/edk2/pull/2216
>>>> * Required for EDK II CI to complete in a reasonable period of time when
>>>> processing the 4000+ source file style changes made by uncrustify.
>>>> * Also fixes critical bugs that can potentially corrupt git state when
>>>> EccCheck is run locally.
>>>> * Status: Review complete. PR pushed.
>>>>
>>>> 3) Update pytool LicenseCheck plugin to use temp directory for diff output
>>> file
>>>> *
>>> INVALID URI REMOVED
>>> d=3746__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKkoslR8D0LLPZTWXL0OKtMZJ
>>> OYpcSVHoWRuHaBw$
>>>> *
>>>>
>>> https://github.com/mdkinney/edk2/tree/Bug_3746_LicenseCheckUseDiffO
>>> utput
>>>> File_V2
>>>> * https://github.com/tianocore/edk2/pull/2217
>>>> * Required to reduce EDK II CI build times.
>>>> * Status: Review complete. PR pushed.
>>>>
>>>> 4) Update max job time from 60 min to 120 minutes
>>> in .azurepipelines/templates
>>>> *
>>> INVALID URI REMOVED
>>> d=3750__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKkoslR8D0LLPZTWXL0OKtMZJ
>>> OYpcSVHozaVshgc$
>>>> *
>>>>
>>> https://github.com/mdkinney/edk2/tree/Bug_3750_IncreaseAzurePipelines
>>> Time
>>>> out
>>>> * https://github.com/tianocore/edk2/pull/2219
>>>> * Required to allow EccCheck of uncrustify changes to complete on Azure
>>>> Pipelines CI agents without timing out.
>>>> * Status: Review complete. PR pushed.
>>>>
>>>> 5) Update Package YAML to ignore specific ECC files/errors
>>>> *
>>> INVALID URI REMOVED
>>> d=3749__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKkoslR8D0LLPZTWXL0OKtMZJ
>>> OYpcSVHo-fY3dMo$
>>>> *
>>>>
>>> https://github.com/mdkinney/edk2/tree/Bug_3749_EccCheckIgnoreFilesErr
>>> ors
>>>> * https://github.com/tianocore/edk2/pull/2218
>>>> * Required to pass EccCheck
>>>> * Status: Review complete. PR pushed
>>>>
>>>> 6) Uncrustify Source Changes
>>>> *
>>> INVALID URI REMOVED
>>> d=3737__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKkoslR8D0LLPZTWXL0OKtMZJ
>>> OYpcSVHoMRa47YU$
>>>> *
>>> INVALID URI REMOVED
>>> d=3739__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKkoslR8D0LLPZTWXL0OKtMZJ
>>> OYpcSVHoPAlckP8$
>>>> *
>>>>
>>> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyC
>>> han
>>>> ges_V5
>>>> * https://github.com/tianocore/edk2/pull/2229
>>>> * Build comparison result PASS:
>>>> https://github.com/mdkinney/edk2/actions/runs/1521618836
>>>> * EFI_D_ -> DEBUG changes required to pass PatchCheck
>>>> * Uncrustify format changes required to pass UncrustifyCheck
>>>> * Status:
>>>> Waiting for review
>>>>
>>>> 7) UncrustifyCheck EDK II CI Plugin
>>>> *
>>> INVALID URI REMOVED
>>> d=3748__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKkoslR8D0LLPZTWXL0OKtMZJ
>>> OYpcSVHoCzEWymQ$
>>>> *
>>>>
>>> https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plug
>>> in_v
>>>> 6
>>>> * Required to enforce all PRs submitted to EDK II CI match uncrustify
>>> format.
>>>> * Unconditionally checks all packages. Can not be committed until all C/H
>>>> source files have been updated.
>>>> * Status: Review complete
>>>>
>>>> Combined Branch/PR for Review/Test
>>>> ==================================
>>>> * Build Comparison results must pass 100% across the full set of PRs before
>>>> the individual PRs can be pushed in the order listed above.
>>>> * Branch:
>>>> https://github.com/mdkinney/edk2/tree/TestOnly_Uncrustify_PR_Series
>>>> * PR: https://github.com/tianocore/edk2/pull/2229
>>>> Status = PASS
>>>> * CompareBuild:
>>>> Branch:
>>>>
>>> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyC
>>> han
>>>> ges_V5
>>>> --ref1: ef9a059cdb15844fe52a49af2bf7d86b9dd3e9bf
>>>> --ref2: Bug_3737_3739_ApplyUncrustifyChanges_V5
>>>> Extra Options: -n 4 --quiet
>>>> Results: https://github.com/mdkinney/edk2/actions/runs/1521618836
>>>> 30 VS2019 build comparisons PASS
>>>> 40 GCC5 build comparisons PASS
>>>> 100% PASS
>>>>
>>>> The following git log shows the set of patches from --ref1 to --ref 2across
>>>> which there are no differences in any of the OBJ/LIB/DLL/EFI/FFS/FV/FD
>>> files.
>>>>
>>>> --ref2
>>>> b7d4bf0675b7 (HEAD -> Bug_3737_3739_ApplyUncrustifyChanges_V5)
>>>> UnitTestFrameworkPkg: Apply uncrusitify changes
>>>> 7f03d25f60e7 UefiPayloadPkg: Apply uncrusitify changes
>>>> 0bfd8d9b5ac9 UefiCpuPkg: Apply uncrusitify changes
>>>> e1cd9bfb9dea StandaloneMmPkg: Apply uncrusitify changes
>>>> 5da2f65be378 SourceLevelDebugPkg: Apply uncrusitify changes
>>>> 95b86de07e5d SignedCapsulePkg: Apply uncrusitify changes
>>>> fe71d97246c4 ShellPkg: Apply uncrusitify changes
>>>> 54c21c952992 SecurityPkg: Apply uncrusitify changes
>>>> 187a3785f12b RedfishPkg: Apply uncrusitify changes
>>>> 810100002a46 PcAtChipsetPkg: Apply uncrusitify changes
>>>> 276a695c0cf2 OvmfPkg: Apply uncrusitify changes
>>>> 303c0a91ab07 NetworkPkg: Apply uncrusitify changes
>>>> bc80792cd1b1 MdePkg: Apply uncrusitify changes
>>>> 3ea86be17a2a MdeModulePkg: Apply uncrusitify changes
>>>> c70ef11ed0cd IntelFsp2WrapperPkg: Apply uncrusitify changes
>>>> c0291221f252 IntelFsp2Pkg: Apply uncrusitify changes
>>>> 6a479952a690 FmpDevicePkg: Apply uncrusitify changes
>>>> 3a7c05b7070d FatPkg: Apply uncrusitify changes
>>>> b789f98c8959 EmulatorPkg: Apply uncrusitify changes
>>>> 952d7a1c9220 EmbeddedPkg: Apply uncrusitify changes
>>>> a1cc9881bab6 DynamicTablesPkg: Apply uncrusitify changes
>>>> 50654dfe5785 CryptoPkg: Apply uncrusitify changes
>>>> ed965a02dfa1 ArmVirtPkg: Apply uncrusitify changes
>>>> 9744023fbc46 ArmPlatformPkg: Apply uncrusitify changes
>>>> 7a1cde5f5bba ArmPkg: Apply uncrusitify changes
>>>> 19d17e0913e8 UefiCpuPkg: Change use of EFI_D_* to DEBUG_*
>>>> ffa718b4f994 SourceLevelDebugPkg: Change use of EFI_D_* to DEBUG_*
>>>> b86cb3c5e5b4 ShellPkg: Change use of EFI_D_* to DEBUG_*
>>>> c7c42204dc07 SecurityPkg: Change use of EFI_D_* to DEBUG_*
>>>> 16b8e6f958e4 PcAtChipsetPkg: Change use of EFI_D_* to DEBUG_*
>>>> 0ac3f8b2dac5 OvmfPkg: Change use of EFI_D_* to DEBUG_*
>>>> bc5004b8d294 NetworkPkg: Change use of EFI_D_* to DEBUG_*
>>>> 6f671a8e2377 MdePkg: Change use of EFI_D_* to DEBUG_*
>>>> a10c610ff9a3 MdeModulePkg: Change use of EFI_D_* to DEBUG_*
>>>> 09a3bddba390 FatPkg: Change use of EFI_D_* to DEBUG_*
>>>> 59c61318246a EmulatorPkg: Change use of EFI_D_* to DEBUG_*
>>>> 3a80367dda3b EmbeddedPkg: Change use of EFI_D_* to DEBUG_*
>>>> 23eb1aaf80ca ArmVirtPkg: Change use of EFI_D_* to DEBUG_*
>>>> 875914b45c54 ArmPlatformPkg: Change use of EFI_D_* to DEBUG_*
>>>> eb2eca82b451 ArmPkg: Change use of EFI_D_* to DEBUG_*
>>>> f0f3f5aae7c4 (origin/master, origin/HEAD, master) UnitTestFrameworkPkg:
>>>> Update YAML to ignore specific ECC files/errors
>>>> c05734797790 UefiPayloadPkg: Update YAML to ignore specific ECC
>>> files/errors
>>>> c30c40d6c63d StandaloneMmPkg: Update YAML to ignore specific ECC
>>>> files/errors
>>>> 9944508e85f1 ShellPkg: Update YAML to ignore specific ECC files/errors
>>>> 60fa40be458d SecurityPkg: Update YAML to ignore specific ECC files/errors
>>>> df790cd6b37e MdePkg: Update YAML to ignore specific ECC files/errors
>>>> 9deb9370766e MdeModulePkg: Update YAML to ignore specific ECC
>>> files/errors
>>>> d7d30e8f219f EmulatorPkg: Update YAML to ignore specific ECC files/errors
>>>> d5744ecba813 CryptoPkg: Update YAML to ignore specific ECC files/errors
>>>> c97fee87f0f9 ArmVirtPkg: Update YAML to ignore specific ECC files/errors
>>>> 1939fc9569f2 ArmPlatformPkg: Update YAML to ignore specific ECC
>>> files/errors
>>>> 365dced2c37a ArmPkg: Update YAML to ignore specific ECC files/errors
>>>> 76a1ce4d5fec .azurepipelines/templates: Update max pipeline job time to
>>> 2
>>>> hours
>>>> 99f84ff47390 .pytools/Plugin/LicenseCheck: Use temp directory for git diff
>>>> output
>>>> 3019f1bbabf1 .pytool/Plugin/EccCheck: Add performance optimizations
>>>> 854462bd3479 .pytool/Plugin/EccCheck: Remove temp directory on
>>> exception
>>>> 69877614fdee .pytool/Plugin/EccCheck: Remove RevertCode()
>>>> --ref1
>>>> ef9a059cdb15 EmulatorPkg/Win/Host: Update CC_FLAGS
>>>> bb1bba3d7767 (tag: edk2-stable202111) NetworkPkg: Fix invalid pointer for
>>> DNS
>>>> response token on error
>>>>
>>>> Best regards,
>>>>
>>>> Mike
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>>
>>>
>>
>>
>>
>>
>>
>
>
>
>
>
>
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
2021-12-01 17:05 ` Michael Kubacki
@ 2021-12-01 17:39 ` Michael D Kinney
2021-12-02 5:48 ` Abner Chang
2021-12-02 11:00 ` Gerd Hoffmann
1 sibling, 1 reply; 39+ messages in thread
From: Michael D Kinney @ 2021-12-01 17:39 UTC (permalink / raw)
To: Michael Kubacki, devel@edk2.groups.io, Chang, Abner, Wang, Jian J,
Michael Kubacki, Andrew Fish (afish@apple.com), Leif Lindholm,
Kinney, Michael D
Hi Michael,
This commit provides the background on why this content was copied into edk2
instead of used as a submodule.
https://github.com/tianocore/edk2/commit/4751a48aeb2ab828b0a5cbdc585fd3642967cda1#diff-9f77cfa3b537eee28489f8713c17d8702330f5cef1795f154201dd463aabed27
I imagine the main reason for this request is to minimize the effort required
to sync these copies with new versions. The scope right now is only 7 files.
One alternative to help with the sync is for Abner to run uncrustify on the
files from the upstream source on these 7 files and then do the diff to see if
there is anything significant that needs to be merged.
Abner, are you ok with doing this 2 step process to sync the files that were copied?
Best regards,
Mike
> -----Original Message-----
> From: Michael Kubacki <mikuback@linux.microsoft.com>
> Sent: Wednesday, December 1, 2021 9:05 AM
> To: devel@edk2.groups.io; Kinney, Michael D <michael.d.kinney@intel.com>; Chang, Abner <abner.chang@hpe.com>; Wang, Jian J
> <jian.j.wang@intel.com>; Michael Kubacki <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com) <afish@apple.com>;
> Leif Lindholm <leif@nuviainc.com>
> Subject: Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
>
> This can of course be done at a technical level but it smells of a code
> cohesion problem.
>
> The code is in the edk2 project so it would reason that it should be
> uniform in style with other code and at least close to the EDK II C
> Coding Standard Specification.
>
> Currently, git submodules and ignored path are already excluded in
> addition to the ability to mark packages in "audit mode" so they do not
> fail the results. I would personally prefer not to allow arbitrary
> subdirectories to diverge from the rest of the codebase / package but if
> there's consensus this is a good idea, I can look into adding it.
>
> Regards,
> Michael
>
> On 12/1/2021 11:43 AM, Michael D Kinney wrote:
> > Hi Abner,
> >
> > Yes. That is possible. We are already excluding BaseTools.
> >
> > In order to get the file lists that apply to uncrustify, we would have to change from:
> >
> > git ls-files *.c *.h :!BaseTools/*
> >
> > To:
> >
> > git ls-files *.c *.h :!BaseTools/* :!RedfishPkg/PrivateLibrary/RedfishLib/edk2libredfish/*
> >
> > However, I do not see a feature in the UncrustifyCheck to check all c/h files in RedFishPkg and exclude one of its
> directories.
> > If we can specify an exclusion like this in the RedFishPkg YAML file, then I think that would work well.
> >
> > Michael Kubacki would have to comment on how we support this in EDK II CI checks.
> >
> > Best regards,
> >
> > Mike
> >
> >
> >> -----Original Message-----
> >> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Abner Chang
> >> Sent: Tuesday, November 30, 2021 11:34 PM
> >> To: devel@edk2.groups.io; Wang, Jian J <jian.j.wang@intel.com>; Kinney, Michael D <michael.d.kinney@intel.com>; Michael
> >> Kubacki <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com) <afish@apple.com>; Leif Lindholm
> >> <leif@nuviainc.com>
> >> Subject: Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
> >>
> >> Hi Mike,
> >> Is that possible to not applying uncrusitify on the source files that cloned from other open source?
> >> e.g. RedfishPkg/PrivateLibrary/RedfishLib/edk2libredfish/*.*
> >>
> >> Thanks
> >> Abner
> >>
> >>> -----Original Message-----
> >>> From: devel@edk2.groups.io [mailto:devel@edk2.groups.io] On Behalf Of
> >>> Wang, Jian J
> >>> Sent: Wednesday, December 1, 2021 2:39 PM
> >>> To: devel@edk2.groups.io; Kinney, Michael D <michael.d.kinney@intel.com>;
> >>> Michael Kubacki <michael.kubacki@microsoft.com>; Andrew Fish
> >>> (afish@apple.com) <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>
> >>> Subject: Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended
> >>> Hard Freeze Update #4
> >>>
> >>> For SignedCapulePkg,
> >>>
> >>> Reviewed-by: Jian J Wang <jian.j.wang@intel.com>
> >>>
> >>> Regards,
> >>> Jian
> >>>
> >>>> -----Original Message-----
> >>>> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of
> >>> Michael D
> >>>> Kinney
> >>>> Sent: Wednesday, December 01, 2021 6:34 AM
> >>>> To: devel@edk2.groups.io; Kinney, Michael D
> >>> <michael.d.kinney@intel.com>;
> >>>> Michael Kubacki <michael.kubacki@microsoft.com>; Andrew Fish
> >>>> (afish@apple.com) <afish@apple.com>; Leif Lindholm
> >>> <leif@nuviainc.com>
> >>>> Subject: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended
> >>> Hard
> >>>> Freeze Update #4
> >>>>
> >>>> Hello,
> >>>>
> >>>> Thank you for your patience during this extended hard freeze.
> >>>>
> >>>> Just one more step to go. There has been a delay in the review of
> >>>> the patch series with the uncrustify source changes. PR(6). This
> >>>> patch series was not sent out as patch review email because of its
> >>>> very large size. It only contains source style changes and the
> >>>> CompareBuild tool and GitHub action has shown there are no binary
> >>>> differences introduced with these source style changes.
> >>>>
> >>>> If you are a package maintainer, then please review the following
> >>>> branch/PR for your package contents and review the EDK II CI results
> >>>> and BuildCompare results. I do not expect a line by line review
> >>>> because we already had time to provide feedback on the source style
> >>>> performed by uncrustify. Instead, a Reviewed-by for your package
> >>>> indicates that you have reviewed the EDK II CI results and CompareBuild
> >>>> tool functionality and results and you accept the source style
> >>>> changes to your package.
> >>>>
> >>>> *
> >>>>
> >>> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyC
> >>> han
> >>>> ges_V5
> >>>> * https://github.com/tianocore/edk2/pull/2229
> >>>> * https://github.com/mdkinney/edk2/actions/runs/1521618836
> >>>>
> >>>> Additional details on this update below.
> >>>>
> >>>> Thank you,
> >>>>
> >>>> Mike
> >>>>
> >>>>
> >>>> Changes from Update #3
> >>>> ----------------------------------------------------------------------------
> >>>> * Pushed PR (5)
> >>>> * Added link to PR(6). EDK II CI Status is PASS. Build Compare PASS.
> >>>> * Waiting for review of PR (6)
> >>>> * Review of PR (7) completed and waiting for review of PR (6)
> >>>> ----------------------------------------------------------------------------
> >>>>
> >>>> Changes from Update #2
> >>>> ----------------------------------------------------------------------------
> >>>> * Changed order of PRs swapping (4) and (5). The PR that activates
> >>>> increases the max CI agent job time is independent of all the other
> >>>> PRs and its review is complete, so it can be committed now.
> >>>> * Pushed PRs (1), (2), (3), (4).
> >>>> * Waiting for review to complete for PRs (5) and (6)
> >>>> * Reviews complete for PR (7)
> >>>> * Identifies steps using git filter-branch to apply uncrustify changes to a
> >>>> code review patch series that was generated before the uncrustify
> >>> changes
> >>>> avoiding manual merge.
> >>>> * Identified steps using git filter-repo to generate an alternate history of
> >>>> the edk2 repo with uncrustify changes applied on every commit. This may
> >>>> be useful when evaluating changes to files using tools like git blame
> >>>> without the large uncrustify patch series.
> >>>> ---------------------------------------------------------------------------
> >>>>
> >>>> Changes from Update #1
> >>>> ----------------------------------------------------------------------------
> >>>> * Changed order of PRs swapping (6) and (7). The PR that activates
> >>>> EDK II CI check UncrustifyCheck has to be last because it unconditionally
> >>>> checks all C/H files in all packages. Not just files that have been
> >>>> modified like some of the other checkers.
> >>>> * Updated link to the branch with the UncrustifyCheck plugin that has been
> >>>> updated with a one line change and Reviewed-by and Tested-by tags.
> >>>>
> >>>>
> >>> https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plug
> >>> in_v
> >>>> 6
> >>>> * Reviews complete for (1), (2), (3), (5), and (7)
> >>>> ---------------------------------------------------------------------------
> >>>>
> >>>> Michael Kubacki and I have prepared the patches required to apply the
> >>>> uncrustify changes and enable EDK II CI to check all submitted
> >>>> patches have been run through uncrustify.
> >>>>
> >>>> We have verified through the CompareBuild GitHub Action that the
> >>>> format changes performed by uncrustify have no functional changes.
> >>>> All of the OBJ, LIB, DLL, EFI, FFS, FV, and FD files match 100%
> >>>> across 70 VS2019/GCC5 builds of all package/platform DSC files in
> >>>> the edk2 repo.
> >>>>
> >>>> The hard freeze will be extended after the edk2-stable202111 tag until
> >>>> all uncrustify related changes are committed. We do not expect this
> >>>> to take more than a few days. Do not push any PRs until the hard
> >>>> freeze is lifted.
> >>>>
> >>>> The changes are broken up into 7 patch series/PRs. The PRs are ordered
> >>>> so they can be submitted using the normal submission process and EDK II
> >>>> CI will pass for each one. Details are listed below.
> >>>>
> >>>> Uncrustify 73.0.3 for EDK II
> >>>> =============================
> >>>> * Sources:
> >>> INVALID URI REMOVED
> >>> stify__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKkoslR8D0LLPZTWXL0OKtMZJOYp
> >>> cSVHoHwMD2t8$
> >>>> * Documentation:
> >>>>
> >>> INVALID URI REMOVED
> >>> /_wiki/wikis/Uncrustify.wiki/1/Proj__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKk
> >>> oslR8D0LLPZTWXL0OKtMZJOYpcSVHoP8OtlxM$
> >>>> ect-Mu-(EDK-II)-Fork-Readme
> >>>> * Download:
> >>>>
> >>> INVALID URI REMOVED
> >>> /_packaging?_a=package&feed=m__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKk
> >>> oslR8D0LLPZTWXL0OKtMZJOYpcSVHo4yTWeAM$
> >>>> u_uncrustify&package=mu-uncrustify-
> >>>> release&protocolType=NuGet&version=73.0.3
> >>>>
> >>>> Installing Uncrustify
> >>>> ======================
> >>>> The Uncrustify tool is installed automatically when the Pytools
> >>>> environment is used and the stuart* commands are run to complete the
> >>>> environment setup. Please see:
> >>>>
> >>>> https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-
> >>> locally
> >>>>
> >>>> Uncrustify can also be installed from the download page listed above
> >>>> or built from sources from the source link above.
> >>>>
> >>>> The Documentation link provides instruction on how to run uncrustify from
> >>>> the command line or install as a Visual Studio Code plugin. The main
> >>>> uncrustify documentation also describes how to integrate with a few other
> >>>> editors.
> >>>>
> >>>> We have also discussed a client side githook. That effort has not started.
> >>>> Let us know if that is a feature you would find useful.
> >>>>
> >>>> Developer impact for new code reviews
> >>>> ======================================
> >>>> Once the uncrustify checker is active in EDK II CI, developers must
> >>>> make sure their patches are run through the uncrustify tool before
> >>>> sending the patches for review.
> >>>>
> >>>> Developers must install and run uncrustify against changes files before
> >>>> sending patch review emails or submitting PR for EDK II CI. If EDK II CI
> >>>> detects and differences in source formatting, then EDK II CI will fail
> >>>> and the developer must run uncrustify and resubmit the patches.
> >>>>
> >>>> Developer impact to patch series/PRs reviewed during edk2-stable201121
> >>>> soft/hard freeze
> >>>>
> >>> ==========================================================
> >>> =======
> >>>> ======================
> >>>> Developers must rebase their changes after the uncrustify source changes
> >>> are
> >>>> committed. The branch with a preview of the uncrustify changes can be
> >>> used
> >>>> to start this rebase work.
> >>>>
> >>>>
> >>>>
> >>> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyC
> >>> han
> >>>> ges_V5
> >>>>
> >>>> The following steps can be used to update an existing branch with the
> >>>> required uncrustify format. This is the Windows version. I will add
> >>>> the Linux version soon.
> >>>>
> >>>> 1) Fetch and checkout and rebase to latest edk2/master
> >>>>
> >>>> git fetch origin
> >>>> git checkout master
> >>>> git rebase origin/master
> >>>>
> >>>> 2) Make a backup copy of plugin UncrustifyCheck outside WORKSPACE.
> >>>> (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable and
> >>>> EDK II specific uncrustify configuration file available when working
> >>>> with a branch that does not have those tools in its scope.
> >>>>
> >>>> xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
> >>>>
> >>>> 3) Check out the patch series branch (e.g. MyBranch)
> >>>>
> >>>> git checkout MyBranch
> >>>>
> >>>> 4) Rebase patch series against edk2-stable202111
> >>>>
> >>>> git rebase edk2-stable202111
> >>>>
> >>>> 5) Create new branch for the uncrustifed version (e.g.
> >>> MyBranch_Uncrustified)
> >>>>
> >>>> git checkout -b MyBranch_Uncrustified
> >>>>
> >>>> 6) Use git filter-branch to uncrustify all the commits in the series
> >>>> between the rebase target from (2) and HEAD of the branch. A filter
> >>>> can be used to scope the uncrustify operations to only the C/H files
> >>>> in the specific package the patch series is against. (e.g.
> >>> DynamicTablesPkg).
> >>>> BaseTools should always be excluded. If the package scoped filter is
> >>>> not used, it will still work, but will take longer to run because
> >>>> uncrustify will rescan every C/H files in the whole repo.
> >>>>
> >>>> git filter-branch --tree-filter "git ls-files DynamicTablesPkg*.c
> >>>> DynamicTablesPkg*.h :!BaseTools/* | c:\\Temp\\UncrustifyCheck\\mu-
> >>>> uncrustify-release_extdep\\Windows-x86\\uncrustify.exe -c
> >>>> c:\\Temp\\UncrustifyCheck\\uncrustify.cfg -F - --replace --no-backup --if-
> >>>> changed" edk2-stable202111..HEAD
> >>>>
> >>>> 7) Now that all the individual patches in the branch are uncrustified,
> >>>> rebase against latest edk2/master that is already uncrustified.
> >>>>
> >>>> git rebase master
> >>>>
> >>>> 8) Verify the patches in this new branch.
> >>>>
> >>>> Impacts to tracing history across the uncrusity changes
> >>>> =======================================================
> >>>> Tools the view file and line history do work with the large uncrustify
> >>>> patch series. One impact is that the operations can be very slow due
> >>>> to the large uncrustify patches.
> >>>>
> >>>> One option to provide a faster experience is to provide an alternate
> >>>> version of the edk2 repository as "documentation" that has the
> >>>> entire history re-written with uncrustify run on every commit.
> >>>> The tool called git-filter-repo can be used to perform this
> >>>> transformation and runs in a reasonable period of time (a few hours)
> >>>>
> >>>> https://github.com/newren/git-filter-repo
> >>>> https://github.com/newren/git-filter-repo/blob/main/contrib/filter-
> >>> repo-
> >>>> demos/lint-history
> >>>>
> >>>> The following steps can be used to perform this transformation.
> >>>> This is the Windows version. I will add the Linux version soon.
> >>>>
> >>>> ** WARNING ** This operation modifies(rewrites) all the commits
> >>>> in the local copy of the repo. Do not perform
> >>>> these steps on a local repo you are using for
> >>>> active development.
> >>>>
> >>>> 1) Clone edk2 into a new directory (see **WARNING**)
> >>>>
> >>>> git clone https://github.com/tianocore/edk2.git edk2-uncrustified
> >>>> cd edk2-uncrustified
> >>>>
> >>>> 2) Setup python virtual env, install pytools, and run stuart commands
> >>>> to setup build environment which includes installing uncrustify tools.
> >>>>
> >>>> https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-
> >>>> locally
> >>>>
> >>>> 3) Make a backup copy of plugin UncrustifyCheck outside WORKSPACE.
> >>>> (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable and
> >>>> EDK II specific uncrustify configuration file available when working
> >>>> with a branch that does not have those tools in its scope.
> >>>>
> >>>> xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
> >>>>
> >>>> 4) Use lint-history.py from git-filter-repo examples
> >>>>
> >>>> https://github.com/newren/git-filter-repo
> >>>> https://github.com/newren/git-filter-repo/blob/main/contrib/filter-
> >>> repo-
> >>>> demos/lint-history
> >>>>
> >>>> Line #127 - Add try except around subprocess.check_call() with except
> >>>> being pass. This is required because there are a few commits of C
> >>>> files in the edk2 repo that have incorrect C syntax and do not
> >>>> build with a C compiler and break the uncrustify parser. Skip reformat
> >>>> of C files that can not be parsed by uncrustify. These rare instances
> >>>> are addressed in the commit that fixes the C syntax error.
> >>>>
> >>>> Run this slightly modified version of lint-history. Include only
> >>>> C/H files and exclude directories that start with 'Tools' or 'BaseTools'.
> >>>> This step took about 2.2 hours on a laptop.
> >>>>
> >>>> lint-history.py
> >>>> --relevant "return (not filename.startswith(b'Tools') and not
> >>>> filename.startswith(b'BaseTools') and (filename.endswith(b'.c') or
> >>>> filename.endswith(b'.h')))"
> >>>> c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\mu-
> >>> uncrustify-
> >>>> release_extdep\\Windows-x86\\uncrustify.exe -c
> >>>> c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\uncrustify.cfg --
> >>> replace --
> >>>> no-backup --if-changed
> >>>>
> >>>> Order of PRs to apply during extended hard freeze
> >>>> ==================================================
> >>>> 1) Update EmulatorPkg Win Host [BuildOptions] MSFT CC_FLAGS to not
> >>> force
> >>>> debug information
> >>>> *
> >>> INVALID URI REMOVED
> >>> d=3747__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKkoslR8D0LLPZTWXL0OKtMZJ
> >>> OYpcSVHo2BEG33c$
> >>>> *
> >>>>
> >>> https://github.com/mdkinney/edk2/tree/Bug_3747_EmulatorPkg_WinHost_
> >>> Rep
> >>>> roducibleBuild
> >>>> * https://github.com/tianocore/edk2/pull/2215
> >>>> * Required for EmulatorPkg to pass CompareBuild for VS2019 IA32/X64
> >>> builds.
> >>>> * Status: Review complete. PR pushed.
> >>>>
> >>>> 2) EccCheck should not revert staged and local changes
> >>>> *
> >>> INVALID URI REMOVED
> >>> d=2986__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKkoslR8D0LLPZTWXL0OKtMZJ
> >>> OYpcSVHotvR0Unk$
> >>>> *
> >>>>
> >>> https://github.com/mdkinney/edk2/tree/Bug_2986_EccCheckRemoveGitRe
> >>> vert
> >>>> _V2
> >>>> * https://github.com/tianocore/edk2/pull/2216
> >>>> * Required for EDK II CI to complete in a reasonable period of time when
> >>>> processing the 4000+ source file style changes made by uncrustify.
> >>>> * Also fixes critical bugs that can potentially corrupt git state when
> >>>> EccCheck is run locally.
> >>>> * Status: Review complete. PR pushed.
> >>>>
> >>>> 3) Update pytool LicenseCheck plugin to use temp directory for diff output
> >>> file
> >>>> *
> >>> INVALID URI REMOVED
> >>> d=3746__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKkoslR8D0LLPZTWXL0OKtMZJ
> >>> OYpcSVHoWRuHaBw$
> >>>> *
> >>>>
> >>> https://github.com/mdkinney/edk2/tree/Bug_3746_LicenseCheckUseDiffO
> >>> utput
> >>>> File_V2
> >>>> * https://github.com/tianocore/edk2/pull/2217
> >>>> * Required to reduce EDK II CI build times.
> >>>> * Status: Review complete. PR pushed.
> >>>>
> >>>> 4) Update max job time from 60 min to 120 minutes
> >>> in .azurepipelines/templates
> >>>> *
> >>> INVALID URI REMOVED
> >>> d=3750__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKkoslR8D0LLPZTWXL0OKtMZJ
> >>> OYpcSVHozaVshgc$
> >>>> *
> >>>>
> >>> https://github.com/mdkinney/edk2/tree/Bug_3750_IncreaseAzurePipelines
> >>> Time
> >>>> out
> >>>> * https://github.com/tianocore/edk2/pull/2219
> >>>> * Required to allow EccCheck of uncrustify changes to complete on Azure
> >>>> Pipelines CI agents without timing out.
> >>>> * Status: Review complete. PR pushed.
> >>>>
> >>>> 5) Update Package YAML to ignore specific ECC files/errors
> >>>> *
> >>> INVALID URI REMOVED
> >>> d=3749__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKkoslR8D0LLPZTWXL0OKtMZJ
> >>> OYpcSVHo-fY3dMo$
> >>>> *
> >>>>
> >>> https://github.com/mdkinney/edk2/tree/Bug_3749_EccCheckIgnoreFilesErr
> >>> ors
> >>>> * https://github.com/tianocore/edk2/pull/2218
> >>>> * Required to pass EccCheck
> >>>> * Status: Review complete. PR pushed
> >>>>
> >>>> 6) Uncrustify Source Changes
> >>>> *
> >>> INVALID URI REMOVED
> >>> d=3737__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKkoslR8D0LLPZTWXL0OKtMZJ
> >>> OYpcSVHoMRa47YU$
> >>>> *
> >>> INVALID URI REMOVED
> >>> d=3739__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKkoslR8D0LLPZTWXL0OKtMZJ
> >>> OYpcSVHoPAlckP8$
> >>>> *
> >>>>
> >>> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyC
> >>> han
> >>>> ges_V5
> >>>> * https://github.com/tianocore/edk2/pull/2229
> >>>> * Build comparison result PASS:
> >>>> https://github.com/mdkinney/edk2/actions/runs/1521618836
> >>>> * EFI_D_ -> DEBUG changes required to pass PatchCheck
> >>>> * Uncrustify format changes required to pass UncrustifyCheck
> >>>> * Status:
> >>>> Waiting for review
> >>>>
> >>>> 7) UncrustifyCheck EDK II CI Plugin
> >>>> *
> >>> INVALID URI REMOVED
> >>> d=3748__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKkoslR8D0LLPZTWXL0OKtMZJ
> >>> OYpcSVHoCzEWymQ$
> >>>> *
> >>>>
> >>> https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plug
> >>> in_v
> >>>> 6
> >>>> * Required to enforce all PRs submitted to EDK II CI match uncrustify
> >>> format.
> >>>> * Unconditionally checks all packages. Can not be committed until all C/H
> >>>> source files have been updated.
> >>>> * Status: Review complete
> >>>>
> >>>> Combined Branch/PR for Review/Test
> >>>> ==================================
> >>>> * Build Comparison results must pass 100% across the full set of PRs before
> >>>> the individual PRs can be pushed in the order listed above.
> >>>> * Branch:
> >>>> https://github.com/mdkinney/edk2/tree/TestOnly_Uncrustify_PR_Series
> >>>> * PR: https://github.com/tianocore/edk2/pull/2229
> >>>> Status = PASS
> >>>> * CompareBuild:
> >>>> Branch:
> >>>>
> >>> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyC
> >>> han
> >>>> ges_V5
> >>>> --ref1: ef9a059cdb15844fe52a49af2bf7d86b9dd3e9bf
> >>>> --ref2: Bug_3737_3739_ApplyUncrustifyChanges_V5
> >>>> Extra Options: -n 4 --quiet
> >>>> Results: https://github.com/mdkinney/edk2/actions/runs/1521618836
> >>>> 30 VS2019 build comparisons PASS
> >>>> 40 GCC5 build comparisons PASS
> >>>> 100% PASS
> >>>>
> >>>> The following git log shows the set of patches from --ref1 to --ref 2across
> >>>> which there are no differences in any of the OBJ/LIB/DLL/EFI/FFS/FV/FD
> >>> files.
> >>>>
> >>>> --ref2
> >>>> b7d4bf0675b7 (HEAD -> Bug_3737_3739_ApplyUncrustifyChanges_V5)
> >>>> UnitTestFrameworkPkg: Apply uncrusitify changes
> >>>> 7f03d25f60e7 UefiPayloadPkg: Apply uncrusitify changes
> >>>> 0bfd8d9b5ac9 UefiCpuPkg: Apply uncrusitify changes
> >>>> e1cd9bfb9dea StandaloneMmPkg: Apply uncrusitify changes
> >>>> 5da2f65be378 SourceLevelDebugPkg: Apply uncrusitify changes
> >>>> 95b86de07e5d SignedCapsulePkg: Apply uncrusitify changes
> >>>> fe71d97246c4 ShellPkg: Apply uncrusitify changes
> >>>> 54c21c952992 SecurityPkg: Apply uncrusitify changes
> >>>> 187a3785f12b RedfishPkg: Apply uncrusitify changes
> >>>> 810100002a46 PcAtChipsetPkg: Apply uncrusitify changes
> >>>> 276a695c0cf2 OvmfPkg: Apply uncrusitify changes
> >>>> 303c0a91ab07 NetworkPkg: Apply uncrusitify changes
> >>>> bc80792cd1b1 MdePkg: Apply uncrusitify changes
> >>>> 3ea86be17a2a MdeModulePkg: Apply uncrusitify changes
> >>>> c70ef11ed0cd IntelFsp2WrapperPkg: Apply uncrusitify changes
> >>>> c0291221f252 IntelFsp2Pkg: Apply uncrusitify changes
> >>>> 6a479952a690 FmpDevicePkg: Apply uncrusitify changes
> >>>> 3a7c05b7070d FatPkg: Apply uncrusitify changes
> >>>> b789f98c8959 EmulatorPkg: Apply uncrusitify changes
> >>>> 952d7a1c9220 EmbeddedPkg: Apply uncrusitify changes
> >>>> a1cc9881bab6 DynamicTablesPkg: Apply uncrusitify changes
> >>>> 50654dfe5785 CryptoPkg: Apply uncrusitify changes
> >>>> ed965a02dfa1 ArmVirtPkg: Apply uncrusitify changes
> >>>> 9744023fbc46 ArmPlatformPkg: Apply uncrusitify changes
> >>>> 7a1cde5f5bba ArmPkg: Apply uncrusitify changes
> >>>> 19d17e0913e8 UefiCpuPkg: Change use of EFI_D_* to DEBUG_*
> >>>> ffa718b4f994 SourceLevelDebugPkg: Change use of EFI_D_* to DEBUG_*
> >>>> b86cb3c5e5b4 ShellPkg: Change use of EFI_D_* to DEBUG_*
> >>>> c7c42204dc07 SecurityPkg: Change use of EFI_D_* to DEBUG_*
> >>>> 16b8e6f958e4 PcAtChipsetPkg: Change use of EFI_D_* to DEBUG_*
> >>>> 0ac3f8b2dac5 OvmfPkg: Change use of EFI_D_* to DEBUG_*
> >>>> bc5004b8d294 NetworkPkg: Change use of EFI_D_* to DEBUG_*
> >>>> 6f671a8e2377 MdePkg: Change use of EFI_D_* to DEBUG_*
> >>>> a10c610ff9a3 MdeModulePkg: Change use of EFI_D_* to DEBUG_*
> >>>> 09a3bddba390 FatPkg: Change use of EFI_D_* to DEBUG_*
> >>>> 59c61318246a EmulatorPkg: Change use of EFI_D_* to DEBUG_*
> >>>> 3a80367dda3b EmbeddedPkg: Change use of EFI_D_* to DEBUG_*
> >>>> 23eb1aaf80ca ArmVirtPkg: Change use of EFI_D_* to DEBUG_*
> >>>> 875914b45c54 ArmPlatformPkg: Change use of EFI_D_* to DEBUG_*
> >>>> eb2eca82b451 ArmPkg: Change use of EFI_D_* to DEBUG_*
> >>>> f0f3f5aae7c4 (origin/master, origin/HEAD, master) UnitTestFrameworkPkg:
> >>>> Update YAML to ignore specific ECC files/errors
> >>>> c05734797790 UefiPayloadPkg: Update YAML to ignore specific ECC
> >>> files/errors
> >>>> c30c40d6c63d StandaloneMmPkg: Update YAML to ignore specific ECC
> >>>> files/errors
> >>>> 9944508e85f1 ShellPkg: Update YAML to ignore specific ECC files/errors
> >>>> 60fa40be458d SecurityPkg: Update YAML to ignore specific ECC files/errors
> >>>> df790cd6b37e MdePkg: Update YAML to ignore specific ECC files/errors
> >>>> 9deb9370766e MdeModulePkg: Update YAML to ignore specific ECC
> >>> files/errors
> >>>> d7d30e8f219f EmulatorPkg: Update YAML to ignore specific ECC files/errors
> >>>> d5744ecba813 CryptoPkg: Update YAML to ignore specific ECC files/errors
> >>>> c97fee87f0f9 ArmVirtPkg: Update YAML to ignore specific ECC files/errors
> >>>> 1939fc9569f2 ArmPlatformPkg: Update YAML to ignore specific ECC
> >>> files/errors
> >>>> 365dced2c37a ArmPkg: Update YAML to ignore specific ECC files/errors
> >>>> 76a1ce4d5fec .azurepipelines/templates: Update max pipeline job time to
> >>> 2
> >>>> hours
> >>>> 99f84ff47390 .pytools/Plugin/LicenseCheck: Use temp directory for git diff
> >>>> output
> >>>> 3019f1bbabf1 .pytool/Plugin/EccCheck: Add performance optimizations
> >>>> 854462bd3479 .pytool/Plugin/EccCheck: Remove temp directory on
> >>> exception
> >>>> 69877614fdee .pytool/Plugin/EccCheck: Remove RevertCode()
> >>>> --ref1
> >>>> ef9a059cdb15 EmulatorPkg/Win/Host: Update CC_FLAGS
> >>>> bb1bba3d7767 (tag: edk2-stable202111) NetworkPkg: Fix invalid pointer for
> >>> DNS
> >>>> response token on error
> >>>>
> >>>> Best regards,
> >>>>
> >>>> Mike
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>
> >>
> >>
> >>
> >>
> >
> >
> >
> >
> >
> >
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
2021-11-30 22:34 Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4 Michael D Kinney
` (6 preceding siblings ...)
2021-12-01 12:59 ` Sami Mujawar
@ 2021-12-01 18:52 ` Bret Barkelew
2021-12-02 18:27 ` [edk2-devel] " Maciej Rabeda
2021-12-03 8:56 ` Gerd Hoffmann
9 siblings, 0 replies; 39+ messages in thread
From: Bret Barkelew @ 2021-12-01 18:52 UTC (permalink / raw)
To: devel@edk2.groups.io, Kinney, Michael D, Michael Kubacki,
Andrew Fish (afish@apple.com), Lindholm, Leif
[-- Attachment #1: Type: text/plain, Size: 36103 bytes --]
So, so, so refreshing to see automated consistency.
For UnitTestFrameworkPkg…
Reviewed-by: Bret Barkelew <bret.barkelew@microsoft.com>
- Bret
From: Michael D Kinney via groups.io<mailto:michael.d.kinney=intel.com@groups.io>
Sent: Tuesday, November 30, 2021 2:34 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Kinney, Michael D<mailto:michael.d.kinney@intel.com>; Michael Kubacki<mailto:Michael.Kubacki@microsoft.com>; Andrew Fish (afish@apple.com)<mailto:afish@apple.com>; Lindholm, Leif<mailto:leif@nuviainc.com>
Subject: [EXTERNAL] [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
Hello,
Thank you for your patience during this extended hard freeze.
Just one more step to go. There has been a delay in the review of
the patch series with the uncrustify source changes. PR(6). This
patch series was not sent out as patch review email because of its
very large size. It only contains source style changes and the
CompareBuild tool and GitHub action has shown there are no binary
differences introduced with these source style changes.
If you are a package maintainer, then please review the following
branch/PR for your package contents and review the EDK II CI results
and BuildCompare results. I do not expect a line by line review
because we already had time to provide feedback on the source style
performed by uncrustify. Instead, a Reviewed-by for your package
indicates that you have reviewed the EDK II CI results and CompareBuild
tool functionality and results and you accept the source style
changes to your package.
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Ftree%2FBug_3737_3739_ApplyUncrustifyChanges_V5&data=04%7C01%7CBret.Barkelew%40microsoft.com%7Cafe73bb0029e491612b908d9b4519560%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637739084803059341%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=6OFgnyfsGhclcj8GTVKchzBmkbOQJtcm4o2iz4RzPU4%3D&reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fpull%2F2229&data=04%7C01%7CBret.Barkelew%40microsoft.com%7Cafe73bb0029e491612b908d9b4519560%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637739084803059341%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=BA8pmxk0T4Uom8%2F4dzyjJ2lvEl3mALyq3wOlVCKMPD0%3D&reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Factions%2Fruns%2F1521618836&data=04%7C01%7CBret.Barkelew%40microsoft.com%7Cafe73bb0029e491612b908d9b4519560%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637739084803059341%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=1EcBi3Mjb9LM%2BhrUDfufOJFmvEEVsb6IW7chJvs3dl0%3D&reserved=0
Additional details on this update below.
Thank you,
Mike
Changes from Update #3
----------------------------------------------------------------------------
* Pushed PR (5)
* Added link to PR(6). EDK II CI Status is PASS. Build Compare PASS.
* Waiting for review of PR (6)
* Review of PR (7) completed and waiting for review of PR (6)
----------------------------------------------------------------------------
Changes from Update #2
----------------------------------------------------------------------------
* Changed order of PRs swapping (4) and (5). The PR that activates
increases the max CI agent job time is independent of all the other
PRs and its review is complete, so it can be committed now.
* Pushed PRs (1), (2), (3), (4).
* Waiting for review to complete for PRs (5) and (6)
* Reviews complete for PR (7)
* Identifies steps using git filter-branch to apply uncrustify changes to a
code review patch series that was generated before the uncrustify changes
avoiding manual merge.
* Identified steps using git filter-repo to generate an alternate history of
the edk2 repo with uncrustify changes applied on every commit. This may
be useful when evaluating changes to files using tools like git blame
without the large uncrustify patch series.
---------------------------------------------------------------------------
Changes from Update #1
----------------------------------------------------------------------------
* Changed order of PRs swapping (6) and (7). The PR that activates
EDK II CI check UncrustifyCheck has to be last because it unconditionally
checks all C/H files in all packages. Not just files that have been
modified like some of the other checkers.
* Updated link to the branch with the UncrustifyCheck plugin that has been
updated with a one line change and Reviewed-by and Tested-by tags.
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Ftree%2FBug_3748_add_uncrustify_ci_plugin_v6&data=04%7C01%7CBret.Barkelew%40microsoft.com%7Cafe73bb0029e491612b908d9b4519560%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637739084803059341%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=j0Xjp7ss0vyQ2oNTRF4VC%2B5AwQrKQuxxGHH14q%2Fw1p8%3D&reserved=0
* Reviews complete for (1), (2), (3), (5), and (7)
---------------------------------------------------------------------------
Michael Kubacki and I have prepared the patches required to apply the
uncrustify changes and enable EDK II CI to check all submitted
patches have been run through uncrustify.
We have verified through the CompareBuild GitHub Action that the
format changes performed by uncrustify have no functional changes.
All of the OBJ, LIB, DLL, EFI, FFS, FV, and FD files match 100%
across 70 VS2019/GCC5 builds of all package/platform DSC files in
the edk2 repo.
The hard freeze will be extended after the edk2-stable202111 tag until
all uncrustify related changes are committed. We do not expect this
to take more than a few days. Do not push any PRs until the hard
freeze is lifted.
The changes are broken up into 7 patch series/PRs. The PRs are ordered
so they can be submitted using the normal submission process and EDK II
CI will pass for each one. Details are listed below.
Uncrustify 73.0.3 for EDK II
=============================
* Sources: https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdev.azure.com%2Fprojectmu%2F_git%2FUncrustify&data=04%7C01%7CBret.Barkelew%40microsoft.com%7Cafe73bb0029e491612b908d9b4519560%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637739084803059341%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=oxs8hpNZPKyuizJMa1Q938NvF07fpc5XN5PzQKvO5B4%3D&reserved=0
* Documentation: https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdev.azure.com%2Fprojectmu%2FUncrustify%2F_wiki%2Fwikis%2FUncrustify.wiki%2F1%2FProject-Mu-&data=04%7C01%7CBret.Barkelew%40microsoft.com%7Cafe73bb0029e491612b908d9b4519560%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637739084803059341%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=FPPTmTUhhXL%2FGybBIdaMcTumBnfibpjNnorESzob8jA%3D&reserved=0(EDK-II)-Fork-Readme
* Download: https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdev.azure.com%2Fprojectmu%2FUncrustify%2F_packaging%3F_a%3Dpackage%26feed%3Dmu_uncrustify%26package%3Dmu-uncrustify-release%26protocolType%3DNuGet%26version%3D73.0.3&data=04%7C01%7CBret.Barkelew%40microsoft.com%7Cafe73bb0029e491612b908d9b4519560%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637739084803059341%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=BUOogU1P194ZKUDhSBoO7QcxPIDjZyM%2Bqvg2kwNaSKw%3D&reserved=0
Installing Uncrustify
======================
The Uncrustify tool is installed automatically when the Pytools
environment is used and the stuart* commands are run to complete the
environment setup. Please see:
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2F.pytool%23running-ci-locally&data=04%7C01%7CBret.Barkelew%40microsoft.com%7Cafe73bb0029e491612b908d9b4519560%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637739084803059341%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Gl8zfFjVrUvpnTO0UXb3bXR2lsIBz5PwLazkKyBvGIE%3D&reserved=0
Uncrustify can also be installed from the download page listed above
or built from sources from the source link above.
The Documentation link provides instruction on how to run uncrustify from
the command line or install as a Visual Studio Code plugin. The main
uncrustify documentation also describes how to integrate with a few other
editors.
We have also discussed a client side githook. That effort has not started.
Let us know if that is a feature you would find useful.
Developer impact for new code reviews
======================================
Once the uncrustify checker is active in EDK II CI, developers must
make sure their patches are run through the uncrustify tool before
sending the patches for review.
Developers must install and run uncrustify against changes files before
sending patch review emails or submitting PR for EDK II CI. If EDK II CI
detects and differences in source formatting, then EDK II CI will fail
and the developer must run uncrustify and resubmit the patches.
Developer impact to patch series/PRs reviewed during edk2-stable201121 soft/hard freeze
=======================================================================================
Developers must rebase their changes after the uncrustify source changes are
committed. The branch with a preview of the uncrustify changes can be used
to start this rebase work.
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Ftree%2FBug_3737_3739_ApplyUncrustifyChanges_V5&data=04%7C01%7CBret.Barkelew%40microsoft.com%7Cafe73bb0029e491612b908d9b4519560%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637739084803059341%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=6OFgnyfsGhclcj8GTVKchzBmkbOQJtcm4o2iz4RzPU4%3D&reserved=0
The following steps can be used to update an existing branch with the
required uncrustify format. This is the Windows version. I will add
the Linux version soon.
1) Fetch and checkout and rebase to latest edk2/master
git fetch origin
git checkout master
git rebase origin/master
2) Make a backup copy of plugin UncrustifyCheck outside WORKSPACE.
(e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable and
EDK II specific uncrustify configuration file available when working
with a branch that does not have those tools in its scope.
xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
3) Check out the patch series branch (e.g. MyBranch)
git checkout MyBranch
4) Rebase patch series against edk2-stable202111
git rebase edk2-stable202111
5) Create new branch for the uncrustifed version (e.g. MyBranch_Uncrustified)
git checkout -b MyBranch_Uncrustified
6) Use git filter-branch to uncrustify all the commits in the series
between the rebase target from (2) and HEAD of the branch. A filter
can be used to scope the uncrustify operations to only the C/H files
in the specific package the patch series is against. (e.g. DynamicTablesPkg).
BaseTools should always be excluded. If the package scoped filter is
not used, it will still work, but will take longer to run because
uncrustify will rescan every C/H files in the whole repo.
git filter-branch --tree-filter "git ls-files DynamicTablesPkg*.c DynamicTablesPkg*.h :!BaseTools/* | c:\\Temp\\UncrustifyCheck\\mu-uncrustify-release_extdep\\Windows-x86\\uncrustify.exe -c c:\\Temp\\UncrustifyCheck\\uncrustify.cfg -F - --replace --no-backup --if-changed" edk2-stable202111..HEAD
7) Now that all the individual patches in the branch are uncrustified,
rebase against latest edk2/master that is already uncrustified.
git rebase master
8) Verify the patches in this new branch.
Impacts to tracing history across the uncrusity changes
=======================================================
Tools the view file and line history do work with the large uncrustify
patch series. One impact is that the operations can be very slow due
to the large uncrustify patches.
One option to provide a faster experience is to provide an alternate
version of the edk2 repository as "documentation" that has the
entire history re-written with uncrustify run on every commit.
The tool called git-filter-repo can be used to perform this
transformation and runs in a reasonable period of time (a few hours)
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnewren%2Fgit-filter-repo&data=04%7C01%7CBret.Barkelew%40microsoft.com%7Cafe73bb0029e491612b908d9b4519560%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637739084803059341%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=ww5Wr752N7sQUbISyV8uXnv4EDRveeU6LXktsw7afIs%3D&reserved=0
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnewren%2Fgit-filter-repo%2Fblob%2Fmain%2Fcontrib%2Ffilter-repo-demos%2Flint-history&data=04%7C01%7CBret.Barkelew%40microsoft.com%7Cafe73bb0029e491612b908d9b4519560%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637739084803059341%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=VeoGuXB8K%2F1vp%2BwDdSRM6HvUhqtK4wkFfJLjdorspWs%3D&reserved=0
The following steps can be used to perform this transformation.
This is the Windows version. I will add the Linux version soon.
** WARNING ** This operation modifies(rewrites) all the commits
in the local copy of the repo. Do not perform
these steps on a local repo you are using for
active development.
1) Clone edk2 into a new directory (see **WARNING**)
git clone https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2.git&data=04%7C01%7CBret.Barkelew%40microsoft.com%7Cafe73bb0029e491612b908d9b4519560%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637739084803059341%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=djea29CcnZTVc2MohA2WzGaLiKMp66sD9qdum7wcaFI%3D&reserved=0 edk2-uncrustified
cd edk2-uncrustified
2) Setup python virtual env, install pytools, and run stuart commands
to setup build environment which includes installing uncrustify tools.
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2F.pytool%23running-ci-locally&data=04%7C01%7CBret.Barkelew%40microsoft.com%7Cafe73bb0029e491612b908d9b4519560%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637739084803059341%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Gl8zfFjVrUvpnTO0UXb3bXR2lsIBz5PwLazkKyBvGIE%3D&reserved=0
3) Make a backup copy of plugin UncrustifyCheck outside WORKSPACE.
(e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable and
EDK II specific uncrustify configuration file available when working
with a branch that does not have those tools in its scope.
xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
4) Use lint-history.py from git-filter-repo examples
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnewren%2Fgit-filter-repo&data=04%7C01%7CBret.Barkelew%40microsoft.com%7Cafe73bb0029e491612b908d9b4519560%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637739084803059341%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=ww5Wr752N7sQUbISyV8uXnv4EDRveeU6LXktsw7afIs%3D&reserved=0
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnewren%2Fgit-filter-repo%2Fblob%2Fmain%2Fcontrib%2Ffilter-repo-demos%2Flint-history&data=04%7C01%7CBret.Barkelew%40microsoft.com%7Cafe73bb0029e491612b908d9b4519560%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637739084803059341%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=VeoGuXB8K%2F1vp%2BwDdSRM6HvUhqtK4wkFfJLjdorspWs%3D&reserved=0
Line #127 - Add try except around subprocess.check_call() with except
being pass. This is required because there are a few commits of C
files in the edk2 repo that have incorrect C syntax and do not
build with a C compiler and break the uncrustify parser. Skip reformat
of C files that can not be parsed by uncrustify. These rare instances
are addressed in the commit that fixes the C syntax error.
Run this slightly modified version of lint-history. Include only
C/H files and exclude directories that start with 'Tools' or 'BaseTools'.
This step took about 2.2 hours on a laptop.
lint-history.py
--relevant "return (not filename.startswith(b'Tools') and not filename.startswith(b'BaseTools') and (filename.endswith(b'.c') or filename.endswith(b'.h')))"
c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\mu-uncrustify-release_extdep\\Windows-x86\\uncrustify.exe -c c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\uncrustify.cfg --replace --no-backup --if-changed
Order of PRs to apply during extended hard freeze
==================================================
1) Update EmulatorPkg Win Host [BuildOptions] MSFT CC_FLAGS to not force debug information
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.tianocore.org%2Fshow_bug.cgi%3Fid%3D3747&data=04%7C01%7CBret.Barkelew%40microsoft.com%7Cafe73bb0029e491612b908d9b4519560%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637739084803059341%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=vRu5LQSjbmwafbvaA6Iv0S%2FzUIbAE%2Bn2Ax9xbckTYLg%3D&reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Ftree%2FBug_3747_EmulatorPkg_WinHost_ReproducibleBuild&data=04%7C01%7CBret.Barkelew%40microsoft.com%7Cafe73bb0029e491612b908d9b4519560%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637739084803059341%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=oFCZQy6C9zf7kD4QugF9jORY%2Feh5%2B9T9odYGvMND%2FE0%3D&reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fpull%2F2215&data=04%7C01%7CBret.Barkelew%40microsoft.com%7Cafe73bb0029e491612b908d9b4519560%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637739084803059341%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=mfZMcsGnEof2DvpnVCz8NV9HTo%2FyqadoGfEquXwLjFI%3D&reserved=0
* Required for EmulatorPkg to pass CompareBuild for VS2019 IA32/X64 builds.
* Status: Review complete. PR pushed.
2) EccCheck should not revert staged and local changes
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.tianocore.org%2Fshow_bug.cgi%3Fid%3D2986&data=04%7C01%7CBret.Barkelew%40microsoft.com%7Cafe73bb0029e491612b908d9b4519560%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637739084803059341%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=SQVOuOT3Fg%2FVlH2471q0UAG9%2FGRgux3pvkRjrsM%2FkRU%3D&reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Ftree%2FBug_2986_EccCheckRemoveGitRevert_V2&data=04%7C01%7CBret.Barkelew%40microsoft.com%7Cafe73bb0029e491612b908d9b4519560%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637739084803059341%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=st1WwSlbkgoJ1nPpZqb9XnvCgLqPlpRAos5LtAM%2BcPA%3D&reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fpull%2F2216&data=04%7C01%7CBret.Barkelew%40microsoft.com%7Cafe73bb0029e491612b908d9b4519560%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637739084803059341%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=6ZQJvN7KLQl5pYsVVIosFbwIdTYtfBVU3rQgvq4cKxs%3D&reserved=0
* Required for EDK II CI to complete in a reasonable period of time when
processing the 4000+ source file style changes made by uncrustify.
* Also fixes critical bugs that can potentially corrupt git state when
EccCheck is run locally.
* Status: Review complete. PR pushed.
3) Update pytool LicenseCheck plugin to use temp directory for diff output file
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.tianocore.org%2Fshow_bug.cgi%3Fid%3D3746&data=04%7C01%7CBret.Barkelew%40microsoft.com%7Cafe73bb0029e491612b908d9b4519560%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637739084803059341%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=cukX7Hc09DRj1v%2BfC9epfr0DDeP0NKunwwbweXjK2hI%3D&reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Ftree%2FBug_3746_LicenseCheckUseDiffOutputFile_V2&data=04%7C01%7CBret.Barkelew%40microsoft.com%7Cafe73bb0029e491612b908d9b4519560%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637739084803059341%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=i7uICY1zgaq3ukuWCvWTq211b2qhHM6lYRfxTg4pJRE%3D&reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fpull%2F2217&data=04%7C01%7CBret.Barkelew%40microsoft.com%7Cafe73bb0029e491612b908d9b4519560%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637739084803059341%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=kNvYpT8TrH%2FxcykLRVAg6Z7JQ7RkBtqhFAWyR3x6%2Faw%3D&reserved=0
* Required to reduce EDK II CI build times.
* Status: Review complete. PR pushed.
4) Update max job time from 60 min to 120 minutes in .azurepipelines/templates
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.tianocore.org%2Fshow_bug.cgi%3Fid%3D3750&data=04%7C01%7CBret.Barkelew%40microsoft.com%7Cafe73bb0029e491612b908d9b4519560%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637739084803059341%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=qlXfBBwzPrz4iLyVunIEF8N5mondsfLjMmYk7hlF5Qg%3D&reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Ftree%2FBug_3750_IncreaseAzurePipelinesTimeout&data=04%7C01%7CBret.Barkelew%40microsoft.com%7Cafe73bb0029e491612b908d9b4519560%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637739084803059341%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=3QhsR365UOEUjyOsrPXvH3oXXSx1F3g6BmpQemEjIxA%3D&reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fpull%2F2219&data=04%7C01%7CBret.Barkelew%40microsoft.com%7Cafe73bb0029e491612b908d9b4519560%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637739084803059341%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=yNZhoLZXW7hYqPNtKqq4X6rNWyblAhFPpEbkalXotVU%3D&reserved=0
* Required to allow EccCheck of uncrustify changes to complete on Azure
Pipelines CI agents without timing out.
* Status: Review complete. PR pushed.
5) Update Package YAML to ignore specific ECC files/errors
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.tianocore.org%2Fshow_bug.cgi%3Fid%3D3749&data=04%7C01%7CBret.Barkelew%40microsoft.com%7Cafe73bb0029e491612b908d9b4519560%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637739084803059341%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=HE2PDTC15WKVaQMPU9HZDhfJ4cx71yYzSFnnwqfVk5U%3D&reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Ftree%2FBug_3749_EccCheckIgnoreFilesErrors&data=04%7C01%7CBret.Barkelew%40microsoft.com%7Cafe73bb0029e491612b908d9b4519560%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637739084803059341%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=d6PNQhqOzohY3OQcX5WVgxB5aWp4qylzFuIzRPsAbZk%3D&reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fpull%2F2218&data=04%7C01%7CBret.Barkelew%40microsoft.com%7Cafe73bb0029e491612b908d9b4519560%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637739084803059341%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Vvf2btUzSOzwILvps3h7cRHRfH%2B5cLV%2FcC0Zdo64QaY%3D&reserved=0
* Required to pass EccCheck
* Status: Review complete. PR pushed
6) Uncrustify Source Changes
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.tianocore.org%2Fshow_bug.cgi%3Fid%3D3737&data=04%7C01%7CBret.Barkelew%40microsoft.com%7Cafe73bb0029e491612b908d9b4519560%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637739084803059341%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=4iMSTuiCXmhNq9v1z65UYEQ4G9eS0TTkKT0gAC%2Fj%2Bjo%3D&reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.tianocore.org%2Fshow_bug.cgi%3Fid%3D3739&data=04%7C01%7CBret.Barkelew%40microsoft.com%7Cafe73bb0029e491612b908d9b4519560%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637739084803059341%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=vbr%2BU6nvHHVcfaMrmfH1tMPC3kNDJyv%2BvDhOoEgUyj4%3D&reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Ftree%2FBug_3737_3739_ApplyUncrustifyChanges_V5&data=04%7C01%7CBret.Barkelew%40microsoft.com%7Cafe73bb0029e491612b908d9b4519560%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637739084803059341%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=6OFgnyfsGhclcj8GTVKchzBmkbOQJtcm4o2iz4RzPU4%3D&reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fpull%2F2229&data=04%7C01%7CBret.Barkelew%40microsoft.com%7Cafe73bb0029e491612b908d9b4519560%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637739084803059341%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=BA8pmxk0T4Uom8%2F4dzyjJ2lvEl3mALyq3wOlVCKMPD0%3D&reserved=0
* Build comparison result PASS: https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Factions%2Fruns%2F1521618836&data=04%7C01%7CBret.Barkelew%40microsoft.com%7Cafe73bb0029e491612b908d9b4519560%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637739084803059341%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=1EcBi3Mjb9LM%2BhrUDfufOJFmvEEVsb6IW7chJvs3dl0%3D&reserved=0
* EFI_D_ -> DEBUG changes required to pass PatchCheck
* Uncrustify format changes required to pass UncrustifyCheck
* Status:
Waiting for review
7) UncrustifyCheck EDK II CI Plugin
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.tianocore.org%2Fshow_bug.cgi%3Fid%3D3748&data=04%7C01%7CBret.Barkelew%40microsoft.com%7Cafe73bb0029e491612b908d9b4519560%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637739084803059341%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=peNoGQGid33y42AXi3qugg4oRJycrj%2FoPZkFofszVVk%3D&reserved=0
* https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Ftree%2FBug_3748_add_uncrustify_ci_plugin_v6&data=04%7C01%7CBret.Barkelew%40microsoft.com%7Cafe73bb0029e491612b908d9b4519560%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637739084803059341%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=j0Xjp7ss0vyQ2oNTRF4VC%2B5AwQrKQuxxGHH14q%2Fw1p8%3D&reserved=0
* Required to enforce all PRs submitted to EDK II CI match uncrustify format.
* Unconditionally checks all packages. Can not be committed until all C/H
source files have been updated.
* Status: Review complete
Combined Branch/PR for Review/Test
==================================
* Build Comparison results must pass 100% across the full set of PRs before
the individual PRs can be pushed in the order listed above.
* Branch: https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Ftree%2FTestOnly_Uncrustify_PR_Series&data=04%7C01%7CBret.Barkelew%40microsoft.com%7Cafe73bb0029e491612b908d9b4519560%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637739084803059341%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=%2Fgacf3RVe6uGU%2BQ2gVdQYOfKLn2zwS1DXwuqRo8k%2Fco%3D&reserved=0
* PR: https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fpull%2F2229&data=04%7C01%7CBret.Barkelew%40microsoft.com%7Cafe73bb0029e491612b908d9b4519560%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637739084803059341%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=BA8pmxk0T4Uom8%2F4dzyjJ2lvEl3mALyq3wOlVCKMPD0%3D&reserved=0
Status = PASS
* CompareBuild:
Branch: https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Ftree%2FBug_3737_3739_ApplyUncrustifyChanges_V5&data=04%7C01%7CBret.Barkelew%40microsoft.com%7Cafe73bb0029e491612b908d9b4519560%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637739084803059341%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=6OFgnyfsGhclcj8GTVKchzBmkbOQJtcm4o2iz4RzPU4%3D&reserved=0
--ref1: ef9a059cdb15844fe52a49af2bf7d86b9dd3e9bf
--ref2: Bug_3737_3739_ApplyUncrustifyChanges_V5
Extra Options: -n 4 --quiet
Results: https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Factions%2Fruns%2F1521618836&data=04%7C01%7CBret.Barkelew%40microsoft.com%7Cafe73bb0029e491612b908d9b4519560%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637739084803059341%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=1EcBi3Mjb9LM%2BhrUDfufOJFmvEEVsb6IW7chJvs3dl0%3D&reserved=0
30 VS2019 build comparisons PASS
40 GCC5 build comparisons PASS
100% PASS
The following git log shows the set of patches from --ref1 to --ref 2across
which there are no differences in any of the OBJ/LIB/DLL/EFI/FFS/FV/FD files.
--ref2
b7d4bf0675b7 (HEAD -> Bug_3737_3739_ApplyUncrustifyChanges_V5) UnitTestFrameworkPkg: Apply uncrusitify changes
7f03d25f60e7 UefiPayloadPkg: Apply uncrusitify changes
0bfd8d9b5ac9 UefiCpuPkg: Apply uncrusitify changes
e1cd9bfb9dea StandaloneMmPkg: Apply uncrusitify changes
5da2f65be378 SourceLevelDebugPkg: Apply uncrusitify changes
95b86de07e5d SignedCapsulePkg: Apply uncrusitify changes
fe71d97246c4 ShellPkg: Apply uncrusitify changes
54c21c952992 SecurityPkg: Apply uncrusitify changes
187a3785f12b RedfishPkg: Apply uncrusitify changes
810100002a46 PcAtChipsetPkg: Apply uncrusitify changes
276a695c0cf2 OvmfPkg: Apply uncrusitify changes
303c0a91ab07 NetworkPkg: Apply uncrusitify changes
bc80792cd1b1 MdePkg: Apply uncrusitify changes
3ea86be17a2a MdeModulePkg: Apply uncrusitify changes
c70ef11ed0cd IntelFsp2WrapperPkg: Apply uncrusitify changes
c0291221f252 IntelFsp2Pkg: Apply uncrusitify changes
6a479952a690 FmpDevicePkg: Apply uncrusitify changes
3a7c05b7070d FatPkg: Apply uncrusitify changes
b789f98c8959 EmulatorPkg: Apply uncrusitify changes
952d7a1c9220 EmbeddedPkg: Apply uncrusitify changes
a1cc9881bab6 DynamicTablesPkg: Apply uncrusitify changes
50654dfe5785 CryptoPkg: Apply uncrusitify changes
ed965a02dfa1 ArmVirtPkg: Apply uncrusitify changes
9744023fbc46 ArmPlatformPkg: Apply uncrusitify changes
7a1cde5f5bba ArmPkg: Apply uncrusitify changes
19d17e0913e8 UefiCpuPkg: Change use of EFI_D_* to DEBUG_*
ffa718b4f994 SourceLevelDebugPkg: Change use of EFI_D_* to DEBUG_*
b86cb3c5e5b4 ShellPkg: Change use of EFI_D_* to DEBUG_*
c7c42204dc07 SecurityPkg: Change use of EFI_D_* to DEBUG_*
16b8e6f958e4 PcAtChipsetPkg: Change use of EFI_D_* to DEBUG_*
0ac3f8b2dac5 OvmfPkg: Change use of EFI_D_* to DEBUG_*
bc5004b8d294 NetworkPkg: Change use of EFI_D_* to DEBUG_*
6f671a8e2377 MdePkg: Change use of EFI_D_* to DEBUG_*
a10c610ff9a3 MdeModulePkg: Change use of EFI_D_* to DEBUG_*
09a3bddba390 FatPkg: Change use of EFI_D_* to DEBUG_*
59c61318246a EmulatorPkg: Change use of EFI_D_* to DEBUG_*
3a80367dda3b EmbeddedPkg: Change use of EFI_D_* to DEBUG_*
23eb1aaf80ca ArmVirtPkg: Change use of EFI_D_* to DEBUG_*
875914b45c54 ArmPlatformPkg: Change use of EFI_D_* to DEBUG_*
eb2eca82b451 ArmPkg: Change use of EFI_D_* to DEBUG_*
f0f3f5aae7c4 (origin/master, origin/HEAD, master) UnitTestFrameworkPkg: Update YAML to ignore specific ECC files/errors
c05734797790 UefiPayloadPkg: Update YAML to ignore specific ECC files/errors
c30c40d6c63d StandaloneMmPkg: Update YAML to ignore specific ECC files/errors
9944508e85f1 ShellPkg: Update YAML to ignore specific ECC files/errors
60fa40be458d SecurityPkg: Update YAML to ignore specific ECC files/errors
df790cd6b37e MdePkg: Update YAML to ignore specific ECC files/errors
9deb9370766e MdeModulePkg: Update YAML to ignore specific ECC files/errors
d7d30e8f219f EmulatorPkg: Update YAML to ignore specific ECC files/errors
d5744ecba813 CryptoPkg: Update YAML to ignore specific ECC files/errors
c97fee87f0f9 ArmVirtPkg: Update YAML to ignore specific ECC files/errors
1939fc9569f2 ArmPlatformPkg: Update YAML to ignore specific ECC files/errors
365dced2c37a ArmPkg: Update YAML to ignore specific ECC files/errors
76a1ce4d5fec .azurepipelines/templates: Update max pipeline job time to 2 hours
99f84ff47390 .pytools/Plugin/LicenseCheck: Use temp directory for git diff output
3019f1bbabf1 .pytool/Plugin/EccCheck: Add performance optimizations
854462bd3479 .pytool/Plugin/EccCheck: Remove temp directory on exception
69877614fdee .pytool/Plugin/EccCheck: Remove RevertCode()
--ref1
ef9a059cdb15 EmulatorPkg/Win/Host: Update CC_FLAGS
bb1bba3d7767 (tag: edk2-stable202111) NetworkPkg: Fix invalid pointer for DNS response token on error
Best regards,
Mike
[-- Attachment #2: Type: text/html, Size: 116501 bytes --]
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
2021-12-01 17:39 ` Michael D Kinney
@ 2021-12-02 5:48 ` Abner Chang
0 siblings, 0 replies; 39+ messages in thread
From: Abner Chang @ 2021-12-02 5:48 UTC (permalink / raw)
To: Kinney, Michael D, Michael Kubacki, devel@edk2.groups.io,
Wang, Jian J, Michael Kubacki, Andrew Fish (afish@apple.com),
Leif Lindholm
> -----Original Message-----
> From: Kinney, Michael D [mailto:michael.d.kinney@intel.com]
> Sent: Thursday, December 2, 2021 1:39 AM
> To: Michael Kubacki <mikuback@linux.microsoft.com>;
> devel@edk2.groups.io; Chang, Abner (HPS SW/FW Technologist)
> <abner.chang@hpe.com>; Wang, Jian J <jian.j.wang@intel.com>; Michael
> Kubacki <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com)
> <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>; Kinney, Michael D
> <michael.d.kinney@intel.com>
> Subject: RE: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended
> Hard Freeze Update #4
>
> Hi Michael,
>
> This commit provides the background on why this content was copied into
> edk2
> instead of used as a submodule.
>
> https://github.com/tianocore/edk2/commit/4751a48aeb2ab828b0a5cbdc585
> fd3642967cda1#diff-
> 9f77cfa3b537eee28489f8713c17d8702330f5cef1795f154201dd463aabed27
>
> I imagine the main reason for this request is to minimize the effort required
> to sync these copies with new versions.
Yes, this is what I worry about applying uncrustify on these files.
>The scope right now is only 7 files.
> One alternative to help with the sync is for Abner to run uncrustify on the
> files from the upstream source on these 7 files and then do the diff to see if
> there is anything significant that needs to be merged.
>
> Abner, are you ok with doing this 2 step process to sync the files that were
> copied?
I also consider to just uncrustify those files, maybe just compare two open source version and add the diff back to edk2 one. Then we can have this work moves forward.
Thanks
For the RedfishPkg/,
Acked-by: Abner Chang
>
> Best regards,
>
> Mike
>
>
> > -----Original Message-----
> > From: Michael Kubacki <mikuback@linux.microsoft.com>
> > Sent: Wednesday, December 1, 2021 9:05 AM
> > To: devel@edk2.groups.io; Kinney, Michael D
> <michael.d.kinney@intel.com>; Chang, Abner <abner.chang@hpe.com>;
> Wang, Jian J
> > <jian.j.wang@intel.com>; Michael Kubacki
> <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com)
> <afish@apple.com>;
> > Leif Lindholm <leif@nuviainc.com>
> > Subject: Re: [edk2-devel] Uncrustify Conversion Detailed Plan and
> Extended Hard Freeze Update #4
> >
> > This can of course be done at a technical level but it smells of a code
> > cohesion problem.
> >
> > The code is in the edk2 project so it would reason that it should be
> > uniform in style with other code and at least close to the EDK II C
> > Coding Standard Specification.
> >
> > Currently, git submodules and ignored path are already excluded in
> > addition to the ability to mark packages in "audit mode" so they do not
> > fail the results. I would personally prefer not to allow arbitrary
> > subdirectories to diverge from the rest of the codebase / package but if
> > there's consensus this is a good idea, I can look into adding it.
> >
> > Regards,
> > Michael
> >
> > On 12/1/2021 11:43 AM, Michael D Kinney wrote:
> > > Hi Abner,
> > >
> > > Yes. That is possible. We are already excluding BaseTools.
> > >
> > > In order to get the file lists that apply to uncrustify, we would have to
> change from:
> > >
> > > git ls-files *.c *.h :!BaseTools/*
> > >
> > > To:
> > >
> > > git ls-files *.c
> *.h :!BaseTools/* :!RedfishPkg/PrivateLibrary/RedfishLib/edk2libredfish/*
> > >
> > > However, I do not see a feature in the UncrustifyCheck to check all c/h
> files in RedFishPkg and exclude one of its
> > directories.
> > > If we can specify an exclusion like this in the RedFishPkg YAML file, then I
> think that would work well.
> > >
> > > Michael Kubacki would have to comment on how we support this in EDK
> II CI checks.
> > >
> > > Best regards,
> > >
> > > Mike
> > >
> > >
> > >> -----Original Message-----
> > >> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of
> Abner Chang
> > >> Sent: Tuesday, November 30, 2021 11:34 PM
> > >> To: devel@edk2.groups.io; Wang, Jian J <jian.j.wang@intel.com>;
> Kinney, Michael D <michael.d.kinney@intel.com>; Michael
> > >> Kubacki <michael.kubacki@microsoft.com>; Andrew Fish
> (afish@apple.com) <afish@apple.com>; Leif Lindholm
> > >> <leif@nuviainc.com>
> > >> Subject: Re: [edk2-devel] Uncrustify Conversion Detailed Plan and
> Extended Hard Freeze Update #4
> > >>
> > >> Hi Mike,
> > >> Is that possible to not applying uncrusitify on the source files that cloned
> from other open source?
> > >> e.g. RedfishPkg/PrivateLibrary/RedfishLib/edk2libredfish/*.*
> > >>
> > >> Thanks
> > >> Abner
> > >>
> > >>> -----Original Message-----
> > >>> From: devel@edk2.groups.io [mailto:devel@edk2.groups.io] On
> Behalf Of
> > >>> Wang, Jian J
> > >>> Sent: Wednesday, December 1, 2021 2:39 PM
> > >>> To: devel@edk2.groups.io; Kinney, Michael D
> <michael.d.kinney@intel.com>;
> > >>> Michael Kubacki <michael.kubacki@microsoft.com>; Andrew Fish
> > >>> (afish@apple.com) <afish@apple.com>; Leif Lindholm
> <leif@nuviainc.com>
> > >>> Subject: Re: [edk2-devel] Uncrustify Conversion Detailed Plan and
> Extended
> > >>> Hard Freeze Update #4
> > >>>
> > >>> For SignedCapulePkg,
> > >>>
> > >>> Reviewed-by: Jian J Wang <jian.j.wang@intel.com>
> > >>>
> > >>> Regards,
> > >>> Jian
> > >>>
> > >>>> -----Original Message-----
> > >>>> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of
> > >>> Michael D
> > >>>> Kinney
> > >>>> Sent: Wednesday, December 01, 2021 6:34 AM
> > >>>> To: devel@edk2.groups.io; Kinney, Michael D
> > >>> <michael.d.kinney@intel.com>;
> > >>>> Michael Kubacki <michael.kubacki@microsoft.com>; Andrew Fish
> > >>>> (afish@apple.com) <afish@apple.com>; Leif Lindholm
> > >>> <leif@nuviainc.com>
> > >>>> Subject: [edk2-devel] Uncrustify Conversion Detailed Plan and
> Extended
> > >>> Hard
> > >>>> Freeze Update #4
> > >>>>
> > >>>> Hello,
> > >>>>
> > >>>> Thank you for your patience during this extended hard freeze.
> > >>>>
> > >>>> Just one more step to go. There has been a delay in the review of
> > >>>> the patch series with the uncrustify source changes. PR(6). This
> > >>>> patch series was not sent out as patch review email because of its
> > >>>> very large size. It only contains source style changes and the
> > >>>> CompareBuild tool and GitHub action has shown there are no binary
> > >>>> differences introduced with these source style changes.
> > >>>>
> > >>>> If you are a package maintainer, then please review the following
> > >>>> branch/PR for your package contents and review the EDK II CI results
> > >>>> and BuildCompare results. I do not expect a line by line review
> > >>>> because we already had time to provide feedback on the source style
> > >>>> performed by uncrustify. Instead, a Reviewed-by for your package
> > >>>> indicates that you have reviewed the EDK II CI results and
> CompareBuild
> > >>>> tool functionality and results and you accept the source style
> > >>>> changes to your package.
> > >>>>
> > >>>> *
> > >>>>
> > >>>
> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyC
> > >>> han
> > >>>> ges_V5
> > >>>> * https://github.com/tianocore/edk2/pull/2229
> > >>>> * https://github.com/mdkinney/edk2/actions/runs/1521618836
> > >>>>
> > >>>> Additional details on this update below.
> > >>>>
> > >>>> Thank you,
> > >>>>
> > >>>> Mike
> > >>>>
> > >>>>
> > >>>> Changes from Update #3
> > >>>> ----------------------------------------------------------------------------
> > >>>> * Pushed PR (5)
> > >>>> * Added link to PR(6). EDK II CI Status is PASS. Build Compare PASS.
> > >>>> * Waiting for review of PR (6)
> > >>>> * Review of PR (7) completed and waiting for review of PR (6)
> > >>>> ----------------------------------------------------------------------------
> > >>>>
> > >>>> Changes from Update #2
> > >>>> ----------------------------------------------------------------------------
> > >>>> * Changed order of PRs swapping (4) and (5). The PR that activates
> > >>>> increases the max CI agent job time is independent of all the other
> > >>>> PRs and its review is complete, so it can be committed now.
> > >>>> * Pushed PRs (1), (2), (3), (4).
> > >>>> * Waiting for review to complete for PRs (5) and (6)
> > >>>> * Reviews complete for PR (7)
> > >>>> * Identifies steps using git filter-branch to apply uncrustify changes to
> a
> > >>>> code review patch series that was generated before the uncrustify
> > >>> changes
> > >>>> avoiding manual merge.
> > >>>> * Identified steps using git filter-repo to generate an alternate history
> of
> > >>>> the edk2 repo with uncrustify changes applied on every commit.
> This may
> > >>>> be useful when evaluating changes to files using tools like git blame
> > >>>> without the large uncrustify patch series.
> > >>>> ---------------------------------------------------------------------------
> > >>>>
> > >>>> Changes from Update #1
> > >>>> ----------------------------------------------------------------------------
> > >>>> * Changed order of PRs swapping (6) and (7). The PR that activates
> > >>>> EDK II CI check UncrustifyCheck has to be last because it
> unconditionally
> > >>>> checks all C/H files in all packages. Not just files that have been
> > >>>> modified like some of the other checkers.
> > >>>> * Updated link to the branch with the UncrustifyCheck plugin that has
> been
> > >>>> updated with a one line change and Reviewed-by and Tested-by
> tags.
> > >>>>
> > >>>>
> > >>>
> https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plug
> > >>> in_v
> > >>>> 6
> > >>>> * Reviews complete for (1), (2), (3), (5), and (7)
> > >>>> ---------------------------------------------------------------------------
> > >>>>
> > >>>> Michael Kubacki and I have prepared the patches required to apply
> the
> > >>>> uncrustify changes and enable EDK II CI to check all submitted
> > >>>> patches have been run through uncrustify.
> > >>>>
> > >>>> We have verified through the CompareBuild GitHub Action that the
> > >>>> format changes performed by uncrustify have no functional changes.
> > >>>> All of the OBJ, LIB, DLL, EFI, FFS, FV, and FD files match 100%
> > >>>> across 70 VS2019/GCC5 builds of all package/platform DSC files in
> > >>>> the edk2 repo.
> > >>>>
> > >>>> The hard freeze will be extended after the edk2-stable202111 tag
> until
> > >>>> all uncrustify related changes are committed. We do not expect this
> > >>>> to take more than a few days. Do not push any PRs until the hard
> > >>>> freeze is lifted.
> > >>>>
> > >>>> The changes are broken up into 7 patch series/PRs. The PRs are
> ordered
> > >>>> so they can be submitted using the normal submission process and
> EDK II
> > >>>> CI will pass for each one. Details are listed below.
> > >>>>
> > >>>> Uncrustify 73.0.3 for EDK II
> > >>>> =============================
> > >>>> * Sources:
> > >>> INVALID URI REMOVED
> > >>>
> stify__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKkoslR8D0LLPZTWXL0OKtMZJOYp
> > >>> cSVHoHwMD2t8$
> > >>>> * Documentation:
> > >>>>
> > >>> INVALID URI REMOVED
> > >>>
> /_wiki/wikis/Uncrustify.wiki/1/Proj__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKk
> > >>> oslR8D0LLPZTWXL0OKtMZJOYpcSVHoP8OtlxM$
> > >>>> ect-Mu-(EDK-II)-Fork-Readme
> > >>>> * Download:
> > >>>>
> > >>> INVALID URI REMOVED
> > >>>
> /_packaging?_a=package&feed=m__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKk
> > >>> oslR8D0LLPZTWXL0OKtMZJOYpcSVHo4yTWeAM$
> > >>>> u_uncrustify&package=mu-uncrustify-
> > >>>> release&protocolType=NuGet&version=73.0.3
> > >>>>
> > >>>> Installing Uncrustify
> > >>>> ======================
> > >>>> The Uncrustify tool is installed automatically when the Pytools
> > >>>> environment is used and the stuart* commands are run to complete
> the
> > >>>> environment setup. Please see:
> > >>>>
> > >>>> https://github.com/tianocore/edk2/tree/master/.pytool#running-
> ci-
> > >>> locally
> > >>>>
> > >>>> Uncrustify can also be installed from the download page listed above
> > >>>> or built from sources from the source link above.
> > >>>>
> > >>>> The Documentation link provides instruction on how to run uncrustify
> from
> > >>>> the command line or install as a Visual Studio Code plugin. The main
> > >>>> uncrustify documentation also describes how to integrate with a few
> other
> > >>>> editors.
> > >>>>
> > >>>> We have also discussed a client side githook. That effort has not
> started.
> > >>>> Let us know if that is a feature you would find useful.
> > >>>>
> > >>>> Developer impact for new code reviews
> > >>>> ======================================
> > >>>> Once the uncrustify checker is active in EDK II CI, developers must
> > >>>> make sure their patches are run through the uncrustify tool before
> > >>>> sending the patches for review.
> > >>>>
> > >>>> Developers must install and run uncrustify against changes files
> before
> > >>>> sending patch review emails or submitting PR for EDK II CI. If EDK II CI
> > >>>> detects and differences in source formatting, then EDK II CI will fail
> > >>>> and the developer must run uncrustify and resubmit the patches.
> > >>>>
> > >>>> Developer impact to patch series/PRs reviewed during edk2-
> stable201121
> > >>>> soft/hard freeze
> > >>>>
> > >>>
> ==========================================================
> > >>> =======
> > >>>> ======================
> > >>>> Developers must rebase their changes after the uncrustify source
> changes
> > >>> are
> > >>>> committed. The branch with a preview of the uncrustify changes can
> be
> > >>> used
> > >>>> to start this rebase work.
> > >>>>
> > >>>>
> > >>>>
> > >>>
> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyC
> > >>> han
> > >>>> ges_V5
> > >>>>
> > >>>> The following steps can be used to update an existing branch with the
> > >>>> required uncrustify format. This is the Windows version. I will add
> > >>>> the Linux version soon.
> > >>>>
> > >>>> 1) Fetch and checkout and rebase to latest edk2/master
> > >>>>
> > >>>> git fetch origin
> > >>>> git checkout master
> > >>>> git rebase origin/master
> > >>>>
> > >>>> 2) Make a backup copy of plugin UncrustifyCheck outside
> WORKSPACE.
> > >>>> (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable
> and
> > >>>> EDK II specific uncrustify configuration file available when working
> > >>>> with a branch that does not have those tools in its scope.
> > >>>>
> > >>>> xcopy .pytool\Plugin\UncrustifyCheck
> C:\Temp\UncrustifyCheck
> > >>>>
> > >>>> 3) Check out the patch series branch (e.g. MyBranch)
> > >>>>
> > >>>> git checkout MyBranch
> > >>>>
> > >>>> 4) Rebase patch series against edk2-stable202111
> > >>>>
> > >>>> git rebase edk2-stable202111
> > >>>>
> > >>>> 5) Create new branch for the uncrustifed version (e.g.
> > >>> MyBranch_Uncrustified)
> > >>>>
> > >>>> git checkout -b MyBranch_Uncrustified
> > >>>>
> > >>>> 6) Use git filter-branch to uncrustify all the commits in the series
> > >>>> between the rebase target from (2) and HEAD of the branch. A
> filter
> > >>>> can be used to scope the uncrustify operations to only the C/H
> files
> > >>>> in the specific package the patch series is against. (e.g.
> > >>> DynamicTablesPkg).
> > >>>> BaseTools should always be excluded. If the package scoped
> filter is
> > >>>> not used, it will still work, but will take longer to run because
> > >>>> uncrustify will rescan every C/H files in the whole repo.
> > >>>>
> > >>>> git filter-branch --tree-filter "git ls-files DynamicTablesPkg*.c
> > >>>> DynamicTablesPkg*.h :!BaseTools/* |
> c:\\Temp\\UncrustifyCheck\\mu-
> > >>>> uncrustify-release_extdep\\Windows-x86\\uncrustify.exe -c
> > >>>> c:\\Temp\\UncrustifyCheck\\uncrustify.cfg -F - --replace --no-backup
> --if-
> > >>>> changed" edk2-stable202111..HEAD
> > >>>>
> > >>>> 7) Now that all the individual patches in the branch are uncrustified,
> > >>>> rebase against latest edk2/master that is already uncrustified.
> > >>>>
> > >>>> git rebase master
> > >>>>
> > >>>> 8) Verify the patches in this new branch.
> > >>>>
> > >>>> Impacts to tracing history across the uncrusity changes
> > >>>>
> =======================================================
> > >>>> Tools the view file and line history do work with the large uncrustify
> > >>>> patch series. One impact is that the operations can be very slow due
> > >>>> to the large uncrustify patches.
> > >>>>
> > >>>> One option to provide a faster experience is to provide an alternate
> > >>>> version of the edk2 repository as "documentation" that has the
> > >>>> entire history re-written with uncrustify run on every commit.
> > >>>> The tool called git-filter-repo can be used to perform this
> > >>>> transformation and runs in a reasonable period of time (a few hours)
> > >>>>
> > >>>> https://github.com/newren/git-filter-repo
> > >>>> https://github.com/newren/git-filter-
> repo/blob/main/contrib/filter-
> > >>> repo-
> > >>>> demos/lint-history
> > >>>>
> > >>>> The following steps can be used to perform this transformation.
> > >>>> This is the Windows version. I will add the Linux version soon.
> > >>>>
> > >>>> ** WARNING ** This operation modifies(rewrites) all the commits
> > >>>> in the local copy of the repo. Do not perform
> > >>>> these steps on a local repo you are using for
> > >>>> active development.
> > >>>>
> > >>>> 1) Clone edk2 into a new directory (see **WARNING**)
> > >>>>
> > >>>> git clone https://github.com/tianocore/edk2.git edk2-
> uncrustified
> > >>>> cd edk2-uncrustified
> > >>>>
> > >>>> 2) Setup python virtual env, install pytools, and run stuart
> commands
> > >>>> to setup build environment which includes installing uncrustify
> tools.
> > >>>>
> > >>>>
> https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-
> > >>>> locally
> > >>>>
> > >>>> 3) Make a backup copy of plugin UncrustifyCheck outside
> WORKSPACE.
> > >>>> (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable
> and
> > >>>> EDK II specific uncrustify configuration file available when working
> > >>>> with a branch that does not have those tools in its scope.
> > >>>>
> > >>>> xcopy .pytool\Plugin\UncrustifyCheck
> C:\Temp\UncrustifyCheck
> > >>>>
> > >>>> 4) Use lint-history.py from git-filter-repo examples
> > >>>>
> > >>>> https://github.com/newren/git-filter-repo
> > >>>> https://github.com/newren/git-filter-
> repo/blob/main/contrib/filter-
> > >>> repo-
> > >>>> demos/lint-history
> > >>>>
> > >>>> Line #127 - Add try except around subprocess.check_call() with
> except
> > >>>> being pass. This is required because there are a few commits of C
> > >>>> files in the edk2 repo that have incorrect C syntax and do not
> > >>>> build with a C compiler and break the uncrustify parser. Skip
> reformat
> > >>>> of C files that can not be parsed by uncrustify. These rare
> instances
> > >>>> are addressed in the commit that fixes the C syntax error.
> > >>>>
> > >>>> Run this slightly modified version of lint-history. Include only
> > >>>> C/H files and exclude directories that start with 'Tools' or
> 'BaseTools'.
> > >>>> This step took about 2.2 hours on a laptop.
> > >>>>
> > >>>> lint-history.py
> > >>>> --relevant "return (not filename.startswith(b'Tools') and not
> > >>>> filename.startswith(b'BaseTools') and (filename.endswith(b'.c') or
> > >>>> filename.endswith(b'.h')))"
> > >>>> c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\mu-
> > >>> uncrustify-
> > >>>> release_extdep\\Windows-x86\\uncrustify.exe -c
> > >>>> c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\uncrustify.cfg
> --
> > >>> replace --
> > >>>> no-backup --if-changed
> > >>>>
> > >>>> Order of PRs to apply during extended hard freeze
> > >>>> ==================================================
> > >>>> 1) Update EmulatorPkg Win Host [BuildOptions] MSFT CC_FLAGS to
> not
> > >>> force
> > >>>> debug information
> > >>>> *
> > >>> INVALID URI REMOVED
> > >>>
> d=3747__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKkoslR8D0LLPZTWXL0OKtMZJ
> > >>> OYpcSVHo2BEG33c$
> > >>>> *
> > >>>>
> > >>>
> https://github.com/mdkinney/edk2/tree/Bug_3747_EmulatorPkg_WinHost_
> > >>> Rep
> > >>>> roducibleBuild
> > >>>> * https://github.com/tianocore/edk2/pull/2215
> > >>>> * Required for EmulatorPkg to pass CompareBuild for VS2019
> IA32/X64
> > >>> builds.
> > >>>> * Status: Review complete. PR pushed.
> > >>>>
> > >>>> 2) EccCheck should not revert staged and local changes
> > >>>> *
> > >>> INVALID URI REMOVED
> > >>>
> d=2986__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKkoslR8D0LLPZTWXL0OKtMZJ
> > >>> OYpcSVHotvR0Unk$
> > >>>> *
> > >>>>
> > >>>
> https://github.com/mdkinney/edk2/tree/Bug_2986_EccCheckRemoveGitRe
> > >>> vert
> > >>>> _V2
> > >>>> * https://github.com/tianocore/edk2/pull/2216
> > >>>> * Required for EDK II CI to complete in a reasonable period of time
> when
> > >>>> processing the 4000+ source file style changes made by uncrustify.
> > >>>> * Also fixes critical bugs that can potentially corrupt git state when
> > >>>> EccCheck is run locally.
> > >>>> * Status: Review complete. PR pushed.
> > >>>>
> > >>>> 3) Update pytool LicenseCheck plugin to use temp directory for diff
> output
> > >>> file
> > >>>> *
> > >>> INVALID URI REMOVED
> > >>>
> d=3746__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKkoslR8D0LLPZTWXL0OKtMZJ
> > >>> OYpcSVHoWRuHaBw$
> > >>>> *
> > >>>>
> > >>>
> https://github.com/mdkinney/edk2/tree/Bug_3746_LicenseCheckUseDiffO
> > >>> utput
> > >>>> File_V2
> > >>>> * https://github.com/tianocore/edk2/pull/2217
> > >>>> * Required to reduce EDK II CI build times.
> > >>>> * Status: Review complete. PR pushed.
> > >>>>
> > >>>> 4) Update max job time from 60 min to 120 minutes
> > >>> in .azurepipelines/templates
> > >>>> *
> > >>> INVALID URI REMOVED
> > >>>
> d=3750__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKkoslR8D0LLPZTWXL0OKtMZJ
> > >>> OYpcSVHozaVshgc$
> > >>>> *
> > >>>>
> > >>>
> https://github.com/mdkinney/edk2/tree/Bug_3750_IncreaseAzurePipelines
> > >>> Time
> > >>>> out
> > >>>> * https://github.com/tianocore/edk2/pull/2219
> > >>>> * Required to allow EccCheck of uncrustify changes to complete on
> Azure
> > >>>> Pipelines CI agents without timing out.
> > >>>> * Status: Review complete. PR pushed.
> > >>>>
> > >>>> 5) Update Package YAML to ignore specific ECC files/errors
> > >>>> *
> > >>> INVALID URI REMOVED
> > >>>
> d=3749__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKkoslR8D0LLPZTWXL0OKtMZJ
> > >>> OYpcSVHo-fY3dMo$
> > >>>> *
> > >>>>
> > >>>
> https://github.com/mdkinney/edk2/tree/Bug_3749_EccCheckIgnoreFilesErr
> > >>> ors
> > >>>> * https://github.com/tianocore/edk2/pull/2218
> > >>>> * Required to pass EccCheck
> > >>>> * Status: Review complete. PR pushed
> > >>>>
> > >>>> 6) Uncrustify Source Changes
> > >>>> *
> > >>> INVALID URI REMOVED
> > >>>
> d=3737__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKkoslR8D0LLPZTWXL0OKtMZJ
> > >>> OYpcSVHoMRa47YU$
> > >>>> *
> > >>> INVALID URI REMOVED
> > >>>
> d=3739__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKkoslR8D0LLPZTWXL0OKtMZJ
> > >>> OYpcSVHoPAlckP8$
> > >>>> *
> > >>>>
> > >>>
> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyC
> > >>> han
> > >>>> ges_V5
> > >>>> * https://github.com/tianocore/edk2/pull/2229
> > >>>> * Build comparison result PASS:
> > >>>> https://github.com/mdkinney/edk2/actions/runs/1521618836
> > >>>> * EFI_D_ -> DEBUG changes required to pass PatchCheck
> > >>>> * Uncrustify format changes required to pass UncrustifyCheck
> > >>>> * Status:
> > >>>> Waiting for review
> > >>>>
> > >>>> 7) UncrustifyCheck EDK II CI Plugin
> > >>>> *
> > >>> INVALID URI REMOVED
> > >>>
> d=3748__;!!NpxR!3rTN8lYPg_5bczi9Qai9VWvHKkoslR8D0LLPZTWXL0OKtMZJ
> > >>> OYpcSVHoCzEWymQ$
> > >>>> *
> > >>>>
> > >>>
> https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plug
> > >>> in_v
> > >>>> 6
> > >>>> * Required to enforce all PRs submitted to EDK II CI match
> uncrustify
> > >>> format.
> > >>>> * Unconditionally checks all packages. Can not be committed until
> all C/H
> > >>>> source files have been updated.
> > >>>> * Status: Review complete
> > >>>>
> > >>>> Combined Branch/PR for Review/Test
> > >>>> ==================================
> > >>>> * Build Comparison results must pass 100% across the full set of PRs
> before
> > >>>> the individual PRs can be pushed in the order listed above.
> > >>>> * Branch:
> > >>>>
> https://github.com/mdkinney/edk2/tree/TestOnly_Uncrustify_PR_Series
> > >>>> * PR: https://github.com/tianocore/edk2/pull/2229
> > >>>> Status = PASS
> > >>>> * CompareBuild:
> > >>>> Branch:
> > >>>>
> > >>>
> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyC
> > >>> han
> > >>>> ges_V5
> > >>>> --ref1: ef9a059cdb15844fe52a49af2bf7d86b9dd3e9bf
> > >>>> --ref2: Bug_3737_3739_ApplyUncrustifyChanges_V5
> > >>>> Extra Options: -n 4 --quiet
> > >>>> Results:
> https://github.com/mdkinney/edk2/actions/runs/1521618836
> > >>>> 30 VS2019 build comparisons PASS
> > >>>> 40 GCC5 build comparisons PASS
> > >>>> 100% PASS
> > >>>>
> > >>>> The following git log shows the set of patches from --ref1 to --ref
> 2across
> > >>>> which there are no differences in any of the
> OBJ/LIB/DLL/EFI/FFS/FV/FD
> > >>> files.
> > >>>>
> > >>>> --ref2
> > >>>> b7d4bf0675b7 (HEAD -> Bug_3737_3739_ApplyUncrustifyChanges_V5)
> > >>>> UnitTestFrameworkPkg: Apply uncrusitify changes
> > >>>> 7f03d25f60e7 UefiPayloadPkg: Apply uncrusitify changes
> > >>>> 0bfd8d9b5ac9 UefiCpuPkg: Apply uncrusitify changes
> > >>>> e1cd9bfb9dea StandaloneMmPkg: Apply uncrusitify changes
> > >>>> 5da2f65be378 SourceLevelDebugPkg: Apply uncrusitify changes
> > >>>> 95b86de07e5d SignedCapsulePkg: Apply uncrusitify changes
> > >>>> fe71d97246c4 ShellPkg: Apply uncrusitify changes
> > >>>> 54c21c952992 SecurityPkg: Apply uncrusitify changes
> > >>>> 187a3785f12b RedfishPkg: Apply uncrusitify changes
> > >>>> 810100002a46 PcAtChipsetPkg: Apply uncrusitify changes
> > >>>> 276a695c0cf2 OvmfPkg: Apply uncrusitify changes
> > >>>> 303c0a91ab07 NetworkPkg: Apply uncrusitify changes
> > >>>> bc80792cd1b1 MdePkg: Apply uncrusitify changes
> > >>>> 3ea86be17a2a MdeModulePkg: Apply uncrusitify changes
> > >>>> c70ef11ed0cd IntelFsp2WrapperPkg: Apply uncrusitify changes
> > >>>> c0291221f252 IntelFsp2Pkg: Apply uncrusitify changes
> > >>>> 6a479952a690 FmpDevicePkg: Apply uncrusitify changes
> > >>>> 3a7c05b7070d FatPkg: Apply uncrusitify changes
> > >>>> b789f98c8959 EmulatorPkg: Apply uncrusitify changes
> > >>>> 952d7a1c9220 EmbeddedPkg: Apply uncrusitify changes
> > >>>> a1cc9881bab6 DynamicTablesPkg: Apply uncrusitify changes
> > >>>> 50654dfe5785 CryptoPkg: Apply uncrusitify changes
> > >>>> ed965a02dfa1 ArmVirtPkg: Apply uncrusitify changes
> > >>>> 9744023fbc46 ArmPlatformPkg: Apply uncrusitify changes
> > >>>> 7a1cde5f5bba ArmPkg: Apply uncrusitify changes
> > >>>> 19d17e0913e8 UefiCpuPkg: Change use of EFI_D_* to DEBUG_*
> > >>>> ffa718b4f994 SourceLevelDebugPkg: Change use of EFI_D_* to
> DEBUG_*
> > >>>> b86cb3c5e5b4 ShellPkg: Change use of EFI_D_* to DEBUG_*
> > >>>> c7c42204dc07 SecurityPkg: Change use of EFI_D_* to DEBUG_*
> > >>>> 16b8e6f958e4 PcAtChipsetPkg: Change use of EFI_D_* to DEBUG_*
> > >>>> 0ac3f8b2dac5 OvmfPkg: Change use of EFI_D_* to DEBUG_*
> > >>>> bc5004b8d294 NetworkPkg: Change use of EFI_D_* to DEBUG_*
> > >>>> 6f671a8e2377 MdePkg: Change use of EFI_D_* to DEBUG_*
> > >>>> a10c610ff9a3 MdeModulePkg: Change use of EFI_D_* to DEBUG_*
> > >>>> 09a3bddba390 FatPkg: Change use of EFI_D_* to DEBUG_*
> > >>>> 59c61318246a EmulatorPkg: Change use of EFI_D_* to DEBUG_*
> > >>>> 3a80367dda3b EmbeddedPkg: Change use of EFI_D_* to DEBUG_*
> > >>>> 23eb1aaf80ca ArmVirtPkg: Change use of EFI_D_* to DEBUG_*
> > >>>> 875914b45c54 ArmPlatformPkg: Change use of EFI_D_* to DEBUG_*
> > >>>> eb2eca82b451 ArmPkg: Change use of EFI_D_* to DEBUG_*
> > >>>> f0f3f5aae7c4 (origin/master, origin/HEAD, master)
> UnitTestFrameworkPkg:
> > >>>> Update YAML to ignore specific ECC files/errors
> > >>>> c05734797790 UefiPayloadPkg: Update YAML to ignore specific ECC
> > >>> files/errors
> > >>>> c30c40d6c63d StandaloneMmPkg: Update YAML to ignore specific
> ECC
> > >>>> files/errors
> > >>>> 9944508e85f1 ShellPkg: Update YAML to ignore specific ECC
> files/errors
> > >>>> 60fa40be458d SecurityPkg: Update YAML to ignore specific ECC
> files/errors
> > >>>> df790cd6b37e MdePkg: Update YAML to ignore specific ECC
> files/errors
> > >>>> 9deb9370766e MdeModulePkg: Update YAML to ignore specific ECC
> > >>> files/errors
> > >>>> d7d30e8f219f EmulatorPkg: Update YAML to ignore specific ECC
> files/errors
> > >>>> d5744ecba813 CryptoPkg: Update YAML to ignore specific ECC
> files/errors
> > >>>> c97fee87f0f9 ArmVirtPkg: Update YAML to ignore specific ECC
> files/errors
> > >>>> 1939fc9569f2 ArmPlatformPkg: Update YAML to ignore specific ECC
> > >>> files/errors
> > >>>> 365dced2c37a ArmPkg: Update YAML to ignore specific ECC
> files/errors
> > >>>> 76a1ce4d5fec .azurepipelines/templates: Update max pipeline job
> time to
> > >>> 2
> > >>>> hours
> > >>>> 99f84ff47390 .pytools/Plugin/LicenseCheck: Use temp directory for
> git diff
> > >>>> output
> > >>>> 3019f1bbabf1 .pytool/Plugin/EccCheck: Add performance
> optimizations
> > >>>> 854462bd3479 .pytool/Plugin/EccCheck: Remove temp directory on
> > >>> exception
> > >>>> 69877614fdee .pytool/Plugin/EccCheck: Remove RevertCode()
> > >>>> --ref1
> > >>>> ef9a059cdb15 EmulatorPkg/Win/Host: Update CC_FLAGS
> > >>>> bb1bba3d7767 (tag: edk2-stable202111) NetworkPkg: Fix invalid
> pointer for
> > >>> DNS
> > >>>> response token on error
> > >>>>
> > >>>> Best regards,
> > >>>>
> > >>>> Mike
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >
> > >
> > >
> > >
> > >
> > >
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
2021-12-01 17:05 ` Michael Kubacki
2021-12-01 17:39 ` Michael D Kinney
@ 2021-12-02 11:00 ` Gerd Hoffmann
2021-12-02 15:33 ` Michael Kubacki
1 sibling, 1 reply; 39+ messages in thread
From: Gerd Hoffmann @ 2021-12-02 11:00 UTC (permalink / raw)
To: devel, mikuback
Cc: michael.d.kinney, Chang, Abner, Wang, Jian J, Michael Kubacki,
Andrew Fish (afish@apple.com), Leif Lindholm
On Wed, Dec 01, 2021 at 12:05:24PM -0500, Michael Kubacki wrote:
> This can of course be done at a technical level but it smells of a code
> cohesion problem.
>
> The code is in the edk2 project so it would reason that it should be uniform
> in style with other code and at least close to the EDK II C Coding Standard
> Specification.
>
> Currently, git submodules and ignored path are already excluded in addition
> to the ability to mark packages in "audit mode" so they do not fail the
> results. I would personally prefer not to allow arbitrary subdirectories to
> diverge from the rest of the codebase / package but if there's consensus
> this is a good idea, I can look into adding it.
What is the plan for generated include files, for example the ones in
CryptoPkg/Library/Include/openssl?
take care,
Gerd
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
2021-12-02 11:00 ` Gerd Hoffmann
@ 2021-12-02 15:33 ` Michael Kubacki
2021-12-02 16:23 ` Gerd Hoffmann
0 siblings, 1 reply; 39+ messages in thread
From: Michael Kubacki @ 2021-12-02 15:33 UTC (permalink / raw)
To: devel, kraxel
Cc: michael.d.kinney, Chang, Abner, Wang, Jian J, Michael Kubacki,
Andrew Fish (afish@apple.com), Leif Lindholm
Hi Gerd,
To help me understand which files you're specifically referring to, can
you please point them out from this commit? Or provide additional details?
https://github.com/tianocore/edk2/pull/2229/commits/50654dfe5785964c9ae72961d13a50b26af77794
CryptoPkg/Library/Include/openssl/opensslconf.h is currently formatted.
Regards,
Michael
On 12/2/2021 6:00 AM, Gerd Hoffmann wrote:
> On Wed, Dec 01, 2021 at 12:05:24PM -0500, Michael Kubacki wrote:
>> This can of course be done at a technical level but it smells of a code
>> cohesion problem.
>>
>> The code is in the edk2 project so it would reason that it should be uniform
>> in style with other code and at least close to the EDK II C Coding Standard
>> Specification.
>>
>> Currently, git submodules and ignored path are already excluded in addition
>> to the ability to mark packages in "audit mode" so they do not fail the
>> results. I would personally prefer not to allow arbitrary subdirectories to
>> diverge from the rest of the codebase / package but if there's consensus
>> this is a good idea, I can look into adding it.
>
> What is the plan for generated include files, for example the ones in
> CryptoPkg/Library/Include/openssl?
>
> take care,
> Gerd
>
>
>
>
>
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
2021-12-02 15:33 ` Michael Kubacki
@ 2021-12-02 16:23 ` Gerd Hoffmann
2021-12-02 17:15 ` Michael Kubacki
0 siblings, 1 reply; 39+ messages in thread
From: Gerd Hoffmann @ 2021-12-02 16:23 UTC (permalink / raw)
To: Michael Kubacki
Cc: devel, michael.d.kinney, Chang, Abner, Wang, Jian J,
Michael Kubacki, Andrew Fish (afish@apple.com), Leif Lindholm
On Thu, Dec 02, 2021 at 10:33:14AM -0500, Michael Kubacki wrote:
> Hi Gerd,
>
> To help me understand which files you're specifically referring to, can you
> please point them out from this commit? Or provide additional details?
>
> https://github.com/tianocore/edk2/pull/2229/commits/50654dfe5785964c9ae72961d13a50b26af77794
>
> CryptoPkg/Library/Include/openssl/opensslconf.h is currently formatted.
Yes, that one, and there is another in CryptoPkg/Library/Include/crypto/
The switch to openssl 3.0 will add more of those files[1]
They are generated from openssl source code,
CryptoPkg/Library/OpensslLib/process_files.pl handles that.
I think the two reasonable options to deal with that are:
(1) exclude those files from formating, or
(2) call uncrustify in process_files.pl to reformat them
each time they are generated.
Given that these files will never be edited manually my personal
preference would be (1).
take care,
Gerd
[1] wip branch @ https://github.com/kraxel/edk2/commits/openssl3
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
2021-12-02 16:23 ` Gerd Hoffmann
@ 2021-12-02 17:15 ` Michael Kubacki
2021-12-02 17:33 ` Michael D Kinney
0 siblings, 1 reply; 39+ messages in thread
From: Michael Kubacki @ 2021-12-02 17:15 UTC (permalink / raw)
To: Gerd Hoffmann
Cc: devel, michael.d.kinney, Chang, Abner, Wang, Jian J,
Michael Kubacki, Andrew Fish (afish@apple.com), Leif Lindholm
Thank you for clarifying. This case is more reasonable to me since the
files are auto generated and not ported into edk2.
There is an option to bring files like this in as a new submodule but I
understand why that might not be desirable and would be disruptive to
the current process.
If there's no objection, I'll add an option to exclude paths within a
package in a v6 patch of the plugin and leave this to maintainer
discretion. I just ask that the spirit of consistent formatting be
considered when making these decisions.
Regards,
Michael
On 12/2/2021 11:23 AM, Gerd Hoffmann wrote:
> On Thu, Dec 02, 2021 at 10:33:14AM -0500, Michael Kubacki wrote:
>> Hi Gerd,
>>
>> To help me understand which files you're specifically referring to, can you
>> please point them out from this commit? Or provide additional details?
>>
>> https://github.com/tianocore/edk2/pull/2229/commits/50654dfe5785964c9ae72961d13a50b26af77794
>>
>> CryptoPkg/Library/Include/openssl/opensslconf.h is currently formatted.
>
> Yes, that one, and there is another in CryptoPkg/Library/Include/crypto/
> The switch to openssl 3.0 will add more of those files[1]
>
> They are generated from openssl source code,
> CryptoPkg/Library/OpensslLib/process_files.pl handles that.
>
> I think the two reasonable options to deal with that are:
>
> (1) exclude those files from formating, or
> (2) call uncrustify in process_files.pl to reformat them
> each time they are generated.
>
> Given that these files will never be edited manually my personal
> preference would be (1).
>
> take care,
> Gerd
>
> [1] wip branch @ https://github.com/kraxel/edk2/commits/openssl3
>
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
2021-12-02 17:15 ` Michael Kubacki
@ 2021-12-02 17:33 ` Michael D Kinney
0 siblings, 0 replies; 39+ messages in thread
From: Michael D Kinney @ 2021-12-02 17:33 UTC (permalink / raw)
To: Michael Kubacki, Gerd Hoffmann, Kinney, Michael D
Cc: devel@edk2.groups.io, Chang, Abner, Wang, Jian J, Michael Kubacki,
Andrew Fish (afish@apple.com), Leif Lindholm
Michael,
Given that there are now 2 requests for this feature, I think it is a
good idea to add the feature.
Package YAML files are part of the review content, so we can make sure
we all look at these YAML changes and provide feedback when this
uncrustify exclusion feature is applied in areas that do not make
sense.
One additional challenge with the exclusion feature is how developers
can run uncrustify locally to prepare their patch reviews. They can
not run it on all c/h files in a package. Now they have to know which
directories to exclude. This applies to running uncrustify from command
line and usages such as Visual Studio Code plugin. In order to make
it easy for developers to always get the right format, you would need
a helper tool that can look at the YAML file to know if there are
areas that are excluded and exclude files from the uncrustify processing.
I am concerned that someone will miss the exclusion list in patch
generation and review and the uncrustified versions will get checked
in by accident at some point. EDK II CI will not care because it will
skip the UncrustifyCheck on excluded files/dirs.
Thanks,
Mike
> -----Original Message-----
> From: Michael Kubacki <mikuback@linux.microsoft.com>
> Sent: Thursday, December 2, 2021 9:15 AM
> To: Gerd Hoffmann <kraxel@redhat.com>
> Cc: devel@edk2.groups.io; Kinney, Michael D <michael.d.kinney@intel.com>; Chang, Abner <abner.chang@hpe.com>; Wang, Jian J
> <jian.j.wang@intel.com>; Michael Kubacki <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com) <afish@apple.com>;
> Leif Lindholm <leif@nuviainc.com>
> Subject: Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
>
> Thank you for clarifying. This case is more reasonable to me since the
> files are auto generated and not ported into edk2.
>
> There is an option to bring files like this in as a new submodule but I
> understand why that might not be desirable and would be disruptive to
> the current process.
>
> If there's no objection, I'll add an option to exclude paths within a
> package in a v6 patch of the plugin and leave this to maintainer
> discretion. I just ask that the spirit of consistent formatting be
> considered when making these decisions.
>
> Regards,
> Michael
>
> On 12/2/2021 11:23 AM, Gerd Hoffmann wrote:
> > On Thu, Dec 02, 2021 at 10:33:14AM -0500, Michael Kubacki wrote:
> >> Hi Gerd,
> >>
> >> To help me understand which files you're specifically referring to, can you
> >> please point them out from this commit? Or provide additional details?
> >>
> >> https://github.com/tianocore/edk2/pull/2229/commits/50654dfe5785964c9ae72961d13a50b26af77794
> >>
> >> CryptoPkg/Library/Include/openssl/opensslconf.h is currently formatted.
> >
> > Yes, that one, and there is another in CryptoPkg/Library/Include/crypto/
> > The switch to openssl 3.0 will add more of those files[1]
> >
> > They are generated from openssl source code,
> > CryptoPkg/Library/OpensslLib/process_files.pl handles that.
> >
> > I think the two reasonable options to deal with that are:
> >
> > (1) exclude those files from formating, or
> > (2) call uncrustify in process_files.pl to reformat them
> > each time they are generated.
> >
> > Given that these files will never be edited manually my personal
> > preference would be (1).
> >
> > take care,
> > Gerd
> >
> > [1] wip branch @ https://github.com/kraxel/edk2/commits/openssl3
> >
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
2021-11-30 22:34 Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4 Michael D Kinney
` (7 preceding siblings ...)
2021-12-01 18:52 ` Bret Barkelew
@ 2021-12-02 18:27 ` Maciej Rabeda
2021-12-02 19:45 ` Michael D Kinney
2021-12-03 8:56 ` Gerd Hoffmann
9 siblings, 1 reply; 39+ messages in thread
From: Maciej Rabeda @ 2021-12-02 18:27 UTC (permalink / raw)
To: devel, michael.d.kinney, Michael Kubacki,
Andrew Fish (afish@apple.com), Leif Lindholm
[-- Attachment #1: Type: text/plain, Size: 20308 bytes --]
Hey Mike,
While most of the changes related to fixing coding style violations,
there are a couple of changes to NetworkPkg in that PR that make the
code less readable. Examples below.
Example 1:
Example 2:
Example 3:
Example 4:
On 30-Nov-21 23:34, Michael D Kinney wrote:
> Hello,
>
> Thank you for your patience during this extended hard freeze.
>
> Just one more step to go. There has been a delay in the review of
> the patch series with the uncrustify source changes. PR(6). This
> patch series was not sent out as patch review email because of its
> very large size. It only contains source style changes and the
> CompareBuild tool and GitHub action has shown there are no binary
> differences introduced with these source style changes.
>
> If you are a package maintainer, then please review the following
> branch/PR for your package contents and review the EDK II CI results
> and BuildCompare results. I do not expect a line by line review
> because we already had time to provide feedback on the source style
> performed by uncrustify. Instead, a Reviewed-by for your package
> indicates that you have reviewed the EDK II CI results and CompareBuild
> tool functionality and results and you accept the source style
> changes to your package.
>
> *https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
> *https://github.com/tianocore/edk2/pull/2229
> *https://github.com/mdkinney/edk2/actions/runs/1521618836
>
> Additional details on this update below.
>
> Thank you,
>
> Mike
>
>
> Changes from Update #3
> ----------------------------------------------------------------------------
> * Pushed PR (5)
> * Added link to PR(6). EDK II CI Status is PASS. Build Compare PASS.
> * Waiting for review of PR (6)
> * Review of PR (7) completed and waiting for review of PR (6)
> ----------------------------------------------------------------------------
>
> Changes from Update #2
> ----------------------------------------------------------------------------
> * Changed order of PRs swapping (4) and (5). The PR that activates
> increases the max CI agent job time is independent of all the other
> PRs and its review is complete, so it can be committed now.
> * Pushed PRs (1), (2), (3), (4).
> * Waiting for review to complete for PRs (5) and (6)
> * Reviews complete for PR (7)
> * Identifies steps using git filter-branch to apply uncrustify changes to a
> code review patch series that was generated before the uncrustify changes
> avoiding manual merge.
> * Identified steps using git filter-repo to generate an alternate history of
> the edk2 repo with uncrustify changes applied on every commit. This may
> be useful when evaluating changes to files using tools like git blame
> without the large uncrustify patch series.
> ---------------------------------------------------------------------------
>
> Changes from Update #1
> ----------------------------------------------------------------------------
> * Changed order of PRs swapping (6) and (7). The PR that activates
> EDK II CI check UncrustifyCheck has to be last because it unconditionally
> checks all C/H files in all packages. Not just files that have been
> modified like some of the other checkers.
> * Updated link to the branch with the UncrustifyCheck plugin that has been
> updated with a one line change and Reviewed-by and Tested-by tags.
> https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v6
> * Reviews complete for (1), (2), (3), (5), and (7)
> ---------------------------------------------------------------------------
>
> Michael Kubacki and I have prepared the patches required to apply the
> uncrustify changes and enable EDK II CI to check all submitted
> patches have been run through uncrustify.
>
> We have verified through the CompareBuild GitHub Action that the
> format changes performed by uncrustify have no functional changes.
> All of the OBJ, LIB, DLL, EFI, FFS, FV, and FD files match 100%
> across 70 VS2019/GCC5 builds of all package/platform DSC files in
> the edk2 repo.
>
> The hard freeze will be extended after the edk2-stable202111 tag until
> all uncrustify related changes are committed. We do not expect this
> to take more than a few days. Do not push any PRs until the hard
> freeze is lifted.
>
> The changes are broken up into 7 patch series/PRs. The PRs are ordered
> so they can be submitted using the normal submission process and EDK II
> CI will pass for each one. Details are listed below.
>
> Uncrustify 73.0.3 for EDK II
> =============================
> * Sources:https://dev.azure.com/projectmu/_git/Uncrustify
> * Documentation:https://dev.azure.com/projectmu/Uncrustify/_wiki/wikis/Uncrustify.wiki/1/Project-Mu-(EDK-II)-Fork-Readme
> * Download:https://dev.azure.com/projectmu/Uncrustify/_packaging?_a=package&feed=mu_uncrustify&package=mu-uncrustify-release&protocolType=NuGet&version=73.0.3
>
> Installing Uncrustify
> ======================
> The Uncrustify tool is installed automatically when the Pytools
> environment is used and the stuart* commands are run to complete the
> environment setup. Please see:
>
> https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally
>
> Uncrustify can also be installed from the download page listed above
> or built from sources from the source link above.
>
> The Documentation link provides instruction on how to run uncrustify from
> the command line or install as a Visual Studio Code plugin. The main
> uncrustify documentation also describes how to integrate with a few other
> editors.
>
> We have also discussed a client side githook. That effort has not started.
> Let us know if that is a feature you would find useful.
>
> Developer impact for new code reviews
> ======================================
> Once the uncrustify checker is active in EDK II CI, developers must
> make sure their patches are run through the uncrustify tool before
> sending the patches for review.
>
> Developers must install and run uncrustify against changes files before
> sending patch review emails or submitting PR for EDK II CI. If EDK II CI
> detects and differences in source formatting, then EDK II CI will fail
> and the developer must run uncrustify and resubmit the patches.
>
> Developer impact to patch series/PRs reviewed during edk2-stable201121 soft/hard freeze
> =======================================================================================
> Developers must rebase their changes after the uncrustify source changes are
> committed. The branch with a preview of the uncrustify changes can be used
> to start this rebase work.
>
> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
>
> The following steps can be used to update an existing branch with the
> required uncrustify format. This is the Windows version. I will add
> the Linux version soon.
>
> 1) Fetch and checkout and rebase to latest edk2/master
>
> git fetch origin
> git checkout master
> git rebase origin/master
>
> 2) Make a backup copy of plugin UncrustifyCheck outside WORKSPACE.
> (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable and
> EDK II specific uncrustify configuration file available when working
> with a branch that does not have those tools in its scope.
>
> xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
>
> 3) Check out the patch series branch (e.g. MyBranch)
>
> git checkout MyBranch
>
> 4) Rebase patch series against edk2-stable202111
>
> git rebase edk2-stable202111
>
> 5) Create new branch for the uncrustifed version (e.g. MyBranch_Uncrustified)
>
> git checkout -b MyBranch_Uncrustified
>
> 6) Use git filter-branch to uncrustify all the commits in the series
> between the rebase target from (2) and HEAD of the branch. A filter
> can be used to scope the uncrustify operations to only the C/H files
> in the specific package the patch series is against. (e.g. DynamicTablesPkg).
> BaseTools should always be excluded. If the package scoped filter is
> not used, it will still work, but will take longer to run because
> uncrustify will rescan every C/H files in the whole repo.
>
> git filter-branch --tree-filter "git ls-files DynamicTablesPkg*.c DynamicTablesPkg*.h :!BaseTools/* | c:\\Temp\\UncrustifyCheck\\mu-uncrustify-release_extdep\\Windows-x86\\uncrustify.exe -c c:\\Temp\\UncrustifyCheck\\uncrustify.cfg -F - --replace --no-backup --if-changed" edk2-stable202111..HEAD
>
> 7) Now that all the individual patches in the branch are uncrustified,
> rebase against latest edk2/master that is already uncrustified.
>
> git rebase master
>
> 8) Verify the patches in this new branch.
>
> Impacts to tracing history across the uncrusity changes
> =======================================================
> Tools the view file and line history do work with the large uncrustify
> patch series. One impact is that the operations can be very slow due
> to the large uncrustify patches.
>
> One option to provide a faster experience is to provide an alternate
> version of the edk2 repository as "documentation" that has the
> entire history re-written with uncrustify run on every commit.
> The tool called git-filter-repo can be used to perform this
> transformation and runs in a reasonable period of time (a few hours)
>
> https://github.com/newren/git-filter-repo
> https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-demos/lint-history
>
> The following steps can be used to perform this transformation.
> This is the Windows version. I will add the Linux version soon.
>
> ** WARNING ** This operation modifies(rewrites) all the commits
> in the local copy of the repo. Do not perform
> these steps on a local repo you are using for
> active development.
>
> 1) Clone edk2 into a new directory (see **WARNING**)
>
> git clonehttps://github.com/tianocore/edk2.git edk2-uncrustified
> cd edk2-uncrustified
>
> 2) Setup python virtual env, install pytools, and run stuart commands
> to setup build environment which includes installing uncrustify tools.
>
> https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally
>
> 3) Make a backup copy of plugin UncrustifyCheck outside WORKSPACE.
> (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable and
> EDK II specific uncrustify configuration file available when working
> with a branch that does not have those tools in its scope.
>
> xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
>
> 4) Use lint-history.py from git-filter-repo examples
>
> https://github.com/newren/git-filter-repo
> https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-demos/lint-history
>
> Line #127 - Add try except around subprocess.check_call() with except
> being pass. This is required because there are a few commits of C
> files in the edk2 repo that have incorrect C syntax and do not
> build with a C compiler and break the uncrustify parser. Skip reformat
> of C files that can not be parsed by uncrustify. These rare instances
> are addressed in the commit that fixes the C syntax error.
>
> Run this slightly modified version of lint-history. Include only
> C/H files and exclude directories that start with 'Tools' or 'BaseTools'.
> This step took about 2.2 hours on a laptop.
>
> lint-history.py
> --relevant "return (not filename.startswith(b'Tools') and not filename.startswith(b'BaseTools') and (filename.endswith(b'.c') or filename.endswith(b'.h')))"
> c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\mu-uncrustify-release_extdep\\Windows-x86\\uncrustify.exe -c c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\uncrustify.cfg --replace --no-backup --if-changed
>
> Order of PRs to apply during extended hard freeze
> ==================================================
> 1) Update EmulatorPkg Win Host [BuildOptions] MSFT CC_FLAGS to not force debug information
> *https://bugzilla.tianocore.org/show_bug.cgi?id=3747
> *https://github.com/mdkinney/edk2/tree/Bug_3747_EmulatorPkg_WinHost_ReproducibleBuild
> *https://github.com/tianocore/edk2/pull/2215
> * Required for EmulatorPkg to pass CompareBuild for VS2019 IA32/X64 builds.
> * Status: Review complete. PR pushed.
>
> 2) EccCheck should not revert staged and local changes
> *https://bugzilla.tianocore.org/show_bug.cgi?id=2986
> *https://github.com/mdkinney/edk2/tree/Bug_2986_EccCheckRemoveGitRevert_V2
> *https://github.com/tianocore/edk2/pull/2216
> * Required for EDK II CI to complete in a reasonable period of time when
> processing the 4000+ source file style changes made by uncrustify.
> * Also fixes critical bugs that can potentially corrupt git state when
> EccCheck is run locally.
> * Status: Review complete. PR pushed.
>
> 3) Update pytool LicenseCheck plugin to use temp directory for diff output file
> *https://bugzilla.tianocore.org/show_bug.cgi?id=3746
> *https://github.com/mdkinney/edk2/tree/Bug_3746_LicenseCheckUseDiffOutputFile_V2
> *https://github.com/tianocore/edk2/pull/2217
> * Required to reduce EDK II CI build times.
> * Status: Review complete. PR pushed.
>
> 4) Update max job time from 60 min to 120 minutes in .azurepipelines/templates
> *https://bugzilla.tianocore.org/show_bug.cgi?id=3750
> *https://github.com/mdkinney/edk2/tree/Bug_3750_IncreaseAzurePipelinesTimeout
> *https://github.com/tianocore/edk2/pull/2219
> * Required to allow EccCheck of uncrustify changes to complete on Azure
> Pipelines CI agents without timing out.
> * Status: Review complete. PR pushed.
>
> 5) Update Package YAML to ignore specific ECC files/errors
> *https://bugzilla.tianocore.org/show_bug.cgi?id=3749
> *https://github.com/mdkinney/edk2/tree/Bug_3749_EccCheckIgnoreFilesErrors
> *https://github.com/tianocore/edk2/pull/2218
> * Required to pass EccCheck
> * Status: Review complete. PR pushed
>
> 6) Uncrustify Source Changes
> *https://bugzilla.tianocore.org/show_bug.cgi?id=3737
> *https://bugzilla.tianocore.org/show_bug.cgi?id=3739
> *https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
> *https://github.com/tianocore/edk2/pull/2229
> * Build comparison result PASS:https://github.com/mdkinney/edk2/actions/runs/1521618836
> * EFI_D_ -> DEBUG changes required to pass PatchCheck
> * Uncrustify format changes required to pass UncrustifyCheck
> * Status:
> Waiting for review
>
> 7) UncrustifyCheck EDK II CI Plugin
> *https://bugzilla.tianocore.org/show_bug.cgi?id=3748
> *https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v6
> * Required to enforce all PRs submitted to EDK II CI match uncrustify format.
> * Unconditionally checks all packages. Can not be committed until all C/H
> source files have been updated.
> * Status: Review complete
>
> Combined Branch/PR for Review/Test
> ==================================
> * Build Comparison results must pass 100% across the full set of PRs before
> the individual PRs can be pushed in the order listed above.
> * Branch:https://github.com/mdkinney/edk2/tree/TestOnly_Uncrustify_PR_Series
> * PR:https://github.com/tianocore/edk2/pull/2229
> Status = PASS
> * CompareBuild:
> Branch:https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
> --ref1: ef9a059cdb15844fe52a49af2bf7d86b9dd3e9bf
> --ref2: Bug_3737_3739_ApplyUncrustifyChanges_V5
> Extra Options: -n 4 --quiet
> Results:https://github.com/mdkinney/edk2/actions/runs/1521618836
> 30 VS2019 build comparisons PASS
> 40 GCC5 build comparisons PASS
> 100% PASS
>
> The following git log shows the set of patches from --ref1 to --ref 2across
> which there are no differences in any of the OBJ/LIB/DLL/EFI/FFS/FV/FD files.
>
> --ref2
> b7d4bf0675b7 (HEAD -> Bug_3737_3739_ApplyUncrustifyChanges_V5) UnitTestFrameworkPkg: Apply uncrusitify changes
> 7f03d25f60e7 UefiPayloadPkg: Apply uncrusitify changes
> 0bfd8d9b5ac9 UefiCpuPkg: Apply uncrusitify changes
> e1cd9bfb9dea StandaloneMmPkg: Apply uncrusitify changes
> 5da2f65be378 SourceLevelDebugPkg: Apply uncrusitify changes
> 95b86de07e5d SignedCapsulePkg: Apply uncrusitify changes
> fe71d97246c4 ShellPkg: Apply uncrusitify changes
> 54c21c952992 SecurityPkg: Apply uncrusitify changes
> 187a3785f12b RedfishPkg: Apply uncrusitify changes
> 810100002a46 PcAtChipsetPkg: Apply uncrusitify changes
> 276a695c0cf2 OvmfPkg: Apply uncrusitify changes
> 303c0a91ab07 NetworkPkg: Apply uncrusitify changes
> bc80792cd1b1 MdePkg: Apply uncrusitify changes
> 3ea86be17a2a MdeModulePkg: Apply uncrusitify changes
> c70ef11ed0cd IntelFsp2WrapperPkg: Apply uncrusitify changes
> c0291221f252 IntelFsp2Pkg: Apply uncrusitify changes
> 6a479952a690 FmpDevicePkg: Apply uncrusitify changes
> 3a7c05b7070d FatPkg: Apply uncrusitify changes
> b789f98c8959 EmulatorPkg: Apply uncrusitify changes
> 952d7a1c9220 EmbeddedPkg: Apply uncrusitify changes
> a1cc9881bab6 DynamicTablesPkg: Apply uncrusitify changes
> 50654dfe5785 CryptoPkg: Apply uncrusitify changes
> ed965a02dfa1 ArmVirtPkg: Apply uncrusitify changes
> 9744023fbc46 ArmPlatformPkg: Apply uncrusitify changes
> 7a1cde5f5bba ArmPkg: Apply uncrusitify changes
> 19d17e0913e8 UefiCpuPkg: Change use of EFI_D_* to DEBUG_*
> ffa718b4f994 SourceLevelDebugPkg: Change use of EFI_D_* to DEBUG_*
> b86cb3c5e5b4 ShellPkg: Change use of EFI_D_* to DEBUG_*
> c7c42204dc07 SecurityPkg: Change use of EFI_D_* to DEBUG_*
> 16b8e6f958e4 PcAtChipsetPkg: Change use of EFI_D_* to DEBUG_*
> 0ac3f8b2dac5 OvmfPkg: Change use of EFI_D_* to DEBUG_*
> bc5004b8d294 NetworkPkg: Change use of EFI_D_* to DEBUG_*
> 6f671a8e2377 MdePkg: Change use of EFI_D_* to DEBUG_*
> a10c610ff9a3 MdeModulePkg: Change use of EFI_D_* to DEBUG_*
> 09a3bddba390 FatPkg: Change use of EFI_D_* to DEBUG_*
> 59c61318246a EmulatorPkg: Change use of EFI_D_* to DEBUG_*
> 3a80367dda3b EmbeddedPkg: Change use of EFI_D_* to DEBUG_*
> 23eb1aaf80ca ArmVirtPkg: Change use of EFI_D_* to DEBUG_*
> 875914b45c54 ArmPlatformPkg: Change use of EFI_D_* to DEBUG_*
> eb2eca82b451 ArmPkg: Change use of EFI_D_* to DEBUG_*
> f0f3f5aae7c4 (origin/master, origin/HEAD, master) UnitTestFrameworkPkg: Update YAML to ignore specific ECC files/errors
> c05734797790 UefiPayloadPkg: Update YAML to ignore specific ECC files/errors
> c30c40d6c63d StandaloneMmPkg: Update YAML to ignore specific ECC files/errors
> 9944508e85f1 ShellPkg: Update YAML to ignore specific ECC files/errors
> 60fa40be458d SecurityPkg: Update YAML to ignore specific ECC files/errors
> df790cd6b37e MdePkg: Update YAML to ignore specific ECC files/errors
> 9deb9370766e MdeModulePkg: Update YAML to ignore specific ECC files/errors
> d7d30e8f219f EmulatorPkg: Update YAML to ignore specific ECC files/errors
> d5744ecba813 CryptoPkg: Update YAML to ignore specific ECC files/errors
> c97fee87f0f9 ArmVirtPkg: Update YAML to ignore specific ECC files/errors
> 1939fc9569f2 ArmPlatformPkg: Update YAML to ignore specific ECC files/errors
> 365dced2c37a ArmPkg: Update YAML to ignore specific ECC files/errors
> 76a1ce4d5fec .azurepipelines/templates: Update max pipeline job time to 2 hours
> 99f84ff47390 .pytools/Plugin/LicenseCheck: Use temp directory for git diff output
> 3019f1bbabf1 .pytool/Plugin/EccCheck: Add performance optimizations
> 854462bd3479 .pytool/Plugin/EccCheck: Remove temp directory on exception
> 69877614fdee .pytool/Plugin/EccCheck: Remove RevertCode()
> --ref1
> ef9a059cdb15 EmulatorPkg/Win/Host: Update CC_FLAGS
> bb1bba3d7767 (tag: edk2-stable202111) NetworkPkg: Fix invalid pointer for DNS response token on error
>
> Best regards,
>
> Mike
>
>
>
>
>
>
>
>
>
>
[-- Attachment #2.1: Type: text/html, Size: 33764 bytes --]
[-- Attachment #2.2: npn4fyUZWeq1j4xo.png --]
[-- Type: image/png, Size: 17255 bytes --]
[-- Attachment #2.3: QsFtWMoACvKgJUpN.png --]
[-- Type: image/png, Size: 20870 bytes --]
[-- Attachment #2.4: 5KBDFgOi0e09cJV6.png --]
[-- Type: image/png, Size: 15290 bytes --]
[-- Attachment #2.5: WaF2lUMqi2bfQd8W.png --]
[-- Type: image/png, Size: 21357 bytes --]
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
2021-12-02 18:27 ` [edk2-devel] " Maciej Rabeda
@ 2021-12-02 19:45 ` Michael D Kinney
2021-12-02 21:56 ` Michael Kubacki
0 siblings, 1 reply; 39+ messages in thread
From: Michael D Kinney @ 2021-12-02 19:45 UTC (permalink / raw)
To: devel@edk2.groups.io, maciej.rabeda@linux.intel.com,
Michael Kubacki, Andrew Fish (afish@apple.com), Leif Lindholm,
Kinney, Michael D
[-- Attachment #1.1: Type: text/plain, Size: 23485 bytes --]
Hi Maciej,
Thanks for the feedback.
* Example #1. This appears to be caused by the following uncrustify settings:
# The threshold for aligning on '=' in assignments.
# Use a negative number for absolute thresholds.
#
# 0: No limit (default).
align_assign_thresh = 0 # number
The edk2 setting for this is:
align_assign_thresh = 4
This means blocks of assignments that are different than more than 4 spaces will be considered a new block.
‘HwAddreLen’ and ‘Xid’ are more than 4 characters in length different. Same for ‘Xid’ and ‘Reserved’. So
uncrustify treats these as 3 different assignment blocks.
If we change to the default value of 0: No limit, this example is treated as a single block and all ‘=’ are aligned.
Is there a reason ‘4’ was selected? Is there is any harm in using the default of 0?
* Example #2: Uncruistfy is confused by the DEBUG_CODE() macro. This is not a traditional macro because the
contents of the macro is a block of C code. I do not know how to convince uncrustify that the contents of a
macro like function call to be treated as a code block from an indent perspective.
* Example #3/#4: Uncrustify is confused by the OPTIONAL keyword. The edk2 config declares it as a QUALIFIER
like IN and OUT. However, OPTIONAL only appears at the end of the line that declares a parameter to a
function. There are 3 forms. One with comma after OPTIONAL. One with comma before OPTIONAL, and
one with no comma if the parameter is the last parameter in the function declaration.
TYPE ParamName OPTIONAL,
TYPE ParamName, OPTIONAL
TYPE ParamName OPTIONAL
OPTIONAL is defined to nothing in edk2 builds. From a uncrustify perspective, we really want is to be
ignored or more correctly treated it as a token that is attached to ParamName and the combination of
ParamName and OPTIONAL treated as one unit to determine indents.
Thanks,
Mike
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Maciej Rabeda
Sent: Thursday, December 2, 2021 10:27 AM
To: devel@edk2.groups.io; Kinney, Michael D <michael.d.kinney@intel.com>; Michael Kubacki <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com) <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>
Subject: Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
Hey Mike,
While most of the changes related to fixing coding style violations, there are a couple of changes to NetworkPkg in that PR that make the code less readable. Examples below.
Example 1:
[cid:image001.png@01D7E76D.5A618130]
[cid:image001.png@01D7E76D.5A618130]
Example 2:
[cid:image002.png@01D7E76D.5A618130]
Example 3:
[cid:image003.png@01D7E76D.5A618130]
Example 4:
[cid:image004.png@01D7E76D.5A618130]
On 30-Nov-21 23:34, Michael D Kinney wrote:
Hello,
Thank you for your patience during this extended hard freeze.
Just one more step to go. There has been a delay in the review of
the patch series with the uncrustify source changes. PR(6). This
patch series was not sent out as patch review email because of its
very large size. It only contains source style changes and the
CompareBuild tool and GitHub action has shown there are no binary
differences introduced with these source style changes.
If you are a package maintainer, then please review the following
branch/PR for your package contents and review the EDK II CI results
and BuildCompare results. I do not expect a line by line review
because we already had time to provide feedback on the source style
performed by uncrustify. Instead, a Reviewed-by for your package
indicates that you have reviewed the EDK II CI results and CompareBuild
tool functionality and results and you accept the source style
changes to your package.
* https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
* https://github.com/tianocore/edk2/pull/2229
* https://github.com/mdkinney/edk2/actions/runs/1521618836
Additional details on this update below.
Thank you,
Mike
Changes from Update #3
----------------------------------------------------------------------------
* Pushed PR (5)
* Added link to PR(6). EDK II CI Status is PASS. Build Compare PASS.
* Waiting for review of PR (6)
* Review of PR (7) completed and waiting for review of PR (6)
----------------------------------------------------------------------------
Changes from Update #2
----------------------------------------------------------------------------
* Changed order of PRs swapping (4) and (5). The PR that activates
increases the max CI agent job time is independent of all the other
PRs and its review is complete, so it can be committed now.
* Pushed PRs (1), (2), (3), (4).
* Waiting for review to complete for PRs (5) and (6)
* Reviews complete for PR (7)
* Identifies steps using git filter-branch to apply uncrustify changes to a
code review patch series that was generated before the uncrustify changes
avoiding manual merge.
* Identified steps using git filter-repo to generate an alternate history of
the edk2 repo with uncrustify changes applied on every commit. This may
be useful when evaluating changes to files using tools like git blame
without the large uncrustify patch series.
---------------------------------------------------------------------------
Changes from Update #1
----------------------------------------------------------------------------
* Changed order of PRs swapping (6) and (7). The PR that activates
EDK II CI check UncrustifyCheck has to be last because it unconditionally
checks all C/H files in all packages. Not just files that have been
modified like some of the other checkers.
* Updated link to the branch with the UncrustifyCheck plugin that has been
updated with a one line change and Reviewed-by and Tested-by tags.
https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v6
* Reviews complete for (1), (2), (3), (5), and (7)
---------------------------------------------------------------------------
Michael Kubacki and I have prepared the patches required to apply the
uncrustify changes and enable EDK II CI to check all submitted
patches have been run through uncrustify.
We have verified through the CompareBuild GitHub Action that the
format changes performed by uncrustify have no functional changes.
All of the OBJ, LIB, DLL, EFI, FFS, FV, and FD files match 100%
across 70 VS2019/GCC5 builds of all package/platform DSC files in
the edk2 repo.
The hard freeze will be extended after the edk2-stable202111 tag until
all uncrustify related changes are committed. We do not expect this
to take more than a few days. Do not push any PRs until the hard
freeze is lifted.
The changes are broken up into 7 patch series/PRs. The PRs are ordered
so they can be submitted using the normal submission process and EDK II
CI will pass for each one. Details are listed below.
Uncrustify 73.0.3 for EDK II
=============================
* Sources: https://dev.azure.com/projectmu/_git/Uncrustify
* Documentation: https://dev.azure.com/projectmu/Uncrustify/_wiki/wikis/Uncrustify.wiki/1/Project-Mu-(EDK-II)-Fork-Readme
* Download: https://dev.azure.com/projectmu/Uncrustify/_packaging?_a=package&feed=mu_uncrustify&package=mu-uncrustify-release&protocolType=NuGet&version=73.0.3
Installing Uncrustify
======================
The Uncrustify tool is installed automatically when the Pytools
environment is used and the stuart* commands are run to complete the
environment setup. Please see:
https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally
Uncrustify can also be installed from the download page listed above
or built from sources from the source link above.
The Documentation link provides instruction on how to run uncrustify from
the command line or install as a Visual Studio Code plugin. The main
uncrustify documentation also describes how to integrate with a few other
editors.
We have also discussed a client side githook. That effort has not started.
Let us know if that is a feature you would find useful.
Developer impact for new code reviews
======================================
Once the uncrustify checker is active in EDK II CI, developers must
make sure their patches are run through the uncrustify tool before
sending the patches for review.
Developers must install and run uncrustify against changes files before
sending patch review emails or submitting PR for EDK II CI. If EDK II CI
detects and differences in source formatting, then EDK II CI will fail
and the developer must run uncrustify and resubmit the patches.
Developer impact to patch series/PRs reviewed during edk2-stable201121 soft/hard freeze
=======================================================================================
Developers must rebase their changes after the uncrustify source changes are
committed. The branch with a preview of the uncrustify changes can be used
to start this rebase work.
https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
The following steps can be used to update an existing branch with the
required uncrustify format. This is the Windows version. I will add
the Linux version soon.
1) Fetch and checkout and rebase to latest edk2/master
git fetch origin
git checkout master
git rebase origin/master
2) Make a backup copy of plugin UncrustifyCheck outside WORKSPACE.
(e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable and
EDK II specific uncrustify configuration file available when working
with a branch that does not have those tools in its scope.
xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
3) Check out the patch series branch (e.g. MyBranch)
git checkout MyBranch
4) Rebase patch series against edk2-stable202111
git rebase edk2-stable202111
5) Create new branch for the uncrustifed version (e.g. MyBranch_Uncrustified)
git checkout -b MyBranch_Uncrustified
6) Use git filter-branch to uncrustify all the commits in the series
between the rebase target from (2) and HEAD of the branch. A filter
can be used to scope the uncrustify operations to only the C/H files
in the specific package the patch series is against. (e.g. DynamicTablesPkg).
BaseTools should always be excluded. If the package scoped filter is
not used, it will still work, but will take longer to run because
uncrustify will rescan every C/H files in the whole repo.
git filter-branch --tree-filter "git ls-files DynamicTablesPkg*.c DynamicTablesPkg*.h :!BaseTools/* | c:\\Temp\\UncrustifyCheck\\mu-uncrustify-release_extdep\\Windows-x86\\uncrustify.exe -c c:\\Temp\\UncrustifyCheck\\uncrustify.cfg -F - --replace --no-backup --if-changed" edk2-stable202111..HEAD
7) Now that all the individual patches in the branch are uncrustified,
rebase against latest edk2/master that is already uncrustified.
git rebase master
8) Verify the patches in this new branch.
Impacts to tracing history across the uncrusity changes
=======================================================
Tools the view file and line history do work with the large uncrustify
patch series. One impact is that the operations can be very slow due
to the large uncrustify patches.
One option to provide a faster experience is to provide an alternate
version of the edk2 repository as "documentation" that has the
entire history re-written with uncrustify run on every commit.
The tool called git-filter-repo can be used to perform this
transformation and runs in a reasonable period of time (a few hours)
https://github.com/newren/git-filter-repo
https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-demos/lint-history
The following steps can be used to perform this transformation.
This is the Windows version. I will add the Linux version soon.
** WARNING ** This operation modifies(rewrites) all the commits
in the local copy of the repo. Do not perform
these steps on a local repo you are using for
active development.
1) Clone edk2 into a new directory (see **WARNING**)
git clone https://github.com/tianocore/edk2.git edk2-uncrustified
cd edk2-uncrustified
2) Setup python virtual env, install pytools, and run stuart commands
to setup build environment which includes installing uncrustify tools.
https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally
3) Make a backup copy of plugin UncrustifyCheck outside WORKSPACE.
(e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable and
EDK II specific uncrustify configuration file available when working
with a branch that does not have those tools in its scope.
xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
4) Use lint-history.py from git-filter-repo examples
https://github.com/newren/git-filter-repo
https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-demos/lint-history
Line #127 - Add try except around subprocess.check_call() with except
being pass. This is required because there are a few commits of C
files in the edk2 repo that have incorrect C syntax and do not
build with a C compiler and break the uncrustify parser. Skip reformat
of C files that can not be parsed by uncrustify. These rare instances
are addressed in the commit that fixes the C syntax error.
Run this slightly modified version of lint-history. Include only
C/H files and exclude directories that start with 'Tools' or 'BaseTools'.
This step took about 2.2 hours on a laptop.
lint-history.py
--relevant "return (not filename.startswith(b'Tools') and not filename.startswith(b'BaseTools') and (filename.endswith(b'.c') or filename.endswith(b'.h')))"
c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\mu-uncrustify-release_extdep\\Windows-x86\\uncrustify.exe -c c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\uncrustify.cfg --replace --no-backup --if-changed
Order of PRs to apply during extended hard freeze
==================================================
1) Update EmulatorPkg Win Host [BuildOptions] MSFT CC_FLAGS to not force debug information
* https://bugzilla.tianocore.org/show_bug.cgi?id=3747
* https://github.com/mdkinney/edk2/tree/Bug_3747_EmulatorPkg_WinHost_ReproducibleBuild
* https://github.com/tianocore/edk2/pull/2215
* Required for EmulatorPkg to pass CompareBuild for VS2019 IA32/X64 builds.
* Status: Review complete. PR pushed.
2) EccCheck should not revert staged and local changes
* https://bugzilla.tianocore.org/show_bug.cgi?id=2986
* https://github.com/mdkinney/edk2/tree/Bug_2986_EccCheckRemoveGitRevert_V2
* https://github.com/tianocore/edk2/pull/2216
* Required for EDK II CI to complete in a reasonable period of time when
processing the 4000+ source file style changes made by uncrustify.
* Also fixes critical bugs that can potentially corrupt git state when
EccCheck is run locally.
* Status: Review complete. PR pushed.
3) Update pytool LicenseCheck plugin to use temp directory for diff output file
* https://bugzilla.tianocore.org/show_bug.cgi?id=3746
* https://github.com/mdkinney/edk2/tree/Bug_3746_LicenseCheckUseDiffOutputFile_V2
* https://github.com/tianocore/edk2/pull/2217
* Required to reduce EDK II CI build times.
* Status: Review complete. PR pushed.
4) Update max job time from 60 min to 120 minutes in .azurepipelines/templates
* https://bugzilla.tianocore.org/show_bug.cgi?id=3750
* https://github.com/mdkinney/edk2/tree/Bug_3750_IncreaseAzurePipelinesTimeout
* https://github.com/tianocore/edk2/pull/2219
* Required to allow EccCheck of uncrustify changes to complete on Azure
Pipelines CI agents without timing out.
* Status: Review complete. PR pushed.
5) Update Package YAML to ignore specific ECC files/errors
* https://bugzilla.tianocore.org/show_bug.cgi?id=3749
* https://github.com/mdkinney/edk2/tree/Bug_3749_EccCheckIgnoreFilesErrors
* https://github.com/tianocore/edk2/pull/2218
* Required to pass EccCheck
* Status: Review complete. PR pushed
6) Uncrustify Source Changes
* https://bugzilla.tianocore.org/show_bug.cgi?id=3737
* https://bugzilla.tianocore.org/show_bug.cgi?id=3739
* https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
* https://github.com/tianocore/edk2/pull/2229
* Build comparison result PASS: https://github.com/mdkinney/edk2/actions/runs/1521618836
* EFI_D_ -> DEBUG changes required to pass PatchCheck
* Uncrustify format changes required to pass UncrustifyCheck
* Status:
Waiting for review
7) UncrustifyCheck EDK II CI Plugin
* https://bugzilla.tianocore.org/show_bug.cgi?id=3748
* https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v6
* Required to enforce all PRs submitted to EDK II CI match uncrustify format.
* Unconditionally checks all packages. Can not be committed until all C/H
source files have been updated.
* Status: Review complete
Combined Branch/PR for Review/Test
==================================
* Build Comparison results must pass 100% across the full set of PRs before
the individual PRs can be pushed in the order listed above.
* Branch: https://github.com/mdkinney/edk2/tree/TestOnly_Uncrustify_PR_Series
* PR: https://github.com/tianocore/edk2/pull/2229
Status = PASS
* CompareBuild:
Branch: https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
--ref1: ef9a059cdb15844fe52a49af2bf7d86b9dd3e9bf
--ref2: Bug_3737_3739_ApplyUncrustifyChanges_V5
Extra Options: -n 4 --quiet
Results: https://github.com/mdkinney/edk2/actions/runs/1521618836
30 VS2019 build comparisons PASS
40 GCC5 build comparisons PASS
100% PASS
The following git log shows the set of patches from --ref1 to --ref 2across
which there are no differences in any of the OBJ/LIB/DLL/EFI/FFS/FV/FD files.
--ref2
b7d4bf0675b7 (HEAD -> Bug_3737_3739_ApplyUncrustifyChanges_V5) UnitTestFrameworkPkg: Apply uncrusitify changes
7f03d25f60e7 UefiPayloadPkg: Apply uncrusitify changes
0bfd8d9b5ac9 UefiCpuPkg: Apply uncrusitify changes
e1cd9bfb9dea StandaloneMmPkg: Apply uncrusitify changes
5da2f65be378 SourceLevelDebugPkg: Apply uncrusitify changes
95b86de07e5d SignedCapsulePkg: Apply uncrusitify changes
fe71d97246c4 ShellPkg: Apply uncrusitify changes
54c21c952992 SecurityPkg: Apply uncrusitify changes
187a3785f12b RedfishPkg: Apply uncrusitify changes
810100002a46 PcAtChipsetPkg: Apply uncrusitify changes
276a695c0cf2 OvmfPkg: Apply uncrusitify changes
303c0a91ab07 NetworkPkg: Apply uncrusitify changes
bc80792cd1b1 MdePkg: Apply uncrusitify changes
3ea86be17a2a MdeModulePkg: Apply uncrusitify changes
c70ef11ed0cd IntelFsp2WrapperPkg: Apply uncrusitify changes
c0291221f252 IntelFsp2Pkg: Apply uncrusitify changes
6a479952a690 FmpDevicePkg: Apply uncrusitify changes
3a7c05b7070d FatPkg: Apply uncrusitify changes
b789f98c8959 EmulatorPkg: Apply uncrusitify changes
952d7a1c9220 EmbeddedPkg: Apply uncrusitify changes
a1cc9881bab6 DynamicTablesPkg: Apply uncrusitify changes
50654dfe5785 CryptoPkg: Apply uncrusitify changes
ed965a02dfa1 ArmVirtPkg: Apply uncrusitify changes
9744023fbc46 ArmPlatformPkg: Apply uncrusitify changes
7a1cde5f5bba ArmPkg: Apply uncrusitify changes
19d17e0913e8 UefiCpuPkg: Change use of EFI_D_* to DEBUG_*
ffa718b4f994 SourceLevelDebugPkg: Change use of EFI_D_* to DEBUG_*
b86cb3c5e5b4 ShellPkg: Change use of EFI_D_* to DEBUG_*
c7c42204dc07 SecurityPkg: Change use of EFI_D_* to DEBUG_*
16b8e6f958e4 PcAtChipsetPkg: Change use of EFI_D_* to DEBUG_*
0ac3f8b2dac5 OvmfPkg: Change use of EFI_D_* to DEBUG_*
bc5004b8d294 NetworkPkg: Change use of EFI_D_* to DEBUG_*
6f671a8e2377 MdePkg: Change use of EFI_D_* to DEBUG_*
a10c610ff9a3 MdeModulePkg: Change use of EFI_D_* to DEBUG_*
09a3bddba390 FatPkg: Change use of EFI_D_* to DEBUG_*
59c61318246a EmulatorPkg: Change use of EFI_D_* to DEBUG_*
3a80367dda3b EmbeddedPkg: Change use of EFI_D_* to DEBUG_*
23eb1aaf80ca ArmVirtPkg: Change use of EFI_D_* to DEBUG_*
875914b45c54 ArmPlatformPkg: Change use of EFI_D_* to DEBUG_*
eb2eca82b451 ArmPkg: Change use of EFI_D_* to DEBUG_*
f0f3f5aae7c4 (origin/master, origin/HEAD, master) UnitTestFrameworkPkg: Update YAML to ignore specific ECC files/errors
c05734797790 UefiPayloadPkg: Update YAML to ignore specific ECC files/errors
c30c40d6c63d StandaloneMmPkg: Update YAML to ignore specific ECC files/errors
9944508e85f1 ShellPkg: Update YAML to ignore specific ECC files/errors
60fa40be458d SecurityPkg: Update YAML to ignore specific ECC files/errors
df790cd6b37e MdePkg: Update YAML to ignore specific ECC files/errors
9deb9370766e MdeModulePkg: Update YAML to ignore specific ECC files/errors
d7d30e8f219f EmulatorPkg: Update YAML to ignore specific ECC files/errors
d5744ecba813 CryptoPkg: Update YAML to ignore specific ECC files/errors
c97fee87f0f9 ArmVirtPkg: Update YAML to ignore specific ECC files/errors
1939fc9569f2 ArmPlatformPkg: Update YAML to ignore specific ECC files/errors
365dced2c37a ArmPkg: Update YAML to ignore specific ECC files/errors
76a1ce4d5fec .azurepipelines/templates: Update max pipeline job time to 2 hours
99f84ff47390 .pytools/Plugin/LicenseCheck: Use temp directory for git diff output
3019f1bbabf1 .pytool/Plugin/EccCheck: Add performance optimizations
854462bd3479 .pytool/Plugin/EccCheck: Remove temp directory on exception
69877614fdee .pytool/Plugin/EccCheck: Remove RevertCode()
--ref1
ef9a059cdb15 EmulatorPkg/Win/Host: Update CC_FLAGS
bb1bba3d7767 (tag: edk2-stable202111) NetworkPkg: Fix invalid pointer for DNS response token on error
Best regards,
Mike
[-- Attachment #1.2: Type: text/html, Size: 155826 bytes --]
[-- Attachment #2: image001.png --]
[-- Type: image/png, Size: 17255 bytes --]
[-- Attachment #3: image002.png --]
[-- Type: image/png, Size: 20870 bytes --]
[-- Attachment #4: image003.png --]
[-- Type: image/png, Size: 15290 bytes --]
[-- Attachment #5: image004.png --]
[-- Type: image/png, Size: 21357 bytes --]
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
2021-12-02 19:45 ` Michael D Kinney
@ 2021-12-02 21:56 ` Michael Kubacki
2021-12-03 0:14 ` Michael D Kinney
0 siblings, 1 reply; 39+ messages in thread
From: Michael Kubacki @ 2021-12-02 21:56 UTC (permalink / raw)
To: devel, michael.d.kinney, maciej.rabeda@linux.intel.com,
Michael Kubacki, Andrew Fish (afish@apple.com), Leif Lindholm
My reply is inline.
Regards,
Michael
On 12/2/2021 2:45 PM, Michael D Kinney wrote:
> Hi Maciej,
>
> Thanks for the feedback.
>
> * Example #1.This appears to be caused by the following uncrustify
> settings:
>
> # The threshold for aligning on '=' in assignments.
>
> # Use a negative number for absolute thresholds.
>
> #
>
> # 0: No limit (default).
>
> align_assign_thresh= 0# number
>
> The edk2 setting for this is:
>
> align_assign_thresh= 4
>
> This means blocks of assignments that are different than more than 4
> spaces will be considered a new block.
>
> ‘HwAddreLen’ and ‘Xid’ are more than 4 characters in length
> different.Same for ‘Xid’ and ‘Reserved’.So
>
> uncrustify treats these as 3 different assignment blocks.
>
> If we change to the default value of 0: No limit, this example is
> treated as a single block and all ‘=’ are aligned.
>
> Is there a reason ‘4’ was selected?Is there is any harm in using the
> default of 0?
>
We can certainly change the threshold. '4' was derived by
experimentation. This is an area that is somewhat subjective and every
case is difficult to cover well. Please feel free to suggest any changes.
For the benefit of others, this file can be a useful refernce
understanding spans, gaps, and thresholds.
https://github.com/uncrustify/uncrustify/blob/master/documentation/htdocs/configuration.txt
> * Example #2: Uncruistfy is confused by the DEBUG_CODE() macro.This is
> not a traditional macro because the
>
> contents of the macro is a block of C code.I do not know how to convince
> uncrustify that the contents of a
>
> macro like function call to be treated as a code block from an indent
> perspective.
>
I believe this would have to be overridden in the Uncrustify fork since
DEBUG_CODE() is being treated as a macro function call and code blocks
are not expected there. In addition, some special treatment might be
needed for alignment in between DEBUG_CODE_BEGIN()/DEBUG_CODE_END().
I'm happy to look at this. However, regression validation can take a
while so I'd like to make sure we need this now. In cases that do not
have additional code blocks, it seems to format fairly well. Is this
prevalent and impactful enough it must be fixed now? Or, could we
revisit it with a follow up patch?
> * Example #3/#4: Uncrustify is confused by the OPTIONAL keyword.The
> edk2 config declares it as a QUALIFIER
>
> like IN and OUT.However, OPTIONAL only appears at the end of the line
> that declares a parameter to a
>
> function.There are 3 forms. One with comma after OPTIONAL.One with comma
> before OPTIONAL, and
>
> one with no comma if the parameter is the last parameter in the function
> declaration.
>
> TYPE ParamName OPTIONAL,
>
> TYPEParamName, OPTIONAL
>
> TYPEParamName OPTIONAL
>
> OPTIONAL is defined to nothing in edk2 builds.From a uncrustify
> perspective, we really want is to be
>
> ignored or more correctly treated it as a token that is attached to
> ParamName and the combination of
>
> ParamName and OPTIONAL treated as one unit to determine indents.
>
"TYPE ParamName, OPTIONAL" seems especially problematic and presents an
inconsistency with the other formats. Mike, can you please let me know
if you have the same observation and your thoughts on a consistent pattern?
> Thanks,
>
> Mike
>
> *From:*devel@edk2.groups.io <devel@edk2.groups.io> *On Behalf Of *Maciej
> Rabeda
> *Sent:* Thursday, December 2, 2021 10:27 AM
> *To:* devel@edk2.groups.io; Kinney, Michael D
> <michael.d.kinney@intel.com>; Michael Kubacki
> <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com)
> <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>
> *Subject:* Re: [edk2-devel] Uncrustify Conversion Detailed Plan and
> Extended Hard Freeze Update #4
>
> Hey Mike,
>
> While most of the changes related to fixing coding style violations,
> there are a couple of changes to NetworkPkg in that PR that make the
> code less readable. Examples below.
>
> Example 1:
>
>
>
> Example 2:
>
>
> Example 3:
>
>
> Example 4:
>
> On 30-Nov-21 23:34, Michael D Kinney wrote:
>
> Hello,
>
> Thank you for your patience during this extended hard freeze.
>
> Just one more step to go.There has been a delay in the review of
>
> the patch series with the uncrustify source changes.PR(6).This
>
> patch series was not sent out as patch review email because of its
>
> very large size.It only contains source style changes and the
>
> CompareBuild tool and GitHub action has shown there are no binary
>
> differences introduced with these source style changes.
>
> If you are a package maintainer, then please review the following
>
> branch/PR for your package contents and review the EDK II CI results
>
> and BuildCompare results.I do not expect a line by line review
>
> because we already had time to provide feedback on the source style
>
> performed by uncrustify.Instead, a Reviewed-by for your package
>
> indicates that you have reviewed the EDK II CI results and CompareBuild
>
> tool functionality and results and you accept the source style
>
> changes to your package.
>
> *https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5 <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5>
>
> *https://github.com/tianocore/edk2/pull/2229 <https://github.com/tianocore/edk2/pull/2229>
>
> *https://github.com/mdkinney/edk2/actions/runs/1521618836 <https://github.com/mdkinney/edk2/actions/runs/1521618836>
>
> Additional details on this update below.
>
> Thank you,
>
> Mike
>
> Changes from Update #3
>
> ----------------------------------------------------------------------------
>
> * Pushed PR (5)
>
> * Added link to PR(6). EDK II CI Status is PASS. Build Compare PASS.
>
> * Waiting for review of PR (6)
>
> * Review of PR (7) completed and waiting for review of PR (6)
>
> ----------------------------------------------------------------------------
>
> Changes from Update #2
>
> ----------------------------------------------------------------------------
>
> * Changed order of PRs swapping (4) and (5).The PR that activates
>
> increases the max CI agent job time is independent of all the other
>
> PRs and its review is complete, so it can be committed now.
>
> * Pushed PRs (1), (2), (3), (4).
>
> * Waiting for review to complete for PRs (5) and (6)
>
> * Reviews complete for PR (7)
>
> * Identifies steps using git filter-branch to apply uncrustify changes to a
>
> code review patch series that was generated before the uncrustify changes
>
> avoiding manual merge.
>
> * Identified steps using git filter-repo to generate an alternate history of
>
> the edk2 repo with uncrustify changes applied on every commit.This may
>
> be useful when evaluating changes to files using tools like git blame
>
> without the large uncrustify patch series.
>
> ---------------------------------------------------------------------------
>
> Changes from Update #1
>
> ----------------------------------------------------------------------------
>
> * Changed order of PRs swapping (6) and (7).The PR that activates
>
> EDK II CI check UncrustifyCheck has to be last because it unconditionally
>
> checks all C/H files in all packages.Not just files that have been
>
> modified like some of the other checkers.
>
> * Updated link to the branch with the UncrustifyCheck plugin that has been
>
> updated with a one line change and Reviewed-by and Tested-by tags.
>
> https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v6 <https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v6>
>
> * Reviews complete for (1), (2), (3), (5), and (7)
>
> ---------------------------------------------------------------------------
>
> Michael Kubacki and I have prepared the patches required to apply the
>
> uncrustify changes and enable EDK II CI to check all submitted
>
> patches have been run through uncrustify.
>
> We have verified through the CompareBuild GitHub Action that the
>
> format changes performed by uncrustify have no functional changes.
>
> All of the OBJ, LIB, DLL, EFI, FFS, FV, and FD files match 100%
>
> across 70 VS2019/GCC5 builds of all package/platform DSC files in
>
> the edk2 repo.
>
> The hard freeze will be extended after the edk2-stable202111 tag until
>
> all uncrustify related changes are committed.We do not expect this
>
> to take more than a few days.Do not push any PRs until the hard
>
> freeze is lifted.
>
> The changes are broken up into 7 patch series/PRs.The PRs are ordered
>
> so they can be submitted using the normal submission process and EDK II
>
> CI will pass for each one.Details are listed below.
>
> Uncrustify 73.0.3 for EDK II
>
> =============================
>
> * Sources:https://dev.azure.com/projectmu/_git/Uncrustify <https://dev.azure.com/projectmu/_git/Uncrustify>
>
> * Documentation:https://dev.azure.com/projectmu/Uncrustify/_wiki/wikis/Uncrustify.wiki/1/Project-Mu-(EDK-II)-Fork-Readme <https://dev.azure.com/projectmu/Uncrustify/_wiki/wikis/Uncrustify.wiki/1/Project-Mu-(EDK-II)-Fork-Readme>
>
> * Download:https://dev.azure.com/projectmu/Uncrustify/_packaging?_a=package&feed=mu_uncrustify&package=mu-uncrustify-release&protocolType=NuGet&version=73.0.3 <https://dev.azure.com/projectmu/Uncrustify/_packaging?_a=package&feed=mu_uncrustify&package=mu-uncrustify-release&protocolType=NuGet&version=73.0.3>
>
> Installing Uncrustify
>
> ======================
>
> The Uncrustify tool is installed automatically when the Pytools
>
> environment is used and the stuart* commands are run to complete the
>
> environment setup.Please see:
>
> https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally <https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally>
>
> Uncrustify can also be installed from the download page listed above
>
> or built from sources from the source link above.
>
> The Documentation link provides instruction on how to run uncrustify from
>
> the command line or install as a Visual Studio Code plugin.The main
>
> uncrustify documentation also describes how to integrate with a few other
>
> editors.
>
> We have also discussed a client side githook.That effort has not started.
>
> Let us know if that is a feature you would find useful.
>
> Developer impact for new code reviews
>
> ======================================
>
> Once the uncrustify checker is active in EDK II CI, developers must
>
> make sure their patches are run through the uncrustify tool before
>
> sending the patches for review.
>
> Developers must install and run uncrustify against changes files before
>
> sending patch review emails or submitting PR for EDK II CI.If EDK II CI
>
> detects and differences in source formatting, then EDK II CI will fail
>
> and the developer must run uncrustify and resubmit the patches.
>
> Developer impact to patch series/PRs reviewed during edk2-stable201121 soft/hard freeze
>
> =======================================================================================
>
> Developers must rebase their changes after the uncrustify source changes are
>
> committed.The branch with a preview of the uncrustify changes can be used
>
> to start this rebase work.
>
> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5 <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5>
>
> The following steps can be used to update an existing branch with the
>
> required uncrustify format.This is the Windows version.I will add
>
> the Linux version soon.
>
> 1) Fetch and checkout and rebase to latest edk2/master
>
> git fetch origin
>
> git checkout master
>
> git rebase origin/master
>
> 2) Make a backup copy of plugin UncrustifyCheck outside WORKSPACE.
>
> (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable and
>
> EDK II specific uncrustify configuration file available when working
>
> with a branch that does not have those tools in its scope.
>
> xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
>
> 3) Check out the patch series branch (e.g. MyBranch)
>
> git checkout MyBranch
>
> 4) Rebase patch series against edk2-stable202111
>
>
>
> git rebase edk2-stable202111
>
> 5) Create new branch for the uncrustifed version (e.g. MyBranch_Uncrustified)
>
> git checkout -b MyBranch_Uncrustified
>
> 6) Use git filter-branch to uncrustify all the commits in the series
>
> between the rebase target from (2) and HEAD of the branch.A filter
>
> can be used to scope the uncrustify operations to only the C/H files
>
> in the specific package the patch series is against. (e.g. DynamicTablesPkg).
>
> BaseTools should always be excluded.If the package scoped filter is
>
> not used, it will still work, but will take longer to run because
>
> uncrustify will rescan every C/H files in the whole repo.
>
> git filter-branch --tree-filter "git ls-files DynamicTablesPkg*.c DynamicTablesPkg*.h :!BaseTools/* | c:\\Temp\\UncrustifyCheck\\mu-uncrustify-release_extdep\\Windows-x86\\uncrustify.exe -c c:\\Temp\\UncrustifyCheck\\uncrustify.cfg -F - --replace --no-backup --if-changed" edk2-stable202111..HEAD
>
> 7) Now that all the individual patches in the branch are uncrustified,
>
> rebase against latest edk2/master that is already uncrustified.
>
> git rebase master
>
> 8) Verify the patches in this new branch.
>
> Impacts to tracing history across the uncrusity changes
>
> =======================================================
>
> Tools the view file and line history do work with the large uncrustify
>
> patch series.One impact is that the operations can be very slow due
>
> to the large uncrustify patches.
>
> One option to provide a faster experience is to provide an alternate
>
> version of the edk2 repository as "documentation" that has the
>
> entire history re-written with uncrustify run on every commit.
>
> The tool called git-filter-repo can be used to perform this
>
> transformation and runs in a reasonable period of time (a few hours)
>
> https://github.com/newren/git-filter-repo <https://github.com/newren/git-filter-repo>
>
> https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-demos/lint-history <https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-demos/lint-history>
>
> The following steps can be used to perform this transformation.
>
> This is the Windows version. I will add the Linux version soon.
>
> ** WARNING **This operation modifies(rewrites) all the commits
>
> in the local copy of the repo.Do not perform
>
> these steps on a local repo you are using for
>
> active development.
>
> 1) Clone edk2 into a new directory (see **WARNING**)
>
> git clonehttps://github.com/tianocore/edk2.git <https://github.com/tianocore/edk2.git> edk2-uncrustified
>
> cd edk2-uncrustified
>
> 2) Setup python virtual env, install pytools, and run stuart commands
>
> to setup build environment which includes installing uncrustify tools.
>
> https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally <https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally>
>
> 3) Make a backup copy of plugin UncrustifyCheck outside WORKSPACE.
>
> (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable and
>
> EDK II specific uncrustify configuration file available when working
>
> with a branch that does not have those tools in its scope.
>
> xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
>
> 4) Use lint-history.py from git-filter-repo examples
>
> https://github.com/newren/git-filter-repo <https://github.com/newren/git-filter-repo>
>
> https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-demos/lint-history <https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-demos/lint-history>
>
> Line #127 - Add try except around subprocess.check_call() with except
>
> being pass.This is required because there are a few commits of C
>
> files in the edk2 repo that have incorrect C syntax and do not
>
> build with a C compiler and break the uncrustify parser.Skip reformat
>
> of C files that can not be parsed by uncrustify.These rare instances
>
> are addressed in the commit that fixes the C syntax error.
>
> Run this slightly modified version of lint-history.Include only
>
> C/H files and exclude directories that start with 'Tools' or 'BaseTools'.
>
> This step took about 2.2 hours on a laptop.
>
> lint-history.py
>
> --relevant "return (not filename.startswith(b'Tools') and not filename.startswith(b'BaseTools') and (filename.endswith(b'.c') or filename.endswith(b'.h')))"
>
> c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\mu-uncrustify-release_extdep\\Windows-x86\\uncrustify.exe -c c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\uncrustify.cfg --replace --no-backup --if-changed
>
> Order of PRs to apply during extended hard freeze
>
> ==================================================
>
> 1) Update EmulatorPkg Win Host [BuildOptions] MSFT CC_FLAGS to not force debug information
>
> *https://bugzilla.tianocore.org/show_bug.cgi?id=3747 <https://bugzilla.tianocore.org/show_bug.cgi?id=3747>
>
> *https://github.com/mdkinney/edk2/tree/Bug_3747_EmulatorPkg_WinHost_ReproducibleBuild <https://github.com/mdkinney/edk2/tree/Bug_3747_EmulatorPkg_WinHost_ReproducibleBuild>
>
> *https://github.com/tianocore/edk2/pull/2215 <https://github.com/tianocore/edk2/pull/2215>
>
> * Required for EmulatorPkg to pass CompareBuild for VS2019 IA32/X64 builds.
>
> * Status: Review complete.PR pushed.
>
> 2) EccCheck should not revert staged and local changes
>
> *https://bugzilla.tianocore.org/show_bug.cgi?id=2986 <https://bugzilla.tianocore.org/show_bug.cgi?id=2986>
>
> *https://github.com/mdkinney/edk2/tree/Bug_2986_EccCheckRemoveGitRevert_V2 <https://github.com/mdkinney/edk2/tree/Bug_2986_EccCheckRemoveGitRevert_V2>
>
> *https://github.com/tianocore/edk2/pull/2216 <https://github.com/tianocore/edk2/pull/2216>
>
> * Required for EDK II CI to complete in a reasonable period of time when
>
> processing the 4000+ source file style changes made by uncrustify.
>
> * Also fixes critical bugs that can potentially corrupt git state when
>
> EccCheck is run locally.
>
> * Status: Review complete.PR pushed.
>
> 3) Update pytool LicenseCheck plugin to use temp directory for diff output file
>
> *https://bugzilla.tianocore.org/show_bug.cgi?id=3746 <https://bugzilla.tianocore.org/show_bug.cgi?id=3746>
>
> *https://github.com/mdkinney/edk2/tree/Bug_3746_LicenseCheckUseDiffOutputFile_V2 <https://github.com/mdkinney/edk2/tree/Bug_3746_LicenseCheckUseDiffOutputFile_V2>
>
> *https://github.com/tianocore/edk2/pull/2217 <https://github.com/tianocore/edk2/pull/2217>
>
> * Required to reduce EDK II CI build times.
>
> * Status: Review complete.PR pushed.
>
> 4) Update max job time from 60 min to 120 minutes in .azurepipelines/templates
>
> *https://bugzilla.tianocore.org/show_bug.cgi?id=3750 <https://bugzilla.tianocore.org/show_bug.cgi?id=3750>
>
> *https://github.com/mdkinney/edk2/tree/Bug_3750_IncreaseAzurePipelinesTimeout <https://github.com/mdkinney/edk2/tree/Bug_3750_IncreaseAzurePipelinesTimeout>
>
> *https://github.com/tianocore/edk2/pull/2219 <https://github.com/tianocore/edk2/pull/2219>
>
> * Required to allow EccCheck of uncrustify changes to complete on Azure
>
> Pipelines CI agents without timing out.
>
> * Status: Review complete.PR pushed.
>
> 5) Update Package YAML to ignore specific ECC files/errors
>
> *https://bugzilla.tianocore.org/show_bug.cgi?id=3749 <https://bugzilla.tianocore.org/show_bug.cgi?id=3749>
>
> *https://github.com/mdkinney/edk2/tree/Bug_3749_EccCheckIgnoreFilesErrors <https://github.com/mdkinney/edk2/tree/Bug_3749_EccCheckIgnoreFilesErrors>
>
> *https://github.com/tianocore/edk2/pull/2218 <https://github.com/tianocore/edk2/pull/2218>
>
> * Required to pass EccCheck
>
> * Status: Review complete. PR pushed
>
> 6) Uncrustify Source Changes
>
> *https://bugzilla.tianocore.org/show_bug.cgi?id=3737 <https://bugzilla.tianocore.org/show_bug.cgi?id=3737>
>
> *https://bugzilla.tianocore.org/show_bug.cgi?id=3739 <https://bugzilla.tianocore.org/show_bug.cgi?id=3739>
>
> *https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5 <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5>
>
> *https://github.com/tianocore/edk2/pull/2229 <https://github.com/tianocore/edk2/pull/2229>
>
> * Build comparison result PASS:https://github.com/mdkinney/edk2/actions/runs/1521618836 <https://github.com/mdkinney/edk2/actions/runs/1521618836>
>
> * EFI_D_ -> DEBUG changes required to pass PatchCheck
>
> * Uncrustify format changes required to pass UncrustifyCheck
>
> * Status:
>
> Waiting for review
>
> 7) UncrustifyCheck EDK II CI Plugin
>
> *https://bugzilla.tianocore.org/show_bug.cgi?id=3748 <https://bugzilla.tianocore.org/show_bug.cgi?id=3748>
>
> *https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v6 <https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v6>
>
> * Required to enforce all PRs submitted to EDK II CI match uncrustify format.
>
> * Unconditionally checks all packages.Can not be committed until all C/H
>
> source files have been updated.
>
> * Status: Review complete
>
> Combined Branch/PR for Review/Test
>
> ==================================
>
> * Build Comparison results must pass 100% across the full set of PRs before
>
> the individual PRs can be pushed in the order listed above.
>
> * Branch:https://github.com/mdkinney/edk2/tree/TestOnly_Uncrustify_PR_Series <https://github.com/mdkinney/edk2/tree/TestOnly_Uncrustify_PR_Series>
>
> * PR:https://github.com/tianocore/edk2/pull/2229 <https://github.com/tianocore/edk2/pull/2229>
>
> Status = PASS
>
> * CompareBuild:
>
> Branch:https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5 <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5>
>
> --ref1:ef9a059cdb15844fe52a49af2bf7d86b9dd3e9bf
>
> --ref2:Bug_3737_3739_ApplyUncrustifyChanges_V5
>
> Extra Options: -n 4 --quiet
>
> Results:https://github.com/mdkinney/edk2/actions/runs/1521618836 <https://github.com/mdkinney/edk2/actions/runs/1521618836>
>
> 30 VS2019 build comparisons PASS
>
> 40 GCC5 build comparisons PASS
>
> 100% PASS
>
> The following git log shows the set of patches from --ref1 to --ref 2across
>
> which there are no differences in any of the OBJ/LIB/DLL/EFI/FFS/FV/FD files.
>
> --ref2
>
> b7d4bf0675b7 (HEAD -> Bug_3737_3739_ApplyUncrustifyChanges_V5) UnitTestFrameworkPkg: Apply uncrusitify changes
>
> 7f03d25f60e7 UefiPayloadPkg: Apply uncrusitify changes
>
> 0bfd8d9b5ac9 UefiCpuPkg: Apply uncrusitify changes
>
> e1cd9bfb9dea StandaloneMmPkg: Apply uncrusitify changes
>
> 5da2f65be378 SourceLevelDebugPkg: Apply uncrusitify changes
>
> 95b86de07e5d SignedCapsulePkg: Apply uncrusitify changes
>
> fe71d97246c4 ShellPkg: Apply uncrusitify changes
>
> 54c21c952992 SecurityPkg: Apply uncrusitify changes
>
> 187a3785f12b RedfishPkg: Apply uncrusitify changes
>
> 810100002a46 PcAtChipsetPkg: Apply uncrusitify changes
>
> 276a695c0cf2 OvmfPkg: Apply uncrusitify changes
>
> 303c0a91ab07 NetworkPkg: Apply uncrusitify changes
>
> bc80792cd1b1 MdePkg: Apply uncrusitify changes
>
> 3ea86be17a2a MdeModulePkg: Apply uncrusitify changes
>
> c70ef11ed0cd IntelFsp2WrapperPkg: Apply uncrusitify changes
>
> c0291221f252 IntelFsp2Pkg: Apply uncrusitify changes
>
> 6a479952a690 FmpDevicePkg: Apply uncrusitify changes
>
> 3a7c05b7070d FatPkg: Apply uncrusitify changes
>
> b789f98c8959 EmulatorPkg: Apply uncrusitify changes
>
> 952d7a1c9220 EmbeddedPkg: Apply uncrusitify changes
>
> a1cc9881bab6 DynamicTablesPkg: Apply uncrusitify changes
>
> 50654dfe5785 CryptoPkg: Apply uncrusitify changes
>
> ed965a02dfa1 ArmVirtPkg: Apply uncrusitify changes
>
> 9744023fbc46 ArmPlatformPkg: Apply uncrusitify changes
>
> 7a1cde5f5bba ArmPkg: Apply uncrusitify changes
>
> 19d17e0913e8 UefiCpuPkg: Change use of EFI_D_* to DEBUG_*
>
> ffa718b4f994 SourceLevelDebugPkg: Change use of EFI_D_* to DEBUG_*
>
> b86cb3c5e5b4 ShellPkg: Change use of EFI_D_* to DEBUG_*
>
> c7c42204dc07 SecurityPkg: Change use of EFI_D_* to DEBUG_*
>
> 16b8e6f958e4 PcAtChipsetPkg: Change use of EFI_D_* to DEBUG_*
>
> 0ac3f8b2dac5 OvmfPkg: Change use of EFI_D_* to DEBUG_*
>
> bc5004b8d294 NetworkPkg: Change use of EFI_D_* to DEBUG_*
>
> 6f671a8e2377 MdePkg: Change use of EFI_D_* to DEBUG_*
>
> a10c610ff9a3 MdeModulePkg: Change use of EFI_D_* to DEBUG_*
>
> 09a3bddba390 FatPkg: Change use of EFI_D_* to DEBUG_*
>
> 59c61318246a EmulatorPkg: Change use of EFI_D_* to DEBUG_*
>
> 3a80367dda3b EmbeddedPkg: Change use of EFI_D_* to DEBUG_*
>
> 23eb1aaf80ca ArmVirtPkg: Change use of EFI_D_* to DEBUG_*
>
> 875914b45c54 ArmPlatformPkg: Change use of EFI_D_* to DEBUG_*
>
> eb2eca82b451 ArmPkg: Change use of EFI_D_* to DEBUG_*
>
> f0f3f5aae7c4 (origin/master, origin/HEAD, master) UnitTestFrameworkPkg: Update YAML to ignore specific ECC files/errors
>
> c05734797790 UefiPayloadPkg: Update YAML to ignore specific ECC files/errors
>
> c30c40d6c63d StandaloneMmPkg: Update YAML to ignore specific ECC files/errors
>
> 9944508e85f1 ShellPkg: Update YAML to ignore specific ECC files/errors
>
> 60fa40be458d SecurityPkg: Update YAML to ignore specific ECC files/errors
>
> df790cd6b37e MdePkg: Update YAML to ignore specific ECC files/errors
>
> 9deb9370766e MdeModulePkg: Update YAML to ignore specific ECC files/errors
>
> d7d30e8f219f EmulatorPkg: Update YAML to ignore specific ECC files/errors
>
> d5744ecba813 CryptoPkg: Update YAML to ignore specific ECC files/errors
>
> c97fee87f0f9 ArmVirtPkg: Update YAML to ignore specific ECC files/errors
>
> 1939fc9569f2 ArmPlatformPkg: Update YAML to ignore specific ECC files/errors
>
> 365dced2c37a ArmPkg: Update YAML to ignore specific ECC files/errors
>
> 76a1ce4d5fec .azurepipelines/templates: Update max pipeline job time to 2 hours
>
> 99f84ff47390 .pytools/Plugin/LicenseCheck: Use temp directory for git diff output
>
> 3019f1bbabf1 .pytool/Plugin/EccCheck: Add performance optimizations
>
> 854462bd3479 .pytool/Plugin/EccCheck: Remove temp directory on exception
>
> 69877614fdee .pytool/Plugin/EccCheck: Remove RevertCode()
>
> --ref1
>
> ef9a059cdb15 EmulatorPkg/Win/Host: Update CC_FLAGS
>
> bb1bba3d7767 (tag: edk2-stable202111) NetworkPkg: Fix invalid pointer for DNS response token on error
>
> Best regards,
>
> Mike
>
>
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
2021-12-02 21:56 ` Michael Kubacki
@ 2021-12-03 0:14 ` Michael D Kinney
2021-12-03 0:18 ` Michael D Kinney
0 siblings, 1 reply; 39+ messages in thread
From: Michael D Kinney @ 2021-12-03 0:14 UTC (permalink / raw)
To: Michael Kubacki, devel@edk2.groups.io,
maciej.rabeda@linux.intel.com, Michael Kubacki,
Andrew Fish (afish@apple.com), Leif Lindholm, Kinney, Michael D
Hi Michael,
Reponses inline below.
I would like to summarize the 4 issues raised in the past day along with the recommendations.
1) Exclusion feature for UncrustifyCheck. There are 2 directories with 8 files total that
maintainers have noted they would like to see not go through uncrustify formatting. Today
the only content that is skipped is BaseTools and submodules.
Adding a general purpose exclusion feature would then require all developers to make
sure their method of using uncrustify also excludes those same areas. This requires
extra steps for all developers and maintainers.
If we do not add the exclusion feature, then the 8 files will require an extra step
to sync with the original source of those files. The rate of changes of these 8 files
is very low today.
RECOMMENDATION: Do not add exclusion feature at this time. Revisit if the extra work
to maintain the files that would be candidates for exclusions increases significantly.
2) Alignment of assignments. The threshold of 4 characters appears to be too low and causes
source files that are already aligned to become unaligned.
RECOMMENDATION: Change threshold to the default value of 0 which means no limit.
align_assign_thresh= 4
3) Alignment of parameters in function declaration not correct. The root cause of this
is the use of the OPTIONAL keyword. If the OPTIONAL keyword is removed, then the
alignment is correct. The alignment is also correct if the OPTIONAL keyword appears
before the ','. If the OPTIONAL keyword appears after the ',', then the format is
not correct. The OPTIONAL keyword indicates that the parameter in the function is
not required and may be passed in as NULL or 0 or some other default value defined by
the API. It makes more sense for this OPTIONAL keyword that follows the parameter
names to appear before the ',' so it is scoped to the parameter on that line. If it
appears after the ',', then C parsers thinks it is a prefix (IN, OUT, CONST, volatile,
static) for the next parameter in the function.
RECOMMENDATION: Update patch series with a global search and replace so OPTIONAL
keyword always appears before the ',' on the same line.
RegEx search string: ',( *)OPTIONAL( *)'
RegEx replace string: ' $1OPTIONAL,$2'
4) Format issues with complex blocks in DEBUG_CODE(), or between DEBUG_CODE_BEGIN() and
DEBUG_CODE_END(). Uncrustify treats these as macros and is not aware that the
parameter passed into the macro call is a block of C code that needs to be formatted.
Complex blocks with if/while/for/case statements are impacted the most.
RECOMMENDATION: Update the uncrustify with an edk2 specific extension to treat these
macros as a block of code as if they were surrounded by an extra set of braces {}.
I have posted a branch for testing purposes that implements (2) and (3).
Branch: https://github.com/mdkinney/edk2/tree/TestOnly_Bug_3737_3739_ApplyUncrustifyChanges_V6_OPTIONAL_Keyword_Fix
PR: https://github.com/tianocore/edk2/pull/2233
Status: PASS
CompareBuild: https://github.com/mdkinney/edk2/actions/runs/1532855914
Status: PASS
You can see what changed by fetching and comparing the following 2 branches:
https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
https://github.com/mdkinney/edk2/tree/TestOnly_Bug_3737_3739_ApplyUncrustifyChanges_V6_OPTIONAL_Keyword_Fix
Please provide feedback on the RECOMMENDATIONS above. I will go ahead and prepare of V6 version of
the patch series now that that test results are all PASS.
Best regards,
Mike
> -----Original Message-----
> From: Michael Kubacki <mikuback@linux.microsoft.com>
> Sent: Thursday, December 2, 2021 1:57 PM
> To: devel@edk2.groups.io; Kinney, Michael D <michael.d.kinney@intel.com>; maciej.rabeda@linux.intel.com; Michael Kubacki
> <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com) <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>
> Subject: Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
>
> My reply is inline.
>
> Regards,
> Michael
>
> On 12/2/2021 2:45 PM, Michael D Kinney wrote:
> > Hi Maciej,
> >
> > Thanks for the feedback.
> >
> > * Example #1.This appears to be caused by the following uncrustify
> > settings:
> >
> > # The threshold for aligning on '=' in assignments.
> >
> > # Use a negative number for absolute thresholds.
> >
> > #
> >
> > # 0: No limit (default).
> >
> > align_assign_thresh= 0# number
> >
> > The edk2 setting for this is:
> >
> > align_assign_thresh= 4
> >
> > This means blocks of assignments that are different than more than 4
> > spaces will be considered a new block.
> >
> > ‘HwAddreLen’ and ‘Xid’ are more than 4 characters in length
> > different.Same for ‘Xid’ and ‘Reserved’.So
> >
> > uncrustify treats these as 3 different assignment blocks.
> >
> > If we change to the default value of 0: No limit, this example is
> > treated as a single block and all ‘=’ are aligned.
> >
> > Is there a reason ‘4’ was selected?Is there is any harm in using the
> > default of 0?
> >
> We can certainly change the threshold. '4' was derived by
> experimentation. This is an area that is somewhat subjective and every
> case is difficult to cover well. Please feel free to suggest any changes.
>
I recommend we use the default value of 0. That way, any code that choose to
align assignments will still be aligned. If a developer does not like
short assignments and long assignments in the same code block to use the
long assignment column, they can always break the block up into multiple
blocks by adding a carriage return.
> For the benefit of others, this file can be a useful refernce
> understanding spans, gaps, and thresholds.
>
> https://github.com/uncrustify/uncrustify/blob/master/documentation/htdocs/configuration.txt
>
> > * Example #2: Uncruistfy is confused by the DEBUG_CODE() macro.This is
> > not a traditional macro because the
> >
> > contents of the macro is a block of C code.I do not know how to convince
> > uncrustify that the contents of a
> >
> > macro like function call to be treated as a code block from an indent
> > perspective.
> >
> I believe this would have to be overridden in the Uncrustify fork since
> DEBUG_CODE() is being treated as a macro function call and code blocks
> are not expected there. In addition, some special treatment might be
> needed for alignment in between DEBUG_CODE_BEGIN()/DEBUG_CODE_END().
>
> I'm happy to look at this. However, regression validation can take a
> while so I'd like to make sure we need this now. In cases that do not
> have additional code blocks, it seems to format fairly well. Is this
> prevalent and impactful enough it must be fixed now? Or, could we
> revisit it with a follow up patch?
How long does regression testing take? I see about 20 instances of
DEBUG_CODE() that are producing bad formatting. If the content inside
DEBUG_CODE() is a single line of a single block of statements without
any if/while/for/case statements that require further indent, then the
format looks ok.
DEBUG_CODE_BEGIN() and DEBUG_CODE_END() would look better if the
contents between are also indented one level.
>
> > * Example #3/#4: Uncrustify is confused by the OPTIONAL keyword.The
> > edk2 config declares it as a QUALIFIER
> >
> > like IN and OUT.However, OPTIONAL only appears at the end of the line
> > that declares a parameter to a
> >
> > function.There are 3 forms. One with comma after OPTIONAL.One with comma
> > before OPTIONAL, and
> >
> > one with no comma if the parameter is the last parameter in the function
> > declaration.
> >
> > TYPE ParamName OPTIONAL,
> >
> > TYPEParamName, OPTIONAL
> >
> > TYPEParamName OPTIONAL
> >
> > OPTIONAL is defined to nothing in edk2 builds.From a uncrustify
> > perspective, we really want is to be
> >
> > ignored or more correctly treated it as a token that is attached to
> > ParamName and the combination of
> >
> > ParamName and OPTIONAL treated as one unit to determine indents.
> >
> "TYPE ParamName, OPTIONAL" seems especially problematic and presents an
> inconsistency with the other formats. Mike, can you please let me know
> if you have the same observation and your thoughts on a consistent pattern?
>
> > Thanks,
> >
> > Mike
> >
> > *From:*devel@edk2.groups.io <devel@edk2.groups.io> *On Behalf Of *Maciej
> > Rabeda
> > *Sent:* Thursday, December 2, 2021 10:27 AM
> > *To:* devel@edk2.groups.io; Kinney, Michael D
> > <michael.d.kinney@intel.com>; Michael Kubacki
> > <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com)
> > <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>
> > *Subject:* Re: [edk2-devel] Uncrustify Conversion Detailed Plan and
> > Extended Hard Freeze Update #4
> >
> > Hey Mike,
> >
> > While most of the changes related to fixing coding style violations,
> > there are a couple of changes to NetworkPkg in that PR that make the
> > code less readable. Examples below.
> >
> > Example 1:
> >
> >
> >
> > Example 2:
> >
> >
> > Example 3:
> >
> >
> > Example 4:
> >
> > On 30-Nov-21 23:34, Michael D Kinney wrote:
> >
> > Hello,
> >
> > Thank you for your patience during this extended hard freeze.
> >
> > Just one more step to go.There has been a delay in the review of
> >
> > the patch series with the uncrustify source changes.PR(6).This
> >
> > patch series was not sent out as patch review email because of its
> >
> > very large size.It only contains source style changes and the
> >
> > CompareBuild tool and GitHub action has shown there are no binary
> >
> > differences introduced with these source style changes.
> >
> > If you are a package maintainer, then please review the following
> >
> > branch/PR for your package contents and review the EDK II CI results
> >
> > and BuildCompare results.I do not expect a line by line review
> >
> > because we already had time to provide feedback on the source style
> >
> > performed by uncrustify.Instead, a Reviewed-by for your package
> >
> > indicates that you have reviewed the EDK II CI results and CompareBuild
> >
> > tool functionality and results and you accept the source style
> >
> > changes to your package.
> >
> > *https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
> <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5>
> >
> > *https://github.com/tianocore/edk2/pull/2229 <https://github.com/tianocore/edk2/pull/2229>
> >
> > *https://github.com/mdkinney/edk2/actions/runs/1521618836
> <https://github.com/mdkinney/edk2/actions/runs/1521618836>
> >
> > Additional details on this update below.
> >
> > Thank you,
> >
> > Mike
> >
> > Changes from Update #3
> >
> > ----------------------------------------------------------------------------
> >
> > * Pushed PR (5)
> >
> > * Added link to PR(6). EDK II CI Status is PASS. Build Compare PASS.
> >
> > * Waiting for review of PR (6)
> >
> > * Review of PR (7) completed and waiting for review of PR (6)
> >
> > ----------------------------------------------------------------------------
> >
> > Changes from Update #2
> >
> > ----------------------------------------------------------------------------
> >
> > * Changed order of PRs swapping (4) and (5).The PR that activates
> >
> > increases the max CI agent job time is independent of all the other
> >
> > PRs and its review is complete, so it can be committed now.
> >
> > * Pushed PRs (1), (2), (3), (4).
> >
> > * Waiting for review to complete for PRs (5) and (6)
> >
> > * Reviews complete for PR (7)
> >
> > * Identifies steps using git filter-branch to apply uncrustify changes to a
> >
> > code review patch series that was generated before the uncrustify changes
> >
> > avoiding manual merge.
> >
> > * Identified steps using git filter-repo to generate an alternate history of
> >
> > the edk2 repo with uncrustify changes applied on every commit.This may
> >
> > be useful when evaluating changes to files using tools like git blame
> >
> > without the large uncrustify patch series.
> >
> > ---------------------------------------------------------------------------
> >
> > Changes from Update #1
> >
> > ----------------------------------------------------------------------------
> >
> > * Changed order of PRs swapping (6) and (7).The PR that activates
> >
> > EDK II CI check UncrustifyCheck has to be last because it unconditionally
> >
> > checks all C/H files in all packages.Not just files that have been
> >
> > modified like some of the other checkers.
> >
> > * Updated link to the branch with the UncrustifyCheck plugin that has been
> >
> > updated with a one line change and Reviewed-by and Tested-by tags.
> >
> > https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v6
> <https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v6>
> >
> > * Reviews complete for (1), (2), (3), (5), and (7)
> >
> > ---------------------------------------------------------------------------
> >
> > Michael Kubacki and I have prepared the patches required to apply the
> >
> > uncrustify changes and enable EDK II CI to check all submitted
> >
> > patches have been run through uncrustify.
> >
> > We have verified through the CompareBuild GitHub Action that the
> >
> > format changes performed by uncrustify have no functional changes.
> >
> > All of the OBJ, LIB, DLL, EFI, FFS, FV, and FD files match 100%
> >
> > across 70 VS2019/GCC5 builds of all package/platform DSC files in
> >
> > the edk2 repo.
> >
> > The hard freeze will be extended after the edk2-stable202111 tag until
> >
> > all uncrustify related changes are committed.We do not expect this
> >
> > to take more than a few days.Do not push any PRs until the hard
> >
> > freeze is lifted.
> >
> > The changes are broken up into 7 patch series/PRs.The PRs are ordered
> >
> > so they can be submitted using the normal submission process and EDK II
> >
> > CI will pass for each one.Details are listed below.
> >
> > Uncrustify 73.0.3 for EDK II
> >
> > =============================
> >
> > * Sources:https://dev.azure.com/projectmu/_git/Uncrustify <https://dev.azure.com/projectmu/_git/Uncrustify>
> >
> > * Documentation:https://dev.azure.com/projectmu/Uncrustify/_wiki/wikis/Uncrustify.wiki/1/Project-Mu-(EDK-II)-Fork-
> Readme <https://dev.azure.com/projectmu/Uncrustify/_wiki/wikis/Uncrustify.wiki/1/Project-Mu-(EDK-II)-Fork-Readme>
> >
> > * Download:https://dev.azure.com/projectmu/Uncrustify/_packaging?_a=package&feed=mu_uncrustify&package=mu-
> uncrustify-release&protocolType=NuGet&version=73.0.3
> <https://dev.azure.com/projectmu/Uncrustify/_packaging?_a=package&feed=mu_uncrustify&package=mu-uncrustify-
> release&protocolType=NuGet&version=73.0.3>
> >
> > Installing Uncrustify
> >
> > ======================
> >
> > The Uncrustify tool is installed automatically when the Pytools
> >
> > environment is used and the stuart* commands are run to complete the
> >
> > environment setup.Please see:
> >
> > https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally
> <https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally>
> >
> > Uncrustify can also be installed from the download page listed above
> >
> > or built from sources from the source link above.
> >
> > The Documentation link provides instruction on how to run uncrustify from
> >
> > the command line or install as a Visual Studio Code plugin.The main
> >
> > uncrustify documentation also describes how to integrate with a few other
> >
> > editors.
> >
> > We have also discussed a client side githook.That effort has not started.
> >
> > Let us know if that is a feature you would find useful.
> >
> > Developer impact for new code reviews
> >
> > ======================================
> >
> > Once the uncrustify checker is active in EDK II CI, developers must
> >
> > make sure their patches are run through the uncrustify tool before
> >
> > sending the patches for review.
> >
> > Developers must install and run uncrustify against changes files before
> >
> > sending patch review emails or submitting PR for EDK II CI.If EDK II CI
> >
> > detects and differences in source formatting, then EDK II CI will fail
> >
> > and the developer must run uncrustify and resubmit the patches.
> >
> > Developer impact to patch series/PRs reviewed during edk2-stable201121 soft/hard freeze
> >
> > =======================================================================================
> >
> > Developers must rebase their changes after the uncrustify source changes are
> >
> > committed.The branch with a preview of the uncrustify changes can be used
> >
> > to start this rebase work.
> >
> > https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
> <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5>
> >
> > The following steps can be used to update an existing branch with the
> >
> > required uncrustify format.This is the Windows version.I will add
> >
> > the Linux version soon.
> >
> > 1) Fetch and checkout and rebase to latest edk2/master
> >
> > git fetch origin
> >
> > git checkout master
> >
> > git rebase origin/master
> >
> > 2) Make a backup copy of plugin UncrustifyCheck outside WORKSPACE.
> >
> > (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable and
> >
> > EDK II specific uncrustify configuration file available when working
> >
> > with a branch that does not have those tools in its scope.
> >
> > xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
> >
> > 3) Check out the patch series branch (e.g. MyBranch)
> >
> > git checkout MyBranch
> >
> > 4) Rebase patch series against edk2-stable202111
> >
> >
> >
> > git rebase edk2-stable202111
> >
> > 5) Create new branch for the uncrustifed version (e.g. MyBranch_Uncrustified)
> >
> > git checkout -b MyBranch_Uncrustified
> >
> > 6) Use git filter-branch to uncrustify all the commits in the series
> >
> > between the rebase target from (2) and HEAD of the branch.A filter
> >
> > can be used to scope the uncrustify operations to only the C/H files
> >
> > in the specific package the patch series is against. (e.g. DynamicTablesPkg).
> >
> > BaseTools should always be excluded.If the package scoped filter is
> >
> > not used, it will still work, but will take longer to run because
> >
> > uncrustify will rescan every C/H files in the whole repo.
> >
> > git filter-branch --tree-filter "git ls-files DynamicTablesPkg*.c DynamicTablesPkg*.h :!BaseTools/* |
> c:\\Temp\\UncrustifyCheck\\mu-uncrustify-release_extdep\\Windows-x86\\uncrustify.exe -c
> c:\\Temp\\UncrustifyCheck\\uncrustify.cfg -F - --replace --no-backup --if-changed" edk2-stable202111..HEAD
> >
> > 7) Now that all the individual patches in the branch are uncrustified,
> >
> > rebase against latest edk2/master that is already uncrustified.
> >
> > git rebase master
> >
> > 8) Verify the patches in this new branch.
> >
> > Impacts to tracing history across the uncrusity changes
> >
> > =======================================================
> >
> > Tools the view file and line history do work with the large uncrustify
> >
> > patch series.One impact is that the operations can be very slow due
> >
> > to the large uncrustify patches.
> >
> > One option to provide a faster experience is to provide an alternate
> >
> > version of the edk2 repository as "documentation" that has the
> >
> > entire history re-written with uncrustify run on every commit.
> >
> > The tool called git-filter-repo can be used to perform this
> >
> > transformation and runs in a reasonable period of time (a few hours)
> >
> > https://github.com/newren/git-filter-repo <https://github.com/newren/git-filter-repo>
> >
> > https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-demos/lint-history
> <https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-demos/lint-history>
> >
> > The following steps can be used to perform this transformation.
> >
> > This is the Windows version. I will add the Linux version soon.
> >
> > ** WARNING **This operation modifies(rewrites) all the commits
> >
> > in the local copy of the repo.Do not perform
> >
> > these steps on a local repo you are using for
> >
> > active development.
> >
> > 1) Clone edk2 into a new directory (see **WARNING**)
> >
> > git clonehttps://github.com/tianocore/edk2.git <https://github.com/tianocore/edk2.git> edk2-uncrustified
> >
> > cd edk2-uncrustified
> >
> > 2) Setup python virtual env, install pytools, and run stuart commands
> >
> > to setup build environment which includes installing uncrustify tools.
> >
> > https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally
> <https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally>
> >
> > 3) Make a backup copy of plugin UncrustifyCheck outside WORKSPACE.
> >
> > (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable and
> >
> > EDK II specific uncrustify configuration file available when working
> >
> > with a branch that does not have those tools in its scope.
> >
> > xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
> >
> > 4) Use lint-history.py from git-filter-repo examples
> >
> > https://github.com/newren/git-filter-repo <https://github.com/newren/git-filter-repo>
> >
> > https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-demos/lint-history
> <https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-demos/lint-history>
> >
> > Line #127 - Add try except around subprocess.check_call() with except
> >
> > being pass.This is required because there are a few commits of C
> >
> > files in the edk2 repo that have incorrect C syntax and do not
> >
> > build with a C compiler and break the uncrustify parser.Skip reformat
> >
> > of C files that can not be parsed by uncrustify.These rare instances
> >
> > are addressed in the commit that fixes the C syntax error.
> >
> > Run this slightly modified version of lint-history.Include only
> >
> > C/H files and exclude directories that start with 'Tools' or 'BaseTools'.
> >
> > This step took about 2.2 hours on a laptop.
> >
> > lint-history.py
> >
> > --relevant "return (not filename.startswith(b'Tools') and not filename.startswith(b'BaseTools') and
> (filename.endswith(b'.c') or filename.endswith(b'.h')))"
> >
> > c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\mu-uncrustify-release_extdep\\Windows-x86\\uncrustify.exe -c
> c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\uncrustify.cfg --replace --no-backup --if-changed
> >
> > Order of PRs to apply during extended hard freeze
> >
> > ==================================================
> >
> > 1) Update EmulatorPkg Win Host [BuildOptions] MSFT CC_FLAGS to not force debug information
> >
> > *https://bugzilla.tianocore.org/show_bug.cgi?id=3747 <https://bugzilla.tianocore.org/show_bug.cgi?id=3747>
> >
> > *https://github.com/mdkinney/edk2/tree/Bug_3747_EmulatorPkg_WinHost_ReproducibleBuild
> <https://github.com/mdkinney/edk2/tree/Bug_3747_EmulatorPkg_WinHost_ReproducibleBuild>
> >
> > *https://github.com/tianocore/edk2/pull/2215 <https://github.com/tianocore/edk2/pull/2215>
> >
> > * Required for EmulatorPkg to pass CompareBuild for VS2019 IA32/X64 builds.
> >
> > * Status: Review complete.PR pushed.
> >
> > 2) EccCheck should not revert staged and local changes
> >
> > *https://bugzilla.tianocore.org/show_bug.cgi?id=2986 <https://bugzilla.tianocore.org/show_bug.cgi?id=2986>
> >
> > *https://github.com/mdkinney/edk2/tree/Bug_2986_EccCheckRemoveGitRevert_V2
> <https://github.com/mdkinney/edk2/tree/Bug_2986_EccCheckRemoveGitRevert_V2>
> >
> > *https://github.com/tianocore/edk2/pull/2216 <https://github.com/tianocore/edk2/pull/2216>
> >
> > * Required for EDK II CI to complete in a reasonable period of time when
> >
> > processing the 4000+ source file style changes made by uncrustify.
> >
> > * Also fixes critical bugs that can potentially corrupt git state when
> >
> > EccCheck is run locally.
> >
> > * Status: Review complete.PR pushed.
> >
> > 3) Update pytool LicenseCheck plugin to use temp directory for diff output file
> >
> > *https://bugzilla.tianocore.org/show_bug.cgi?id=3746 <https://bugzilla.tianocore.org/show_bug.cgi?id=3746>
> >
> > *https://github.com/mdkinney/edk2/tree/Bug_3746_LicenseCheckUseDiffOutputFile_V2
> <https://github.com/mdkinney/edk2/tree/Bug_3746_LicenseCheckUseDiffOutputFile_V2>
> >
> > *https://github.com/tianocore/edk2/pull/2217 <https://github.com/tianocore/edk2/pull/2217>
> >
> > * Required to reduce EDK II CI build times.
> >
> > * Status: Review complete.PR pushed.
> >
> > 4) Update max job time from 60 min to 120 minutes in .azurepipelines/templates
> >
> > *https://bugzilla.tianocore.org/show_bug.cgi?id=3750 <https://bugzilla.tianocore.org/show_bug.cgi?id=3750>
> >
> > *https://github.com/mdkinney/edk2/tree/Bug_3750_IncreaseAzurePipelinesTimeout
> <https://github.com/mdkinney/edk2/tree/Bug_3750_IncreaseAzurePipelinesTimeout>
> >
> > *https://github.com/tianocore/edk2/pull/2219 <https://github.com/tianocore/edk2/pull/2219>
> >
> > * Required to allow EccCheck of uncrustify changes to complete on Azure
> >
> > Pipelines CI agents without timing out.
> >
> > * Status: Review complete.PR pushed.
> >
> > 5) Update Package YAML to ignore specific ECC files/errors
> >
> > *https://bugzilla.tianocore.org/show_bug.cgi?id=3749 <https://bugzilla.tianocore.org/show_bug.cgi?id=3749>
> >
> > *https://github.com/mdkinney/edk2/tree/Bug_3749_EccCheckIgnoreFilesErrors
> <https://github.com/mdkinney/edk2/tree/Bug_3749_EccCheckIgnoreFilesErrors>
> >
> > *https://github.com/tianocore/edk2/pull/2218 <https://github.com/tianocore/edk2/pull/2218>
> >
> > * Required to pass EccCheck
> >
> > * Status: Review complete. PR pushed
> >
> > 6) Uncrustify Source Changes
> >
> > *https://bugzilla.tianocore.org/show_bug.cgi?id=3737 <https://bugzilla.tianocore.org/show_bug.cgi?id=3737>
> >
> > *https://bugzilla.tianocore.org/show_bug.cgi?id=3739 <https://bugzilla.tianocore.org/show_bug.cgi?id=3739>
> >
> > *https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
> <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5>
> >
> > *https://github.com/tianocore/edk2/pull/2229 <https://github.com/tianocore/edk2/pull/2229>
> >
> > * Build comparison result PASS:https://github.com/mdkinney/edk2/actions/runs/1521618836
> <https://github.com/mdkinney/edk2/actions/runs/1521618836>
> >
> > * EFI_D_ -> DEBUG changes required to pass PatchCheck
> >
> > * Uncrustify format changes required to pass UncrustifyCheck
> >
> > * Status:
> >
> > Waiting for review
> >
> > 7) UncrustifyCheck EDK II CI Plugin
> >
> > *https://bugzilla.tianocore.org/show_bug.cgi?id=3748 <https://bugzilla.tianocore.org/show_bug.cgi?id=3748>
> >
> > *https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v6
> <https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v6>
> >
> > * Required to enforce all PRs submitted to EDK II CI match uncrustify format.
> >
> > * Unconditionally checks all packages.Can not be committed until all C/H
> >
> > source files have been updated.
> >
> > * Status: Review complete
> >
> > Combined Branch/PR for Review/Test
> >
> > ==================================
> >
> > * Build Comparison results must pass 100% across the full set of PRs before
> >
> > the individual PRs can be pushed in the order listed above.
> >
> > * Branch:https://github.com/mdkinney/edk2/tree/TestOnly_Uncrustify_PR_Series
> <https://github.com/mdkinney/edk2/tree/TestOnly_Uncrustify_PR_Series>
> >
> > * PR:https://github.com/tianocore/edk2/pull/2229 <https://github.com/tianocore/edk2/pull/2229>
> >
> > Status = PASS
> >
> > * CompareBuild:
> >
> > Branch:https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
> <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5>
> >
> > --ref1:ef9a059cdb15844fe52a49af2bf7d86b9dd3e9bf
> >
> > --ref2:Bug_3737_3739_ApplyUncrustifyChanges_V5
> >
> > Extra Options: -n 4 --quiet
> >
> > Results:https://github.com/mdkinney/edk2/actions/runs/1521618836
> <https://github.com/mdkinney/edk2/actions/runs/1521618836>
> >
> > 30 VS2019 build comparisons PASS
> >
> > 40 GCC5 build comparisons PASS
> >
> > 100% PASS
> >
> > The following git log shows the set of patches from --ref1 to --ref 2across
> >
> > which there are no differences in any of the OBJ/LIB/DLL/EFI/FFS/FV/FD files.
> >
> > --ref2
> >
> > b7d4bf0675b7 (HEAD -> Bug_3737_3739_ApplyUncrustifyChanges_V5) UnitTestFrameworkPkg: Apply uncrusitify changes
> >
> > 7f03d25f60e7 UefiPayloadPkg: Apply uncrusitify changes
> >
> > 0bfd8d9b5ac9 UefiCpuPkg: Apply uncrusitify changes
> >
> > e1cd9bfb9dea StandaloneMmPkg: Apply uncrusitify changes
> >
> > 5da2f65be378 SourceLevelDebugPkg: Apply uncrusitify changes
> >
> > 95b86de07e5d SignedCapsulePkg: Apply uncrusitify changes
> >
> > fe71d97246c4 ShellPkg: Apply uncrusitify changes
> >
> > 54c21c952992 SecurityPkg: Apply uncrusitify changes
> >
> > 187a3785f12b RedfishPkg: Apply uncrusitify changes
> >
> > 810100002a46 PcAtChipsetPkg: Apply uncrusitify changes
> >
> > 276a695c0cf2 OvmfPkg: Apply uncrusitify changes
> >
> > 303c0a91ab07 NetworkPkg: Apply uncrusitify changes
> >
> > bc80792cd1b1 MdePkg: Apply uncrusitify changes
> >
> > 3ea86be17a2a MdeModulePkg: Apply uncrusitify changes
> >
> > c70ef11ed0cd IntelFsp2WrapperPkg: Apply uncrusitify changes
> >
> > c0291221f252 IntelFsp2Pkg: Apply uncrusitify changes
> >
> > 6a479952a690 FmpDevicePkg: Apply uncrusitify changes
> >
> > 3a7c05b7070d FatPkg: Apply uncrusitify changes
> >
> > b789f98c8959 EmulatorPkg: Apply uncrusitify changes
> >
> > 952d7a1c9220 EmbeddedPkg: Apply uncrusitify changes
> >
> > a1cc9881bab6 DynamicTablesPkg: Apply uncrusitify changes
> >
> > 50654dfe5785 CryptoPkg: Apply uncrusitify changes
> >
> > ed965a02dfa1 ArmVirtPkg: Apply uncrusitify changes
> >
> > 9744023fbc46 ArmPlatformPkg: Apply uncrusitify changes
> >
> > 7a1cde5f5bba ArmPkg: Apply uncrusitify changes
> >
> > 19d17e0913e8 UefiCpuPkg: Change use of EFI_D_* to DEBUG_*
> >
> > ffa718b4f994 SourceLevelDebugPkg: Change use of EFI_D_* to DEBUG_*
> >
> > b86cb3c5e5b4 ShellPkg: Change use of EFI_D_* to DEBUG_*
> >
> > c7c42204dc07 SecurityPkg: Change use of EFI_D_* to DEBUG_*
> >
> > 16b8e6f958e4 PcAtChipsetPkg: Change use of EFI_D_* to DEBUG_*
> >
> > 0ac3f8b2dac5 OvmfPkg: Change use of EFI_D_* to DEBUG_*
> >
> > bc5004b8d294 NetworkPkg: Change use of EFI_D_* to DEBUG_*
> >
> > 6f671a8e2377 MdePkg: Change use of EFI_D_* to DEBUG_*
> >
> > a10c610ff9a3 MdeModulePkg: Change use of EFI_D_* to DEBUG_*
> >
> > 09a3bddba390 FatPkg: Change use of EFI_D_* to DEBUG_*
> >
> > 59c61318246a EmulatorPkg: Change use of EFI_D_* to DEBUG_*
> >
> > 3a80367dda3b EmbeddedPkg: Change use of EFI_D_* to DEBUG_*
> >
> > 23eb1aaf80ca ArmVirtPkg: Change use of EFI_D_* to DEBUG_*
> >
> > 875914b45c54 ArmPlatformPkg: Change use of EFI_D_* to DEBUG_*
> >
> > eb2eca82b451 ArmPkg: Change use of EFI_D_* to DEBUG_*
> >
> > f0f3f5aae7c4 (origin/master, origin/HEAD, master) UnitTestFrameworkPkg: Update YAML to ignore specific ECC
> files/errors
> >
> > c05734797790 UefiPayloadPkg: Update YAML to ignore specific ECC files/errors
> >
> > c30c40d6c63d StandaloneMmPkg: Update YAML to ignore specific ECC files/errors
> >
> > 9944508e85f1 ShellPkg: Update YAML to ignore specific ECC files/errors
> >
> > 60fa40be458d SecurityPkg: Update YAML to ignore specific ECC files/errors
> >
> > df790cd6b37e MdePkg: Update YAML to ignore specific ECC files/errors
> >
> > 9deb9370766e MdeModulePkg: Update YAML to ignore specific ECC files/errors
> >
> > d7d30e8f219f EmulatorPkg: Update YAML to ignore specific ECC files/errors
> >
> > d5744ecba813 CryptoPkg: Update YAML to ignore specific ECC files/errors
> >
> > c97fee87f0f9 ArmVirtPkg: Update YAML to ignore specific ECC files/errors
> >
> > 1939fc9569f2 ArmPlatformPkg: Update YAML to ignore specific ECC files/errors
> >
> > 365dced2c37a ArmPkg: Update YAML to ignore specific ECC files/errors
> >
> > 76a1ce4d5fec .azurepipelines/templates: Update max pipeline job time to 2 hours
> >
> > 99f84ff47390 .pytools/Plugin/LicenseCheck: Use temp directory for git diff output
> >
> > 3019f1bbabf1 .pytool/Plugin/EccCheck: Add performance optimizations
> >
> > 854462bd3479 .pytool/Plugin/EccCheck: Remove temp directory on exception
> >
> > 69877614fdee .pytool/Plugin/EccCheck: Remove RevertCode()
> >
> > --ref1
> >
> > ef9a059cdb15 EmulatorPkg/Win/Host: Update CC_FLAGS
> >
> > bb1bba3d7767 (tag: edk2-stable202111) NetworkPkg: Fix invalid pointer for DNS response token on error
> >
> > Best regards,
> >
> > Mike
> >
> >
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
2021-12-03 0:14 ` Michael D Kinney
@ 2021-12-03 0:18 ` Michael D Kinney
2021-12-03 0:31 ` Michael Kubacki
0 siblings, 1 reply; 39+ messages in thread
From: Michael D Kinney @ 2021-12-03 0:18 UTC (permalink / raw)
To: Michael Kubacki, devel@edk2.groups.io,
maciej.rabeda@linux.intel.com, Michael Kubacki,
Andrew Fish (afish@apple.com), Leif Lindholm, Kinney, Michael D
Hi Michael,
CORRECTION: set align_assign_threshold to 0.
Reponses inline below.
I would like to summarize the 4 issues raised in the past day along with the recommendations.
1) Exclusion feature for UncrustifyCheck. There are 2 directories with 8 files total that
maintainers have noted they would like to see not go through uncrustify formatting. Today
the only content that is skipped is BaseTools and submodules.
Adding a general purpose exclusion feature would then require all developers to make
sure their method of using uncrustify also excludes those same areas. This requires
extra steps for all developers and maintainers.
If we do not add the exclusion feature, then the 8 files will require an extra step
to sync with the original source of those files. The rate of changes of these 8 files
is very low today.
RECOMMENDATION: Do not add exclusion feature at this time. Revisit if the extra work
to maintain the files that would be candidates for exclusions increases significantly.
2) Alignment of assignments. The threshold of 4 characters appears to be too low and causes
source files that are already aligned to become unaligned.
RECOMMENDATION: Change threshold to the default value of 0 which means no limit.
align_assign_thresh= 0
3) Alignment of parameters in function declaration not correct. The root cause of this
is the use of the OPTIONAL keyword. If the OPTIONAL keyword is removed, then the
alignment is correct. The alignment is also correct if the OPTIONAL keyword appears
before the ','. If the OPTIONAL keyword appears after the ',', then the format is
not correct. The OPTIONAL keyword indicates that the parameter in the function is
not required and may be passed in as NULL or 0 or some other default value defined by
the API. It makes more sense for this OPTIONAL keyword that follows the parameter
names to appear before the ',' so it is scoped to the parameter on that line. If it
appears after the ',', then C parsers thinks it is a prefix (IN, OUT, CONST, volatile,
static) for the next parameter in the function.
RECOMMENDATION: Update patch series with a global search and replace so OPTIONAL
keyword always appears before the ',' on the same line.
RegEx search string: ',( *)OPTIONAL( *)'
RegEx replace string: ' $1OPTIONAL,$2'
4) Format issues with complex blocks in DEBUG_CODE(), or between DEBUG_CODE_BEGIN() and
DEBUG_CODE_END(). Uncrustify treats these as macros and is not aware that the
parameter passed into the macro call is a block of C code that needs to be formatted.
Complex blocks with if/while/for/case statements are impacted the most.
RECOMMENDATION: Update the uncrustify with an edk2 specific extension to treat these
macros as a block of code as if they were surrounded by an extra set of braces {}.
I have posted a branch for testing purposes that implements (2) and (3).
Branch: https://github.com/mdkinney/edk2/tree/TestOnly_Bug_3737_3739_ApplyUncrustifyChanges_V6_OPTIONAL_Keyword_Fix
PR: https://github.com/tianocore/edk2/pull/2233
Status: PASS
CompareBuild: https://github.com/mdkinney/edk2/actions/runs/1532855914
Status: PASS
You can see what changed by fetching and comparing the following 2 branches:
https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
https://github.com/mdkinney/edk2/tree/TestOnly_Bug_3737_3739_ApplyUncrustifyChanges_V6_OPTIONAL_Keyword_Fix
Please provide feedback on the RECOMMENDATIONS above. I will go ahead and prepare of V6 version of
the patch series now that that test results are all PASS.
Best regards,
Mike
> -----Original Message-----
> From: Kinney, Michael D <michael.d.kinney@intel.com>
> Sent: Thursday, December 2, 2021 4:15 PM
> To: Michael Kubacki <mikuback@linux.microsoft.com>; devel@edk2.groups.io; maciej.rabeda@linux.intel.com; Michael Kubacki
> <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com) <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>;
> Kinney, Michael D <michael.d.kinney@intel.com>
> Subject: RE: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
>
> Hi Michael,
>
> Reponses inline below.
>
> I would like to summarize the 4 issues raised in the past day along with the recommendations.
>
> 1) Exclusion feature for UncrustifyCheck. There are 2 directories with 8 files total that
> maintainers have noted they would like to see not go through uncrustify formatting. Today
> the only content that is skipped is BaseTools and submodules.
>
> Adding a general purpose exclusion feature would then require all developers to make
> sure their method of using uncrustify also excludes those same areas. This requires
> extra steps for all developers and maintainers.
>
> If we do not add the exclusion feature, then the 8 files will require an extra step
> to sync with the original source of those files. The rate of changes of these 8 files
> is very low today.
>
> RECOMMENDATION: Do not add exclusion feature at this time. Revisit if the extra work
> to maintain the files that would be candidates for exclusions increases significantly.
>
> 2) Alignment of assignments. The threshold of 4 characters appears to be too low and causes
> source files that are already aligned to become unaligned.
>
> RECOMMENDATION: Change threshold to the default value of 0 which means no limit.
>
> align_assign_thresh= 4
>
> 3) Alignment of parameters in function declaration not correct. The root cause of this
> is the use of the OPTIONAL keyword. If the OPTIONAL keyword is removed, then the
> alignment is correct. The alignment is also correct if the OPTIONAL keyword appears
> before the ','. If the OPTIONAL keyword appears after the ',', then the format is
> not correct. The OPTIONAL keyword indicates that the parameter in the function is
> not required and may be passed in as NULL or 0 or some other default value defined by
> the API. It makes more sense for this OPTIONAL keyword that follows the parameter
> names to appear before the ',' so it is scoped to the parameter on that line. If it
> appears after the ',', then C parsers thinks it is a prefix (IN, OUT, CONST, volatile,
> static) for the next parameter in the function.
>
> RECOMMENDATION: Update patch series with a global search and replace so OPTIONAL
> keyword always appears before the ',' on the same line.
>
> RegEx search string: ',( *)OPTIONAL( *)'
> RegEx replace string: ' $1OPTIONAL,$2'
>
> 4) Format issues with complex blocks in DEBUG_CODE(), or between DEBUG_CODE_BEGIN() and
> DEBUG_CODE_END(). Uncrustify treats these as macros and is not aware that the
> parameter passed into the macro call is a block of C code that needs to be formatted.
> Complex blocks with if/while/for/case statements are impacted the most.
>
> RECOMMENDATION: Update the uncrustify with an edk2 specific extension to treat these
> macros as a block of code as if they were surrounded by an extra set of braces {}.
>
>
> I have posted a branch for testing purposes that implements (2) and (3).
>
> Branch: https://github.com/mdkinney/edk2/tree/TestOnly_Bug_3737_3739_ApplyUncrustifyChanges_V6_OPTIONAL_Keyword_Fix
> PR: https://github.com/tianocore/edk2/pull/2233
> Status: PASS
> CompareBuild: https://github.com/mdkinney/edk2/actions/runs/1532855914
> Status: PASS
>
> You can see what changed by fetching and comparing the following 2 branches:
>
> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
> https://github.com/mdkinney/edk2/tree/TestOnly_Bug_3737_3739_ApplyUncrustifyChanges_V6_OPTIONAL_Keyword_Fix
>
> Please provide feedback on the RECOMMENDATIONS above. I will go ahead and prepare of V6 version of
> the patch series now that that test results are all PASS.
>
> Best regards,
>
> Mike
>
>
> > -----Original Message-----
> > From: Michael Kubacki <mikuback@linux.microsoft.com>
> > Sent: Thursday, December 2, 2021 1:57 PM
> > To: devel@edk2.groups.io; Kinney, Michael D <michael.d.kinney@intel.com>; maciej.rabeda@linux.intel.com; Michael Kubacki
> > <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com) <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>
> > Subject: Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
> >
> > My reply is inline.
> >
> > Regards,
> > Michael
> >
> > On 12/2/2021 2:45 PM, Michael D Kinney wrote:
> > > Hi Maciej,
> > >
> > > Thanks for the feedback.
> > >
> > > * Example #1.This appears to be caused by the following uncrustify
> > > settings:
> > >
> > > # The threshold for aligning on '=' in assignments.
> > >
> > > # Use a negative number for absolute thresholds.
> > >
> > > #
> > >
> > > # 0: No limit (default).
> > >
> > > align_assign_thresh= 0# number
> > >
> > > The edk2 setting for this is:
> > >
> > > align_assign_thresh= 4
> > >
> > > This means blocks of assignments that are different than more than 4
> > > spaces will be considered a new block.
> > >
> > > ‘HwAddreLen’ and ‘Xid’ are more than 4 characters in length
> > > different.Same for ‘Xid’ and ‘Reserved’.So
> > >
> > > uncrustify treats these as 3 different assignment blocks.
> > >
> > > If we change to the default value of 0: No limit, this example is
> > > treated as a single block and all ‘=’ are aligned.
> > >
> > > Is there a reason ‘4’ was selected?Is there is any harm in using the
> > > default of 0?
> > >
> > We can certainly change the threshold. '4' was derived by
> > experimentation. This is an area that is somewhat subjective and every
> > case is difficult to cover well. Please feel free to suggest any changes.
> >
>
> I recommend we use the default value of 0. That way, any code that choose to
> align assignments will still be aligned. If a developer does not like
> short assignments and long assignments in the same code block to use the
> long assignment column, they can always break the block up into multiple
> blocks by adding a carriage return.
>
> > For the benefit of others, this file can be a useful refernce
> > understanding spans, gaps, and thresholds.
> >
> > https://github.com/uncrustify/uncrustify/blob/master/documentation/htdocs/configuration.txt
> >
> > > * Example #2: Uncruistfy is confused by the DEBUG_CODE() macro.This is
> > > not a traditional macro because the
> > >
> > > contents of the macro is a block of C code.I do not know how to convince
> > > uncrustify that the contents of a
> > >
> > > macro like function call to be treated as a code block from an indent
> > > perspective.
> > >
> > I believe this would have to be overridden in the Uncrustify fork since
> > DEBUG_CODE() is being treated as a macro function call and code blocks
> > are not expected there. In addition, some special treatment might be
> > needed for alignment in between DEBUG_CODE_BEGIN()/DEBUG_CODE_END().
> >
> > I'm happy to look at this. However, regression validation can take a
> > while so I'd like to make sure we need this now. In cases that do not
> > have additional code blocks, it seems to format fairly well. Is this
> > prevalent and impactful enough it must be fixed now? Or, could we
> > revisit it with a follow up patch?
>
> How long does regression testing take? I see about 20 instances of
> DEBUG_CODE() that are producing bad formatting. If the content inside
> DEBUG_CODE() is a single line of a single block of statements without
> any if/while/for/case statements that require further indent, then the
> format looks ok.
>
> DEBUG_CODE_BEGIN() and DEBUG_CODE_END() would look better if the
> contents between are also indented one level.
>
> >
> > > * Example #3/#4: Uncrustify is confused by the OPTIONAL keyword.The
> > > edk2 config declares it as a QUALIFIER
> > >
> > > like IN and OUT.However, OPTIONAL only appears at the end of the line
> > > that declares a parameter to a
> > >
> > > function.There are 3 forms. One with comma after OPTIONAL.One with comma
> > > before OPTIONAL, and
> > >
> > > one with no comma if the parameter is the last parameter in the function
> > > declaration.
> > >
> > > TYPE ParamName OPTIONAL,
> > >
> > > TYPEParamName, OPTIONAL
> > >
> > > TYPEParamName OPTIONAL
> > >
> > > OPTIONAL is defined to nothing in edk2 builds.From a uncrustify
> > > perspective, we really want is to be
> > >
> > > ignored or more correctly treated it as a token that is attached to
> > > ParamName and the combination of
> > >
> > > ParamName and OPTIONAL treated as one unit to determine indents.
> > >
> > "TYPE ParamName, OPTIONAL" seems especially problematic and presents an
> > inconsistency with the other formats. Mike, can you please let me know
> > if you have the same observation and your thoughts on a consistent pattern?
> >
> > > Thanks,
> > >
> > > Mike
> > >
> > > *From:*devel@edk2.groups.io <devel@edk2.groups.io> *On Behalf Of *Maciej
> > > Rabeda
> > > *Sent:* Thursday, December 2, 2021 10:27 AM
> > > *To:* devel@edk2.groups.io; Kinney, Michael D
> > > <michael.d.kinney@intel.com>; Michael Kubacki
> > > <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com)
> > > <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>
> > > *Subject:* Re: [edk2-devel] Uncrustify Conversion Detailed Plan and
> > > Extended Hard Freeze Update #4
> > >
> > > Hey Mike,
> > >
> > > While most of the changes related to fixing coding style violations,
> > > there are a couple of changes to NetworkPkg in that PR that make the
> > > code less readable. Examples below.
> > >
> > > Example 1:
> > >
> > >
> > >
> > > Example 2:
> > >
> > >
> > > Example 3:
> > >
> > >
> > > Example 4:
> > >
> > > On 30-Nov-21 23:34, Michael D Kinney wrote:
> > >
> > > Hello,
> > >
> > > Thank you for your patience during this extended hard freeze.
> > >
> > > Just one more step to go.There has been a delay in the review of
> > >
> > > the patch series with the uncrustify source changes.PR(6).This
> > >
> > > patch series was not sent out as patch review email because of its
> > >
> > > very large size.It only contains source style changes and the
> > >
> > > CompareBuild tool and GitHub action has shown there are no binary
> > >
> > > differences introduced with these source style changes.
> > >
> > > If you are a package maintainer, then please review the following
> > >
> > > branch/PR for your package contents and review the EDK II CI results
> > >
> > > and BuildCompare results.I do not expect a line by line review
> > >
> > > because we already had time to provide feedback on the source style
> > >
> > > performed by uncrustify.Instead, a Reviewed-by for your package
> > >
> > > indicates that you have reviewed the EDK II CI results and CompareBuild
> > >
> > > tool functionality and results and you accept the source style
> > >
> > > changes to your package.
> > >
> > > *https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
> > <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5>
> > >
> > > *https://github.com/tianocore/edk2/pull/2229 <https://github.com/tianocore/edk2/pull/2229>
> > >
> > > *https://github.com/mdkinney/edk2/actions/runs/1521618836
> > <https://github.com/mdkinney/edk2/actions/runs/1521618836>
> > >
> > > Additional details on this update below.
> > >
> > > Thank you,
> > >
> > > Mike
> > >
> > > Changes from Update #3
> > >
> > > ----------------------------------------------------------------------------
> > >
> > > * Pushed PR (5)
> > >
> > > * Added link to PR(6). EDK II CI Status is PASS. Build Compare PASS.
> > >
> > > * Waiting for review of PR (6)
> > >
> > > * Review of PR (7) completed and waiting for review of PR (6)
> > >
> > > ----------------------------------------------------------------------------
> > >
> > > Changes from Update #2
> > >
> > > ----------------------------------------------------------------------------
> > >
> > > * Changed order of PRs swapping (4) and (5).The PR that activates
> > >
> > > increases the max CI agent job time is independent of all the other
> > >
> > > PRs and its review is complete, so it can be committed now.
> > >
> > > * Pushed PRs (1), (2), (3), (4).
> > >
> > > * Waiting for review to complete for PRs (5) and (6)
> > >
> > > * Reviews complete for PR (7)
> > >
> > > * Identifies steps using git filter-branch to apply uncrustify changes to a
> > >
> > > code review patch series that was generated before the uncrustify changes
> > >
> > > avoiding manual merge.
> > >
> > > * Identified steps using git filter-repo to generate an alternate history of
> > >
> > > the edk2 repo with uncrustify changes applied on every commit.This may
> > >
> > > be useful when evaluating changes to files using tools like git blame
> > >
> > > without the large uncrustify patch series.
> > >
> > > ---------------------------------------------------------------------------
> > >
> > > Changes from Update #1
> > >
> > > ----------------------------------------------------------------------------
> > >
> > > * Changed order of PRs swapping (6) and (7).The PR that activates
> > >
> > > EDK II CI check UncrustifyCheck has to be last because it unconditionally
> > >
> > > checks all C/H files in all packages.Not just files that have been
> > >
> > > modified like some of the other checkers.
> > >
> > > * Updated link to the branch with the UncrustifyCheck plugin that has been
> > >
> > > updated with a one line change and Reviewed-by and Tested-by tags.
> > >
> > > https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v6
> > <https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v6>
> > >
> > > * Reviews complete for (1), (2), (3), (5), and (7)
> > >
> > > ---------------------------------------------------------------------------
> > >
> > > Michael Kubacki and I have prepared the patches required to apply the
> > >
> > > uncrustify changes and enable EDK II CI to check all submitted
> > >
> > > patches have been run through uncrustify.
> > >
> > > We have verified through the CompareBuild GitHub Action that the
> > >
> > > format changes performed by uncrustify have no functional changes.
> > >
> > > All of the OBJ, LIB, DLL, EFI, FFS, FV, and FD files match 100%
> > >
> > > across 70 VS2019/GCC5 builds of all package/platform DSC files in
> > >
> > > the edk2 repo.
> > >
> > > The hard freeze will be extended after the edk2-stable202111 tag until
> > >
> > > all uncrustify related changes are committed.We do not expect this
> > >
> > > to take more than a few days.Do not push any PRs until the hard
> > >
> > > freeze is lifted.
> > >
> > > The changes are broken up into 7 patch series/PRs.The PRs are ordered
> > >
> > > so they can be submitted using the normal submission process and EDK II
> > >
> > > CI will pass for each one.Details are listed below.
> > >
> > > Uncrustify 73.0.3 for EDK II
> > >
> > > =============================
> > >
> > > * Sources:https://dev.azure.com/projectmu/_git/Uncrustify <https://dev.azure.com/projectmu/_git/Uncrustify>
> > >
> > > * Documentation:https://dev.azure.com/projectmu/Uncrustify/_wiki/wikis/Uncrustify.wiki/1/Project-Mu-(EDK-II)-Fork-
> > Readme <https://dev.azure.com/projectmu/Uncrustify/_wiki/wikis/Uncrustify.wiki/1/Project-Mu-(EDK-II)-Fork-Readme>
> > >
> > > * Download:https://dev.azure.com/projectmu/Uncrustify/_packaging?_a=package&feed=mu_uncrustify&package=mu-
> > uncrustify-release&protocolType=NuGet&version=73.0.3
> > <https://dev.azure.com/projectmu/Uncrustify/_packaging?_a=package&feed=mu_uncrustify&package=mu-uncrustify-
> > release&protocolType=NuGet&version=73.0.3>
> > >
> > > Installing Uncrustify
> > >
> > > ======================
> > >
> > > The Uncrustify tool is installed automatically when the Pytools
> > >
> > > environment is used and the stuart* commands are run to complete the
> > >
> > > environment setup.Please see:
> > >
> > > https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally
> > <https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally>
> > >
> > > Uncrustify can also be installed from the download page listed above
> > >
> > > or built from sources from the source link above.
> > >
> > > The Documentation link provides instruction on how to run uncrustify from
> > >
> > > the command line or install as a Visual Studio Code plugin.The main
> > >
> > > uncrustify documentation also describes how to integrate with a few other
> > >
> > > editors.
> > >
> > > We have also discussed a client side githook.That effort has not started.
> > >
> > > Let us know if that is a feature you would find useful.
> > >
> > > Developer impact for new code reviews
> > >
> > > ======================================
> > >
> > > Once the uncrustify checker is active in EDK II CI, developers must
> > >
> > > make sure their patches are run through the uncrustify tool before
> > >
> > > sending the patches for review.
> > >
> > > Developers must install and run uncrustify against changes files before
> > >
> > > sending patch review emails or submitting PR for EDK II CI.If EDK II CI
> > >
> > > detects and differences in source formatting, then EDK II CI will fail
> > >
> > > and the developer must run uncrustify and resubmit the patches.
> > >
> > > Developer impact to patch series/PRs reviewed during edk2-stable201121 soft/hard freeze
> > >
> > > =======================================================================================
> > >
> > > Developers must rebase their changes after the uncrustify source changes are
> > >
> > > committed.The branch with a preview of the uncrustify changes can be used
> > >
> > > to start this rebase work.
> > >
> > > https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
> > <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5>
> > >
> > > The following steps can be used to update an existing branch with the
> > >
> > > required uncrustify format.This is the Windows version.I will add
> > >
> > > the Linux version soon.
> > >
> > > 1) Fetch and checkout and rebase to latest edk2/master
> > >
> > > git fetch origin
> > >
> > > git checkout master
> > >
> > > git rebase origin/master
> > >
> > > 2) Make a backup copy of plugin UncrustifyCheck outside WORKSPACE.
> > >
> > > (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable and
> > >
> > > EDK II specific uncrustify configuration file available when working
> > >
> > > with a branch that does not have those tools in its scope.
> > >
> > > xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
> > >
> > > 3) Check out the patch series branch (e.g. MyBranch)
> > >
> > > git checkout MyBranch
> > >
> > > 4) Rebase patch series against edk2-stable202111
> > >
> > >
> > >
> > > git rebase edk2-stable202111
> > >
> > > 5) Create new branch for the uncrustifed version (e.g. MyBranch_Uncrustified)
> > >
> > > git checkout -b MyBranch_Uncrustified
> > >
> > > 6) Use git filter-branch to uncrustify all the commits in the series
> > >
> > > between the rebase target from (2) and HEAD of the branch.A filter
> > >
> > > can be used to scope the uncrustify operations to only the C/H files
> > >
> > > in the specific package the patch series is against. (e.g. DynamicTablesPkg).
> > >
> > > BaseTools should always be excluded.If the package scoped filter is
> > >
> > > not used, it will still work, but will take longer to run because
> > >
> > > uncrustify will rescan every C/H files in the whole repo.
> > >
> > > git filter-branch --tree-filter "git ls-files DynamicTablesPkg*.c DynamicTablesPkg*.h :!BaseTools/* |
> > c:\\Temp\\UncrustifyCheck\\mu-uncrustify-release_extdep\\Windows-x86\\uncrustify.exe -c
> > c:\\Temp\\UncrustifyCheck\\uncrustify.cfg -F - --replace --no-backup --if-changed" edk2-stable202111..HEAD
> > >
> > > 7) Now that all the individual patches in the branch are uncrustified,
> > >
> > > rebase against latest edk2/master that is already uncrustified.
> > >
> > > git rebase master
> > >
> > > 8) Verify the patches in this new branch.
> > >
> > > Impacts to tracing history across the uncrusity changes
> > >
> > > =======================================================
> > >
> > > Tools the view file and line history do work with the large uncrustify
> > >
> > > patch series.One impact is that the operations can be very slow due
> > >
> > > to the large uncrustify patches.
> > >
> > > One option to provide a faster experience is to provide an alternate
> > >
> > > version of the edk2 repository as "documentation" that has the
> > >
> > > entire history re-written with uncrustify run on every commit.
> > >
> > > The tool called git-filter-repo can be used to perform this
> > >
> > > transformation and runs in a reasonable period of time (a few hours)
> > >
> > > https://github.com/newren/git-filter-repo <https://github.com/newren/git-filter-repo>
> > >
> > > https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-demos/lint-history
> > <https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-demos/lint-history>
> > >
> > > The following steps can be used to perform this transformation.
> > >
> > > This is the Windows version. I will add the Linux version soon.
> > >
> > > ** WARNING **This operation modifies(rewrites) all the commits
> > >
> > > in the local copy of the repo.Do not perform
> > >
> > > these steps on a local repo you are using for
> > >
> > > active development.
> > >
> > > 1) Clone edk2 into a new directory (see **WARNING**)
> > >
> > > git clonehttps://github.com/tianocore/edk2.git <https://github.com/tianocore/edk2.git> edk2-uncrustified
> > >
> > > cd edk2-uncrustified
> > >
> > > 2) Setup python virtual env, install pytools, and run stuart commands
> > >
> > > to setup build environment which includes installing uncrustify tools.
> > >
> > > https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally
> > <https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally>
> > >
> > > 3) Make a backup copy of plugin UncrustifyCheck outside WORKSPACE.
> > >
> > > (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable and
> > >
> > > EDK II specific uncrustify configuration file available when working
> > >
> > > with a branch that does not have those tools in its scope.
> > >
> > > xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
> > >
> > > 4) Use lint-history.py from git-filter-repo examples
> > >
> > > https://github.com/newren/git-filter-repo <https://github.com/newren/git-filter-repo>
> > >
> > > https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-demos/lint-history
> > <https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-demos/lint-history>
> > >
> > > Line #127 - Add try except around subprocess.check_call() with except
> > >
> > > being pass.This is required because there are a few commits of C
> > >
> > > files in the edk2 repo that have incorrect C syntax and do not
> > >
> > > build with a C compiler and break the uncrustify parser.Skip reformat
> > >
> > > of C files that can not be parsed by uncrustify.These rare instances
> > >
> > > are addressed in the commit that fixes the C syntax error.
> > >
> > > Run this slightly modified version of lint-history.Include only
> > >
> > > C/H files and exclude directories that start with 'Tools' or 'BaseTools'.
> > >
> > > This step took about 2.2 hours on a laptop.
> > >
> > > lint-history.py
> > >
> > > --relevant "return (not filename.startswith(b'Tools') and not filename.startswith(b'BaseTools') and
> > (filename.endswith(b'.c') or filename.endswith(b'.h')))"
> > >
> > > c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\mu-uncrustify-release_extdep\\Windows-x86\\uncrustify.exe -c
> > c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\uncrustify.cfg --replace --no-backup --if-changed
> > >
> > > Order of PRs to apply during extended hard freeze
> > >
> > > ==================================================
> > >
> > > 1) Update EmulatorPkg Win Host [BuildOptions] MSFT CC_FLAGS to not force debug information
> > >
> > > *https://bugzilla.tianocore.org/show_bug.cgi?id=3747 <https://bugzilla.tianocore.org/show_bug.cgi?id=3747>
> > >
> > > *https://github.com/mdkinney/edk2/tree/Bug_3747_EmulatorPkg_WinHost_ReproducibleBuild
> > <https://github.com/mdkinney/edk2/tree/Bug_3747_EmulatorPkg_WinHost_ReproducibleBuild>
> > >
> > > *https://github.com/tianocore/edk2/pull/2215 <https://github.com/tianocore/edk2/pull/2215>
> > >
> > > * Required for EmulatorPkg to pass CompareBuild for VS2019 IA32/X64 builds.
> > >
> > > * Status: Review complete.PR pushed.
> > >
> > > 2) EccCheck should not revert staged and local changes
> > >
> > > *https://bugzilla.tianocore.org/show_bug.cgi?id=2986 <https://bugzilla.tianocore.org/show_bug.cgi?id=2986>
> > >
> > > *https://github.com/mdkinney/edk2/tree/Bug_2986_EccCheckRemoveGitRevert_V2
> > <https://github.com/mdkinney/edk2/tree/Bug_2986_EccCheckRemoveGitRevert_V2>
> > >
> > > *https://github.com/tianocore/edk2/pull/2216 <https://github.com/tianocore/edk2/pull/2216>
> > >
> > > * Required for EDK II CI to complete in a reasonable period of time when
> > >
> > > processing the 4000+ source file style changes made by uncrustify.
> > >
> > > * Also fixes critical bugs that can potentially corrupt git state when
> > >
> > > EccCheck is run locally.
> > >
> > > * Status: Review complete.PR pushed.
> > >
> > > 3) Update pytool LicenseCheck plugin to use temp directory for diff output file
> > >
> > > *https://bugzilla.tianocore.org/show_bug.cgi?id=3746 <https://bugzilla.tianocore.org/show_bug.cgi?id=3746>
> > >
> > > *https://github.com/mdkinney/edk2/tree/Bug_3746_LicenseCheckUseDiffOutputFile_V2
> > <https://github.com/mdkinney/edk2/tree/Bug_3746_LicenseCheckUseDiffOutputFile_V2>
> > >
> > > *https://github.com/tianocore/edk2/pull/2217 <https://github.com/tianocore/edk2/pull/2217>
> > >
> > > * Required to reduce EDK II CI build times.
> > >
> > > * Status: Review complete.PR pushed.
> > >
> > > 4) Update max job time from 60 min to 120 minutes in .azurepipelines/templates
> > >
> > > *https://bugzilla.tianocore.org/show_bug.cgi?id=3750 <https://bugzilla.tianocore.org/show_bug.cgi?id=3750>
> > >
> > > *https://github.com/mdkinney/edk2/tree/Bug_3750_IncreaseAzurePipelinesTimeout
> > <https://github.com/mdkinney/edk2/tree/Bug_3750_IncreaseAzurePipelinesTimeout>
> > >
> > > *https://github.com/tianocore/edk2/pull/2219 <https://github.com/tianocore/edk2/pull/2219>
> > >
> > > * Required to allow EccCheck of uncrustify changes to complete on Azure
> > >
> > > Pipelines CI agents without timing out.
> > >
> > > * Status: Review complete.PR pushed.
> > >
> > > 5) Update Package YAML to ignore specific ECC files/errors
> > >
> > > *https://bugzilla.tianocore.org/show_bug.cgi?id=3749 <https://bugzilla.tianocore.org/show_bug.cgi?id=3749>
> > >
> > > *https://github.com/mdkinney/edk2/tree/Bug_3749_EccCheckIgnoreFilesErrors
> > <https://github.com/mdkinney/edk2/tree/Bug_3749_EccCheckIgnoreFilesErrors>
> > >
> > > *https://github.com/tianocore/edk2/pull/2218 <https://github.com/tianocore/edk2/pull/2218>
> > >
> > > * Required to pass EccCheck
> > >
> > > * Status: Review complete. PR pushed
> > >
> > > 6) Uncrustify Source Changes
> > >
> > > *https://bugzilla.tianocore.org/show_bug.cgi?id=3737 <https://bugzilla.tianocore.org/show_bug.cgi?id=3737>
> > >
> > > *https://bugzilla.tianocore.org/show_bug.cgi?id=3739 <https://bugzilla.tianocore.org/show_bug.cgi?id=3739>
> > >
> > > *https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
> > <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5>
> > >
> > > *https://github.com/tianocore/edk2/pull/2229 <https://github.com/tianocore/edk2/pull/2229>
> > >
> > > * Build comparison result PASS:https://github.com/mdkinney/edk2/actions/runs/1521618836
> > <https://github.com/mdkinney/edk2/actions/runs/1521618836>
> > >
> > > * EFI_D_ -> DEBUG changes required to pass PatchCheck
> > >
> > > * Uncrustify format changes required to pass UncrustifyCheck
> > >
> > > * Status:
> > >
> > > Waiting for review
> > >
> > > 7) UncrustifyCheck EDK II CI Plugin
> > >
> > > *https://bugzilla.tianocore.org/show_bug.cgi?id=3748 <https://bugzilla.tianocore.org/show_bug.cgi?id=3748>
> > >
> > > *https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v6
> > <https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v6>
> > >
> > > * Required to enforce all PRs submitted to EDK II CI match uncrustify format.
> > >
> > > * Unconditionally checks all packages.Can not be committed until all C/H
> > >
> > > source files have been updated.
> > >
> > > * Status: Review complete
> > >
> > > Combined Branch/PR for Review/Test
> > >
> > > ==================================
> > >
> > > * Build Comparison results must pass 100% across the full set of PRs before
> > >
> > > the individual PRs can be pushed in the order listed above.
> > >
> > > * Branch:https://github.com/mdkinney/edk2/tree/TestOnly_Uncrustify_PR_Series
> > <https://github.com/mdkinney/edk2/tree/TestOnly_Uncrustify_PR_Series>
> > >
> > > * PR:https://github.com/tianocore/edk2/pull/2229 <https://github.com/tianocore/edk2/pull/2229>
> > >
> > > Status = PASS
> > >
> > > * CompareBuild:
> > >
> > > Branch:https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
> > <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5>
> > >
> > > --ref1:ef9a059cdb15844fe52a49af2bf7d86b9dd3e9bf
> > >
> > > --ref2:Bug_3737_3739_ApplyUncrustifyChanges_V5
> > >
> > > Extra Options: -n 4 --quiet
> > >
> > > Results:https://github.com/mdkinney/edk2/actions/runs/1521618836
> > <https://github.com/mdkinney/edk2/actions/runs/1521618836>
> > >
> > > 30 VS2019 build comparisons PASS
> > >
> > > 40 GCC5 build comparisons PASS
> > >
> > > 100% PASS
> > >
> > > The following git log shows the set of patches from --ref1 to --ref 2across
> > >
> > > which there are no differences in any of the OBJ/LIB/DLL/EFI/FFS/FV/FD files.
> > >
> > > --ref2
> > >
> > > b7d4bf0675b7 (HEAD -> Bug_3737_3739_ApplyUncrustifyChanges_V5) UnitTestFrameworkPkg: Apply uncrusitify changes
> > >
> > > 7f03d25f60e7 UefiPayloadPkg: Apply uncrusitify changes
> > >
> > > 0bfd8d9b5ac9 UefiCpuPkg: Apply uncrusitify changes
> > >
> > > e1cd9bfb9dea StandaloneMmPkg: Apply uncrusitify changes
> > >
> > > 5da2f65be378 SourceLevelDebugPkg: Apply uncrusitify changes
> > >
> > > 95b86de07e5d SignedCapsulePkg: Apply uncrusitify changes
> > >
> > > fe71d97246c4 ShellPkg: Apply uncrusitify changes
> > >
> > > 54c21c952992 SecurityPkg: Apply uncrusitify changes
> > >
> > > 187a3785f12b RedfishPkg: Apply uncrusitify changes
> > >
> > > 810100002a46 PcAtChipsetPkg: Apply uncrusitify changes
> > >
> > > 276a695c0cf2 OvmfPkg: Apply uncrusitify changes
> > >
> > > 303c0a91ab07 NetworkPkg: Apply uncrusitify changes
> > >
> > > bc80792cd1b1 MdePkg: Apply uncrusitify changes
> > >
> > > 3ea86be17a2a MdeModulePkg: Apply uncrusitify changes
> > >
> > > c70ef11ed0cd IntelFsp2WrapperPkg: Apply uncrusitify changes
> > >
> > > c0291221f252 IntelFsp2Pkg: Apply uncrusitify changes
> > >
> > > 6a479952a690 FmpDevicePkg: Apply uncrusitify changes
> > >
> > > 3a7c05b7070d FatPkg: Apply uncrusitify changes
> > >
> > > b789f98c8959 EmulatorPkg: Apply uncrusitify changes
> > >
> > > 952d7a1c9220 EmbeddedPkg: Apply uncrusitify changes
> > >
> > > a1cc9881bab6 DynamicTablesPkg: Apply uncrusitify changes
> > >
> > > 50654dfe5785 CryptoPkg: Apply uncrusitify changes
> > >
> > > ed965a02dfa1 ArmVirtPkg: Apply uncrusitify changes
> > >
> > > 9744023fbc46 ArmPlatformPkg: Apply uncrusitify changes
> > >
> > > 7a1cde5f5bba ArmPkg: Apply uncrusitify changes
> > >
> > > 19d17e0913e8 UefiCpuPkg: Change use of EFI_D_* to DEBUG_*
> > >
> > > ffa718b4f994 SourceLevelDebugPkg: Change use of EFI_D_* to DEBUG_*
> > >
> > > b86cb3c5e5b4 ShellPkg: Change use of EFI_D_* to DEBUG_*
> > >
> > > c7c42204dc07 SecurityPkg: Change use of EFI_D_* to DEBUG_*
> > >
> > > 16b8e6f958e4 PcAtChipsetPkg: Change use of EFI_D_* to DEBUG_*
> > >
> > > 0ac3f8b2dac5 OvmfPkg: Change use of EFI_D_* to DEBUG_*
> > >
> > > bc5004b8d294 NetworkPkg: Change use of EFI_D_* to DEBUG_*
> > >
> > > 6f671a8e2377 MdePkg: Change use of EFI_D_* to DEBUG_*
> > >
> > > a10c610ff9a3 MdeModulePkg: Change use of EFI_D_* to DEBUG_*
> > >
> > > 09a3bddba390 FatPkg: Change use of EFI_D_* to DEBUG_*
> > >
> > > 59c61318246a EmulatorPkg: Change use of EFI_D_* to DEBUG_*
> > >
> > > 3a80367dda3b EmbeddedPkg: Change use of EFI_D_* to DEBUG_*
> > >
> > > 23eb1aaf80ca ArmVirtPkg: Change use of EFI_D_* to DEBUG_*
> > >
> > > 875914b45c54 ArmPlatformPkg: Change use of EFI_D_* to DEBUG_*
> > >
> > > eb2eca82b451 ArmPkg: Change use of EFI_D_* to DEBUG_*
> > >
> > > f0f3f5aae7c4 (origin/master, origin/HEAD, master) UnitTestFrameworkPkg: Update YAML to ignore specific ECC
> > files/errors
> > >
> > > c05734797790 UefiPayloadPkg: Update YAML to ignore specific ECC files/errors
> > >
> > > c30c40d6c63d StandaloneMmPkg: Update YAML to ignore specific ECC files/errors
> > >
> > > 9944508e85f1 ShellPkg: Update YAML to ignore specific ECC files/errors
> > >
> > > 60fa40be458d SecurityPkg: Update YAML to ignore specific ECC files/errors
> > >
> > > df790cd6b37e MdePkg: Update YAML to ignore specific ECC files/errors
> > >
> > > 9deb9370766e MdeModulePkg: Update YAML to ignore specific ECC files/errors
> > >
> > > d7d30e8f219f EmulatorPkg: Update YAML to ignore specific ECC files/errors
> > >
> > > d5744ecba813 CryptoPkg: Update YAML to ignore specific ECC files/errors
> > >
> > > c97fee87f0f9 ArmVirtPkg: Update YAML to ignore specific ECC files/errors
> > >
> > > 1939fc9569f2 ArmPlatformPkg: Update YAML to ignore specific ECC files/errors
> > >
> > > 365dced2c37a ArmPkg: Update YAML to ignore specific ECC files/errors
> > >
> > > 76a1ce4d5fec .azurepipelines/templates: Update max pipeline job time to 2 hours
> > >
> > > 99f84ff47390 .pytools/Plugin/LicenseCheck: Use temp directory for git diff output
> > >
> > > 3019f1bbabf1 .pytool/Plugin/EccCheck: Add performance optimizations
> > >
> > > 854462bd3479 .pytool/Plugin/EccCheck: Remove temp directory on exception
> > >
> > > 69877614fdee .pytool/Plugin/EccCheck: Remove RevertCode()
> > >
> > > --ref1
> > >
> > > ef9a059cdb15 EmulatorPkg/Win/Host: Update CC_FLAGS
> > >
> > > bb1bba3d7767 (tag: edk2-stable202111) NetworkPkg: Fix invalid pointer for DNS response token on error
> > >
> > > Best regards,
> > >
> > > Mike
> > >
> > >
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
2021-12-03 0:18 ` Michael D Kinney
@ 2021-12-03 0:31 ` Michael Kubacki
2021-12-03 0:53 ` Michael D Kinney
0 siblings, 1 reply; 39+ messages in thread
From: Michael Kubacki @ 2021-12-03 0:31 UTC (permalink / raw)
To: Kinney, Michael D, devel@edk2.groups.io,
maciej.rabeda@linux.intel.com, Michael Kubacki,
Andrew Fish (afish@apple.com), Leif Lindholm
Hi Mike,
Thank you for the detailed analysis and recommendations.
I agree with the recommendations and I will try to have an Uncrustify
tool update by tomorrow for (4).
That will require an update in uncrustify_ext_dep.yaml to pull in the
new version. I'm assuming I should also update uncrustify.cfg to set
align_assign_thresh = 0 in that new patch series?
Regards,
Michael
On 12/2/2021 7:18 PM, Kinney, Michael D wrote:
> Hi Michael,
>
> CORRECTION: set align_assign_threshold to 0.
>
> Reponses inline below.
>
> I would like to summarize the 4 issues raised in the past day along with the recommendations.
>
> 1) Exclusion feature for UncrustifyCheck. There are 2 directories with 8 files total that
> maintainers have noted they would like to see not go through uncrustify formatting. Today
> the only content that is skipped is BaseTools and submodules.
>
> Adding a general purpose exclusion feature would then require all developers to make
> sure their method of using uncrustify also excludes those same areas. This requires
> extra steps for all developers and maintainers.
>
> If we do not add the exclusion feature, then the 8 files will require an extra step
> to sync with the original source of those files. The rate of changes of these 8 files
> is very low today.
>
> RECOMMENDATION: Do not add exclusion feature at this time. Revisit if the extra work
> to maintain the files that would be candidates for exclusions increases significantly.
>
> 2) Alignment of assignments. The threshold of 4 characters appears to be too low and causes
> source files that are already aligned to become unaligned.
>
> RECOMMENDATION: Change threshold to the default value of 0 which means no limit.
>
> align_assign_thresh= 0
>
> 3) Alignment of parameters in function declaration not correct. The root cause of this
> is the use of the OPTIONAL keyword. If the OPTIONAL keyword is removed, then the
> alignment is correct. The alignment is also correct if the OPTIONAL keyword appears
> before the ','. If the OPTIONAL keyword appears after the ',', then the format is
> not correct. The OPTIONAL keyword indicates that the parameter in the function is
> not required and may be passed in as NULL or 0 or some other default value defined by
> the API. It makes more sense for this OPTIONAL keyword that follows the parameter
> names to appear before the ',' so it is scoped to the parameter on that line. If it
> appears after the ',', then C parsers thinks it is a prefix (IN, OUT, CONST, volatile,
> static) for the next parameter in the function.
>
> RECOMMENDATION: Update patch series with a global search and replace so OPTIONAL
> keyword always appears before the ',' on the same line.
>
> RegEx search string: ',( *)OPTIONAL( *)'
> RegEx replace string: ' $1OPTIONAL,$2'
>
> 4) Format issues with complex blocks in DEBUG_CODE(), or between DEBUG_CODE_BEGIN() and
> DEBUG_CODE_END(). Uncrustify treats these as macros and is not aware that the
> parameter passed into the macro call is a block of C code that needs to be formatted.
> Complex blocks with if/while/for/case statements are impacted the most.
>
> RECOMMENDATION: Update the uncrustify with an edk2 specific extension to treat these
> macros as a block of code as if they were surrounded by an extra set of braces {}.
>
>
> I have posted a branch for testing purposes that implements (2) and (3).
>
> Branch: https://github.com/mdkinney/edk2/tree/TestOnly_Bug_3737_3739_ApplyUncrustifyChanges_V6_OPTIONAL_Keyword_Fix
> PR: https://github.com/tianocore/edk2/pull/2233
> Status: PASS
> CompareBuild: https://github.com/mdkinney/edk2/actions/runs/1532855914
> Status: PASS
>
> You can see what changed by fetching and comparing the following 2 branches:
>
> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
> https://github.com/mdkinney/edk2/tree/TestOnly_Bug_3737_3739_ApplyUncrustifyChanges_V6_OPTIONAL_Keyword_Fix
>
> Please provide feedback on the RECOMMENDATIONS above. I will go ahead and prepare of V6 version of
> the patch series now that that test results are all PASS.
>
> Best regards,
>
> Mike
>
>
>> -----Original Message-----
>> From: Kinney, Michael D <michael.d.kinney@intel.com>
>> Sent: Thursday, December 2, 2021 4:15 PM
>> To: Michael Kubacki <mikuback@linux.microsoft.com>; devel@edk2.groups.io; maciej.rabeda@linux.intel.com; Michael Kubacki
>> <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com) <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>;
>> Kinney, Michael D <michael.d.kinney@intel.com>
>> Subject: RE: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
>>
>> Hi Michael,
>>
>> Reponses inline below.
>>
>> I would like to summarize the 4 issues raised in the past day along with the recommendations.
>>
>> 1) Exclusion feature for UncrustifyCheck. There are 2 directories with 8 files total that
>> maintainers have noted they would like to see not go through uncrustify formatting. Today
>> the only content that is skipped is BaseTools and submodules.
>>
>> Adding a general purpose exclusion feature would then require all developers to make
>> sure their method of using uncrustify also excludes those same areas. This requires
>> extra steps for all developers and maintainers.
>>
>> If we do not add the exclusion feature, then the 8 files will require an extra step
>> to sync with the original source of those files. The rate of changes of these 8 files
>> is very low today.
>>
>> RECOMMENDATION: Do not add exclusion feature at this time. Revisit if the extra work
>> to maintain the files that would be candidates for exclusions increases significantly.
>>
>> 2) Alignment of assignments. The threshold of 4 characters appears to be too low and causes
>> source files that are already aligned to become unaligned.
>>
>> RECOMMENDATION: Change threshold to the default value of 0 which means no limit.
>>
>> align_assign_thresh= 4
>>
>> 3) Alignment of parameters in function declaration not correct. The root cause of this
>> is the use of the OPTIONAL keyword. If the OPTIONAL keyword is removed, then the
>> alignment is correct. The alignment is also correct if the OPTIONAL keyword appears
>> before the ','. If the OPTIONAL keyword appears after the ',', then the format is
>> not correct. The OPTIONAL keyword indicates that the parameter in the function is
>> not required and may be passed in as NULL or 0 or some other default value defined by
>> the API. It makes more sense for this OPTIONAL keyword that follows the parameter
>> names to appear before the ',' so it is scoped to the parameter on that line. If it
>> appears after the ',', then C parsers thinks it is a prefix (IN, OUT, CONST, volatile,
>> static) for the next parameter in the function.
>>
>> RECOMMENDATION: Update patch series with a global search and replace so OPTIONAL
>> keyword always appears before the ',' on the same line.
>>
>> RegEx search string: ',( *)OPTIONAL( *)'
>> RegEx replace string: ' $1OPTIONAL,$2'
>>
>> 4) Format issues with complex blocks in DEBUG_CODE(), or between DEBUG_CODE_BEGIN() and
>> DEBUG_CODE_END(). Uncrustify treats these as macros and is not aware that the
>> parameter passed into the macro call is a block of C code that needs to be formatted.
>> Complex blocks with if/while/for/case statements are impacted the most.
>>
>> RECOMMENDATION: Update the uncrustify with an edk2 specific extension to treat these
>> macros as a block of code as if they were surrounded by an extra set of braces {}.
>>
>>
>> I have posted a branch for testing purposes that implements (2) and (3).
>>
>> Branch: https://github.com/mdkinney/edk2/tree/TestOnly_Bug_3737_3739_ApplyUncrustifyChanges_V6_OPTIONAL_Keyword_Fix
>> PR: https://github.com/tianocore/edk2/pull/2233
>> Status: PASS
>> CompareBuild: https://github.com/mdkinney/edk2/actions/runs/1532855914
>> Status: PASS
>>
>> You can see what changed by fetching and comparing the following 2 branches:
>>
>> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
>> https://github.com/mdkinney/edk2/tree/TestOnly_Bug_3737_3739_ApplyUncrustifyChanges_V6_OPTIONAL_Keyword_Fix
>>
>> Please provide feedback on the RECOMMENDATIONS above. I will go ahead and prepare of V6 version of
>> the patch series now that that test results are all PASS.
>>
>> Best regards,
>>
>> Mike
>>
>>
>>> -----Original Message-----
>>> From: Michael Kubacki <mikuback@linux.microsoft.com>
>>> Sent: Thursday, December 2, 2021 1:57 PM
>>> To: devel@edk2.groups.io; Kinney, Michael D <michael.d.kinney@intel.com>; maciej.rabeda@linux.intel.com; Michael Kubacki
>>> <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com) <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>
>>> Subject: Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
>>>
>>> My reply is inline.
>>>
>>> Regards,
>>> Michael
>>>
>>> On 12/2/2021 2:45 PM, Michael D Kinney wrote:
>>>> Hi Maciej,
>>>>
>>>> Thanks for the feedback.
>>>>
>>>> * Example #1.This appears to be caused by the following uncrustify
>>>> settings:
>>>>
>>>> # The threshold for aligning on '=' in assignments.
>>>>
>>>> # Use a negative number for absolute thresholds.
>>>>
>>>> #
>>>>
>>>> # 0: No limit (default).
>>>>
>>>> align_assign_thresh= 0# number
>>>>
>>>> The edk2 setting for this is:
>>>>
>>>> align_assign_thresh= 4
>>>>
>>>> This means blocks of assignments that are different than more than 4
>>>> spaces will be considered a new block.
>>>>
>>>> ‘HwAddreLen’ and ‘Xid’ are more than 4 characters in length
>>>> different.Same for ‘Xid’ and ‘Reserved’.So
>>>>
>>>> uncrustify treats these as 3 different assignment blocks.
>>>>
>>>> If we change to the default value of 0: No limit, this example is
>>>> treated as a single block and all ‘=’ are aligned.
>>>>
>>>> Is there a reason ‘4’ was selected?Is there is any harm in using the
>>>> default of 0?
>>>>
>>> We can certainly change the threshold. '4' was derived by
>>> experimentation. This is an area that is somewhat subjective and every
>>> case is difficult to cover well. Please feel free to suggest any changes.
>>>
>>
>> I recommend we use the default value of 0. That way, any code that choose to
>> align assignments will still be aligned. If a developer does not like
>> short assignments and long assignments in the same code block to use the
>> long assignment column, they can always break the block up into multiple
>> blocks by adding a carriage return.
>>
>>> For the benefit of others, this file can be a useful refernce
>>> understanding spans, gaps, and thresholds.
>>>
>>> https://github.com/uncrustify/uncrustify/blob/master/documentation/htdocs/configuration.txt
>>>
>>>> * Example #2: Uncruistfy is confused by the DEBUG_CODE() macro.This is
>>>> not a traditional macro because the
>>>>
>>>> contents of the macro is a block of C code.I do not know how to convince
>>>> uncrustify that the contents of a
>>>>
>>>> macro like function call to be treated as a code block from an indent
>>>> perspective.
>>>>
>>> I believe this would have to be overridden in the Uncrustify fork since
>>> DEBUG_CODE() is being treated as a macro function call and code blocks
>>> are not expected there. In addition, some special treatment might be
>>> needed for alignment in between DEBUG_CODE_BEGIN()/DEBUG_CODE_END().
>>>
>>> I'm happy to look at this. However, regression validation can take a
>>> while so I'd like to make sure we need this now. In cases that do not
>>> have additional code blocks, it seems to format fairly well. Is this
>>> prevalent and impactful enough it must be fixed now? Or, could we
>>> revisit it with a follow up patch?
>>
>> How long does regression testing take? I see about 20 instances of
>> DEBUG_CODE() that are producing bad formatting. If the content inside
>> DEBUG_CODE() is a single line of a single block of statements without
>> any if/while/for/case statements that require further indent, then the
>> format looks ok.
>>
>> DEBUG_CODE_BEGIN() and DEBUG_CODE_END() would look better if the
>> contents between are also indented one level.
>>
>>>
>>>> * Example #3/#4: Uncrustify is confused by the OPTIONAL keyword.The
>>>> edk2 config declares it as a QUALIFIER
>>>>
>>>> like IN and OUT.However, OPTIONAL only appears at the end of the line
>>>> that declares a parameter to a
>>>>
>>>> function.There are 3 forms. One with comma after OPTIONAL.One with comma
>>>> before OPTIONAL, and
>>>>
>>>> one with no comma if the parameter is the last parameter in the function
>>>> declaration.
>>>>
>>>> TYPE ParamName OPTIONAL,
>>>>
>>>> TYPEParamName, OPTIONAL
>>>>
>>>> TYPEParamName OPTIONAL
>>>>
>>>> OPTIONAL is defined to nothing in edk2 builds.From a uncrustify
>>>> perspective, we really want is to be
>>>>
>>>> ignored or more correctly treated it as a token that is attached to
>>>> ParamName and the combination of
>>>>
>>>> ParamName and OPTIONAL treated as one unit to determine indents.
>>>>
>>> "TYPE ParamName, OPTIONAL" seems especially problematic and presents an
>>> inconsistency with the other formats. Mike, can you please let me know
>>> if you have the same observation and your thoughts on a consistent pattern?
>>>
>>>> Thanks,
>>>>
>>>> Mike
>>>>
>>>> *From:*devel@edk2.groups.io <devel@edk2.groups.io> *On Behalf Of *Maciej
>>>> Rabeda
>>>> *Sent:* Thursday, December 2, 2021 10:27 AM
>>>> *To:* devel@edk2.groups.io; Kinney, Michael D
>>>> <michael.d.kinney@intel.com>; Michael Kubacki
>>>> <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com)
>>>> <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>
>>>> *Subject:* Re: [edk2-devel] Uncrustify Conversion Detailed Plan and
>>>> Extended Hard Freeze Update #4
>>>>
>>>> Hey Mike,
>>>>
>>>> While most of the changes related to fixing coding style violations,
>>>> there are a couple of changes to NetworkPkg in that PR that make the
>>>> code less readable. Examples below.
>>>>
>>>> Example 1:
>>>>
>>>>
>>>>
>>>> Example 2:
>>>>
>>>>
>>>> Example 3:
>>>>
>>>>
>>>> Example 4:
>>>>
>>>> On 30-Nov-21 23:34, Michael D Kinney wrote:
>>>>
>>>> Hello,
>>>>
>>>> Thank you for your patience during this extended hard freeze.
>>>>
>>>> Just one more step to go.There has been a delay in the review of
>>>>
>>>> the patch series with the uncrustify source changes.PR(6).This
>>>>
>>>> patch series was not sent out as patch review email because of its
>>>>
>>>> very large size.It only contains source style changes and the
>>>>
>>>> CompareBuild tool and GitHub action has shown there are no binary
>>>>
>>>> differences introduced with these source style changes.
>>>>
>>>> If you are a package maintainer, then please review the following
>>>>
>>>> branch/PR for your package contents and review the EDK II CI results
>>>>
>>>> and BuildCompare results.I do not expect a line by line review
>>>>
>>>> because we already had time to provide feedback on the source style
>>>>
>>>> performed by uncrustify.Instead, a Reviewed-by for your package
>>>>
>>>> indicates that you have reviewed the EDK II CI results and CompareBuild
>>>>
>>>> tool functionality and results and you accept the source style
>>>>
>>>> changes to your package.
>>>>
>>>> *https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
>>> <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5>
>>>>
>>>> *https://github.com/tianocore/edk2/pull/2229 <https://github.com/tianocore/edk2/pull/2229>
>>>>
>>>> *https://github.com/mdkinney/edk2/actions/runs/1521618836
>>> <https://github.com/mdkinney/edk2/actions/runs/1521618836>
>>>>
>>>> Additional details on this update below.
>>>>
>>>> Thank you,
>>>>
>>>> Mike
>>>>
>>>> Changes from Update #3
>>>>
>>>> ----------------------------------------------------------------------------
>>>>
>>>> * Pushed PR (5)
>>>>
>>>> * Added link to PR(6). EDK II CI Status is PASS. Build Compare PASS.
>>>>
>>>> * Waiting for review of PR (6)
>>>>
>>>> * Review of PR (7) completed and waiting for review of PR (6)
>>>>
>>>> ----------------------------------------------------------------------------
>>>>
>>>> Changes from Update #2
>>>>
>>>> ----------------------------------------------------------------------------
>>>>
>>>> * Changed order of PRs swapping (4) and (5).The PR that activates
>>>>
>>>> increases the max CI agent job time is independent of all the other
>>>>
>>>> PRs and its review is complete, so it can be committed now.
>>>>
>>>> * Pushed PRs (1), (2), (3), (4).
>>>>
>>>> * Waiting for review to complete for PRs (5) and (6)
>>>>
>>>> * Reviews complete for PR (7)
>>>>
>>>> * Identifies steps using git filter-branch to apply uncrustify changes to a
>>>>
>>>> code review patch series that was generated before the uncrustify changes
>>>>
>>>> avoiding manual merge.
>>>>
>>>> * Identified steps using git filter-repo to generate an alternate history of
>>>>
>>>> the edk2 repo with uncrustify changes applied on every commit.This may
>>>>
>>>> be useful when evaluating changes to files using tools like git blame
>>>>
>>>> without the large uncrustify patch series.
>>>>
>>>> ---------------------------------------------------------------------------
>>>>
>>>> Changes from Update #1
>>>>
>>>> ----------------------------------------------------------------------------
>>>>
>>>> * Changed order of PRs swapping (6) and (7).The PR that activates
>>>>
>>>> EDK II CI check UncrustifyCheck has to be last because it unconditionally
>>>>
>>>> checks all C/H files in all packages.Not just files that have been
>>>>
>>>> modified like some of the other checkers.
>>>>
>>>> * Updated link to the branch with the UncrustifyCheck plugin that has been
>>>>
>>>> updated with a one line change and Reviewed-by and Tested-by tags.
>>>>
>>>> https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v6
>>> <https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v6>
>>>>
>>>> * Reviews complete for (1), (2), (3), (5), and (7)
>>>>
>>>> ---------------------------------------------------------------------------
>>>>
>>>> Michael Kubacki and I have prepared the patches required to apply the
>>>>
>>>> uncrustify changes and enable EDK II CI to check all submitted
>>>>
>>>> patches have been run through uncrustify.
>>>>
>>>> We have verified through the CompareBuild GitHub Action that the
>>>>
>>>> format changes performed by uncrustify have no functional changes.
>>>>
>>>> All of the OBJ, LIB, DLL, EFI, FFS, FV, and FD files match 100%
>>>>
>>>> across 70 VS2019/GCC5 builds of all package/platform DSC files in
>>>>
>>>> the edk2 repo.
>>>>
>>>> The hard freeze will be extended after the edk2-stable202111 tag until
>>>>
>>>> all uncrustify related changes are committed.We do not expect this
>>>>
>>>> to take more than a few days.Do not push any PRs until the hard
>>>>
>>>> freeze is lifted.
>>>>
>>>> The changes are broken up into 7 patch series/PRs.The PRs are ordered
>>>>
>>>> so they can be submitted using the normal submission process and EDK II
>>>>
>>>> CI will pass for each one.Details are listed below.
>>>>
>>>> Uncrustify 73.0.3 for EDK II
>>>>
>>>> =============================
>>>>
>>>> * Sources:https://dev.azure.com/projectmu/_git/Uncrustify <https://dev.azure.com/projectmu/_git/Uncrustify>
>>>>
>>>> * Documentation:https://dev.azure.com/projectmu/Uncrustify/_wiki/wikis/Uncrustify.wiki/1/Project-Mu-(EDK-II)-Fork-
>>> Readme <https://dev.azure.com/projectmu/Uncrustify/_wiki/wikis/Uncrustify.wiki/1/Project-Mu-(EDK-II)-Fork-Readme>
>>>>
>>>> * Download:https://dev.azure.com/projectmu/Uncrustify/_packaging?_a=package&feed=mu_uncrustify&package=mu-
>>> uncrustify-release&protocolType=NuGet&version=73.0.3
>>> <https://dev.azure.com/projectmu/Uncrustify/_packaging?_a=package&feed=mu_uncrustify&package=mu-uncrustify-
>>> release&protocolType=NuGet&version=73.0.3>
>>>>
>>>> Installing Uncrustify
>>>>
>>>> ======================
>>>>
>>>> The Uncrustify tool is installed automatically when the Pytools
>>>>
>>>> environment is used and the stuart* commands are run to complete the
>>>>
>>>> environment setup.Please see:
>>>>
>>>> https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally
>>> <https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally>
>>>>
>>>> Uncrustify can also be installed from the download page listed above
>>>>
>>>> or built from sources from the source link above.
>>>>
>>>> The Documentation link provides instruction on how to run uncrustify from
>>>>
>>>> the command line or install as a Visual Studio Code plugin.The main
>>>>
>>>> uncrustify documentation also describes how to integrate with a few other
>>>>
>>>> editors.
>>>>
>>>> We have also discussed a client side githook.That effort has not started.
>>>>
>>>> Let us know if that is a feature you would find useful.
>>>>
>>>> Developer impact for new code reviews
>>>>
>>>> ======================================
>>>>
>>>> Once the uncrustify checker is active in EDK II CI, developers must
>>>>
>>>> make sure their patches are run through the uncrustify tool before
>>>>
>>>> sending the patches for review.
>>>>
>>>> Developers must install and run uncrustify against changes files before
>>>>
>>>> sending patch review emails or submitting PR for EDK II CI.If EDK II CI
>>>>
>>>> detects and differences in source formatting, then EDK II CI will fail
>>>>
>>>> and the developer must run uncrustify and resubmit the patches.
>>>>
>>>> Developer impact to patch series/PRs reviewed during edk2-stable201121 soft/hard freeze
>>>>
>>>> =======================================================================================
>>>>
>>>> Developers must rebase their changes after the uncrustify source changes are
>>>>
>>>> committed.The branch with a preview of the uncrustify changes can be used
>>>>
>>>> to start this rebase work.
>>>>
>>>> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
>>> <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5>
>>>>
>>>> The following steps can be used to update an existing branch with the
>>>>
>>>> required uncrustify format.This is the Windows version.I will add
>>>>
>>>> the Linux version soon.
>>>>
>>>> 1) Fetch and checkout and rebase to latest edk2/master
>>>>
>>>> git fetch origin
>>>>
>>>> git checkout master
>>>>
>>>> git rebase origin/master
>>>>
>>>> 2) Make a backup copy of plugin UncrustifyCheck outside WORKSPACE.
>>>>
>>>> (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable and
>>>>
>>>> EDK II specific uncrustify configuration file available when working
>>>>
>>>> with a branch that does not have those tools in its scope.
>>>>
>>>> xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
>>>>
>>>> 3) Check out the patch series branch (e.g. MyBranch)
>>>>
>>>> git checkout MyBranch
>>>>
>>>> 4) Rebase patch series against edk2-stable202111
>>>>
>>>>
>>>>
>>>> git rebase edk2-stable202111
>>>>
>>>> 5) Create new branch for the uncrustifed version (e.g. MyBranch_Uncrustified)
>>>>
>>>> git checkout -b MyBranch_Uncrustified
>>>>
>>>> 6) Use git filter-branch to uncrustify all the commits in the series
>>>>
>>>> between the rebase target from (2) and HEAD of the branch.A filter
>>>>
>>>> can be used to scope the uncrustify operations to only the C/H files
>>>>
>>>> in the specific package the patch series is against. (e.g. DynamicTablesPkg).
>>>>
>>>> BaseTools should always be excluded.If the package scoped filter is
>>>>
>>>> not used, it will still work, but will take longer to run because
>>>>
>>>> uncrustify will rescan every C/H files in the whole repo.
>>>>
>>>> git filter-branch --tree-filter "git ls-files DynamicTablesPkg*.c DynamicTablesPkg*.h :!BaseTools/* |
>>> c:\\Temp\\UncrustifyCheck\\mu-uncrustify-release_extdep\\Windows-x86\\uncrustify.exe -c
>>> c:\\Temp\\UncrustifyCheck\\uncrustify.cfg -F - --replace --no-backup --if-changed" edk2-stable202111..HEAD
>>>>
>>>> 7) Now that all the individual patches in the branch are uncrustified,
>>>>
>>>> rebase against latest edk2/master that is already uncrustified.
>>>>
>>>> git rebase master
>>>>
>>>> 8) Verify the patches in this new branch.
>>>>
>>>> Impacts to tracing history across the uncrusity changes
>>>>
>>>> =======================================================
>>>>
>>>> Tools the view file and line history do work with the large uncrustify
>>>>
>>>> patch series.One impact is that the operations can be very slow due
>>>>
>>>> to the large uncrustify patches.
>>>>
>>>> One option to provide a faster experience is to provide an alternate
>>>>
>>>> version of the edk2 repository as "documentation" that has the
>>>>
>>>> entire history re-written with uncrustify run on every commit.
>>>>
>>>> The tool called git-filter-repo can be used to perform this
>>>>
>>>> transformation and runs in a reasonable period of time (a few hours)
>>>>
>>>> https://github.com/newren/git-filter-repo <https://github.com/newren/git-filter-repo>
>>>>
>>>> https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-demos/lint-history
>>> <https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-demos/lint-history>
>>>>
>>>> The following steps can be used to perform this transformation.
>>>>
>>>> This is the Windows version. I will add the Linux version soon.
>>>>
>>>> ** WARNING **This operation modifies(rewrites) all the commits
>>>>
>>>> in the local copy of the repo.Do not perform
>>>>
>>>> these steps on a local repo you are using for
>>>>
>>>> active development.
>>>>
>>>> 1) Clone edk2 into a new directory (see **WARNING**)
>>>>
>>>> git clonehttps://github.com/tianocore/edk2.git <https://github.com/tianocore/edk2.git> edk2-uncrustified
>>>>
>>>> cd edk2-uncrustified
>>>>
>>>> 2) Setup python virtual env, install pytools, and run stuart commands
>>>>
>>>> to setup build environment which includes installing uncrustify tools.
>>>>
>>>> https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally
>>> <https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally>
>>>>
>>>> 3) Make a backup copy of plugin UncrustifyCheck outside WORKSPACE.
>>>>
>>>> (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable and
>>>>
>>>> EDK II specific uncrustify configuration file available when working
>>>>
>>>> with a branch that does not have those tools in its scope.
>>>>
>>>> xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
>>>>
>>>> 4) Use lint-history.py from git-filter-repo examples
>>>>
>>>> https://github.com/newren/git-filter-repo <https://github.com/newren/git-filter-repo>
>>>>
>>>> https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-demos/lint-history
>>> <https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-demos/lint-history>
>>>>
>>>> Line #127 - Add try except around subprocess.check_call() with except
>>>>
>>>> being pass.This is required because there are a few commits of C
>>>>
>>>> files in the edk2 repo that have incorrect C syntax and do not
>>>>
>>>> build with a C compiler and break the uncrustify parser.Skip reformat
>>>>
>>>> of C files that can not be parsed by uncrustify.These rare instances
>>>>
>>>> are addressed in the commit that fixes the C syntax error.
>>>>
>>>> Run this slightly modified version of lint-history.Include only
>>>>
>>>> C/H files and exclude directories that start with 'Tools' or 'BaseTools'.
>>>>
>>>> This step took about 2.2 hours on a laptop.
>>>>
>>>> lint-history.py
>>>>
>>>> --relevant "return (not filename.startswith(b'Tools') and not filename.startswith(b'BaseTools') and
>>> (filename.endswith(b'.c') or filename.endswith(b'.h')))"
>>>>
>>>> c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\mu-uncrustify-release_extdep\\Windows-x86\\uncrustify.exe -c
>>> c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\uncrustify.cfg --replace --no-backup --if-changed
>>>>
>>>> Order of PRs to apply during extended hard freeze
>>>>
>>>> ==================================================
>>>>
>>>> 1) Update EmulatorPkg Win Host [BuildOptions] MSFT CC_FLAGS to not force debug information
>>>>
>>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3747 <https://bugzilla.tianocore.org/show_bug.cgi?id=3747>
>>>>
>>>> *https://github.com/mdkinney/edk2/tree/Bug_3747_EmulatorPkg_WinHost_ReproducibleBuild
>>> <https://github.com/mdkinney/edk2/tree/Bug_3747_EmulatorPkg_WinHost_ReproducibleBuild>
>>>>
>>>> *https://github.com/tianocore/edk2/pull/2215 <https://github.com/tianocore/edk2/pull/2215>
>>>>
>>>> * Required for EmulatorPkg to pass CompareBuild for VS2019 IA32/X64 builds.
>>>>
>>>> * Status: Review complete.PR pushed.
>>>>
>>>> 2) EccCheck should not revert staged and local changes
>>>>
>>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=2986 <https://bugzilla.tianocore.org/show_bug.cgi?id=2986>
>>>>
>>>> *https://github.com/mdkinney/edk2/tree/Bug_2986_EccCheckRemoveGitRevert_V2
>>> <https://github.com/mdkinney/edk2/tree/Bug_2986_EccCheckRemoveGitRevert_V2>
>>>>
>>>> *https://github.com/tianocore/edk2/pull/2216 <https://github.com/tianocore/edk2/pull/2216>
>>>>
>>>> * Required for EDK II CI to complete in a reasonable period of time when
>>>>
>>>> processing the 4000+ source file style changes made by uncrustify.
>>>>
>>>> * Also fixes critical bugs that can potentially corrupt git state when
>>>>
>>>> EccCheck is run locally.
>>>>
>>>> * Status: Review complete.PR pushed.
>>>>
>>>> 3) Update pytool LicenseCheck plugin to use temp directory for diff output file
>>>>
>>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3746 <https://bugzilla.tianocore.org/show_bug.cgi?id=3746>
>>>>
>>>> *https://github.com/mdkinney/edk2/tree/Bug_3746_LicenseCheckUseDiffOutputFile_V2
>>> <https://github.com/mdkinney/edk2/tree/Bug_3746_LicenseCheckUseDiffOutputFile_V2>
>>>>
>>>> *https://github.com/tianocore/edk2/pull/2217 <https://github.com/tianocore/edk2/pull/2217>
>>>>
>>>> * Required to reduce EDK II CI build times.
>>>>
>>>> * Status: Review complete.PR pushed.
>>>>
>>>> 4) Update max job time from 60 min to 120 minutes in .azurepipelines/templates
>>>>
>>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3750 <https://bugzilla.tianocore.org/show_bug.cgi?id=3750>
>>>>
>>>> *https://github.com/mdkinney/edk2/tree/Bug_3750_IncreaseAzurePipelinesTimeout
>>> <https://github.com/mdkinney/edk2/tree/Bug_3750_IncreaseAzurePipelinesTimeout>
>>>>
>>>> *https://github.com/tianocore/edk2/pull/2219 <https://github.com/tianocore/edk2/pull/2219>
>>>>
>>>> * Required to allow EccCheck of uncrustify changes to complete on Azure
>>>>
>>>> Pipelines CI agents without timing out.
>>>>
>>>> * Status: Review complete.PR pushed.
>>>>
>>>> 5) Update Package YAML to ignore specific ECC files/errors
>>>>
>>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3749 <https://bugzilla.tianocore.org/show_bug.cgi?id=3749>
>>>>
>>>> *https://github.com/mdkinney/edk2/tree/Bug_3749_EccCheckIgnoreFilesErrors
>>> <https://github.com/mdkinney/edk2/tree/Bug_3749_EccCheckIgnoreFilesErrors>
>>>>
>>>> *https://github.com/tianocore/edk2/pull/2218 <https://github.com/tianocore/edk2/pull/2218>
>>>>
>>>> * Required to pass EccCheck
>>>>
>>>> * Status: Review complete. PR pushed
>>>>
>>>> 6) Uncrustify Source Changes
>>>>
>>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3737 <https://bugzilla.tianocore.org/show_bug.cgi?id=3737>
>>>>
>>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3739 <https://bugzilla.tianocore.org/show_bug.cgi?id=3739>
>>>>
>>>> *https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
>>> <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5>
>>>>
>>>> *https://github.com/tianocore/edk2/pull/2229 <https://github.com/tianocore/edk2/pull/2229>
>>>>
>>>> * Build comparison result PASS:https://github.com/mdkinney/edk2/actions/runs/1521618836
>>> <https://github.com/mdkinney/edk2/actions/runs/1521618836>
>>>>
>>>> * EFI_D_ -> DEBUG changes required to pass PatchCheck
>>>>
>>>> * Uncrustify format changes required to pass UncrustifyCheck
>>>>
>>>> * Status:
>>>>
>>>> Waiting for review
>>>>
>>>> 7) UncrustifyCheck EDK II CI Plugin
>>>>
>>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3748 <https://bugzilla.tianocore.org/show_bug.cgi?id=3748>
>>>>
>>>> *https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v6
>>> <https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v6>
>>>>
>>>> * Required to enforce all PRs submitted to EDK II CI match uncrustify format.
>>>>
>>>> * Unconditionally checks all packages.Can not be committed until all C/H
>>>>
>>>> source files have been updated.
>>>>
>>>> * Status: Review complete
>>>>
>>>> Combined Branch/PR for Review/Test
>>>>
>>>> ==================================
>>>>
>>>> * Build Comparison results must pass 100% across the full set of PRs before
>>>>
>>>> the individual PRs can be pushed in the order listed above.
>>>>
>>>> * Branch:https://github.com/mdkinney/edk2/tree/TestOnly_Uncrustify_PR_Series
>>> <https://github.com/mdkinney/edk2/tree/TestOnly_Uncrustify_PR_Series>
>>>>
>>>> * PR:https://github.com/tianocore/edk2/pull/2229 <https://github.com/tianocore/edk2/pull/2229>
>>>>
>>>> Status = PASS
>>>>
>>>> * CompareBuild:
>>>>
>>>> Branch:https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
>>> <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5>
>>>>
>>>> --ref1:ef9a059cdb15844fe52a49af2bf7d86b9dd3e9bf
>>>>
>>>> --ref2:Bug_3737_3739_ApplyUncrustifyChanges_V5
>>>>
>>>> Extra Options: -n 4 --quiet
>>>>
>>>> Results:https://github.com/mdkinney/edk2/actions/runs/1521618836
>>> <https://github.com/mdkinney/edk2/actions/runs/1521618836>
>>>>
>>>> 30 VS2019 build comparisons PASS
>>>>
>>>> 40 GCC5 build comparisons PASS
>>>>
>>>> 100% PASS
>>>>
>>>> The following git log shows the set of patches from --ref1 to --ref 2across
>>>>
>>>> which there are no differences in any of the OBJ/LIB/DLL/EFI/FFS/FV/FD files.
>>>>
>>>> --ref2
>>>>
>>>> b7d4bf0675b7 (HEAD -> Bug_3737_3739_ApplyUncrustifyChanges_V5) UnitTestFrameworkPkg: Apply uncrusitify changes
>>>>
>>>> 7f03d25f60e7 UefiPayloadPkg: Apply uncrusitify changes
>>>>
>>>> 0bfd8d9b5ac9 UefiCpuPkg: Apply uncrusitify changes
>>>>
>>>> e1cd9bfb9dea StandaloneMmPkg: Apply uncrusitify changes
>>>>
>>>> 5da2f65be378 SourceLevelDebugPkg: Apply uncrusitify changes
>>>>
>>>> 95b86de07e5d SignedCapsulePkg: Apply uncrusitify changes
>>>>
>>>> fe71d97246c4 ShellPkg: Apply uncrusitify changes
>>>>
>>>> 54c21c952992 SecurityPkg: Apply uncrusitify changes
>>>>
>>>> 187a3785f12b RedfishPkg: Apply uncrusitify changes
>>>>
>>>> 810100002a46 PcAtChipsetPkg: Apply uncrusitify changes
>>>>
>>>> 276a695c0cf2 OvmfPkg: Apply uncrusitify changes
>>>>
>>>> 303c0a91ab07 NetworkPkg: Apply uncrusitify changes
>>>>
>>>> bc80792cd1b1 MdePkg: Apply uncrusitify changes
>>>>
>>>> 3ea86be17a2a MdeModulePkg: Apply uncrusitify changes
>>>>
>>>> c70ef11ed0cd IntelFsp2WrapperPkg: Apply uncrusitify changes
>>>>
>>>> c0291221f252 IntelFsp2Pkg: Apply uncrusitify changes
>>>>
>>>> 6a479952a690 FmpDevicePkg: Apply uncrusitify changes
>>>>
>>>> 3a7c05b7070d FatPkg: Apply uncrusitify changes
>>>>
>>>> b789f98c8959 EmulatorPkg: Apply uncrusitify changes
>>>>
>>>> 952d7a1c9220 EmbeddedPkg: Apply uncrusitify changes
>>>>
>>>> a1cc9881bab6 DynamicTablesPkg: Apply uncrusitify changes
>>>>
>>>> 50654dfe5785 CryptoPkg: Apply uncrusitify changes
>>>>
>>>> ed965a02dfa1 ArmVirtPkg: Apply uncrusitify changes
>>>>
>>>> 9744023fbc46 ArmPlatformPkg: Apply uncrusitify changes
>>>>
>>>> 7a1cde5f5bba ArmPkg: Apply uncrusitify changes
>>>>
>>>> 19d17e0913e8 UefiCpuPkg: Change use of EFI_D_* to DEBUG_*
>>>>
>>>> ffa718b4f994 SourceLevelDebugPkg: Change use of EFI_D_* to DEBUG_*
>>>>
>>>> b86cb3c5e5b4 ShellPkg: Change use of EFI_D_* to DEBUG_*
>>>>
>>>> c7c42204dc07 SecurityPkg: Change use of EFI_D_* to DEBUG_*
>>>>
>>>> 16b8e6f958e4 PcAtChipsetPkg: Change use of EFI_D_* to DEBUG_*
>>>>
>>>> 0ac3f8b2dac5 OvmfPkg: Change use of EFI_D_* to DEBUG_*
>>>>
>>>> bc5004b8d294 NetworkPkg: Change use of EFI_D_* to DEBUG_*
>>>>
>>>> 6f671a8e2377 MdePkg: Change use of EFI_D_* to DEBUG_*
>>>>
>>>> a10c610ff9a3 MdeModulePkg: Change use of EFI_D_* to DEBUG_*
>>>>
>>>> 09a3bddba390 FatPkg: Change use of EFI_D_* to DEBUG_*
>>>>
>>>> 59c61318246a EmulatorPkg: Change use of EFI_D_* to DEBUG_*
>>>>
>>>> 3a80367dda3b EmbeddedPkg: Change use of EFI_D_* to DEBUG_*
>>>>
>>>> 23eb1aaf80ca ArmVirtPkg: Change use of EFI_D_* to DEBUG_*
>>>>
>>>> 875914b45c54 ArmPlatformPkg: Change use of EFI_D_* to DEBUG_*
>>>>
>>>> eb2eca82b451 ArmPkg: Change use of EFI_D_* to DEBUG_*
>>>>
>>>> f0f3f5aae7c4 (origin/master, origin/HEAD, master) UnitTestFrameworkPkg: Update YAML to ignore specific ECC
>>> files/errors
>>>>
>>>> c05734797790 UefiPayloadPkg: Update YAML to ignore specific ECC files/errors
>>>>
>>>> c30c40d6c63d StandaloneMmPkg: Update YAML to ignore specific ECC files/errors
>>>>
>>>> 9944508e85f1 ShellPkg: Update YAML to ignore specific ECC files/errors
>>>>
>>>> 60fa40be458d SecurityPkg: Update YAML to ignore specific ECC files/errors
>>>>
>>>> df790cd6b37e MdePkg: Update YAML to ignore specific ECC files/errors
>>>>
>>>> 9deb9370766e MdeModulePkg: Update YAML to ignore specific ECC files/errors
>>>>
>>>> d7d30e8f219f EmulatorPkg: Update YAML to ignore specific ECC files/errors
>>>>
>>>> d5744ecba813 CryptoPkg: Update YAML to ignore specific ECC files/errors
>>>>
>>>> c97fee87f0f9 ArmVirtPkg: Update YAML to ignore specific ECC files/errors
>>>>
>>>> 1939fc9569f2 ArmPlatformPkg: Update YAML to ignore specific ECC files/errors
>>>>
>>>> 365dced2c37a ArmPkg: Update YAML to ignore specific ECC files/errors
>>>>
>>>> 76a1ce4d5fec .azurepipelines/templates: Update max pipeline job time to 2 hours
>>>>
>>>> 99f84ff47390 .pytools/Plugin/LicenseCheck: Use temp directory for git diff output
>>>>
>>>> 3019f1bbabf1 .pytool/Plugin/EccCheck: Add performance optimizations
>>>>
>>>> 854462bd3479 .pytool/Plugin/EccCheck: Remove temp directory on exception
>>>>
>>>> 69877614fdee .pytool/Plugin/EccCheck: Remove RevertCode()
>>>>
>>>> --ref1
>>>>
>>>> ef9a059cdb15 EmulatorPkg/Win/Host: Update CC_FLAGS
>>>>
>>>> bb1bba3d7767 (tag: edk2-stable202111) NetworkPkg: Fix invalid pointer for DNS response token on error
>>>>
>>>> Best regards,
>>>>
>>>> Mike
>>>>
>>>>
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
2021-12-03 0:31 ` Michael Kubacki
@ 2021-12-03 0:53 ` Michael D Kinney
2021-12-03 2:22 ` Michael D Kinney
0 siblings, 1 reply; 39+ messages in thread
From: Michael D Kinney @ 2021-12-03 0:53 UTC (permalink / raw)
To: Michael Kubacki, devel@edk2.groups.io,
maciej.rabeda@linux.intel.com, Michael Kubacki,
Andrew Fish (afish@apple.com), Leif Lindholm, Kinney, Michael D
Michael,
Yes. Please update the patch series that adds UncrustifyCheck with those changes.
Thanks,
Mike
> -----Original Message-----
> From: Michael Kubacki <mikuback@linux.microsoft.com>
> Sent: Thursday, December 2, 2021 4:31 PM
> To: Kinney, Michael D <michael.d.kinney@intel.com>; devel@edk2.groups.io; maciej.rabeda@linux.intel.com; Michael Kubacki
> <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com) <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>
> Subject: Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
>
> Hi Mike,
>
> Thank you for the detailed analysis and recommendations.
>
> I agree with the recommendations and I will try to have an Uncrustify
> tool update by tomorrow for (4).
>
> That will require an update in uncrustify_ext_dep.yaml to pull in the
> new version. I'm assuming I should also update uncrustify.cfg to set
> align_assign_thresh = 0 in that new patch series?
>
> Regards,
> Michael
>
> On 12/2/2021 7:18 PM, Kinney, Michael D wrote:
> > Hi Michael,
> >
> > CORRECTION: set align_assign_threshold to 0.
> >
> > Reponses inline below.
> >
> > I would like to summarize the 4 issues raised in the past day along with the recommendations.
> >
> > 1) Exclusion feature for UncrustifyCheck. There are 2 directories with 8 files total that
> > maintainers have noted they would like to see not go through uncrustify formatting. Today
> > the only content that is skipped is BaseTools and submodules.
> >
> > Adding a general purpose exclusion feature would then require all developers to make
> > sure their method of using uncrustify also excludes those same areas. This requires
> > extra steps for all developers and maintainers.
> >
> > If we do not add the exclusion feature, then the 8 files will require an extra step
> > to sync with the original source of those files. The rate of changes of these 8 files
> > is very low today.
> >
> > RECOMMENDATION: Do not add exclusion feature at this time. Revisit if the extra work
> > to maintain the files that would be candidates for exclusions increases significantly.
> >
> > 2) Alignment of assignments. The threshold of 4 characters appears to be too low and causes
> > source files that are already aligned to become unaligned.
> >
> > RECOMMENDATION: Change threshold to the default value of 0 which means no limit.
> >
> > align_assign_thresh= 0
> >
> > 3) Alignment of parameters in function declaration not correct. The root cause of this
> > is the use of the OPTIONAL keyword. If the OPTIONAL keyword is removed, then the
> > alignment is correct. The alignment is also correct if the OPTIONAL keyword appears
> > before the ','. If the OPTIONAL keyword appears after the ',', then the format is
> > not correct. The OPTIONAL keyword indicates that the parameter in the function is
> > not required and may be passed in as NULL or 0 or some other default value defined by
> > the API. It makes more sense for this OPTIONAL keyword that follows the parameter
> > names to appear before the ',' so it is scoped to the parameter on that line. If it
> > appears after the ',', then C parsers thinks it is a prefix (IN, OUT, CONST, volatile,
> > static) for the next parameter in the function.
> >
> > RECOMMENDATION: Update patch series with a global search and replace so OPTIONAL
> > keyword always appears before the ',' on the same line.
> >
> > RegEx search string: ',( *)OPTIONAL( *)'
> > RegEx replace string: ' $1OPTIONAL,$2'
> >
> > 4) Format issues with complex blocks in DEBUG_CODE(), or between DEBUG_CODE_BEGIN() and
> > DEBUG_CODE_END(). Uncrustify treats these as macros and is not aware that the
> > parameter passed into the macro call is a block of C code that needs to be formatted.
> > Complex blocks with if/while/for/case statements are impacted the most.
> >
> > RECOMMENDATION: Update the uncrustify with an edk2 specific extension to treat these
> > macros as a block of code as if they were surrounded by an extra set of braces {}.
> >
> >
> > I have posted a branch for testing purposes that implements (2) and (3).
> >
> > Branch: https://github.com/mdkinney/edk2/tree/TestOnly_Bug_3737_3739_ApplyUncrustifyChanges_V6_OPTIONAL_Keyword_Fix
> > PR: https://github.com/tianocore/edk2/pull/2233
> > Status: PASS
> > CompareBuild: https://github.com/mdkinney/edk2/actions/runs/1532855914
> > Status: PASS
> >
> > You can see what changed by fetching and comparing the following 2 branches:
> >
> > https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
> > https://github.com/mdkinney/edk2/tree/TestOnly_Bug_3737_3739_ApplyUncrustifyChanges_V6_OPTIONAL_Keyword_Fix
> >
> > Please provide feedback on the RECOMMENDATIONS above. I will go ahead and prepare of V6 version of
> > the patch series now that that test results are all PASS.
> >
> > Best regards,
> >
> > Mike
> >
> >
> >> -----Original Message-----
> >> From: Kinney, Michael D <michael.d.kinney@intel.com>
> >> Sent: Thursday, December 2, 2021 4:15 PM
> >> To: Michael Kubacki <mikuback@linux.microsoft.com>; devel@edk2.groups.io; maciej.rabeda@linux.intel.com; Michael
> Kubacki
> >> <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com) <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>;
> >> Kinney, Michael D <michael.d.kinney@intel.com>
> >> Subject: RE: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
> >>
> >> Hi Michael,
> >>
> >> Reponses inline below.
> >>
> >> I would like to summarize the 4 issues raised in the past day along with the recommendations.
> >>
> >> 1) Exclusion feature for UncrustifyCheck. There are 2 directories with 8 files total that
> >> maintainers have noted they would like to see not go through uncrustify formatting. Today
> >> the only content that is skipped is BaseTools and submodules.
> >>
> >> Adding a general purpose exclusion feature would then require all developers to make
> >> sure their method of using uncrustify also excludes those same areas. This requires
> >> extra steps for all developers and maintainers.
> >>
> >> If we do not add the exclusion feature, then the 8 files will require an extra step
> >> to sync with the original source of those files. The rate of changes of these 8 files
> >> is very low today.
> >>
> >> RECOMMENDATION: Do not add exclusion feature at this time. Revisit if the extra work
> >> to maintain the files that would be candidates for exclusions increases significantly.
> >>
> >> 2) Alignment of assignments. The threshold of 4 characters appears to be too low and causes
> >> source files that are already aligned to become unaligned.
> >>
> >> RECOMMENDATION: Change threshold to the default value of 0 which means no limit.
> >>
> >> align_assign_thresh= 4
> >>
> >> 3) Alignment of parameters in function declaration not correct. The root cause of this
> >> is the use of the OPTIONAL keyword. If the OPTIONAL keyword is removed, then the
> >> alignment is correct. The alignment is also correct if the OPTIONAL keyword appears
> >> before the ','. If the OPTIONAL keyword appears after the ',', then the format is
> >> not correct. The OPTIONAL keyword indicates that the parameter in the function is
> >> not required and may be passed in as NULL or 0 or some other default value defined by
> >> the API. It makes more sense for this OPTIONAL keyword that follows the parameter
> >> names to appear before the ',' so it is scoped to the parameter on that line. If it
> >> appears after the ',', then C parsers thinks it is a prefix (IN, OUT, CONST, volatile,
> >> static) for the next parameter in the function.
> >>
> >> RECOMMENDATION: Update patch series with a global search and replace so OPTIONAL
> >> keyword always appears before the ',' on the same line.
> >>
> >> RegEx search string: ',( *)OPTIONAL( *)'
> >> RegEx replace string: ' $1OPTIONAL,$2'
> >>
> >> 4) Format issues with complex blocks in DEBUG_CODE(), or between DEBUG_CODE_BEGIN() and
> >> DEBUG_CODE_END(). Uncrustify treats these as macros and is not aware that the
> >> parameter passed into the macro call is a block of C code that needs to be formatted.
> >> Complex blocks with if/while/for/case statements are impacted the most.
> >>
> >> RECOMMENDATION: Update the uncrustify with an edk2 specific extension to treat these
> >> macros as a block of code as if they were surrounded by an extra set of braces {}.
> >>
> >>
> >> I have posted a branch for testing purposes that implements (2) and (3).
> >>
> >> Branch: https://github.com/mdkinney/edk2/tree/TestOnly_Bug_3737_3739_ApplyUncrustifyChanges_V6_OPTIONAL_Keyword_Fix
> >> PR: https://github.com/tianocore/edk2/pull/2233
> >> Status: PASS
> >> CompareBuild: https://github.com/mdkinney/edk2/actions/runs/1532855914
> >> Status: PASS
> >>
> >> You can see what changed by fetching and comparing the following 2 branches:
> >>
> >> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
> >> https://github.com/mdkinney/edk2/tree/TestOnly_Bug_3737_3739_ApplyUncrustifyChanges_V6_OPTIONAL_Keyword_Fix
> >>
> >> Please provide feedback on the RECOMMENDATIONS above. I will go ahead and prepare of V6 version of
> >> the patch series now that that test results are all PASS.
> >>
> >> Best regards,
> >>
> >> Mike
> >>
> >>
> >>> -----Original Message-----
> >>> From: Michael Kubacki <mikuback@linux.microsoft.com>
> >>> Sent: Thursday, December 2, 2021 1:57 PM
> >>> To: devel@edk2.groups.io; Kinney, Michael D <michael.d.kinney@intel.com>; maciej.rabeda@linux.intel.com; Michael
> Kubacki
> >>> <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com) <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>
> >>> Subject: Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
> >>>
> >>> My reply is inline.
> >>>
> >>> Regards,
> >>> Michael
> >>>
> >>> On 12/2/2021 2:45 PM, Michael D Kinney wrote:
> >>>> Hi Maciej,
> >>>>
> >>>> Thanks for the feedback.
> >>>>
> >>>> * Example #1.This appears to be caused by the following uncrustify
> >>>> settings:
> >>>>
> >>>> # The threshold for aligning on '=' in assignments.
> >>>>
> >>>> # Use a negative number for absolute thresholds.
> >>>>
> >>>> #
> >>>>
> >>>> # 0: No limit (default).
> >>>>
> >>>> align_assign_thresh= 0# number
> >>>>
> >>>> The edk2 setting for this is:
> >>>>
> >>>> align_assign_thresh= 4
> >>>>
> >>>> This means blocks of assignments that are different than more than 4
> >>>> spaces will be considered a new block.
> >>>>
> >>>> ‘HwAddreLen’ and ‘Xid’ are more than 4 characters in length
> >>>> different.Same for ‘Xid’ and ‘Reserved’.So
> >>>>
> >>>> uncrustify treats these as 3 different assignment blocks.
> >>>>
> >>>> If we change to the default value of 0: No limit, this example is
> >>>> treated as a single block and all ‘=’ are aligned.
> >>>>
> >>>> Is there a reason ‘4’ was selected?Is there is any harm in using the
> >>>> default of 0?
> >>>>
> >>> We can certainly change the threshold. '4' was derived by
> >>> experimentation. This is an area that is somewhat subjective and every
> >>> case is difficult to cover well. Please feel free to suggest any changes.
> >>>
> >>
> >> I recommend we use the default value of 0. That way, any code that choose to
> >> align assignments will still be aligned. If a developer does not like
> >> short assignments and long assignments in the same code block to use the
> >> long assignment column, they can always break the block up into multiple
> >> blocks by adding a carriage return.
> >>
> >>> For the benefit of others, this file can be a useful refernce
> >>> understanding spans, gaps, and thresholds.
> >>>
> >>> https://github.com/uncrustify/uncrustify/blob/master/documentation/htdocs/configuration.txt
> >>>
> >>>> * Example #2: Uncruistfy is confused by the DEBUG_CODE() macro.This is
> >>>> not a traditional macro because the
> >>>>
> >>>> contents of the macro is a block of C code.I do not know how to convince
> >>>> uncrustify that the contents of a
> >>>>
> >>>> macro like function call to be treated as a code block from an indent
> >>>> perspective.
> >>>>
> >>> I believe this would have to be overridden in the Uncrustify fork since
> >>> DEBUG_CODE() is being treated as a macro function call and code blocks
> >>> are not expected there. In addition, some special treatment might be
> >>> needed for alignment in between DEBUG_CODE_BEGIN()/DEBUG_CODE_END().
> >>>
> >>> I'm happy to look at this. However, regression validation can take a
> >>> while so I'd like to make sure we need this now. In cases that do not
> >>> have additional code blocks, it seems to format fairly well. Is this
> >>> prevalent and impactful enough it must be fixed now? Or, could we
> >>> revisit it with a follow up patch?
> >>
> >> How long does regression testing take? I see about 20 instances of
> >> DEBUG_CODE() that are producing bad formatting. If the content inside
> >> DEBUG_CODE() is a single line of a single block of statements without
> >> any if/while/for/case statements that require further indent, then the
> >> format looks ok.
> >>
> >> DEBUG_CODE_BEGIN() and DEBUG_CODE_END() would look better if the
> >> contents between are also indented one level.
> >>
> >>>
> >>>> * Example #3/#4: Uncrustify is confused by the OPTIONAL keyword.The
> >>>> edk2 config declares it as a QUALIFIER
> >>>>
> >>>> like IN and OUT.However, OPTIONAL only appears at the end of the line
> >>>> that declares a parameter to a
> >>>>
> >>>> function.There are 3 forms. One with comma after OPTIONAL.One with comma
> >>>> before OPTIONAL, and
> >>>>
> >>>> one with no comma if the parameter is the last parameter in the function
> >>>> declaration.
> >>>>
> >>>> TYPE ParamName OPTIONAL,
> >>>>
> >>>> TYPEParamName, OPTIONAL
> >>>>
> >>>> TYPEParamName OPTIONAL
> >>>>
> >>>> OPTIONAL is defined to nothing in edk2 builds.From a uncrustify
> >>>> perspective, we really want is to be
> >>>>
> >>>> ignored or more correctly treated it as a token that is attached to
> >>>> ParamName and the combination of
> >>>>
> >>>> ParamName and OPTIONAL treated as one unit to determine indents.
> >>>>
> >>> "TYPE ParamName, OPTIONAL" seems especially problematic and presents an
> >>> inconsistency with the other formats. Mike, can you please let me know
> >>> if you have the same observation and your thoughts on a consistent pattern?
> >>>
> >>>> Thanks,
> >>>>
> >>>> Mike
> >>>>
> >>>> *From:*devel@edk2.groups.io <devel@edk2.groups.io> *On Behalf Of *Maciej
> >>>> Rabeda
> >>>> *Sent:* Thursday, December 2, 2021 10:27 AM
> >>>> *To:* devel@edk2.groups.io; Kinney, Michael D
> >>>> <michael.d.kinney@intel.com>; Michael Kubacki
> >>>> <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com)
> >>>> <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>
> >>>> *Subject:* Re: [edk2-devel] Uncrustify Conversion Detailed Plan and
> >>>> Extended Hard Freeze Update #4
> >>>>
> >>>> Hey Mike,
> >>>>
> >>>> While most of the changes related to fixing coding style violations,
> >>>> there are a couple of changes to NetworkPkg in that PR that make the
> >>>> code less readable. Examples below.
> >>>>
> >>>> Example 1:
> >>>>
> >>>>
> >>>>
> >>>> Example 2:
> >>>>
> >>>>
> >>>> Example 3:
> >>>>
> >>>>
> >>>> Example 4:
> >>>>
> >>>> On 30-Nov-21 23:34, Michael D Kinney wrote:
> >>>>
> >>>> Hello,
> >>>>
> >>>> Thank you for your patience during this extended hard freeze.
> >>>>
> >>>> Just one more step to go.There has been a delay in the review of
> >>>>
> >>>> the patch series with the uncrustify source changes.PR(6).This
> >>>>
> >>>> patch series was not sent out as patch review email because of its
> >>>>
> >>>> very large size.It only contains source style changes and the
> >>>>
> >>>> CompareBuild tool and GitHub action has shown there are no binary
> >>>>
> >>>> differences introduced with these source style changes.
> >>>>
> >>>> If you are a package maintainer, then please review the following
> >>>>
> >>>> branch/PR for your package contents and review the EDK II CI results
> >>>>
> >>>> and BuildCompare results.I do not expect a line by line review
> >>>>
> >>>> because we already had time to provide feedback on the source style
> >>>>
> >>>> performed by uncrustify.Instead, a Reviewed-by for your package
> >>>>
> >>>> indicates that you have reviewed the EDK II CI results and CompareBuild
> >>>>
> >>>> tool functionality and results and you accept the source style
> >>>>
> >>>> changes to your package.
> >>>>
> >>>> *https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
> >>> <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5>
> >>>>
> >>>> *https://github.com/tianocore/edk2/pull/2229 <https://github.com/tianocore/edk2/pull/2229>
> >>>>
> >>>> *https://github.com/mdkinney/edk2/actions/runs/1521618836
> >>> <https://github.com/mdkinney/edk2/actions/runs/1521618836>
> >>>>
> >>>> Additional details on this update below.
> >>>>
> >>>> Thank you,
> >>>>
> >>>> Mike
> >>>>
> >>>> Changes from Update #3
> >>>>
> >>>> ----------------------------------------------------------------------------
> >>>>
> >>>> * Pushed PR (5)
> >>>>
> >>>> * Added link to PR(6). EDK II CI Status is PASS. Build Compare PASS.
> >>>>
> >>>> * Waiting for review of PR (6)
> >>>>
> >>>> * Review of PR (7) completed and waiting for review of PR (6)
> >>>>
> >>>> ----------------------------------------------------------------------------
> >>>>
> >>>> Changes from Update #2
> >>>>
> >>>> ----------------------------------------------------------------------------
> >>>>
> >>>> * Changed order of PRs swapping (4) and (5).The PR that activates
> >>>>
> >>>> increases the max CI agent job time is independent of all the other
> >>>>
> >>>> PRs and its review is complete, so it can be committed now.
> >>>>
> >>>> * Pushed PRs (1), (2), (3), (4).
> >>>>
> >>>> * Waiting for review to complete for PRs (5) and (6)
> >>>>
> >>>> * Reviews complete for PR (7)
> >>>>
> >>>> * Identifies steps using git filter-branch to apply uncrustify changes to a
> >>>>
> >>>> code review patch series that was generated before the uncrustify changes
> >>>>
> >>>> avoiding manual merge.
> >>>>
> >>>> * Identified steps using git filter-repo to generate an alternate history of
> >>>>
> >>>> the edk2 repo with uncrustify changes applied on every commit.This may
> >>>>
> >>>> be useful when evaluating changes to files using tools like git blame
> >>>>
> >>>> without the large uncrustify patch series.
> >>>>
> >>>> ---------------------------------------------------------------------------
> >>>>
> >>>> Changes from Update #1
> >>>>
> >>>> ----------------------------------------------------------------------------
> >>>>
> >>>> * Changed order of PRs swapping (6) and (7).The PR that activates
> >>>>
> >>>> EDK II CI check UncrustifyCheck has to be last because it unconditionally
> >>>>
> >>>> checks all C/H files in all packages.Not just files that have been
> >>>>
> >>>> modified like some of the other checkers.
> >>>>
> >>>> * Updated link to the branch with the UncrustifyCheck plugin that has been
> >>>>
> >>>> updated with a one line change and Reviewed-by and Tested-by tags.
> >>>>
> >>>> https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v6
> >>> <https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v6>
> >>>>
> >>>> * Reviews complete for (1), (2), (3), (5), and (7)
> >>>>
> >>>> ---------------------------------------------------------------------------
> >>>>
> >>>> Michael Kubacki and I have prepared the patches required to apply the
> >>>>
> >>>> uncrustify changes and enable EDK II CI to check all submitted
> >>>>
> >>>> patches have been run through uncrustify.
> >>>>
> >>>> We have verified through the CompareBuild GitHub Action that the
> >>>>
> >>>> format changes performed by uncrustify have no functional changes.
> >>>>
> >>>> All of the OBJ, LIB, DLL, EFI, FFS, FV, and FD files match 100%
> >>>>
> >>>> across 70 VS2019/GCC5 builds of all package/platform DSC files in
> >>>>
> >>>> the edk2 repo.
> >>>>
> >>>> The hard freeze will be extended after the edk2-stable202111 tag until
> >>>>
> >>>> all uncrustify related changes are committed.We do not expect this
> >>>>
> >>>> to take more than a few days.Do not push any PRs until the hard
> >>>>
> >>>> freeze is lifted.
> >>>>
> >>>> The changes are broken up into 7 patch series/PRs.The PRs are ordered
> >>>>
> >>>> so they can be submitted using the normal submission process and EDK II
> >>>>
> >>>> CI will pass for each one.Details are listed below.
> >>>>
> >>>> Uncrustify 73.0.3 for EDK II
> >>>>
> >>>> =============================
> >>>>
> >>>> * Sources:https://dev.azure.com/projectmu/_git/Uncrustify <https://dev.azure.com/projectmu/_git/Uncrustify>
> >>>>
> >>>> * Documentation:https://dev.azure.com/projectmu/Uncrustify/_wiki/wikis/Uncrustify.wiki/1/Project-Mu-(EDK-II)-
> Fork-
> >>> Readme <https://dev.azure.com/projectmu/Uncrustify/_wiki/wikis/Uncrustify.wiki/1/Project-Mu-(EDK-II)-Fork-Readme>
> >>>>
> >>>> * Download:https://dev.azure.com/projectmu/Uncrustify/_packaging?_a=package&feed=mu_uncrustify&package=mu-
> >>> uncrustify-release&protocolType=NuGet&version=73.0.3
> >>> <https://dev.azure.com/projectmu/Uncrustify/_packaging?_a=package&feed=mu_uncrustify&package=mu-uncrustify-
> >>> release&protocolType=NuGet&version=73.0.3>
> >>>>
> >>>> Installing Uncrustify
> >>>>
> >>>> ======================
> >>>>
> >>>> The Uncrustify tool is installed automatically when the Pytools
> >>>>
> >>>> environment is used and the stuart* commands are run to complete the
> >>>>
> >>>> environment setup.Please see:
> >>>>
> >>>> https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally
> >>> <https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally>
> >>>>
> >>>> Uncrustify can also be installed from the download page listed above
> >>>>
> >>>> or built from sources from the source link above.
> >>>>
> >>>> The Documentation link provides instruction on how to run uncrustify from
> >>>>
> >>>> the command line or install as a Visual Studio Code plugin.The main
> >>>>
> >>>> uncrustify documentation also describes how to integrate with a few other
> >>>>
> >>>> editors.
> >>>>
> >>>> We have also discussed a client side githook.That effort has not started.
> >>>>
> >>>> Let us know if that is a feature you would find useful.
> >>>>
> >>>> Developer impact for new code reviews
> >>>>
> >>>> ======================================
> >>>>
> >>>> Once the uncrustify checker is active in EDK II CI, developers must
> >>>>
> >>>> make sure their patches are run through the uncrustify tool before
> >>>>
> >>>> sending the patches for review.
> >>>>
> >>>> Developers must install and run uncrustify against changes files before
> >>>>
> >>>> sending patch review emails or submitting PR for EDK II CI.If EDK II CI
> >>>>
> >>>> detects and differences in source formatting, then EDK II CI will fail
> >>>>
> >>>> and the developer must run uncrustify and resubmit the patches.
> >>>>
> >>>> Developer impact to patch series/PRs reviewed during edk2-stable201121 soft/hard freeze
> >>>>
> >>>> =======================================================================================
> >>>>
> >>>> Developers must rebase their changes after the uncrustify source changes are
> >>>>
> >>>> committed.The branch with a preview of the uncrustify changes can be used
> >>>>
> >>>> to start this rebase work.
> >>>>
> >>>> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
> >>> <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5>
> >>>>
> >>>> The following steps can be used to update an existing branch with the
> >>>>
> >>>> required uncrustify format.This is the Windows version.I will add
> >>>>
> >>>> the Linux version soon.
> >>>>
> >>>> 1) Fetch and checkout and rebase to latest edk2/master
> >>>>
> >>>> git fetch origin
> >>>>
> >>>> git checkout master
> >>>>
> >>>> git rebase origin/master
> >>>>
> >>>> 2) Make a backup copy of plugin UncrustifyCheck outside WORKSPACE.
> >>>>
> >>>> (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable and
> >>>>
> >>>> EDK II specific uncrustify configuration file available when working
> >>>>
> >>>> with a branch that does not have those tools in its scope.
> >>>>
> >>>> xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
> >>>>
> >>>> 3) Check out the patch series branch (e.g. MyBranch)
> >>>>
> >>>> git checkout MyBranch
> >>>>
> >>>> 4) Rebase patch series against edk2-stable202111
> >>>>
> >>>>
> >>>>
> >>>> git rebase edk2-stable202111
> >>>>
> >>>> 5) Create new branch for the uncrustifed version (e.g. MyBranch_Uncrustified)
> >>>>
> >>>> git checkout -b MyBranch_Uncrustified
> >>>>
> >>>> 6) Use git filter-branch to uncrustify all the commits in the series
> >>>>
> >>>> between the rebase target from (2) and HEAD of the branch.A filter
> >>>>
> >>>> can be used to scope the uncrustify operations to only the C/H files
> >>>>
> >>>> in the specific package the patch series is against. (e.g. DynamicTablesPkg).
> >>>>
> >>>> BaseTools should always be excluded.If the package scoped filter is
> >>>>
> >>>> not used, it will still work, but will take longer to run because
> >>>>
> >>>> uncrustify will rescan every C/H files in the whole repo.
> >>>>
> >>>> git filter-branch --tree-filter "git ls-files DynamicTablesPkg*.c DynamicTablesPkg*.h :!BaseTools/* |
> >>> c:\\Temp\\UncrustifyCheck\\mu-uncrustify-release_extdep\\Windows-x86\\uncrustify.exe -c
> >>> c:\\Temp\\UncrustifyCheck\\uncrustify.cfg -F - --replace --no-backup --if-changed" edk2-stable202111..HEAD
> >>>>
> >>>> 7) Now that all the individual patches in the branch are uncrustified,
> >>>>
> >>>> rebase against latest edk2/master that is already uncrustified.
> >>>>
> >>>> git rebase master
> >>>>
> >>>> 8) Verify the patches in this new branch.
> >>>>
> >>>> Impacts to tracing history across the uncrusity changes
> >>>>
> >>>> =======================================================
> >>>>
> >>>> Tools the view file and line history do work with the large uncrustify
> >>>>
> >>>> patch series.One impact is that the operations can be very slow due
> >>>>
> >>>> to the large uncrustify patches.
> >>>>
> >>>> One option to provide a faster experience is to provide an alternate
> >>>>
> >>>> version of the edk2 repository as "documentation" that has the
> >>>>
> >>>> entire history re-written with uncrustify run on every commit.
> >>>>
> >>>> The tool called git-filter-repo can be used to perform this
> >>>>
> >>>> transformation and runs in a reasonable period of time (a few hours)
> >>>>
> >>>> https://github.com/newren/git-filter-repo <https://github.com/newren/git-filter-repo>
> >>>>
> >>>> https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-demos/lint-history
> >>> <https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-demos/lint-history>
> >>>>
> >>>> The following steps can be used to perform this transformation.
> >>>>
> >>>> This is the Windows version. I will add the Linux version soon.
> >>>>
> >>>> ** WARNING **This operation modifies(rewrites) all the commits
> >>>>
> >>>> in the local copy of the repo.Do not perform
> >>>>
> >>>> these steps on a local repo you are using for
> >>>>
> >>>> active development.
> >>>>
> >>>> 1) Clone edk2 into a new directory (see **WARNING**)
> >>>>
> >>>> git clonehttps://github.com/tianocore/edk2.git <https://github.com/tianocore/edk2.git> edk2-uncrustified
> >>>>
> >>>> cd edk2-uncrustified
> >>>>
> >>>> 2) Setup python virtual env, install pytools, and run stuart commands
> >>>>
> >>>> to setup build environment which includes installing uncrustify tools.
> >>>>
> >>>> https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally
> >>> <https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally>
> >>>>
> >>>> 3) Make a backup copy of plugin UncrustifyCheck outside WORKSPACE.
> >>>>
> >>>> (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable and
> >>>>
> >>>> EDK II specific uncrustify configuration file available when working
> >>>>
> >>>> with a branch that does not have those tools in its scope.
> >>>>
> >>>> xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
> >>>>
> >>>> 4) Use lint-history.py from git-filter-repo examples
> >>>>
> >>>> https://github.com/newren/git-filter-repo <https://github.com/newren/git-filter-repo>
> >>>>
> >>>> https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-demos/lint-history
> >>> <https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-demos/lint-history>
> >>>>
> >>>> Line #127 - Add try except around subprocess.check_call() with except
> >>>>
> >>>> being pass.This is required because there are a few commits of C
> >>>>
> >>>> files in the edk2 repo that have incorrect C syntax and do not
> >>>>
> >>>> build with a C compiler and break the uncrustify parser.Skip reformat
> >>>>
> >>>> of C files that can not be parsed by uncrustify.These rare instances
> >>>>
> >>>> are addressed in the commit that fixes the C syntax error.
> >>>>
> >>>> Run this slightly modified version of lint-history.Include only
> >>>>
> >>>> C/H files and exclude directories that start with 'Tools' or 'BaseTools'.
> >>>>
> >>>> This step took about 2.2 hours on a laptop.
> >>>>
> >>>> lint-history.py
> >>>>
> >>>> --relevant "return (not filename.startswith(b'Tools') and not filename.startswith(b'BaseTools') and
> >>> (filename.endswith(b'.c') or filename.endswith(b'.h')))"
> >>>>
> >>>> c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\mu-uncrustify-release_extdep\\Windows-x86\\uncrustify.exe -c
> >>> c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\uncrustify.cfg --replace --no-backup --if-changed
> >>>>
> >>>> Order of PRs to apply during extended hard freeze
> >>>>
> >>>> ==================================================
> >>>>
> >>>> 1) Update EmulatorPkg Win Host [BuildOptions] MSFT CC_FLAGS to not force debug information
> >>>>
> >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3747 <https://bugzilla.tianocore.org/show_bug.cgi?id=3747>
> >>>>
> >>>> *https://github.com/mdkinney/edk2/tree/Bug_3747_EmulatorPkg_WinHost_ReproducibleBuild
> >>> <https://github.com/mdkinney/edk2/tree/Bug_3747_EmulatorPkg_WinHost_ReproducibleBuild>
> >>>>
> >>>> *https://github.com/tianocore/edk2/pull/2215 <https://github.com/tianocore/edk2/pull/2215>
> >>>>
> >>>> * Required for EmulatorPkg to pass CompareBuild for VS2019 IA32/X64 builds.
> >>>>
> >>>> * Status: Review complete.PR pushed.
> >>>>
> >>>> 2) EccCheck should not revert staged and local changes
> >>>>
> >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=2986 <https://bugzilla.tianocore.org/show_bug.cgi?id=2986>
> >>>>
> >>>> *https://github.com/mdkinney/edk2/tree/Bug_2986_EccCheckRemoveGitRevert_V2
> >>> <https://github.com/mdkinney/edk2/tree/Bug_2986_EccCheckRemoveGitRevert_V2>
> >>>>
> >>>> *https://github.com/tianocore/edk2/pull/2216 <https://github.com/tianocore/edk2/pull/2216>
> >>>>
> >>>> * Required for EDK II CI to complete in a reasonable period of time when
> >>>>
> >>>> processing the 4000+ source file style changes made by uncrustify.
> >>>>
> >>>> * Also fixes critical bugs that can potentially corrupt git state when
> >>>>
> >>>> EccCheck is run locally.
> >>>>
> >>>> * Status: Review complete.PR pushed.
> >>>>
> >>>> 3) Update pytool LicenseCheck plugin to use temp directory for diff output file
> >>>>
> >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3746 <https://bugzilla.tianocore.org/show_bug.cgi?id=3746>
> >>>>
> >>>> *https://github.com/mdkinney/edk2/tree/Bug_3746_LicenseCheckUseDiffOutputFile_V2
> >>> <https://github.com/mdkinney/edk2/tree/Bug_3746_LicenseCheckUseDiffOutputFile_V2>
> >>>>
> >>>> *https://github.com/tianocore/edk2/pull/2217 <https://github.com/tianocore/edk2/pull/2217>
> >>>>
> >>>> * Required to reduce EDK II CI build times.
> >>>>
> >>>> * Status: Review complete.PR pushed.
> >>>>
> >>>> 4) Update max job time from 60 min to 120 minutes in .azurepipelines/templates
> >>>>
> >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3750 <https://bugzilla.tianocore.org/show_bug.cgi?id=3750>
> >>>>
> >>>> *https://github.com/mdkinney/edk2/tree/Bug_3750_IncreaseAzurePipelinesTimeout
> >>> <https://github.com/mdkinney/edk2/tree/Bug_3750_IncreaseAzurePipelinesTimeout>
> >>>>
> >>>> *https://github.com/tianocore/edk2/pull/2219 <https://github.com/tianocore/edk2/pull/2219>
> >>>>
> >>>> * Required to allow EccCheck of uncrustify changes to complete on Azure
> >>>>
> >>>> Pipelines CI agents without timing out.
> >>>>
> >>>> * Status: Review complete.PR pushed.
> >>>>
> >>>> 5) Update Package YAML to ignore specific ECC files/errors
> >>>>
> >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3749 <https://bugzilla.tianocore.org/show_bug.cgi?id=3749>
> >>>>
> >>>> *https://github.com/mdkinney/edk2/tree/Bug_3749_EccCheckIgnoreFilesErrors
> >>> <https://github.com/mdkinney/edk2/tree/Bug_3749_EccCheckIgnoreFilesErrors>
> >>>>
> >>>> *https://github.com/tianocore/edk2/pull/2218 <https://github.com/tianocore/edk2/pull/2218>
> >>>>
> >>>> * Required to pass EccCheck
> >>>>
> >>>> * Status: Review complete. PR pushed
> >>>>
> >>>> 6) Uncrustify Source Changes
> >>>>
> >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3737 <https://bugzilla.tianocore.org/show_bug.cgi?id=3737>
> >>>>
> >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3739 <https://bugzilla.tianocore.org/show_bug.cgi?id=3739>
> >>>>
> >>>> *https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
> >>> <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5>
> >>>>
> >>>> *https://github.com/tianocore/edk2/pull/2229 <https://github.com/tianocore/edk2/pull/2229>
> >>>>
> >>>> * Build comparison result PASS:https://github.com/mdkinney/edk2/actions/runs/1521618836
> >>> <https://github.com/mdkinney/edk2/actions/runs/1521618836>
> >>>>
> >>>> * EFI_D_ -> DEBUG changes required to pass PatchCheck
> >>>>
> >>>> * Uncrustify format changes required to pass UncrustifyCheck
> >>>>
> >>>> * Status:
> >>>>
> >>>> Waiting for review
> >>>>
> >>>> 7) UncrustifyCheck EDK II CI Plugin
> >>>>
> >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3748 <https://bugzilla.tianocore.org/show_bug.cgi?id=3748>
> >>>>
> >>>> *https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v6
> >>> <https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v6>
> >>>>
> >>>> * Required to enforce all PRs submitted to EDK II CI match uncrustify format.
> >>>>
> >>>> * Unconditionally checks all packages.Can not be committed until all C/H
> >>>>
> >>>> source files have been updated.
> >>>>
> >>>> * Status: Review complete
> >>>>
> >>>> Combined Branch/PR for Review/Test
> >>>>
> >>>> ==================================
> >>>>
> >>>> * Build Comparison results must pass 100% across the full set of PRs before
> >>>>
> >>>> the individual PRs can be pushed in the order listed above.
> >>>>
> >>>> * Branch:https://github.com/mdkinney/edk2/tree/TestOnly_Uncrustify_PR_Series
> >>> <https://github.com/mdkinney/edk2/tree/TestOnly_Uncrustify_PR_Series>
> >>>>
> >>>> * PR:https://github.com/tianocore/edk2/pull/2229 <https://github.com/tianocore/edk2/pull/2229>
> >>>>
> >>>> Status = PASS
> >>>>
> >>>> * CompareBuild:
> >>>>
> >>>> Branch:https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
> >>> <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5>
> >>>>
> >>>> --ref1:ef9a059cdb15844fe52a49af2bf7d86b9dd3e9bf
> >>>>
> >>>> --ref2:Bug_3737_3739_ApplyUncrustifyChanges_V5
> >>>>
> >>>> Extra Options: -n 4 --quiet
> >>>>
> >>>> Results:https://github.com/mdkinney/edk2/actions/runs/1521618836
> >>> <https://github.com/mdkinney/edk2/actions/runs/1521618836>
> >>>>
> >>>> 30 VS2019 build comparisons PASS
> >>>>
> >>>> 40 GCC5 build comparisons PASS
> >>>>
> >>>> 100% PASS
> >>>>
> >>>> The following git log shows the set of patches from --ref1 to --ref 2across
> >>>>
> >>>> which there are no differences in any of the OBJ/LIB/DLL/EFI/FFS/FV/FD files.
> >>>>
> >>>> --ref2
> >>>>
> >>>> b7d4bf0675b7 (HEAD -> Bug_3737_3739_ApplyUncrustifyChanges_V5) UnitTestFrameworkPkg: Apply uncrusitify changes
> >>>>
> >>>> 7f03d25f60e7 UefiPayloadPkg: Apply uncrusitify changes
> >>>>
> >>>> 0bfd8d9b5ac9 UefiCpuPkg: Apply uncrusitify changes
> >>>>
> >>>> e1cd9bfb9dea StandaloneMmPkg: Apply uncrusitify changes
> >>>>
> >>>> 5da2f65be378 SourceLevelDebugPkg: Apply uncrusitify changes
> >>>>
> >>>> 95b86de07e5d SignedCapsulePkg: Apply uncrusitify changes
> >>>>
> >>>> fe71d97246c4 ShellPkg: Apply uncrusitify changes
> >>>>
> >>>> 54c21c952992 SecurityPkg: Apply uncrusitify changes
> >>>>
> >>>> 187a3785f12b RedfishPkg: Apply uncrusitify changes
> >>>>
> >>>> 810100002a46 PcAtChipsetPkg: Apply uncrusitify changes
> >>>>
> >>>> 276a695c0cf2 OvmfPkg: Apply uncrusitify changes
> >>>>
> >>>> 303c0a91ab07 NetworkPkg: Apply uncrusitify changes
> >>>>
> >>>> bc80792cd1b1 MdePkg: Apply uncrusitify changes
> >>>>
> >>>> 3ea86be17a2a MdeModulePkg: Apply uncrusitify changes
> >>>>
> >>>> c70ef11ed0cd IntelFsp2WrapperPkg: Apply uncrusitify changes
> >>>>
> >>>> c0291221f252 IntelFsp2Pkg: Apply uncrusitify changes
> >>>>
> >>>> 6a479952a690 FmpDevicePkg: Apply uncrusitify changes
> >>>>
> >>>> 3a7c05b7070d FatPkg: Apply uncrusitify changes
> >>>>
> >>>> b789f98c8959 EmulatorPkg: Apply uncrusitify changes
> >>>>
> >>>> 952d7a1c9220 EmbeddedPkg: Apply uncrusitify changes
> >>>>
> >>>> a1cc9881bab6 DynamicTablesPkg: Apply uncrusitify changes
> >>>>
> >>>> 50654dfe5785 CryptoPkg: Apply uncrusitify changes
> >>>>
> >>>> ed965a02dfa1 ArmVirtPkg: Apply uncrusitify changes
> >>>>
> >>>> 9744023fbc46 ArmPlatformPkg: Apply uncrusitify changes
> >>>>
> >>>> 7a1cde5f5bba ArmPkg: Apply uncrusitify changes
> >>>>
> >>>> 19d17e0913e8 UefiCpuPkg: Change use of EFI_D_* to DEBUG_*
> >>>>
> >>>> ffa718b4f994 SourceLevelDebugPkg: Change use of EFI_D_* to DEBUG_*
> >>>>
> >>>> b86cb3c5e5b4 ShellPkg: Change use of EFI_D_* to DEBUG_*
> >>>>
> >>>> c7c42204dc07 SecurityPkg: Change use of EFI_D_* to DEBUG_*
> >>>>
> >>>> 16b8e6f958e4 PcAtChipsetPkg: Change use of EFI_D_* to DEBUG_*
> >>>>
> >>>> 0ac3f8b2dac5 OvmfPkg: Change use of EFI_D_* to DEBUG_*
> >>>>
> >>>> bc5004b8d294 NetworkPkg: Change use of EFI_D_* to DEBUG_*
> >>>>
> >>>> 6f671a8e2377 MdePkg: Change use of EFI_D_* to DEBUG_*
> >>>>
> >>>> a10c610ff9a3 MdeModulePkg: Change use of EFI_D_* to DEBUG_*
> >>>>
> >>>> 09a3bddba390 FatPkg: Change use of EFI_D_* to DEBUG_*
> >>>>
> >>>> 59c61318246a EmulatorPkg: Change use of EFI_D_* to DEBUG_*
> >>>>
> >>>> 3a80367dda3b EmbeddedPkg: Change use of EFI_D_* to DEBUG_*
> >>>>
> >>>> 23eb1aaf80ca ArmVirtPkg: Change use of EFI_D_* to DEBUG_*
> >>>>
> >>>> 875914b45c54 ArmPlatformPkg: Change use of EFI_D_* to DEBUG_*
> >>>>
> >>>> eb2eca82b451 ArmPkg: Change use of EFI_D_* to DEBUG_*
> >>>>
> >>>> f0f3f5aae7c4 (origin/master, origin/HEAD, master) UnitTestFrameworkPkg: Update YAML to ignore specific ECC
> >>> files/errors
> >>>>
> >>>> c05734797790 UefiPayloadPkg: Update YAML to ignore specific ECC files/errors
> >>>>
> >>>> c30c40d6c63d StandaloneMmPkg: Update YAML to ignore specific ECC files/errors
> >>>>
> >>>> 9944508e85f1 ShellPkg: Update YAML to ignore specific ECC files/errors
> >>>>
> >>>> 60fa40be458d SecurityPkg: Update YAML to ignore specific ECC files/errors
> >>>>
> >>>> df790cd6b37e MdePkg: Update YAML to ignore specific ECC files/errors
> >>>>
> >>>> 9deb9370766e MdeModulePkg: Update YAML to ignore specific ECC files/errors
> >>>>
> >>>> d7d30e8f219f EmulatorPkg: Update YAML to ignore specific ECC files/errors
> >>>>
> >>>> d5744ecba813 CryptoPkg: Update YAML to ignore specific ECC files/errors
> >>>>
> >>>> c97fee87f0f9 ArmVirtPkg: Update YAML to ignore specific ECC files/errors
> >>>>
> >>>> 1939fc9569f2 ArmPlatformPkg: Update YAML to ignore specific ECC files/errors
> >>>>
> >>>> 365dced2c37a ArmPkg: Update YAML to ignore specific ECC files/errors
> >>>>
> >>>> 76a1ce4d5fec .azurepipelines/templates: Update max pipeline job time to 2 hours
> >>>>
> >>>> 99f84ff47390 .pytools/Plugin/LicenseCheck: Use temp directory for git diff output
> >>>>
> >>>> 3019f1bbabf1 .pytool/Plugin/EccCheck: Add performance optimizations
> >>>>
> >>>> 854462bd3479 .pytool/Plugin/EccCheck: Remove temp directory on exception
> >>>>
> >>>> 69877614fdee .pytool/Plugin/EccCheck: Remove RevertCode()
> >>>>
> >>>> --ref1
> >>>>
> >>>> ef9a059cdb15 EmulatorPkg/Win/Host: Update CC_FLAGS
> >>>>
> >>>> bb1bba3d7767 (tag: edk2-stable202111) NetworkPkg: Fix invalid pointer for DNS response token on error
> >>>>
> >>>> Best regards,
> >>>>
> >>>> Mike
> >>>>
> >>>>
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
2021-12-03 0:53 ` Michael D Kinney
@ 2021-12-03 2:22 ` Michael D Kinney
2021-12-06 1:17 ` Michael D Kinney
0 siblings, 1 reply; 39+ messages in thread
From: Michael D Kinney @ 2021-12-03 2:22 UTC (permalink / raw)
To: Michael Kubacki, devel@edk2.groups.io,
maciej.rabeda@linux.intel.com, Michael Kubacki,
Andrew Fish (afish@apple.com), Leif Lindholm, Kinney, Michael D
Hello EDK II Maintainers,
I have entered BZ 3760 to make the use of the OPTIONAL keyword style consistent for all of edk2 repo
and to be compatible with uncrustify.
I have posted the following V6 branch that does the EFI_D_* to DEBUG_* changes, the OPTIONAL keyword
style changes, and the uncrustify changes with the one configuration change for assignment alignment.
https://github.com/mdkinney/edk2/tree/Bug_3737_3760_3739_ApplyUncrustifyChanges_V6
Please provide feedback on the code style in this branch with the known DEBUG_CODE() issue still
present.
If we are able to quickly update uncrustify to handle DEBUG_CODE(), I will generate a V7.
Thanks,
Mike
> -----Original Message-----
> From: Kinney, Michael D <michael.d.kinney@intel.com>
> Sent: Thursday, December 2, 2021 4:53 PM
> To: Michael Kubacki <mikuback@linux.microsoft.com>; devel@edk2.groups.io; maciej.rabeda@linux.intel.com; Michael Kubacki
> <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com) <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>;
> Kinney, Michael D <michael.d.kinney@intel.com>
> Subject: RE: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
>
> Michael,
>
> Yes. Please update the patch series that adds UncrustifyCheck with those changes.
>
> Thanks,
>
> Mike
>
> > -----Original Message-----
> > From: Michael Kubacki <mikuback@linux.microsoft.com>
> > Sent: Thursday, December 2, 2021 4:31 PM
> > To: Kinney, Michael D <michael.d.kinney@intel.com>; devel@edk2.groups.io; maciej.rabeda@linux.intel.com; Michael Kubacki
> > <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com) <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>
> > Subject: Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
> >
> > Hi Mike,
> >
> > Thank you for the detailed analysis and recommendations.
> >
> > I agree with the recommendations and I will try to have an Uncrustify
> > tool update by tomorrow for (4).
> >
> > That will require an update in uncrustify_ext_dep.yaml to pull in the
> > new version. I'm assuming I should also update uncrustify.cfg to set
> > align_assign_thresh = 0 in that new patch series?
> >
> > Regards,
> > Michael
> >
> > On 12/2/2021 7:18 PM, Kinney, Michael D wrote:
> > > Hi Michael,
> > >
> > > CORRECTION: set align_assign_threshold to 0.
> > >
> > > Reponses inline below.
> > >
> > > I would like to summarize the 4 issues raised in the past day along with the recommendations.
> > >
> > > 1) Exclusion feature for UncrustifyCheck. There are 2 directories with 8 files total that
> > > maintainers have noted they would like to see not go through uncrustify formatting. Today
> > > the only content that is skipped is BaseTools and submodules.
> > >
> > > Adding a general purpose exclusion feature would then require all developers to make
> > > sure their method of using uncrustify also excludes those same areas. This requires
> > > extra steps for all developers and maintainers.
> > >
> > > If we do not add the exclusion feature, then the 8 files will require an extra step
> > > to sync with the original source of those files. The rate of changes of these 8 files
> > > is very low today.
> > >
> > > RECOMMENDATION: Do not add exclusion feature at this time. Revisit if the extra work
> > > to maintain the files that would be candidates for exclusions increases significantly.
> > >
> > > 2) Alignment of assignments. The threshold of 4 characters appears to be too low and causes
> > > source files that are already aligned to become unaligned.
> > >
> > > RECOMMENDATION: Change threshold to the default value of 0 which means no limit.
> > >
> > > align_assign_thresh= 0
> > >
> > > 3) Alignment of parameters in function declaration not correct. The root cause of this
> > > is the use of the OPTIONAL keyword. If the OPTIONAL keyword is removed, then the
> > > alignment is correct. The alignment is also correct if the OPTIONAL keyword appears
> > > before the ','. If the OPTIONAL keyword appears after the ',', then the format is
> > > not correct. The OPTIONAL keyword indicates that the parameter in the function is
> > > not required and may be passed in as NULL or 0 or some other default value defined by
> > > the API. It makes more sense for this OPTIONAL keyword that follows the parameter
> > > names to appear before the ',' so it is scoped to the parameter on that line. If it
> > > appears after the ',', then C parsers thinks it is a prefix (IN, OUT, CONST, volatile,
> > > static) for the next parameter in the function.
> > >
> > > RECOMMENDATION: Update patch series with a global search and replace so OPTIONAL
> > > keyword always appears before the ',' on the same line.
> > >
> > > RegEx search string: ',( *)OPTIONAL( *)'
> > > RegEx replace string: ' $1OPTIONAL,$2'
> > >
> > > 4) Format issues with complex blocks in DEBUG_CODE(), or between DEBUG_CODE_BEGIN() and
> > > DEBUG_CODE_END(). Uncrustify treats these as macros and is not aware that the
> > > parameter passed into the macro call is a block of C code that needs to be formatted.
> > > Complex blocks with if/while/for/case statements are impacted the most.
> > >
> > > RECOMMENDATION: Update the uncrustify with an edk2 specific extension to treat these
> > > macros as a block of code as if they were surrounded by an extra set of braces {}.
> > >
> > >
> > > I have posted a branch for testing purposes that implements (2) and (3).
> > >
> > > Branch: https://github.com/mdkinney/edk2/tree/TestOnly_Bug_3737_3739_ApplyUncrustifyChanges_V6_OPTIONAL_Keyword_Fix
> > > PR: https://github.com/tianocore/edk2/pull/2233
> > > Status: PASS
> > > CompareBuild: https://github.com/mdkinney/edk2/actions/runs/1532855914
> > > Status: PASS
> > >
> > > You can see what changed by fetching and comparing the following 2 branches:
> > >
> > > https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
> > > https://github.com/mdkinney/edk2/tree/TestOnly_Bug_3737_3739_ApplyUncrustifyChanges_V6_OPTIONAL_Keyword_Fix
> > >
> > > Please provide feedback on the RECOMMENDATIONS above. I will go ahead and prepare of V6 version of
> > > the patch series now that that test results are all PASS.
> > >
> > > Best regards,
> > >
> > > Mike
> > >
> > >
> > >> -----Original Message-----
> > >> From: Kinney, Michael D <michael.d.kinney@intel.com>
> > >> Sent: Thursday, December 2, 2021 4:15 PM
> > >> To: Michael Kubacki <mikuback@linux.microsoft.com>; devel@edk2.groups.io; maciej.rabeda@linux.intel.com; Michael
> > Kubacki
> > >> <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com) <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>;
> > >> Kinney, Michael D <michael.d.kinney@intel.com>
> > >> Subject: RE: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
> > >>
> > >> Hi Michael,
> > >>
> > >> Reponses inline below.
> > >>
> > >> I would like to summarize the 4 issues raised in the past day along with the recommendations.
> > >>
> > >> 1) Exclusion feature for UncrustifyCheck. There are 2 directories with 8 files total that
> > >> maintainers have noted they would like to see not go through uncrustify formatting. Today
> > >> the only content that is skipped is BaseTools and submodules.
> > >>
> > >> Adding a general purpose exclusion feature would then require all developers to make
> > >> sure their method of using uncrustify also excludes those same areas. This requires
> > >> extra steps for all developers and maintainers.
> > >>
> > >> If we do not add the exclusion feature, then the 8 files will require an extra step
> > >> to sync with the original source of those files. The rate of changes of these 8 files
> > >> is very low today.
> > >>
> > >> RECOMMENDATION: Do not add exclusion feature at this time. Revisit if the extra work
> > >> to maintain the files that would be candidates for exclusions increases significantly.
> > >>
> > >> 2) Alignment of assignments. The threshold of 4 characters appears to be too low and causes
> > >> source files that are already aligned to become unaligned.
> > >>
> > >> RECOMMENDATION: Change threshold to the default value of 0 which means no limit.
> > >>
> > >> align_assign_thresh= 4
> > >>
> > >> 3) Alignment of parameters in function declaration not correct. The root cause of this
> > >> is the use of the OPTIONAL keyword. If the OPTIONAL keyword is removed, then the
> > >> alignment is correct. The alignment is also correct if the OPTIONAL keyword appears
> > >> before the ','. If the OPTIONAL keyword appears after the ',', then the format is
> > >> not correct. The OPTIONAL keyword indicates that the parameter in the function is
> > >> not required and may be passed in as NULL or 0 or some other default value defined by
> > >> the API. It makes more sense for this OPTIONAL keyword that follows the parameter
> > >> names to appear before the ',' so it is scoped to the parameter on that line. If it
> > >> appears after the ',', then C parsers thinks it is a prefix (IN, OUT, CONST, volatile,
> > >> static) for the next parameter in the function.
> > >>
> > >> RECOMMENDATION: Update patch series with a global search and replace so OPTIONAL
> > >> keyword always appears before the ',' on the same line.
> > >>
> > >> RegEx search string: ',( *)OPTIONAL( *)'
> > >> RegEx replace string: ' $1OPTIONAL,$2'
> > >>
> > >> 4) Format issues with complex blocks in DEBUG_CODE(), or between DEBUG_CODE_BEGIN() and
> > >> DEBUG_CODE_END(). Uncrustify treats these as macros and is not aware that the
> > >> parameter passed into the macro call is a block of C code that needs to be formatted.
> > >> Complex blocks with if/while/for/case statements are impacted the most.
> > >>
> > >> RECOMMENDATION: Update the uncrustify with an edk2 specific extension to treat these
> > >> macros as a block of code as if they were surrounded by an extra set of braces {}.
> > >>
> > >>
> > >> I have posted a branch for testing purposes that implements (2) and (3).
> > >>
> > >> Branch: https://github.com/mdkinney/edk2/tree/TestOnly_Bug_3737_3739_ApplyUncrustifyChanges_V6_OPTIONAL_Keyword_Fix
> > >> PR: https://github.com/tianocore/edk2/pull/2233
> > >> Status: PASS
> > >> CompareBuild: https://github.com/mdkinney/edk2/actions/runs/1532855914
> > >> Status: PASS
> > >>
> > >> You can see what changed by fetching and comparing the following 2 branches:
> > >>
> > >> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
> > >> https://github.com/mdkinney/edk2/tree/TestOnly_Bug_3737_3739_ApplyUncrustifyChanges_V6_OPTIONAL_Keyword_Fix
> > >>
> > >> Please provide feedback on the RECOMMENDATIONS above. I will go ahead and prepare of V6 version of
> > >> the patch series now that that test results are all PASS.
> > >>
> > >> Best regards,
> > >>
> > >> Mike
> > >>
> > >>
> > >>> -----Original Message-----
> > >>> From: Michael Kubacki <mikuback@linux.microsoft.com>
> > >>> Sent: Thursday, December 2, 2021 1:57 PM
> > >>> To: devel@edk2.groups.io; Kinney, Michael D <michael.d.kinney@intel.com>; maciej.rabeda@linux.intel.com; Michael
> > Kubacki
> > >>> <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com) <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>
> > >>> Subject: Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
> > >>>
> > >>> My reply is inline.
> > >>>
> > >>> Regards,
> > >>> Michael
> > >>>
> > >>> On 12/2/2021 2:45 PM, Michael D Kinney wrote:
> > >>>> Hi Maciej,
> > >>>>
> > >>>> Thanks for the feedback.
> > >>>>
> > >>>> * Example #1.This appears to be caused by the following uncrustify
> > >>>> settings:
> > >>>>
> > >>>> # The threshold for aligning on '=' in assignments.
> > >>>>
> > >>>> # Use a negative number for absolute thresholds.
> > >>>>
> > >>>> #
> > >>>>
> > >>>> # 0: No limit (default).
> > >>>>
> > >>>> align_assign_thresh= 0# number
> > >>>>
> > >>>> The edk2 setting for this is:
> > >>>>
> > >>>> align_assign_thresh= 4
> > >>>>
> > >>>> This means blocks of assignments that are different than more than 4
> > >>>> spaces will be considered a new block.
> > >>>>
> > >>>> ‘HwAddreLen’ and ‘Xid’ are more than 4 characters in length
> > >>>> different.Same for ‘Xid’ and ‘Reserved’.So
> > >>>>
> > >>>> uncrustify treats these as 3 different assignment blocks.
> > >>>>
> > >>>> If we change to the default value of 0: No limit, this example is
> > >>>> treated as a single block and all ‘=’ are aligned.
> > >>>>
> > >>>> Is there a reason ‘4’ was selected?Is there is any harm in using the
> > >>>> default of 0?
> > >>>>
> > >>> We can certainly change the threshold. '4' was derived by
> > >>> experimentation. This is an area that is somewhat subjective and every
> > >>> case is difficult to cover well. Please feel free to suggest any changes.
> > >>>
> > >>
> > >> I recommend we use the default value of 0. That way, any code that choose to
> > >> align assignments will still be aligned. If a developer does not like
> > >> short assignments and long assignments in the same code block to use the
> > >> long assignment column, they can always break the block up into multiple
> > >> blocks by adding a carriage return.
> > >>
> > >>> For the benefit of others, this file can be a useful refernce
> > >>> understanding spans, gaps, and thresholds.
> > >>>
> > >>> https://github.com/uncrustify/uncrustify/blob/master/documentation/htdocs/configuration.txt
> > >>>
> > >>>> * Example #2: Uncruistfy is confused by the DEBUG_CODE() macro.This is
> > >>>> not a traditional macro because the
> > >>>>
> > >>>> contents of the macro is a block of C code.I do not know how to convince
> > >>>> uncrustify that the contents of a
> > >>>>
> > >>>> macro like function call to be treated as a code block from an indent
> > >>>> perspective.
> > >>>>
> > >>> I believe this would have to be overridden in the Uncrustify fork since
> > >>> DEBUG_CODE() is being treated as a macro function call and code blocks
> > >>> are not expected there. In addition, some special treatment might be
> > >>> needed for alignment in between DEBUG_CODE_BEGIN()/DEBUG_CODE_END().
> > >>>
> > >>> I'm happy to look at this. However, regression validation can take a
> > >>> while so I'd like to make sure we need this now. In cases that do not
> > >>> have additional code blocks, it seems to format fairly well. Is this
> > >>> prevalent and impactful enough it must be fixed now? Or, could we
> > >>> revisit it with a follow up patch?
> > >>
> > >> How long does regression testing take? I see about 20 instances of
> > >> DEBUG_CODE() that are producing bad formatting. If the content inside
> > >> DEBUG_CODE() is a single line of a single block of statements without
> > >> any if/while/for/case statements that require further indent, then the
> > >> format looks ok.
> > >>
> > >> DEBUG_CODE_BEGIN() and DEBUG_CODE_END() would look better if the
> > >> contents between are also indented one level.
> > >>
> > >>>
> > >>>> * Example #3/#4: Uncrustify is confused by the OPTIONAL keyword.The
> > >>>> edk2 config declares it as a QUALIFIER
> > >>>>
> > >>>> like IN and OUT.However, OPTIONAL only appears at the end of the line
> > >>>> that declares a parameter to a
> > >>>>
> > >>>> function.There are 3 forms. One with comma after OPTIONAL.One with comma
> > >>>> before OPTIONAL, and
> > >>>>
> > >>>> one with no comma if the parameter is the last parameter in the function
> > >>>> declaration.
> > >>>>
> > >>>> TYPE ParamName OPTIONAL,
> > >>>>
> > >>>> TYPEParamName, OPTIONAL
> > >>>>
> > >>>> TYPEParamName OPTIONAL
> > >>>>
> > >>>> OPTIONAL is defined to nothing in edk2 builds.From a uncrustify
> > >>>> perspective, we really want is to be
> > >>>>
> > >>>> ignored or more correctly treated it as a token that is attached to
> > >>>> ParamName and the combination of
> > >>>>
> > >>>> ParamName and OPTIONAL treated as one unit to determine indents.
> > >>>>
> > >>> "TYPE ParamName, OPTIONAL" seems especially problematic and presents an
> > >>> inconsistency with the other formats. Mike, can you please let me know
> > >>> if you have the same observation and your thoughts on a consistent pattern?
> > >>>
> > >>>> Thanks,
> > >>>>
> > >>>> Mike
> > >>>>
> > >>>> *From:*devel@edk2.groups.io <devel@edk2.groups.io> *On Behalf Of *Maciej
> > >>>> Rabeda
> > >>>> *Sent:* Thursday, December 2, 2021 10:27 AM
> > >>>> *To:* devel@edk2.groups.io; Kinney, Michael D
> > >>>> <michael.d.kinney@intel.com>; Michael Kubacki
> > >>>> <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com)
> > >>>> <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>
> > >>>> *Subject:* Re: [edk2-devel] Uncrustify Conversion Detailed Plan and
> > >>>> Extended Hard Freeze Update #4
> > >>>>
> > >>>> Hey Mike,
> > >>>>
> > >>>> While most of the changes related to fixing coding style violations,
> > >>>> there are a couple of changes to NetworkPkg in that PR that make the
> > >>>> code less readable. Examples below.
> > >>>>
> > >>>> Example 1:
> > >>>>
> > >>>>
> > >>>>
> > >>>> Example 2:
> > >>>>
> > >>>>
> > >>>> Example 3:
> > >>>>
> > >>>>
> > >>>> Example 4:
> > >>>>
> > >>>> On 30-Nov-21 23:34, Michael D Kinney wrote:
> > >>>>
> > >>>> Hello,
> > >>>>
> > >>>> Thank you for your patience during this extended hard freeze.
> > >>>>
> > >>>> Just one more step to go.There has been a delay in the review of
> > >>>>
> > >>>> the patch series with the uncrustify source changes.PR(6).This
> > >>>>
> > >>>> patch series was not sent out as patch review email because of its
> > >>>>
> > >>>> very large size.It only contains source style changes and the
> > >>>>
> > >>>> CompareBuild tool and GitHub action has shown there are no binary
> > >>>>
> > >>>> differences introduced with these source style changes.
> > >>>>
> > >>>> If you are a package maintainer, then please review the following
> > >>>>
> > >>>> branch/PR for your package contents and review the EDK II CI results
> > >>>>
> > >>>> and BuildCompare results.I do not expect a line by line review
> > >>>>
> > >>>> because we already had time to provide feedback on the source style
> > >>>>
> > >>>> performed by uncrustify.Instead, a Reviewed-by for your package
> > >>>>
> > >>>> indicates that you have reviewed the EDK II CI results and CompareBuild
> > >>>>
> > >>>> tool functionality and results and you accept the source style
> > >>>>
> > >>>> changes to your package.
> > >>>>
> > >>>> *https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
> > >>> <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5>
> > >>>>
> > >>>> *https://github.com/tianocore/edk2/pull/2229 <https://github.com/tianocore/edk2/pull/2229>
> > >>>>
> > >>>> *https://github.com/mdkinney/edk2/actions/runs/1521618836
> > >>> <https://github.com/mdkinney/edk2/actions/runs/1521618836>
> > >>>>
> > >>>> Additional details on this update below.
> > >>>>
> > >>>> Thank you,
> > >>>>
> > >>>> Mike
> > >>>>
> > >>>> Changes from Update #3
> > >>>>
> > >>>> ----------------------------------------------------------------------------
> > >>>>
> > >>>> * Pushed PR (5)
> > >>>>
> > >>>> * Added link to PR(6). EDK II CI Status is PASS. Build Compare PASS.
> > >>>>
> > >>>> * Waiting for review of PR (6)
> > >>>>
> > >>>> * Review of PR (7) completed and waiting for review of PR (6)
> > >>>>
> > >>>> ----------------------------------------------------------------------------
> > >>>>
> > >>>> Changes from Update #2
> > >>>>
> > >>>> ----------------------------------------------------------------------------
> > >>>>
> > >>>> * Changed order of PRs swapping (4) and (5).The PR that activates
> > >>>>
> > >>>> increases the max CI agent job time is independent of all the other
> > >>>>
> > >>>> PRs and its review is complete, so it can be committed now.
> > >>>>
> > >>>> * Pushed PRs (1), (2), (3), (4).
> > >>>>
> > >>>> * Waiting for review to complete for PRs (5) and (6)
> > >>>>
> > >>>> * Reviews complete for PR (7)
> > >>>>
> > >>>> * Identifies steps using git filter-branch to apply uncrustify changes to a
> > >>>>
> > >>>> code review patch series that was generated before the uncrustify changes
> > >>>>
> > >>>> avoiding manual merge.
> > >>>>
> > >>>> * Identified steps using git filter-repo to generate an alternate history of
> > >>>>
> > >>>> the edk2 repo with uncrustify changes applied on every commit.This may
> > >>>>
> > >>>> be useful when evaluating changes to files using tools like git blame
> > >>>>
> > >>>> without the large uncrustify patch series.
> > >>>>
> > >>>> ---------------------------------------------------------------------------
> > >>>>
> > >>>> Changes from Update #1
> > >>>>
> > >>>> ----------------------------------------------------------------------------
> > >>>>
> > >>>> * Changed order of PRs swapping (6) and (7).The PR that activates
> > >>>>
> > >>>> EDK II CI check UncrustifyCheck has to be last because it unconditionally
> > >>>>
> > >>>> checks all C/H files in all packages.Not just files that have been
> > >>>>
> > >>>> modified like some of the other checkers.
> > >>>>
> > >>>> * Updated link to the branch with the UncrustifyCheck plugin that has been
> > >>>>
> > >>>> updated with a one line change and Reviewed-by and Tested-by tags.
> > >>>>
> > >>>> https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v6
> > >>> <https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v6>
> > >>>>
> > >>>> * Reviews complete for (1), (2), (3), (5), and (7)
> > >>>>
> > >>>> ---------------------------------------------------------------------------
> > >>>>
> > >>>> Michael Kubacki and I have prepared the patches required to apply the
> > >>>>
> > >>>> uncrustify changes and enable EDK II CI to check all submitted
> > >>>>
> > >>>> patches have been run through uncrustify.
> > >>>>
> > >>>> We have verified through the CompareBuild GitHub Action that the
> > >>>>
> > >>>> format changes performed by uncrustify have no functional changes.
> > >>>>
> > >>>> All of the OBJ, LIB, DLL, EFI, FFS, FV, and FD files match 100%
> > >>>>
> > >>>> across 70 VS2019/GCC5 builds of all package/platform DSC files in
> > >>>>
> > >>>> the edk2 repo.
> > >>>>
> > >>>> The hard freeze will be extended after the edk2-stable202111 tag until
> > >>>>
> > >>>> all uncrustify related changes are committed.We do not expect this
> > >>>>
> > >>>> to take more than a few days.Do not push any PRs until the hard
> > >>>>
> > >>>> freeze is lifted.
> > >>>>
> > >>>> The changes are broken up into 7 patch series/PRs.The PRs are ordered
> > >>>>
> > >>>> so they can be submitted using the normal submission process and EDK II
> > >>>>
> > >>>> CI will pass for each one.Details are listed below.
> > >>>>
> > >>>> Uncrustify 73.0.3 for EDK II
> > >>>>
> > >>>> =============================
> > >>>>
> > >>>> * Sources:https://dev.azure.com/projectmu/_git/Uncrustify <https://dev.azure.com/projectmu/_git/Uncrustify>
> > >>>>
> > >>>> * Documentation:https://dev.azure.com/projectmu/Uncrustify/_wiki/wikis/Uncrustify.wiki/1/Project-Mu-(EDK-II)-
> > Fork-
> > >>> Readme <https://dev.azure.com/projectmu/Uncrustify/_wiki/wikis/Uncrustify.wiki/1/Project-Mu-(EDK-II)-Fork-Readme>
> > >>>>
> > >>>> * Download:https://dev.azure.com/projectmu/Uncrustify/_packaging?_a=package&feed=mu_uncrustify&package=mu-
> > >>> uncrustify-release&protocolType=NuGet&version=73.0.3
> > >>> <https://dev.azure.com/projectmu/Uncrustify/_packaging?_a=package&feed=mu_uncrustify&package=mu-uncrustify-
> > >>> release&protocolType=NuGet&version=73.0.3>
> > >>>>
> > >>>> Installing Uncrustify
> > >>>>
> > >>>> ======================
> > >>>>
> > >>>> The Uncrustify tool is installed automatically when the Pytools
> > >>>>
> > >>>> environment is used and the stuart* commands are run to complete the
> > >>>>
> > >>>> environment setup.Please see:
> > >>>>
> > >>>> https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally
> > >>> <https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally>
> > >>>>
> > >>>> Uncrustify can also be installed from the download page listed above
> > >>>>
> > >>>> or built from sources from the source link above.
> > >>>>
> > >>>> The Documentation link provides instruction on how to run uncrustify from
> > >>>>
> > >>>> the command line or install as a Visual Studio Code plugin.The main
> > >>>>
> > >>>> uncrustify documentation also describes how to integrate with a few other
> > >>>>
> > >>>> editors.
> > >>>>
> > >>>> We have also discussed a client side githook.That effort has not started.
> > >>>>
> > >>>> Let us know if that is a feature you would find useful.
> > >>>>
> > >>>> Developer impact for new code reviews
> > >>>>
> > >>>> ======================================
> > >>>>
> > >>>> Once the uncrustify checker is active in EDK II CI, developers must
> > >>>>
> > >>>> make sure their patches are run through the uncrustify tool before
> > >>>>
> > >>>> sending the patches for review.
> > >>>>
> > >>>> Developers must install and run uncrustify against changes files before
> > >>>>
> > >>>> sending patch review emails or submitting PR for EDK II CI.If EDK II CI
> > >>>>
> > >>>> detects and differences in source formatting, then EDK II CI will fail
> > >>>>
> > >>>> and the developer must run uncrustify and resubmit the patches.
> > >>>>
> > >>>> Developer impact to patch series/PRs reviewed during edk2-stable201121 soft/hard freeze
> > >>>>
> > >>>> =======================================================================================
> > >>>>
> > >>>> Developers must rebase their changes after the uncrustify source changes are
> > >>>>
> > >>>> committed.The branch with a preview of the uncrustify changes can be used
> > >>>>
> > >>>> to start this rebase work.
> > >>>>
> > >>>> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
> > >>> <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5>
> > >>>>
> > >>>> The following steps can be used to update an existing branch with the
> > >>>>
> > >>>> required uncrustify format.This is the Windows version.I will add
> > >>>>
> > >>>> the Linux version soon.
> > >>>>
> > >>>> 1) Fetch and checkout and rebase to latest edk2/master
> > >>>>
> > >>>> git fetch origin
> > >>>>
> > >>>> git checkout master
> > >>>>
> > >>>> git rebase origin/master
> > >>>>
> > >>>> 2) Make a backup copy of plugin UncrustifyCheck outside WORKSPACE.
> > >>>>
> > >>>> (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable and
> > >>>>
> > >>>> EDK II specific uncrustify configuration file available when working
> > >>>>
> > >>>> with a branch that does not have those tools in its scope.
> > >>>>
> > >>>> xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
> > >>>>
> > >>>> 3) Check out the patch series branch (e.g. MyBranch)
> > >>>>
> > >>>> git checkout MyBranch
> > >>>>
> > >>>> 4) Rebase patch series against edk2-stable202111
> > >>>>
> > >>>>
> > >>>>
> > >>>> git rebase edk2-stable202111
> > >>>>
> > >>>> 5) Create new branch for the uncrustifed version (e.g. MyBranch_Uncrustified)
> > >>>>
> > >>>> git checkout -b MyBranch_Uncrustified
> > >>>>
> > >>>> 6) Use git filter-branch to uncrustify all the commits in the series
> > >>>>
> > >>>> between the rebase target from (2) and HEAD of the branch.A filter
> > >>>>
> > >>>> can be used to scope the uncrustify operations to only the C/H files
> > >>>>
> > >>>> in the specific package the patch series is against. (e.g. DynamicTablesPkg).
> > >>>>
> > >>>> BaseTools should always be excluded.If the package scoped filter is
> > >>>>
> > >>>> not used, it will still work, but will take longer to run because
> > >>>>
> > >>>> uncrustify will rescan every C/H files in the whole repo.
> > >>>>
> > >>>> git filter-branch --tree-filter "git ls-files DynamicTablesPkg*.c DynamicTablesPkg*.h :!BaseTools/* |
> > >>> c:\\Temp\\UncrustifyCheck\\mu-uncrustify-release_extdep\\Windows-x86\\uncrustify.exe -c
> > >>> c:\\Temp\\UncrustifyCheck\\uncrustify.cfg -F - --replace --no-backup --if-changed" edk2-stable202111..HEAD
> > >>>>
> > >>>> 7) Now that all the individual patches in the branch are uncrustified,
> > >>>>
> > >>>> rebase against latest edk2/master that is already uncrustified.
> > >>>>
> > >>>> git rebase master
> > >>>>
> > >>>> 8) Verify the patches in this new branch.
> > >>>>
> > >>>> Impacts to tracing history across the uncrusity changes
> > >>>>
> > >>>> =======================================================
> > >>>>
> > >>>> Tools the view file and line history do work with the large uncrustify
> > >>>>
> > >>>> patch series.One impact is that the operations can be very slow due
> > >>>>
> > >>>> to the large uncrustify patches.
> > >>>>
> > >>>> One option to provide a faster experience is to provide an alternate
> > >>>>
> > >>>> version of the edk2 repository as "documentation" that has the
> > >>>>
> > >>>> entire history re-written with uncrustify run on every commit.
> > >>>>
> > >>>> The tool called git-filter-repo can be used to perform this
> > >>>>
> > >>>> transformation and runs in a reasonable period of time (a few hours)
> > >>>>
> > >>>> https://github.com/newren/git-filter-repo <https://github.com/newren/git-filter-repo>
> > >>>>
> > >>>> https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-demos/lint-history
> > >>> <https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-demos/lint-history>
> > >>>>
> > >>>> The following steps can be used to perform this transformation.
> > >>>>
> > >>>> This is the Windows version. I will add the Linux version soon.
> > >>>>
> > >>>> ** WARNING **This operation modifies(rewrites) all the commits
> > >>>>
> > >>>> in the local copy of the repo.Do not perform
> > >>>>
> > >>>> these steps on a local repo you are using for
> > >>>>
> > >>>> active development.
> > >>>>
> > >>>> 1) Clone edk2 into a new directory (see **WARNING**)
> > >>>>
> > >>>> git clonehttps://github.com/tianocore/edk2.git <https://github.com/tianocore/edk2.git> edk2-uncrustified
> > >>>>
> > >>>> cd edk2-uncrustified
> > >>>>
> > >>>> 2) Setup python virtual env, install pytools, and run stuart commands
> > >>>>
> > >>>> to setup build environment which includes installing uncrustify tools.
> > >>>>
> > >>>> https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally
> > >>> <https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally>
> > >>>>
> > >>>> 3) Make a backup copy of plugin UncrustifyCheck outside WORKSPACE.
> > >>>>
> > >>>> (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable and
> > >>>>
> > >>>> EDK II specific uncrustify configuration file available when working
> > >>>>
> > >>>> with a branch that does not have those tools in its scope.
> > >>>>
> > >>>> xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
> > >>>>
> > >>>> 4) Use lint-history.py from git-filter-repo examples
> > >>>>
> > >>>> https://github.com/newren/git-filter-repo <https://github.com/newren/git-filter-repo>
> > >>>>
> > >>>> https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-demos/lint-history
> > >>> <https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-demos/lint-history>
> > >>>>
> > >>>> Line #127 - Add try except around subprocess.check_call() with except
> > >>>>
> > >>>> being pass.This is required because there are a few commits of C
> > >>>>
> > >>>> files in the edk2 repo that have incorrect C syntax and do not
> > >>>>
> > >>>> build with a C compiler and break the uncrustify parser.Skip reformat
> > >>>>
> > >>>> of C files that can not be parsed by uncrustify.These rare instances
> > >>>>
> > >>>> are addressed in the commit that fixes the C syntax error.
> > >>>>
> > >>>> Run this slightly modified version of lint-history.Include only
> > >>>>
> > >>>> C/H files and exclude directories that start with 'Tools' or 'BaseTools'.
> > >>>>
> > >>>> This step took about 2.2 hours on a laptop.
> > >>>>
> > >>>> lint-history.py
> > >>>>
> > >>>> --relevant "return (not filename.startswith(b'Tools') and not filename.startswith(b'BaseTools') and
> > >>> (filename.endswith(b'.c') or filename.endswith(b'.h')))"
> > >>>>
> > >>>> c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\mu-uncrustify-release_extdep\\Windows-x86\\uncrustify.exe -
> c
> > >>> c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\uncrustify.cfg --replace --no-backup --if-changed
> > >>>>
> > >>>> Order of PRs to apply during extended hard freeze
> > >>>>
> > >>>> ==================================================
> > >>>>
> > >>>> 1) Update EmulatorPkg Win Host [BuildOptions] MSFT CC_FLAGS to not force debug information
> > >>>>
> > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3747 <https://bugzilla.tianocore.org/show_bug.cgi?id=3747>
> > >>>>
> > >>>> *https://github.com/mdkinney/edk2/tree/Bug_3747_EmulatorPkg_WinHost_ReproducibleBuild
> > >>> <https://github.com/mdkinney/edk2/tree/Bug_3747_EmulatorPkg_WinHost_ReproducibleBuild>
> > >>>>
> > >>>> *https://github.com/tianocore/edk2/pull/2215 <https://github.com/tianocore/edk2/pull/2215>
> > >>>>
> > >>>> * Required for EmulatorPkg to pass CompareBuild for VS2019 IA32/X64 builds.
> > >>>>
> > >>>> * Status: Review complete.PR pushed.
> > >>>>
> > >>>> 2) EccCheck should not revert staged and local changes
> > >>>>
> > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=2986 <https://bugzilla.tianocore.org/show_bug.cgi?id=2986>
> > >>>>
> > >>>> *https://github.com/mdkinney/edk2/tree/Bug_2986_EccCheckRemoveGitRevert_V2
> > >>> <https://github.com/mdkinney/edk2/tree/Bug_2986_EccCheckRemoveGitRevert_V2>
> > >>>>
> > >>>> *https://github.com/tianocore/edk2/pull/2216 <https://github.com/tianocore/edk2/pull/2216>
> > >>>>
> > >>>> * Required for EDK II CI to complete in a reasonable period of time when
> > >>>>
> > >>>> processing the 4000+ source file style changes made by uncrustify.
> > >>>>
> > >>>> * Also fixes critical bugs that can potentially corrupt git state when
> > >>>>
> > >>>> EccCheck is run locally.
> > >>>>
> > >>>> * Status: Review complete.PR pushed.
> > >>>>
> > >>>> 3) Update pytool LicenseCheck plugin to use temp directory for diff output file
> > >>>>
> > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3746 <https://bugzilla.tianocore.org/show_bug.cgi?id=3746>
> > >>>>
> > >>>> *https://github.com/mdkinney/edk2/tree/Bug_3746_LicenseCheckUseDiffOutputFile_V2
> > >>> <https://github.com/mdkinney/edk2/tree/Bug_3746_LicenseCheckUseDiffOutputFile_V2>
> > >>>>
> > >>>> *https://github.com/tianocore/edk2/pull/2217 <https://github.com/tianocore/edk2/pull/2217>
> > >>>>
> > >>>> * Required to reduce EDK II CI build times.
> > >>>>
> > >>>> * Status: Review complete.PR pushed.
> > >>>>
> > >>>> 4) Update max job time from 60 min to 120 minutes in .azurepipelines/templates
> > >>>>
> > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3750 <https://bugzilla.tianocore.org/show_bug.cgi?id=3750>
> > >>>>
> > >>>> *https://github.com/mdkinney/edk2/tree/Bug_3750_IncreaseAzurePipelinesTimeout
> > >>> <https://github.com/mdkinney/edk2/tree/Bug_3750_IncreaseAzurePipelinesTimeout>
> > >>>>
> > >>>> *https://github.com/tianocore/edk2/pull/2219 <https://github.com/tianocore/edk2/pull/2219>
> > >>>>
> > >>>> * Required to allow EccCheck of uncrustify changes to complete on Azure
> > >>>>
> > >>>> Pipelines CI agents without timing out.
> > >>>>
> > >>>> * Status: Review complete.PR pushed.
> > >>>>
> > >>>> 5) Update Package YAML to ignore specific ECC files/errors
> > >>>>
> > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3749 <https://bugzilla.tianocore.org/show_bug.cgi?id=3749>
> > >>>>
> > >>>> *https://github.com/mdkinney/edk2/tree/Bug_3749_EccCheckIgnoreFilesErrors
> > >>> <https://github.com/mdkinney/edk2/tree/Bug_3749_EccCheckIgnoreFilesErrors>
> > >>>>
> > >>>> *https://github.com/tianocore/edk2/pull/2218 <https://github.com/tianocore/edk2/pull/2218>
> > >>>>
> > >>>> * Required to pass EccCheck
> > >>>>
> > >>>> * Status: Review complete. PR pushed
> > >>>>
> > >>>> 6) Uncrustify Source Changes
> > >>>>
> > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3737 <https://bugzilla.tianocore.org/show_bug.cgi?id=3737>
> > >>>>
> > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3739 <https://bugzilla.tianocore.org/show_bug.cgi?id=3739>
> > >>>>
> > >>>> *https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
> > >>> <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5>
> > >>>>
> > >>>> *https://github.com/tianocore/edk2/pull/2229 <https://github.com/tianocore/edk2/pull/2229>
> > >>>>
> > >>>> * Build comparison result PASS:https://github.com/mdkinney/edk2/actions/runs/1521618836
> > >>> <https://github.com/mdkinney/edk2/actions/runs/1521618836>
> > >>>>
> > >>>> * EFI_D_ -> DEBUG changes required to pass PatchCheck
> > >>>>
> > >>>> * Uncrustify format changes required to pass UncrustifyCheck
> > >>>>
> > >>>> * Status:
> > >>>>
> > >>>> Waiting for review
> > >>>>
> > >>>> 7) UncrustifyCheck EDK II CI Plugin
> > >>>>
> > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3748 <https://bugzilla.tianocore.org/show_bug.cgi?id=3748>
> > >>>>
> > >>>> *https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v6
> > >>> <https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v6>
> > >>>>
> > >>>> * Required to enforce all PRs submitted to EDK II CI match uncrustify format.
> > >>>>
> > >>>> * Unconditionally checks all packages.Can not be committed until all C/H
> > >>>>
> > >>>> source files have been updated.
> > >>>>
> > >>>> * Status: Review complete
> > >>>>
> > >>>> Combined Branch/PR for Review/Test
> > >>>>
> > >>>> ==================================
> > >>>>
> > >>>> * Build Comparison results must pass 100% across the full set of PRs before
> > >>>>
> > >>>> the individual PRs can be pushed in the order listed above.
> > >>>>
> > >>>> * Branch:https://github.com/mdkinney/edk2/tree/TestOnly_Uncrustify_PR_Series
> > >>> <https://github.com/mdkinney/edk2/tree/TestOnly_Uncrustify_PR_Series>
> > >>>>
> > >>>> * PR:https://github.com/tianocore/edk2/pull/2229 <https://github.com/tianocore/edk2/pull/2229>
> > >>>>
> > >>>> Status = PASS
> > >>>>
> > >>>> * CompareBuild:
> > >>>>
> > >>>> Branch:https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
> > >>> <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5>
> > >>>>
> > >>>> --ref1:ef9a059cdb15844fe52a49af2bf7d86b9dd3e9bf
> > >>>>
> > >>>> --ref2:Bug_3737_3739_ApplyUncrustifyChanges_V5
> > >>>>
> > >>>> Extra Options: -n 4 --quiet
> > >>>>
> > >>>> Results:https://github.com/mdkinney/edk2/actions/runs/1521618836
> > >>> <https://github.com/mdkinney/edk2/actions/runs/1521618836>
> > >>>>
> > >>>> 30 VS2019 build comparisons PASS
> > >>>>
> > >>>> 40 GCC5 build comparisons PASS
> > >>>>
> > >>>> 100% PASS
> > >>>>
> > >>>> The following git log shows the set of patches from --ref1 to --ref 2across
> > >>>>
> > >>>> which there are no differences in any of the OBJ/LIB/DLL/EFI/FFS/FV/FD files.
> > >>>>
> > >>>> --ref2
> > >>>>
> > >>>> b7d4bf0675b7 (HEAD -> Bug_3737_3739_ApplyUncrustifyChanges_V5) UnitTestFrameworkPkg: Apply uncrusitify changes
> > >>>>
> > >>>> 7f03d25f60e7 UefiPayloadPkg: Apply uncrusitify changes
> > >>>>
> > >>>> 0bfd8d9b5ac9 UefiCpuPkg: Apply uncrusitify changes
> > >>>>
> > >>>> e1cd9bfb9dea StandaloneMmPkg: Apply uncrusitify changes
> > >>>>
> > >>>> 5da2f65be378 SourceLevelDebugPkg: Apply uncrusitify changes
> > >>>>
> > >>>> 95b86de07e5d SignedCapsulePkg: Apply uncrusitify changes
> > >>>>
> > >>>> fe71d97246c4 ShellPkg: Apply uncrusitify changes
> > >>>>
> > >>>> 54c21c952992 SecurityPkg: Apply uncrusitify changes
> > >>>>
> > >>>> 187a3785f12b RedfishPkg: Apply uncrusitify changes
> > >>>>
> > >>>> 810100002a46 PcAtChipsetPkg: Apply uncrusitify changes
> > >>>>
> > >>>> 276a695c0cf2 OvmfPkg: Apply uncrusitify changes
> > >>>>
> > >>>> 303c0a91ab07 NetworkPkg: Apply uncrusitify changes
> > >>>>
> > >>>> bc80792cd1b1 MdePkg: Apply uncrusitify changes
> > >>>>
> > >>>> 3ea86be17a2a MdeModulePkg: Apply uncrusitify changes
> > >>>>
> > >>>> c70ef11ed0cd IntelFsp2WrapperPkg: Apply uncrusitify changes
> > >>>>
> > >>>> c0291221f252 IntelFsp2Pkg: Apply uncrusitify changes
> > >>>>
> > >>>> 6a479952a690 FmpDevicePkg: Apply uncrusitify changes
> > >>>>
> > >>>> 3a7c05b7070d FatPkg: Apply uncrusitify changes
> > >>>>
> > >>>> b789f98c8959 EmulatorPkg: Apply uncrusitify changes
> > >>>>
> > >>>> 952d7a1c9220 EmbeddedPkg: Apply uncrusitify changes
> > >>>>
> > >>>> a1cc9881bab6 DynamicTablesPkg: Apply uncrusitify changes
> > >>>>
> > >>>> 50654dfe5785 CryptoPkg: Apply uncrusitify changes
> > >>>>
> > >>>> ed965a02dfa1 ArmVirtPkg: Apply uncrusitify changes
> > >>>>
> > >>>> 9744023fbc46 ArmPlatformPkg: Apply uncrusitify changes
> > >>>>
> > >>>> 7a1cde5f5bba ArmPkg: Apply uncrusitify changes
> > >>>>
> > >>>> 19d17e0913e8 UefiCpuPkg: Change use of EFI_D_* to DEBUG_*
> > >>>>
> > >>>> ffa718b4f994 SourceLevelDebugPkg: Change use of EFI_D_* to DEBUG_*
> > >>>>
> > >>>> b86cb3c5e5b4 ShellPkg: Change use of EFI_D_* to DEBUG_*
> > >>>>
> > >>>> c7c42204dc07 SecurityPkg: Change use of EFI_D_* to DEBUG_*
> > >>>>
> > >>>> 16b8e6f958e4 PcAtChipsetPkg: Change use of EFI_D_* to DEBUG_*
> > >>>>
> > >>>> 0ac3f8b2dac5 OvmfPkg: Change use of EFI_D_* to DEBUG_*
> > >>>>
> > >>>> bc5004b8d294 NetworkPkg: Change use of EFI_D_* to DEBUG_*
> > >>>>
> > >>>> 6f671a8e2377 MdePkg: Change use of EFI_D_* to DEBUG_*
> > >>>>
> > >>>> a10c610ff9a3 MdeModulePkg: Change use of EFI_D_* to DEBUG_*
> > >>>>
> > >>>> 09a3bddba390 FatPkg: Change use of EFI_D_* to DEBUG_*
> > >>>>
> > >>>> 59c61318246a EmulatorPkg: Change use of EFI_D_* to DEBUG_*
> > >>>>
> > >>>> 3a80367dda3b EmbeddedPkg: Change use of EFI_D_* to DEBUG_*
> > >>>>
> > >>>> 23eb1aaf80ca ArmVirtPkg: Change use of EFI_D_* to DEBUG_*
> > >>>>
> > >>>> 875914b45c54 ArmPlatformPkg: Change use of EFI_D_* to DEBUG_*
> > >>>>
> > >>>> eb2eca82b451 ArmPkg: Change use of EFI_D_* to DEBUG_*
> > >>>>
> > >>>> f0f3f5aae7c4 (origin/master, origin/HEAD, master) UnitTestFrameworkPkg: Update YAML to ignore specific ECC
> > >>> files/errors
> > >>>>
> > >>>> c05734797790 UefiPayloadPkg: Update YAML to ignore specific ECC files/errors
> > >>>>
> > >>>> c30c40d6c63d StandaloneMmPkg: Update YAML to ignore specific ECC files/errors
> > >>>>
> > >>>> 9944508e85f1 ShellPkg: Update YAML to ignore specific ECC files/errors
> > >>>>
> > >>>> 60fa40be458d SecurityPkg: Update YAML to ignore specific ECC files/errors
> > >>>>
> > >>>> df790cd6b37e MdePkg: Update YAML to ignore specific ECC files/errors
> > >>>>
> > >>>> 9deb9370766e MdeModulePkg: Update YAML to ignore specific ECC files/errors
> > >>>>
> > >>>> d7d30e8f219f EmulatorPkg: Update YAML to ignore specific ECC files/errors
> > >>>>
> > >>>> d5744ecba813 CryptoPkg: Update YAML to ignore specific ECC files/errors
> > >>>>
> > >>>> c97fee87f0f9 ArmVirtPkg: Update YAML to ignore specific ECC files/errors
> > >>>>
> > >>>> 1939fc9569f2 ArmPlatformPkg: Update YAML to ignore specific ECC files/errors
> > >>>>
> > >>>> 365dced2c37a ArmPkg: Update YAML to ignore specific ECC files/errors
> > >>>>
> > >>>> 76a1ce4d5fec .azurepipelines/templates: Update max pipeline job time to 2 hours
> > >>>>
> > >>>> 99f84ff47390 .pytools/Plugin/LicenseCheck: Use temp directory for git diff output
> > >>>>
> > >>>> 3019f1bbabf1 .pytool/Plugin/EccCheck: Add performance optimizations
> > >>>>
> > >>>> 854462bd3479 .pytool/Plugin/EccCheck: Remove temp directory on exception
> > >>>>
> > >>>> 69877614fdee .pytool/Plugin/EccCheck: Remove RevertCode()
> > >>>>
> > >>>> --ref1
> > >>>>
> > >>>> ef9a059cdb15 EmulatorPkg/Win/Host: Update CC_FLAGS
> > >>>>
> > >>>> bb1bba3d7767 (tag: edk2-stable202111) NetworkPkg: Fix invalid pointer for DNS response token on error
> > >>>>
> > >>>> Best regards,
> > >>>>
> > >>>> Mike
> > >>>>
> > >>>>
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
2021-11-30 22:34 Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4 Michael D Kinney
` (8 preceding siblings ...)
2021-12-02 18:27 ` [edk2-devel] " Maciej Rabeda
@ 2021-12-03 8:56 ` Gerd Hoffmann
2021-12-03 16:25 ` Michael D Kinney
9 siblings, 1 reply; 39+ messages in thread
From: Gerd Hoffmann @ 2021-12-03 8:56 UTC (permalink / raw)
To: devel, michael.d.kinney
Cc: Michael Kubacki, Andrew Fish (afish@apple.com), Leif Lindholm
Hi,
> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
> git fetch origin
> git checkout master
> git rebase origin/master
> 2) Make a backup copy of plugin UncrustifyCheck outside WORKSPACE.
> (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable and
> EDK II specific uncrustify configuration file available when working
> with a branch that does not have those tools in its scope.
>
> xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
That directory doesn't exist (neither in master nor in the
ApplyUncrustifyChanges branch).
take care,
Gerd
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
2021-12-03 8:56 ` Gerd Hoffmann
@ 2021-12-03 16:25 ` Michael D Kinney
0 siblings, 0 replies; 39+ messages in thread
From: Michael D Kinney @ 2021-12-03 16:25 UTC (permalink / raw)
To: Gerd Hoffmann, devel@edk2.groups.io, Kinney, Michael D
Cc: Michael Kubacki, Andrew Fish (afish@apple.com), Leif Lindholm
Hi Gerd,
> -----Original Message-----
> From: Gerd Hoffmann <kraxel@redhat.com>
> Sent: Friday, December 3, 2021 12:56 AM
> To: devel@edk2.groups.io; Kinney, Michael D <michael.d.kinney@intel.com>
> Cc: Michael Kubacki <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com) <afish@apple.com>; Leif Lindholm
> <leif@nuviainc.com>
> Subject: Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
>
> Hi,
>
> > https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
>
> > git fetch origin
> > git checkout master
> > git rebase origin/master
>
> > 2) Make a backup copy of plugin UncrustifyCheck outside WORKSPACE.
> > (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable and
> > EDK II specific uncrustify configuration file available when working
> > with a branch that does not have those tools in its scope.
> >
> > xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
>
> That directory doesn't exist (neither in master nor in the
> ApplyUncrustifyChanges branch).
It is not checked in yet, It will be the last PR in the series of PRs to enable
UncrustifyCheck as part of EDK II CI. There is a branch available to get this
content now to prepare for this being enabled.
7) UncrustifyCheck EDK II CI Plugin
* https://bugzilla.tianocore.org/show_bug.cgi?id=3748
* https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v6
* Required to enforce all PRs submitted to EDK II CI match uncrustify format.
* Unconditionally checks all packages. Cannot be committed until all C/H
source files have been updated.
* Status: Review complete
>
> take care,
> Gerd
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
2021-12-03 2:22 ` Michael D Kinney
@ 2021-12-06 1:17 ` Michael D Kinney
2021-12-06 1:23 ` Wu, Hao A
` (7 more replies)
0 siblings, 8 replies; 39+ messages in thread
From: Michael D Kinney @ 2021-12-06 1:17 UTC (permalink / raw)
To: Michael Kubacki, devel@edk2.groups.io,
maciej.rabeda@linux.intel.com, Michael Kubacki,
Andrew Fish (afish@apple.com), Leif Lindholm, Kinney, Michael D
Hello EDK II Maintainers,
A detailed evaluation of the DEBUG_CODE() formatting issue has been completed.
The reason DEBUG_CODE() is a challenge is that this looks like a macro from a
C parsing perspective, but the EDK II usage places C statements or blocks of
C code as the parameter to this macro.
There are actually 2 methods available to mark a statement or a block of code
to be included when the DEBUG_PROPERTY_DEBUG_CODE_ENABLED bit is enabled in
PcdDebugPropertyMask. One is DEBUG_CODE() and the other is to mark the beginning
and end of a code block with DEBUG_CODE_BEGIN() and DEBUG_CODE_END(). In fact,
DEBUG_CODE() is implemented using DEBUG_CODE_BEGIN() and DEBUG_CODE_END() macros.
#define DEBUG_CODE(Expression) \
DEBUG_CODE_BEGIN (); \
Expression \
DEBUG_CODE_END ()
A complete review for the use of these DEBUG_CODE macros was performed on the
edk2 repo. Uncrustify performs good formatting for code blocks between
DEBUG_CODE_BEGIN() and DEBUG_CODE_END(). This is because these look like simple
macros calls with no parameters and the lines of C code between these 2 macros
is formatted correctly.
The uncrustify formatting issues are only present with the use of DEBUG_CODE().
Simple use cases of DEBUG_CODE(Expression) where Expression is a single C
statement also look correct. A medium complexity use case where Expression is
a code block of simple statements or even some local variables and simple
statements also look correct. It is only complex code blocks that use C
statements such as if/for/while/case that include the use of braces {} does
uncrustify perform incorrect formatting.
The recommended solution to this issue is to convert the use of DEBUG_CODE()
to DEBUG_CODE_BEGIN() / DEBUG_CODE_END() for cases where the Expression
passed to DEBUG_CODE() is the complex use case that contains statements that
use braces {}. There are 57 instances of this pattern across 40 files in the
edk2 repo.
I have posted a branch with these additional patches:
https://github.com/mdkinney/edk2/tree/Bug_3737_3760_3739_ApplyUncrustifyChanges_V7
I have performed CompareBuild tests with this revised patch series with
the DEBUG_CODE changes. It passes 100% showing no binary differences.
https://github.com/mdkinney/edk2/actions/runs/1542454606
I have opened a PR to run this patch series through EDK II CI. It also passes 100%.
https://github.com/tianocore/edk2/pull/2236
The summary of changes made since the V6 review are:
1) Change uncrustify configuration assignment alignment threshold to 0
align_assign_thresh = 0
2) Replace "<param>, OPTIONAL" with "<param> OPTIONAL,"
3) Replace DEBUG_CODE(Expression) with
DEBUG_CODE_BEGIN();
Expression
DEBUG_CODE_END()
if Expression is complex and contains braces {}.
4) No changes to uncrustify tool required.
Please review the differences between the following 2 branches and provide
feedback or a Series Reviewed-by if you agree with these additional changes.
https://github.com/mdkinney/edk2/tree/Bug_3737_3760_3739_ApplyUncrustifyChanges_V6
https://github.com/mdkinney/edk2/tree/Bug_3737_3760_3739_ApplyUncrustifyChanges_V7
The goal is to complete the review and get the uncrustify change committed
tomorrow so the extended hard freeze can be lifted.
Thanks,
Mike
> -----Original Message-----
> From: Kinney, Michael D <michael.d.kinney@intel.com>
> Sent: Thursday, December 2, 2021 6:23 PM
> To: Michael Kubacki <mikuback@linux.microsoft.com>; devel@edk2.groups.io; maciej.rabeda@linux.intel.com; Michael Kubacki
> <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com) <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>;
> Kinney, Michael D <michael.d.kinney@intel.com>
> Subject: RE: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
>
> Hello EDK II Maintainers,
>
> I have entered BZ 3760 to make the use of the OPTIONAL keyword style consistent for all of edk2 repo
> and to be compatible with uncrustify.
>
> I have posted the following V6 branch that does the EFI_D_* to DEBUG_* changes, the OPTIONAL keyword
> style changes, and the uncrustify changes with the one configuration change for assignment alignment.
>
> https://github.com/mdkinney/edk2/tree/Bug_3737_3760_3739_ApplyUncrustifyChanges_V6
>
> Please provide feedback on the code style in this branch with the known DEBUG_CODE() issue still
> present.
>
> If we are able to quickly update uncrustify to handle DEBUG_CODE(), I will generate a V7.
>
> Thanks,
>
> Mike
>
> > -----Original Message-----
> > From: Kinney, Michael D <michael.d.kinney@intel.com>
> > Sent: Thursday, December 2, 2021 4:53 PM
> > To: Michael Kubacki <mikuback@linux.microsoft.com>; devel@edk2.groups.io; maciej.rabeda@linux.intel.com; Michael Kubacki
> > <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com) <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>;
> > Kinney, Michael D <michael.d.kinney@intel.com>
> > Subject: RE: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
> >
> > Michael,
> >
> > Yes. Please update the patch series that adds UncrustifyCheck with those changes.
> >
> > Thanks,
> >
> > Mike
> >
> > > -----Original Message-----
> > > From: Michael Kubacki <mikuback@linux.microsoft.com>
> > > Sent: Thursday, December 2, 2021 4:31 PM
> > > To: Kinney, Michael D <michael.d.kinney@intel.com>; devel@edk2.groups.io; maciej.rabeda@linux.intel.com; Michael
> Kubacki
> > > <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com) <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>
> > > Subject: Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
> > >
> > > Hi Mike,
> > >
> > > Thank you for the detailed analysis and recommendations.
> > >
> > > I agree with the recommendations and I will try to have an Uncrustify
> > > tool update by tomorrow for (4).
> > >
> > > That will require an update in uncrustify_ext_dep.yaml to pull in the
> > > new version. I'm assuming I should also update uncrustify.cfg to set
> > > align_assign_thresh = 0 in that new patch series?
> > >
> > > Regards,
> > > Michael
> > >
> > > On 12/2/2021 7:18 PM, Kinney, Michael D wrote:
> > > > Hi Michael,
> > > >
> > > > CORRECTION: set align_assign_threshold to 0.
> > > >
> > > > Reponses inline below.
> > > >
> > > > I would like to summarize the 4 issues raised in the past day along with the recommendations.
> > > >
> > > > 1) Exclusion feature for UncrustifyCheck. There are 2 directories with 8 files total that
> > > > maintainers have noted they would like to see not go through uncrustify formatting. Today
> > > > the only content that is skipped is BaseTools and submodules.
> > > >
> > > > Adding a general purpose exclusion feature would then require all developers to make
> > > > sure their method of using uncrustify also excludes those same areas. This requires
> > > > extra steps for all developers and maintainers.
> > > >
> > > > If we do not add the exclusion feature, then the 8 files will require an extra step
> > > > to sync with the original source of those files. The rate of changes of these 8 files
> > > > is very low today.
> > > >
> > > > RECOMMENDATION: Do not add exclusion feature at this time. Revisit if the extra work
> > > > to maintain the files that would be candidates for exclusions increases significantly.
> > > >
> > > > 2) Alignment of assignments. The threshold of 4 characters appears to be too low and causes
> > > > source files that are already aligned to become unaligned.
> > > >
> > > > RECOMMENDATION: Change threshold to the default value of 0 which means no limit.
> > > >
> > > > align_assign_thresh= 0
> > > >
> > > > 3) Alignment of parameters in function declaration not correct. The root cause of this
> > > > is the use of the OPTIONAL keyword. If the OPTIONAL keyword is removed, then the
> > > > alignment is correct. The alignment is also correct if the OPTIONAL keyword appears
> > > > before the ','. If the OPTIONAL keyword appears after the ',', then the format is
> > > > not correct. The OPTIONAL keyword indicates that the parameter in the function is
> > > > not required and may be passed in as NULL or 0 or some other default value defined by
> > > > the API. It makes more sense for this OPTIONAL keyword that follows the parameter
> > > > names to appear before the ',' so it is scoped to the parameter on that line. If it
> > > > appears after the ',', then C parsers thinks it is a prefix (IN, OUT, CONST, volatile,
> > > > static) for the next parameter in the function.
> > > >
> > > > RECOMMENDATION: Update patch series with a global search and replace so OPTIONAL
> > > > keyword always appears before the ',' on the same line.
> > > >
> > > > RegEx search string: ',( *)OPTIONAL( *)'
> > > > RegEx replace string: ' $1OPTIONAL,$2'
> > > >
> > > > 4) Format issues with complex blocks in DEBUG_CODE(), or between DEBUG_CODE_BEGIN() and
> > > > DEBUG_CODE_END(). Uncrustify treats these as macros and is not aware that the
> > > > parameter passed into the macro call is a block of C code that needs to be formatted.
> > > > Complex blocks with if/while/for/case statements are impacted the most.
> > > >
> > > > RECOMMENDATION: Update the uncrustify with an edk2 specific extension to treat these
> > > > macros as a block of code as if they were surrounded by an extra set of braces {}.
> > > >
> > > >
> > > > I have posted a branch for testing purposes that implements (2) and (3).
> > > >
> > > > Branch: https://github.com/mdkinney/edk2/tree/TestOnly_Bug_3737_3739_ApplyUncrustifyChanges_V6_OPTIONAL_Keyword_Fix
> > > > PR: https://github.com/tianocore/edk2/pull/2233
> > > > Status: PASS
> > > > CompareBuild: https://github.com/mdkinney/edk2/actions/runs/1532855914
> > > > Status: PASS
> > > >
> > > > You can see what changed by fetching and comparing the following 2 branches:
> > > >
> > > > https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
> > > > https://github.com/mdkinney/edk2/tree/TestOnly_Bug_3737_3739_ApplyUncrustifyChanges_V6_OPTIONAL_Keyword_Fix
> > > >
> > > > Please provide feedback on the RECOMMENDATIONS above. I will go ahead and prepare of V6 version of
> > > > the patch series now that that test results are all PASS.
> > > >
> > > > Best regards,
> > > >
> > > > Mike
> > > >
> > > >
> > > >> -----Original Message-----
> > > >> From: Kinney, Michael D <michael.d.kinney@intel.com>
> > > >> Sent: Thursday, December 2, 2021 4:15 PM
> > > >> To: Michael Kubacki <mikuback@linux.microsoft.com>; devel@edk2.groups.io; maciej.rabeda@linux.intel.com; Michael
> > > Kubacki
> > > >> <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com) <afish@apple.com>; Leif Lindholm
> <leif@nuviainc.com>;
> > > >> Kinney, Michael D <michael.d.kinney@intel.com>
> > > >> Subject: RE: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
> > > >>
> > > >> Hi Michael,
> > > >>
> > > >> Reponses inline below.
> > > >>
> > > >> I would like to summarize the 4 issues raised in the past day along with the recommendations.
> > > >>
> > > >> 1) Exclusion feature for UncrustifyCheck. There are 2 directories with 8 files total that
> > > >> maintainers have noted they would like to see not go through uncrustify formatting. Today
> > > >> the only content that is skipped is BaseTools and submodules.
> > > >>
> > > >> Adding a general purpose exclusion feature would then require all developers to make
> > > >> sure their method of using uncrustify also excludes those same areas. This requires
> > > >> extra steps for all developers and maintainers.
> > > >>
> > > >> If we do not add the exclusion feature, then the 8 files will require an extra step
> > > >> to sync with the original source of those files. The rate of changes of these 8 files
> > > >> is very low today.
> > > >>
> > > >> RECOMMENDATION: Do not add exclusion feature at this time. Revisit if the extra work
> > > >> to maintain the files that would be candidates for exclusions increases significantly.
> > > >>
> > > >> 2) Alignment of assignments. The threshold of 4 characters appears to be too low and causes
> > > >> source files that are already aligned to become unaligned.
> > > >>
> > > >> RECOMMENDATION: Change threshold to the default value of 0 which means no limit.
> > > >>
> > > >> align_assign_thresh= 4
> > > >>
> > > >> 3) Alignment of parameters in function declaration not correct. The root cause of this
> > > >> is the use of the OPTIONAL keyword. If the OPTIONAL keyword is removed, then the
> > > >> alignment is correct. The alignment is also correct if the OPTIONAL keyword appears
> > > >> before the ','. If the OPTIONAL keyword appears after the ',', then the format is
> > > >> not correct. The OPTIONAL keyword indicates that the parameter in the function is
> > > >> not required and may be passed in as NULL or 0 or some other default value defined by
> > > >> the API. It makes more sense for this OPTIONAL keyword that follows the parameter
> > > >> names to appear before the ',' so it is scoped to the parameter on that line. If it
> > > >> appears after the ',', then C parsers thinks it is a prefix (IN, OUT, CONST, volatile,
> > > >> static) for the next parameter in the function.
> > > >>
> > > >> RECOMMENDATION: Update patch series with a global search and replace so OPTIONAL
> > > >> keyword always appears before the ',' on the same line.
> > > >>
> > > >> RegEx search string: ',( *)OPTIONAL( *)'
> > > >> RegEx replace string: ' $1OPTIONAL,$2'
> > > >>
> > > >> 4) Format issues with complex blocks in DEBUG_CODE(), or between DEBUG_CODE_BEGIN() and
> > > >> DEBUG_CODE_END(). Uncrustify treats these as macros and is not aware that the
> > > >> parameter passed into the macro call is a block of C code that needs to be formatted.
> > > >> Complex blocks with if/while/for/case statements are impacted the most.
> > > >>
> > > >> RECOMMENDATION: Update the uncrustify with an edk2 specific extension to treat these
> > > >> macros as a block of code as if they were surrounded by an extra set of braces {}.
> > > >>
> > > >>
> > > >> I have posted a branch for testing purposes that implements (2) and (3).
> > > >>
> > > >> Branch: https://github.com/mdkinney/edk2/tree/TestOnly_Bug_3737_3739_ApplyUncrustifyChanges_V6_OPTIONAL_Keyword_Fix
> > > >> PR: https://github.com/tianocore/edk2/pull/2233
> > > >> Status: PASS
> > > >> CompareBuild: https://github.com/mdkinney/edk2/actions/runs/1532855914
> > > >> Status: PASS
> > > >>
> > > >> You can see what changed by fetching and comparing the following 2 branches:
> > > >>
> > > >> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
> > > >> https://github.com/mdkinney/edk2/tree/TestOnly_Bug_3737_3739_ApplyUncrustifyChanges_V6_OPTIONAL_Keyword_Fix
> > > >>
> > > >> Please provide feedback on the RECOMMENDATIONS above. I will go ahead and prepare of V6 version of
> > > >> the patch series now that that test results are all PASS.
> > > >>
> > > >> Best regards,
> > > >>
> > > >> Mike
> > > >>
> > > >>
> > > >>> -----Original Message-----
> > > >>> From: Michael Kubacki <mikuback@linux.microsoft.com>
> > > >>> Sent: Thursday, December 2, 2021 1:57 PM
> > > >>> To: devel@edk2.groups.io; Kinney, Michael D <michael.d.kinney@intel.com>; maciej.rabeda@linux.intel.com; Michael
> > > Kubacki
> > > >>> <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com) <afish@apple.com>; Leif Lindholm
> <leif@nuviainc.com>
> > > >>> Subject: Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
> > > >>>
> > > >>> My reply is inline.
> > > >>>
> > > >>> Regards,
> > > >>> Michael
> > > >>>
> > > >>> On 12/2/2021 2:45 PM, Michael D Kinney wrote:
> > > >>>> Hi Maciej,
> > > >>>>
> > > >>>> Thanks for the feedback.
> > > >>>>
> > > >>>> * Example #1.This appears to be caused by the following uncrustify
> > > >>>> settings:
> > > >>>>
> > > >>>> # The threshold for aligning on '=' in assignments.
> > > >>>>
> > > >>>> # Use a negative number for absolute thresholds.
> > > >>>>
> > > >>>> #
> > > >>>>
> > > >>>> # 0: No limit (default).
> > > >>>>
> > > >>>> align_assign_thresh= 0# number
> > > >>>>
> > > >>>> The edk2 setting for this is:
> > > >>>>
> > > >>>> align_assign_thresh= 4
> > > >>>>
> > > >>>> This means blocks of assignments that are different than more than 4
> > > >>>> spaces will be considered a new block.
> > > >>>>
> > > >>>> ‘HwAddreLen’ and ‘Xid’ are more than 4 characters in length
> > > >>>> different.Same for ‘Xid’ and ‘Reserved’.So
> > > >>>>
> > > >>>> uncrustify treats these as 3 different assignment blocks.
> > > >>>>
> > > >>>> If we change to the default value of 0: No limit, this example is
> > > >>>> treated as a single block and all ‘=’ are aligned.
> > > >>>>
> > > >>>> Is there a reason ‘4’ was selected?Is there is any harm in using the
> > > >>>> default of 0?
> > > >>>>
> > > >>> We can certainly change the threshold. '4' was derived by
> > > >>> experimentation. This is an area that is somewhat subjective and every
> > > >>> case is difficult to cover well. Please feel free to suggest any changes.
> > > >>>
> > > >>
> > > >> I recommend we use the default value of 0. That way, any code that choose to
> > > >> align assignments will still be aligned. If a developer does not like
> > > >> short assignments and long assignments in the same code block to use the
> > > >> long assignment column, they can always break the block up into multiple
> > > >> blocks by adding a carriage return.
> > > >>
> > > >>> For the benefit of others, this file can be a useful refernce
> > > >>> understanding spans, gaps, and thresholds.
> > > >>>
> > > >>> https://github.com/uncrustify/uncrustify/blob/master/documentation/htdocs/configuration.txt
> > > >>>
> > > >>>> * Example #2: Uncruistfy is confused by the DEBUG_CODE() macro.This is
> > > >>>> not a traditional macro because the
> > > >>>>
> > > >>>> contents of the macro is a block of C code.I do not know how to convince
> > > >>>> uncrustify that the contents of a
> > > >>>>
> > > >>>> macro like function call to be treated as a code block from an indent
> > > >>>> perspective.
> > > >>>>
> > > >>> I believe this would have to be overridden in the Uncrustify fork since
> > > >>> DEBUG_CODE() is being treated as a macro function call and code blocks
> > > >>> are not expected there. In addition, some special treatment might be
> > > >>> needed for alignment in between DEBUG_CODE_BEGIN()/DEBUG_CODE_END().
> > > >>>
> > > >>> I'm happy to look at this. However, regression validation can take a
> > > >>> while so I'd like to make sure we need this now. In cases that do not
> > > >>> have additional code blocks, it seems to format fairly well. Is this
> > > >>> prevalent and impactful enough it must be fixed now? Or, could we
> > > >>> revisit it with a follow up patch?
> > > >>
> > > >> How long does regression testing take? I see about 20 instances of
> > > >> DEBUG_CODE() that are producing bad formatting. If the content inside
> > > >> DEBUG_CODE() is a single line of a single block of statements without
> > > >> any if/while/for/case statements that require further indent, then the
> > > >> format looks ok.
> > > >>
> > > >> DEBUG_CODE_BEGIN() and DEBUG_CODE_END() would look better if the
> > > >> contents between are also indented one level.
> > > >>
> > > >>>
> > > >>>> * Example #3/#4: Uncrustify is confused by the OPTIONAL keyword.The
> > > >>>> edk2 config declares it as a QUALIFIER
> > > >>>>
> > > >>>> like IN and OUT.However, OPTIONAL only appears at the end of the line
> > > >>>> that declares a parameter to a
> > > >>>>
> > > >>>> function.There are 3 forms. One with comma after OPTIONAL.One with comma
> > > >>>> before OPTIONAL, and
> > > >>>>
> > > >>>> one with no comma if the parameter is the last parameter in the function
> > > >>>> declaration.
> > > >>>>
> > > >>>> TYPE ParamName OPTIONAL,
> > > >>>>
> > > >>>> TYPEParamName, OPTIONAL
> > > >>>>
> > > >>>> TYPEParamName OPTIONAL
> > > >>>>
> > > >>>> OPTIONAL is defined to nothing in edk2 builds.From a uncrustify
> > > >>>> perspective, we really want is to be
> > > >>>>
> > > >>>> ignored or more correctly treated it as a token that is attached to
> > > >>>> ParamName and the combination of
> > > >>>>
> > > >>>> ParamName and OPTIONAL treated as one unit to determine indents.
> > > >>>>
> > > >>> "TYPE ParamName, OPTIONAL" seems especially problematic and presents an
> > > >>> inconsistency with the other formats. Mike, can you please let me know
> > > >>> if you have the same observation and your thoughts on a consistent pattern?
> > > >>>
> > > >>>> Thanks,
> > > >>>>
> > > >>>> Mike
> > > >>>>
> > > >>>> *From:*devel@edk2.groups.io <devel@edk2.groups.io> *On Behalf Of *Maciej
> > > >>>> Rabeda
> > > >>>> *Sent:* Thursday, December 2, 2021 10:27 AM
> > > >>>> *To:* devel@edk2.groups.io; Kinney, Michael D
> > > >>>> <michael.d.kinney@intel.com>; Michael Kubacki
> > > >>>> <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com)
> > > >>>> <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>
> > > >>>> *Subject:* Re: [edk2-devel] Uncrustify Conversion Detailed Plan and
> > > >>>> Extended Hard Freeze Update #4
> > > >>>>
> > > >>>> Hey Mike,
> > > >>>>
> > > >>>> While most of the changes related to fixing coding style violations,
> > > >>>> there are a couple of changes to NetworkPkg in that PR that make the
> > > >>>> code less readable. Examples below.
> > > >>>>
> > > >>>> Example 1:
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>> Example 2:
> > > >>>>
> > > >>>>
> > > >>>> Example 3:
> > > >>>>
> > > >>>>
> > > >>>> Example 4:
> > > >>>>
> > > >>>> On 30-Nov-21 23:34, Michael D Kinney wrote:
> > > >>>>
> > > >>>> Hello,
> > > >>>>
> > > >>>> Thank you for your patience during this extended hard freeze.
> > > >>>>
> > > >>>> Just one more step to go.There has been a delay in the review of
> > > >>>>
> > > >>>> the patch series with the uncrustify source changes.PR(6).This
> > > >>>>
> > > >>>> patch series was not sent out as patch review email because of its
> > > >>>>
> > > >>>> very large size.It only contains source style changes and the
> > > >>>>
> > > >>>> CompareBuild tool and GitHub action has shown there are no binary
> > > >>>>
> > > >>>> differences introduced with these source style changes.
> > > >>>>
> > > >>>> If you are a package maintainer, then please review the following
> > > >>>>
> > > >>>> branch/PR for your package contents and review the EDK II CI results
> > > >>>>
> > > >>>> and BuildCompare results.I do not expect a line by line review
> > > >>>>
> > > >>>> because we already had time to provide feedback on the source style
> > > >>>>
> > > >>>> performed by uncrustify.Instead, a Reviewed-by for your package
> > > >>>>
> > > >>>> indicates that you have reviewed the EDK II CI results and CompareBuild
> > > >>>>
> > > >>>> tool functionality and results and you accept the source style
> > > >>>>
> > > >>>> changes to your package.
> > > >>>>
> > > >>>> *https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
> > > >>> <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5>
> > > >>>>
> > > >>>> *https://github.com/tianocore/edk2/pull/2229 <https://github.com/tianocore/edk2/pull/2229>
> > > >>>>
> > > >>>> *https://github.com/mdkinney/edk2/actions/runs/1521618836
> > > >>> <https://github.com/mdkinney/edk2/actions/runs/1521618836>
> > > >>>>
> > > >>>> Additional details on this update below.
> > > >>>>
> > > >>>> Thank you,
> > > >>>>
> > > >>>> Mike
> > > >>>>
> > > >>>> Changes from Update #3
> > > >>>>
> > > >>>> ----------------------------------------------------------------------------
> > > >>>>
> > > >>>> * Pushed PR (5)
> > > >>>>
> > > >>>> * Added link to PR(6). EDK II CI Status is PASS. Build Compare PASS.
> > > >>>>
> > > >>>> * Waiting for review of PR (6)
> > > >>>>
> > > >>>> * Review of PR (7) completed and waiting for review of PR (6)
> > > >>>>
> > > >>>> ----------------------------------------------------------------------------
> > > >>>>
> > > >>>> Changes from Update #2
> > > >>>>
> > > >>>> ----------------------------------------------------------------------------
> > > >>>>
> > > >>>> * Changed order of PRs swapping (4) and (5).The PR that activates
> > > >>>>
> > > >>>> increases the max CI agent job time is independent of all the other
> > > >>>>
> > > >>>> PRs and its review is complete, so it can be committed now.
> > > >>>>
> > > >>>> * Pushed PRs (1), (2), (3), (4).
> > > >>>>
> > > >>>> * Waiting for review to complete for PRs (5) and (6)
> > > >>>>
> > > >>>> * Reviews complete for PR (7)
> > > >>>>
> > > >>>> * Identifies steps using git filter-branch to apply uncrustify changes to a
> > > >>>>
> > > >>>> code review patch series that was generated before the uncrustify changes
> > > >>>>
> > > >>>> avoiding manual merge.
> > > >>>>
> > > >>>> * Identified steps using git filter-repo to generate an alternate history of
> > > >>>>
> > > >>>> the edk2 repo with uncrustify changes applied on every commit.This may
> > > >>>>
> > > >>>> be useful when evaluating changes to files using tools like git blame
> > > >>>>
> > > >>>> without the large uncrustify patch series.
> > > >>>>
> > > >>>> ---------------------------------------------------------------------------
> > > >>>>
> > > >>>> Changes from Update #1
> > > >>>>
> > > >>>> ----------------------------------------------------------------------------
> > > >>>>
> > > >>>> * Changed order of PRs swapping (6) and (7).The PR that activates
> > > >>>>
> > > >>>> EDK II CI check UncrustifyCheck has to be last because it unconditionally
> > > >>>>
> > > >>>> checks all C/H files in all packages.Not just files that have been
> > > >>>>
> > > >>>> modified like some of the other checkers.
> > > >>>>
> > > >>>> * Updated link to the branch with the UncrustifyCheck plugin that has been
> > > >>>>
> > > >>>> updated with a one line change and Reviewed-by and Tested-by tags.
> > > >>>>
> > > >>>> https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v6
> > > >>> <https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v6>
> > > >>>>
> > > >>>> * Reviews complete for (1), (2), (3), (5), and (7)
> > > >>>>
> > > >>>> ---------------------------------------------------------------------------
> > > >>>>
> > > >>>> Michael Kubacki and I have prepared the patches required to apply the
> > > >>>>
> > > >>>> uncrustify changes and enable EDK II CI to check all submitted
> > > >>>>
> > > >>>> patches have been run through uncrustify.
> > > >>>>
> > > >>>> We have verified through the CompareBuild GitHub Action that the
> > > >>>>
> > > >>>> format changes performed by uncrustify have no functional changes.
> > > >>>>
> > > >>>> All of the OBJ, LIB, DLL, EFI, FFS, FV, and FD files match 100%
> > > >>>>
> > > >>>> across 70 VS2019/GCC5 builds of all package/platform DSC files in
> > > >>>>
> > > >>>> the edk2 repo.
> > > >>>>
> > > >>>> The hard freeze will be extended after the edk2-stable202111 tag until
> > > >>>>
> > > >>>> all uncrustify related changes are committed.We do not expect this
> > > >>>>
> > > >>>> to take more than a few days.Do not push any PRs until the hard
> > > >>>>
> > > >>>> freeze is lifted.
> > > >>>>
> > > >>>> The changes are broken up into 7 patch series/PRs.The PRs are ordered
> > > >>>>
> > > >>>> so they can be submitted using the normal submission process and EDK II
> > > >>>>
> > > >>>> CI will pass for each one.Details are listed below.
> > > >>>>
> > > >>>> Uncrustify 73.0.3 for EDK II
> > > >>>>
> > > >>>> =============================
> > > >>>>
> > > >>>> * Sources:https://dev.azure.com/projectmu/_git/Uncrustify <https://dev.azure.com/projectmu/_git/Uncrustify>
> > > >>>>
> > > >>>> * Documentation:https://dev.azure.com/projectmu/Uncrustify/_wiki/wikis/Uncrustify.wiki/1/Project-Mu-(EDK-
> II)-
> > > Fork-
> > > >>> Readme <https://dev.azure.com/projectmu/Uncrustify/_wiki/wikis/Uncrustify.wiki/1/Project-Mu-(EDK-II)-Fork-Readme>
> > > >>>>
> > > >>>> * Download:https://dev.azure.com/projectmu/Uncrustify/_packaging?_a=package&feed=mu_uncrustify&package=mu-
> > > >>> uncrustify-release&protocolType=NuGet&version=73.0.3
> > > >>> <https://dev.azure.com/projectmu/Uncrustify/_packaging?_a=package&feed=mu_uncrustify&package=mu-uncrustify-
> > > >>> release&protocolType=NuGet&version=73.0.3>
> > > >>>>
> > > >>>> Installing Uncrustify
> > > >>>>
> > > >>>> ======================
> > > >>>>
> > > >>>> The Uncrustify tool is installed automatically when the Pytools
> > > >>>>
> > > >>>> environment is used and the stuart* commands are run to complete the
> > > >>>>
> > > >>>> environment setup.Please see:
> > > >>>>
> > > >>>> https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally
> > > >>> <https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally>
> > > >>>>
> > > >>>> Uncrustify can also be installed from the download page listed above
> > > >>>>
> > > >>>> or built from sources from the source link above.
> > > >>>>
> > > >>>> The Documentation link provides instruction on how to run uncrustify from
> > > >>>>
> > > >>>> the command line or install as a Visual Studio Code plugin.The main
> > > >>>>
> > > >>>> uncrustify documentation also describes how to integrate with a few other
> > > >>>>
> > > >>>> editors.
> > > >>>>
> > > >>>> We have also discussed a client side githook.That effort has not started.
> > > >>>>
> > > >>>> Let us know if that is a feature you would find useful.
> > > >>>>
> > > >>>> Developer impact for new code reviews
> > > >>>>
> > > >>>> ======================================
> > > >>>>
> > > >>>> Once the uncrustify checker is active in EDK II CI, developers must
> > > >>>>
> > > >>>> make sure their patches are run through the uncrustify tool before
> > > >>>>
> > > >>>> sending the patches for review.
> > > >>>>
> > > >>>> Developers must install and run uncrustify against changes files before
> > > >>>>
> > > >>>> sending patch review emails or submitting PR for EDK II CI.If EDK II CI
> > > >>>>
> > > >>>> detects and differences in source formatting, then EDK II CI will fail
> > > >>>>
> > > >>>> and the developer must run uncrustify and resubmit the patches.
> > > >>>>
> > > >>>> Developer impact to patch series/PRs reviewed during edk2-stable201121 soft/hard freeze
> > > >>>>
> > > >>>> =======================================================================================
> > > >>>>
> > > >>>> Developers must rebase their changes after the uncrustify source changes are
> > > >>>>
> > > >>>> committed.The branch with a preview of the uncrustify changes can be used
> > > >>>>
> > > >>>> to start this rebase work.
> > > >>>>
> > > >>>> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
> > > >>> <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5>
> > > >>>>
> > > >>>> The following steps can be used to update an existing branch with the
> > > >>>>
> > > >>>> required uncrustify format.This is the Windows version.I will add
> > > >>>>
> > > >>>> the Linux version soon.
> > > >>>>
> > > >>>> 1) Fetch and checkout and rebase to latest edk2/master
> > > >>>>
> > > >>>> git fetch origin
> > > >>>>
> > > >>>> git checkout master
> > > >>>>
> > > >>>> git rebase origin/master
> > > >>>>
> > > >>>> 2) Make a backup copy of plugin UncrustifyCheck outside WORKSPACE.
> > > >>>>
> > > >>>> (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable and
> > > >>>>
> > > >>>> EDK II specific uncrustify configuration file available when working
> > > >>>>
> > > >>>> with a branch that does not have those tools in its scope.
> > > >>>>
> > > >>>> xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
> > > >>>>
> > > >>>> 3) Check out the patch series branch (e.g. MyBranch)
> > > >>>>
> > > >>>> git checkout MyBranch
> > > >>>>
> > > >>>> 4) Rebase patch series against edk2-stable202111
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>> git rebase edk2-stable202111
> > > >>>>
> > > >>>> 5) Create new branch for the uncrustifed version (e.g. MyBranch_Uncrustified)
> > > >>>>
> > > >>>> git checkout -b MyBranch_Uncrustified
> > > >>>>
> > > >>>> 6) Use git filter-branch to uncrustify all the commits in the series
> > > >>>>
> > > >>>> between the rebase target from (2) and HEAD of the branch.A filter
> > > >>>>
> > > >>>> can be used to scope the uncrustify operations to only the C/H files
> > > >>>>
> > > >>>> in the specific package the patch series is against. (e.g. DynamicTablesPkg).
> > > >>>>
> > > >>>> BaseTools should always be excluded.If the package scoped filter is
> > > >>>>
> > > >>>> not used, it will still work, but will take longer to run because
> > > >>>>
> > > >>>> uncrustify will rescan every C/H files in the whole repo.
> > > >>>>
> > > >>>> git filter-branch --tree-filter "git ls-files DynamicTablesPkg*.c DynamicTablesPkg*.h :!BaseTools/* |
> > > >>> c:\\Temp\\UncrustifyCheck\\mu-uncrustify-release_extdep\\Windows-x86\\uncrustify.exe -c
> > > >>> c:\\Temp\\UncrustifyCheck\\uncrustify.cfg -F - --replace --no-backup --if-changed" edk2-stable202111..HEAD
> > > >>>>
> > > >>>> 7) Now that all the individual patches in the branch are uncrustified,
> > > >>>>
> > > >>>> rebase against latest edk2/master that is already uncrustified.
> > > >>>>
> > > >>>> git rebase master
> > > >>>>
> > > >>>> 8) Verify the patches in this new branch.
> > > >>>>
> > > >>>> Impacts to tracing history across the uncrusity changes
> > > >>>>
> > > >>>> =======================================================
> > > >>>>
> > > >>>> Tools the view file and line history do work with the large uncrustify
> > > >>>>
> > > >>>> patch series.One impact is that the operations can be very slow due
> > > >>>>
> > > >>>> to the large uncrustify patches.
> > > >>>>
> > > >>>> One option to provide a faster experience is to provide an alternate
> > > >>>>
> > > >>>> version of the edk2 repository as "documentation" that has the
> > > >>>>
> > > >>>> entire history re-written with uncrustify run on every commit.
> > > >>>>
> > > >>>> The tool called git-filter-repo can be used to perform this
> > > >>>>
> > > >>>> transformation and runs in a reasonable period of time (a few hours)
> > > >>>>
> > > >>>> https://github.com/newren/git-filter-repo <https://github.com/newren/git-filter-repo>
> > > >>>>
> > > >>>> https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-demos/lint-history
> > > >>> <https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-demos/lint-history>
> > > >>>>
> > > >>>> The following steps can be used to perform this transformation.
> > > >>>>
> > > >>>> This is the Windows version. I will add the Linux version soon.
> > > >>>>
> > > >>>> ** WARNING **This operation modifies(rewrites) all the commits
> > > >>>>
> > > >>>> in the local copy of the repo.Do not perform
> > > >>>>
> > > >>>> these steps on a local repo you are using for
> > > >>>>
> > > >>>> active development.
> > > >>>>
> > > >>>> 1) Clone edk2 into a new directory (see **WARNING**)
> > > >>>>
> > > >>>> git clonehttps://github.com/tianocore/edk2.git <https://github.com/tianocore/edk2.git> edk2-uncrustified
> > > >>>>
> > > >>>> cd edk2-uncrustified
> > > >>>>
> > > >>>> 2) Setup python virtual env, install pytools, and run stuart commands
> > > >>>>
> > > >>>> to setup build environment which includes installing uncrustify tools.
> > > >>>>
> > > >>>> https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally
> > > >>> <https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally>
> > > >>>>
> > > >>>> 3) Make a backup copy of plugin UncrustifyCheck outside WORKSPACE.
> > > >>>>
> > > >>>> (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable and
> > > >>>>
> > > >>>> EDK II specific uncrustify configuration file available when working
> > > >>>>
> > > >>>> with a branch that does not have those tools in its scope.
> > > >>>>
> > > >>>> xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
> > > >>>>
> > > >>>> 4) Use lint-history.py from git-filter-repo examples
> > > >>>>
> > > >>>> https://github.com/newren/git-filter-repo <https://github.com/newren/git-filter-repo>
> > > >>>>
> > > >>>> https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-demos/lint-history
> > > >>> <https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-demos/lint-history>
> > > >>>>
> > > >>>> Line #127 - Add try except around subprocess.check_call() with except
> > > >>>>
> > > >>>> being pass.This is required because there are a few commits of C
> > > >>>>
> > > >>>> files in the edk2 repo that have incorrect C syntax and do not
> > > >>>>
> > > >>>> build with a C compiler and break the uncrustify parser.Skip reformat
> > > >>>>
> > > >>>> of C files that can not be parsed by uncrustify.These rare instances
> > > >>>>
> > > >>>> are addressed in the commit that fixes the C syntax error.
> > > >>>>
> > > >>>> Run this slightly modified version of lint-history.Include only
> > > >>>>
> > > >>>> C/H files and exclude directories that start with 'Tools' or 'BaseTools'.
> > > >>>>
> > > >>>> This step took about 2.2 hours on a laptop.
> > > >>>>
> > > >>>> lint-history.py
> > > >>>>
> > > >>>> --relevant "return (not filename.startswith(b'Tools') and not filename.startswith(b'BaseTools') and
> > > >>> (filename.endswith(b'.c') or filename.endswith(b'.h')))"
> > > >>>>
> > > >>>> c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\mu-uncrustify-release_extdep\\Windows-x86\\uncrustify.exe
> -
> > c
> > > >>> c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\uncrustify.cfg --replace --no-backup --if-changed
> > > >>>>
> > > >>>> Order of PRs to apply during extended hard freeze
> > > >>>>
> > > >>>> ==================================================
> > > >>>>
> > > >>>> 1) Update EmulatorPkg Win Host [BuildOptions] MSFT CC_FLAGS to not force debug information
> > > >>>>
> > > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3747 <https://bugzilla.tianocore.org/show_bug.cgi?id=3747>
> > > >>>>
> > > >>>> *https://github.com/mdkinney/edk2/tree/Bug_3747_EmulatorPkg_WinHost_ReproducibleBuild
> > > >>> <https://github.com/mdkinney/edk2/tree/Bug_3747_EmulatorPkg_WinHost_ReproducibleBuild>
> > > >>>>
> > > >>>> *https://github.com/tianocore/edk2/pull/2215 <https://github.com/tianocore/edk2/pull/2215>
> > > >>>>
> > > >>>> * Required for EmulatorPkg to pass CompareBuild for VS2019 IA32/X64 builds.
> > > >>>>
> > > >>>> * Status: Review complete.PR pushed.
> > > >>>>
> > > >>>> 2) EccCheck should not revert staged and local changes
> > > >>>>
> > > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=2986 <https://bugzilla.tianocore.org/show_bug.cgi?id=2986>
> > > >>>>
> > > >>>> *https://github.com/mdkinney/edk2/tree/Bug_2986_EccCheckRemoveGitRevert_V2
> > > >>> <https://github.com/mdkinney/edk2/tree/Bug_2986_EccCheckRemoveGitRevert_V2>
> > > >>>>
> > > >>>> *https://github.com/tianocore/edk2/pull/2216 <https://github.com/tianocore/edk2/pull/2216>
> > > >>>>
> > > >>>> * Required for EDK II CI to complete in a reasonable period of time when
> > > >>>>
> > > >>>> processing the 4000+ source file style changes made by uncrustify.
> > > >>>>
> > > >>>> * Also fixes critical bugs that can potentially corrupt git state when
> > > >>>>
> > > >>>> EccCheck is run locally.
> > > >>>>
> > > >>>> * Status: Review complete.PR pushed.
> > > >>>>
> > > >>>> 3) Update pytool LicenseCheck plugin to use temp directory for diff output file
> > > >>>>
> > > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3746 <https://bugzilla.tianocore.org/show_bug.cgi?id=3746>
> > > >>>>
> > > >>>> *https://github.com/mdkinney/edk2/tree/Bug_3746_LicenseCheckUseDiffOutputFile_V2
> > > >>> <https://github.com/mdkinney/edk2/tree/Bug_3746_LicenseCheckUseDiffOutputFile_V2>
> > > >>>>
> > > >>>> *https://github.com/tianocore/edk2/pull/2217 <https://github.com/tianocore/edk2/pull/2217>
> > > >>>>
> > > >>>> * Required to reduce EDK II CI build times.
> > > >>>>
> > > >>>> * Status: Review complete.PR pushed.
> > > >>>>
> > > >>>> 4) Update max job time from 60 min to 120 minutes in .azurepipelines/templates
> > > >>>>
> > > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3750 <https://bugzilla.tianocore.org/show_bug.cgi?id=3750>
> > > >>>>
> > > >>>> *https://github.com/mdkinney/edk2/tree/Bug_3750_IncreaseAzurePipelinesTimeout
> > > >>> <https://github.com/mdkinney/edk2/tree/Bug_3750_IncreaseAzurePipelinesTimeout>
> > > >>>>
> > > >>>> *https://github.com/tianocore/edk2/pull/2219 <https://github.com/tianocore/edk2/pull/2219>
> > > >>>>
> > > >>>> * Required to allow EccCheck of uncrustify changes to complete on Azure
> > > >>>>
> > > >>>> Pipelines CI agents without timing out.
> > > >>>>
> > > >>>> * Status: Review complete.PR pushed.
> > > >>>>
> > > >>>> 5) Update Package YAML to ignore specific ECC files/errors
> > > >>>>
> > > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3749 <https://bugzilla.tianocore.org/show_bug.cgi?id=3749>
> > > >>>>
> > > >>>> *https://github.com/mdkinney/edk2/tree/Bug_3749_EccCheckIgnoreFilesErrors
> > > >>> <https://github.com/mdkinney/edk2/tree/Bug_3749_EccCheckIgnoreFilesErrors>
> > > >>>>
> > > >>>> *https://github.com/tianocore/edk2/pull/2218 <https://github.com/tianocore/edk2/pull/2218>
> > > >>>>
> > > >>>> * Required to pass EccCheck
> > > >>>>
> > > >>>> * Status: Review complete. PR pushed
> > > >>>>
> > > >>>> 6) Uncrustify Source Changes
> > > >>>>
> > > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3737 <https://bugzilla.tianocore.org/show_bug.cgi?id=3737>
> > > >>>>
> > > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3739 <https://bugzilla.tianocore.org/show_bug.cgi?id=3739>
> > > >>>>
> > > >>>> *https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
> > > >>> <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5>
> > > >>>>
> > > >>>> *https://github.com/tianocore/edk2/pull/2229 <https://github.com/tianocore/edk2/pull/2229>
> > > >>>>
> > > >>>> * Build comparison result PASS:https://github.com/mdkinney/edk2/actions/runs/1521618836
> > > >>> <https://github.com/mdkinney/edk2/actions/runs/1521618836>
> > > >>>>
> > > >>>> * EFI_D_ -> DEBUG changes required to pass PatchCheck
> > > >>>>
> > > >>>> * Uncrustify format changes required to pass UncrustifyCheck
> > > >>>>
> > > >>>> * Status:
> > > >>>>
> > > >>>> Waiting for review
> > > >>>>
> > > >>>> 7) UncrustifyCheck EDK II CI Plugin
> > > >>>>
> > > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3748 <https://bugzilla.tianocore.org/show_bug.cgi?id=3748>
> > > >>>>
> > > >>>> *https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v6
> > > >>> <https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v6>
> > > >>>>
> > > >>>> * Required to enforce all PRs submitted to EDK II CI match uncrustify format.
> > > >>>>
> > > >>>> * Unconditionally checks all packages.Can not be committed until all C/H
> > > >>>>
> > > >>>> source files have been updated.
> > > >>>>
> > > >>>> * Status: Review complete
> > > >>>>
> > > >>>> Combined Branch/PR for Review/Test
> > > >>>>
> > > >>>> ==================================
> > > >>>>
> > > >>>> * Build Comparison results must pass 100% across the full set of PRs before
> > > >>>>
> > > >>>> the individual PRs can be pushed in the order listed above.
> > > >>>>
> > > >>>> * Branch:https://github.com/mdkinney/edk2/tree/TestOnly_Uncrustify_PR_Series
> > > >>> <https://github.com/mdkinney/edk2/tree/TestOnly_Uncrustify_PR_Series>
> > > >>>>
> > > >>>> * PR:https://github.com/tianocore/edk2/pull/2229 <https://github.com/tianocore/edk2/pull/2229>
> > > >>>>
> > > >>>> Status = PASS
> > > >>>>
> > > >>>> * CompareBuild:
> > > >>>>
> > > >>>> Branch:https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
> > > >>> <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5>
> > > >>>>
> > > >>>> --ref1:ef9a059cdb15844fe52a49af2bf7d86b9dd3e9bf
> > > >>>>
> > > >>>> --ref2:Bug_3737_3739_ApplyUncrustifyChanges_V5
> > > >>>>
> > > >>>> Extra Options: -n 4 --quiet
> > > >>>>
> > > >>>> Results:https://github.com/mdkinney/edk2/actions/runs/1521618836
> > > >>> <https://github.com/mdkinney/edk2/actions/runs/1521618836>
> > > >>>>
> > > >>>> 30 VS2019 build comparisons PASS
> > > >>>>
> > > >>>> 40 GCC5 build comparisons PASS
> > > >>>>
> > > >>>> 100% PASS
> > > >>>>
> > > >>>> The following git log shows the set of patches from --ref1 to --ref 2across
> > > >>>>
> > > >>>> which there are no differences in any of the OBJ/LIB/DLL/EFI/FFS/FV/FD files.
> > > >>>>
> > > >>>> --ref2
> > > >>>>
> > > >>>> b7d4bf0675b7 (HEAD -> Bug_3737_3739_ApplyUncrustifyChanges_V5) UnitTestFrameworkPkg: Apply uncrusitify
> changes
> > > >>>>
> > > >>>> 7f03d25f60e7 UefiPayloadPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> 0bfd8d9b5ac9 UefiCpuPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> e1cd9bfb9dea StandaloneMmPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> 5da2f65be378 SourceLevelDebugPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> 95b86de07e5d SignedCapsulePkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> fe71d97246c4 ShellPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> 54c21c952992 SecurityPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> 187a3785f12b RedfishPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> 810100002a46 PcAtChipsetPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> 276a695c0cf2 OvmfPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> 303c0a91ab07 NetworkPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> bc80792cd1b1 MdePkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> 3ea86be17a2a MdeModulePkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> c70ef11ed0cd IntelFsp2WrapperPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> c0291221f252 IntelFsp2Pkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> 6a479952a690 FmpDevicePkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> 3a7c05b7070d FatPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> b789f98c8959 EmulatorPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> 952d7a1c9220 EmbeddedPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> a1cc9881bab6 DynamicTablesPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> 50654dfe5785 CryptoPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> ed965a02dfa1 ArmVirtPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> 9744023fbc46 ArmPlatformPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> 7a1cde5f5bba ArmPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> 19d17e0913e8 UefiCpuPkg: Change use of EFI_D_* to DEBUG_*
> > > >>>>
> > > >>>> ffa718b4f994 SourceLevelDebugPkg: Change use of EFI_D_* to DEBUG_*
> > > >>>>
> > > >>>> b86cb3c5e5b4 ShellPkg: Change use of EFI_D_* to DEBUG_*
> > > >>>>
> > > >>>> c7c42204dc07 SecurityPkg: Change use of EFI_D_* to DEBUG_*
> > > >>>>
> > > >>>> 16b8e6f958e4 PcAtChipsetPkg: Change use of EFI_D_* to DEBUG_*
> > > >>>>
> > > >>>> 0ac3f8b2dac5 OvmfPkg: Change use of EFI_D_* to DEBUG_*
> > > >>>>
> > > >>>> bc5004b8d294 NetworkPkg: Change use of EFI_D_* to DEBUG_*
> > > >>>>
> > > >>>> 6f671a8e2377 MdePkg: Change use of EFI_D_* to DEBUG_*
> > > >>>>
> > > >>>> a10c610ff9a3 MdeModulePkg: Change use of EFI_D_* to DEBUG_*
> > > >>>>
> > > >>>> 09a3bddba390 FatPkg: Change use of EFI_D_* to DEBUG_*
> > > >>>>
> > > >>>> 59c61318246a EmulatorPkg: Change use of EFI_D_* to DEBUG_*
> > > >>>>
> > > >>>> 3a80367dda3b EmbeddedPkg: Change use of EFI_D_* to DEBUG_*
> > > >>>>
> > > >>>> 23eb1aaf80ca ArmVirtPkg: Change use of EFI_D_* to DEBUG_*
> > > >>>>
> > > >>>> 875914b45c54 ArmPlatformPkg: Change use of EFI_D_* to DEBUG_*
> > > >>>>
> > > >>>> eb2eca82b451 ArmPkg: Change use of EFI_D_* to DEBUG_*
> > > >>>>
> > > >>>> f0f3f5aae7c4 (origin/master, origin/HEAD, master) UnitTestFrameworkPkg: Update YAML to ignore specific ECC
> > > >>> files/errors
> > > >>>>
> > > >>>> c05734797790 UefiPayloadPkg: Update YAML to ignore specific ECC files/errors
> > > >>>>
> > > >>>> c30c40d6c63d StandaloneMmPkg: Update YAML to ignore specific ECC files/errors
> > > >>>>
> > > >>>> 9944508e85f1 ShellPkg: Update YAML to ignore specific ECC files/errors
> > > >>>>
> > > >>>> 60fa40be458d SecurityPkg: Update YAML to ignore specific ECC files/errors
> > > >>>>
> > > >>>> df790cd6b37e MdePkg: Update YAML to ignore specific ECC files/errors
> > > >>>>
> > > >>>> 9deb9370766e MdeModulePkg: Update YAML to ignore specific ECC files/errors
> > > >>>>
> > > >>>> d7d30e8f219f EmulatorPkg: Update YAML to ignore specific ECC files/errors
> > > >>>>
> > > >>>> d5744ecba813 CryptoPkg: Update YAML to ignore specific ECC files/errors
> > > >>>>
> > > >>>> c97fee87f0f9 ArmVirtPkg: Update YAML to ignore specific ECC files/errors
> > > >>>>
> > > >>>> 1939fc9569f2 ArmPlatformPkg: Update YAML to ignore specific ECC files/errors
> > > >>>>
> > > >>>> 365dced2c37a ArmPkg: Update YAML to ignore specific ECC files/errors
> > > >>>>
> > > >>>> 76a1ce4d5fec .azurepipelines/templates: Update max pipeline job time to 2 hours
> > > >>>>
> > > >>>> 99f84ff47390 .pytools/Plugin/LicenseCheck: Use temp directory for git diff output
> > > >>>>
> > > >>>> 3019f1bbabf1 .pytool/Plugin/EccCheck: Add performance optimizations
> > > >>>>
> > > >>>> 854462bd3479 .pytool/Plugin/EccCheck: Remove temp directory on exception
> > > >>>>
> > > >>>> 69877614fdee .pytool/Plugin/EccCheck: Remove RevertCode()
> > > >>>>
> > > >>>> --ref1
> > > >>>>
> > > >>>> ef9a059cdb15 EmulatorPkg/Win/Host: Update CC_FLAGS
> > > >>>>
> > > >>>> bb1bba3d7767 (tag: edk2-stable202111) NetworkPkg: Fix invalid pointer for DNS response token on error
> > > >>>>
> > > >>>> Best regards,
> > > >>>>
> > > >>>> Mike
> > > >>>>
> > > >>>>
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
2021-12-06 1:17 ` Michael D Kinney
@ 2021-12-06 1:23 ` Wu, Hao A
2021-12-06 2:29 ` Ni, Ray
2021-12-06 9:31 ` 回复: " gaoliming
` (6 subsequent siblings)
7 siblings, 1 reply; 39+ messages in thread
From: Wu, Hao A @ 2021-12-06 1:23 UTC (permalink / raw)
To: devel@edk2.groups.io, Kinney, Michael D, Michael Kubacki,
maciej.rabeda@linux.intel.com, Michael Kubacki,
Andrew Fish (afish@apple.com), Leif Lindholm
For SourceLevelDebugPkg:
Reviewed-by: Hao A Wu <hao.a.wu@intel.com>
Best Regards,
Hao Wu
> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Michael
> D Kinney
> Sent: Monday, December 6, 2021 9:18 AM
> To: Michael Kubacki <mikuback@linux.microsoft.com>;
> devel@edk2.groups.io; maciej.rabeda@linux.intel.com; Michael Kubacki
> <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com)
> <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>; Kinney, Michael D
> <michael.d.kinney@intel.com>
> Subject: Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended
> Hard Freeze Update #4
>
> Hello EDK II Maintainers,
>
> A detailed evaluation of the DEBUG_CODE() formatting issue has been
> completed.
> The reason DEBUG_CODE() is a challenge is that this looks like a macro from a
> C parsing perspective, but the EDK II usage places C statements or blocks of
> C code as the parameter to this macro.
>
> There are actually 2 methods available to mark a statement or a block of code
> to be included when the DEBUG_PROPERTY_DEBUG_CODE_ENABLED bit is
> enabled in
> PcdDebugPropertyMask. One is DEBUG_CODE() and the other is to mark the
> beginning
> and end of a code block with DEBUG_CODE_BEGIN() and
> DEBUG_CODE_END(). In fact,
> DEBUG_CODE() is implemented using DEBUG_CODE_BEGIN() and
> DEBUG_CODE_END() macros.
>
> #define DEBUG_CODE(Expression) \
> DEBUG_CODE_BEGIN (); \
> Expression \
> DEBUG_CODE_END ()
>
> A complete review for the use of these DEBUG_CODE macros was
> performed on the
> edk2 repo. Uncrustify performs good formatting for code blocks between
> DEBUG_CODE_BEGIN() and DEBUG_CODE_END(). This is because these look
> like simple
> macros calls with no parameters and the lines of C code between these 2
> macros
> is formatted correctly.
>
> The uncrustify formatting issues are only present with the use of
> DEBUG_CODE().
> Simple use cases of DEBUG_CODE(Expression) where Expression is a single C
> statement also look correct. A medium complexity use case where
> Expression is
> a code block of simple statements or even some local variables and simple
> statements also look correct. It is only complex code blocks that use C
> statements such as if/for/while/case that include the use of braces {} does
> uncrustify perform incorrect formatting.
>
> The recommended solution to this issue is to convert the use of
> DEBUG_CODE()
> to DEBUG_CODE_BEGIN() / DEBUG_CODE_END() for cases where the
> Expression
> passed to DEBUG_CODE() is the complex use case that contains statements
> that
> use braces {}. There are 57 instances of this pattern across 40 files in the
> edk2 repo.
>
> I have posted a branch with these additional patches:
>
>
> https://github.com/mdkinney/edk2/tree/Bug_3737_3760_3739_ApplyUncru
> stifyChanges_V7
>
> I have performed CompareBuild tests with this revised patch series with
> the DEBUG_CODE changes. It passes 100% showing no binary differences.
>
> https://github.com/mdkinney/edk2/actions/runs/1542454606
>
> I have opened a PR to run this patch series through EDK II CI. It also passes
> 100%.
>
> https://github.com/tianocore/edk2/pull/2236
>
> The summary of changes made since the V6 review are:
>
> 1) Change uncrustify configuration assignment alignment threshold to 0
>
> align_assign_thresh = 0
>
> 2) Replace "<param>, OPTIONAL" with "<param> OPTIONAL,"
>
> 3) Replace DEBUG_CODE(Expression) with
>
> DEBUG_CODE_BEGIN();
> Expression
> DEBUG_CODE_END()
>
> if Expression is complex and contains braces {}.
>
> 4) No changes to uncrustify tool required.
>
> Please review the differences between the following 2 branches and provide
> feedback or a Series Reviewed-by if you agree with these additional changes.
>
>
> https://github.com/mdkinney/edk2/tree/Bug_3737_3760_3739_ApplyUncru
> stifyChanges_V6
>
> https://github.com/mdkinney/edk2/tree/Bug_3737_3760_3739_ApplyUncru
> stifyChanges_V7
>
> The goal is to complete the review and get the uncrustify change committed
> tomorrow so the extended hard freeze can be lifted.
>
> Thanks,
>
> Mike
>
> > -----Original Message-----
> > From: Kinney, Michael D <michael.d.kinney@intel.com>
> > Sent: Thursday, December 2, 2021 6:23 PM
> > To: Michael Kubacki <mikuback@linux.microsoft.com>;
> devel@edk2.groups.io; maciej.rabeda@linux.intel.com; Michael Kubacki
> > <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com)
> <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>;
> > Kinney, Michael D <michael.d.kinney@intel.com>
> > Subject: RE: [edk2-devel] Uncrustify Conversion Detailed Plan and
> Extended Hard Freeze Update #4
> >
> > Hello EDK II Maintainers,
> >
> > I have entered BZ 3760 to make the use of the OPTIONAL keyword style
> consistent for all of edk2 repo
> > and to be compatible with uncrustify.
> >
> > I have posted the following V6 branch that does the EFI_D_* to DEBUG_*
> changes, the OPTIONAL keyword
> > style changes, and the uncrustify changes with the one configuration
> change for assignment alignment.
> >
> >
> https://github.com/mdkinney/edk2/tree/Bug_3737_3760_3739_ApplyUncru
> stifyChanges_V6
> >
> > Please provide feedback on the code style in this branch with the known
> DEBUG_CODE() issue still
> > present.
> >
> > If we are able to quickly update uncrustify to handle DEBUG_CODE(), I will
> generate a V7.
> >
> > Thanks,
> >
> > Mike
> >
> > > -----Original Message-----
> > > From: Kinney, Michael D <michael.d.kinney@intel.com>
> > > Sent: Thursday, December 2, 2021 4:53 PM
> > > To: Michael Kubacki <mikuback@linux.microsoft.com>;
> devel@edk2.groups.io; maciej.rabeda@linux.intel.com; Michael Kubacki
> > > <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com)
> <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>;
> > > Kinney, Michael D <michael.d.kinney@intel.com>
> > > Subject: RE: [edk2-devel] Uncrustify Conversion Detailed Plan and
> Extended Hard Freeze Update #4
> > >
> > > Michael,
> > >
> > > Yes. Please update the patch series that adds UncrustifyCheck with
> those changes.
> > >
> > > Thanks,
> > >
> > > Mike
> > >
> > > > -----Original Message-----
> > > > From: Michael Kubacki <mikuback@linux.microsoft.com>
> > > > Sent: Thursday, December 2, 2021 4:31 PM
> > > > To: Kinney, Michael D <michael.d.kinney@intel.com>;
> devel@edk2.groups.io; maciej.rabeda@linux.intel.com; Michael
> > Kubacki
> > > > <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com)
> <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>
> > > > Subject: Re: [edk2-devel] Uncrustify Conversion Detailed Plan and
> Extended Hard Freeze Update #4
> > > >
> > > > Hi Mike,
> > > >
> > > > Thank you for the detailed analysis and recommendations.
> > > >
> > > > I agree with the recommendations and I will try to have an Uncrustify
> > > > tool update by tomorrow for (4).
> > > >
> > > > That will require an update in uncrustify_ext_dep.yaml to pull in the
> > > > new version. I'm assuming I should also update uncrustify.cfg to set
> > > > align_assign_thresh = 0 in that new patch series?
> > > >
> > > > Regards,
> > > > Michael
> > > >
> > > > On 12/2/2021 7:18 PM, Kinney, Michael D wrote:
> > > > > Hi Michael,
> > > > >
> > > > > CORRECTION: set align_assign_threshold to 0.
> > > > >
> > > > > Reponses inline below.
> > > > >
> > > > > I would like to summarize the 4 issues raised in the past day along
> with the recommendations.
> > > > >
> > > > > 1) Exclusion feature for UncrustifyCheck. There are 2 directories with
> 8 files total that
> > > > > maintainers have noted they would like to see not go through
> uncrustify formatting. Today
> > > > > the only content that is skipped is BaseTools and submodules.
> > > > >
> > > > > Adding a general purpose exclusion feature would then require all
> developers to make
> > > > > sure their method of using uncrustify also excludes those same
> areas. This requires
> > > > > extra steps for all developers and maintainers.
> > > > >
> > > > > If we do not add the exclusion feature, then the 8 files will require
> an extra step
> > > > > to sync with the original source of those files. The rate of changes
> of these 8 files
> > > > > is very low today.
> > > > >
> > > > > RECOMMENDATION: Do not add exclusion feature at this time.
> Revisit if the extra work
> > > > > to maintain the files that would be candidates for exclusions
> increases significantly.
> > > > >
> > > > > 2) Alignment of assignments. The threshold of 4 characters appears
> to be too low and causes
> > > > > source files that are already aligned to become unaligned.
> > > > >
> > > > > RECOMMENDATION: Change threshold to the default value of 0
> which means no limit.
> > > > >
> > > > > align_assign_thresh= 0
> > > > >
> > > > > 3) Alignment of parameters in function declaration not correct. The
> root cause of this
> > > > > is the use of the OPTIONAL keyword. If the OPTIONAL keyword is
> removed, then the
> > > > > alignment is correct. The alignment is also correct if the OPTIONAL
> keyword appears
> > > > > before the ','. If the OPTIONAL keyword appears after the ',', then
> the format is
> > > > > not correct. The OPTIONAL keyword indicates that the parameter
> in the function is
> > > > > not required and may be passed in as NULL or 0 or some other
> default value defined by
> > > > > the API. It makes more sense for this OPTIONAL keyword that
> follows the parameter
> > > > > names to appear before the ',' so it is scoped to the parameter on
> that line. If it
> > > > > appears after the ',', then C parsers thinks it is a prefix (IN, OUT,
> CONST, volatile,
> > > > > static) for the next parameter in the function.
> > > > >
> > > > > RECOMMENDATION: Update patch series with a global search and
> replace so OPTIONAL
> > > > > keyword always appears before the ',' on the same line.
> > > > >
> > > > > RegEx search string: ',( *)OPTIONAL( *)'
> > > > > RegEx replace string: ' $1OPTIONAL,$2'
> > > > >
> > > > > 4) Format issues with complex blocks in DEBUG_CODE(), or between
> DEBUG_CODE_BEGIN() and
> > > > > DEBUG_CODE_END(). Uncrustify treats these as macros and is not
> aware that the
> > > > > parameter passed into the macro call is a block of C code that needs
> to be formatted.
> > > > > Complex blocks with if/while/for/case statements are impacted the
> most.
> > > > >
> > > > > RECOMMENDATION: Update the uncrustify with an edk2 specific
> extension to treat these
> > > > > macros as a block of code as if they were surrounded by an extra
> set of braces {}.
> > > > >
> > > > >
> > > > > I have posted a branch for testing purposes that implements (2) and
> (3).
> > > > >
> > > > > Branch:
> https://github.com/mdkinney/edk2/tree/TestOnly_Bug_3737_3739_ApplyU
> ncrustifyChanges_V6_OPTIONAL_Keyword_Fix
> > > > > PR: https://github.com/tianocore/edk2/pull/2233
> > > > > Status: PASS
> > > > > CompareBuild:
> https://github.com/mdkinney/edk2/actions/runs/1532855914
> > > > > Status: PASS
> > > > >
> > > > > You can see what changed by fetching and comparing the following 2
> branches:
> > > > >
> > > > >
> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyC
> hanges_V5
> > > > >
> https://github.com/mdkinney/edk2/tree/TestOnly_Bug_3737_3739_ApplyU
> ncrustifyChanges_V6_OPTIONAL_Keyword_Fix
> > > > >
> > > > > Please provide feedback on the RECOMMENDATIONS above. I will go
> ahead and prepare of V6 version of
> > > > > the patch series now that that test results are all PASS.
> > > > >
> > > > > Best regards,
> > > > >
> > > > > Mike
> > > > >
> > > > >
> > > > >> -----Original Message-----
> > > > >> From: Kinney, Michael D <michael.d.kinney@intel.com>
> > > > >> Sent: Thursday, December 2, 2021 4:15 PM
> > > > >> To: Michael Kubacki <mikuback@linux.microsoft.com>;
> devel@edk2.groups.io; maciej.rabeda@linux.intel.com; Michael
> > > > Kubacki
> > > > >> <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com)
> <afish@apple.com>; Leif Lindholm
> > <leif@nuviainc.com>;
> > > > >> Kinney, Michael D <michael.d.kinney@intel.com>
> > > > >> Subject: RE: [edk2-devel] Uncrustify Conversion Detailed Plan and
> Extended Hard Freeze Update #4
> > > > >>
> > > > >> Hi Michael,
> > > > >>
> > > > >> Reponses inline below.
> > > > >>
> > > > >> I would like to summarize the 4 issues raised in the past day along
> with the recommendations.
> > > > >>
> > > > >> 1) Exclusion feature for UncrustifyCheck. There are 2 directories
> with 8 files total that
> > > > >> maintainers have noted they would like to see not go through
> uncrustify formatting. Today
> > > > >> the only content that is skipped is BaseTools and submodules.
> > > > >>
> > > > >> Adding a general purpose exclusion feature would then require all
> developers to make
> > > > >> sure their method of using uncrustify also excludes those same
> areas. This requires
> > > > >> extra steps for all developers and maintainers.
> > > > >>
> > > > >> If we do not add the exclusion feature, then the 8 files will require
> an extra step
> > > > >> to sync with the original source of those files. The rate of changes
> of these 8 files
> > > > >> is very low today.
> > > > >>
> > > > >> RECOMMENDATION: Do not add exclusion feature at this time.
> Revisit if the extra work
> > > > >> to maintain the files that would be candidates for exclusions
> increases significantly.
> > > > >>
> > > > >> 2) Alignment of assignments. The threshold of 4 characters appears
> to be too low and causes
> > > > >> source files that are already aligned to become unaligned.
> > > > >>
> > > > >> RECOMMENDATION: Change threshold to the default value of 0
> which means no limit.
> > > > >>
> > > > >> align_assign_thresh= 4
> > > > >>
> > > > >> 3) Alignment of parameters in function declaration not correct. The
> root cause of this
> > > > >> is the use of the OPTIONAL keyword. If the OPTIONAL keyword is
> removed, then the
> > > > >> alignment is correct. The alignment is also correct if the OPTIONAL
> keyword appears
> > > > >> before the ','. If the OPTIONAL keyword appears after the ',', then
> the format is
> > > > >> not correct. The OPTIONAL keyword indicates that the parameter
> in the function is
> > > > >> not required and may be passed in as NULL or 0 or some other
> default value defined by
> > > > >> the API. It makes more sense for this OPTIONAL keyword that
> follows the parameter
> > > > >> names to appear before the ',' so it is scoped to the parameter on
> that line. If it
> > > > >> appears after the ',', then C parsers thinks it is a prefix (IN, OUT,
> CONST, volatile,
> > > > >> static) for the next parameter in the function.
> > > > >>
> > > > >> RECOMMENDATION: Update patch series with a global search and
> replace so OPTIONAL
> > > > >> keyword always appears before the ',' on the same line.
> > > > >>
> > > > >> RegEx search string: ',( *)OPTIONAL( *)'
> > > > >> RegEx replace string: ' $1OPTIONAL,$2'
> > > > >>
> > > > >> 4) Format issues with complex blocks in DEBUG_CODE(), or between
> DEBUG_CODE_BEGIN() and
> > > > >> DEBUG_CODE_END(). Uncrustify treats these as macros and is not
> aware that the
> > > > >> parameter passed into the macro call is a block of C code that
> needs to be formatted.
> > > > >> Complex blocks with if/while/for/case statements are impacted
> the most.
> > > > >>
> > > > >> RECOMMENDATION: Update the uncrustify with an edk2 specific
> extension to treat these
> > > > >> macros as a block of code as if they were surrounded by an extra
> set of braces {}.
> > > > >>
> > > > >>
> > > > >> I have posted a branch for testing purposes that implements (2) and
> (3).
> > > > >>
> > > > >> Branch:
> https://github.com/mdkinney/edk2/tree/TestOnly_Bug_3737_3739_ApplyU
> ncrustifyChanges_V6_OPTIONAL_Keyword_Fix
> > > > >> PR: https://github.com/tianocore/edk2/pull/2233
> > > > >> Status: PASS
> > > > >> CompareBuild:
> https://github.com/mdkinney/edk2/actions/runs/1532855914
> > > > >> Status: PASS
> > > > >>
> > > > >> You can see what changed by fetching and comparing the following 2
> branches:
> > > > >>
> > > > >>
> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyC
> hanges_V5
> > > > >>
> https://github.com/mdkinney/edk2/tree/TestOnly_Bug_3737_3739_ApplyU
> ncrustifyChanges_V6_OPTIONAL_Keyword_Fix
> > > > >>
> > > > >> Please provide feedback on the RECOMMENDATIONS above. I will
> go ahead and prepare of V6 version of
> > > > >> the patch series now that that test results are all PASS.
> > > > >>
> > > > >> Best regards,
> > > > >>
> > > > >> Mike
> > > > >>
> > > > >>
> > > > >>> -----Original Message-----
> > > > >>> From: Michael Kubacki <mikuback@linux.microsoft.com>
> > > > >>> Sent: Thursday, December 2, 2021 1:57 PM
> > > > >>> To: devel@edk2.groups.io; Kinney, Michael D
> <michael.d.kinney@intel.com>; maciej.rabeda@linux.intel.com; Michael
> > > > Kubacki
> > > > >>> <michael.kubacki@microsoft.com>; Andrew Fish
> (afish@apple.com) <afish@apple.com>; Leif Lindholm
> > <leif@nuviainc.com>
> > > > >>> Subject: Re: [edk2-devel] Uncrustify Conversion Detailed Plan and
> Extended Hard Freeze Update #4
> > > > >>>
> > > > >>> My reply is inline.
> > > > >>>
> > > > >>> Regards,
> > > > >>> Michael
> > > > >>>
> > > > >>> On 12/2/2021 2:45 PM, Michael D Kinney wrote:
> > > > >>>> Hi Maciej,
> > > > >>>>
> > > > >>>> Thanks for the feedback.
> > > > >>>>
> > > > >>>> * Example #1.This appears to be caused by the following
> uncrustify
> > > > >>>> settings:
> > > > >>>>
> > > > >>>> # The threshold for aligning on '=' in assignments.
> > > > >>>>
> > > > >>>> # Use a negative number for absolute thresholds.
> > > > >>>>
> > > > >>>> #
> > > > >>>>
> > > > >>>> # 0: No limit (default).
> > > > >>>>
> > > > >>>> align_assign_thresh= 0# number
> > > > >>>>
> > > > >>>> The edk2 setting for this is:
> > > > >>>>
> > > > >>>> align_assign_thresh= 4
> > > > >>>>
> > > > >>>> This means blocks of assignments that are different than more
> than 4
> > > > >>>> spaces will be considered a new block.
> > > > >>>>
> > > > >>>> ‘HwAddreLen’ and ‘Xid’ are more than 4 characters in length
> > > > >>>> different.Same for ‘Xid’ and ‘Reserved’.So
> > > > >>>>
> > > > >>>> uncrustify treats these as 3 different assignment blocks.
> > > > >>>>
> > > > >>>> If we change to the default value of 0: No limit, this example is
> > > > >>>> treated as a single block and all ‘=’ are aligned.
> > > > >>>>
> > > > >>>> Is there a reason ‘4’ was selected?Is there is any harm in using the
> > > > >>>> default of 0?
> > > > >>>>
> > > > >>> We can certainly change the threshold. '4' was derived by
> > > > >>> experimentation. This is an area that is somewhat subjective and
> every
> > > > >>> case is difficult to cover well. Please feel free to suggest any
> changes.
> > > > >>>
> > > > >>
> > > > >> I recommend we use the default value of 0. That way, any code that
> choose to
> > > > >> align assignments will still be aligned. If a developer does not like
> > > > >> short assignments and long assignments in the same code block to
> use the
> > > > >> long assignment column, they can always break the block up into
> multiple
> > > > >> blocks by adding a carriage return.
> > > > >>
> > > > >>> For the benefit of others, this file can be a useful refernce
> > > > >>> understanding spans, gaps, and thresholds.
> > > > >>>
> > > > >>>
> https://github.com/uncrustify/uncrustify/blob/master/documentation/htdo
> cs/configuration.txt
> > > > >>>
> > > > >>>> * Example #2: Uncruistfy is confused by the DEBUG_CODE()
> macro.This is
> > > > >>>> not a traditional macro because the
> > > > >>>>
> > > > >>>> contents of the macro is a block of C code.I do not know how to
> convince
> > > > >>>> uncrustify that the contents of a
> > > > >>>>
> > > > >>>> macro like function call to be treated as a code block from an
> indent
> > > > >>>> perspective.
> > > > >>>>
> > > > >>> I believe this would have to be overridden in the Uncrustify fork
> since
> > > > >>> DEBUG_CODE() is being treated as a macro function call and code
> blocks
> > > > >>> are not expected there. In addition, some special treatment might
> be
> > > > >>> needed for alignment in between
> DEBUG_CODE_BEGIN()/DEBUG_CODE_END().
> > > > >>>
> > > > >>> I'm happy to look at this. However, regression validation can take a
> > > > >>> while so I'd like to make sure we need this now. In cases that do
> not
> > > > >>> have additional code blocks, it seems to format fairly well. Is this
> > > > >>> prevalent and impactful enough it must be fixed now? Or, could we
> > > > >>> revisit it with a follow up patch?
> > > > >>
> > > > >> How long does regression testing take? I see about 20 instances of
> > > > >> DEBUG_CODE() that are producing bad formatting. If the content
> inside
> > > > >> DEBUG_CODE() is a single line of a single block of statements
> without
> > > > >> any if/while/for/case statements that require further indent, then
> the
> > > > >> format looks ok.
> > > > >>
> > > > >> DEBUG_CODE_BEGIN() and DEBUG_CODE_END() would look better
> if the
> > > > >> contents between are also indented one level.
> > > > >>
> > > > >>>
> > > > >>>> * Example #3/#4: Uncrustify is confused by the OPTIONAL
> keyword.The
> > > > >>>> edk2 config declares it as a QUALIFIER
> > > > >>>>
> > > > >>>> like IN and OUT.However, OPTIONAL only appears at the end of
> the line
> > > > >>>> that declares a parameter to a
> > > > >>>>
> > > > >>>> function.There are 3 forms. One with comma after
> OPTIONAL.One with comma
> > > > >>>> before OPTIONAL, and
> > > > >>>>
> > > > >>>> one with no comma if the parameter is the last parameter in the
> function
> > > > >>>> declaration.
> > > > >>>>
> > > > >>>> TYPE ParamName OPTIONAL,
> > > > >>>>
> > > > >>>> TYPEParamName, OPTIONAL
> > > > >>>>
> > > > >>>> TYPEParamName OPTIONAL
> > > > >>>>
> > > > >>>> OPTIONAL is defined to nothing in edk2 builds.From a uncrustify
> > > > >>>> perspective, we really want is to be
> > > > >>>>
> > > > >>>> ignored or more correctly treated it as a token that is attached to
> > > > >>>> ParamName and the combination of
> > > > >>>>
> > > > >>>> ParamName and OPTIONAL treated as one unit to determine
> indents.
> > > > >>>>
> > > > >>> "TYPE ParamName, OPTIONAL" seems especially problematic and
> presents an
> > > > >>> inconsistency with the other formats. Mike, can you please let me
> know
> > > > >>> if you have the same observation and your thoughts on a
> consistent pattern?
> > > > >>>
> > > > >>>> Thanks,
> > > > >>>>
> > > > >>>> Mike
> > > > >>>>
> > > > >>>> *From:*devel@edk2.groups.io <devel@edk2.groups.io> *On
> Behalf Of *Maciej
> > > > >>>> Rabeda
> > > > >>>> *Sent:* Thursday, December 2, 2021 10:27 AM
> > > > >>>> *To:* devel@edk2.groups.io; Kinney, Michael D
> > > > >>>> <michael.d.kinney@intel.com>; Michael Kubacki
> > > > >>>> <michael.kubacki@microsoft.com>; Andrew Fish
> (afish@apple.com)
> > > > >>>> <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>
> > > > >>>> *Subject:* Re: [edk2-devel] Uncrustify Conversion Detailed Plan
> and
> > > > >>>> Extended Hard Freeze Update #4
> > > > >>>>
> > > > >>>> Hey Mike,
> > > > >>>>
> > > > >>>> While most of the changes related to fixing coding style violations,
> > > > >>>> there are a couple of changes to NetworkPkg in that PR that make
> the
> > > > >>>> code less readable. Examples below.
> > > > >>>>
> > > > >>>> Example 1:
> > > > >>>>
> > > > >>>>
> > > > >>>>
> > > > >>>> Example 2:
> > > > >>>>
> > > > >>>>
> > > > >>>> Example 3:
> > > > >>>>
> > > > >>>>
> > > > >>>> Example 4:
> > > > >>>>
> > > > >>>> On 30-Nov-21 23:34, Michael D Kinney wrote:
> > > > >>>>
> > > > >>>> Hello,
> > > > >>>>
> > > > >>>> Thank you for your patience during this extended hard freeze.
> > > > >>>>
> > > > >>>> Just one more step to go.There has been a delay in the review
> of
> > > > >>>>
> > > > >>>> the patch series with the uncrustify source changes.PR(6).This
> > > > >>>>
> > > > >>>> patch series was not sent out as patch review email because of
> its
> > > > >>>>
> > > > >>>> very large size.It only contains source style changes and the
> > > > >>>>
> > > > >>>> CompareBuild tool and GitHub action has shown there are no
> binary
> > > > >>>>
> > > > >>>> differences introduced with these source style changes.
> > > > >>>>
> > > > >>>> If you are a package maintainer, then please review the
> following
> > > > >>>>
> > > > >>>> branch/PR for your package contents and review the EDK II CI
> results
> > > > >>>>
> > > > >>>> and BuildCompare results.I do not expect a line by line review
> > > > >>>>
> > > > >>>> because we already had time to provide feedback on the
> source style
> > > > >>>>
> > > > >>>> performed by uncrustify.Instead, a Reviewed-by for your
> package
> > > > >>>>
> > > > >>>> indicates that you have reviewed the EDK II CI results and
> CompareBuild
> > > > >>>>
> > > > >>>> tool functionality and results and you accept the source style
> > > > >>>>
> > > > >>>> changes to your package.
> > > > >>>>
> > > > >>>>
> *https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustify
> Changes_V5
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustify
> Changes_V5>
> > > > >>>>
> > > > >>>> *https://github.com/tianocore/edk2/pull/2229
> <https://github.com/tianocore/edk2/pull/2229>
> > > > >>>>
> > > > >>>> *https://github.com/mdkinney/edk2/actions/runs/1521618836
> > > > >>> <https://github.com/mdkinney/edk2/actions/runs/1521618836>
> > > > >>>>
> > > > >>>> Additional details on this update below.
> > > > >>>>
> > > > >>>> Thank you,
> > > > >>>>
> > > > >>>> Mike
> > > > >>>>
> > > > >>>> Changes from Update #3
> > > > >>>>
> > > > >>>> ----------------------------------------------------------------------------
> > > > >>>>
> > > > >>>> * Pushed PR (5)
> > > > >>>>
> > > > >>>> * Added link to PR(6). EDK II CI Status is PASS. Build Compare
> PASS.
> > > > >>>>
> > > > >>>> * Waiting for review of PR (6)
> > > > >>>>
> > > > >>>> * Review of PR (7) completed and waiting for review of PR (6)
> > > > >>>>
> > > > >>>> ----------------------------------------------------------------------------
> > > > >>>>
> > > > >>>> Changes from Update #2
> > > > >>>>
> > > > >>>> ----------------------------------------------------------------------------
> > > > >>>>
> > > > >>>> * Changed order of PRs swapping (4) and (5).The PR that
> activates
> > > > >>>>
> > > > >>>> increases the max CI agent job time is independent of all the
> other
> > > > >>>>
> > > > >>>> PRs and its review is complete, so it can be committed now.
> > > > >>>>
> > > > >>>> * Pushed PRs (1), (2), (3), (4).
> > > > >>>>
> > > > >>>> * Waiting for review to complete for PRs (5) and (6)
> > > > >>>>
> > > > >>>> * Reviews complete for PR (7)
> > > > >>>>
> > > > >>>> * Identifies steps using git filter-branch to apply uncrustify
> changes to a
> > > > >>>>
> > > > >>>> code review patch series that was generated before the
> uncrustify changes
> > > > >>>>
> > > > >>>> avoiding manual merge.
> > > > >>>>
> > > > >>>> * Identified steps using git filter-repo to generate an alternate
> history of
> > > > >>>>
> > > > >>>> the edk2 repo with uncrustify changes applied on every
> commit.This may
> > > > >>>>
> > > > >>>> be useful when evaluating changes to files using tools like git
> blame
> > > > >>>>
> > > > >>>> without the large uncrustify patch series.
> > > > >>>>
> > > > >>>> ---------------------------------------------------------------------------
> > > > >>>>
> > > > >>>> Changes from Update #1
> > > > >>>>
> > > > >>>> ----------------------------------------------------------------------------
> > > > >>>>
> > > > >>>> * Changed order of PRs swapping (6) and (7).The PR that
> activates
> > > > >>>>
> > > > >>>> EDK II CI check UncrustifyCheck has to be last because it
> unconditionally
> > > > >>>>
> > > > >>>> checks all C/H files in all packages.Not just files that have been
> > > > >>>>
> > > > >>>> modified like some of the other checkers.
> > > > >>>>
> > > > >>>> * Updated link to the branch with the UncrustifyCheck plugin
> that has been
> > > > >>>>
> > > > >>>> updated with a one line change and Reviewed-by and Tested-
> by tags.
> > > > >>>>
> > > > >>>>
> https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plug
> in_v6
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plu
> gin_v6>
> > > > >>>>
> > > > >>>> * Reviews complete for (1), (2), (3), (5), and (7)
> > > > >>>>
> > > > >>>> ---------------------------------------------------------------------------
> > > > >>>>
> > > > >>>> Michael Kubacki and I have prepared the patches required to
> apply the
> > > > >>>>
> > > > >>>> uncrustify changes and enable EDK II CI to check all submitted
> > > > >>>>
> > > > >>>> patches have been run through uncrustify.
> > > > >>>>
> > > > >>>> We have verified through the CompareBuild GitHub Action that
> the
> > > > >>>>
> > > > >>>> format changes performed by uncrustify have no functional
> changes.
> > > > >>>>
> > > > >>>> All of the OBJ, LIB, DLL, EFI, FFS, FV, and FD files match 100%
> > > > >>>>
> > > > >>>> across 70 VS2019/GCC5 builds of all package/platform DSC files
> in
> > > > >>>>
> > > > >>>> the edk2 repo.
> > > > >>>>
> > > > >>>> The hard freeze will be extended after the edk2-stable202111
> tag until
> > > > >>>>
> > > > >>>> all uncrustify related changes are committed.We do not expect
> this
> > > > >>>>
> > > > >>>> to take more than a few days.Do not push any PRs until the
> hard
> > > > >>>>
> > > > >>>> freeze is lifted.
> > > > >>>>
> > > > >>>> The changes are broken up into 7 patch series/PRs.The PRs are
> ordered
> > > > >>>>
> > > > >>>> so they can be submitted using the normal submission process
> and EDK II
> > > > >>>>
> > > > >>>> CI will pass for each one.Details are listed below.
> > > > >>>>
> > > > >>>> Uncrustify 73.0.3 for EDK II
> > > > >>>>
> > > > >>>> =============================
> > > > >>>>
> > > > >>>> * Sources:https://dev.azure.com/projectmu/_git/Uncrustify
> <https://dev.azure.com/projectmu/_git/Uncrustify>
> > > > >>>>
> > > > >>>> *
> Documentation:https://dev.azure.com/projectmu/Uncrustify/_wiki/wikis/U
> ncrustify.wiki/1/Project-Mu-(EDK-
> > II)-
> > > > Fork-
> > > > >>> Readme
> <https://dev.azure.com/projectmu/Uncrustify/_wiki/wikis/Uncrustify.wiki/1
> /Project-Mu-(EDK-II)-Fork-Readme>
> > > > >>>>
> > > > >>>> *
> Download:https://dev.azure.com/projectmu/Uncrustify/_packaging?_a=pac
> kage&feed=mu_uncrustify&package=mu-
> > > > >>> uncrustify-release&protocolType=NuGet&version=73.0.3
> > > > >>>
> <https://dev.azure.com/projectmu/Uncrustify/_packaging?_a=package&fe
> ed=mu_uncrustify&package=mu-uncrustify-
> > > > >>> release&protocolType=NuGet&version=73.0.3>
> > > > >>>>
> > > > >>>> Installing Uncrustify
> > > > >>>>
> > > > >>>> ======================
> > > > >>>>
> > > > >>>> The Uncrustify tool is installed automatically when the Pytools
> > > > >>>>
> > > > >>>> environment is used and the stuart* commands are run to
> complete the
> > > > >>>>
> > > > >>>> environment setup.Please see:
> > > > >>>>
> > > > >>>>
> https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally
> > > > >>>
> <https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally>
> > > > >>>>
> > > > >>>> Uncrustify can also be installed from the download page listed
> above
> > > > >>>>
> > > > >>>> or built from sources from the source link above.
> > > > >>>>
> > > > >>>> The Documentation link provides instruction on how to run
> uncrustify from
> > > > >>>>
> > > > >>>> the command line or install as a Visual Studio Code plugin.The
> main
> > > > >>>>
> > > > >>>> uncrustify documentation also describes how to integrate with
> a few other
> > > > >>>>
> > > > >>>> editors.
> > > > >>>>
> > > > >>>> We have also discussed a client side githook.That effort has not
> started.
> > > > >>>>
> > > > >>>> Let us know if that is a feature you would find useful.
> > > > >>>>
> > > > >>>> Developer impact for new code reviews
> > > > >>>>
> > > > >>>> ======================================
> > > > >>>>
> > > > >>>> Once the uncrustify checker is active in EDK II CI, developers
> must
> > > > >>>>
> > > > >>>> make sure their patches are run through the uncrustify tool
> before
> > > > >>>>
> > > > >>>> sending the patches for review.
> > > > >>>>
> > > > >>>> Developers must install and run uncrustify against changes files
> before
> > > > >>>>
> > > > >>>> sending patch review emails or submitting PR for EDK II CI.If
> EDK II CI
> > > > >>>>
> > > > >>>> detects and differences in source formatting, then EDK II CI will
> fail
> > > > >>>>
> > > > >>>> and the developer must run uncrustify and resubmit the
> patches.
> > > > >>>>
> > > > >>>> Developer impact to patch series/PRs reviewed during edk2-
> stable201121 soft/hard freeze
> > > > >>>>
> > > > >>>>
> ==========================================================
> =============================
> > > > >>>>
> > > > >>>> Developers must rebase their changes after the uncrustify
> source changes are
> > > > >>>>
> > > > >>>> committed.The branch with a preview of the uncrustify
> changes can be used
> > > > >>>>
> > > > >>>> to start this rebase work.
> > > > >>>>
> > > > >>>>
> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyC
> hanges_V5
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustify
> Changes_V5>
> > > > >>>>
> > > > >>>> The following steps can be used to update an existing branch
> with the
> > > > >>>>
> > > > >>>> required uncrustify format.This is the Windows version.I will
> add
> > > > >>>>
> > > > >>>> the Linux version soon.
> > > > >>>>
> > > > >>>> 1) Fetch and checkout and rebase to latest edk2/master
> > > > >>>>
> > > > >>>> git fetch origin
> > > > >>>>
> > > > >>>> git checkout master
> > > > >>>>
> > > > >>>> git rebase origin/master
> > > > >>>>
> > > > >>>> 2) Make a backup copy of plugin UncrustifyCheck outside
> WORKSPACE.
> > > > >>>>
> > > > >>>> (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool
> executable and
> > > > >>>>
> > > > >>>> EDK II specific uncrustify configuration file available when
> working
> > > > >>>>
> > > > >>>> with a branch that does not have those tools in its scope.
> > > > >>>>
> > > > >>>> xcopy .pytool\Plugin\UncrustifyCheck
> C:\Temp\UncrustifyCheck
> > > > >>>>
> > > > >>>> 3) Check out the patch series branch (e.g. MyBranch)
> > > > >>>>
> > > > >>>> git checkout MyBranch
> > > > >>>>
> > > > >>>> 4) Rebase patch series against edk2-stable202111
> > > > >>>>
> > > > >>>>
> > > > >>>>
> > > > >>>> git rebase edk2-stable202111
> > > > >>>>
> > > > >>>> 5) Create new branch for the uncrustifed version (e.g.
> MyBranch_Uncrustified)
> > > > >>>>
> > > > >>>> git checkout -b MyBranch_Uncrustified
> > > > >>>>
> > > > >>>> 6) Use git filter-branch to uncrustify all the commits in the
> series
> > > > >>>>
> > > > >>>> between the rebase target from (2) and HEAD of the branch.A
> filter
> > > > >>>>
> > > > >>>> can be used to scope the uncrustify operations to only the C/H
> files
> > > > >>>>
> > > > >>>> in the specific package the patch series is against. (e.g.
> DynamicTablesPkg).
> > > > >>>>
> > > > >>>> BaseTools should always be excluded.If the package scoped
> filter is
> > > > >>>>
> > > > >>>> not used, it will still work, but will take longer to run because
> > > > >>>>
> > > > >>>> uncrustify will rescan every C/H files in the whole repo.
> > > > >>>>
> > > > >>>> git filter-branch --tree-filter "git ls-files DynamicTablesPkg*.c
> DynamicTablesPkg*.h :!BaseTools/* |
> > > > >>> c:\\Temp\\UncrustifyCheck\\mu-uncrustify-
> release_extdep\\Windows-x86\\uncrustify.exe -c
> > > > >>> c:\\Temp\\UncrustifyCheck\\uncrustify.cfg -F - --replace --no-
> backup --if-changed" edk2-stable202111..HEAD
> > > > >>>>
> > > > >>>> 7) Now that all the individual patches in the branch are
> uncrustified,
> > > > >>>>
> > > > >>>> rebase against latest edk2/master that is already uncrustified.
> > > > >>>>
> > > > >>>> git rebase master
> > > > >>>>
> > > > >>>> 8) Verify the patches in this new branch.
> > > > >>>>
> > > > >>>> Impacts to tracing history across the uncrusity changes
> > > > >>>>
> > > > >>>>
> =======================================================
> > > > >>>>
> > > > >>>> Tools the view file and line history do work with the large
> uncrustify
> > > > >>>>
> > > > >>>> patch series.One impact is that the operations can be very slow
> due
> > > > >>>>
> > > > >>>> to the large uncrustify patches.
> > > > >>>>
> > > > >>>> One option to provide a faster experience is to provide an
> alternate
> > > > >>>>
> > > > >>>> version of the edk2 repository as "documentation" that has the
> > > > >>>>
> > > > >>>> entire history re-written with uncrustify run on every commit.
> > > > >>>>
> > > > >>>> The tool called git-filter-repo can be used to perform this
> > > > >>>>
> > > > >>>> transformation and runs in a reasonable period of time (a few
> hours)
> > > > >>>>
> > > > >>>> https://github.com/newren/git-filter-repo
> <https://github.com/newren/git-filter-repo>
> > > > >>>>
> > > > >>>> https://github.com/newren/git-filter-
> repo/blob/main/contrib/filter-repo-demos/lint-history
> > > > >>> <https://github.com/newren/git-filter-
> repo/blob/main/contrib/filter-repo-demos/lint-history>
> > > > >>>>
> > > > >>>> The following steps can be used to perform this transformation.
> > > > >>>>
> > > > >>>> This is the Windows version. I will add the Linux version soon.
> > > > >>>>
> > > > >>>> ** WARNING **This operation modifies(rewrites) all the
> commits
> > > > >>>>
> > > > >>>> in the local copy of the repo.Do not perform
> > > > >>>>
> > > > >>>> these steps on a local repo you are using for
> > > > >>>>
> > > > >>>> active development.
> > > > >>>>
> > > > >>>> 1) Clone edk2 into a new directory (see **WARNING**)
> > > > >>>>
> > > > >>>> git clonehttps://github.com/tianocore/edk2.git
> <https://github.com/tianocore/edk2.git> edk2-uncrustified
> > > > >>>>
> > > > >>>> cd edk2-uncrustified
> > > > >>>>
> > > > >>>> 2) Setup python virtual env, install pytools, and run stuart
> commands
> > > > >>>>
> > > > >>>> to setup build environment which includes installing uncrustify
> tools.
> > > > >>>>
> > > > >>>>
> https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally
> > > > >>>
> <https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally>
> > > > >>>>
> > > > >>>> 3) Make a backup copy of plugin UncrustifyCheck outside
> WORKSPACE.
> > > > >>>>
> > > > >>>> (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool
> executable and
> > > > >>>>
> > > > >>>> EDK II specific uncrustify configuration file available when
> working
> > > > >>>>
> > > > >>>> with a branch that does not have those tools in its scope.
> > > > >>>>
> > > > >>>> xcopy .pytool\Plugin\UncrustifyCheck
> C:\Temp\UncrustifyCheck
> > > > >>>>
> > > > >>>> 4) Use lint-history.py from git-filter-repo examples
> > > > >>>>
> > > > >>>> https://github.com/newren/git-filter-repo
> <https://github.com/newren/git-filter-repo>
> > > > >>>>
> > > > >>>> https://github.com/newren/git-filter-
> repo/blob/main/contrib/filter-repo-demos/lint-history
> > > > >>> <https://github.com/newren/git-filter-
> repo/blob/main/contrib/filter-repo-demos/lint-history>
> > > > >>>>
> > > > >>>> Line #127 - Add try except around subprocess.check_call() with
> except
> > > > >>>>
> > > > >>>> being pass.This is required because there are a few commits of
> C
> > > > >>>>
> > > > >>>> files in the edk2 repo that have incorrect C syntax and do not
> > > > >>>>
> > > > >>>> build with a C compiler and break the uncrustify parser.Skip
> reformat
> > > > >>>>
> > > > >>>> of C files that can not be parsed by uncrustify.These rare
> instances
> > > > >>>>
> > > > >>>> are addressed in the commit that fixes the C syntax error.
> > > > >>>>
> > > > >>>> Run this slightly modified version of lint-history.Include only
> > > > >>>>
> > > > >>>> C/H files and exclude directories that start with 'Tools' or
> 'BaseTools'.
> > > > >>>>
> > > > >>>> This step took about 2.2 hours on a laptop.
> > > > >>>>
> > > > >>>> lint-history.py
> > > > >>>>
> > > > >>>> --relevant "return (not filename.startswith(b'Tools') and not
> filename.startswith(b'BaseTools') and
> > > > >>> (filename.endswith(b'.c') or filename.endswith(b'.h')))"
> > > > >>>>
> > > > >>>> c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\mu-
> uncrustify-release_extdep\\Windows-x86\\uncrustify.exe
> > -
> > > c
> > > > >>>
> c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\uncrustify.cfg --
> replace --no-backup --if-changed
> > > > >>>>
> > > > >>>> Order of PRs to apply during extended hard freeze
> > > > >>>>
> > > > >>>>
> ==================================================
> > > > >>>>
> > > > >>>> 1) Update EmulatorPkg Win Host [BuildOptions] MSFT
> CC_FLAGS to not force debug information
> > > > >>>>
> > > > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3747
> <https://bugzilla.tianocore.org/show_bug.cgi?id=3747>
> > > > >>>>
> > > > >>>>
> *https://github.com/mdkinney/edk2/tree/Bug_3747_EmulatorPkg_WinHost
> _ReproducibleBuild
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_3747_EmulatorPkg_WinHost
> _ReproducibleBuild>
> > > > >>>>
> > > > >>>> *https://github.com/tianocore/edk2/pull/2215
> <https://github.com/tianocore/edk2/pull/2215>
> > > > >>>>
> > > > >>>> * Required for EmulatorPkg to pass CompareBuild for VS2019
> IA32/X64 builds.
> > > > >>>>
> > > > >>>> * Status: Review complete.PR pushed.
> > > > >>>>
> > > > >>>> 2) EccCheck should not revert staged and local changes
> > > > >>>>
> > > > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=2986
> <https://bugzilla.tianocore.org/show_bug.cgi?id=2986>
> > > > >>>>
> > > > >>>>
> *https://github.com/mdkinney/edk2/tree/Bug_2986_EccCheckRemoveGitR
> evert_V2
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_2986_EccCheckRemoveGitR
> evert_V2>
> > > > >>>>
> > > > >>>> *https://github.com/tianocore/edk2/pull/2216
> <https://github.com/tianocore/edk2/pull/2216>
> > > > >>>>
> > > > >>>> * Required for EDK II CI to complete in a reasonable period of
> time when
> > > > >>>>
> > > > >>>> processing the 4000+ source file style changes made by
> uncrustify.
> > > > >>>>
> > > > >>>> * Also fixes critical bugs that can potentially corrupt git state
> when
> > > > >>>>
> > > > >>>> EccCheck is run locally.
> > > > >>>>
> > > > >>>> * Status: Review complete.PR pushed.
> > > > >>>>
> > > > >>>> 3) Update pytool LicenseCheck plugin to use temp directory for
> diff output file
> > > > >>>>
> > > > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3746
> <https://bugzilla.tianocore.org/show_bug.cgi?id=3746>
> > > > >>>>
> > > > >>>>
> *https://github.com/mdkinney/edk2/tree/Bug_3746_LicenseCheckUseDiff
> OutputFile_V2
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_3746_LicenseCheckUseDiff
> OutputFile_V2>
> > > > >>>>
> > > > >>>> *https://github.com/tianocore/edk2/pull/2217
> <https://github.com/tianocore/edk2/pull/2217>
> > > > >>>>
> > > > >>>> * Required to reduce EDK II CI build times.
> > > > >>>>
> > > > >>>> * Status: Review complete.PR pushed.
> > > > >>>>
> > > > >>>> 4) Update max job time from 60 min to 120 minutes
> in .azurepipelines/templates
> > > > >>>>
> > > > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3750
> <https://bugzilla.tianocore.org/show_bug.cgi?id=3750>
> > > > >>>>
> > > > >>>>
> *https://github.com/mdkinney/edk2/tree/Bug_3750_IncreaseAzurePipeline
> sTimeout
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_3750_IncreaseAzurePipeline
> sTimeout>
> > > > >>>>
> > > > >>>> *https://github.com/tianocore/edk2/pull/2219
> <https://github.com/tianocore/edk2/pull/2219>
> > > > >>>>
> > > > >>>> * Required to allow EccCheck of uncrustify changes to complete
> on Azure
> > > > >>>>
> > > > >>>> Pipelines CI agents without timing out.
> > > > >>>>
> > > > >>>> * Status: Review complete.PR pushed.
> > > > >>>>
> > > > >>>> 5) Update Package YAML to ignore specific ECC files/errors
> > > > >>>>
> > > > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3749
> <https://bugzilla.tianocore.org/show_bug.cgi?id=3749>
> > > > >>>>
> > > > >>>>
> *https://github.com/mdkinney/edk2/tree/Bug_3749_EccCheckIgnoreFilesEr
> rors
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_3749_EccCheckIgnoreFilesEr
> rors>
> > > > >>>>
> > > > >>>> *https://github.com/tianocore/edk2/pull/2218
> <https://github.com/tianocore/edk2/pull/2218>
> > > > >>>>
> > > > >>>> * Required to pass EccCheck
> > > > >>>>
> > > > >>>> * Status: Review complete. PR pushed
> > > > >>>>
> > > > >>>> 6) Uncrustify Source Changes
> > > > >>>>
> > > > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3737
> <https://bugzilla.tianocore.org/show_bug.cgi?id=3737>
> > > > >>>>
> > > > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3739
> <https://bugzilla.tianocore.org/show_bug.cgi?id=3739>
> > > > >>>>
> > > > >>>>
> *https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustify
> Changes_V5
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustify
> Changes_V5>
> > > > >>>>
> > > > >>>> *https://github.com/tianocore/edk2/pull/2229
> <https://github.com/tianocore/edk2/pull/2229>
> > > > >>>>
> > > > >>>> * Build comparison result
> PASS:https://github.com/mdkinney/edk2/actions/runs/1521618836
> > > > >>> <https://github.com/mdkinney/edk2/actions/runs/1521618836>
> > > > >>>>
> > > > >>>> * EFI_D_ -> DEBUG changes required to pass PatchCheck
> > > > >>>>
> > > > >>>> * Uncrustify format changes required to pass UncrustifyCheck
> > > > >>>>
> > > > >>>> * Status:
> > > > >>>>
> > > > >>>> Waiting for review
> > > > >>>>
> > > > >>>> 7) UncrustifyCheck EDK II CI Plugin
> > > > >>>>
> > > > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3748
> <https://bugzilla.tianocore.org/show_bug.cgi?id=3748>
> > > > >>>>
> > > > >>>>
> *https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plu
> gin_v6
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plu
> gin_v6>
> > > > >>>>
> > > > >>>> * Required to enforce all PRs submitted to EDK II CI match
> uncrustify format.
> > > > >>>>
> > > > >>>> * Unconditionally checks all packages.Can not be committed
> until all C/H
> > > > >>>>
> > > > >>>> source files have been updated.
> > > > >>>>
> > > > >>>> * Status: Review complete
> > > > >>>>
> > > > >>>> Combined Branch/PR for Review/Test
> > > > >>>>
> > > > >>>> ==================================
> > > > >>>>
> > > > >>>> * Build Comparison results must pass 100% across the full set of
> PRs before
> > > > >>>>
> > > > >>>> the individual PRs can be pushed in the order listed above.
> > > > >>>>
> > > > >>>> *
> Branch:https://github.com/mdkinney/edk2/tree/TestOnly_Uncrustify_PR_S
> eries
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/TestOnly_Uncrustify_PR_Series>
> > > > >>>>
> > > > >>>> * PR:https://github.com/tianocore/edk2/pull/2229
> <https://github.com/tianocore/edk2/pull/2229>
> > > > >>>>
> > > > >>>> Status = PASS
> > > > >>>>
> > > > >>>> * CompareBuild:
> > > > >>>>
> > > > >>>>
> Branch:https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUnc
> rustifyChanges_V5
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustify
> Changes_V5>
> > > > >>>>
> > > > >>>> --ref1:ef9a059cdb15844fe52a49af2bf7d86b9dd3e9bf
> > > > >>>>
> > > > >>>> --ref2:Bug_3737_3739_ApplyUncrustifyChanges_V5
> > > > >>>>
> > > > >>>> Extra Options: -n 4 --quiet
> > > > >>>>
> > > > >>>>
> Results:https://github.com/mdkinney/edk2/actions/runs/1521618836
> > > > >>> <https://github.com/mdkinney/edk2/actions/runs/1521618836>
> > > > >>>>
> > > > >>>> 30 VS2019 build comparisons PASS
> > > > >>>>
> > > > >>>> 40 GCC5 build comparisons PASS
> > > > >>>>
> > > > >>>> 100% PASS
> > > > >>>>
> > > > >>>> The following git log shows the set of patches from --ref1 to --
> ref 2across
> > > > >>>>
> > > > >>>> which there are no differences in any of the
> OBJ/LIB/DLL/EFI/FFS/FV/FD files.
> > > > >>>>
> > > > >>>> --ref2
> > > > >>>>
> > > > >>>> b7d4bf0675b7 (HEAD ->
> Bug_3737_3739_ApplyUncrustifyChanges_V5) UnitTestFrameworkPkg:
> Apply uncrusitify
> > changes
> > > > >>>>
> > > > >>>> 7f03d25f60e7 UefiPayloadPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 0bfd8d9b5ac9 UefiCpuPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> e1cd9bfb9dea StandaloneMmPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 5da2f65be378 SourceLevelDebugPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 95b86de07e5d SignedCapsulePkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> fe71d97246c4 ShellPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 54c21c952992 SecurityPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 187a3785f12b RedfishPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 810100002a46 PcAtChipsetPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 276a695c0cf2 OvmfPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 303c0a91ab07 NetworkPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> bc80792cd1b1 MdePkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 3ea86be17a2a MdeModulePkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> c70ef11ed0cd IntelFsp2WrapperPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> c0291221f252 IntelFsp2Pkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 6a479952a690 FmpDevicePkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 3a7c05b7070d FatPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> b789f98c8959 EmulatorPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 952d7a1c9220 EmbeddedPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> a1cc9881bab6 DynamicTablesPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 50654dfe5785 CryptoPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> ed965a02dfa1 ArmVirtPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 9744023fbc46 ArmPlatformPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 7a1cde5f5bba ArmPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 19d17e0913e8 UefiCpuPkg: Change use of EFI_D_* to
> DEBUG_*
> > > > >>>>
> > > > >>>> ffa718b4f994 SourceLevelDebugPkg: Change use of EFI_D_* to
> DEBUG_*
> > > > >>>>
> > > > >>>> b86cb3c5e5b4 ShellPkg: Change use of EFI_D_* to DEBUG_*
> > > > >>>>
> > > > >>>> c7c42204dc07 SecurityPkg: Change use of EFI_D_* to DEBUG_*
> > > > >>>>
> > > > >>>> 16b8e6f958e4 PcAtChipsetPkg: Change use of EFI_D_* to
> DEBUG_*
> > > > >>>>
> > > > >>>> 0ac3f8b2dac5 OvmfPkg: Change use of EFI_D_* to DEBUG_*
> > > > >>>>
> > > > >>>> bc5004b8d294 NetworkPkg: Change use of EFI_D_* to
> DEBUG_*
> > > > >>>>
> > > > >>>> 6f671a8e2377 MdePkg: Change use of EFI_D_* to DEBUG_*
> > > > >>>>
> > > > >>>> a10c610ff9a3 MdeModulePkg: Change use of EFI_D_* to
> DEBUG_*
> > > > >>>>
> > > > >>>> 09a3bddba390 FatPkg: Change use of EFI_D_* to DEBUG_*
> > > > >>>>
> > > > >>>> 59c61318246a EmulatorPkg: Change use of EFI_D_* to
> DEBUG_*
> > > > >>>>
> > > > >>>> 3a80367dda3b EmbeddedPkg: Change use of EFI_D_* to
> DEBUG_*
> > > > >>>>
> > > > >>>> 23eb1aaf80ca ArmVirtPkg: Change use of EFI_D_* to DEBUG_*
> > > > >>>>
> > > > >>>> 875914b45c54 ArmPlatformPkg: Change use of EFI_D_* to
> DEBUG_*
> > > > >>>>
> > > > >>>> eb2eca82b451 ArmPkg: Change use of EFI_D_* to DEBUG_*
> > > > >>>>
> > > > >>>> f0f3f5aae7c4 (origin/master, origin/HEAD, master)
> UnitTestFrameworkPkg: Update YAML to ignore specific ECC
> > > > >>> files/errors
> > > > >>>>
> > > > >>>> c05734797790 UefiPayloadPkg: Update YAML to ignore specific
> ECC files/errors
> > > > >>>>
> > > > >>>> c30c40d6c63d StandaloneMmPkg: Update YAML to ignore
> specific ECC files/errors
> > > > >>>>
> > > > >>>> 9944508e85f1 ShellPkg: Update YAML to ignore specific ECC
> files/errors
> > > > >>>>
> > > > >>>> 60fa40be458d SecurityPkg: Update YAML to ignore specific ECC
> files/errors
> > > > >>>>
> > > > >>>> df790cd6b37e MdePkg: Update YAML to ignore specific ECC
> files/errors
> > > > >>>>
> > > > >>>> 9deb9370766e MdeModulePkg: Update YAML to ignore specific
> ECC files/errors
> > > > >>>>
> > > > >>>> d7d30e8f219f EmulatorPkg: Update YAML to ignore specific ECC
> files/errors
> > > > >>>>
> > > > >>>> d5744ecba813 CryptoPkg: Update YAML to ignore specific ECC
> files/errors
> > > > >>>>
> > > > >>>> c97fee87f0f9 ArmVirtPkg: Update YAML to ignore specific ECC
> files/errors
> > > > >>>>
> > > > >>>> 1939fc9569f2 ArmPlatformPkg: Update YAML to ignore specific
> ECC files/errors
> > > > >>>>
> > > > >>>> 365dced2c37a ArmPkg: Update YAML to ignore specific ECC
> files/errors
> > > > >>>>
> > > > >>>> 76a1ce4d5fec .azurepipelines/templates: Update max pipeline
> job time to 2 hours
> > > > >>>>
> > > > >>>> 99f84ff47390 .pytools/Plugin/LicenseCheck: Use temp directory
> for git diff output
> > > > >>>>
> > > > >>>> 3019f1bbabf1 .pytool/Plugin/EccCheck: Add performance
> optimizations
> > > > >>>>
> > > > >>>> 854462bd3479 .pytool/Plugin/EccCheck: Remove temp
> directory on exception
> > > > >>>>
> > > > >>>> 69877614fdee .pytool/Plugin/EccCheck: Remove RevertCode()
> > > > >>>>
> > > > >>>> --ref1
> > > > >>>>
> > > > >>>> ef9a059cdb15 EmulatorPkg/Win/Host: Update CC_FLAGS
> > > > >>>>
> > > > >>>> bb1bba3d7767 (tag: edk2-stable202111) NetworkPkg: Fix invalid
> pointer for DNS response token on error
> > > > >>>>
> > > > >>>> Best regards,
> > > > >>>>
> > > > >>>> Mike
> > > > >>>>
> > > > >>>>
>
>
>
>
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
2021-12-06 1:23 ` Wu, Hao A
@ 2021-12-06 2:29 ` Ni, Ray
0 siblings, 0 replies; 39+ messages in thread
From: Ni, Ray @ 2021-12-06 2:29 UTC (permalink / raw)
To: Wu, Hao A, devel
[-- Attachment #1: Type: text/plain, Size: 337 bytes --]
Mike,
Thank you! I know it's a bit hard to get every code to meet human expectations using auto-tools.
But it's the first time that edk2 code is more consistent than before.
Reviewed-by: Ray Ni <ray.ni@intel.com>
for:
UefiCpuPkg
PcAtChipsetPkg
SourceLevelDebugPkg
ShellPkg
EmulatorPkg
FatPkg
UefiPayloadPkg
[-- Attachment #2: Type: text/html, Size: 410 bytes --]
^ permalink raw reply [flat|nested] 39+ messages in thread
* 回复: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
2021-12-06 1:17 ` Michael D Kinney
2021-12-06 1:23 ` Wu, Hao A
@ 2021-12-06 9:31 ` gaoliming
2021-12-06 19:45 ` Bret Barkelew
` (5 subsequent siblings)
7 siblings, 0 replies; 39+ messages in thread
From: gaoliming @ 2021-12-06 9:31 UTC (permalink / raw)
To: devel, michael.d.kinney, 'Michael Kubacki', maciej.rabeda,
'Michael Kubacki', 'Andrew Fish',
'Leif Lindholm'
Mike:
I have no comments for these changes. For MdePkg, MdeModulePkg, FmpDevicePkg, Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Thanks
Liming
> -----邮件原件-----
> 发件人: devel@edk2.groups.io <devel@edk2.groups.io> 代表 Michael D
> Kinney
> 发送时间: 2021年12月6日 9:18
> 收件人: Michael Kubacki <mikuback@linux.microsoft.com>;
> devel@edk2.groups.io; maciej.rabeda@linux.intel.com; Michael Kubacki
> <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com)
> <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>; Kinney, Michael D
> <michael.d.kinney@intel.com>
> 主题: Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended
> Hard Freeze Update #4
>
> Hello EDK II Maintainers,
>
> A detailed evaluation of the DEBUG_CODE() formatting issue has been
> completed.
> The reason DEBUG_CODE() is a challenge is that this looks like a macro from a
> C parsing perspective, but the EDK II usage places C statements or blocks of
> C code as the parameter to this macro.
>
> There are actually 2 methods available to mark a statement or a block of code
> to be included when the DEBUG_PROPERTY_DEBUG_CODE_ENABLED bit is
> enabled in
> PcdDebugPropertyMask. One is DEBUG_CODE() and the other is to mark the
> beginning
> and end of a code block with DEBUG_CODE_BEGIN() and
> DEBUG_CODE_END(). In fact,
> DEBUG_CODE() is implemented using DEBUG_CODE_BEGIN() and
> DEBUG_CODE_END() macros.
>
> #define DEBUG_CODE(Expression) \
> DEBUG_CODE_BEGIN (); \
> Expression \
> DEBUG_CODE_END ()
>
> A complete review for the use of these DEBUG_CODE macros was performed
> on the
> edk2 repo. Uncrustify performs good formatting for code blocks between
> DEBUG_CODE_BEGIN() and DEBUG_CODE_END(). This is because these
> look like simple
> macros calls with no parameters and the lines of C code between these 2
> macros
> is formatted correctly.
>
> The uncrustify formatting issues are only present with the use of
> DEBUG_CODE().
> Simple use cases of DEBUG_CODE(Expression) where Expression is a single C
> statement also look correct. A medium complexity use case where
> Expression is
> a code block of simple statements or even some local variables and simple
> statements also look correct. It is only complex code blocks that use C
> statements such as if/for/while/case that include the use of braces {} does
> uncrustify perform incorrect formatting.
>
> The recommended solution to this issue is to convert the use of
> DEBUG_CODE()
> to DEBUG_CODE_BEGIN() / DEBUG_CODE_END() for cases where the
> Expression
> passed to DEBUG_CODE() is the complex use case that contains statements
> that
> use braces {}. There are 57 instances of this pattern across 40 files in the
> edk2 repo.
>
> I have posted a branch with these additional patches:
>
>
> https://github.com/mdkinney/edk2/tree/Bug_3737_3760_3739_ApplyUncrus
> tifyChanges_V7
>
> I have performed CompareBuild tests with this revised patch series with
> the DEBUG_CODE changes. It passes 100% showing no binary differences.
>
> https://github.com/mdkinney/edk2/actions/runs/1542454606
>
> I have opened a PR to run this patch series through EDK II CI. It also passes
> 100%.
>
> https://github.com/tianocore/edk2/pull/2236
>
> The summary of changes made since the V6 review are:
>
> 1) Change uncrustify configuration assignment alignment threshold to 0
>
> align_assign_thresh = 0
>
> 2) Replace "<param>, OPTIONAL" with "<param> OPTIONAL,"
>
> 3) Replace DEBUG_CODE(Expression) with
>
> DEBUG_CODE_BEGIN();
> Expression
> DEBUG_CODE_END()
>
> if Expression is complex and contains braces {}.
>
> 4) No changes to uncrustify tool required.
>
> Please review the differences between the following 2 branches and provide
> feedback or a Series Reviewed-by if you agree with these additional changes.
>
>
> https://github.com/mdkinney/edk2/tree/Bug_3737_3760_3739_ApplyUncrus
> tifyChanges_V6
>
> https://github.com/mdkinney/edk2/tree/Bug_3737_3760_3739_ApplyUncrus
> tifyChanges_V7
>
> The goal is to complete the review and get the uncrustify change committed
> tomorrow so the extended hard freeze can be lifted.
>
> Thanks,
>
> Mike
>
> > -----Original Message-----
> > From: Kinney, Michael D <michael.d.kinney@intel.com>
> > Sent: Thursday, December 2, 2021 6:23 PM
> > To: Michael Kubacki <mikuback@linux.microsoft.com>;
> devel@edk2.groups.io; maciej.rabeda@linux.intel.com; Michael Kubacki
> > <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com)
> <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>;
> > Kinney, Michael D <michael.d.kinney@intel.com>
> > Subject: RE: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended
> Hard Freeze Update #4
> >
> > Hello EDK II Maintainers,
> >
> > I have entered BZ 3760 to make the use of the OPTIONAL keyword style
> consistent for all of edk2 repo
> > and to be compatible with uncrustify.
> >
> > I have posted the following V6 branch that does the EFI_D_* to DEBUG_*
> changes, the OPTIONAL keyword
> > style changes, and the uncrustify changes with the one configuration change
> for assignment alignment.
> >
> >
> https://github.com/mdkinney/edk2/tree/Bug_3737_3760_3739_ApplyUncrus
> tifyChanges_V6
> >
> > Please provide feedback on the code style in this branch with the known
> DEBUG_CODE() issue still
> > present.
> >
> > If we are able to quickly update uncrustify to handle DEBUG_CODE(), I will
> generate a V7.
> >
> > Thanks,
> >
> > Mike
> >
> > > -----Original Message-----
> > > From: Kinney, Michael D <michael.d.kinney@intel.com>
> > > Sent: Thursday, December 2, 2021 4:53 PM
> > > To: Michael Kubacki <mikuback@linux.microsoft.com>;
> devel@edk2.groups.io; maciej.rabeda@linux.intel.com; Michael Kubacki
> > > <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com)
> <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>;
> > > Kinney, Michael D <michael.d.kinney@intel.com>
> > > Subject: RE: [edk2-devel] Uncrustify Conversion Detailed Plan and
> Extended Hard Freeze Update #4
> > >
> > > Michael,
> > >
> > > Yes. Please update the patch series that adds UncrustifyCheck with
> those changes.
> > >
> > > Thanks,
> > >
> > > Mike
> > >
> > > > -----Original Message-----
> > > > From: Michael Kubacki <mikuback@linux.microsoft.com>
> > > > Sent: Thursday, December 2, 2021 4:31 PM
> > > > To: Kinney, Michael D <michael.d.kinney@intel.com>;
> devel@edk2.groups.io; maciej.rabeda@linux.intel.com; Michael
> > Kubacki
> > > > <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com)
> <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>
> > > > Subject: Re: [edk2-devel] Uncrustify Conversion Detailed Plan and
> Extended Hard Freeze Update #4
> > > >
> > > > Hi Mike,
> > > >
> > > > Thank you for the detailed analysis and recommendations.
> > > >
> > > > I agree with the recommendations and I will try to have an Uncrustify
> > > > tool update by tomorrow for (4).
> > > >
> > > > That will require an update in uncrustify_ext_dep.yaml to pull in the
> > > > new version. I'm assuming I should also update uncrustify.cfg to set
> > > > align_assign_thresh = 0 in that new patch series?
> > > >
> > > > Regards,
> > > > Michael
> > > >
> > > > On 12/2/2021 7:18 PM, Kinney, Michael D wrote:
> > > > > Hi Michael,
> > > > >
> > > > > CORRECTION: set align_assign_threshold to 0.
> > > > >
> > > > > Reponses inline below.
> > > > >
> > > > > I would like to summarize the 4 issues raised in the past day along with
> the recommendations.
> > > > >
> > > > > 1) Exclusion feature for UncrustifyCheck. There are 2 directories
> with 8 files total that
> > > > > maintainers have noted they would like to see not go through
> uncrustify formatting. Today
> > > > > the only content that is skipped is BaseTools and submodules.
> > > > >
> > > > > Adding a general purpose exclusion feature would then require
> all developers to make
> > > > > sure their method of using uncrustify also excludes those same
> areas. This requires
> > > > > extra steps for all developers and maintainers.
> > > > >
> > > > > If we do not add the exclusion feature, then the 8 files will
> require an extra step
> > > > > to sync with the original source of those files. The rate of
> changes of these 8 files
> > > > > is very low today.
> > > > >
> > > > > RECOMMENDATION: Do not add exclusion feature at this time.
> Revisit if the extra work
> > > > > to maintain the files that would be candidates for exclusions
> increases significantly.
> > > > >
> > > > > 2) Alignment of assignments. The threshold of 4 characters appears
> to be too low and causes
> > > > > source files that are already aligned to become unaligned.
> > > > >
> > > > > RECOMMENDATION: Change threshold to the default value of 0
> which means no limit.
> > > > >
> > > > > align_assign_thresh= 0
> > > > >
> > > > > 3) Alignment of parameters in function declaration not correct. The
> root cause of this
> > > > > is the use of the OPTIONAL keyword. If the OPTIONAL
> keyword is removed, then the
> > > > > alignment is correct. The alignment is also correct if the
> OPTIONAL keyword appears
> > > > > before the ','. If the OPTIONAL keyword appears after the ',',
> then the format is
> > > > > not correct. The OPTIONAL keyword indicates that the
> parameter in the function is
> > > > > not required and may be passed in as NULL or 0 or some other
> default value defined by
> > > > > the API. It makes more sense for this OPTIONAL keyword that
> follows the parameter
> > > > > names to appear before the ',' so it is scoped to the parameter
> on that line. If it
> > > > > appears after the ',', then C parsers thinks it is a prefix (IN, OUT,
> CONST, volatile,
> > > > > static) for the next parameter in the function.
> > > > >
> > > > > RECOMMENDATION: Update patch series with a global search
> and replace so OPTIONAL
> > > > > keyword always appears before the ',' on the same line.
> > > > >
> > > > > RegEx search string: ',( *)OPTIONAL( *)'
> > > > > RegEx replace string: ' $1OPTIONAL,$2'
> > > > >
> > > > > 4) Format issues with complex blocks in DEBUG_CODE(), or between
> DEBUG_CODE_BEGIN() and
> > > > > DEBUG_CODE_END(). Uncrustify treats these as macros and is
> not aware that the
> > > > > parameter passed into the macro call is a block of C code that
> needs to be formatted.
> > > > > Complex blocks with if/while/for/case statements are impacted
> the most.
> > > > >
> > > > > RECOMMENDATION: Update the uncrustify with an edk2
> specific extension to treat these
> > > > > macros as a block of code as if they were surrounded by an extra
> set of braces {}.
> > > > >
> > > > >
> > > > > I have posted a branch for testing purposes that implements (2) and
> (3).
> > > > >
> > > > > Branch:
> https://github.com/mdkinney/edk2/tree/TestOnly_Bug_3737_3739_ApplyUn
> crustifyChanges_V6_OPTIONAL_Keyword_Fix
> > > > > PR: https://github.com/tianocore/edk2/pull/2233
> > > > > Status: PASS
> > > > > CompareBuild:
> https://github.com/mdkinney/edk2/actions/runs/1532855914
> > > > > Status: PASS
> > > > >
> > > > > You can see what changed by fetching and comparing the following 2
> branches:
> > > > >
> > > > >
> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyCh
> anges_V5
> > > > >
> https://github.com/mdkinney/edk2/tree/TestOnly_Bug_3737_3739_ApplyUn
> crustifyChanges_V6_OPTIONAL_Keyword_Fix
> > > > >
> > > > > Please provide feedback on the RECOMMENDATIONS above. I will
> go ahead and prepare of V6 version of
> > > > > the patch series now that that test results are all PASS.
> > > > >
> > > > > Best regards,
> > > > >
> > > > > Mike
> > > > >
> > > > >
> > > > >> -----Original Message-----
> > > > >> From: Kinney, Michael D <michael.d.kinney@intel.com>
> > > > >> Sent: Thursday, December 2, 2021 4:15 PM
> > > > >> To: Michael Kubacki <mikuback@linux.microsoft.com>;
> devel@edk2.groups.io; maciej.rabeda@linux.intel.com; Michael
> > > > Kubacki
> > > > >> <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com)
> <afish@apple.com>; Leif Lindholm
> > <leif@nuviainc.com>;
> > > > >> Kinney, Michael D <michael.d.kinney@intel.com>
> > > > >> Subject: RE: [edk2-devel] Uncrustify Conversion Detailed Plan and
> Extended Hard Freeze Update #4
> > > > >>
> > > > >> Hi Michael,
> > > > >>
> > > > >> Reponses inline below.
> > > > >>
> > > > >> I would like to summarize the 4 issues raised in the past day along
> with the recommendations.
> > > > >>
> > > > >> 1) Exclusion feature for UncrustifyCheck. There are 2 directories
> with 8 files total that
> > > > >> maintainers have noted they would like to see not go through
> uncrustify formatting. Today
> > > > >> the only content that is skipped is BaseTools and submodules.
> > > > >>
> > > > >> Adding a general purpose exclusion feature would then require
> all developers to make
> > > > >> sure their method of using uncrustify also excludes those same
> areas. This requires
> > > > >> extra steps for all developers and maintainers.
> > > > >>
> > > > >> If we do not add the exclusion feature, then the 8 files will
> require an extra step
> > > > >> to sync with the original source of those files. The rate of
> changes of these 8 files
> > > > >> is very low today.
> > > > >>
> > > > >> RECOMMENDATION: Do not add exclusion feature at this time.
> Revisit if the extra work
> > > > >> to maintain the files that would be candidates for exclusions
> increases significantly.
> > > > >>
> > > > >> 2) Alignment of assignments. The threshold of 4 characters appears
> to be too low and causes
> > > > >> source files that are already aligned to become unaligned.
> > > > >>
> > > > >> RECOMMENDATION: Change threshold to the default value of
> 0 which means no limit.
> > > > >>
> > > > >> align_assign_thresh= 4
> > > > >>
> > > > >> 3) Alignment of parameters in function declaration not correct. The
> root cause of this
> > > > >> is the use of the OPTIONAL keyword. If the OPTIONAL
> keyword is removed, then the
> > > > >> alignment is correct. The alignment is also correct if the
> OPTIONAL keyword appears
> > > > >> before the ','. If the OPTIONAL keyword appears after the ',',
> then the format is
> > > > >> not correct. The OPTIONAL keyword indicates that the
> parameter in the function is
> > > > >> not required and may be passed in as NULL or 0 or some other
> default value defined by
> > > > >> the API. It makes more sense for this OPTIONAL keyword that
> follows the parameter
> > > > >> names to appear before the ',' so it is scoped to the parameter
> on that line. If it
> > > > >> appears after the ',', then C parsers thinks it is a prefix (IN, OUT,
> CONST, volatile,
> > > > >> static) for the next parameter in the function.
> > > > >>
> > > > >> RECOMMENDATION: Update patch series with a global search
> and replace so OPTIONAL
> > > > >> keyword always appears before the ',' on the same line.
> > > > >>
> > > > >> RegEx search string: ',( *)OPTIONAL( *)'
> > > > >> RegEx replace string: ' $1OPTIONAL,$2'
> > > > >>
> > > > >> 4) Format issues with complex blocks in DEBUG_CODE(), or between
> DEBUG_CODE_BEGIN() and
> > > > >> DEBUG_CODE_END(). Uncrustify treats these as macros and
> is not aware that the
> > > > >> parameter passed into the macro call is a block of C code that
> needs to be formatted.
> > > > >> Complex blocks with if/while/for/case statements are impacted
> the most.
> > > > >>
> > > > >> RECOMMENDATION: Update the uncrustify with an edk2
> specific extension to treat these
> > > > >> macros as a block of code as if they were surrounded by an
> extra set of braces {}.
> > > > >>
> > > > >>
> > > > >> I have posted a branch for testing purposes that implements (2) and
> (3).
> > > > >>
> > > > >> Branch:
> https://github.com/mdkinney/edk2/tree/TestOnly_Bug_3737_3739_ApplyUn
> crustifyChanges_V6_OPTIONAL_Keyword_Fix
> > > > >> PR: https://github.com/tianocore/edk2/pull/2233
> > > > >> Status: PASS
> > > > >> CompareBuild:
> https://github.com/mdkinney/edk2/actions/runs/1532855914
> > > > >> Status: PASS
> > > > >>
> > > > >> You can see what changed by fetching and comparing the following 2
> branches:
> > > > >>
> > > > >>
> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyCh
> anges_V5
> > > > >>
> https://github.com/mdkinney/edk2/tree/TestOnly_Bug_3737_3739_ApplyUn
> crustifyChanges_V6_OPTIONAL_Keyword_Fix
> > > > >>
> > > > >> Please provide feedback on the RECOMMENDATIONS above. I will
> go ahead and prepare of V6 version of
> > > > >> the patch series now that that test results are all PASS.
> > > > >>
> > > > >> Best regards,
> > > > >>
> > > > >> Mike
> > > > >>
> > > > >>
> > > > >>> -----Original Message-----
> > > > >>> From: Michael Kubacki <mikuback@linux.microsoft.com>
> > > > >>> Sent: Thursday, December 2, 2021 1:57 PM
> > > > >>> To: devel@edk2.groups.io; Kinney, Michael D
> <michael.d.kinney@intel.com>; maciej.rabeda@linux.intel.com; Michael
> > > > Kubacki
> > > > >>> <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com)
> <afish@apple.com>; Leif Lindholm
> > <leif@nuviainc.com>
> > > > >>> Subject: Re: [edk2-devel] Uncrustify Conversion Detailed Plan and
> Extended Hard Freeze Update #4
> > > > >>>
> > > > >>> My reply is inline.
> > > > >>>
> > > > >>> Regards,
> > > > >>> Michael
> > > > >>>
> > > > >>> On 12/2/2021 2:45 PM, Michael D Kinney wrote:
> > > > >>>> Hi Maciej,
> > > > >>>>
> > > > >>>> Thanks for the feedback.
> > > > >>>>
> > > > >>>> * Example #1.This appears to be caused by the following
> uncrustify
> > > > >>>> settings:
> > > > >>>>
> > > > >>>> # The threshold for aligning on '=' in assignments.
> > > > >>>>
> > > > >>>> # Use a negative number for absolute thresholds.
> > > > >>>>
> > > > >>>> #
> > > > >>>>
> > > > >>>> # 0: No limit (default).
> > > > >>>>
> > > > >>>> align_assign_thresh= 0# number
> > > > >>>>
> > > > >>>> The edk2 setting for this is:
> > > > >>>>
> > > > >>>> align_assign_thresh= 4
> > > > >>>>
> > > > >>>> This means blocks of assignments that are different than more
> than 4
> > > > >>>> spaces will be considered a new block.
> > > > >>>>
> > > > >>>> ‘HwAddreLen’ and ‘Xid’ are more than 4 characters in length
> > > > >>>> different.Same for ‘Xid’ and ‘Reserved’.So
> > > > >>>>
> > > > >>>> uncrustify treats these as 3 different assignment blocks.
> > > > >>>>
> > > > >>>> If we change to the default value of 0: No limit, this example is
> > > > >>>> treated as a single block and all ‘=’ are aligned.
> > > > >>>>
> > > > >>>> Is there a reason ‘4’ was selected?Is there is any harm in using
> the
> > > > >>>> default of 0?
> > > > >>>>
> > > > >>> We can certainly change the threshold. '4' was derived by
> > > > >>> experimentation. This is an area that is somewhat subjective and
> every
> > > > >>> case is difficult to cover well. Please feel free to suggest any
> changes.
> > > > >>>
> > > > >>
> > > > >> I recommend we use the default value of 0. That way, any code that
> choose to
> > > > >> align assignments will still be aligned. If a developer does not like
> > > > >> short assignments and long assignments in the same code block to
> use the
> > > > >> long assignment column, they can always break the block up into
> multiple
> > > > >> blocks by adding a carriage return.
> > > > >>
> > > > >>> For the benefit of others, this file can be a useful refernce
> > > > >>> understanding spans, gaps, and thresholds.
> > > > >>>
> > > > >>>
> https://github.com/uncrustify/uncrustify/blob/master/documentation/htdocs
> /configuration.txt
> > > > >>>
> > > > >>>> * Example #2: Uncruistfy is confused by the DEBUG_CODE()
> macro.This is
> > > > >>>> not a traditional macro because the
> > > > >>>>
> > > > >>>> contents of the macro is a block of C code.I do not know how to
> convince
> > > > >>>> uncrustify that the contents of a
> > > > >>>>
> > > > >>>> macro like function call to be treated as a code block from an
> indent
> > > > >>>> perspective.
> > > > >>>>
> > > > >>> I believe this would have to be overridden in the Uncrustify fork
> since
> > > > >>> DEBUG_CODE() is being treated as a macro function call and code
> blocks
> > > > >>> are not expected there. In addition, some special treatment might
> be
> > > > >>> needed for alignment in between
> DEBUG_CODE_BEGIN()/DEBUG_CODE_END().
> > > > >>>
> > > > >>> I'm happy to look at this. However, regression validation can take a
> > > > >>> while so I'd like to make sure we need this now. In cases that do not
> > > > >>> have additional code blocks, it seems to format fairly well. Is this
> > > > >>> prevalent and impactful enough it must be fixed now? Or, could we
> > > > >>> revisit it with a follow up patch?
> > > > >>
> > > > >> How long does regression testing take? I see about 20 instances of
> > > > >> DEBUG_CODE() that are producing bad formatting. If the content
> inside
> > > > >> DEBUG_CODE() is a single line of a single block of statements without
> > > > >> any if/while/for/case statements that require further indent, then the
> > > > >> format looks ok.
> > > > >>
> > > > >> DEBUG_CODE_BEGIN() and DEBUG_CODE_END() would look better
> if the
> > > > >> contents between are also indented one level.
> > > > >>
> > > > >>>
> > > > >>>> * Example #3/#4: Uncrustify is confused by the OPTIONAL
> keyword.The
> > > > >>>> edk2 config declares it as a QUALIFIER
> > > > >>>>
> > > > >>>> like IN and OUT.However, OPTIONAL only appears at the end of
> the line
> > > > >>>> that declares a parameter to a
> > > > >>>>
> > > > >>>> function.There are 3 forms. One with comma after OPTIONAL.One
> with comma
> > > > >>>> before OPTIONAL, and
> > > > >>>>
> > > > >>>> one with no comma if the parameter is the last parameter in the
> function
> > > > >>>> declaration.
> > > > >>>>
> > > > >>>> TYPE ParamName OPTIONAL,
> > > > >>>>
> > > > >>>> TYPEParamName, OPTIONAL
> > > > >>>>
> > > > >>>> TYPEParamName OPTIONAL
> > > > >>>>
> > > > >>>> OPTIONAL is defined to nothing in edk2 builds.From a uncrustify
> > > > >>>> perspective, we really want is to be
> > > > >>>>
> > > > >>>> ignored or more correctly treated it as a token that is attached to
> > > > >>>> ParamName and the combination of
> > > > >>>>
> > > > >>>> ParamName and OPTIONAL treated as one unit to determine
> indents.
> > > > >>>>
> > > > >>> "TYPE ParamName, OPTIONAL" seems especially problematic and
> presents an
> > > > >>> inconsistency with the other formats. Mike, can you please let me
> know
> > > > >>> if you have the same observation and your thoughts on a consistent
> pattern?
> > > > >>>
> > > > >>>> Thanks,
> > > > >>>>
> > > > >>>> Mike
> > > > >>>>
> > > > >>>> *From:*devel@edk2.groups.io <devel@edk2.groups.io> *On
> Behalf Of *Maciej
> > > > >>>> Rabeda
> > > > >>>> *Sent:* Thursday, December 2, 2021 10:27 AM
> > > > >>>> *To:* devel@edk2.groups.io; Kinney, Michael D
> > > > >>>> <michael.d.kinney@intel.com>; Michael Kubacki
> > > > >>>> <michael.kubacki@microsoft.com>; Andrew Fish
> (afish@apple.com)
> > > > >>>> <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>
> > > > >>>> *Subject:* Re: [edk2-devel] Uncrustify Conversion Detailed Plan
> and
> > > > >>>> Extended Hard Freeze Update #4
> > > > >>>>
> > > > >>>> Hey Mike,
> > > > >>>>
> > > > >>>> While most of the changes related to fixing coding style violations,
> > > > >>>> there are a couple of changes to NetworkPkg in that PR that make
> the
> > > > >>>> code less readable. Examples below.
> > > > >>>>
> > > > >>>> Example 1:
> > > > >>>>
> > > > >>>>
> > > > >>>>
> > > > >>>> Example 2:
> > > > >>>>
> > > > >>>>
> > > > >>>> Example 3:
> > > > >>>>
> > > > >>>>
> > > > >>>> Example 4:
> > > > >>>>
> > > > >>>> On 30-Nov-21 23:34, Michael D Kinney wrote:
> > > > >>>>
> > > > >>>> Hello,
> > > > >>>>
> > > > >>>> Thank you for your patience during this extended hard
> freeze.
> > > > >>>>
> > > > >>>> Just one more step to go.There has been a delay in the
> review of
> > > > >>>>
> > > > >>>> the patch series with the uncrustify source
> changes.PR(6).This
> > > > >>>>
> > > > >>>> patch series was not sent out as patch review email
> because of its
> > > > >>>>
> > > > >>>> very large size.It only contains source style changes and the
> > > > >>>>
> > > > >>>> CompareBuild tool and GitHub action has shown there are
> no binary
> > > > >>>>
> > > > >>>> differences introduced with these source style changes.
> > > > >>>>
> > > > >>>> If you are a package maintainer, then please review the
> following
> > > > >>>>
> > > > >>>> branch/PR for your package contents and review the EDK II
> CI results
> > > > >>>>
> > > > >>>> and BuildCompare results.I do not expect a line by line
> review
> > > > >>>>
> > > > >>>> because we already had time to provide feedback on the
> source style
> > > > >>>>
> > > > >>>> performed by uncrustify.Instead, a Reviewed-by for your
> package
> > > > >>>>
> > > > >>>> indicates that you have reviewed the EDK II CI results and
> CompareBuild
> > > > >>>>
> > > > >>>> tool functionality and results and you accept the source
> style
> > > > >>>>
> > > > >>>> changes to your package.
> > > > >>>>
> > > > >>>>
> *https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyC
> hanges_V5
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyC
> hanges_V5>
> > > > >>>>
> > > > >>>> *https://github.com/tianocore/edk2/pull/2229
> <https://github.com/tianocore/edk2/pull/2229>
> > > > >>>>
> > > > >>>>
> *https://github.com/mdkinney/edk2/actions/runs/1521618836
> > > > >>> <https://github.com/mdkinney/edk2/actions/runs/1521618836>
> > > > >>>>
> > > > >>>> Additional details on this update below.
> > > > >>>>
> > > > >>>> Thank you,
> > > > >>>>
> > > > >>>> Mike
> > > > >>>>
> > > > >>>> Changes from Update #3
> > > > >>>>
> > > > >>>>
> ----------------------------------------------------------------------------
> > > > >>>>
> > > > >>>> * Pushed PR (5)
> > > > >>>>
> > > > >>>> * Added link to PR(6). EDK II CI Status is PASS. Build
> Compare PASS.
> > > > >>>>
> > > > >>>> * Waiting for review of PR (6)
> > > > >>>>
> > > > >>>> * Review of PR (7) completed and waiting for review of PR
> (6)
> > > > >>>>
> > > > >>>>
> ----------------------------------------------------------------------------
> > > > >>>>
> > > > >>>> Changes from Update #2
> > > > >>>>
> > > > >>>>
> ----------------------------------------------------------------------------
> > > > >>>>
> > > > >>>> * Changed order of PRs swapping (4) and (5).The PR that
> activates
> > > > >>>>
> > > > >>>> increases the max CI agent job time is independent of all
> the other
> > > > >>>>
> > > > >>>> PRs and its review is complete, so it can be committed now.
> > > > >>>>
> > > > >>>> * Pushed PRs (1), (2), (3), (4).
> > > > >>>>
> > > > >>>> * Waiting for review to complete for PRs (5) and (6)
> > > > >>>>
> > > > >>>> * Reviews complete for PR (7)
> > > > >>>>
> > > > >>>> * Identifies steps using git filter-branch to apply uncrustify
> changes to a
> > > > >>>>
> > > > >>>> code review patch series that was generated before the
> uncrustify changes
> > > > >>>>
> > > > >>>> avoiding manual merge.
> > > > >>>>
> > > > >>>> * Identified steps using git filter-repo to generate an
> alternate history of
> > > > >>>>
> > > > >>>> the edk2 repo with uncrustify changes applied on every
> commit.This may
> > > > >>>>
> > > > >>>> be useful when evaluating changes to files using tools like
> git blame
> > > > >>>>
> > > > >>>> without the large uncrustify patch series.
> > > > >>>>
> > > > >>>>
> ---------------------------------------------------------------------------
> > > > >>>>
> > > > >>>> Changes from Update #1
> > > > >>>>
> > > > >>>>
> ----------------------------------------------------------------------------
> > > > >>>>
> > > > >>>> * Changed order of PRs swapping (6) and (7).The PR that
> activates
> > > > >>>>
> > > > >>>> EDK II CI check UncrustifyCheck has to be last because it
> unconditionally
> > > > >>>>
> > > > >>>> checks all C/H files in all packages.Not just files that have
> been
> > > > >>>>
> > > > >>>> modified like some of the other checkers.
> > > > >>>>
> > > > >>>> * Updated link to the branch with the UncrustifyCheck
> plugin that has been
> > > > >>>>
> > > > >>>> updated with a one line change and Reviewed-by and
> Tested-by tags.
> > > > >>>>
> > > > >>>>
> https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin
> _v6
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugi
> n_v6>
> > > > >>>>
> > > > >>>> * Reviews complete for (1), (2), (3), (5), and (7)
> > > > >>>>
> > > > >>>>
> ---------------------------------------------------------------------------
> > > > >>>>
> > > > >>>> Michael Kubacki and I have prepared the patches required
> to apply the
> > > > >>>>
> > > > >>>> uncrustify changes and enable EDK II CI to check all
> submitted
> > > > >>>>
> > > > >>>> patches have been run through uncrustify.
> > > > >>>>
> > > > >>>> We have verified through the CompareBuild GitHub Action
> that the
> > > > >>>>
> > > > >>>> format changes performed by uncrustify have no functional
> changes.
> > > > >>>>
> > > > >>>> All of the OBJ, LIB, DLL, EFI, FFS, FV, and FD files match
> 100%
> > > > >>>>
> > > > >>>> across 70 VS2019/GCC5 builds of all package/platform DSC
> files in
> > > > >>>>
> > > > >>>> the edk2 repo.
> > > > >>>>
> > > > >>>> The hard freeze will be extended after the
> edk2-stable202111 tag until
> > > > >>>>
> > > > >>>> all uncrustify related changes are committed.We do not
> expect this
> > > > >>>>
> > > > >>>> to take more than a few days.Do not push any PRs until the
> hard
> > > > >>>>
> > > > >>>> freeze is lifted.
> > > > >>>>
> > > > >>>> The changes are broken up into 7 patch series/PRs.The PRs
> are ordered
> > > > >>>>
> > > > >>>> so they can be submitted using the normal submission
> process and EDK II
> > > > >>>>
> > > > >>>> CI will pass for each one.Details are listed below.
> > > > >>>>
> > > > >>>> Uncrustify 73.0.3 for EDK II
> > > > >>>>
> > > > >>>> =============================
> > > > >>>>
> > > > >>>> * Sources:https://dev.azure.com/projectmu/_git/Uncrustify
> <https://dev.azure.com/projectmu/_git/Uncrustify>
> > > > >>>>
> > > > >>>> *
> Documentation:https://dev.azure.com/projectmu/Uncrustify/_wiki/wikis/Un
> crustify.wiki/1/Project-Mu-(EDK-
> > II)-
> > > > Fork-
> > > > >>> Readme
> <https://dev.azure.com/projectmu/Uncrustify/_wiki/wikis/Uncrustify.wiki/1/
> Project-Mu-(EDK-II)-Fork-Readme>
> > > > >>>>
> > > > >>>> *
> Download:https://dev.azure.com/projectmu/Uncrustify/_packaging?_a=pack
> age&feed=mu_uncrustify&package=mu-
> > > > >>> uncrustify-release&protocolType=NuGet&version=73.0.3
> > > > >>>
> <https://dev.azure.com/projectmu/Uncrustify/_packaging?_a=package&feed
> =mu_uncrustify&package=mu-uncrustify-
> > > > >>> release&protocolType=NuGet&version=73.0.3>
> > > > >>>>
> > > > >>>> Installing Uncrustify
> > > > >>>>
> > > > >>>> ======================
> > > > >>>>
> > > > >>>> The Uncrustify tool is installed automatically when the
> Pytools
> > > > >>>>
> > > > >>>> environment is used and the stuart* commands are run to
> complete the
> > > > >>>>
> > > > >>>> environment setup.Please see:
> > > > >>>>
> > > > >>>>
> https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally
> > > > >>>
> <https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally>
> > > > >>>>
> > > > >>>> Uncrustify can also be installed from the download page
> listed above
> > > > >>>>
> > > > >>>> or built from sources from the source link above.
> > > > >>>>
> > > > >>>> The Documentation link provides instruction on how to run
> uncrustify from
> > > > >>>>
> > > > >>>> the command line or install as a Visual Studio Code
> plugin.The main
> > > > >>>>
> > > > >>>> uncrustify documentation also describes how to integrate
> with a few other
> > > > >>>>
> > > > >>>> editors.
> > > > >>>>
> > > > >>>> We have also discussed a client side githook.That effort has
> not started.
> > > > >>>>
> > > > >>>> Let us know if that is a feature you would find useful.
> > > > >>>>
> > > > >>>> Developer impact for new code reviews
> > > > >>>>
> > > > >>>> ======================================
> > > > >>>>
> > > > >>>> Once the uncrustify checker is active in EDK II CI,
> developers must
> > > > >>>>
> > > > >>>> make sure their patches are run through the uncrustify tool
> before
> > > > >>>>
> > > > >>>> sending the patches for review.
> > > > >>>>
> > > > >>>> Developers must install and run uncrustify against changes
> files before
> > > > >>>>
> > > > >>>> sending patch review emails or submitting PR for EDK II CI.If
> EDK II CI
> > > > >>>>
> > > > >>>> detects and differences in source formatting, then EDK II CI
> will fail
> > > > >>>>
> > > > >>>> and the developer must run uncrustify and resubmit the
> patches.
> > > > >>>>
> > > > >>>> Developer impact to patch series/PRs reviewed during
> edk2-stable201121 soft/hard freeze
> > > > >>>>
> > > > >>>>
> ==============================================================
> =========================
> > > > >>>>
> > > > >>>> Developers must rebase their changes after the uncrustify
> source changes are
> > > > >>>>
> > > > >>>> committed.The branch with a preview of the uncrustify
> changes can be used
> > > > >>>>
> > > > >>>> to start this rebase work.
> > > > >>>>
> > > > >>>>
> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyCh
> anges_V5
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyC
> hanges_V5>
> > > > >>>>
> > > > >>>> The following steps can be used to update an existing
> branch with the
> > > > >>>>
> > > > >>>> required uncrustify format.This is the Windows version.I will
> add
> > > > >>>>
> > > > >>>> the Linux version soon.
> > > > >>>>
> > > > >>>> 1) Fetch and checkout and rebase to latest edk2/master
> > > > >>>>
> > > > >>>> git fetch origin
> > > > >>>>
> > > > >>>> git checkout master
> > > > >>>>
> > > > >>>> git rebase origin/master
> > > > >>>>
> > > > >>>> 2) Make a backup copy of plugin UncrustifyCheck outside
> WORKSPACE.
> > > > >>>>
> > > > >>>> (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool
> executable and
> > > > >>>>
> > > > >>>> EDK II specific uncrustify configuration file available when
> working
> > > > >>>>
> > > > >>>> with a branch that does not have those tools in its scope.
> > > > >>>>
> > > > >>>> xcopy .pytool\Plugin\UncrustifyCheck
> C:\Temp\UncrustifyCheck
> > > > >>>>
> > > > >>>> 3) Check out the patch series branch (e.g. MyBranch)
> > > > >>>>
> > > > >>>> git checkout MyBranch
> > > > >>>>
> > > > >>>> 4) Rebase patch series against edk2-stable202111
> > > > >>>>
> > > > >>>>
> > > > >>>>
> > > > >>>> git rebase edk2-stable202111
> > > > >>>>
> > > > >>>> 5) Create new branch for the uncrustifed version (e.g.
> MyBranch_Uncrustified)
> > > > >>>>
> > > > >>>> git checkout -b MyBranch_Uncrustified
> > > > >>>>
> > > > >>>> 6) Use git filter-branch to uncrustify all the commits in the
> series
> > > > >>>>
> > > > >>>> between the rebase target from (2) and HEAD of the
> branch.A filter
> > > > >>>>
> > > > >>>> can be used to scope the uncrustify operations to only the
> C/H files
> > > > >>>>
> > > > >>>> in the specific package the patch series is against. (e.g.
> DynamicTablesPkg).
> > > > >>>>
> > > > >>>> BaseTools should always be excluded.If the package scoped
> filter is
> > > > >>>>
> > > > >>>> not used, it will still work, but will take longer to run
> because
> > > > >>>>
> > > > >>>> uncrustify will rescan every C/H files in the whole repo.
> > > > >>>>
> > > > >>>> git filter-branch --tree-filter "git ls-files
> DynamicTablesPkg*.c DynamicTablesPkg*.h :!BaseTools/* |
> > > > >>>
> c:\\Temp\\UncrustifyCheck\\mu-uncrustify-release_extdep\\Windows-x86\\
> uncrustify.exe -c
> > > > >>> c:\\Temp\\UncrustifyCheck\\uncrustify.cfg -F - --replace
> --no-backup --if-changed" edk2-stable202111..HEAD
> > > > >>>>
> > > > >>>> 7) Now that all the individual patches in the branch are
> uncrustified,
> > > > >>>>
> > > > >>>> rebase against latest edk2/master that is already
> uncrustified.
> > > > >>>>
> > > > >>>> git rebase master
> > > > >>>>
> > > > >>>> 8) Verify the patches in this new branch.
> > > > >>>>
> > > > >>>> Impacts to tracing history across the uncrusity changes
> > > > >>>>
> > > > >>>>
> =======================================================
> > > > >>>>
> > > > >>>> Tools the view file and line history do work with the large
> uncrustify
> > > > >>>>
> > > > >>>> patch series.One impact is that the operations can be very
> slow due
> > > > >>>>
> > > > >>>> to the large uncrustify patches.
> > > > >>>>
> > > > >>>> One option to provide a faster experience is to provide an
> alternate
> > > > >>>>
> > > > >>>> version of the edk2 repository as "documentation" that has
> the
> > > > >>>>
> > > > >>>> entire history re-written with uncrustify run on every
> commit.
> > > > >>>>
> > > > >>>> The tool called git-filter-repo can be used to perform this
> > > > >>>>
> > > > >>>> transformation and runs in a reasonable period of time (a
> few hours)
> > > > >>>>
> > > > >>>> https://github.com/newren/git-filter-repo
> <https://github.com/newren/git-filter-repo>
> > > > >>>>
> > > > >>>>
> https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-dem
> os/lint-history
> > > > >>>
> <https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-de
> mos/lint-history>
> > > > >>>>
> > > > >>>> The following steps can be used to perform this
> transformation.
> > > > >>>>
> > > > >>>> This is the Windows version. I will add the Linux version
> soon.
> > > > >>>>
> > > > >>>> ** WARNING **This operation modifies(rewrites) all the
> commits
> > > > >>>>
> > > > >>>> in the local copy of the repo.Do not perform
> > > > >>>>
> > > > >>>> these steps on a local repo you are using for
> > > > >>>>
> > > > >>>> active development.
> > > > >>>>
> > > > >>>> 1) Clone edk2 into a new directory (see **WARNING**)
> > > > >>>>
> > > > >>>> git clonehttps://github.com/tianocore/edk2.git
> <https://github.com/tianocore/edk2.git> edk2-uncrustified
> > > > >>>>
> > > > >>>> cd edk2-uncrustified
> > > > >>>>
> > > > >>>> 2) Setup python virtual env, install pytools, and run stuart
> commands
> > > > >>>>
> > > > >>>> to setup build environment which includes installing
> uncrustify tools.
> > > > >>>>
> > > > >>>>
> https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally
> > > > >>>
> <https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally>
> > > > >>>>
> > > > >>>> 3) Make a backup copy of plugin UncrustifyCheck outside
> WORKSPACE.
> > > > >>>>
> > > > >>>> (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool
> executable and
> > > > >>>>
> > > > >>>> EDK II specific uncrustify configuration file available when
> working
> > > > >>>>
> > > > >>>> with a branch that does not have those tools in its scope.
> > > > >>>>
> > > > >>>> xcopy .pytool\Plugin\UncrustifyCheck
> C:\Temp\UncrustifyCheck
> > > > >>>>
> > > > >>>> 4) Use lint-history.py from git-filter-repo examples
> > > > >>>>
> > > > >>>> https://github.com/newren/git-filter-repo
> <https://github.com/newren/git-filter-repo>
> > > > >>>>
> > > > >>>>
> https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-dem
> os/lint-history
> > > > >>>
> <https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-de
> mos/lint-history>
> > > > >>>>
> > > > >>>> Line #127 - Add try except around subprocess.check_call()
> with except
> > > > >>>>
> > > > >>>> being pass.This is required because there are a few commits
> of C
> > > > >>>>
> > > > >>>> files in the edk2 repo that have incorrect C syntax and do
> not
> > > > >>>>
> > > > >>>> build with a C compiler and break the uncrustify parser.Skip
> reformat
> > > > >>>>
> > > > >>>> of C files that can not be parsed by uncrustify.These rare
> instances
> > > > >>>>
> > > > >>>> are addressed in the commit that fixes the C syntax error.
> > > > >>>>
> > > > >>>> Run this slightly modified version of lint-history.Include only
> > > > >>>>
> > > > >>>> C/H files and exclude directories that start with 'Tools' or
> 'BaseTools'.
> > > > >>>>
> > > > >>>> This step took about 2.2 hours on a laptop.
> > > > >>>>
> > > > >>>> lint-history.py
> > > > >>>>
> > > > >>>> --relevant "return (not filename.startswith(b'Tools') and not
> filename.startswith(b'BaseTools') and
> > > > >>> (filename.endswith(b'.c') or filename.endswith(b'.h')))"
> > > > >>>>
> > > > >>>>
> c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\mu-uncrustify-release
> _extdep\\Windows-x86\\uncrustify.exe
> > -
> > > c
> > > > >>> c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\uncrustify.cfg
> --replace --no-backup --if-changed
> > > > >>>>
> > > > >>>> Order of PRs to apply during extended hard freeze
> > > > >>>>
> > > > >>>>
> ==================================================
> > > > >>>>
> > > > >>>> 1) Update EmulatorPkg Win Host [BuildOptions] MSFT
> CC_FLAGS to not force debug information
> > > > >>>>
> > > > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3747
> <https://bugzilla.tianocore.org/show_bug.cgi?id=3747>
> > > > >>>>
> > > > >>>>
> *https://github.com/mdkinney/edk2/tree/Bug_3747_EmulatorPkg_WinHost_
> ReproducibleBuild
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_3747_EmulatorPkg_WinHost_
> ReproducibleBuild>
> > > > >>>>
> > > > >>>> *https://github.com/tianocore/edk2/pull/2215
> <https://github.com/tianocore/edk2/pull/2215>
> > > > >>>>
> > > > >>>> * Required for EmulatorPkg to pass CompareBuild for
> VS2019 IA32/X64 builds.
> > > > >>>>
> > > > >>>> * Status: Review complete.PR pushed.
> > > > >>>>
> > > > >>>> 2) EccCheck should not revert staged and local changes
> > > > >>>>
> > > > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=2986
> <https://bugzilla.tianocore.org/show_bug.cgi?id=2986>
> > > > >>>>
> > > > >>>>
> *https://github.com/mdkinney/edk2/tree/Bug_2986_EccCheckRemoveGitRe
> vert_V2
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_2986_EccCheckRemoveGitRe
> vert_V2>
> > > > >>>>
> > > > >>>> *https://github.com/tianocore/edk2/pull/2216
> <https://github.com/tianocore/edk2/pull/2216>
> > > > >>>>
> > > > >>>> * Required for EDK II CI to complete in a reasonable period
> of time when
> > > > >>>>
> > > > >>>> processing the 4000+ source file style changes made by
> uncrustify.
> > > > >>>>
> > > > >>>> * Also fixes critical bugs that can potentially corrupt git state
> when
> > > > >>>>
> > > > >>>> EccCheck is run locally.
> > > > >>>>
> > > > >>>> * Status: Review complete.PR pushed.
> > > > >>>>
> > > > >>>> 3) Update pytool LicenseCheck plugin to use temp directory
> for diff output file
> > > > >>>>
> > > > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3746
> <https://bugzilla.tianocore.org/show_bug.cgi?id=3746>
> > > > >>>>
> > > > >>>>
> *https://github.com/mdkinney/edk2/tree/Bug_3746_LicenseCheckUseDiffOu
> tputFile_V2
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_3746_LicenseCheckUseDiffOu
> tputFile_V2>
> > > > >>>>
> > > > >>>> *https://github.com/tianocore/edk2/pull/2217
> <https://github.com/tianocore/edk2/pull/2217>
> > > > >>>>
> > > > >>>> * Required to reduce EDK II CI build times.
> > > > >>>>
> > > > >>>> * Status: Review complete.PR pushed.
> > > > >>>>
> > > > >>>> 4) Update max job time from 60 min to 120 minutes
> in .azurepipelines/templates
> > > > >>>>
> > > > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3750
> <https://bugzilla.tianocore.org/show_bug.cgi?id=3750>
> > > > >>>>
> > > > >>>>
> *https://github.com/mdkinney/edk2/tree/Bug_3750_IncreaseAzurePipelines
> Timeout
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_3750_IncreaseAzurePipelines
> Timeout>
> > > > >>>>
> > > > >>>> *https://github.com/tianocore/edk2/pull/2219
> <https://github.com/tianocore/edk2/pull/2219>
> > > > >>>>
> > > > >>>> * Required to allow EccCheck of uncrustify changes to
> complete on Azure
> > > > >>>>
> > > > >>>> Pipelines CI agents without timing out.
> > > > >>>>
> > > > >>>> * Status: Review complete.PR pushed.
> > > > >>>>
> > > > >>>> 5) Update Package YAML to ignore specific ECC files/errors
> > > > >>>>
> > > > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3749
> <https://bugzilla.tianocore.org/show_bug.cgi?id=3749>
> > > > >>>>
> > > > >>>>
> *https://github.com/mdkinney/edk2/tree/Bug_3749_EccCheckIgnoreFilesErr
> ors
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_3749_EccCheckIgnoreFilesErr
> ors>
> > > > >>>>
> > > > >>>> *https://github.com/tianocore/edk2/pull/2218
> <https://github.com/tianocore/edk2/pull/2218>
> > > > >>>>
> > > > >>>> * Required to pass EccCheck
> > > > >>>>
> > > > >>>> * Status: Review complete. PR pushed
> > > > >>>>
> > > > >>>> 6) Uncrustify Source Changes
> > > > >>>>
> > > > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3737
> <https://bugzilla.tianocore.org/show_bug.cgi?id=3737>
> > > > >>>>
> > > > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3739
> <https://bugzilla.tianocore.org/show_bug.cgi?id=3739>
> > > > >>>>
> > > > >>>>
> *https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyC
> hanges_V5
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyC
> hanges_V5>
> > > > >>>>
> > > > >>>> *https://github.com/tianocore/edk2/pull/2229
> <https://github.com/tianocore/edk2/pull/2229>
> > > > >>>>
> > > > >>>> * Build comparison result
> PASS:https://github.com/mdkinney/edk2/actions/runs/1521618836
> > > > >>> <https://github.com/mdkinney/edk2/actions/runs/1521618836>
> > > > >>>>
> > > > >>>> * EFI_D_ -> DEBUG changes required to pass PatchCheck
> > > > >>>>
> > > > >>>> * Uncrustify format changes required to pass
> UncrustifyCheck
> > > > >>>>
> > > > >>>> * Status:
> > > > >>>>
> > > > >>>> Waiting for review
> > > > >>>>
> > > > >>>> 7) UncrustifyCheck EDK II CI Plugin
> > > > >>>>
> > > > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3748
> <https://bugzilla.tianocore.org/show_bug.cgi?id=3748>
> > > > >>>>
> > > > >>>>
> *https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugi
> n_v6
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugi
> n_v6>
> > > > >>>>
> > > > >>>> * Required to enforce all PRs submitted to EDK II CI match
> uncrustify format.
> > > > >>>>
> > > > >>>> * Unconditionally checks all packages.Can not be
> committed until all C/H
> > > > >>>>
> > > > >>>> source files have been updated.
> > > > >>>>
> > > > >>>> * Status: Review complete
> > > > >>>>
> > > > >>>> Combined Branch/PR for Review/Test
> > > > >>>>
> > > > >>>> ==================================
> > > > >>>>
> > > > >>>> * Build Comparison results must pass 100% across the full
> set of PRs before
> > > > >>>>
> > > > >>>> the individual PRs can be pushed in the order listed above.
> > > > >>>>
> > > > >>>> *
> Branch:https://github.com/mdkinney/edk2/tree/TestOnly_Uncrustify_PR_Ser
> ies
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/TestOnly_Uncrustify_PR_Series>
> > > > >>>>
> > > > >>>> * PR:https://github.com/tianocore/edk2/pull/2229
> <https://github.com/tianocore/edk2/pull/2229>
> > > > >>>>
> > > > >>>> Status = PASS
> > > > >>>>
> > > > >>>> * CompareBuild:
> > > > >>>>
> > > > >>>>
> Branch:https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncr
> ustifyChanges_V5
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyC
> hanges_V5>
> > > > >>>>
> > > > >>>> --ref1:ef9a059cdb15844fe52a49af2bf7d86b9dd3e9bf
> > > > >>>>
> > > > >>>> --ref2:Bug_3737_3739_ApplyUncrustifyChanges_V5
> > > > >>>>
> > > > >>>> Extra Options: -n 4 --quiet
> > > > >>>>
> > > > >>>>
> Results:https://github.com/mdkinney/edk2/actions/runs/1521618836
> > > > >>> <https://github.com/mdkinney/edk2/actions/runs/1521618836>
> > > > >>>>
> > > > >>>> 30 VS2019 build comparisons PASS
> > > > >>>>
> > > > >>>> 40 GCC5 build comparisons PASS
> > > > >>>>
> > > > >>>> 100% PASS
> > > > >>>>
> > > > >>>> The following git log shows the set of patches from --ref1 to
> --ref 2across
> > > > >>>>
> > > > >>>> which there are no differences in any of the
> OBJ/LIB/DLL/EFI/FFS/FV/FD files.
> > > > >>>>
> > > > >>>> --ref2
> > > > >>>>
> > > > >>>> b7d4bf0675b7 (HEAD ->
> Bug_3737_3739_ApplyUncrustifyChanges_V5) UnitTestFrameworkPkg: Apply
> uncrusitify
> > changes
> > > > >>>>
> > > > >>>> 7f03d25f60e7 UefiPayloadPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 0bfd8d9b5ac9 UefiCpuPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> e1cd9bfb9dea StandaloneMmPkg: Apply uncrusitify
> changes
> > > > >>>>
> > > > >>>> 5da2f65be378 SourceLevelDebugPkg: Apply uncrusitify
> changes
> > > > >>>>
> > > > >>>> 95b86de07e5d SignedCapsulePkg: Apply uncrusitify
> changes
> > > > >>>>
> > > > >>>> fe71d97246c4 ShellPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 54c21c952992 SecurityPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 187a3785f12b RedfishPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 810100002a46 PcAtChipsetPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 276a695c0cf2 OvmfPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 303c0a91ab07 NetworkPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> bc80792cd1b1 MdePkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 3ea86be17a2a MdeModulePkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> c70ef11ed0cd IntelFsp2WrapperPkg: Apply uncrusitify
> changes
> > > > >>>>
> > > > >>>> c0291221f252 IntelFsp2Pkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 6a479952a690 FmpDevicePkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 3a7c05b7070d FatPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> b789f98c8959 EmulatorPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 952d7a1c9220 EmbeddedPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> a1cc9881bab6 DynamicTablesPkg: Apply uncrusitify
> changes
> > > > >>>>
> > > > >>>> 50654dfe5785 CryptoPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> ed965a02dfa1 ArmVirtPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 9744023fbc46 ArmPlatformPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 7a1cde5f5bba ArmPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 19d17e0913e8 UefiCpuPkg: Change use of EFI_D_* to
> DEBUG_*
> > > > >>>>
> > > > >>>> ffa718b4f994 SourceLevelDebugPkg: Change use of EFI_D_*
> to DEBUG_*
> > > > >>>>
> > > > >>>> b86cb3c5e5b4 ShellPkg: Change use of EFI_D_* to
> DEBUG_*
> > > > >>>>
> > > > >>>> c7c42204dc07 SecurityPkg: Change use of EFI_D_* to
> DEBUG_*
> > > > >>>>
> > > > >>>> 16b8e6f958e4 PcAtChipsetPkg: Change use of EFI_D_* to
> DEBUG_*
> > > > >>>>
> > > > >>>> 0ac3f8b2dac5 OvmfPkg: Change use of EFI_D_* to
> DEBUG_*
> > > > >>>>
> > > > >>>> bc5004b8d294 NetworkPkg: Change use of EFI_D_* to
> DEBUG_*
> > > > >>>>
> > > > >>>> 6f671a8e2377 MdePkg: Change use of EFI_D_* to
> DEBUG_*
> > > > >>>>
> > > > >>>> a10c610ff9a3 MdeModulePkg: Change use of EFI_D_* to
> DEBUG_*
> > > > >>>>
> > > > >>>> 09a3bddba390 FatPkg: Change use of EFI_D_* to DEBUG_*
> > > > >>>>
> > > > >>>> 59c61318246a EmulatorPkg: Change use of EFI_D_* to
> DEBUG_*
> > > > >>>>
> > > > >>>> 3a80367dda3b EmbeddedPkg: Change use of EFI_D_* to
> DEBUG_*
> > > > >>>>
> > > > >>>> 23eb1aaf80ca ArmVirtPkg: Change use of EFI_D_* to
> DEBUG_*
> > > > >>>>
> > > > >>>> 875914b45c54 ArmPlatformPkg: Change use of EFI_D_* to
> DEBUG_*
> > > > >>>>
> > > > >>>> eb2eca82b451 ArmPkg: Change use of EFI_D_* to
> DEBUG_*
> > > > >>>>
> > > > >>>> f0f3f5aae7c4 (origin/master, origin/HEAD, master)
> UnitTestFrameworkPkg: Update YAML to ignore specific ECC
> > > > >>> files/errors
> > > > >>>>
> > > > >>>> c05734797790 UefiPayloadPkg: Update YAML to ignore
> specific ECC files/errors
> > > > >>>>
> > > > >>>> c30c40d6c63d StandaloneMmPkg: Update YAML to ignore
> specific ECC files/errors
> > > > >>>>
> > > > >>>> 9944508e85f1 ShellPkg: Update YAML to ignore specific ECC
> files/errors
> > > > >>>>
> > > > >>>> 60fa40be458d SecurityPkg: Update YAML to ignore specific
> ECC files/errors
> > > > >>>>
> > > > >>>> df790cd6b37e MdePkg: Update YAML to ignore specific ECC
> files/errors
> > > > >>>>
> > > > >>>> 9deb9370766e MdeModulePkg: Update YAML to ignore
> specific ECC files/errors
> > > > >>>>
> > > > >>>> d7d30e8f219f EmulatorPkg: Update YAML to ignore specific
> ECC files/errors
> > > > >>>>
> > > > >>>> d5744ecba813 CryptoPkg: Update YAML to ignore specific
> ECC files/errors
> > > > >>>>
> > > > >>>> c97fee87f0f9 ArmVirtPkg: Update YAML to ignore specific
> ECC files/errors
> > > > >>>>
> > > > >>>> 1939fc9569f2 ArmPlatformPkg: Update YAML to ignore
> specific ECC files/errors
> > > > >>>>
> > > > >>>> 365dced2c37a ArmPkg: Update YAML to ignore specific ECC
> files/errors
> > > > >>>>
> > > > >>>> 76a1ce4d5fec .azurepipelines/templates: Update max
> pipeline job time to 2 hours
> > > > >>>>
> > > > >>>> 99f84ff47390 .pytools/Plugin/LicenseCheck: Use temp
> directory for git diff output
> > > > >>>>
> > > > >>>> 3019f1bbabf1 .pytool/Plugin/EccCheck: Add performance
> optimizations
> > > > >>>>
> > > > >>>> 854462bd3479 .pytool/Plugin/EccCheck: Remove temp
> directory on exception
> > > > >>>>
> > > > >>>> 69877614fdee .pytool/Plugin/EccCheck: Remove
> RevertCode()
> > > > >>>>
> > > > >>>> --ref1
> > > > >>>>
> > > > >>>> ef9a059cdb15 EmulatorPkg/Win/Host: Update CC_FLAGS
> > > > >>>>
> > > > >>>> bb1bba3d7767 (tag: edk2-stable202111) NetworkPkg: Fix
> invalid pointer for DNS response token on error
> > > > >>>>
> > > > >>>> Best regards,
> > > > >>>>
> > > > >>>> Mike
> > > > >>>>
> > > > >>>>
>
>
>
>
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
2021-12-06 1:17 ` Michael D Kinney
2021-12-06 1:23 ` Wu, Hao A
2021-12-06 9:31 ` 回复: " gaoliming
@ 2021-12-06 19:45 ` Bret Barkelew
2021-12-06 20:28 ` Maciej Rabeda
` (4 subsequent siblings)
7 siblings, 0 replies; 39+ messages in thread
From: Bret Barkelew @ 2021-12-06 19:45 UTC (permalink / raw)
To: devel@edk2.groups.io, Kinney, Michael D, Michael Kubacki,
maciej.rabeda@linux.intel.com, Michael Kubacki,
Andrew Fish (afish@apple.com), Lindholm, Leif
[-- Attachment #1: Type: text/plain, Size: 92182 bytes --]
Series Reviewed-by: Bret Barkelew <bret.barkelew@microsoft.com>
- Bret
From: Michael D Kinney via groups.io<mailto:michael.d.kinney=intel.com@groups.io>
Sent: Sunday, December 5, 2021 5:18 PM
To: Michael Kubacki<mailto:mikuback@linux.microsoft.com>; devel@edk2.groups.io<mailto:devel@edk2.groups.io>; maciej.rabeda@linux.intel.com<mailto:maciej.rabeda@linux.intel.com>; Michael Kubacki<mailto:Michael.Kubacki@microsoft.com>; Andrew Fish (afish@apple.com)<mailto:afish@apple.com>; Lindholm, Leif<mailto:leif@nuviainc.com>; Kinney, Michael D<mailto:michael.d.kinney@intel.com>
Subject: [EXTERNAL] Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
Hello EDK II Maintainers,
A detailed evaluation of the DEBUG_CODE() formatting issue has been completed.
The reason DEBUG_CODE() is a challenge is that this looks like a macro from a
C parsing perspective, but the EDK II usage places C statements or blocks of
C code as the parameter to this macro.
There are actually 2 methods available to mark a statement or a block of code
to be included when the DEBUG_PROPERTY_DEBUG_CODE_ENABLED bit is enabled in
PcdDebugPropertyMask. One is DEBUG_CODE() and the other is to mark the beginning
and end of a code block with DEBUG_CODE_BEGIN() and DEBUG_CODE_END(). In fact,
DEBUG_CODE() is implemented using DEBUG_CODE_BEGIN() and DEBUG_CODE_END() macros.
#define DEBUG_CODE(Expression) \
DEBUG_CODE_BEGIN (); \
Expression \
DEBUG_CODE_END ()
A complete review for the use of these DEBUG_CODE macros was performed on the
edk2 repo. Uncrustify performs good formatting for code blocks between
DEBUG_CODE_BEGIN() and DEBUG_CODE_END(). This is because these look like simple
macros calls with no parameters and the lines of C code between these 2 macros
is formatted correctly.
The uncrustify formatting issues are only present with the use of DEBUG_CODE().
Simple use cases of DEBUG_CODE(Expression) where Expression is a single C
statement also look correct. A medium complexity use case where Expression is
a code block of simple statements or even some local variables and simple
statements also look correct. It is only complex code blocks that use C
statements such as if/for/while/case that include the use of braces {} does
uncrustify perform incorrect formatting.
The recommended solution to this issue is to convert the use of DEBUG_CODE()
to DEBUG_CODE_BEGIN() / DEBUG_CODE_END() for cases where the Expression
passed to DEBUG_CODE() is the complex use case that contains statements that
use braces {}. There are 57 instances of this pattern across 40 files in the
edk2 repo.
I have posted a branch with these additional patches:
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Ftree%2FBug_3737_3760_3739_ApplyUncrustifyChanges_V7&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=NRfBw8TYqA1BkWCsfb6x5G%2FlnKOFpVbRbS%2BbCNJsKnE%3D&reserved=0
I have performed CompareBuild tests with this revised patch series with
the DEBUG_CODE changes. It passes 100% showing no binary differences.
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Factions%2Fruns%2F1542454606&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=UUnES9Tdyt6tps56RAoBA%2Fm47WqD4qdl4kBmlRAENCA%3D&reserved=0
I have opened a PR to run this patch series through EDK II CI. It also passes 100%.
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fpull%2F2236&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Xdg6OtNtet5JJhLdC0F7LHzLHGnJ6fQklsYfLXMkcXs%3D&reserved=0
The summary of changes made since the V6 review are:
1) Change uncrustify configuration assignment alignment threshold to 0
align_assign_thresh = 0
2) Replace "<param>, OPTIONAL" with "<param> OPTIONAL,"
3) Replace DEBUG_CODE(Expression) with
DEBUG_CODE_BEGIN();
Expression
DEBUG_CODE_END()
if Expression is complex and contains braces {}.
4) No changes to uncrustify tool required.
Please review the differences between the following 2 branches and provide
feedback or a Series Reviewed-by if you agree with these additional changes.
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Ftree%2FBug_3737_3760_3739_ApplyUncrustifyChanges_V6&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=oJyk1mqXffFT5dA8zwV3OpeQIWAydy%2FgXGccCGPiSOw%3D&reserved=0
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Ftree%2FBug_3737_3760_3739_ApplyUncrustifyChanges_V7&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=NRfBw8TYqA1BkWCsfb6x5G%2FlnKOFpVbRbS%2BbCNJsKnE%3D&reserved=0
The goal is to complete the review and get the uncrustify change committed
tomorrow so the extended hard freeze can be lifted.
Thanks,
Mike
> -----Original Message-----
> From: Kinney, Michael D <michael.d.kinney@intel.com>
> Sent: Thursday, December 2, 2021 6:23 PM
> To: Michael Kubacki <mikuback@linux.microsoft.com>; devel@edk2.groups.io; maciej.rabeda@linux.intel.com; Michael Kubacki
> <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com) <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>;
> Kinney, Michael D <michael.d.kinney@intel.com>
> Subject: RE: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
>
> Hello EDK II Maintainers,
>
> I have entered BZ 3760 to make the use of the OPTIONAL keyword style consistent for all of edk2 repo
> and to be compatible with uncrustify.
>
> I have posted the following V6 branch that does the EFI_D_* to DEBUG_* changes, the OPTIONAL keyword
> style changes, and the uncrustify changes with the one configuration change for assignment alignment.
>
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Ftree%2FBug_3737_3760_3739_ApplyUncrustifyChanges_V6&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=oJyk1mqXffFT5dA8zwV3OpeQIWAydy%2FgXGccCGPiSOw%3D&reserved=0
>
> Please provide feedback on the code style in this branch with the known DEBUG_CODE() issue still
> present.
>
> If we are able to quickly update uncrustify to handle DEBUG_CODE(), I will generate a V7.
>
> Thanks,
>
> Mike
>
> > -----Original Message-----
> > From: Kinney, Michael D <michael.d.kinney@intel.com>
> > Sent: Thursday, December 2, 2021 4:53 PM
> > To: Michael Kubacki <mikuback@linux.microsoft.com>; devel@edk2.groups.io; maciej.rabeda@linux.intel.com; Michael Kubacki
> > <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com) <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>;
> > Kinney, Michael D <michael.d.kinney@intel.com>
> > Subject: RE: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
> >
> > Michael,
> >
> > Yes. Please update the patch series that adds UncrustifyCheck with those changes.
> >
> > Thanks,
> >
> > Mike
> >
> > > -----Original Message-----
> > > From: Michael Kubacki <mikuback@linux.microsoft.com>
> > > Sent: Thursday, December 2, 2021 4:31 PM
> > > To: Kinney, Michael D <michael.d.kinney@intel.com>; devel@edk2.groups.io; maciej.rabeda@linux.intel.com; Michael
> Kubacki
> > > <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com) <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>
> > > Subject: Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
> > >
> > > Hi Mike,
> > >
> > > Thank you for the detailed analysis and recommendations.
> > >
> > > I agree with the recommendations and I will try to have an Uncrustify
> > > tool update by tomorrow for (4).
> > >
> > > That will require an update in uncrustify_ext_dep.yaml to pull in the
> > > new version. I'm assuming I should also update uncrustify.cfg to set
> > > align_assign_thresh = 0 in that new patch series?
> > >
> > > Regards,
> > > Michael
> > >
> > > On 12/2/2021 7:18 PM, Kinney, Michael D wrote:
> > > > Hi Michael,
> > > >
> > > > CORRECTION: set align_assign_threshold to 0.
> > > >
> > > > Reponses inline below.
> > > >
> > > > I would like to summarize the 4 issues raised in the past day along with the recommendations.
> > > >
> > > > 1) Exclusion feature for UncrustifyCheck. There are 2 directories with 8 files total that
> > > > maintainers have noted they would like to see not go through uncrustify formatting. Today
> > > > the only content that is skipped is BaseTools and submodules.
> > > >
> > > > Adding a general purpose exclusion feature would then require all developers to make
> > > > sure their method of using uncrustify also excludes those same areas. This requires
> > > > extra steps for all developers and maintainers.
> > > >
> > > > If we do not add the exclusion feature, then the 8 files will require an extra step
> > > > to sync with the original source of those files. The rate of changes of these 8 files
> > > > is very low today.
> > > >
> > > > RECOMMENDATION: Do not add exclusion feature at this time. Revisit if the extra work
> > > > to maintain the files that would be candidates for exclusions increases significantly.
> > > >
> > > > 2) Alignment of assignments. The threshold of 4 characters appears to be too low and causes
> > > > source files that are already aligned to become unaligned.
> > > >
> > > > RECOMMENDATION: Change threshold to the default value of 0 which means no limit.
> > > >
> > > > align_assign_thresh= 0
> > > >
> > > > 3) Alignment of parameters in function declaration not correct. The root cause of this
> > > > is the use of the OPTIONAL keyword. If the OPTIONAL keyword is removed, then the
> > > > alignment is correct. The alignment is also correct if the OPTIONAL keyword appears
> > > > before the ','. If the OPTIONAL keyword appears after the ',', then the format is
> > > > not correct. The OPTIONAL keyword indicates that the parameter in the function is
> > > > not required and may be passed in as NULL or 0 or some other default value defined by
> > > > the API. It makes more sense for this OPTIONAL keyword that follows the parameter
> > > > names to appear before the ',' so it is scoped to the parameter on that line. If it
> > > > appears after the ',', then C parsers thinks it is a prefix (IN, OUT, CONST, volatile,
> > > > static) for the next parameter in the function.
> > > >
> > > > RECOMMENDATION: Update patch series with a global search and replace so OPTIONAL
> > > > keyword always appears before the ',' on the same line.
> > > >
> > > > RegEx search string: ',( *)OPTIONAL( *)'
> > > > RegEx replace string: ' $1OPTIONAL,$2'
> > > >
> > > > 4) Format issues with complex blocks in DEBUG_CODE(), or between DEBUG_CODE_BEGIN() and
> > > > DEBUG_CODE_END(). Uncrustify treats these as macros and is not aware that the
> > > > parameter passed into the macro call is a block of C code that needs to be formatted.
> > > > Complex blocks with if/while/for/case statements are impacted the most.
> > > >
> > > > RECOMMENDATION: Update the uncrustify with an edk2 specific extension to treat these
> > > > macros as a block of code as if they were surrounded by an extra set of braces {}.
> > > >
> > > >
> > > > I have posted a branch for testing purposes that implements (2) and (3).
> > > >
> > > > Branch: https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Ftree%2FTestOnly_Bug_3737_3739_ApplyUncrustifyChanges_V6_OPTIONAL_Keyword_Fix&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=2cKi%2Bh5ZY%2FJqIJaa3o2ItdcxB%2BsM8yYNAWhnhL84g2s%3D&reserved=0
> > > > PR: https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fpull%2F2233&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=xVpoVwUPysxKWbuMPkPnGP1zbqUWwgvYRiy6P5VUvE4%3D&reserved=0
> > > > Status: PASS
> > > > CompareBuild: https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Factions%2Fruns%2F1532855914&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=L7BkXP8Gfc5wS2r8JJUyO%2F4HCmR37zvs9XVXQyvo8bs%3D&reserved=0
> > > > Status: PASS
> > > >
> > > > You can see what changed by fetching and comparing the following 2 branches:
> > > >
> > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Ftree%2FBug_3737_3739_ApplyUncrustifyChanges_V5&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=w6EsAK6%2FCt%2FMDRQI%2F2jnqYT173Ww2VLY0Mc7Lz2yuRw%3D&reserved=0
> > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Ftree%2FTestOnly_Bug_3737_3739_ApplyUncrustifyChanges_V6_OPTIONAL_Keyword_Fix&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=2cKi%2Bh5ZY%2FJqIJaa3o2ItdcxB%2BsM8yYNAWhnhL84g2s%3D&reserved=0
> > > >
> > > > Please provide feedback on the RECOMMENDATIONS above. I will go ahead and prepare of V6 version of
> > > > the patch series now that that test results are all PASS.
> > > >
> > > > Best regards,
> > > >
> > > > Mike
> > > >
> > > >
> > > >> -----Original Message-----
> > > >> From: Kinney, Michael D <michael.d.kinney@intel.com>
> > > >> Sent: Thursday, December 2, 2021 4:15 PM
> > > >> To: Michael Kubacki <mikuback@linux.microsoft.com>; devel@edk2.groups.io; maciej.rabeda@linux.intel.com; Michael
> > > Kubacki
> > > >> <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com) <afish@apple.com>; Leif Lindholm
> <leif@nuviainc.com>;
> > > >> Kinney, Michael D <michael.d.kinney@intel.com>
> > > >> Subject: RE: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
> > > >>
> > > >> Hi Michael,
> > > >>
> > > >> Reponses inline below.
> > > >>
> > > >> I would like to summarize the 4 issues raised in the past day along with the recommendations.
> > > >>
> > > >> 1) Exclusion feature for UncrustifyCheck. There are 2 directories with 8 files total that
> > > >> maintainers have noted they would like to see not go through uncrustify formatting. Today
> > > >> the only content that is skipped is BaseTools and submodules.
> > > >>
> > > >> Adding a general purpose exclusion feature would then require all developers to make
> > > >> sure their method of using uncrustify also excludes those same areas. This requires
> > > >> extra steps for all developers and maintainers.
> > > >>
> > > >> If we do not add the exclusion feature, then the 8 files will require an extra step
> > > >> to sync with the original source of those files. The rate of changes of these 8 files
> > > >> is very low today.
> > > >>
> > > >> RECOMMENDATION: Do not add exclusion feature at this time. Revisit if the extra work
> > > >> to maintain the files that would be candidates for exclusions increases significantly.
> > > >>
> > > >> 2) Alignment of assignments. The threshold of 4 characters appears to be too low and causes
> > > >> source files that are already aligned to become unaligned.
> > > >>
> > > >> RECOMMENDATION: Change threshold to the default value of 0 which means no limit.
> > > >>
> > > >> align_assign_thresh= 4
> > > >>
> > > >> 3) Alignment of parameters in function declaration not correct. The root cause of this
> > > >> is the use of the OPTIONAL keyword. If the OPTIONAL keyword is removed, then the
> > > >> alignment is correct. The alignment is also correct if the OPTIONAL keyword appears
> > > >> before the ','. If the OPTIONAL keyword appears after the ',', then the format is
> > > >> not correct. The OPTIONAL keyword indicates that the parameter in the function is
> > > >> not required and may be passed in as NULL or 0 or some other default value defined by
> > > >> the API. It makes more sense for this OPTIONAL keyword that follows the parameter
> > > >> names to appear before the ',' so it is scoped to the parameter on that line. If it
> > > >> appears after the ',', then C parsers thinks it is a prefix (IN, OUT, CONST, volatile,
> > > >> static) for the next parameter in the function.
> > > >>
> > > >> RECOMMENDATION: Update patch series with a global search and replace so OPTIONAL
> > > >> keyword always appears before the ',' on the same line.
> > > >>
> > > >> RegEx search string: ',( *)OPTIONAL( *)'
> > > >> RegEx replace string: ' $1OPTIONAL,$2'
> > > >>
> > > >> 4) Format issues with complex blocks in DEBUG_CODE(), or between DEBUG_CODE_BEGIN() and
> > > >> DEBUG_CODE_END(). Uncrustify treats these as macros and is not aware that the
> > > >> parameter passed into the macro call is a block of C code that needs to be formatted.
> > > >> Complex blocks with if/while/for/case statements are impacted the most.
> > > >>
> > > >> RECOMMENDATION: Update the uncrustify with an edk2 specific extension to treat these
> > > >> macros as a block of code as if they were surrounded by an extra set of braces {}.
> > > >>
> > > >>
> > > >> I have posted a branch for testing purposes that implements (2) and (3).
> > > >>
> > > >> Branch: https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Ftree%2FTestOnly_Bug_3737_3739_ApplyUncrustifyChanges_V6_OPTIONAL_Keyword_Fix&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=2cKi%2Bh5ZY%2FJqIJaa3o2ItdcxB%2BsM8yYNAWhnhL84g2s%3D&reserved=0
> > > >> PR: https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fpull%2F2233&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=xVpoVwUPysxKWbuMPkPnGP1zbqUWwgvYRiy6P5VUvE4%3D&reserved=0
> > > >> Status: PASS
> > > >> CompareBuild: https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Factions%2Fruns%2F1532855914&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=L7BkXP8Gfc5wS2r8JJUyO%2F4HCmR37zvs9XVXQyvo8bs%3D&reserved=0
> > > >> Status: PASS
> > > >>
> > > >> You can see what changed by fetching and comparing the following 2 branches:
> > > >>
> > > >> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Ftree%2FBug_3737_3739_ApplyUncrustifyChanges_V5&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=w6EsAK6%2FCt%2FMDRQI%2F2jnqYT173Ww2VLY0Mc7Lz2yuRw%3D&reserved=0
> > > >> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Ftree%2FTestOnly_Bug_3737_3739_ApplyUncrustifyChanges_V6_OPTIONAL_Keyword_Fix&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=2cKi%2Bh5ZY%2FJqIJaa3o2ItdcxB%2BsM8yYNAWhnhL84g2s%3D&reserved=0
> > > >>
> > > >> Please provide feedback on the RECOMMENDATIONS above. I will go ahead and prepare of V6 version of
> > > >> the patch series now that that test results are all PASS.
> > > >>
> > > >> Best regards,
> > > >>
> > > >> Mike
> > > >>
> > > >>
> > > >>> -----Original Message-----
> > > >>> From: Michael Kubacki <mikuback@linux.microsoft.com>
> > > >>> Sent: Thursday, December 2, 2021 1:57 PM
> > > >>> To: devel@edk2.groups.io; Kinney, Michael D <michael.d.kinney@intel.com>; maciej.rabeda@linux.intel.com; Michael
> > > Kubacki
> > > >>> <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com) <afish@apple.com>; Leif Lindholm
> <leif@nuviainc.com>
> > > >>> Subject: Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
> > > >>>
> > > >>> My reply is inline.
> > > >>>
> > > >>> Regards,
> > > >>> Michael
> > > >>>
> > > >>> On 12/2/2021 2:45 PM, Michael D Kinney wrote:
> > > >>>> Hi Maciej,
> > > >>>>
> > > >>>> Thanks for the feedback.
> > > >>>>
> > > >>>> * Example #1.This appears to be caused by the following uncrustify
> > > >>>> settings:
> > > >>>>
> > > >>>> # The threshold for aligning on '=' in assignments.
> > > >>>>
> > > >>>> # Use a negative number for absolute thresholds.
> > > >>>>
> > > >>>> #
> > > >>>>
> > > >>>> # 0: No limit (default).
> > > >>>>
> > > >>>> align_assign_thresh= 0# number
> > > >>>>
> > > >>>> The edk2 setting for this is:
> > > >>>>
> > > >>>> align_assign_thresh= 4
> > > >>>>
> > > >>>> This means blocks of assignments that are different than more than 4
> > > >>>> spaces will be considered a new block.
> > > >>>>
> > > >>>> 'HwAddreLen' and 'Xid' are more than 4 characters in length
> > > >>>> different.Same for 'Xid' and 'Reserved'.So
> > > >>>>
> > > >>>> uncrustify treats these as 3 different assignment blocks.
> > > >>>>
> > > >>>> If we change to the default value of 0: No limit, this example is
> > > >>>> treated as a single block and all '=' are aligned.
> > > >>>>
> > > >>>> Is there a reason '4' was selected?Is there is any harm in using the
> > > >>>> default of 0?
> > > >>>>
> > > >>> We can certainly change the threshold. '4' was derived by
> > > >>> experimentation. This is an area that is somewhat subjective and every
> > > >>> case is difficult to cover well. Please feel free to suggest any changes.
> > > >>>
> > > >>
> > > >> I recommend we use the default value of 0. That way, any code that choose to
> > > >> align assignments will still be aligned. If a developer does not like
> > > >> short assignments and long assignments in the same code block to use the
> > > >> long assignment column, they can always break the block up into multiple
> > > >> blocks by adding a carriage return.
> > > >>
> > > >>> For the benefit of others, this file can be a useful refernce
> > > >>> understanding spans, gaps, and thresholds.
> > > >>>
> > > >>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Funcrustify%2Funcrustify%2Fblob%2Fmaster%2Fdocumentation%2Fhtdocs%2Fconfiguration.txt&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=8kCcSNIphkcTJgrS9sgjSQJyRjFtDuy2u8HGz0LDadI%3D&reserved=0
> > > >>>
> > > >>>> * Example #2: Uncruistfy is confused by the DEBUG_CODE() macro.This is
> > > >>>> not a traditional macro because the
> > > >>>>
> > > >>>> contents of the macro is a block of C code.I do not know how to convince
> > > >>>> uncrustify that the contents of a
> > > >>>>
> > > >>>> macro like function call to be treated as a code block from an indent
> > > >>>> perspective.
> > > >>>>
> > > >>> I believe this would have to be overridden in the Uncrustify fork since
> > > >>> DEBUG_CODE() is being treated as a macro function call and code blocks
> > > >>> are not expected there. In addition, some special treatment might be
> > > >>> needed for alignment in between DEBUG_CODE_BEGIN()/DEBUG_CODE_END().
> > > >>>
> > > >>> I'm happy to look at this. However, regression validation can take a
> > > >>> while so I'd like to make sure we need this now. In cases that do not
> > > >>> have additional code blocks, it seems to format fairly well. Is this
> > > >>> prevalent and impactful enough it must be fixed now? Or, could we
> > > >>> revisit it with a follow up patch?
> > > >>
> > > >> How long does regression testing take? I see about 20 instances of
> > > >> DEBUG_CODE() that are producing bad formatting. If the content inside
> > > >> DEBUG_CODE() is a single line of a single block of statements without
> > > >> any if/while/for/case statements that require further indent, then the
> > > >> format looks ok.
> > > >>
> > > >> DEBUG_CODE_BEGIN() and DEBUG_CODE_END() would look better if the
> > > >> contents between are also indented one level.
> > > >>
> > > >>>
> > > >>>> * Example #3/#4: Uncrustify is confused by the OPTIONAL keyword.The
> > > >>>> edk2 config declares it as a QUALIFIER
> > > >>>>
> > > >>>> like IN and OUT.However, OPTIONAL only appears at the end of the line
> > > >>>> that declares a parameter to a
> > > >>>>
> > > >>>> function.There are 3 forms. One with comma after OPTIONAL.One with comma
> > > >>>> before OPTIONAL, and
> > > >>>>
> > > >>>> one with no comma if the parameter is the last parameter in the function
> > > >>>> declaration.
> > > >>>>
> > > >>>> TYPE ParamName OPTIONAL,
> > > >>>>
> > > >>>> TYPEParamName, OPTIONAL
> > > >>>>
> > > >>>> TYPEParamName OPTIONAL
> > > >>>>
> > > >>>> OPTIONAL is defined to nothing in edk2 builds.From a uncrustify
> > > >>>> perspective, we really want is to be
> > > >>>>
> > > >>>> ignored or more correctly treated it as a token that is attached to
> > > >>>> ParamName and the combination of
> > > >>>>
> > > >>>> ParamName and OPTIONAL treated as one unit to determine indents.
> > > >>>>
> > > >>> "TYPE ParamName, OPTIONAL" seems especially problematic and presents an
> > > >>> inconsistency with the other formats. Mike, can you please let me know
> > > >>> if you have the same observation and your thoughts on a consistent pattern?
> > > >>>
> > > >>>> Thanks,
> > > >>>>
> > > >>>> Mike
> > > >>>>
> > > >>>> *From:*devel@edk2.groups.io <devel@edk2.groups.io> *On Behalf Of *Maciej
> > > >>>> Rabeda
> > > >>>> *Sent:* Thursday, December 2, 2021 10:27 AM
> > > >>>> *To:* devel@edk2.groups.io; Kinney, Michael D
> > > >>>> <michael.d.kinney@intel.com>; Michael Kubacki
> > > >>>> <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com)
> > > >>>> <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>
> > > >>>> *Subject:* Re: [edk2-devel] Uncrustify Conversion Detailed Plan and
> > > >>>> Extended Hard Freeze Update #4
> > > >>>>
> > > >>>> Hey Mike,
> > > >>>>
> > > >>>> While most of the changes related to fixing coding style violations,
> > > >>>> there are a couple of changes to NetworkPkg in that PR that make the
> > > >>>> code less readable. Examples below.
> > > >>>>
> > > >>>> Example 1:
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>> Example 2:
> > > >>>>
> > > >>>>
> > > >>>> Example 3:
> > > >>>>
> > > >>>>
> > > >>>> Example 4:
> > > >>>>
> > > >>>> On 30-Nov-21 23:34, Michael D Kinney wrote:
> > > >>>>
> > > >>>> Hello,
> > > >>>>
> > > >>>> Thank you for your patience during this extended hard freeze.
> > > >>>>
> > > >>>> Just one more step to go.There has been a delay in the review of
> > > >>>>
> > > >>>> the patch series with the uncrustify source changes.PR(6).This
> > > >>>>
> > > >>>> patch series was not sent out as patch review email because of its
> > > >>>>
> > > >>>> very large size.It only contains source style changes and the
> > > >>>>
> > > >>>> CompareBuild tool and GitHub action has shown there are no binary
> > > >>>>
> > > >>>> differences introduced with these source style changes.
> > > >>>>
> > > >>>> If you are a package maintainer, then please review the following
> > > >>>>
> > > >>>> branch/PR for your package contents and review the EDK II CI results
> > > >>>>
> > > >>>> and BuildCompare results.I do not expect a line by line review
> > > >>>>
> > > >>>> because we already had time to provide feedback on the source style
> > > >>>>
> > > >>>> performed by uncrustify.Instead, a Reviewed-by for your package
> > > >>>>
> > > >>>> indicates that you have reviewed the EDK II CI results and CompareBuild
> > > >>>>
> > > >>>> tool functionality and results and you accept the source style
> > > >>>>
> > > >>>> changes to your package.
> > > >>>>
> > > >>>> *https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Ftree%2FBug_3737_3739_ApplyUncrustifyChanges_V5&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=w6EsAK6%2FCt%2FMDRQI%2F2jnqYT173Ww2VLY0Mc7Lz2yuRw%3D&reserved=0
> > > >>> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Ftree%2FBug_3737_3739_ApplyUncrustifyChanges_V5&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=w6EsAK6%2FCt%2FMDRQI%2F2jnqYT173Ww2VLY0Mc7Lz2yuRw%3D&reserved=0>
> > > >>>>
> > > >>>> *https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fpull%2F2229&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=YOlhPHAaa7FMDPmDcXhVZBKMPIFSqcddc4voMd5it7g%3D&reserved=0 <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fpull%2F2229&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=YOlhPHAaa7FMDPmDcXhVZBKMPIFSqcddc4voMd5it7g%3D&reserved=0>
> > > >>>>
> > > >>>> *https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Factions%2Fruns%2F1521618836&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=jtB8aYxhDScxzKf1uPlK0qSEiScqW5xZ41tFwBd%2Bh0Q%3D&reserved=0
> > > >>> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Factions%2Fruns%2F1521618836&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=jtB8aYxhDScxzKf1uPlK0qSEiScqW5xZ41tFwBd%2Bh0Q%3D&reserved=0>
> > > >>>>
> > > >>>> Additional details on this update below.
> > > >>>>
> > > >>>> Thank you,
> > > >>>>
> > > >>>> Mike
> > > >>>>
> > > >>>> Changes from Update #3
> > > >>>>
> > > >>>> ----------------------------------------------------------------------------
> > > >>>>
> > > >>>> * Pushed PR (5)
> > > >>>>
> > > >>>> * Added link to PR(6). EDK II CI Status is PASS. Build Compare PASS.
> > > >>>>
> > > >>>> * Waiting for review of PR (6)
> > > >>>>
> > > >>>> * Review of PR (7) completed and waiting for review of PR (6)
> > > >>>>
> > > >>>> ----------------------------------------------------------------------------
> > > >>>>
> > > >>>> Changes from Update #2
> > > >>>>
> > > >>>> ----------------------------------------------------------------------------
> > > >>>>
> > > >>>> * Changed order of PRs swapping (4) and (5).The PR that activates
> > > >>>>
> > > >>>> increases the max CI agent job time is independent of all the other
> > > >>>>
> > > >>>> PRs and its review is complete, so it can be committed now.
> > > >>>>
> > > >>>> * Pushed PRs (1), (2), (3), (4).
> > > >>>>
> > > >>>> * Waiting for review to complete for PRs (5) and (6)
> > > >>>>
> > > >>>> * Reviews complete for PR (7)
> > > >>>>
> > > >>>> * Identifies steps using git filter-branch to apply uncrustify changes to a
> > > >>>>
> > > >>>> code review patch series that was generated before the uncrustify changes
> > > >>>>
> > > >>>> avoiding manual merge.
> > > >>>>
> > > >>>> * Identified steps using git filter-repo to generate an alternate history of
> > > >>>>
> > > >>>> the edk2 repo with uncrustify changes applied on every commit.This may
> > > >>>>
> > > >>>> be useful when evaluating changes to files using tools like git blame
> > > >>>>
> > > >>>> without the large uncrustify patch series.
> > > >>>>
> > > >>>> ---------------------------------------------------------------------------
> > > >>>>
> > > >>>> Changes from Update #1
> > > >>>>
> > > >>>> ----------------------------------------------------------------------------
> > > >>>>
> > > >>>> * Changed order of PRs swapping (6) and (7).The PR that activates
> > > >>>>
> > > >>>> EDK II CI check UncrustifyCheck has to be last because it unconditionally
> > > >>>>
> > > >>>> checks all C/H files in all packages.Not just files that have been
> > > >>>>
> > > >>>> modified like some of the other checkers.
> > > >>>>
> > > >>>> * Updated link to the branch with the UncrustifyCheck plugin that has been
> > > >>>>
> > > >>>> updated with a one line change and Reviewed-by and Tested-by tags.
> > > >>>>
> > > >>>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Ftree%2FBug_3748_add_uncrustify_ci_plugin_v6&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=jrViYKgnChmrQCFQNufpKFrHbPv2a0Rce%2BaHAVqXHDs%3D&reserved=0
> > > >>> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Ftree%2FBug_3748_add_uncrustify_ci_plugin_v6&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=jrViYKgnChmrQCFQNufpKFrHbPv2a0Rce%2BaHAVqXHDs%3D&reserved=0>
> > > >>>>
> > > >>>> * Reviews complete for (1), (2), (3), (5), and (7)
> > > >>>>
> > > >>>> ---------------------------------------------------------------------------
> > > >>>>
> > > >>>> Michael Kubacki and I have prepared the patches required to apply the
> > > >>>>
> > > >>>> uncrustify changes and enable EDK II CI to check all submitted
> > > >>>>
> > > >>>> patches have been run through uncrustify.
> > > >>>>
> > > >>>> We have verified through the CompareBuild GitHub Action that the
> > > >>>>
> > > >>>> format changes performed by uncrustify have no functional changes.
> > > >>>>
> > > >>>> All of the OBJ, LIB, DLL, EFI, FFS, FV, and FD files match 100%
> > > >>>>
> > > >>>> across 70 VS2019/GCC5 builds of all package/platform DSC files in
> > > >>>>
> > > >>>> the edk2 repo.
> > > >>>>
> > > >>>> The hard freeze will be extended after the edk2-stable202111 tag until
> > > >>>>
> > > >>>> all uncrustify related changes are committed.We do not expect this
> > > >>>>
> > > >>>> to take more than a few days.Do not push any PRs until the hard
> > > >>>>
> > > >>>> freeze is lifted.
> > > >>>>
> > > >>>> The changes are broken up into 7 patch series/PRs.The PRs are ordered
> > > >>>>
> > > >>>> so they can be submitted using the normal submission process and EDK II
> > > >>>>
> > > >>>> CI will pass for each one.Details are listed below.
> > > >>>>
> > > >>>> Uncrustify 73.0.3 for EDK II
> > > >>>>
> > > >>>> =============================
> > > >>>>
> > > >>>> * Sources:https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdev.azure.com%2Fprojectmu%2F_git%2FUncrustify&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=oCLdWT7EjDK7KX2apqawcHLufS%2BTjPcHnN0n5A86iJQ%3D&reserved=0 <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdev.azure.com%2Fprojectmu%2F_git%2FUncrustify&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=oCLdWT7EjDK7KX2apqawcHLufS%2BTjPcHnN0n5A86iJQ%3D&reserved=0>
> > > >>>>
> > > >>>> * Documentation:https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdev.azure.com%2Fprojectmu%2FUncrustify%2F_wiki%2Fwikis%2FUncrustify.wiki%2F1%2FProject-Mu-&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=rnYkCXUpEi0SAFob0G3QCM%2F597t5B3yup37%2BW4CbuLo%3D&reserved=0(EDK-
> II)-
> > > Fork-
> > > >>> Readme <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdev.azure.com%2Fprojectmu%2FUncrustify%2F_wiki%2Fwikis%2FUncrustify.wiki%2F1%2FProject-Mu-&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=rnYkCXUpEi0SAFob0G3QCM%2F597t5B3yup37%2BW4CbuLo%3D&reserved=0(EDK-II)-Fork-Readme>
> > > >>>>
> > > >>>> * Download:https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdev.azure.com%2Fprojectmu%2FUncrustify%2F_packaging%3F_a%3Dpackage%26feed%3Dmu_uncrustify%26package%3Dmu-&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=psK3sLLB40youbw2XzPiCXaHhFTuEkBakbTdwe7N3Yg%3D&reserved=0
> > > >>> uncrustify-release&protocolType=NuGet&version=73.0.3
> > > >>> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdev.azure.com%2Fprojectmu%2FUncrustify%2F_packaging%3F_a%3Dpackage%26feed%3Dmu_uncrustify%26package%3Dmu-uncrustify-&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=TVyY15YO7%2FjqovaV15GY5k%2BkJiXFS4X9nqkeXtBhgZ8%3D&reserved=0
> > > >>> release&protocolType=NuGet&version=73.0.3>
> > > >>>>
> > > >>>> Installing Uncrustify
> > > >>>>
> > > >>>> ======================
> > > >>>>
> > > >>>> The Uncrustify tool is installed automatically when the Pytools
> > > >>>>
> > > >>>> environment is used and the stuart* commands are run to complete the
> > > >>>>
> > > >>>> environment setup.Please see:
> > > >>>>
> > > >>>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2F.pytool%23running-ci-locally&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=RaZxkrLB%2FFH6leSzUDxJWwSIF6phK8fbPyM1%2FyjLT1g%3D&reserved=0
> > > >>> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2F.pytool%23running-ci-locally&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=RaZxkrLB%2FFH6leSzUDxJWwSIF6phK8fbPyM1%2FyjLT1g%3D&reserved=0>
> > > >>>>
> > > >>>> Uncrustify can also be installed from the download page listed above
> > > >>>>
> > > >>>> or built from sources from the source link above.
> > > >>>>
> > > >>>> The Documentation link provides instruction on how to run uncrustify from
> > > >>>>
> > > >>>> the command line or install as a Visual Studio Code plugin.The main
> > > >>>>
> > > >>>> uncrustify documentation also describes how to integrate with a few other
> > > >>>>
> > > >>>> editors.
> > > >>>>
> > > >>>> We have also discussed a client side githook.That effort has not started.
> > > >>>>
> > > >>>> Let us know if that is a feature you would find useful.
> > > >>>>
> > > >>>> Developer impact for new code reviews
> > > >>>>
> > > >>>> ======================================
> > > >>>>
> > > >>>> Once the uncrustify checker is active in EDK II CI, developers must
> > > >>>>
> > > >>>> make sure their patches are run through the uncrustify tool before
> > > >>>>
> > > >>>> sending the patches for review.
> > > >>>>
> > > >>>> Developers must install and run uncrustify against changes files before
> > > >>>>
> > > >>>> sending patch review emails or submitting PR for EDK II CI.If EDK II CI
> > > >>>>
> > > >>>> detects and differences in source formatting, then EDK II CI will fail
> > > >>>>
> > > >>>> and the developer must run uncrustify and resubmit the patches.
> > > >>>>
> > > >>>> Developer impact to patch series/PRs reviewed during edk2-stable201121 soft/hard freeze
> > > >>>>
> > > >>>> =======================================================================================
> > > >>>>
> > > >>>> Developers must rebase their changes after the uncrustify source changes are
> > > >>>>
> > > >>>> committed.The branch with a preview of the uncrustify changes can be used
> > > >>>>
> > > >>>> to start this rebase work.
> > > >>>>
> > > >>>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Ftree%2FBug_3737_3739_ApplyUncrustifyChanges_V5&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=w6EsAK6%2FCt%2FMDRQI%2F2jnqYT173Ww2VLY0Mc7Lz2yuRw%3D&reserved=0
> > > >>> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Ftree%2FBug_3737_3739_ApplyUncrustifyChanges_V5&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=w6EsAK6%2FCt%2FMDRQI%2F2jnqYT173Ww2VLY0Mc7Lz2yuRw%3D&reserved=0>
> > > >>>>
> > > >>>> The following steps can be used to update an existing branch with the
> > > >>>>
> > > >>>> required uncrustify format.This is the Windows version.I will add
> > > >>>>
> > > >>>> the Linux version soon.
> > > >>>>
> > > >>>> 1) Fetch and checkout and rebase to latest edk2/master
> > > >>>>
> > > >>>> git fetch origin
> > > >>>>
> > > >>>> git checkout master
> > > >>>>
> > > >>>> git rebase origin/master
> > > >>>>
> > > >>>> 2) Make a backup copy of plugin UncrustifyCheck outside WORKSPACE.
> > > >>>>
> > > >>>> (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable and
> > > >>>>
> > > >>>> EDK II specific uncrustify configuration file available when working
> > > >>>>
> > > >>>> with a branch that does not have those tools in its scope.
> > > >>>>
> > > >>>> xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
> > > >>>>
> > > >>>> 3) Check out the patch series branch (e.g. MyBranch)
> > > >>>>
> > > >>>> git checkout MyBranch
> > > >>>>
> > > >>>> 4) Rebase patch series against edk2-stable202111
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>> git rebase edk2-stable202111
> > > >>>>
> > > >>>> 5) Create new branch for the uncrustifed version (e.g. MyBranch_Uncrustified)
> > > >>>>
> > > >>>> git checkout -b MyBranch_Uncrustified
> > > >>>>
> > > >>>> 6) Use git filter-branch to uncrustify all the commits in the series
> > > >>>>
> > > >>>> between the rebase target from (2) and HEAD of the branch.A filter
> > > >>>>
> > > >>>> can be used to scope the uncrustify operations to only the C/H files
> > > >>>>
> > > >>>> in the specific package the patch series is against. (e.g. DynamicTablesPkg).
> > > >>>>
> > > >>>> BaseTools should always be excluded.If the package scoped filter is
> > > >>>>
> > > >>>> not used, it will still work, but will take longer to run because
> > > >>>>
> > > >>>> uncrustify will rescan every C/H files in the whole repo.
> > > >>>>
> > > >>>> git filter-branch --tree-filter "git ls-files DynamicTablesPkg*.c DynamicTablesPkg*.h :!BaseTools/* |
> > > >>> c:\\Temp\\UncrustifyCheck\\mu-uncrustify-release_extdep\\Windows-x86\\uncrustify.exe -c
> > > >>> c:\\Temp\\UncrustifyCheck\\uncrustify.cfg -F - --replace --no-backup --if-changed" edk2-stable202111..HEAD
> > > >>>>
> > > >>>> 7) Now that all the individual patches in the branch are uncrustified,
> > > >>>>
> > > >>>> rebase against latest edk2/master that is already uncrustified.
> > > >>>>
> > > >>>> git rebase master
> > > >>>>
> > > >>>> 8) Verify the patches in this new branch.
> > > >>>>
> > > >>>> Impacts to tracing history across the uncrusity changes
> > > >>>>
> > > >>>> =======================================================
> > > >>>>
> > > >>>> Tools the view file and line history do work with the large uncrustify
> > > >>>>
> > > >>>> patch series.One impact is that the operations can be very slow due
> > > >>>>
> > > >>>> to the large uncrustify patches.
> > > >>>>
> > > >>>> One option to provide a faster experience is to provide an alternate
> > > >>>>
> > > >>>> version of the edk2 repository as "documentation" that has the
> > > >>>>
> > > >>>> entire history re-written with uncrustify run on every commit.
> > > >>>>
> > > >>>> The tool called git-filter-repo can be used to perform this
> > > >>>>
> > > >>>> transformation and runs in a reasonable period of time (a few hours)
> > > >>>>
> > > >>>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnewren%2Fgit-filter-repo&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=yu4XP75X6Xb84xIAIbxCR%2B4x5r0vdaCO1CKCNSg8ShU%3D&reserved=0 <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnewren%2Fgit-filter-repo&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=yu4XP75X6Xb84xIAIbxCR%2B4x5r0vdaCO1CKCNSg8ShU%3D&reserved=0>
> > > >>>>
> > > >>>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnewren%2Fgit-filter-repo%2Fblob%2Fmain%2Fcontrib%2Ffilter-repo-demos%2Flint-history&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=5Tlum9FHkxfL83%2F6TY8NGkqpYx7Hg6YE96TdIYBGy0U%3D&reserved=0
> > > >>> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnewren%2Fgit-filter-repo%2Fblob%2Fmain%2Fcontrib%2Ffilter-repo-demos%2Flint-history&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=5Tlum9FHkxfL83%2F6TY8NGkqpYx7Hg6YE96TdIYBGy0U%3D&reserved=0>
> > > >>>>
> > > >>>> The following steps can be used to perform this transformation.
> > > >>>>
> > > >>>> This is the Windows version. I will add the Linux version soon.
> > > >>>>
> > > >>>> ** WARNING **This operation modifies(rewrites) all the commits
> > > >>>>
> > > >>>> in the local copy of the repo.Do not perform
> > > >>>>
> > > >>>> these steps on a local repo you are using for
> > > >>>>
> > > >>>> active development.
> > > >>>>
> > > >>>> 1) Clone edk2 into a new directory (see **WARNING**)
> > > >>>>
> > > >>>> git clonehttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2.git&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Cnn64ezsb77fHWu%2FRR4sPbuNDb4WkPbv75vXJFOf4bc%3D&reserved=0 <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2.git&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Cnn64ezsb77fHWu%2FRR4sPbuNDb4WkPbv75vXJFOf4bc%3D&reserved=0> edk2-uncrustified
> > > >>>>
> > > >>>> cd edk2-uncrustified
> > > >>>>
> > > >>>> 2) Setup python virtual env, install pytools, and run stuart commands
> > > >>>>
> > > >>>> to setup build environment which includes installing uncrustify tools.
> > > >>>>
> > > >>>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2F.pytool%23running-ci-locally&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=RaZxkrLB%2FFH6leSzUDxJWwSIF6phK8fbPyM1%2FyjLT1g%3D&reserved=0
> > > >>> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Ftree%2Fmaster%2F.pytool%23running-ci-locally&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=RaZxkrLB%2FFH6leSzUDxJWwSIF6phK8fbPyM1%2FyjLT1g%3D&reserved=0>
> > > >>>>
> > > >>>> 3) Make a backup copy of plugin UncrustifyCheck outside WORKSPACE.
> > > >>>>
> > > >>>> (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable and
> > > >>>>
> > > >>>> EDK II specific uncrustify configuration file available when working
> > > >>>>
> > > >>>> with a branch that does not have those tools in its scope.
> > > >>>>
> > > >>>> xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
> > > >>>>
> > > >>>> 4) Use lint-history.py from git-filter-repo examples
> > > >>>>
> > > >>>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnewren%2Fgit-filter-repo&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=yu4XP75X6Xb84xIAIbxCR%2B4x5r0vdaCO1CKCNSg8ShU%3D&reserved=0 <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnewren%2Fgit-filter-repo&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=yu4XP75X6Xb84xIAIbxCR%2B4x5r0vdaCO1CKCNSg8ShU%3D&reserved=0>
> > > >>>>
> > > >>>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnewren%2Fgit-filter-repo%2Fblob%2Fmain%2Fcontrib%2Ffilter-repo-demos%2Flint-history&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=5Tlum9FHkxfL83%2F6TY8NGkqpYx7Hg6YE96TdIYBGy0U%3D&reserved=0
> > > >>> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnewren%2Fgit-filter-repo%2Fblob%2Fmain%2Fcontrib%2Ffilter-repo-demos%2Flint-history&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=5Tlum9FHkxfL83%2F6TY8NGkqpYx7Hg6YE96TdIYBGy0U%3D&reserved=0>
> > > >>>>
> > > >>>> Line #127 - Add try except around subprocess.check_call() with except
> > > >>>>
> > > >>>> being pass.This is required because there are a few commits of C
> > > >>>>
> > > >>>> files in the edk2 repo that have incorrect C syntax and do not
> > > >>>>
> > > >>>> build with a C compiler and break the uncrustify parser.Skip reformat
> > > >>>>
> > > >>>> of C files that can not be parsed by uncrustify.These rare instances
> > > >>>>
> > > >>>> are addressed in the commit that fixes the C syntax error.
> > > >>>>
> > > >>>> Run this slightly modified version of lint-history.Include only
> > > >>>>
> > > >>>> C/H files and exclude directories that start with 'Tools' or 'BaseTools'.
> > > >>>>
> > > >>>> This step took about 2.2 hours on a laptop.
> > > >>>>
> > > >>>> lint-history.py
> > > >>>>
> > > >>>> --relevant "return (not filename.startswith(b'Tools') and not filename.startswith(b'BaseTools') and
> > > >>> (filename.endswith(b'.c') or filename.endswith(b'.h')))"
> > > >>>>
> > > >>>> c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\mu-uncrustify-release_extdep\\Windows-x86\\uncrustify.exe
> -
> > c
> > > >>> c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\uncrustify.cfg --replace --no-backup --if-changed
> > > >>>>
> > > >>>> Order of PRs to apply during extended hard freeze
> > > >>>>
> > > >>>> ==================================================
> > > >>>>
> > > >>>> 1) Update EmulatorPkg Win Host [BuildOptions] MSFT CC_FLAGS to not force debug information
> > > >>>>
> > > >>>> *https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.tianocore.org%2Fshow_bug.cgi%3Fid%3D3747&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=X1qJcou7a0aUfw%2FKblXQTw%2FSgSD7sRuUlyuSPUHwRaM%3D&reserved=0 <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.tianocore.org%2Fshow_bug.cgi%3Fid%3D3747&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=X1qJcou7a0aUfw%2FKblXQTw%2FSgSD7sRuUlyuSPUHwRaM%3D&reserved=0>
> > > >>>>
> > > >>>> *https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Ftree%2FBug_3747_EmulatorPkg_WinHost_ReproducibleBuild&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Bl%2BnAhb%2BnT870EY1lNWkXO4HvQlLV2XglB70bEpeaLo%3D&reserved=0
> > > >>> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Ftree%2FBug_3747_EmulatorPkg_WinHost_ReproducibleBuild&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Bl%2BnAhb%2BnT870EY1lNWkXO4HvQlLV2XglB70bEpeaLo%3D&reserved=0>
> > > >>>>
> > > >>>> *https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fpull%2F2215&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Vf5DwaLDALYwWg2GOMjYQB0ButPMDkjH%2FkT8f2Ly7Uc%3D&reserved=0 <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fpull%2F2215&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Vf5DwaLDALYwWg2GOMjYQB0ButPMDkjH%2FkT8f2Ly7Uc%3D&reserved=0>
> > > >>>>
> > > >>>> * Required for EmulatorPkg to pass CompareBuild for VS2019 IA32/X64 builds.
> > > >>>>
> > > >>>> * Status: Review complete.PR pushed.
> > > >>>>
> > > >>>> 2) EccCheck should not revert staged and local changes
> > > >>>>
> > > >>>> *https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.tianocore.org%2Fshow_bug.cgi%3Fid%3D2986&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=zmosiV3fCJnTPSbPNgoZrSUq7k56CDgALpInKU%2FwhHU%3D&reserved=0 <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.tianocore.org%2Fshow_bug.cgi%3Fid%3D2986&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=zmosiV3fCJnTPSbPNgoZrSUq7k56CDgALpInKU%2FwhHU%3D&reserved=0>
> > > >>>>
> > > >>>> *https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Ftree%2FBug_2986_EccCheckRemoveGitRevert_V2&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=az3bUmXNkoMeDx%2FSmBRR2nStYGkt9Tpz9IaXih98WQU%3D&reserved=0
> > > >>> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Ftree%2FBug_2986_EccCheckRemoveGitRevert_V2&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=az3bUmXNkoMeDx%2FSmBRR2nStYGkt9Tpz9IaXih98WQU%3D&reserved=0>
> > > >>>>
> > > >>>> *https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fpull%2F2216&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=yjDqVLhGUO87muogbV%2F0yCsVJYk9WWtbyhhkHhjRZk8%3D&reserved=0 <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fpull%2F2216&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=yjDqVLhGUO87muogbV%2F0yCsVJYk9WWtbyhhkHhjRZk8%3D&reserved=0>
> > > >>>>
> > > >>>> * Required for EDK II CI to complete in a reasonable period of time when
> > > >>>>
> > > >>>> processing the 4000+ source file style changes made by uncrustify.
> > > >>>>
> > > >>>> * Also fixes critical bugs that can potentially corrupt git state when
> > > >>>>
> > > >>>> EccCheck is run locally.
> > > >>>>
> > > >>>> * Status: Review complete.PR pushed.
> > > >>>>
> > > >>>> 3) Update pytool LicenseCheck plugin to use temp directory for diff output file
> > > >>>>
> > > >>>> *https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.tianocore.org%2Fshow_bug.cgi%3Fid%3D3746&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=YVC65q61WSdDl5LTMz%2F%2FdjYSEU0AdLXTj0EIaQ6GlCg%3D&reserved=0 <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.tianocore.org%2Fshow_bug.cgi%3Fid%3D3746&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=YVC65q61WSdDl5LTMz%2F%2FdjYSEU0AdLXTj0EIaQ6GlCg%3D&reserved=0>
> > > >>>>
> > > >>>> *https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Ftree%2FBug_3746_LicenseCheckUseDiffOutputFile_V2&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=ezbmHR%2F2TrU%2FkK4rZPKCw6Q4g5WI62U2NXqjlzvZ58s%3D&reserved=0
> > > >>> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Ftree%2FBug_3746_LicenseCheckUseDiffOutputFile_V2&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846900446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=ezbmHR%2F2TrU%2FkK4rZPKCw6Q4g5WI62U2NXqjlzvZ58s%3D&reserved=0>
> > > >>>>
> > > >>>> *https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fpull%2F2217&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846950023%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=nyfn6S0DcLvCzCTS2pM5inenIdwr9WjZ9LjFoM87yRQ%3D&reserved=0 <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fpull%2F2217&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846950023%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=nyfn6S0DcLvCzCTS2pM5inenIdwr9WjZ9LjFoM87yRQ%3D&reserved=0>
> > > >>>>
> > > >>>> * Required to reduce EDK II CI build times.
> > > >>>>
> > > >>>> * Status: Review complete.PR pushed.
> > > >>>>
> > > >>>> 4) Update max job time from 60 min to 120 minutes in .azurepipelines/templates
> > > >>>>
> > > >>>> *https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.tianocore.org%2Fshow_bug.cgi%3Fid%3D3750&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846950023%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=iiRirOEeR2cFaMo9PXtR7P5e2SDyXScNPM%2F4m%2Fe5hSc%3D&reserved=0 <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.tianocore.org%2Fshow_bug.cgi%3Fid%3D3750&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846950023%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=iiRirOEeR2cFaMo9PXtR7P5e2SDyXScNPM%2F4m%2Fe5hSc%3D&reserved=0>
> > > >>>>
> > > >>>> *https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Ftree%2FBug_3750_IncreaseAzurePipelinesTimeout&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846950023%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=ZZtHjt%2BAqmtCpjCcWqE1hgpBlB5V9LWNWMMCwunjMME%3D&reserved=0
> > > >>> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Ftree%2FBug_3750_IncreaseAzurePipelinesTimeout&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846950023%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=ZZtHjt%2BAqmtCpjCcWqE1hgpBlB5V9LWNWMMCwunjMME%3D&reserved=0>
> > > >>>>
> > > >>>> *https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fpull%2F2219&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846950023%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=qFBdTXm0Ra2KErfExHy%2F7HOMKQHWA5uXPaZVg7mo69w%3D&reserved=0 <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fpull%2F2219&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846950023%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=qFBdTXm0Ra2KErfExHy%2F7HOMKQHWA5uXPaZVg7mo69w%3D&reserved=0>
> > > >>>>
> > > >>>> * Required to allow EccCheck of uncrustify changes to complete on Azure
> > > >>>>
> > > >>>> Pipelines CI agents without timing out.
> > > >>>>
> > > >>>> * Status: Review complete.PR pushed.
> > > >>>>
> > > >>>> 5) Update Package YAML to ignore specific ECC files/errors
> > > >>>>
> > > >>>> *https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.tianocore.org%2Fshow_bug.cgi%3Fid%3D3749&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846950023%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=ArcC69Yf2ui%2B44INRbhPCtawEWZByAeLMR0mq8J7Zgo%3D&reserved=0 <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.tianocore.org%2Fshow_bug.cgi%3Fid%3D3749&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846950023%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=ArcC69Yf2ui%2B44INRbhPCtawEWZByAeLMR0mq8J7Zgo%3D&reserved=0>
> > > >>>>
> > > >>>> *https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Ftree%2FBug_3749_EccCheckIgnoreFilesErrors&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846950023%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=kzclVkC9M6%2BmLCFpR8j%2FXxiC%2BeCJnz0lppLr4KS10PI%3D&reserved=0
> > > >>> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Ftree%2FBug_3749_EccCheckIgnoreFilesErrors&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846950023%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=kzclVkC9M6%2BmLCFpR8j%2FXxiC%2BeCJnz0lppLr4KS10PI%3D&reserved=0>
> > > >>>>
> > > >>>> *https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fpull%2F2218&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846950023%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=jipiCJ%2FY8vOoS3ZKis23tW666wHbc7btiIxWajn5e88%3D&reserved=0 <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fpull%2F2218&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846950023%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=jipiCJ%2FY8vOoS3ZKis23tW666wHbc7btiIxWajn5e88%3D&reserved=0>
> > > >>>>
> > > >>>> * Required to pass EccCheck
> > > >>>>
> > > >>>> * Status: Review complete. PR pushed
> > > >>>>
> > > >>>> 6) Uncrustify Source Changes
> > > >>>>
> > > >>>> *https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.tianocore.org%2Fshow_bug.cgi%3Fid%3D3737&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846950023%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=fVmyMK86f6P33MJI8zT1BHjAGL%2FMz57DIn7MibuEuPQ%3D&reserved=0 <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.tianocore.org%2Fshow_bug.cgi%3Fid%3D3737&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846950023%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=fVmyMK86f6P33MJI8zT1BHjAGL%2FMz57DIn7MibuEuPQ%3D&reserved=0>
> > > >>>>
> > > >>>> *https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.tianocore.org%2Fshow_bug.cgi%3Fid%3D3739&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846950023%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Iuw6pRgAnzSDxdc7TcRXDHCSALDtsff8fB%2BbU7tRMC4%3D&reserved=0 <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.tianocore.org%2Fshow_bug.cgi%3Fid%3D3739&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846950023%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Iuw6pRgAnzSDxdc7TcRXDHCSALDtsff8fB%2BbU7tRMC4%3D&reserved=0>
> > > >>>>
> > > >>>> *https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Ftree%2FBug_3737_3739_ApplyUncrustifyChanges_V5&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846950023%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=wo0bG8v8FzilmFMRT6VIbhPfzaqMYBWymqW0UFIxVnU%3D&reserved=0
> > > >>> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Ftree%2FBug_3737_3739_ApplyUncrustifyChanges_V5&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846950023%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=wo0bG8v8FzilmFMRT6VIbhPfzaqMYBWymqW0UFIxVnU%3D&reserved=0>
> > > >>>>
> > > >>>> *https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fpull%2F2229&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846950023%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=6MxSHE5gDG821vOioUqEcOMTsZKgf8ZD9k9xvliLNYQ%3D&reserved=0 <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fpull%2F2229&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846950023%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=6MxSHE5gDG821vOioUqEcOMTsZKgf8ZD9k9xvliLNYQ%3D&reserved=0>
> > > >>>>
> > > >>>> * Build comparison result PASS:https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Factions%2Fruns%2F1521618836&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846950023%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=ZBXI03u6MigKHPiUu9u30LnH3DrU6%2B0uYySdpNZAi8A%3D&reserved=0
> > > >>> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Factions%2Fruns%2F1521618836&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846950023%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=ZBXI03u6MigKHPiUu9u30LnH3DrU6%2B0uYySdpNZAi8A%3D&reserved=0>
> > > >>>>
> > > >>>> * EFI_D_ -> DEBUG changes required to pass PatchCheck
> > > >>>>
> > > >>>> * Uncrustify format changes required to pass UncrustifyCheck
> > > >>>>
> > > >>>> * Status:
> > > >>>>
> > > >>>> Waiting for review
> > > >>>>
> > > >>>> 7) UncrustifyCheck EDK II CI Plugin
> > > >>>>
> > > >>>> *https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.tianocore.org%2Fshow_bug.cgi%3Fid%3D3748&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846950023%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=OdqUuXlQdkqSargQzagNzCpMbzJjUDOi7AAy6gn8rE4%3D&reserved=0 <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.tianocore.org%2Fshow_bug.cgi%3Fid%3D3748&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846950023%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=OdqUuXlQdkqSargQzagNzCpMbzJjUDOi7AAy6gn8rE4%3D&reserved=0>
> > > >>>>
> > > >>>> *https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Ftree%2FBug_3748_add_uncrustify_ci_plugin_v6&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846950023%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=9qajrPed4l4R7MhfKg1EKGF0KzulLsoRIfBQYAvwJQU%3D&reserved=0
> > > >>> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Ftree%2FBug_3748_add_uncrustify_ci_plugin_v6&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846950023%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=9qajrPed4l4R7MhfKg1EKGF0KzulLsoRIfBQYAvwJQU%3D&reserved=0>
> > > >>>>
> > > >>>> * Required to enforce all PRs submitted to EDK II CI match uncrustify format.
> > > >>>>
> > > >>>> * Unconditionally checks all packages.Can not be committed until all C/H
> > > >>>>
> > > >>>> source files have been updated.
> > > >>>>
> > > >>>> * Status: Review complete
> > > >>>>
> > > >>>> Combined Branch/PR for Review/Test
> > > >>>>
> > > >>>> ==================================
> > > >>>>
> > > >>>> * Build Comparison results must pass 100% across the full set of PRs before
> > > >>>>
> > > >>>> the individual PRs can be pushed in the order listed above.
> > > >>>>
> > > >>>> * Branch:https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Ftree%2FTestOnly_Uncrustify_PR_Series&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846950023%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=wVD2UbHrSEYsB%2FufOR3255oj%2BcfztzynQSkpGNw0N7o%3D&reserved=0
> > > >>> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Ftree%2FTestOnly_Uncrustify_PR_Series&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846950023%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=wVD2UbHrSEYsB%2FufOR3255oj%2BcfztzynQSkpGNw0N7o%3D&reserved=0>
> > > >>>>
> > > >>>> * PR:https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fpull%2F2229&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846950023%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=6MxSHE5gDG821vOioUqEcOMTsZKgf8ZD9k9xvliLNYQ%3D&reserved=0 <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fpull%2F2229&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846950023%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=6MxSHE5gDG821vOioUqEcOMTsZKgf8ZD9k9xvliLNYQ%3D&reserved=0>
> > > >>>>
> > > >>>> Status = PASS
> > > >>>>
> > > >>>> * CompareBuild:
> > > >>>>
> > > >>>> Branch:https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Ftree%2FBug_3737_3739_ApplyUncrustifyChanges_V5&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846950023%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=wo0bG8v8FzilmFMRT6VIbhPfzaqMYBWymqW0UFIxVnU%3D&reserved=0
> > > >>> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Ftree%2FBug_3737_3739_ApplyUncrustifyChanges_V5&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846950023%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=wo0bG8v8FzilmFMRT6VIbhPfzaqMYBWymqW0UFIxVnU%3D&reserved=0>
> > > >>>>
> > > >>>> --ref1:ef9a059cdb15844fe52a49af2bf7d86b9dd3e9bf
> > > >>>>
> > > >>>> --ref2:Bug_3737_3739_ApplyUncrustifyChanges_V5
> > > >>>>
> > > >>>> Extra Options: -n 4 --quiet
> > > >>>>
> > > >>>> Results:https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Factions%2Fruns%2F1521618836&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846950023%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=ZBXI03u6MigKHPiUu9u30LnH3DrU6%2B0uYySdpNZAi8A%3D&reserved=0
> > > >>> <https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmdkinney%2Fedk2%2Factions%2Fruns%2F1521618836&data=04%7C01%7CBret.Barkelew%40microsoft.com%7C3b84c1ee3b344b6668d508d9b8563e4a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637743502846950023%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=ZBXI03u6MigKHPiUu9u30LnH3DrU6%2B0uYySdpNZAi8A%3D&reserved=0>
> > > >>>>
> > > >>>> 30 VS2019 build comparisons PASS
> > > >>>>
> > > >>>> 40 GCC5 build comparisons PASS
> > > >>>>
> > > >>>> 100% PASS
> > > >>>>
> > > >>>> The following git log shows the set of patches from --ref1 to --ref 2across
> > > >>>>
> > > >>>> which there are no differences in any of the OBJ/LIB/DLL/EFI/FFS/FV/FD files.
> > > >>>>
> > > >>>> --ref2
> > > >>>>
> > > >>>> b7d4bf0675b7 (HEAD -> Bug_3737_3739_ApplyUncrustifyChanges_V5) UnitTestFrameworkPkg: Apply uncrusitify
> changes
> > > >>>>
> > > >>>> 7f03d25f60e7 UefiPayloadPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> 0bfd8d9b5ac9 UefiCpuPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> e1cd9bfb9dea StandaloneMmPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> 5da2f65be378 SourceLevelDebugPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> 95b86de07e5d SignedCapsulePkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> fe71d97246c4 ShellPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> 54c21c952992 SecurityPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> 187a3785f12b RedfishPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> 810100002a46 PcAtChipsetPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> 276a695c0cf2 OvmfPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> 303c0a91ab07 NetworkPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> bc80792cd1b1 MdePkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> 3ea86be17a2a MdeModulePkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> c70ef11ed0cd IntelFsp2WrapperPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> c0291221f252 IntelFsp2Pkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> 6a479952a690 FmpDevicePkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> 3a7c05b7070d FatPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> b789f98c8959 EmulatorPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> 952d7a1c9220 EmbeddedPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> a1cc9881bab6 DynamicTablesPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> 50654dfe5785 CryptoPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> ed965a02dfa1 ArmVirtPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> 9744023fbc46 ArmPlatformPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> 7a1cde5f5bba ArmPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> 19d17e0913e8 UefiCpuPkg: Change use of EFI_D_* to DEBUG_*
> > > >>>>
> > > >>>> ffa718b4f994 SourceLevelDebugPkg: Change use of EFI_D_* to DEBUG_*
> > > >>>>
> > > >>>> b86cb3c5e5b4 ShellPkg: Change use of EFI_D_* to DEBUG_*
> > > >>>>
> > > >>>> c7c42204dc07 SecurityPkg: Change use of EFI_D_* to DEBUG_*
> > > >>>>
> > > >>>> 16b8e6f958e4 PcAtChipsetPkg: Change use of EFI_D_* to DEBUG_*
> > > >>>>
> > > >>>> 0ac3f8b2dac5 OvmfPkg: Change use of EFI_D_* to DEBUG_*
> > > >>>>
> > > >>>> bc5004b8d294 NetworkPkg: Change use of EFI_D_* to DEBUG_*
> > > >>>>
> > > >>>> 6f671a8e2377 MdePkg: Change use of EFI_D_* to DEBUG_*
> > > >>>>
> > > >>>> a10c610ff9a3 MdeModulePkg: Change use of EFI_D_* to DEBUG_*
> > > >>>>
> > > >>>> 09a3bddba390 FatPkg: Change use of EFI_D_* to DEBUG_*
> > > >>>>
> > > >>>> 59c61318246a EmulatorPkg: Change use of EFI_D_* to DEBUG_*
> > > >>>>
> > > >>>> 3a80367dda3b EmbeddedPkg: Change use of EFI_D_* to DEBUG_*
> > > >>>>
> > > >>>> 23eb1aaf80ca ArmVirtPkg: Change use of EFI_D_* to DEBUG_*
> > > >>>>
> > > >>>> 875914b45c54 ArmPlatformPkg: Change use of EFI_D_* to DEBUG_*
> > > >>>>
> > > >>>> eb2eca82b451 ArmPkg: Change use of EFI_D_* to DEBUG_*
> > > >>>>
> > > >>>> f0f3f5aae7c4 (origin/master, origin/HEAD, master) UnitTestFrameworkPkg: Update YAML to ignore specific ECC
> > > >>> files/errors
> > > >>>>
> > > >>>> c05734797790 UefiPayloadPkg: Update YAML to ignore specific ECC files/errors
> > > >>>>
> > > >>>> c30c40d6c63d StandaloneMmPkg: Update YAML to ignore specific ECC files/errors
> > > >>>>
> > > >>>> 9944508e85f1 ShellPkg: Update YAML to ignore specific ECC files/errors
> > > >>>>
> > > >>>> 60fa40be458d SecurityPkg: Update YAML to ignore specific ECC files/errors
> > > >>>>
> > > >>>> df790cd6b37e MdePkg: Update YAML to ignore specific ECC files/errors
> > > >>>>
> > > >>>> 9deb9370766e MdeModulePkg: Update YAML to ignore specific ECC files/errors
> > > >>>>
> > > >>>> d7d30e8f219f EmulatorPkg: Update YAML to ignore specific ECC files/errors
> > > >>>>
> > > >>>> d5744ecba813 CryptoPkg: Update YAML to ignore specific ECC files/errors
> > > >>>>
> > > >>>> c97fee87f0f9 ArmVirtPkg: Update YAML to ignore specific ECC files/errors
> > > >>>>
> > > >>>> 1939fc9569f2 ArmPlatformPkg: Update YAML to ignore specific ECC files/errors
> > > >>>>
> > > >>>> 365dced2c37a ArmPkg: Update YAML to ignore specific ECC files/errors
> > > >>>>
> > > >>>> 76a1ce4d5fec .azurepipelines/templates: Update max pipeline job time to 2 hours
> > > >>>>
> > > >>>> 99f84ff47390 .pytools/Plugin/LicenseCheck: Use temp directory for git diff output
> > > >>>>
> > > >>>> 3019f1bbabf1 .pytool/Plugin/EccCheck: Add performance optimizations
> > > >>>>
> > > >>>> 854462bd3479 .pytool/Plugin/EccCheck: Remove temp directory on exception
> > > >>>>
> > > >>>> 69877614fdee .pytool/Plugin/EccCheck: Remove RevertCode()
> > > >>>>
> > > >>>> --ref1
> > > >>>>
> > > >>>> ef9a059cdb15 EmulatorPkg/Win/Host: Update CC_FLAGS
> > > >>>>
> > > >>>> bb1bba3d7767 (tag: edk2-stable202111) NetworkPkg: Fix invalid pointer for DNS response token on error
> > > >>>>
> > > >>>> Best regards,
> > > >>>>
> > > >>>> Mike
> > > >>>>
> > > >>>>
[-- Attachment #2: Type: text/html, Size: 161441 bytes --]
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
2021-12-06 1:17 ` Michael D Kinney
` (2 preceding siblings ...)
2021-12-06 19:45 ` Bret Barkelew
@ 2021-12-06 20:28 ` Maciej Rabeda
2021-12-07 0:26 ` Abner Chang
` (3 subsequent siblings)
7 siblings, 0 replies; 39+ messages in thread
From: Maciej Rabeda @ 2021-12-06 20:28 UTC (permalink / raw)
To: devel, michael.d.kinney, Michael Kubacki, Michael Kubacki,
Andrew Fish (afish@apple.com), Leif Lindholm
Reviewed-by: Maciej Rabeda <maciej.rabeda@linux.intel.com>
On 06-Dec-21 02:17, Michael D Kinney wrote:
> Hello EDK II Maintainers,
>
> A detailed evaluation of the DEBUG_CODE() formatting issue has been completed.
> The reason DEBUG_CODE() is a challenge is that this looks like a macro from a
> C parsing perspective, but the EDK II usage places C statements or blocks of
> C code as the parameter to this macro.
>
> There are actually 2 methods available to mark a statement or a block of code
> to be included when the DEBUG_PROPERTY_DEBUG_CODE_ENABLED bit is enabled in
> PcdDebugPropertyMask. One is DEBUG_CODE() and the other is to mark the beginning
> and end of a code block with DEBUG_CODE_BEGIN() and DEBUG_CODE_END(). In fact,
> DEBUG_CODE() is implemented using DEBUG_CODE_BEGIN() and DEBUG_CODE_END() macros.
>
> #define DEBUG_CODE(Expression) \
> DEBUG_CODE_BEGIN (); \
> Expression \
> DEBUG_CODE_END ()
>
> A complete review for the use of these DEBUG_CODE macros was performed on the
> edk2 repo. Uncrustify performs good formatting for code blocks between
> DEBUG_CODE_BEGIN() and DEBUG_CODE_END(). This is because these look like simple
> macros calls with no parameters and the lines of C code between these 2 macros
> is formatted correctly.
>
> The uncrustify formatting issues are only present with the use of DEBUG_CODE().
> Simple use cases of DEBUG_CODE(Expression) where Expression is a single C
> statement also look correct. A medium complexity use case where Expression is
> a code block of simple statements or even some local variables and simple
> statements also look correct. It is only complex code blocks that use C
> statements such as if/for/while/case that include the use of braces {} does
> uncrustify perform incorrect formatting.
>
> The recommended solution to this issue is to convert the use of DEBUG_CODE()
> to DEBUG_CODE_BEGIN() / DEBUG_CODE_END() for cases where the Expression
> passed to DEBUG_CODE() is the complex use case that contains statements that
> use braces {}. There are 57 instances of this pattern across 40 files in the
> edk2 repo.
>
> I have posted a branch with these additional patches:
>
> https://github.com/mdkinney/edk2/tree/Bug_3737_3760_3739_ApplyUncrustifyChanges_V7
>
> I have performed CompareBuild tests with this revised patch series with
> the DEBUG_CODE changes. It passes 100% showing no binary differences.
>
> https://github.com/mdkinney/edk2/actions/runs/1542454606
>
> I have opened a PR to run this patch series through EDK II CI. It also passes 100%.
>
> https://github.com/tianocore/edk2/pull/2236
>
> The summary of changes made since the V6 review are:
>
> 1) Change uncrustify configuration assignment alignment threshold to 0
>
> align_assign_thresh = 0
>
> 2) Replace "<param>, OPTIONAL" with "<param> OPTIONAL,"
>
> 3) Replace DEBUG_CODE(Expression) with
>
> DEBUG_CODE_BEGIN();
> Expression
> DEBUG_CODE_END()
>
> if Expression is complex and contains braces {}.
>
> 4) No changes to uncrustify tool required.
>
> Please review the differences between the following 2 branches and provide
> feedback or a Series Reviewed-by if you agree with these additional changes.
>
> https://github.com/mdkinney/edk2/tree/Bug_3737_3760_3739_ApplyUncrustifyChanges_V6
> https://github.com/mdkinney/edk2/tree/Bug_3737_3760_3739_ApplyUncrustifyChanges_V7
>
> The goal is to complete the review and get the uncrustify change committed
> tomorrow so the extended hard freeze can be lifted.
>
> Thanks,
>
> Mike
>
>> -----Original Message-----
>> From: Kinney, Michael D <michael.d.kinney@intel.com>
>> Sent: Thursday, December 2, 2021 6:23 PM
>> To: Michael Kubacki <mikuback@linux.microsoft.com>; devel@edk2.groups.io; maciej.rabeda@linux.intel.com; Michael Kubacki
>> <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com) <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>;
>> Kinney, Michael D <michael.d.kinney@intel.com>
>> Subject: RE: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
>>
>> Hello EDK II Maintainers,
>>
>> I have entered BZ 3760 to make the use of the OPTIONAL keyword style consistent for all of edk2 repo
>> and to be compatible with uncrustify.
>>
>> I have posted the following V6 branch that does the EFI_D_* to DEBUG_* changes, the OPTIONAL keyword
>> style changes, and the uncrustify changes with the one configuration change for assignment alignment.
>>
>> https://github.com/mdkinney/edk2/tree/Bug_3737_3760_3739_ApplyUncrustifyChanges_V6
>>
>> Please provide feedback on the code style in this branch with the known DEBUG_CODE() issue still
>> present.
>>
>> If we are able to quickly update uncrustify to handle DEBUG_CODE(), I will generate a V7.
>>
>> Thanks,
>>
>> Mike
>>
>>> -----Original Message-----
>>> From: Kinney, Michael D <michael.d.kinney@intel.com>
>>> Sent: Thursday, December 2, 2021 4:53 PM
>>> To: Michael Kubacki <mikuback@linux.microsoft.com>; devel@edk2.groups.io; maciej.rabeda@linux.intel.com; Michael Kubacki
>>> <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com) <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>;
>>> Kinney, Michael D <michael.d.kinney@intel.com>
>>> Subject: RE: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
>>>
>>> Michael,
>>>
>>> Yes. Please update the patch series that adds UncrustifyCheck with those changes.
>>>
>>> Thanks,
>>>
>>> Mike
>>>
>>>> -----Original Message-----
>>>> From: Michael Kubacki <mikuback@linux.microsoft.com>
>>>> Sent: Thursday, December 2, 2021 4:31 PM
>>>> To: Kinney, Michael D <michael.d.kinney@intel.com>; devel@edk2.groups.io; maciej.rabeda@linux.intel.com; Michael
>> Kubacki
>>>> <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com) <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>
>>>> Subject: Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
>>>>
>>>> Hi Mike,
>>>>
>>>> Thank you for the detailed analysis and recommendations.
>>>>
>>>> I agree with the recommendations and I will try to have an Uncrustify
>>>> tool update by tomorrow for (4).
>>>>
>>>> That will require an update in uncrustify_ext_dep.yaml to pull in the
>>>> new version. I'm assuming I should also update uncrustify.cfg to set
>>>> align_assign_thresh = 0 in that new patch series?
>>>>
>>>> Regards,
>>>> Michael
>>>>
>>>> On 12/2/2021 7:18 PM, Kinney, Michael D wrote:
>>>>> Hi Michael,
>>>>>
>>>>> CORRECTION: set align_assign_threshold to 0.
>>>>>
>>>>> Reponses inline below.
>>>>>
>>>>> I would like to summarize the 4 issues raised in the past day along with the recommendations.
>>>>>
>>>>> 1) Exclusion feature for UncrustifyCheck. There are 2 directories with 8 files total that
>>>>> maintainers have noted they would like to see not go through uncrustify formatting. Today
>>>>> the only content that is skipped is BaseTools and submodules.
>>>>>
>>>>> Adding a general purpose exclusion feature would then require all developers to make
>>>>> sure their method of using uncrustify also excludes those same areas. This requires
>>>>> extra steps for all developers and maintainers.
>>>>>
>>>>> If we do not add the exclusion feature, then the 8 files will require an extra step
>>>>> to sync with the original source of those files. The rate of changes of these 8 files
>>>>> is very low today.
>>>>>
>>>>> RECOMMENDATION: Do not add exclusion feature at this time. Revisit if the extra work
>>>>> to maintain the files that would be candidates for exclusions increases significantly.
>>>>>
>>>>> 2) Alignment of assignments. The threshold of 4 characters appears to be too low and causes
>>>>> source files that are already aligned to become unaligned.
>>>>>
>>>>> RECOMMENDATION: Change threshold to the default value of 0 which means no limit.
>>>>>
>>>>> align_assign_thresh= 0
>>>>>
>>>>> 3) Alignment of parameters in function declaration not correct. The root cause of this
>>>>> is the use of the OPTIONAL keyword. If the OPTIONAL keyword is removed, then the
>>>>> alignment is correct. The alignment is also correct if the OPTIONAL keyword appears
>>>>> before the ','. If the OPTIONAL keyword appears after the ',', then the format is
>>>>> not correct. The OPTIONAL keyword indicates that the parameter in the function is
>>>>> not required and may be passed in as NULL or 0 or some other default value defined by
>>>>> the API. It makes more sense for this OPTIONAL keyword that follows the parameter
>>>>> names to appear before the ',' so it is scoped to the parameter on that line. If it
>>>>> appears after the ',', then C parsers thinks it is a prefix (IN, OUT, CONST, volatile,
>>>>> static) for the next parameter in the function.
>>>>>
>>>>> RECOMMENDATION: Update patch series with a global search and replace so OPTIONAL
>>>>> keyword always appears before the ',' on the same line.
>>>>>
>>>>> RegEx search string: ',( *)OPTIONAL( *)'
>>>>> RegEx replace string: ' $1OPTIONAL,$2'
>>>>>
>>>>> 4) Format issues with complex blocks in DEBUG_CODE(), or between DEBUG_CODE_BEGIN() and
>>>>> DEBUG_CODE_END(). Uncrustify treats these as macros and is not aware that the
>>>>> parameter passed into the macro call is a block of C code that needs to be formatted.
>>>>> Complex blocks with if/while/for/case statements are impacted the most.
>>>>>
>>>>> RECOMMENDATION: Update the uncrustify with an edk2 specific extension to treat these
>>>>> macros as a block of code as if they were surrounded by an extra set of braces {}.
>>>>>
>>>>>
>>>>> I have posted a branch for testing purposes that implements (2) and (3).
>>>>>
>>>>> Branch: https://github.com/mdkinney/edk2/tree/TestOnly_Bug_3737_3739_ApplyUncrustifyChanges_V6_OPTIONAL_Keyword_Fix
>>>>> PR: https://github.com/tianocore/edk2/pull/2233
>>>>> Status: PASS
>>>>> CompareBuild: https://github.com/mdkinney/edk2/actions/runs/1532855914
>>>>> Status: PASS
>>>>>
>>>>> You can see what changed by fetching and comparing the following 2 branches:
>>>>>
>>>>> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
>>>>> https://github.com/mdkinney/edk2/tree/TestOnly_Bug_3737_3739_ApplyUncrustifyChanges_V6_OPTIONAL_Keyword_Fix
>>>>>
>>>>> Please provide feedback on the RECOMMENDATIONS above. I will go ahead and prepare of V6 version of
>>>>> the patch series now that that test results are all PASS.
>>>>>
>>>>> Best regards,
>>>>>
>>>>> Mike
>>>>>
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Kinney, Michael D <michael.d.kinney@intel.com>
>>>>>> Sent: Thursday, December 2, 2021 4:15 PM
>>>>>> To: Michael Kubacki <mikuback@linux.microsoft.com>; devel@edk2.groups.io; maciej.rabeda@linux.intel.com; Michael
>>>> Kubacki
>>>>>> <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com) <afish@apple.com>; Leif Lindholm
>> <leif@nuviainc.com>;
>>>>>> Kinney, Michael D <michael.d.kinney@intel.com>
>>>>>> Subject: RE: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
>>>>>>
>>>>>> Hi Michael,
>>>>>>
>>>>>> Reponses inline below.
>>>>>>
>>>>>> I would like to summarize the 4 issues raised in the past day along with the recommendations.
>>>>>>
>>>>>> 1) Exclusion feature for UncrustifyCheck. There are 2 directories with 8 files total that
>>>>>> maintainers have noted they would like to see not go through uncrustify formatting. Today
>>>>>> the only content that is skipped is BaseTools and submodules.
>>>>>>
>>>>>> Adding a general purpose exclusion feature would then require all developers to make
>>>>>> sure their method of using uncrustify also excludes those same areas. This requires
>>>>>> extra steps for all developers and maintainers.
>>>>>>
>>>>>> If we do not add the exclusion feature, then the 8 files will require an extra step
>>>>>> to sync with the original source of those files. The rate of changes of these 8 files
>>>>>> is very low today.
>>>>>>
>>>>>> RECOMMENDATION: Do not add exclusion feature at this time. Revisit if the extra work
>>>>>> to maintain the files that would be candidates for exclusions increases significantly.
>>>>>>
>>>>>> 2) Alignment of assignments. The threshold of 4 characters appears to be too low and causes
>>>>>> source files that are already aligned to become unaligned.
>>>>>>
>>>>>> RECOMMENDATION: Change threshold to the default value of 0 which means no limit.
>>>>>>
>>>>>> align_assign_thresh= 4
>>>>>>
>>>>>> 3) Alignment of parameters in function declaration not correct. The root cause of this
>>>>>> is the use of the OPTIONAL keyword. If the OPTIONAL keyword is removed, then the
>>>>>> alignment is correct. The alignment is also correct if the OPTIONAL keyword appears
>>>>>> before the ','. If the OPTIONAL keyword appears after the ',', then the format is
>>>>>> not correct. The OPTIONAL keyword indicates that the parameter in the function is
>>>>>> not required and may be passed in as NULL or 0 or some other default value defined by
>>>>>> the API. It makes more sense for this OPTIONAL keyword that follows the parameter
>>>>>> names to appear before the ',' so it is scoped to the parameter on that line. If it
>>>>>> appears after the ',', then C parsers thinks it is a prefix (IN, OUT, CONST, volatile,
>>>>>> static) for the next parameter in the function.
>>>>>>
>>>>>> RECOMMENDATION: Update patch series with a global search and replace so OPTIONAL
>>>>>> keyword always appears before the ',' on the same line.
>>>>>>
>>>>>> RegEx search string: ',( *)OPTIONAL( *)'
>>>>>> RegEx replace string: ' $1OPTIONAL,$2'
>>>>>>
>>>>>> 4) Format issues with complex blocks in DEBUG_CODE(), or between DEBUG_CODE_BEGIN() and
>>>>>> DEBUG_CODE_END(). Uncrustify treats these as macros and is not aware that the
>>>>>> parameter passed into the macro call is a block of C code that needs to be formatted.
>>>>>> Complex blocks with if/while/for/case statements are impacted the most.
>>>>>>
>>>>>> RECOMMENDATION: Update the uncrustify with an edk2 specific extension to treat these
>>>>>> macros as a block of code as if they were surrounded by an extra set of braces {}.
>>>>>>
>>>>>>
>>>>>> I have posted a branch for testing purposes that implements (2) and (3).
>>>>>>
>>>>>> Branch: https://github.com/mdkinney/edk2/tree/TestOnly_Bug_3737_3739_ApplyUncrustifyChanges_V6_OPTIONAL_Keyword_Fix
>>>>>> PR: https://github.com/tianocore/edk2/pull/2233
>>>>>> Status: PASS
>>>>>> CompareBuild: https://github.com/mdkinney/edk2/actions/runs/1532855914
>>>>>> Status: PASS
>>>>>>
>>>>>> You can see what changed by fetching and comparing the following 2 branches:
>>>>>>
>>>>>> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
>>>>>> https://github.com/mdkinney/edk2/tree/TestOnly_Bug_3737_3739_ApplyUncrustifyChanges_V6_OPTIONAL_Keyword_Fix
>>>>>>
>>>>>> Please provide feedback on the RECOMMENDATIONS above. I will go ahead and prepare of V6 version of
>>>>>> the patch series now that that test results are all PASS.
>>>>>>
>>>>>> Best regards,
>>>>>>
>>>>>> Mike
>>>>>>
>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: Michael Kubacki <mikuback@linux.microsoft.com>
>>>>>>> Sent: Thursday, December 2, 2021 1:57 PM
>>>>>>> To: devel@edk2.groups.io; Kinney, Michael D <michael.d.kinney@intel.com>; maciej.rabeda@linux.intel.com; Michael
>>>> Kubacki
>>>>>>> <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com) <afish@apple.com>; Leif Lindholm
>> <leif@nuviainc.com>
>>>>>>> Subject: Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
>>>>>>>
>>>>>>> My reply is inline.
>>>>>>>
>>>>>>> Regards,
>>>>>>> Michael
>>>>>>>
>>>>>>> On 12/2/2021 2:45 PM, Michael D Kinney wrote:
>>>>>>>> Hi Maciej,
>>>>>>>>
>>>>>>>> Thanks for the feedback.
>>>>>>>>
>>>>>>>> * Example #1.This appears to be caused by the following uncrustify
>>>>>>>> settings:
>>>>>>>>
>>>>>>>> # The threshold for aligning on '=' in assignments.
>>>>>>>>
>>>>>>>> # Use a negative number for absolute thresholds.
>>>>>>>>
>>>>>>>> #
>>>>>>>>
>>>>>>>> # 0: No limit (default).
>>>>>>>>
>>>>>>>> align_assign_thresh= 0# number
>>>>>>>>
>>>>>>>> The edk2 setting for this is:
>>>>>>>>
>>>>>>>> align_assign_thresh= 4
>>>>>>>>
>>>>>>>> This means blocks of assignments that are different than more than 4
>>>>>>>> spaces will be considered a new block.
>>>>>>>>
>>>>>>>> ‘HwAddreLen’ and ‘Xid’ are more than 4 characters in length
>>>>>>>> different.Same for ‘Xid’ and ‘Reserved’.So
>>>>>>>>
>>>>>>>> uncrustify treats these as 3 different assignment blocks.
>>>>>>>>
>>>>>>>> If we change to the default value of 0: No limit, this example is
>>>>>>>> treated as a single block and all ‘=’ are aligned.
>>>>>>>>
>>>>>>>> Is there a reason ‘4’ was selected?Is there is any harm in using the
>>>>>>>> default of 0?
>>>>>>>>
>>>>>>> We can certainly change the threshold. '4' was derived by
>>>>>>> experimentation. This is an area that is somewhat subjective and every
>>>>>>> case is difficult to cover well. Please feel free to suggest any changes.
>>>>>>>
>>>>>> I recommend we use the default value of 0. That way, any code that choose to
>>>>>> align assignments will still be aligned. If a developer does not like
>>>>>> short assignments and long assignments in the same code block to use the
>>>>>> long assignment column, they can always break the block up into multiple
>>>>>> blocks by adding a carriage return.
>>>>>>
>>>>>>> For the benefit of others, this file can be a useful refernce
>>>>>>> understanding spans, gaps, and thresholds.
>>>>>>>
>>>>>>> https://github.com/uncrustify/uncrustify/blob/master/documentation/htdocs/configuration.txt
>>>>>>>
>>>>>>>> * Example #2: Uncruistfy is confused by the DEBUG_CODE() macro.This is
>>>>>>>> not a traditional macro because the
>>>>>>>>
>>>>>>>> contents of the macro is a block of C code.I do not know how to convince
>>>>>>>> uncrustify that the contents of a
>>>>>>>>
>>>>>>>> macro like function call to be treated as a code block from an indent
>>>>>>>> perspective.
>>>>>>>>
>>>>>>> I believe this would have to be overridden in the Uncrustify fork since
>>>>>>> DEBUG_CODE() is being treated as a macro function call and code blocks
>>>>>>> are not expected there. In addition, some special treatment might be
>>>>>>> needed for alignment in between DEBUG_CODE_BEGIN()/DEBUG_CODE_END().
>>>>>>>
>>>>>>> I'm happy to look at this. However, regression validation can take a
>>>>>>> while so I'd like to make sure we need this now. In cases that do not
>>>>>>> have additional code blocks, it seems to format fairly well. Is this
>>>>>>> prevalent and impactful enough it must be fixed now? Or, could we
>>>>>>> revisit it with a follow up patch?
>>>>>> How long does regression testing take? I see about 20 instances of
>>>>>> DEBUG_CODE() that are producing bad formatting. If the content inside
>>>>>> DEBUG_CODE() is a single line of a single block of statements without
>>>>>> any if/while/for/case statements that require further indent, then the
>>>>>> format looks ok.
>>>>>>
>>>>>> DEBUG_CODE_BEGIN() and DEBUG_CODE_END() would look better if the
>>>>>> contents between are also indented one level.
>>>>>>
>>>>>>>> * Example #3/#4: Uncrustify is confused by the OPTIONAL keyword.The
>>>>>>>> edk2 config declares it as a QUALIFIER
>>>>>>>>
>>>>>>>> like IN and OUT.However, OPTIONAL only appears at the end of the line
>>>>>>>> that declares a parameter to a
>>>>>>>>
>>>>>>>> function.There are 3 forms. One with comma after OPTIONAL.One with comma
>>>>>>>> before OPTIONAL, and
>>>>>>>>
>>>>>>>> one with no comma if the parameter is the last parameter in the function
>>>>>>>> declaration.
>>>>>>>>
>>>>>>>> TYPE ParamName OPTIONAL,
>>>>>>>>
>>>>>>>> TYPEParamName, OPTIONAL
>>>>>>>>
>>>>>>>> TYPEParamName OPTIONAL
>>>>>>>>
>>>>>>>> OPTIONAL is defined to nothing in edk2 builds.From a uncrustify
>>>>>>>> perspective, we really want is to be
>>>>>>>>
>>>>>>>> ignored or more correctly treated it as a token that is attached to
>>>>>>>> ParamName and the combination of
>>>>>>>>
>>>>>>>> ParamName and OPTIONAL treated as one unit to determine indents.
>>>>>>>>
>>>>>>> "TYPE ParamName, OPTIONAL" seems especially problematic and presents an
>>>>>>> inconsistency with the other formats. Mike, can you please let me know
>>>>>>> if you have the same observation and your thoughts on a consistent pattern?
>>>>>>>
>>>>>>>> Thanks,
>>>>>>>>
>>>>>>>> Mike
>>>>>>>>
>>>>>>>> *From:*devel@edk2.groups.io <devel@edk2.groups.io> *On Behalf Of *Maciej
>>>>>>>> Rabeda
>>>>>>>> *Sent:* Thursday, December 2, 2021 10:27 AM
>>>>>>>> *To:* devel@edk2.groups.io; Kinney, Michael D
>>>>>>>> <michael.d.kinney@intel.com>; Michael Kubacki
>>>>>>>> <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com)
>>>>>>>> <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>
>>>>>>>> *Subject:* Re: [edk2-devel] Uncrustify Conversion Detailed Plan and
>>>>>>>> Extended Hard Freeze Update #4
>>>>>>>>
>>>>>>>> Hey Mike,
>>>>>>>>
>>>>>>>> While most of the changes related to fixing coding style violations,
>>>>>>>> there are a couple of changes to NetworkPkg in that PR that make the
>>>>>>>> code less readable. Examples below.
>>>>>>>>
>>>>>>>> Example 1:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Example 2:
>>>>>>>>
>>>>>>>>
>>>>>>>> Example 3:
>>>>>>>>
>>>>>>>>
>>>>>>>> Example 4:
>>>>>>>>
>>>>>>>> On 30-Nov-21 23:34, Michael D Kinney wrote:
>>>>>>>>
>>>>>>>> Hello,
>>>>>>>>
>>>>>>>> Thank you for your patience during this extended hard freeze.
>>>>>>>>
>>>>>>>> Just one more step to go.There has been a delay in the review of
>>>>>>>>
>>>>>>>> the patch series with the uncrustify source changes.PR(6).This
>>>>>>>>
>>>>>>>> patch series was not sent out as patch review email because of its
>>>>>>>>
>>>>>>>> very large size.It only contains source style changes and the
>>>>>>>>
>>>>>>>> CompareBuild tool and GitHub action has shown there are no binary
>>>>>>>>
>>>>>>>> differences introduced with these source style changes.
>>>>>>>>
>>>>>>>> If you are a package maintainer, then please review the following
>>>>>>>>
>>>>>>>> branch/PR for your package contents and review the EDK II CI results
>>>>>>>>
>>>>>>>> and BuildCompare results.I do not expect a line by line review
>>>>>>>>
>>>>>>>> because we already had time to provide feedback on the source style
>>>>>>>>
>>>>>>>> performed by uncrustify.Instead, a Reviewed-by for your package
>>>>>>>>
>>>>>>>> indicates that you have reviewed the EDK II CI results and CompareBuild
>>>>>>>>
>>>>>>>> tool functionality and results and you accept the source style
>>>>>>>>
>>>>>>>> changes to your package.
>>>>>>>>
>>>>>>>> *https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
>>>>>>> <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5>
>>>>>>>> *https://github.com/tianocore/edk2/pull/2229 <https://github.com/tianocore/edk2/pull/2229>
>>>>>>>>
>>>>>>>> *https://github.com/mdkinney/edk2/actions/runs/1521618836
>>>>>>> <https://github.com/mdkinney/edk2/actions/runs/1521618836>
>>>>>>>> Additional details on this update below.
>>>>>>>>
>>>>>>>> Thank you,
>>>>>>>>
>>>>>>>> Mike
>>>>>>>>
>>>>>>>> Changes from Update #3
>>>>>>>>
>>>>>>>> ----------------------------------------------------------------------------
>>>>>>>>
>>>>>>>> * Pushed PR (5)
>>>>>>>>
>>>>>>>> * Added link to PR(6). EDK II CI Status is PASS. Build Compare PASS.
>>>>>>>>
>>>>>>>> * Waiting for review of PR (6)
>>>>>>>>
>>>>>>>> * Review of PR (7) completed and waiting for review of PR (6)
>>>>>>>>
>>>>>>>> ----------------------------------------------------------------------------
>>>>>>>>
>>>>>>>> Changes from Update #2
>>>>>>>>
>>>>>>>> ----------------------------------------------------------------------------
>>>>>>>>
>>>>>>>> * Changed order of PRs swapping (4) and (5).The PR that activates
>>>>>>>>
>>>>>>>> increases the max CI agent job time is independent of all the other
>>>>>>>>
>>>>>>>> PRs and its review is complete, so it can be committed now.
>>>>>>>>
>>>>>>>> * Pushed PRs (1), (2), (3), (4).
>>>>>>>>
>>>>>>>> * Waiting for review to complete for PRs (5) and (6)
>>>>>>>>
>>>>>>>> * Reviews complete for PR (7)
>>>>>>>>
>>>>>>>> * Identifies steps using git filter-branch to apply uncrustify changes to a
>>>>>>>>
>>>>>>>> code review patch series that was generated before the uncrustify changes
>>>>>>>>
>>>>>>>> avoiding manual merge.
>>>>>>>>
>>>>>>>> * Identified steps using git filter-repo to generate an alternate history of
>>>>>>>>
>>>>>>>> the edk2 repo with uncrustify changes applied on every commit.This may
>>>>>>>>
>>>>>>>> be useful when evaluating changes to files using tools like git blame
>>>>>>>>
>>>>>>>> without the large uncrustify patch series.
>>>>>>>>
>>>>>>>> ---------------------------------------------------------------------------
>>>>>>>>
>>>>>>>> Changes from Update #1
>>>>>>>>
>>>>>>>> ----------------------------------------------------------------------------
>>>>>>>>
>>>>>>>> * Changed order of PRs swapping (6) and (7).The PR that activates
>>>>>>>>
>>>>>>>> EDK II CI check UncrustifyCheck has to be last because it unconditionally
>>>>>>>>
>>>>>>>> checks all C/H files in all packages.Not just files that have been
>>>>>>>>
>>>>>>>> modified like some of the other checkers.
>>>>>>>>
>>>>>>>> * Updated link to the branch with the UncrustifyCheck plugin that has been
>>>>>>>>
>>>>>>>> updated with a one line change and Reviewed-by and Tested-by tags.
>>>>>>>>
>>>>>>>> https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v6
>>>>>>> <https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v6>
>>>>>>>> * Reviews complete for (1), (2), (3), (5), and (7)
>>>>>>>>
>>>>>>>> ---------------------------------------------------------------------------
>>>>>>>>
>>>>>>>> Michael Kubacki and I have prepared the patches required to apply the
>>>>>>>>
>>>>>>>> uncrustify changes and enable EDK II CI to check all submitted
>>>>>>>>
>>>>>>>> patches have been run through uncrustify.
>>>>>>>>
>>>>>>>> We have verified through the CompareBuild GitHub Action that the
>>>>>>>>
>>>>>>>> format changes performed by uncrustify have no functional changes.
>>>>>>>>
>>>>>>>> All of the OBJ, LIB, DLL, EFI, FFS, FV, and FD files match 100%
>>>>>>>>
>>>>>>>> across 70 VS2019/GCC5 builds of all package/platform DSC files in
>>>>>>>>
>>>>>>>> the edk2 repo.
>>>>>>>>
>>>>>>>> The hard freeze will be extended after the edk2-stable202111 tag until
>>>>>>>>
>>>>>>>> all uncrustify related changes are committed.We do not expect this
>>>>>>>>
>>>>>>>> to take more than a few days.Do not push any PRs until the hard
>>>>>>>>
>>>>>>>> freeze is lifted.
>>>>>>>>
>>>>>>>> The changes are broken up into 7 patch series/PRs.The PRs are ordered
>>>>>>>>
>>>>>>>> so they can be submitted using the normal submission process and EDK II
>>>>>>>>
>>>>>>>> CI will pass for each one.Details are listed below.
>>>>>>>>
>>>>>>>> Uncrustify 73.0.3 for EDK II
>>>>>>>>
>>>>>>>> =============================
>>>>>>>>
>>>>>>>> * Sources:https://dev.azure.com/projectmu/_git/Uncrustify <https://dev.azure.com/projectmu/_git/Uncrustify>
>>>>>>>>
>>>>>>>> * Documentation:https://dev.azure.com/projectmu/Uncrustify/_wiki/wikis/Uncrustify.wiki/1/Project-Mu-(EDK-
>> II)-
>>>> Fork-
>>>>>>> Readme <https://dev.azure.com/projectmu/Uncrustify/_wiki/wikis/Uncrustify.wiki/1/Project-Mu-(EDK-II)-Fork-Readme>
>>>>>>>> * Download:https://dev.azure.com/projectmu/Uncrustify/_packaging?_a=package&feed=mu_uncrustify&package=mu-
>>>>>>> uncrustify-release&protocolType=NuGet&version=73.0.3
>>>>>>> <https://dev.azure.com/projectmu/Uncrustify/_packaging?_a=package&feed=mu_uncrustify&package=mu-uncrustify-
>>>>>>> release&protocolType=NuGet&version=73.0.3>
>>>>>>>> Installing Uncrustify
>>>>>>>>
>>>>>>>> ======================
>>>>>>>>
>>>>>>>> The Uncrustify tool is installed automatically when the Pytools
>>>>>>>>
>>>>>>>> environment is used and the stuart* commands are run to complete the
>>>>>>>>
>>>>>>>> environment setup.Please see:
>>>>>>>>
>>>>>>>> https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally
>>>>>>> <https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally>
>>>>>>>> Uncrustify can also be installed from the download page listed above
>>>>>>>>
>>>>>>>> or built from sources from the source link above.
>>>>>>>>
>>>>>>>> The Documentation link provides instruction on how to run uncrustify from
>>>>>>>>
>>>>>>>> the command line or install as a Visual Studio Code plugin.The main
>>>>>>>>
>>>>>>>> uncrustify documentation also describes how to integrate with a few other
>>>>>>>>
>>>>>>>> editors.
>>>>>>>>
>>>>>>>> We have also discussed a client side githook.That effort has not started.
>>>>>>>>
>>>>>>>> Let us know if that is a feature you would find useful.
>>>>>>>>
>>>>>>>> Developer impact for new code reviews
>>>>>>>>
>>>>>>>> ======================================
>>>>>>>>
>>>>>>>> Once the uncrustify checker is active in EDK II CI, developers must
>>>>>>>>
>>>>>>>> make sure their patches are run through the uncrustify tool before
>>>>>>>>
>>>>>>>> sending the patches for review.
>>>>>>>>
>>>>>>>> Developers must install and run uncrustify against changes files before
>>>>>>>>
>>>>>>>> sending patch review emails or submitting PR for EDK II CI.If EDK II CI
>>>>>>>>
>>>>>>>> detects and differences in source formatting, then EDK II CI will fail
>>>>>>>>
>>>>>>>> and the developer must run uncrustify and resubmit the patches.
>>>>>>>>
>>>>>>>> Developer impact to patch series/PRs reviewed during edk2-stable201121 soft/hard freeze
>>>>>>>>
>>>>>>>> =======================================================================================
>>>>>>>>
>>>>>>>> Developers must rebase their changes after the uncrustify source changes are
>>>>>>>>
>>>>>>>> committed.The branch with a preview of the uncrustify changes can be used
>>>>>>>>
>>>>>>>> to start this rebase work.
>>>>>>>>
>>>>>>>> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
>>>>>>> <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5>
>>>>>>>> The following steps can be used to update an existing branch with the
>>>>>>>>
>>>>>>>> required uncrustify format.This is the Windows version.I will add
>>>>>>>>
>>>>>>>> the Linux version soon.
>>>>>>>>
>>>>>>>> 1) Fetch and checkout and rebase to latest edk2/master
>>>>>>>>
>>>>>>>> git fetch origin
>>>>>>>>
>>>>>>>> git checkout master
>>>>>>>>
>>>>>>>> git rebase origin/master
>>>>>>>>
>>>>>>>> 2) Make a backup copy of plugin UncrustifyCheck outside WORKSPACE.
>>>>>>>>
>>>>>>>> (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable and
>>>>>>>>
>>>>>>>> EDK II specific uncrustify configuration file available when working
>>>>>>>>
>>>>>>>> with a branch that does not have those tools in its scope.
>>>>>>>>
>>>>>>>> xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
>>>>>>>>
>>>>>>>> 3) Check out the patch series branch (e.g. MyBranch)
>>>>>>>>
>>>>>>>> git checkout MyBranch
>>>>>>>>
>>>>>>>> 4) Rebase patch series against edk2-stable202111
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> git rebase edk2-stable202111
>>>>>>>>
>>>>>>>> 5) Create new branch for the uncrustifed version (e.g. MyBranch_Uncrustified)
>>>>>>>>
>>>>>>>> git checkout -b MyBranch_Uncrustified
>>>>>>>>
>>>>>>>> 6) Use git filter-branch to uncrustify all the commits in the series
>>>>>>>>
>>>>>>>> between the rebase target from (2) and HEAD of the branch.A filter
>>>>>>>>
>>>>>>>> can be used to scope the uncrustify operations to only the C/H files
>>>>>>>>
>>>>>>>> in the specific package the patch series is against. (e.g. DynamicTablesPkg).
>>>>>>>>
>>>>>>>> BaseTools should always be excluded.If the package scoped filter is
>>>>>>>>
>>>>>>>> not used, it will still work, but will take longer to run because
>>>>>>>>
>>>>>>>> uncrustify will rescan every C/H files in the whole repo.
>>>>>>>>
>>>>>>>> git filter-branch --tree-filter "git ls-files DynamicTablesPkg*.c DynamicTablesPkg*.h :!BaseTools/* |
>>>>>>> c:\\Temp\\UncrustifyCheck\\mu-uncrustify-release_extdep\\Windows-x86\\uncrustify.exe -c
>>>>>>> c:\\Temp\\UncrustifyCheck\\uncrustify.cfg -F - --replace --no-backup --if-changed" edk2-stable202111..HEAD
>>>>>>>> 7) Now that all the individual patches in the branch are uncrustified,
>>>>>>>>
>>>>>>>> rebase against latest edk2/master that is already uncrustified.
>>>>>>>>
>>>>>>>> git rebase master
>>>>>>>>
>>>>>>>> 8) Verify the patches in this new branch.
>>>>>>>>
>>>>>>>> Impacts to tracing history across the uncrusity changes
>>>>>>>>
>>>>>>>> =======================================================
>>>>>>>>
>>>>>>>> Tools the view file and line history do work with the large uncrustify
>>>>>>>>
>>>>>>>> patch series.One impact is that the operations can be very slow due
>>>>>>>>
>>>>>>>> to the large uncrustify patches.
>>>>>>>>
>>>>>>>> One option to provide a faster experience is to provide an alternate
>>>>>>>>
>>>>>>>> version of the edk2 repository as "documentation" that has the
>>>>>>>>
>>>>>>>> entire history re-written with uncrustify run on every commit.
>>>>>>>>
>>>>>>>> The tool called git-filter-repo can be used to perform this
>>>>>>>>
>>>>>>>> transformation and runs in a reasonable period of time (a few hours)
>>>>>>>>
>>>>>>>> https://github.com/newren/git-filter-repo <https://github.com/newren/git-filter-repo>
>>>>>>>>
>>>>>>>> https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-demos/lint-history
>>>>>>> <https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-demos/lint-history>
>>>>>>>> The following steps can be used to perform this transformation.
>>>>>>>>
>>>>>>>> This is the Windows version. I will add the Linux version soon.
>>>>>>>>
>>>>>>>> ** WARNING **This operation modifies(rewrites) all the commits
>>>>>>>>
>>>>>>>> in the local copy of the repo.Do not perform
>>>>>>>>
>>>>>>>> these steps on a local repo you are using for
>>>>>>>>
>>>>>>>> active development.
>>>>>>>>
>>>>>>>> 1) Clone edk2 into a new directory (see **WARNING**)
>>>>>>>>
>>>>>>>> git clonehttps://github.com/tianocore/edk2.git <https://github.com/tianocore/edk2.git> edk2-uncrustified
>>>>>>>>
>>>>>>>> cd edk2-uncrustified
>>>>>>>>
>>>>>>>> 2) Setup python virtual env, install pytools, and run stuart commands
>>>>>>>>
>>>>>>>> to setup build environment which includes installing uncrustify tools.
>>>>>>>>
>>>>>>>> https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally
>>>>>>> <https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally>
>>>>>>>> 3) Make a backup copy of plugin UncrustifyCheck outside WORKSPACE.
>>>>>>>>
>>>>>>>> (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable and
>>>>>>>>
>>>>>>>> EDK II specific uncrustify configuration file available when working
>>>>>>>>
>>>>>>>> with a branch that does not have those tools in its scope.
>>>>>>>>
>>>>>>>> xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
>>>>>>>>
>>>>>>>> 4) Use lint-history.py from git-filter-repo examples
>>>>>>>>
>>>>>>>> https://github.com/newren/git-filter-repo <https://github.com/newren/git-filter-repo>
>>>>>>>>
>>>>>>>> https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-demos/lint-history
>>>>>>> <https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-demos/lint-history>
>>>>>>>> Line #127 - Add try except around subprocess.check_call() with except
>>>>>>>>
>>>>>>>> being pass.This is required because there are a few commits of C
>>>>>>>>
>>>>>>>> files in the edk2 repo that have incorrect C syntax and do not
>>>>>>>>
>>>>>>>> build with a C compiler and break the uncrustify parser.Skip reformat
>>>>>>>>
>>>>>>>> of C files that can not be parsed by uncrustify.These rare instances
>>>>>>>>
>>>>>>>> are addressed in the commit that fixes the C syntax error.
>>>>>>>>
>>>>>>>> Run this slightly modified version of lint-history.Include only
>>>>>>>>
>>>>>>>> C/H files and exclude directories that start with 'Tools' or 'BaseTools'.
>>>>>>>>
>>>>>>>> This step took about 2.2 hours on a laptop.
>>>>>>>>
>>>>>>>> lint-history.py
>>>>>>>>
>>>>>>>> --relevant "return (not filename.startswith(b'Tools') and not filename.startswith(b'BaseTools') and
>>>>>>> (filename.endswith(b'.c') or filename.endswith(b'.h')))"
>>>>>>>> c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\mu-uncrustify-release_extdep\\Windows-x86\\uncrustify.exe
>> -
>>> c
>>>>>>> c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\uncrustify.cfg --replace --no-backup --if-changed
>>>>>>>> Order of PRs to apply during extended hard freeze
>>>>>>>>
>>>>>>>> ==================================================
>>>>>>>>
>>>>>>>> 1) Update EmulatorPkg Win Host [BuildOptions] MSFT CC_FLAGS to not force debug information
>>>>>>>>
>>>>>>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3747 <https://bugzilla.tianocore.org/show_bug.cgi?id=3747>
>>>>>>>>
>>>>>>>> *https://github.com/mdkinney/edk2/tree/Bug_3747_EmulatorPkg_WinHost_ReproducibleBuild
>>>>>>> <https://github.com/mdkinney/edk2/tree/Bug_3747_EmulatorPkg_WinHost_ReproducibleBuild>
>>>>>>>> *https://github.com/tianocore/edk2/pull/2215 <https://github.com/tianocore/edk2/pull/2215>
>>>>>>>>
>>>>>>>> * Required for EmulatorPkg to pass CompareBuild for VS2019 IA32/X64 builds.
>>>>>>>>
>>>>>>>> * Status: Review complete.PR pushed.
>>>>>>>>
>>>>>>>> 2) EccCheck should not revert staged and local changes
>>>>>>>>
>>>>>>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=2986 <https://bugzilla.tianocore.org/show_bug.cgi?id=2986>
>>>>>>>>
>>>>>>>> *https://github.com/mdkinney/edk2/tree/Bug_2986_EccCheckRemoveGitRevert_V2
>>>>>>> <https://github.com/mdkinney/edk2/tree/Bug_2986_EccCheckRemoveGitRevert_V2>
>>>>>>>> *https://github.com/tianocore/edk2/pull/2216 <https://github.com/tianocore/edk2/pull/2216>
>>>>>>>>
>>>>>>>> * Required for EDK II CI to complete in a reasonable period of time when
>>>>>>>>
>>>>>>>> processing the 4000+ source file style changes made by uncrustify.
>>>>>>>>
>>>>>>>> * Also fixes critical bugs that can potentially corrupt git state when
>>>>>>>>
>>>>>>>> EccCheck is run locally.
>>>>>>>>
>>>>>>>> * Status: Review complete.PR pushed.
>>>>>>>>
>>>>>>>> 3) Update pytool LicenseCheck plugin to use temp directory for diff output file
>>>>>>>>
>>>>>>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3746 <https://bugzilla.tianocore.org/show_bug.cgi?id=3746>
>>>>>>>>
>>>>>>>> *https://github.com/mdkinney/edk2/tree/Bug_3746_LicenseCheckUseDiffOutputFile_V2
>>>>>>> <https://github.com/mdkinney/edk2/tree/Bug_3746_LicenseCheckUseDiffOutputFile_V2>
>>>>>>>> *https://github.com/tianocore/edk2/pull/2217 <https://github.com/tianocore/edk2/pull/2217>
>>>>>>>>
>>>>>>>> * Required to reduce EDK II CI build times.
>>>>>>>>
>>>>>>>> * Status: Review complete.PR pushed.
>>>>>>>>
>>>>>>>> 4) Update max job time from 60 min to 120 minutes in .azurepipelines/templates
>>>>>>>>
>>>>>>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3750 <https://bugzilla.tianocore.org/show_bug.cgi?id=3750>
>>>>>>>>
>>>>>>>> *https://github.com/mdkinney/edk2/tree/Bug_3750_IncreaseAzurePipelinesTimeout
>>>>>>> <https://github.com/mdkinney/edk2/tree/Bug_3750_IncreaseAzurePipelinesTimeout>
>>>>>>>> *https://github.com/tianocore/edk2/pull/2219 <https://github.com/tianocore/edk2/pull/2219>
>>>>>>>>
>>>>>>>> * Required to allow EccCheck of uncrustify changes to complete on Azure
>>>>>>>>
>>>>>>>> Pipelines CI agents without timing out.
>>>>>>>>
>>>>>>>> * Status: Review complete.PR pushed.
>>>>>>>>
>>>>>>>> 5) Update Package YAML to ignore specific ECC files/errors
>>>>>>>>
>>>>>>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3749 <https://bugzilla.tianocore.org/show_bug.cgi?id=3749>
>>>>>>>>
>>>>>>>> *https://github.com/mdkinney/edk2/tree/Bug_3749_EccCheckIgnoreFilesErrors
>>>>>>> <https://github.com/mdkinney/edk2/tree/Bug_3749_EccCheckIgnoreFilesErrors>
>>>>>>>> *https://github.com/tianocore/edk2/pull/2218 <https://github.com/tianocore/edk2/pull/2218>
>>>>>>>>
>>>>>>>> * Required to pass EccCheck
>>>>>>>>
>>>>>>>> * Status: Review complete. PR pushed
>>>>>>>>
>>>>>>>> 6) Uncrustify Source Changes
>>>>>>>>
>>>>>>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3737 <https://bugzilla.tianocore.org/show_bug.cgi?id=3737>
>>>>>>>>
>>>>>>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3739 <https://bugzilla.tianocore.org/show_bug.cgi?id=3739>
>>>>>>>>
>>>>>>>> *https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
>>>>>>> <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5>
>>>>>>>> *https://github.com/tianocore/edk2/pull/2229 <https://github.com/tianocore/edk2/pull/2229>
>>>>>>>>
>>>>>>>> * Build comparison result PASS:https://github.com/mdkinney/edk2/actions/runs/1521618836
>>>>>>> <https://github.com/mdkinney/edk2/actions/runs/1521618836>
>>>>>>>> * EFI_D_ -> DEBUG changes required to pass PatchCheck
>>>>>>>>
>>>>>>>> * Uncrustify format changes required to pass UncrustifyCheck
>>>>>>>>
>>>>>>>> * Status:
>>>>>>>>
>>>>>>>> Waiting for review
>>>>>>>>
>>>>>>>> 7) UncrustifyCheck EDK II CI Plugin
>>>>>>>>
>>>>>>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3748 <https://bugzilla.tianocore.org/show_bug.cgi?id=3748>
>>>>>>>>
>>>>>>>> *https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v6
>>>>>>> <https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v6>
>>>>>>>> * Required to enforce all PRs submitted to EDK II CI match uncrustify format.
>>>>>>>>
>>>>>>>> * Unconditionally checks all packages.Can not be committed until all C/H
>>>>>>>>
>>>>>>>> source files have been updated.
>>>>>>>>
>>>>>>>> * Status: Review complete
>>>>>>>>
>>>>>>>> Combined Branch/PR for Review/Test
>>>>>>>>
>>>>>>>> ==================================
>>>>>>>>
>>>>>>>> * Build Comparison results must pass 100% across the full set of PRs before
>>>>>>>>
>>>>>>>> the individual PRs can be pushed in the order listed above.
>>>>>>>>
>>>>>>>> * Branch:https://github.com/mdkinney/edk2/tree/TestOnly_Uncrustify_PR_Series
>>>>>>> <https://github.com/mdkinney/edk2/tree/TestOnly_Uncrustify_PR_Series>
>>>>>>>> * PR:https://github.com/tianocore/edk2/pull/2229 <https://github.com/tianocore/edk2/pull/2229>
>>>>>>>>
>>>>>>>> Status = PASS
>>>>>>>>
>>>>>>>> * CompareBuild:
>>>>>>>>
>>>>>>>> Branch:https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
>>>>>>> <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5>
>>>>>>>> --ref1:ef9a059cdb15844fe52a49af2bf7d86b9dd3e9bf
>>>>>>>>
>>>>>>>> --ref2:Bug_3737_3739_ApplyUncrustifyChanges_V5
>>>>>>>>
>>>>>>>> Extra Options: -n 4 --quiet
>>>>>>>>
>>>>>>>> Results:https://github.com/mdkinney/edk2/actions/runs/1521618836
>>>>>>> <https://github.com/mdkinney/edk2/actions/runs/1521618836>
>>>>>>>> 30 VS2019 build comparisons PASS
>>>>>>>>
>>>>>>>> 40 GCC5 build comparisons PASS
>>>>>>>>
>>>>>>>> 100% PASS
>>>>>>>>
>>>>>>>> The following git log shows the set of patches from --ref1 to --ref 2across
>>>>>>>>
>>>>>>>> which there are no differences in any of the OBJ/LIB/DLL/EFI/FFS/FV/FD files.
>>>>>>>>
>>>>>>>> --ref2
>>>>>>>>
>>>>>>>> b7d4bf0675b7 (HEAD -> Bug_3737_3739_ApplyUncrustifyChanges_V5) UnitTestFrameworkPkg: Apply uncrusitify
>> changes
>>>>>>>> 7f03d25f60e7 UefiPayloadPkg: Apply uncrusitify changes
>>>>>>>>
>>>>>>>> 0bfd8d9b5ac9 UefiCpuPkg: Apply uncrusitify changes
>>>>>>>>
>>>>>>>> e1cd9bfb9dea StandaloneMmPkg: Apply uncrusitify changes
>>>>>>>>
>>>>>>>> 5da2f65be378 SourceLevelDebugPkg: Apply uncrusitify changes
>>>>>>>>
>>>>>>>> 95b86de07e5d SignedCapsulePkg: Apply uncrusitify changes
>>>>>>>>
>>>>>>>> fe71d97246c4 ShellPkg: Apply uncrusitify changes
>>>>>>>>
>>>>>>>> 54c21c952992 SecurityPkg: Apply uncrusitify changes
>>>>>>>>
>>>>>>>> 187a3785f12b RedfishPkg: Apply uncrusitify changes
>>>>>>>>
>>>>>>>> 810100002a46 PcAtChipsetPkg: Apply uncrusitify changes
>>>>>>>>
>>>>>>>> 276a695c0cf2 OvmfPkg: Apply uncrusitify changes
>>>>>>>>
>>>>>>>> 303c0a91ab07 NetworkPkg: Apply uncrusitify changes
>>>>>>>>
>>>>>>>> bc80792cd1b1 MdePkg: Apply uncrusitify changes
>>>>>>>>
>>>>>>>> 3ea86be17a2a MdeModulePkg: Apply uncrusitify changes
>>>>>>>>
>>>>>>>> c70ef11ed0cd IntelFsp2WrapperPkg: Apply uncrusitify changes
>>>>>>>>
>>>>>>>> c0291221f252 IntelFsp2Pkg: Apply uncrusitify changes
>>>>>>>>
>>>>>>>> 6a479952a690 FmpDevicePkg: Apply uncrusitify changes
>>>>>>>>
>>>>>>>> 3a7c05b7070d FatPkg: Apply uncrusitify changes
>>>>>>>>
>>>>>>>> b789f98c8959 EmulatorPkg: Apply uncrusitify changes
>>>>>>>>
>>>>>>>> 952d7a1c9220 EmbeddedPkg: Apply uncrusitify changes
>>>>>>>>
>>>>>>>> a1cc9881bab6 DynamicTablesPkg: Apply uncrusitify changes
>>>>>>>>
>>>>>>>> 50654dfe5785 CryptoPkg: Apply uncrusitify changes
>>>>>>>>
>>>>>>>> ed965a02dfa1 ArmVirtPkg: Apply uncrusitify changes
>>>>>>>>
>>>>>>>> 9744023fbc46 ArmPlatformPkg: Apply uncrusitify changes
>>>>>>>>
>>>>>>>> 7a1cde5f5bba ArmPkg: Apply uncrusitify changes
>>>>>>>>
>>>>>>>> 19d17e0913e8 UefiCpuPkg: Change use of EFI_D_* to DEBUG_*
>>>>>>>>
>>>>>>>> ffa718b4f994 SourceLevelDebugPkg: Change use of EFI_D_* to DEBUG_*
>>>>>>>>
>>>>>>>> b86cb3c5e5b4 ShellPkg: Change use of EFI_D_* to DEBUG_*
>>>>>>>>
>>>>>>>> c7c42204dc07 SecurityPkg: Change use of EFI_D_* to DEBUG_*
>>>>>>>>
>>>>>>>> 16b8e6f958e4 PcAtChipsetPkg: Change use of EFI_D_* to DEBUG_*
>>>>>>>>
>>>>>>>> 0ac3f8b2dac5 OvmfPkg: Change use of EFI_D_* to DEBUG_*
>>>>>>>>
>>>>>>>> bc5004b8d294 NetworkPkg: Change use of EFI_D_* to DEBUG_*
>>>>>>>>
>>>>>>>> 6f671a8e2377 MdePkg: Change use of EFI_D_* to DEBUG_*
>>>>>>>>
>>>>>>>> a10c610ff9a3 MdeModulePkg: Change use of EFI_D_* to DEBUG_*
>>>>>>>>
>>>>>>>> 09a3bddba390 FatPkg: Change use of EFI_D_* to DEBUG_*
>>>>>>>>
>>>>>>>> 59c61318246a EmulatorPkg: Change use of EFI_D_* to DEBUG_*
>>>>>>>>
>>>>>>>> 3a80367dda3b EmbeddedPkg: Change use of EFI_D_* to DEBUG_*
>>>>>>>>
>>>>>>>> 23eb1aaf80ca ArmVirtPkg: Change use of EFI_D_* to DEBUG_*
>>>>>>>>
>>>>>>>> 875914b45c54 ArmPlatformPkg: Change use of EFI_D_* to DEBUG_*
>>>>>>>>
>>>>>>>> eb2eca82b451 ArmPkg: Change use of EFI_D_* to DEBUG_*
>>>>>>>>
>>>>>>>> f0f3f5aae7c4 (origin/master, origin/HEAD, master) UnitTestFrameworkPkg: Update YAML to ignore specific ECC
>>>>>>> files/errors
>>>>>>>> c05734797790 UefiPayloadPkg: Update YAML to ignore specific ECC files/errors
>>>>>>>>
>>>>>>>> c30c40d6c63d StandaloneMmPkg: Update YAML to ignore specific ECC files/errors
>>>>>>>>
>>>>>>>> 9944508e85f1 ShellPkg: Update YAML to ignore specific ECC files/errors
>>>>>>>>
>>>>>>>> 60fa40be458d SecurityPkg: Update YAML to ignore specific ECC files/errors
>>>>>>>>
>>>>>>>> df790cd6b37e MdePkg: Update YAML to ignore specific ECC files/errors
>>>>>>>>
>>>>>>>> 9deb9370766e MdeModulePkg: Update YAML to ignore specific ECC files/errors
>>>>>>>>
>>>>>>>> d7d30e8f219f EmulatorPkg: Update YAML to ignore specific ECC files/errors
>>>>>>>>
>>>>>>>> d5744ecba813 CryptoPkg: Update YAML to ignore specific ECC files/errors
>>>>>>>>
>>>>>>>> c97fee87f0f9 ArmVirtPkg: Update YAML to ignore specific ECC files/errors
>>>>>>>>
>>>>>>>> 1939fc9569f2 ArmPlatformPkg: Update YAML to ignore specific ECC files/errors
>>>>>>>>
>>>>>>>> 365dced2c37a ArmPkg: Update YAML to ignore specific ECC files/errors
>>>>>>>>
>>>>>>>> 76a1ce4d5fec .azurepipelines/templates: Update max pipeline job time to 2 hours
>>>>>>>>
>>>>>>>> 99f84ff47390 .pytools/Plugin/LicenseCheck: Use temp directory for git diff output
>>>>>>>>
>>>>>>>> 3019f1bbabf1 .pytool/Plugin/EccCheck: Add performance optimizations
>>>>>>>>
>>>>>>>> 854462bd3479 .pytool/Plugin/EccCheck: Remove temp directory on exception
>>>>>>>>
>>>>>>>> 69877614fdee .pytool/Plugin/EccCheck: Remove RevertCode()
>>>>>>>>
>>>>>>>> --ref1
>>>>>>>>
>>>>>>>> ef9a059cdb15 EmulatorPkg/Win/Host: Update CC_FLAGS
>>>>>>>>
>>>>>>>> bb1bba3d7767 (tag: edk2-stable202111) NetworkPkg: Fix invalid pointer for DNS response token on error
>>>>>>>>
>>>>>>>> Best regards,
>>>>>>>>
>>>>>>>> Mike
>>>>>>>>
>>>>>>>>
>
>
>
>
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
2021-12-06 1:17 ` Michael D Kinney
` (3 preceding siblings ...)
2021-12-06 20:28 ` Maciej Rabeda
@ 2021-12-07 0:26 ` Abner Chang
2021-12-07 3:47 ` Wang, Jian J
` (2 subsequent siblings)
7 siblings, 0 replies; 39+ messages in thread
From: Abner Chang @ 2021-12-07 0:26 UTC (permalink / raw)
To: devel@edk2.groups.io, michael.d.kinney@intel.com, Michael Kubacki,
maciej.rabeda@linux.intel.com, Michael Kubacki,
Andrew Fish (afish@apple.com), Leif Lindholm
For RedfishPkg on the branch https://github.com/mdkinney/edk2/tree/Bug_3737_3760_3739_ApplyUncrustifyChanges_V7
Reviewed-by: Abner Chang
> -----Original Message-----
> From: devel@edk2.groups.io [mailto:devel@edk2.groups.io] On Behalf Of
> Michael D Kinney
> Sent: Monday, December 6, 2021 9:18 AM
> To: Michael Kubacki <mikuback@linux.microsoft.com>;
> devel@edk2.groups.io; maciej.rabeda@linux.intel.com; Michael Kubacki
> <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com)
> <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>; Kinney, Michael D
> <michael.d.kinney@intel.com>
> Subject: Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended
> Hard Freeze Update #4
>
> Hello EDK II Maintainers,
>
> A detailed evaluation of the DEBUG_CODE() formatting issue has been
> completed.
> The reason DEBUG_CODE() is a challenge is that this looks like a macro from a
> C parsing perspective, but the EDK II usage places C statements or blocks of
> C code as the parameter to this macro.
>
> There are actually 2 methods available to mark a statement or a block of code
> to be included when the DEBUG_PROPERTY_DEBUG_CODE_ENABLED bit is
> enabled in
> PcdDebugPropertyMask. One is DEBUG_CODE() and the other is to mark the
> beginning
> and end of a code block with DEBUG_CODE_BEGIN() and
> DEBUG_CODE_END(). In fact,
> DEBUG_CODE() is implemented using DEBUG_CODE_BEGIN() and
> DEBUG_CODE_END() macros.
>
> #define DEBUG_CODE(Expression) \
> DEBUG_CODE_BEGIN (); \
> Expression \
> DEBUG_CODE_END ()
>
> A complete review for the use of these DEBUG_CODE macros was
> performed on the
> edk2 repo. Uncrustify performs good formatting for code blocks between
> DEBUG_CODE_BEGIN() and DEBUG_CODE_END(). This is because these look
> like simple
> macros calls with no parameters and the lines of C code between these 2
> macros
> is formatted correctly.
>
> The uncrustify formatting issues are only present with the use of
> DEBUG_CODE().
> Simple use cases of DEBUG_CODE(Expression) where Expression is a single C
> statement also look correct. A medium complexity use case where
> Expression is
> a code block of simple statements or even some local variables and simple
> statements also look correct. It is only complex code blocks that use C
> statements such as if/for/while/case that include the use of braces {} does
> uncrustify perform incorrect formatting.
>
> The recommended solution to this issue is to convert the use of
> DEBUG_CODE()
> to DEBUG_CODE_BEGIN() / DEBUG_CODE_END() for cases where the
> Expression
> passed to DEBUG_CODE() is the complex use case that contains statements
> that
> use braces {}. There are 57 instances of this pattern across 40 files in the
> edk2 repo.
>
> I have posted a branch with these additional patches:
>
>
> https://github.com/mdkinney/edk2/tree/Bug_3737_3760_3739_ApplyUncru
> stifyChanges_V7
>
> I have performed CompareBuild tests with this revised patch series with
> the DEBUG_CODE changes. It passes 100% showing no binary differences.
>
> https://github.com/mdkinney/edk2/actions/runs/1542454606
>
> I have opened a PR to run this patch series through EDK II CI. It also passes
> 100%.
>
> https://github.com/tianocore/edk2/pull/2236
>
> The summary of changes made since the V6 review are:
>
> 1) Change uncrustify configuration assignment alignment threshold to 0
>
> align_assign_thresh = 0
>
> 2) Replace "<param>, OPTIONAL" with "<param> OPTIONAL,"
>
> 3) Replace DEBUG_CODE(Expression) with
>
> DEBUG_CODE_BEGIN();
> Expression
> DEBUG_CODE_END()
>
> if Expression is complex and contains braces {}.
>
> 4) No changes to uncrustify tool required.
>
> Please review the differences between the following 2 branches and provide
> feedback or a Series Reviewed-by if you agree with these additional changes.
>
>
> https://github.com/mdkinney/edk2/tree/Bug_3737_3760_3739_ApplyUncru
> stifyChanges_V6
>
> https://github.com/mdkinney/edk2/tree/Bug_3737_3760_3739_ApplyUncru
> stifyChanges_V7
>
> The goal is to complete the review and get the uncrustify change committed
> tomorrow so the extended hard freeze can be lifted.
>
> Thanks,
>
> Mike
>
> > -----Original Message-----
> > From: Kinney, Michael D <michael.d.kinney@intel.com>
> > Sent: Thursday, December 2, 2021 6:23 PM
> > To: Michael Kubacki <mikuback@linux.microsoft.com>;
> devel@edk2.groups.io; maciej.rabeda@linux.intel.com; Michael Kubacki
> > <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com)
> <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>;
> > Kinney, Michael D <michael.d.kinney@intel.com>
> > Subject: RE: [edk2-devel] Uncrustify Conversion Detailed Plan and
> Extended Hard Freeze Update #4
> >
> > Hello EDK II Maintainers,
> >
> > I have entered BZ 3760 to make the use of the OPTIONAL keyword style
> consistent for all of edk2 repo
> > and to be compatible with uncrustify.
> >
> > I have posted the following V6 branch that does the EFI_D_* to DEBUG_*
> changes, the OPTIONAL keyword
> > style changes, and the uncrustify changes with the one configuration
> change for assignment alignment.
> >
> >
> https://github.com/mdkinney/edk2/tree/Bug_3737_3760_3739_ApplyUncru
> stifyChanges_V6
> >
> > Please provide feedback on the code style in this branch with the known
> DEBUG_CODE() issue still
> > present.
> >
> > If we are able to quickly update uncrustify to handle DEBUG_CODE(), I will
> generate a V7.
> >
> > Thanks,
> >
> > Mike
> >
> > > -----Original Message-----
> > > From: Kinney, Michael D <michael.d.kinney@intel.com>
> > > Sent: Thursday, December 2, 2021 4:53 PM
> > > To: Michael Kubacki <mikuback@linux.microsoft.com>;
> devel@edk2.groups.io; maciej.rabeda@linux.intel.com; Michael Kubacki
> > > <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com)
> <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>;
> > > Kinney, Michael D <michael.d.kinney@intel.com>
> > > Subject: RE: [edk2-devel] Uncrustify Conversion Detailed Plan and
> Extended Hard Freeze Update #4
> > >
> > > Michael,
> > >
> > > Yes. Please update the patch series that adds UncrustifyCheck with
> those changes.
> > >
> > > Thanks,
> > >
> > > Mike
> > >
> > > > -----Original Message-----
> > > > From: Michael Kubacki <mikuback@linux.microsoft.com>
> > > > Sent: Thursday, December 2, 2021 4:31 PM
> > > > To: Kinney, Michael D <michael.d.kinney@intel.com>;
> devel@edk2.groups.io; maciej.rabeda@linux.intel.com; Michael
> > Kubacki
> > > > <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com)
> <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>
> > > > Subject: Re: [edk2-devel] Uncrustify Conversion Detailed Plan and
> Extended Hard Freeze Update #4
> > > >
> > > > Hi Mike,
> > > >
> > > > Thank you for the detailed analysis and recommendations.
> > > >
> > > > I agree with the recommendations and I will try to have an Uncrustify
> > > > tool update by tomorrow for (4).
> > > >
> > > > That will require an update in uncrustify_ext_dep.yaml to pull in the
> > > > new version. I'm assuming I should also update uncrustify.cfg to set
> > > > align_assign_thresh = 0 in that new patch series?
> > > >
> > > > Regards,
> > > > Michael
> > > >
> > > > On 12/2/2021 7:18 PM, Kinney, Michael D wrote:
> > > > > Hi Michael,
> > > > >
> > > > > CORRECTION: set align_assign_threshold to 0.
> > > > >
> > > > > Reponses inline below.
> > > > >
> > > > > I would like to summarize the 4 issues raised in the past day along
> with the recommendations.
> > > > >
> > > > > 1) Exclusion feature for UncrustifyCheck. There are 2 directories with
> 8 files total that
> > > > > maintainers have noted they would like to see not go through
> uncrustify formatting. Today
> > > > > the only content that is skipped is BaseTools and submodules.
> > > > >
> > > > > Adding a general purpose exclusion feature would then require all
> developers to make
> > > > > sure their method of using uncrustify also excludes those same
> areas. This requires
> > > > > extra steps for all developers and maintainers.
> > > > >
> > > > > If we do not add the exclusion feature, then the 8 files will require
> an extra step
> > > > > to sync with the original source of those files. The rate of changes
> of these 8 files
> > > > > is very low today.
> > > > >
> > > > > RECOMMENDATION: Do not add exclusion feature at this time.
> Revisit if the extra work
> > > > > to maintain the files that would be candidates for exclusions
> increases significantly.
> > > > >
> > > > > 2) Alignment of assignments. The threshold of 4 characters appears
> to be too low and causes
> > > > > source files that are already aligned to become unaligned.
> > > > >
> > > > > RECOMMENDATION: Change threshold to the default value of 0
> which means no limit.
> > > > >
> > > > > align_assign_thresh= 0
> > > > >
> > > > > 3) Alignment of parameters in function declaration not correct. The
> root cause of this
> > > > > is the use of the OPTIONAL keyword. If the OPTIONAL keyword is
> removed, then the
> > > > > alignment is correct. The alignment is also correct if the OPTIONAL
> keyword appears
> > > > > before the ','. If the OPTIONAL keyword appears after the ',', then
> the format is
> > > > > not correct. The OPTIONAL keyword indicates that the parameter
> in the function is
> > > > > not required and may be passed in as NULL or 0 or some other
> default value defined by
> > > > > the API. It makes more sense for this OPTIONAL keyword that
> follows the parameter
> > > > > names to appear before the ',' so it is scoped to the parameter on
> that line. If it
> > > > > appears after the ',', then C parsers thinks it is a prefix (IN, OUT,
> CONST, volatile,
> > > > > static) for the next parameter in the function.
> > > > >
> > > > > RECOMMENDATION: Update patch series with a global search and
> replace so OPTIONAL
> > > > > keyword always appears before the ',' on the same line.
> > > > >
> > > > > RegEx search string: ',( *)OPTIONAL( *)'
> > > > > RegEx replace string: ' $1OPTIONAL,$2'
> > > > >
> > > > > 4) Format issues with complex blocks in DEBUG_CODE(), or between
> DEBUG_CODE_BEGIN() and
> > > > > DEBUG_CODE_END(). Uncrustify treats these as macros and is not
> aware that the
> > > > > parameter passed into the macro call is a block of C code that needs
> to be formatted.
> > > > > Complex blocks with if/while/for/case statements are impacted the
> most.
> > > > >
> > > > > RECOMMENDATION: Update the uncrustify with an edk2 specific
> extension to treat these
> > > > > macros as a block of code as if they were surrounded by an extra
> set of braces {}.
> > > > >
> > > > >
> > > > > I have posted a branch for testing purposes that implements (2) and
> (3).
> > > > >
> > > > > Branch:
> https://github.com/mdkinney/edk2/tree/TestOnly_Bug_3737_3739_ApplyU
> ncrustifyChanges_V6_OPTIONAL_Keyword_Fix
> > > > > PR: https://github.com/tianocore/edk2/pull/2233
> > > > > Status: PASS
> > > > > CompareBuild:
> https://github.com/mdkinney/edk2/actions/runs/1532855914
> > > > > Status: PASS
> > > > >
> > > > > You can see what changed by fetching and comparing the following 2
> branches:
> > > > >
> > > > >
> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyC
> hanges_V5
> > > > >
> https://github.com/mdkinney/edk2/tree/TestOnly_Bug_3737_3739_ApplyU
> ncrustifyChanges_V6_OPTIONAL_Keyword_Fix
> > > > >
> > > > > Please provide feedback on the RECOMMENDATIONS above. I will go
> ahead and prepare of V6 version of
> > > > > the patch series now that that test results are all PASS.
> > > > >
> > > > > Best regards,
> > > > >
> > > > > Mike
> > > > >
> > > > >
> > > > >> -----Original Message-----
> > > > >> From: Kinney, Michael D <michael.d.kinney@intel.com>
> > > > >> Sent: Thursday, December 2, 2021 4:15 PM
> > > > >> To: Michael Kubacki <mikuback@linux.microsoft.com>;
> devel@edk2.groups.io; maciej.rabeda@linux.intel.com; Michael
> > > > Kubacki
> > > > >> <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com)
> <afish@apple.com>; Leif Lindholm
> > <leif@nuviainc.com>;
> > > > >> Kinney, Michael D <michael.d.kinney@intel.com>
> > > > >> Subject: RE: [edk2-devel] Uncrustify Conversion Detailed Plan and
> Extended Hard Freeze Update #4
> > > > >>
> > > > >> Hi Michael,
> > > > >>
> > > > >> Reponses inline below.
> > > > >>
> > > > >> I would like to summarize the 4 issues raised in the past day along
> with the recommendations.
> > > > >>
> > > > >> 1) Exclusion feature for UncrustifyCheck. There are 2 directories
> with 8 files total that
> > > > >> maintainers have noted they would like to see not go through
> uncrustify formatting. Today
> > > > >> the only content that is skipped is BaseTools and submodules.
> > > > >>
> > > > >> Adding a general purpose exclusion feature would then require all
> developers to make
> > > > >> sure their method of using uncrustify also excludes those same
> areas. This requires
> > > > >> extra steps for all developers and maintainers.
> > > > >>
> > > > >> If we do not add the exclusion feature, then the 8 files will require
> an extra step
> > > > >> to sync with the original source of those files. The rate of changes
> of these 8 files
> > > > >> is very low today.
> > > > >>
> > > > >> RECOMMENDATION: Do not add exclusion feature at this time.
> Revisit if the extra work
> > > > >> to maintain the files that would be candidates for exclusions
> increases significantly.
> > > > >>
> > > > >> 2) Alignment of assignments. The threshold of 4 characters appears
> to be too low and causes
> > > > >> source files that are already aligned to become unaligned.
> > > > >>
> > > > >> RECOMMENDATION: Change threshold to the default value of 0
> which means no limit.
> > > > >>
> > > > >> align_assign_thresh= 4
> > > > >>
> > > > >> 3) Alignment of parameters in function declaration not correct. The
> root cause of this
> > > > >> is the use of the OPTIONAL keyword. If the OPTIONAL keyword is
> removed, then the
> > > > >> alignment is correct. The alignment is also correct if the OPTIONAL
> keyword appears
> > > > >> before the ','. If the OPTIONAL keyword appears after the ',', then
> the format is
> > > > >> not correct. The OPTIONAL keyword indicates that the parameter
> in the function is
> > > > >> not required and may be passed in as NULL or 0 or some other
> default value defined by
> > > > >> the API. It makes more sense for this OPTIONAL keyword that
> follows the parameter
> > > > >> names to appear before the ',' so it is scoped to the parameter on
> that line. If it
> > > > >> appears after the ',', then C parsers thinks it is a prefix (IN, OUT,
> CONST, volatile,
> > > > >> static) for the next parameter in the function.
> > > > >>
> > > > >> RECOMMENDATION: Update patch series with a global search and
> replace so OPTIONAL
> > > > >> keyword always appears before the ',' on the same line.
> > > > >>
> > > > >> RegEx search string: ',( *)OPTIONAL( *)'
> > > > >> RegEx replace string: ' $1OPTIONAL,$2'
> > > > >>
> > > > >> 4) Format issues with complex blocks in DEBUG_CODE(), or between
> DEBUG_CODE_BEGIN() and
> > > > >> DEBUG_CODE_END(). Uncrustify treats these as macros and is not
> aware that the
> > > > >> parameter passed into the macro call is a block of C code that
> needs to be formatted.
> > > > >> Complex blocks with if/while/for/case statements are impacted
> the most.
> > > > >>
> > > > >> RECOMMENDATION: Update the uncrustify with an edk2 specific
> extension to treat these
> > > > >> macros as a block of code as if they were surrounded by an extra
> set of braces {}.
> > > > >>
> > > > >>
> > > > >> I have posted a branch for testing purposes that implements (2) and
> (3).
> > > > >>
> > > > >> Branch:
> https://github.com/mdkinney/edk2/tree/TestOnly_Bug_3737_3739_ApplyU
> ncrustifyChanges_V6_OPTIONAL_Keyword_Fix
> > > > >> PR: https://github.com/tianocore/edk2/pull/2233
> > > > >> Status: PASS
> > > > >> CompareBuild:
> https://github.com/mdkinney/edk2/actions/runs/1532855914
> > > > >> Status: PASS
> > > > >>
> > > > >> You can see what changed by fetching and comparing the following 2
> branches:
> > > > >>
> > > > >>
> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyC
> hanges_V5
> > > > >>
> https://github.com/mdkinney/edk2/tree/TestOnly_Bug_3737_3739_ApplyU
> ncrustifyChanges_V6_OPTIONAL_Keyword_Fix
> > > > >>
> > > > >> Please provide feedback on the RECOMMENDATIONS above. I will
> go ahead and prepare of V6 version of
> > > > >> the patch series now that that test results are all PASS.
> > > > >>
> > > > >> Best regards,
> > > > >>
> > > > >> Mike
> > > > >>
> > > > >>
> > > > >>> -----Original Message-----
> > > > >>> From: Michael Kubacki <mikuback@linux.microsoft.com>
> > > > >>> Sent: Thursday, December 2, 2021 1:57 PM
> > > > >>> To: devel@edk2.groups.io; Kinney, Michael D
> <michael.d.kinney@intel.com>; maciej.rabeda@linux.intel.com; Michael
> > > > Kubacki
> > > > >>> <michael.kubacki@microsoft.com>; Andrew Fish
> (afish@apple.com) <afish@apple.com>; Leif Lindholm
> > <leif@nuviainc.com>
> > > > >>> Subject: Re: [edk2-devel] Uncrustify Conversion Detailed Plan and
> Extended Hard Freeze Update #4
> > > > >>>
> > > > >>> My reply is inline.
> > > > >>>
> > > > >>> Regards,
> > > > >>> Michael
> > > > >>>
> > > > >>> On 12/2/2021 2:45 PM, Michael D Kinney wrote:
> > > > >>>> Hi Maciej,
> > > > >>>>
> > > > >>>> Thanks for the feedback.
> > > > >>>>
> > > > >>>> * Example #1.This appears to be caused by the following
> uncrustify
> > > > >>>> settings:
> > > > >>>>
> > > > >>>> # The threshold for aligning on '=' in assignments.
> > > > >>>>
> > > > >>>> # Use a negative number for absolute thresholds.
> > > > >>>>
> > > > >>>> #
> > > > >>>>
> > > > >>>> # 0: No limit (default).
> > > > >>>>
> > > > >>>> align_assign_thresh= 0# number
> > > > >>>>
> > > > >>>> The edk2 setting for this is:
> > > > >>>>
> > > > >>>> align_assign_thresh= 4
> > > > >>>>
> > > > >>>> This means blocks of assignments that are different than more
> than 4
> > > > >>>> spaces will be considered a new block.
> > > > >>>>
> > > > >>>> ‘HwAddreLen’ and ‘Xid’ are more than 4 characters in length
> > > > >>>> different.Same for ‘Xid’ and ‘Reserved’.So
> > > > >>>>
> > > > >>>> uncrustify treats these as 3 different assignment blocks.
> > > > >>>>
> > > > >>>> If we change to the default value of 0: No limit, this example is
> > > > >>>> treated as a single block and all ‘=’ are aligned.
> > > > >>>>
> > > > >>>> Is there a reason ‘4’ was selected?Is there is any harm in using the
> > > > >>>> default of 0?
> > > > >>>>
> > > > >>> We can certainly change the threshold. '4' was derived by
> > > > >>> experimentation. This is an area that is somewhat subjective and
> every
> > > > >>> case is difficult to cover well. Please feel free to suggest any
> changes.
> > > > >>>
> > > > >>
> > > > >> I recommend we use the default value of 0. That way, any code that
> choose to
> > > > >> align assignments will still be aligned. If a developer does not like
> > > > >> short assignments and long assignments in the same code block to
> use the
> > > > >> long assignment column, they can always break the block up into
> multiple
> > > > >> blocks by adding a carriage return.
> > > > >>
> > > > >>> For the benefit of others, this file can be a useful refernce
> > > > >>> understanding spans, gaps, and thresholds.
> > > > >>>
> > > > >>>
> https://github.com/uncrustify/uncrustify/blob/master/documentation/htdo
> cs/configuration.txt
> > > > >>>
> > > > >>>> * Example #2: Uncruistfy is confused by the DEBUG_CODE()
> macro.This is
> > > > >>>> not a traditional macro because the
> > > > >>>>
> > > > >>>> contents of the macro is a block of C code.I do not know how to
> convince
> > > > >>>> uncrustify that the contents of a
> > > > >>>>
> > > > >>>> macro like function call to be treated as a code block from an
> indent
> > > > >>>> perspective.
> > > > >>>>
> > > > >>> I believe this would have to be overridden in the Uncrustify fork
> since
> > > > >>> DEBUG_CODE() is being treated as a macro function call and code
> blocks
> > > > >>> are not expected there. In addition, some special treatment might
> be
> > > > >>> needed for alignment in between
> DEBUG_CODE_BEGIN()/DEBUG_CODE_END().
> > > > >>>
> > > > >>> I'm happy to look at this. However, regression validation can take a
> > > > >>> while so I'd like to make sure we need this now. In cases that do
> not
> > > > >>> have additional code blocks, it seems to format fairly well. Is this
> > > > >>> prevalent and impactful enough it must be fixed now? Or, could we
> > > > >>> revisit it with a follow up patch?
> > > > >>
> > > > >> How long does regression testing take? I see about 20 instances of
> > > > >> DEBUG_CODE() that are producing bad formatting. If the content
> inside
> > > > >> DEBUG_CODE() is a single line of a single block of statements
> without
> > > > >> any if/while/for/case statements that require further indent, then
> the
> > > > >> format looks ok.
> > > > >>
> > > > >> DEBUG_CODE_BEGIN() and DEBUG_CODE_END() would look better
> if the
> > > > >> contents between are also indented one level.
> > > > >>
> > > > >>>
> > > > >>>> * Example #3/#4: Uncrustify is confused by the OPTIONAL
> keyword.The
> > > > >>>> edk2 config declares it as a QUALIFIER
> > > > >>>>
> > > > >>>> like IN and OUT.However, OPTIONAL only appears at the end of
> the line
> > > > >>>> that declares a parameter to a
> > > > >>>>
> > > > >>>> function.There are 3 forms. One with comma after
> OPTIONAL.One with comma
> > > > >>>> before OPTIONAL, and
> > > > >>>>
> > > > >>>> one with no comma if the parameter is the last parameter in the
> function
> > > > >>>> declaration.
> > > > >>>>
> > > > >>>> TYPE ParamName OPTIONAL,
> > > > >>>>
> > > > >>>> TYPEParamName, OPTIONAL
> > > > >>>>
> > > > >>>> TYPEParamName OPTIONAL
> > > > >>>>
> > > > >>>> OPTIONAL is defined to nothing in edk2 builds.From a uncrustify
> > > > >>>> perspective, we really want is to be
> > > > >>>>
> > > > >>>> ignored or more correctly treated it as a token that is attached to
> > > > >>>> ParamName and the combination of
> > > > >>>>
> > > > >>>> ParamName and OPTIONAL treated as one unit to determine
> indents.
> > > > >>>>
> > > > >>> "TYPE ParamName, OPTIONAL" seems especially problematic and
> presents an
> > > > >>> inconsistency with the other formats. Mike, can you please let me
> know
> > > > >>> if you have the same observation and your thoughts on a
> consistent pattern?
> > > > >>>
> > > > >>>> Thanks,
> > > > >>>>
> > > > >>>> Mike
> > > > >>>>
> > > > >>>> *From:*devel@edk2.groups.io <devel@edk2.groups.io> *On
> Behalf Of *Maciej
> > > > >>>> Rabeda
> > > > >>>> *Sent:* Thursday, December 2, 2021 10:27 AM
> > > > >>>> *To:* devel@edk2.groups.io; Kinney, Michael D
> > > > >>>> <michael.d.kinney@intel.com>; Michael Kubacki
> > > > >>>> <michael.kubacki@microsoft.com>; Andrew Fish
> (afish@apple.com)
> > > > >>>> <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>
> > > > >>>> *Subject:* Re: [edk2-devel] Uncrustify Conversion Detailed Plan
> and
> > > > >>>> Extended Hard Freeze Update #4
> > > > >>>>
> > > > >>>> Hey Mike,
> > > > >>>>
> > > > >>>> While most of the changes related to fixing coding style violations,
> > > > >>>> there are a couple of changes to NetworkPkg in that PR that make
> the
> > > > >>>> code less readable. Examples below.
> > > > >>>>
> > > > >>>> Example 1:
> > > > >>>>
> > > > >>>>
> > > > >>>>
> > > > >>>> Example 2:
> > > > >>>>
> > > > >>>>
> > > > >>>> Example 3:
> > > > >>>>
> > > > >>>>
> > > > >>>> Example 4:
> > > > >>>>
> > > > >>>> On 30-Nov-21 23:34, Michael D Kinney wrote:
> > > > >>>>
> > > > >>>> Hello,
> > > > >>>>
> > > > >>>> Thank you for your patience during this extended hard freeze.
> > > > >>>>
> > > > >>>> Just one more step to go.There has been a delay in the review
> of
> > > > >>>>
> > > > >>>> the patch series with the uncrustify source changes.PR(6).This
> > > > >>>>
> > > > >>>> patch series was not sent out as patch review email because of
> its
> > > > >>>>
> > > > >>>> very large size.It only contains source style changes and the
> > > > >>>>
> > > > >>>> CompareBuild tool and GitHub action has shown there are no
> binary
> > > > >>>>
> > > > >>>> differences introduced with these source style changes.
> > > > >>>>
> > > > >>>> If you are a package maintainer, then please review the
> following
> > > > >>>>
> > > > >>>> branch/PR for your package contents and review the EDK II CI
> results
> > > > >>>>
> > > > >>>> and BuildCompare results.I do not expect a line by line review
> > > > >>>>
> > > > >>>> because we already had time to provide feedback on the
> source style
> > > > >>>>
> > > > >>>> performed by uncrustify.Instead, a Reviewed-by for your
> package
> > > > >>>>
> > > > >>>> indicates that you have reviewed the EDK II CI results and
> CompareBuild
> > > > >>>>
> > > > >>>> tool functionality and results and you accept the source style
> > > > >>>>
> > > > >>>> changes to your package.
> > > > >>>>
> > > > >>>>
> *https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustify
> Changes_V5
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustify
> Changes_V5>
> > > > >>>>
> > > > >>>> *https://github.com/tianocore/edk2/pull/2229
> <https://github.com/tianocore/edk2/pull/2229>
> > > > >>>>
> > > > >>>> *https://github.com/mdkinney/edk2/actions/runs/1521618836
> > > > >>> <https://github.com/mdkinney/edk2/actions/runs/1521618836>
> > > > >>>>
> > > > >>>> Additional details on this update below.
> > > > >>>>
> > > > >>>> Thank you,
> > > > >>>>
> > > > >>>> Mike
> > > > >>>>
> > > > >>>> Changes from Update #3
> > > > >>>>
> > > > >>>> ----------------------------------------------------------------------------
> > > > >>>>
> > > > >>>> * Pushed PR (5)
> > > > >>>>
> > > > >>>> * Added link to PR(6). EDK II CI Status is PASS. Build Compare
> PASS.
> > > > >>>>
> > > > >>>> * Waiting for review of PR (6)
> > > > >>>>
> > > > >>>> * Review of PR (7) completed and waiting for review of PR (6)
> > > > >>>>
> > > > >>>> ----------------------------------------------------------------------------
> > > > >>>>
> > > > >>>> Changes from Update #2
> > > > >>>>
> > > > >>>> ----------------------------------------------------------------------------
> > > > >>>>
> > > > >>>> * Changed order of PRs swapping (4) and (5).The PR that
> activates
> > > > >>>>
> > > > >>>> increases the max CI agent job time is independent of all the
> other
> > > > >>>>
> > > > >>>> PRs and its review is complete, so it can be committed now.
> > > > >>>>
> > > > >>>> * Pushed PRs (1), (2), (3), (4).
> > > > >>>>
> > > > >>>> * Waiting for review to complete for PRs (5) and (6)
> > > > >>>>
> > > > >>>> * Reviews complete for PR (7)
> > > > >>>>
> > > > >>>> * Identifies steps using git filter-branch to apply uncrustify
> changes to a
> > > > >>>>
> > > > >>>> code review patch series that was generated before the
> uncrustify changes
> > > > >>>>
> > > > >>>> avoiding manual merge.
> > > > >>>>
> > > > >>>> * Identified steps using git filter-repo to generate an alternate
> history of
> > > > >>>>
> > > > >>>> the edk2 repo with uncrustify changes applied on every
> commit.This may
> > > > >>>>
> > > > >>>> be useful when evaluating changes to files using tools like git
> blame
> > > > >>>>
> > > > >>>> without the large uncrustify patch series.
> > > > >>>>
> > > > >>>> ---------------------------------------------------------------------------
> > > > >>>>
> > > > >>>> Changes from Update #1
> > > > >>>>
> > > > >>>> ----------------------------------------------------------------------------
> > > > >>>>
> > > > >>>> * Changed order of PRs swapping (6) and (7).The PR that
> activates
> > > > >>>>
> > > > >>>> EDK II CI check UncrustifyCheck has to be last because it
> unconditionally
> > > > >>>>
> > > > >>>> checks all C/H files in all packages.Not just files that have been
> > > > >>>>
> > > > >>>> modified like some of the other checkers.
> > > > >>>>
> > > > >>>> * Updated link to the branch with the UncrustifyCheck plugin
> that has been
> > > > >>>>
> > > > >>>> updated with a one line change and Reviewed-by and Tested-
> by tags.
> > > > >>>>
> > > > >>>>
> https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plug
> in_v6
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plu
> gin_v6>
> > > > >>>>
> > > > >>>> * Reviews complete for (1), (2), (3), (5), and (7)
> > > > >>>>
> > > > >>>> ---------------------------------------------------------------------------
> > > > >>>>
> > > > >>>> Michael Kubacki and I have prepared the patches required to
> apply the
> > > > >>>>
> > > > >>>> uncrustify changes and enable EDK II CI to check all submitted
> > > > >>>>
> > > > >>>> patches have been run through uncrustify.
> > > > >>>>
> > > > >>>> We have verified through the CompareBuild GitHub Action that
> the
> > > > >>>>
> > > > >>>> format changes performed by uncrustify have no functional
> changes.
> > > > >>>>
> > > > >>>> All of the OBJ, LIB, DLL, EFI, FFS, FV, and FD files match 100%
> > > > >>>>
> > > > >>>> across 70 VS2019/GCC5 builds of all package/platform DSC files
> in
> > > > >>>>
> > > > >>>> the edk2 repo.
> > > > >>>>
> > > > >>>> The hard freeze will be extended after the edk2-stable202111
> tag until
> > > > >>>>
> > > > >>>> all uncrustify related changes are committed.We do not expect
> this
> > > > >>>>
> > > > >>>> to take more than a few days.Do not push any PRs until the
> hard
> > > > >>>>
> > > > >>>> freeze is lifted.
> > > > >>>>
> > > > >>>> The changes are broken up into 7 patch series/PRs.The PRs are
> ordered
> > > > >>>>
> > > > >>>> so they can be submitted using the normal submission process
> and EDK II
> > > > >>>>
> > > > >>>> CI will pass for each one.Details are listed below.
> > > > >>>>
> > > > >>>> Uncrustify 73.0.3 for EDK II
> > > > >>>>
> > > > >>>> =============================
> > > > >>>>
> > > > >>>> *
> Sources:INVALID URI REMOVED
> it/Uncrustify__;!!NpxR!wyxWLyTo0gh2I5SrYc7-gsYq-
> rGWWEEM7gUPYPsGRkd-czEU0wq0W6cMJgY37oA$
> <INVALID URI REMOVED
> ustify__;!!NpxR!wyxWLyTo0gh2I5SrYc7-gsYq-rGWWEEM7gUPYPsGRkd-
> czEU0wq0W6cMJgY37oA$ >
> > > > >>>>
> > > > >>>> *
> Documentation:INVALID URI REMOVED
> tmu/Uncrustify/_wiki/wikis/Uncrustify.wiki/1/Project-Mu-(EDK-
> __;!!NpxR!wyxWLyTo0gh2I5SrYc7-gsYq-rGWWEEM7gUPYPsGRkd-
> czEU0wq0W6cMvGARQHs$
> > II)-
> > > > Fork-
> > > > >>> Readme
> <INVALID URI REMOVED
> y/_wiki/wikis/Uncrustify.wiki/1/Project-Mu-(EDK-II)-Fork-
> Readme__;!!NpxR!wyxWLyTo0gh2I5SrYc7-gsYq-rGWWEEM7gUPYPsGRkd-
> czEU0wq0W6cMSt11EtE$ >
> > > > >>>>
> > > > >>>> *
> Download:INVALID URI REMOVED
> Uncrustify/_packaging?_a=package&feed=mu_uncrustify&package=mu-
> __;!!NpxR!wyxWLyTo0gh2I5SrYc7-gsYq-rGWWEEM7gUPYPsGRkd-
> czEU0wq0W6cMSysRS-o$
> > > > >>> uncrustify-release&protocolType=NuGet&version=73.0.3
> > > > >>>
> <INVALID URI REMOVED
> y/_packaging?_a=package&feed=mu_uncrustify&package=mu-uncrustify-
> __;!!NpxR!wyxWLyTo0gh2I5SrYc7-gsYq-rGWWEEM7gUPYPsGRkd-
> czEU0wq0W6cMmLuf_PM$
> > > > >>> release&protocolType=NuGet&version=73.0.3>
> > > > >>>>
> > > > >>>> Installing Uncrustify
> > > > >>>>
> > > > >>>> ======================
> > > > >>>>
> > > > >>>> The Uncrustify tool is installed automatically when the Pytools
> > > > >>>>
> > > > >>>> environment is used and the stuart* commands are run to
> complete the
> > > > >>>>
> > > > >>>> environment setup.Please see:
> > > > >>>>
> > > > >>>>
> https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally
> > > > >>>
> <https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-
> locally>
> > > > >>>>
> > > > >>>> Uncrustify can also be installed from the download page listed
> above
> > > > >>>>
> > > > >>>> or built from sources from the source link above.
> > > > >>>>
> > > > >>>> The Documentation link provides instruction on how to run
> uncrustify from
> > > > >>>>
> > > > >>>> the command line or install as a Visual Studio Code plugin.The
> main
> > > > >>>>
> > > > >>>> uncrustify documentation also describes how to integrate with
> a few other
> > > > >>>>
> > > > >>>> editors.
> > > > >>>>
> > > > >>>> We have also discussed a client side githook.That effort has not
> started.
> > > > >>>>
> > > > >>>> Let us know if that is a feature you would find useful.
> > > > >>>>
> > > > >>>> Developer impact for new code reviews
> > > > >>>>
> > > > >>>> ======================================
> > > > >>>>
> > > > >>>> Once the uncrustify checker is active in EDK II CI, developers
> must
> > > > >>>>
> > > > >>>> make sure their patches are run through the uncrustify tool
> before
> > > > >>>>
> > > > >>>> sending the patches for review.
> > > > >>>>
> > > > >>>> Developers must install and run uncrustify against changes files
> before
> > > > >>>>
> > > > >>>> sending patch review emails or submitting PR for EDK II CI.If
> EDK II CI
> > > > >>>>
> > > > >>>> detects and differences in source formatting, then EDK II CI will
> fail
> > > > >>>>
> > > > >>>> and the developer must run uncrustify and resubmit the
> patches.
> > > > >>>>
> > > > >>>> Developer impact to patch series/PRs reviewed during edk2-
> stable201121 soft/hard freeze
> > > > >>>>
> > > > >>>>
> ==========================================================
> =============================
> > > > >>>>
> > > > >>>> Developers must rebase their changes after the uncrustify
> source changes are
> > > > >>>>
> > > > >>>> committed.The branch with a preview of the uncrustify
> changes can be used
> > > > >>>>
> > > > >>>> to start this rebase work.
> > > > >>>>
> > > > >>>>
> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyC
> hanges_V5
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustify
> Changes_V5>
> > > > >>>>
> > > > >>>> The following steps can be used to update an existing branch
> with the
> > > > >>>>
> > > > >>>> required uncrustify format.This is the Windows version.I will
> add
> > > > >>>>
> > > > >>>> the Linux version soon.
> > > > >>>>
> > > > >>>> 1) Fetch and checkout and rebase to latest edk2/master
> > > > >>>>
> > > > >>>> git fetch origin
> > > > >>>>
> > > > >>>> git checkout master
> > > > >>>>
> > > > >>>> git rebase origin/master
> > > > >>>>
> > > > >>>> 2) Make a backup copy of plugin UncrustifyCheck outside
> WORKSPACE.
> > > > >>>>
> > > > >>>> (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool
> executable and
> > > > >>>>
> > > > >>>> EDK II specific uncrustify configuration file available when
> working
> > > > >>>>
> > > > >>>> with a branch that does not have those tools in its scope.
> > > > >>>>
> > > > >>>> xcopy .pytool\Plugin\UncrustifyCheck
> C:\Temp\UncrustifyCheck
> > > > >>>>
> > > > >>>> 3) Check out the patch series branch (e.g. MyBranch)
> > > > >>>>
> > > > >>>> git checkout MyBranch
> > > > >>>>
> > > > >>>> 4) Rebase patch series against edk2-stable202111
> > > > >>>>
> > > > >>>>
> > > > >>>>
> > > > >>>> git rebase edk2-stable202111
> > > > >>>>
> > > > >>>> 5) Create new branch for the uncrustifed version (e.g.
> MyBranch_Uncrustified)
> > > > >>>>
> > > > >>>> git checkout -b MyBranch_Uncrustified
> > > > >>>>
> > > > >>>> 6) Use git filter-branch to uncrustify all the commits in the
> series
> > > > >>>>
> > > > >>>> between the rebase target from (2) and HEAD of the branch.A
> filter
> > > > >>>>
> > > > >>>> can be used to scope the uncrustify operations to only the C/H
> files
> > > > >>>>
> > > > >>>> in the specific package the patch series is against. (e.g.
> DynamicTablesPkg).
> > > > >>>>
> > > > >>>> BaseTools should always be excluded.If the package scoped
> filter is
> > > > >>>>
> > > > >>>> not used, it will still work, but will take longer to run because
> > > > >>>>
> > > > >>>> uncrustify will rescan every C/H files in the whole repo.
> > > > >>>>
> > > > >>>> git filter-branch --tree-filter "git ls-files DynamicTablesPkg*.c
> DynamicTablesPkg*.h :!BaseTools/* |
> > > > >>> c:\\Temp\\UncrustifyCheck\\mu-uncrustify-
> release_extdep\\Windows-x86\\uncrustify.exe -c
> > > > >>> c:\\Temp\\UncrustifyCheck\\uncrustify.cfg -F - --replace --no-
> backup --if-changed" edk2-stable202111..HEAD
> > > > >>>>
> > > > >>>> 7) Now that all the individual patches in the branch are
> uncrustified,
> > > > >>>>
> > > > >>>> rebase against latest edk2/master that is already uncrustified.
> > > > >>>>
> > > > >>>> git rebase master
> > > > >>>>
> > > > >>>> 8) Verify the patches in this new branch.
> > > > >>>>
> > > > >>>> Impacts to tracing history across the uncrusity changes
> > > > >>>>
> > > > >>>>
> =======================================================
> > > > >>>>
> > > > >>>> Tools the view file and line history do work with the large
> uncrustify
> > > > >>>>
> > > > >>>> patch series.One impact is that the operations can be very slow
> due
> > > > >>>>
> > > > >>>> to the large uncrustify patches.
> > > > >>>>
> > > > >>>> One option to provide a faster experience is to provide an
> alternate
> > > > >>>>
> > > > >>>> version of the edk2 repository as "documentation" that has the
> > > > >>>>
> > > > >>>> entire history re-written with uncrustify run on every commit.
> > > > >>>>
> > > > >>>> The tool called git-filter-repo can be used to perform this
> > > > >>>>
> > > > >>>> transformation and runs in a reasonable period of time (a few
> hours)
> > > > >>>>
> > > > >>>> https://github.com/newren/git-filter-repo
> <https://github.com/newren/git-filter-repo>
> > > > >>>>
> > > > >>>> https://github.com/newren/git-filter-
> repo/blob/main/contrib/filter-repo-demos/lint-history
> > > > >>> <https://github.com/newren/git-filter-
> repo/blob/main/contrib/filter-repo-demos/lint-history>
> > > > >>>>
> > > > >>>> The following steps can be used to perform this transformation.
> > > > >>>>
> > > > >>>> This is the Windows version. I will add the Linux version soon.
> > > > >>>>
> > > > >>>> ** WARNING **This operation modifies(rewrites) all the
> commits
> > > > >>>>
> > > > >>>> in the local copy of the repo.Do not perform
> > > > >>>>
> > > > >>>> these steps on a local repo you are using for
> > > > >>>>
> > > > >>>> active development.
> > > > >>>>
> > > > >>>> 1) Clone edk2 into a new directory (see **WARNING**)
> > > > >>>>
> > > > >>>> git clonehttps://github.com/tianocore/edk2.git
> <https://github.com/tianocore/edk2.git> edk2-uncrustified
> > > > >>>>
> > > > >>>> cd edk2-uncrustified
> > > > >>>>
> > > > >>>> 2) Setup python virtual env, install pytools, and run stuart
> commands
> > > > >>>>
> > > > >>>> to setup build environment which includes installing uncrustify
> tools.
> > > > >>>>
> > > > >>>>
> https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally
> > > > >>>
> <https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-
> locally>
> > > > >>>>
> > > > >>>> 3) Make a backup copy of plugin UncrustifyCheck outside
> WORKSPACE.
> > > > >>>>
> > > > >>>> (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool
> executable and
> > > > >>>>
> > > > >>>> EDK II specific uncrustify configuration file available when
> working
> > > > >>>>
> > > > >>>> with a branch that does not have those tools in its scope.
> > > > >>>>
> > > > >>>> xcopy .pytool\Plugin\UncrustifyCheck
> C:\Temp\UncrustifyCheck
> > > > >>>>
> > > > >>>> 4) Use lint-history.py from git-filter-repo examples
> > > > >>>>
> > > > >>>> https://github.com/newren/git-filter-repo
> <https://github.com/newren/git-filter-repo>
> > > > >>>>
> > > > >>>> https://github.com/newren/git-filter-
> repo/blob/main/contrib/filter-repo-demos/lint-history
> > > > >>> <https://github.com/newren/git-filter-
> repo/blob/main/contrib/filter-repo-demos/lint-history>
> > > > >>>>
> > > > >>>> Line #127 - Add try except around subprocess.check_call() with
> except
> > > > >>>>
> > > > >>>> being pass.This is required because there are a few commits of
> C
> > > > >>>>
> > > > >>>> files in the edk2 repo that have incorrect C syntax and do not
> > > > >>>>
> > > > >>>> build with a C compiler and break the uncrustify parser.Skip
> reformat
> > > > >>>>
> > > > >>>> of C files that can not be parsed by uncrustify.These rare
> instances
> > > > >>>>
> > > > >>>> are addressed in the commit that fixes the C syntax error.
> > > > >>>>
> > > > >>>> Run this slightly modified version of lint-history.Include only
> > > > >>>>
> > > > >>>> C/H files and exclude directories that start with 'Tools' or
> 'BaseTools'.
> > > > >>>>
> > > > >>>> This step took about 2.2 hours on a laptop.
> > > > >>>>
> > > > >>>> lint-history.py
> > > > >>>>
> > > > >>>> --relevant "return (not filename.startswith(b'Tools') and not
> filename.startswith(b'BaseTools') and
> > > > >>> (filename.endswith(b'.c') or filename.endswith(b'.h')))"
> > > > >>>>
> > > > >>>> c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\mu-
> uncrustify-release_extdep\\Windows-x86\\uncrustify.exe
> > -
> > > c
> > > > >>>
> c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\uncrustify.cfg --
> replace --no-backup --if-changed
> > > > >>>>
> > > > >>>> Order of PRs to apply during extended hard freeze
> > > > >>>>
> > > > >>>>
> ==================================================
> > > > >>>>
> > > > >>>> 1) Update EmulatorPkg Win Host [BuildOptions] MSFT
> CC_FLAGS to not force debug information
> > > > >>>>
> > > > >>>>
> *INVALID URI REMOVED
> ?id=3747__;!!NpxR!wyxWLyTo0gh2I5SrYc7-gsYq-rGWWEEM7gUPYPsGRkd-
> czEU0wq0W6cMJoceXwM$
> <INVALID URI REMOVED
> ?id=3747__;!!NpxR!wyxWLyTo0gh2I5SrYc7-gsYq-rGWWEEM7gUPYPsGRkd-
> czEU0wq0W6cMJoceXwM$ >
> > > > >>>>
> > > > >>>>
> *https://github.com/mdkinney/edk2/tree/Bug_3747_EmulatorPkg_WinHost
> _ReproducibleBuild
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_3747_EmulatorPkg_WinHost
> _ReproducibleBuild>
> > > > >>>>
> > > > >>>> *https://github.com/tianocore/edk2/pull/2215
> <https://github.com/tianocore/edk2/pull/2215>
> > > > >>>>
> > > > >>>> * Required for EmulatorPkg to pass CompareBuild for VS2019
> IA32/X64 builds.
> > > > >>>>
> > > > >>>> * Status: Review complete.PR pushed.
> > > > >>>>
> > > > >>>> 2) EccCheck should not revert staged and local changes
> > > > >>>>
> > > > >>>>
> *INVALID URI REMOVED
> ?id=2986__;!!NpxR!wyxWLyTo0gh2I5SrYc7-gsYq-rGWWEEM7gUPYPsGRkd-
> czEU0wq0W6cMrRbK35g$
> <INVALID URI REMOVED
> ?id=2986__;!!NpxR!wyxWLyTo0gh2I5SrYc7-gsYq-rGWWEEM7gUPYPsGRkd-
> czEU0wq0W6cMrRbK35g$ >
> > > > >>>>
> > > > >>>>
> *https://github.com/mdkinney/edk2/tree/Bug_2986_EccCheckRemoveGitR
> evert_V2
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_2986_EccCheckRemoveGitR
> evert_V2>
> > > > >>>>
> > > > >>>> *https://github.com/tianocore/edk2/pull/2216
> <https://github.com/tianocore/edk2/pull/2216>
> > > > >>>>
> > > > >>>> * Required for EDK II CI to complete in a reasonable period of
> time when
> > > > >>>>
> > > > >>>> processing the 4000+ source file style changes made by
> uncrustify.
> > > > >>>>
> > > > >>>> * Also fixes critical bugs that can potentially corrupt git state
> when
> > > > >>>>
> > > > >>>> EccCheck is run locally.
> > > > >>>>
> > > > >>>> * Status: Review complete.PR pushed.
> > > > >>>>
> > > > >>>> 3) Update pytool LicenseCheck plugin to use temp directory for
> diff output file
> > > > >>>>
> > > > >>>>
> *INVALID URI REMOVED
> ?id=3746__;!!NpxR!wyxWLyTo0gh2I5SrYc7-gsYq-rGWWEEM7gUPYPsGRkd-
> czEU0wq0W6cMWqBIyO4$
> <INVALID URI REMOVED
> ?id=3746__;!!NpxR!wyxWLyTo0gh2I5SrYc7-gsYq-rGWWEEM7gUPYPsGRkd-
> czEU0wq0W6cMWqBIyO4$ >
> > > > >>>>
> > > > >>>>
> *https://github.com/mdkinney/edk2/tree/Bug_3746_LicenseCheckUseDiff
> OutputFile_V2
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_3746_LicenseCheckUseDiff
> OutputFile_V2>
> > > > >>>>
> > > > >>>> *https://github.com/tianocore/edk2/pull/2217
> <https://github.com/tianocore/edk2/pull/2217>
> > > > >>>>
> > > > >>>> * Required to reduce EDK II CI build times.
> > > > >>>>
> > > > >>>> * Status: Review complete.PR pushed.
> > > > >>>>
> > > > >>>> 4) Update max job time from 60 min to 120 minutes
> in .azurepipelines/templates
> > > > >>>>
> > > > >>>>
> *INVALID URI REMOVED
> ?id=3750__;!!NpxR!wyxWLyTo0gh2I5SrYc7-gsYq-rGWWEEM7gUPYPsGRkd-
> czEU0wq0W6cMuU_Qgec$
> <INVALID URI REMOVED
> ?id=3750__;!!NpxR!wyxWLyTo0gh2I5SrYc7-gsYq-rGWWEEM7gUPYPsGRkd-
> czEU0wq0W6cMuU_Qgec$ >
> > > > >>>>
> > > > >>>>
> *https://github.com/mdkinney/edk2/tree/Bug_3750_IncreaseAzurePipeline
> sTimeout
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_3750_IncreaseAzurePipeline
> sTimeout>
> > > > >>>>
> > > > >>>> *https://github.com/tianocore/edk2/pull/2219
> <https://github.com/tianocore/edk2/pull/2219>
> > > > >>>>
> > > > >>>> * Required to allow EccCheck of uncrustify changes to complete
> on Azure
> > > > >>>>
> > > > >>>> Pipelines CI agents without timing out.
> > > > >>>>
> > > > >>>> * Status: Review complete.PR pushed.
> > > > >>>>
> > > > >>>> 5) Update Package YAML to ignore specific ECC files/errors
> > > > >>>>
> > > > >>>>
> *INVALID URI REMOVED
> ?id=3749__;!!NpxR!wyxWLyTo0gh2I5SrYc7-gsYq-rGWWEEM7gUPYPsGRkd-
> czEU0wq0W6cMtG-6eCA$
> <INVALID URI REMOVED
> ?id=3749__;!!NpxR!wyxWLyTo0gh2I5SrYc7-gsYq-rGWWEEM7gUPYPsGRkd-
> czEU0wq0W6cMtG-6eCA$ >
> > > > >>>>
> > > > >>>>
> *https://github.com/mdkinney/edk2/tree/Bug_3749_EccCheckIgnoreFilesEr
> rors
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_3749_EccCheckIgnoreFilesEr
> rors>
> > > > >>>>
> > > > >>>> *https://github.com/tianocore/edk2/pull/2218
> <https://github.com/tianocore/edk2/pull/2218>
> > > > >>>>
> > > > >>>> * Required to pass EccCheck
> > > > >>>>
> > > > >>>> * Status: Review complete. PR pushed
> > > > >>>>
> > > > >>>> 6) Uncrustify Source Changes
> > > > >>>>
> > > > >>>>
> *INVALID URI REMOVED
> ?id=3737__;!!NpxR!wyxWLyTo0gh2I5SrYc7-gsYq-rGWWEEM7gUPYPsGRkd-
> czEU0wq0W6cMggxM5rY$
> <INVALID URI REMOVED
> ?id=3737__;!!NpxR!wyxWLyTo0gh2I5SrYc7-gsYq-rGWWEEM7gUPYPsGRkd-
> czEU0wq0W6cMggxM5rY$ >
> > > > >>>>
> > > > >>>>
> *INVALID URI REMOVED
> ?id=3739__;!!NpxR!wyxWLyTo0gh2I5SrYc7-gsYq-rGWWEEM7gUPYPsGRkd-
> czEU0wq0W6cM0tCrXqg$
> <INVALID URI REMOVED
> ?id=3739__;!!NpxR!wyxWLyTo0gh2I5SrYc7-gsYq-rGWWEEM7gUPYPsGRkd-
> czEU0wq0W6cM0tCrXqg$ >
> > > > >>>>
> > > > >>>>
> *https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustify
> Changes_V5
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustify
> Changes_V5>
> > > > >>>>
> > > > >>>> *https://github.com/tianocore/edk2/pull/2229
> <https://github.com/tianocore/edk2/pull/2229>
> > > > >>>>
> > > > >>>> * Build comparison result
> PASS:https://github.com/mdkinney/edk2/actions/runs/1521618836
> > > > >>> <https://github.com/mdkinney/edk2/actions/runs/1521618836>
> > > > >>>>
> > > > >>>> * EFI_D_ -> DEBUG changes required to pass PatchCheck
> > > > >>>>
> > > > >>>> * Uncrustify format changes required to pass UncrustifyCheck
> > > > >>>>
> > > > >>>> * Status:
> > > > >>>>
> > > > >>>> Waiting for review
> > > > >>>>
> > > > >>>> 7) UncrustifyCheck EDK II CI Plugin
> > > > >>>>
> > > > >>>>
> *INVALID URI REMOVED
> ?id=3748__;!!NpxR!wyxWLyTo0gh2I5SrYc7-gsYq-rGWWEEM7gUPYPsGRkd-
> czEU0wq0W6cMw172ZYE$
> <INVALID URI REMOVED
> ?id=3748__;!!NpxR!wyxWLyTo0gh2I5SrYc7-gsYq-rGWWEEM7gUPYPsGRkd-
> czEU0wq0W6cMw172ZYE$ >
> > > > >>>>
> > > > >>>>
> *https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plu
> gin_v6
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plu
> gin_v6>
> > > > >>>>
> > > > >>>> * Required to enforce all PRs submitted to EDK II CI match
> uncrustify format.
> > > > >>>>
> > > > >>>> * Unconditionally checks all packages.Can not be committed
> until all C/H
> > > > >>>>
> > > > >>>> source files have been updated.
> > > > >>>>
> > > > >>>> * Status: Review complete
> > > > >>>>
> > > > >>>> Combined Branch/PR for Review/Test
> > > > >>>>
> > > > >>>> ==================================
> > > > >>>>
> > > > >>>> * Build Comparison results must pass 100% across the full set of
> PRs before
> > > > >>>>
> > > > >>>> the individual PRs can be pushed in the order listed above.
> > > > >>>>
> > > > >>>> *
> Branch:https://github.com/mdkinney/edk2/tree/TestOnly_Uncrustify_PR_S
> eries
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/TestOnly_Uncrustify_PR_Series>
> > > > >>>>
> > > > >>>> * PR:https://github.com/tianocore/edk2/pull/2229
> <https://github.com/tianocore/edk2/pull/2229>
> > > > >>>>
> > > > >>>> Status = PASS
> > > > >>>>
> > > > >>>> * CompareBuild:
> > > > >>>>
> > > > >>>>
> Branch:https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUnc
> rustifyChanges_V5
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustify
> Changes_V5>
> > > > >>>>
> > > > >>>> --ref1:ef9a059cdb15844fe52a49af2bf7d86b9dd3e9bf
> > > > >>>>
> > > > >>>> --ref2:Bug_3737_3739_ApplyUncrustifyChanges_V5
> > > > >>>>
> > > > >>>> Extra Options: -n 4 --quiet
> > > > >>>>
> > > > >>>>
> Results:https://github.com/mdkinney/edk2/actions/runs/1521618836
> > > > >>> <https://github.com/mdkinney/edk2/actions/runs/1521618836>
> > > > >>>>
> > > > >>>> 30 VS2019 build comparisons PASS
> > > > >>>>
> > > > >>>> 40 GCC5 build comparisons PASS
> > > > >>>>
> > > > >>>> 100% PASS
> > > > >>>>
> > > > >>>> The following git log shows the set of patches from --ref1 to --
> ref 2across
> > > > >>>>
> > > > >>>> which there are no differences in any of the
> OBJ/LIB/DLL/EFI/FFS/FV/FD files.
> > > > >>>>
> > > > >>>> --ref2
> > > > >>>>
> > > > >>>> b7d4bf0675b7 (HEAD ->
> Bug_3737_3739_ApplyUncrustifyChanges_V5) UnitTestFrameworkPkg:
> Apply uncrusitify
> > changes
> > > > >>>>
> > > > >>>> 7f03d25f60e7 UefiPayloadPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 0bfd8d9b5ac9 UefiCpuPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> e1cd9bfb9dea StandaloneMmPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 5da2f65be378 SourceLevelDebugPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 95b86de07e5d SignedCapsulePkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> fe71d97246c4 ShellPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 54c21c952992 SecurityPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 187a3785f12b RedfishPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 810100002a46 PcAtChipsetPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 276a695c0cf2 OvmfPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 303c0a91ab07 NetworkPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> bc80792cd1b1 MdePkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 3ea86be17a2a MdeModulePkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> c70ef11ed0cd IntelFsp2WrapperPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> c0291221f252 IntelFsp2Pkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 6a479952a690 FmpDevicePkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 3a7c05b7070d FatPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> b789f98c8959 EmulatorPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 952d7a1c9220 EmbeddedPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> a1cc9881bab6 DynamicTablesPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 50654dfe5785 CryptoPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> ed965a02dfa1 ArmVirtPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 9744023fbc46 ArmPlatformPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 7a1cde5f5bba ArmPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 19d17e0913e8 UefiCpuPkg: Change use of EFI_D_* to
> DEBUG_*
> > > > >>>>
> > > > >>>> ffa718b4f994 SourceLevelDebugPkg: Change use of EFI_D_* to
> DEBUG_*
> > > > >>>>
> > > > >>>> b86cb3c5e5b4 ShellPkg: Change use of EFI_D_* to DEBUG_*
> > > > >>>>
> > > > >>>> c7c42204dc07 SecurityPkg: Change use of EFI_D_* to DEBUG_*
> > > > >>>>
> > > > >>>> 16b8e6f958e4 PcAtChipsetPkg: Change use of EFI_D_* to
> DEBUG_*
> > > > >>>>
> > > > >>>> 0ac3f8b2dac5 OvmfPkg: Change use of EFI_D_* to DEBUG_*
> > > > >>>>
> > > > >>>> bc5004b8d294 NetworkPkg: Change use of EFI_D_* to
> DEBUG_*
> > > > >>>>
> > > > >>>> 6f671a8e2377 MdePkg: Change use of EFI_D_* to DEBUG_*
> > > > >>>>
> > > > >>>> a10c610ff9a3 MdeModulePkg: Change use of EFI_D_* to
> DEBUG_*
> > > > >>>>
> > > > >>>> 09a3bddba390 FatPkg: Change use of EFI_D_* to DEBUG_*
> > > > >>>>
> > > > >>>> 59c61318246a EmulatorPkg: Change use of EFI_D_* to
> DEBUG_*
> > > > >>>>
> > > > >>>> 3a80367dda3b EmbeddedPkg: Change use of EFI_D_* to
> DEBUG_*
> > > > >>>>
> > > > >>>> 23eb1aaf80ca ArmVirtPkg: Change use of EFI_D_* to DEBUG_*
> > > > >>>>
> > > > >>>> 875914b45c54 ArmPlatformPkg: Change use of EFI_D_* to
> DEBUG_*
> > > > >>>>
> > > > >>>> eb2eca82b451 ArmPkg: Change use of EFI_D_* to DEBUG_*
> > > > >>>>
> > > > >>>> f0f3f5aae7c4 (origin/master, origin/HEAD, master)
> UnitTestFrameworkPkg: Update YAML to ignore specific ECC
> > > > >>> files/errors
> > > > >>>>
> > > > >>>> c05734797790 UefiPayloadPkg: Update YAML to ignore specific
> ECC files/errors
> > > > >>>>
> > > > >>>> c30c40d6c63d StandaloneMmPkg: Update YAML to ignore
> specific ECC files/errors
> > > > >>>>
> > > > >>>> 9944508e85f1 ShellPkg: Update YAML to ignore specific ECC
> files/errors
> > > > >>>>
> > > > >>>> 60fa40be458d SecurityPkg: Update YAML to ignore specific ECC
> files/errors
> > > > >>>>
> > > > >>>> df790cd6b37e MdePkg: Update YAML to ignore specific ECC
> files/errors
> > > > >>>>
> > > > >>>> 9deb9370766e MdeModulePkg: Update YAML to ignore specific
> ECC files/errors
> > > > >>>>
> > > > >>>> d7d30e8f219f EmulatorPkg: Update YAML to ignore specific ECC
> files/errors
> > > > >>>>
> > > > >>>> d5744ecba813 CryptoPkg: Update YAML to ignore specific ECC
> files/errors
> > > > >>>>
> > > > >>>> c97fee87f0f9 ArmVirtPkg: Update YAML to ignore specific ECC
> files/errors
> > > > >>>>
> > > > >>>> 1939fc9569f2 ArmPlatformPkg: Update YAML to ignore specific
> ECC files/errors
> > > > >>>>
> > > > >>>> 365dced2c37a ArmPkg: Update YAML to ignore specific ECC
> files/errors
> > > > >>>>
> > > > >>>> 76a1ce4d5fec .azurepipelines/templates: Update max pipeline
> job time to 2 hours
> > > > >>>>
> > > > >>>> 99f84ff47390 .pytools/Plugin/LicenseCheck: Use temp directory
> for git diff output
> > > > >>>>
> > > > >>>> 3019f1bbabf1 .pytool/Plugin/EccCheck: Add performance
> optimizations
> > > > >>>>
> > > > >>>> 854462bd3479 .pytool/Plugin/EccCheck: Remove temp
> directory on exception
> > > > >>>>
> > > > >>>> 69877614fdee .pytool/Plugin/EccCheck: Remove RevertCode()
> > > > >>>>
> > > > >>>> --ref1
> > > > >>>>
> > > > >>>> ef9a059cdb15 EmulatorPkg/Win/Host: Update CC_FLAGS
> > > > >>>>
> > > > >>>> bb1bba3d7767 (tag: edk2-stable202111) NetworkPkg: Fix invalid
> pointer for DNS response token on error
> > > > >>>>
> > > > >>>> Best regards,
> > > > >>>>
> > > > >>>> Mike
> > > > >>>>
> > > > >>>>
>
>
>
>
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
2021-12-06 1:17 ` Michael D Kinney
` (4 preceding siblings ...)
2021-12-07 0:26 ` Abner Chang
@ 2021-12-07 3:47 ` Wang, Jian J
2021-12-07 4:21 ` Chiu, Chasel
2021-12-07 10:02 ` Sami Mujawar
7 siblings, 0 replies; 39+ messages in thread
From: Wang, Jian J @ 2021-12-07 3:47 UTC (permalink / raw)
To: devel@edk2.groups.io, Kinney, Michael D, Michael Kubacki,
maciej.rabeda@linux.intel.com, Michael Kubacki,
Andrew Fish (afish@apple.com), Leif Lindholm
For CryptoPkg, SecurityPkg and SignedCapsulePkg,
Reviewed-by: Jian J Wang <jian.j.wang@intel.com>
Regards,
Jian
> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Michael D
> Kinney
> Sent: Monday, December 06, 2021 9:18 AM
> To: Michael Kubacki <mikuback@linux.microsoft.com>; devel@edk2.groups.io;
> maciej.rabeda@linux.intel.com; Michael Kubacki
> <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com)
> <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>; Kinney, Michael D
> <michael.d.kinney@intel.com>
> Subject: Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended
> Hard Freeze Update #4
>
> Hello EDK II Maintainers,
>
> A detailed evaluation of the DEBUG_CODE() formatting issue has been
> completed.
> The reason DEBUG_CODE() is a challenge is that this looks like a macro from a
> C parsing perspective, but the EDK II usage places C statements or blocks of
> C code as the parameter to this macro.
>
> There are actually 2 methods available to mark a statement or a block of code
> to be included when the DEBUG_PROPERTY_DEBUG_CODE_ENABLED bit is
> enabled in
> PcdDebugPropertyMask. One is DEBUG_CODE() and the other is to mark the
> beginning
> and end of a code block with DEBUG_CODE_BEGIN() and DEBUG_CODE_END().
> In fact,
> DEBUG_CODE() is implemented using DEBUG_CODE_BEGIN() and
> DEBUG_CODE_END() macros.
>
> #define DEBUG_CODE(Expression) \
> DEBUG_CODE_BEGIN (); \
> Expression \
> DEBUG_CODE_END ()
>
> A complete review for the use of these DEBUG_CODE macros was performed on
> the
> edk2 repo. Uncrustify performs good formatting for code blocks between
> DEBUG_CODE_BEGIN() and DEBUG_CODE_END(). This is because these look
> like simple
> macros calls with no parameters and the lines of C code between these 2
> macros
> is formatted correctly.
>
> The uncrustify formatting issues are only present with the use of DEBUG_CODE().
> Simple use cases of DEBUG_CODE(Expression) where Expression is a single C
> statement also look correct. A medium complexity use case where Expression is
> a code block of simple statements or even some local variables and simple
> statements also look correct. It is only complex code blocks that use C
> statements such as if/for/while/case that include the use of braces {} does
> uncrustify perform incorrect formatting.
>
> The recommended solution to this issue is to convert the use of DEBUG_CODE()
> to DEBUG_CODE_BEGIN() / DEBUG_CODE_END() for cases where the
> Expression
> passed to DEBUG_CODE() is the complex use case that contains statements that
> use braces {}. There are 57 instances of this pattern across 40 files in the
> edk2 repo.
>
> I have posted a branch with these additional patches:
>
>
> https://github.com/mdkinney/edk2/tree/Bug_3737_3760_3739_ApplyUncrustif
> yChanges_V7
>
> I have performed CompareBuild tests with this revised patch series with
> the DEBUG_CODE changes. It passes 100% showing no binary differences.
>
> https://github.com/mdkinney/edk2/actions/runs/1542454606
>
> I have opened a PR to run this patch series through EDK II CI. It also passes 100%.
>
> https://github.com/tianocore/edk2/pull/2236
>
> The summary of changes made since the V6 review are:
>
> 1) Change uncrustify configuration assignment alignment threshold to 0
>
> align_assign_thresh = 0
>
> 2) Replace "<param>, OPTIONAL" with "<param> OPTIONAL,"
>
> 3) Replace DEBUG_CODE(Expression) with
>
> DEBUG_CODE_BEGIN();
> Expression
> DEBUG_CODE_END()
>
> if Expression is complex and contains braces {}.
>
> 4) No changes to uncrustify tool required.
>
> Please review the differences between the following 2 branches and provide
> feedback or a Series Reviewed-by if you agree with these additional changes.
>
>
> https://github.com/mdkinney/edk2/tree/Bug_3737_3760_3739_ApplyUncrustif
> yChanges_V6
>
> https://github.com/mdkinney/edk2/tree/Bug_3737_3760_3739_ApplyUncrustif
> yChanges_V7
>
> The goal is to complete the review and get the uncrustify change committed
> tomorrow so the extended hard freeze can be lifted.
>
> Thanks,
>
> Mike
>
> > -----Original Message-----
> > From: Kinney, Michael D <michael.d.kinney@intel.com>
> > Sent: Thursday, December 2, 2021 6:23 PM
> > To: Michael Kubacki <mikuback@linux.microsoft.com>; devel@edk2.groups.io;
> maciej.rabeda@linux.intel.com; Michael Kubacki
> > <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com)
> <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>;
> > Kinney, Michael D <michael.d.kinney@intel.com>
> > Subject: RE: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended
> Hard Freeze Update #4
> >
> > Hello EDK II Maintainers,
> >
> > I have entered BZ 3760 to make the use of the OPTIONAL keyword style
> consistent for all of edk2 repo
> > and to be compatible with uncrustify.
> >
> > I have posted the following V6 branch that does the EFI_D_* to DEBUG_*
> changes, the OPTIONAL keyword
> > style changes, and the uncrustify changes with the one configuration change
> for assignment alignment.
> >
> >
> https://github.com/mdkinney/edk2/tree/Bug_3737_3760_3739_ApplyUncrustif
> yChanges_V6
> >
> > Please provide feedback on the code style in this branch with the known
> DEBUG_CODE() issue still
> > present.
> >
> > If we are able to quickly update uncrustify to handle DEBUG_CODE(), I will
> generate a V7.
> >
> > Thanks,
> >
> > Mike
> >
> > > -----Original Message-----
> > > From: Kinney, Michael D <michael.d.kinney@intel.com>
> > > Sent: Thursday, December 2, 2021 4:53 PM
> > > To: Michael Kubacki <mikuback@linux.microsoft.com>;
> devel@edk2.groups.io; maciej.rabeda@linux.intel.com; Michael Kubacki
> > > <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com)
> <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>;
> > > Kinney, Michael D <michael.d.kinney@intel.com>
> > > Subject: RE: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended
> Hard Freeze Update #4
> > >
> > > Michael,
> > >
> > > Yes. Please update the patch series that adds UncrustifyCheck with those
> changes.
> > >
> > > Thanks,
> > >
> > > Mike
> > >
> > > > -----Original Message-----
> > > > From: Michael Kubacki <mikuback@linux.microsoft.com>
> > > > Sent: Thursday, December 2, 2021 4:31 PM
> > > > To: Kinney, Michael D <michael.d.kinney@intel.com>;
> devel@edk2.groups.io; maciej.rabeda@linux.intel.com; Michael
> > Kubacki
> > > > <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com)
> <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>
> > > > Subject: Re: [edk2-devel] Uncrustify Conversion Detailed Plan and
> Extended Hard Freeze Update #4
> > > >
> > > > Hi Mike,
> > > >
> > > > Thank you for the detailed analysis and recommendations.
> > > >
> > > > I agree with the recommendations and I will try to have an Uncrustify
> > > > tool update by tomorrow for (4).
> > > >
> > > > That will require an update in uncrustify_ext_dep.yaml to pull in the
> > > > new version. I'm assuming I should also update uncrustify.cfg to set
> > > > align_assign_thresh = 0 in that new patch series?
> > > >
> > > > Regards,
> > > > Michael
> > > >
> > > > On 12/2/2021 7:18 PM, Kinney, Michael D wrote:
> > > > > Hi Michael,
> > > > >
> > > > > CORRECTION: set align_assign_threshold to 0.
> > > > >
> > > > > Reponses inline below.
> > > > >
> > > > > I would like to summarize the 4 issues raised in the past day along with
> the recommendations.
> > > > >
> > > > > 1) Exclusion feature for UncrustifyCheck. There are 2 directories with 8
> files total that
> > > > > maintainers have noted they would like to see not go through
> uncrustify formatting. Today
> > > > > the only content that is skipped is BaseTools and submodules.
> > > > >
> > > > > Adding a general purpose exclusion feature would then require all
> developers to make
> > > > > sure their method of using uncrustify also excludes those same areas.
> This requires
> > > > > extra steps for all developers and maintainers.
> > > > >
> > > > > If we do not add the exclusion feature, then the 8 files will require an
> extra step
> > > > > to sync with the original source of those files. The rate of changes of
> these 8 files
> > > > > is very low today.
> > > > >
> > > > > RECOMMENDATION: Do not add exclusion feature at this time. Revisit
> if the extra work
> > > > > to maintain the files that would be candidates for exclusions increases
> significantly.
> > > > >
> > > > > 2) Alignment of assignments. The threshold of 4 characters appears to
> be too low and causes
> > > > > source files that are already aligned to become unaligned.
> > > > >
> > > > > RECOMMENDATION: Change threshold to the default value of 0 which
> means no limit.
> > > > >
> > > > > align_assign_thresh= 0
> > > > >
> > > > > 3) Alignment of parameters in function declaration not correct. The root
> cause of this
> > > > > is the use of the OPTIONAL keyword. If the OPTIONAL keyword is
> removed, then the
> > > > > alignment is correct. The alignment is also correct if the OPTIONAL
> keyword appears
> > > > > before the ','. If the OPTIONAL keyword appears after the ',', then the
> format is
> > > > > not correct. The OPTIONAL keyword indicates that the parameter in
> the function is
> > > > > not required and may be passed in as NULL or 0 or some other default
> value defined by
> > > > > the API. It makes more sense for this OPTIONAL keyword that follows
> the parameter
> > > > > names to appear before the ',' so it is scoped to the parameter on that
> line. If it
> > > > > appears after the ',', then C parsers thinks it is a prefix (IN, OUT, CONST,
> volatile,
> > > > > static) for the next parameter in the function.
> > > > >
> > > > > RECOMMENDATION: Update patch series with a global search and
> replace so OPTIONAL
> > > > > keyword always appears before the ',' on the same line.
> > > > >
> > > > > RegEx search string: ',( *)OPTIONAL( *)'
> > > > > RegEx replace string: ' $1OPTIONAL,$2'
> > > > >
> > > > > 4) Format issues with complex blocks in DEBUG_CODE(), or between
> DEBUG_CODE_BEGIN() and
> > > > > DEBUG_CODE_END(). Uncrustify treats these as macros and is not
> aware that the
> > > > > parameter passed into the macro call is a block of C code that needs to
> be formatted.
> > > > > Complex blocks with if/while/for/case statements are impacted the
> most.
> > > > >
> > > > > RECOMMENDATION: Update the uncrustify with an edk2 specific
> extension to treat these
> > > > > macros as a block of code as if they were surrounded by an extra set of
> braces {}.
> > > > >
> > > > >
> > > > > I have posted a branch for testing purposes that implements (2) and (3).
> > > > >
> > > > > Branch:
> https://github.com/mdkinney/edk2/tree/TestOnly_Bug_3737_3739_ApplyUncru
> stifyChanges_V6_OPTIONAL_Keyword_Fix
> > > > > PR: https://github.com/tianocore/edk2/pull/2233
> > > > > Status: PASS
> > > > > CompareBuild:
> https://github.com/mdkinney/edk2/actions/runs/1532855914
> > > > > Status: PASS
> > > > >
> > > > > You can see what changed by fetching and comparing the following 2
> branches:
> > > > >
> > > > >
> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChan
> ges_V5
> > > > >
> https://github.com/mdkinney/edk2/tree/TestOnly_Bug_3737_3739_ApplyUncru
> stifyChanges_V6_OPTIONAL_Keyword_Fix
> > > > >
> > > > > Please provide feedback on the RECOMMENDATIONS above. I will go
> ahead and prepare of V6 version of
> > > > > the patch series now that that test results are all PASS.
> > > > >
> > > > > Best regards,
> > > > >
> > > > > Mike
> > > > >
> > > > >
> > > > >> -----Original Message-----
> > > > >> From: Kinney, Michael D <michael.d.kinney@intel.com>
> > > > >> Sent: Thursday, December 2, 2021 4:15 PM
> > > > >> To: Michael Kubacki <mikuback@linux.microsoft.com>;
> devel@edk2.groups.io; maciej.rabeda@linux.intel.com; Michael
> > > > Kubacki
> > > > >> <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com)
> <afish@apple.com>; Leif Lindholm
> > <leif@nuviainc.com>;
> > > > >> Kinney, Michael D <michael.d.kinney@intel.com>
> > > > >> Subject: RE: [edk2-devel] Uncrustify Conversion Detailed Plan and
> Extended Hard Freeze Update #4
> > > > >>
> > > > >> Hi Michael,
> > > > >>
> > > > >> Reponses inline below.
> > > > >>
> > > > >> I would like to summarize the 4 issues raised in the past day along with
> the recommendations.
> > > > >>
> > > > >> 1) Exclusion feature for UncrustifyCheck. There are 2 directories with 8
> files total that
> > > > >> maintainers have noted they would like to see not go through
> uncrustify formatting. Today
> > > > >> the only content that is skipped is BaseTools and submodules.
> > > > >>
> > > > >> Adding a general purpose exclusion feature would then require all
> developers to make
> > > > >> sure their method of using uncrustify also excludes those same areas.
> This requires
> > > > >> extra steps for all developers and maintainers.
> > > > >>
> > > > >> If we do not add the exclusion feature, then the 8 files will require an
> extra step
> > > > >> to sync with the original source of those files. The rate of changes of
> these 8 files
> > > > >> is very low today.
> > > > >>
> > > > >> RECOMMENDATION: Do not add exclusion feature at this time.
> Revisit if the extra work
> > > > >> to maintain the files that would be candidates for exclusions increases
> significantly.
> > > > >>
> > > > >> 2) Alignment of assignments. The threshold of 4 characters appears to
> be too low and causes
> > > > >> source files that are already aligned to become unaligned.
> > > > >>
> > > > >> RECOMMENDATION: Change threshold to the default value of 0
> which means no limit.
> > > > >>
> > > > >> align_assign_thresh= 4
> > > > >>
> > > > >> 3) Alignment of parameters in function declaration not correct. The
> root cause of this
> > > > >> is the use of the OPTIONAL keyword. If the OPTIONAL keyword is
> removed, then the
> > > > >> alignment is correct. The alignment is also correct if the OPTIONAL
> keyword appears
> > > > >> before the ','. If the OPTIONAL keyword appears after the ',', then the
> format is
> > > > >> not correct. The OPTIONAL keyword indicates that the parameter in
> the function is
> > > > >> not required and may be passed in as NULL or 0 or some other default
> value defined by
> > > > >> the API. It makes more sense for this OPTIONAL keyword that follows
> the parameter
> > > > >> names to appear before the ',' so it is scoped to the parameter on
> that line. If it
> > > > >> appears after the ',', then C parsers thinks it is a prefix (IN, OUT,
> CONST, volatile,
> > > > >> static) for the next parameter in the function.
> > > > >>
> > > > >> RECOMMENDATION: Update patch series with a global search and
> replace so OPTIONAL
> > > > >> keyword always appears before the ',' on the same line.
> > > > >>
> > > > >> RegEx search string: ',( *)OPTIONAL( *)'
> > > > >> RegEx replace string: ' $1OPTIONAL,$2'
> > > > >>
> > > > >> 4) Format issues with complex blocks in DEBUG_CODE(), or between
> DEBUG_CODE_BEGIN() and
> > > > >> DEBUG_CODE_END(). Uncrustify treats these as macros and is not
> aware that the
> > > > >> parameter passed into the macro call is a block of C code that needs
> to be formatted.
> > > > >> Complex blocks with if/while/for/case statements are impacted the
> most.
> > > > >>
> > > > >> RECOMMENDATION: Update the uncrustify with an edk2 specific
> extension to treat these
> > > > >> macros as a block of code as if they were surrounded by an extra set
> of braces {}.
> > > > >>
> > > > >>
> > > > >> I have posted a branch for testing purposes that implements (2) and (3).
> > > > >>
> > > > >> Branch:
> https://github.com/mdkinney/edk2/tree/TestOnly_Bug_3737_3739_ApplyUncru
> stifyChanges_V6_OPTIONAL_Keyword_Fix
> > > > >> PR: https://github.com/tianocore/edk2/pull/2233
> > > > >> Status: PASS
> > > > >> CompareBuild:
> https://github.com/mdkinney/edk2/actions/runs/1532855914
> > > > >> Status: PASS
> > > > >>
> > > > >> You can see what changed by fetching and comparing the following 2
> branches:
> > > > >>
> > > > >>
> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChan
> ges_V5
> > > > >>
> https://github.com/mdkinney/edk2/tree/TestOnly_Bug_3737_3739_ApplyUncru
> stifyChanges_V6_OPTIONAL_Keyword_Fix
> > > > >>
> > > > >> Please provide feedback on the RECOMMENDATIONS above. I will go
> ahead and prepare of V6 version of
> > > > >> the patch series now that that test results are all PASS.
> > > > >>
> > > > >> Best regards,
> > > > >>
> > > > >> Mike
> > > > >>
> > > > >>
> > > > >>> -----Original Message-----
> > > > >>> From: Michael Kubacki <mikuback@linux.microsoft.com>
> > > > >>> Sent: Thursday, December 2, 2021 1:57 PM
> > > > >>> To: devel@edk2.groups.io; Kinney, Michael D
> <michael.d.kinney@intel.com>; maciej.rabeda@linux.intel.com; Michael
> > > > Kubacki
> > > > >>> <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com)
> <afish@apple.com>; Leif Lindholm
> > <leif@nuviainc.com>
> > > > >>> Subject: Re: [edk2-devel] Uncrustify Conversion Detailed Plan and
> Extended Hard Freeze Update #4
> > > > >>>
> > > > >>> My reply is inline.
> > > > >>>
> > > > >>> Regards,
> > > > >>> Michael
> > > > >>>
> > > > >>> On 12/2/2021 2:45 PM, Michael D Kinney wrote:
> > > > >>>> Hi Maciej,
> > > > >>>>
> > > > >>>> Thanks for the feedback.
> > > > >>>>
> > > > >>>> * Example #1.This appears to be caused by the following uncrustify
> > > > >>>> settings:
> > > > >>>>
> > > > >>>> # The threshold for aligning on '=' in assignments.
> > > > >>>>
> > > > >>>> # Use a negative number for absolute thresholds.
> > > > >>>>
> > > > >>>> #
> > > > >>>>
> > > > >>>> # 0: No limit (default).
> > > > >>>>
> > > > >>>> align_assign_thresh= 0# number
> > > > >>>>
> > > > >>>> The edk2 setting for this is:
> > > > >>>>
> > > > >>>> align_assign_thresh= 4
> > > > >>>>
> > > > >>>> This means blocks of assignments that are different than more than 4
> > > > >>>> spaces will be considered a new block.
> > > > >>>>
> > > > >>>> ‘HwAddreLen’ and ‘Xid’ are more than 4 characters in length
> > > > >>>> different.Same for ‘Xid’ and ‘Reserved’.So
> > > > >>>>
> > > > >>>> uncrustify treats these as 3 different assignment blocks.
> > > > >>>>
> > > > >>>> If we change to the default value of 0: No limit, this example is
> > > > >>>> treated as a single block and all ‘=’ are aligned.
> > > > >>>>
> > > > >>>> Is there a reason ‘4’ was selected?Is there is any harm in using the
> > > > >>>> default of 0?
> > > > >>>>
> > > > >>> We can certainly change the threshold. '4' was derived by
> > > > >>> experimentation. This is an area that is somewhat subjective and every
> > > > >>> case is difficult to cover well. Please feel free to suggest any changes.
> > > > >>>
> > > > >>
> > > > >> I recommend we use the default value of 0. That way, any code that
> choose to
> > > > >> align assignments will still be aligned. If a developer does not like
> > > > >> short assignments and long assignments in the same code block to use
> the
> > > > >> long assignment column, they can always break the block up into
> multiple
> > > > >> blocks by adding a carriage return.
> > > > >>
> > > > >>> For the benefit of others, this file can be a useful refernce
> > > > >>> understanding spans, gaps, and thresholds.
> > > > >>>
> > > > >>>
> https://github.com/uncrustify/uncrustify/blob/master/documentation/htdocs/c
> onfiguration.txt
> > > > >>>
> > > > >>>> * Example #2: Uncruistfy is confused by the DEBUG_CODE()
> macro.This is
> > > > >>>> not a traditional macro because the
> > > > >>>>
> > > > >>>> contents of the macro is a block of C code.I do not know how to
> convince
> > > > >>>> uncrustify that the contents of a
> > > > >>>>
> > > > >>>> macro like function call to be treated as a code block from an indent
> > > > >>>> perspective.
> > > > >>>>
> > > > >>> I believe this would have to be overridden in the Uncrustify fork since
> > > > >>> DEBUG_CODE() is being treated as a macro function call and code
> blocks
> > > > >>> are not expected there. In addition, some special treatment might be
> > > > >>> needed for alignment in between
> DEBUG_CODE_BEGIN()/DEBUG_CODE_END().
> > > > >>>
> > > > >>> I'm happy to look at this. However, regression validation can take a
> > > > >>> while so I'd like to make sure we need this now. In cases that do not
> > > > >>> have additional code blocks, it seems to format fairly well. Is this
> > > > >>> prevalent and impactful enough it must be fixed now? Or, could we
> > > > >>> revisit it with a follow up patch?
> > > > >>
> > > > >> How long does regression testing take? I see about 20 instances of
> > > > >> DEBUG_CODE() that are producing bad formatting. If the content inside
> > > > >> DEBUG_CODE() is a single line of a single block of statements without
> > > > >> any if/while/for/case statements that require further indent, then the
> > > > >> format looks ok.
> > > > >>
> > > > >> DEBUG_CODE_BEGIN() and DEBUG_CODE_END() would look better if
> the
> > > > >> contents between are also indented one level.
> > > > >>
> > > > >>>
> > > > >>>> * Example #3/#4: Uncrustify is confused by the OPTIONAL
> keyword.The
> > > > >>>> edk2 config declares it as a QUALIFIER
> > > > >>>>
> > > > >>>> like IN and OUT.However, OPTIONAL only appears at the end of the
> line
> > > > >>>> that declares a parameter to a
> > > > >>>>
> > > > >>>> function.There are 3 forms. One with comma after OPTIONAL.One
> with comma
> > > > >>>> before OPTIONAL, and
> > > > >>>>
> > > > >>>> one with no comma if the parameter is the last parameter in the
> function
> > > > >>>> declaration.
> > > > >>>>
> > > > >>>> TYPE ParamName OPTIONAL,
> > > > >>>>
> > > > >>>> TYPEParamName, OPTIONAL
> > > > >>>>
> > > > >>>> TYPEParamName OPTIONAL
> > > > >>>>
> > > > >>>> OPTIONAL is defined to nothing in edk2 builds.From a uncrustify
> > > > >>>> perspective, we really want is to be
> > > > >>>>
> > > > >>>> ignored or more correctly treated it as a token that is attached to
> > > > >>>> ParamName and the combination of
> > > > >>>>
> > > > >>>> ParamName and OPTIONAL treated as one unit to determine indents.
> > > > >>>>
> > > > >>> "TYPE ParamName, OPTIONAL" seems especially problematic and
> presents an
> > > > >>> inconsistency with the other formats. Mike, can you please let me
> know
> > > > >>> if you have the same observation and your thoughts on a consistent
> pattern?
> > > > >>>
> > > > >>>> Thanks,
> > > > >>>>
> > > > >>>> Mike
> > > > >>>>
> > > > >>>> *From:*devel@edk2.groups.io <devel@edk2.groups.io> *On Behalf
> Of *Maciej
> > > > >>>> Rabeda
> > > > >>>> *Sent:* Thursday, December 2, 2021 10:27 AM
> > > > >>>> *To:* devel@edk2.groups.io; Kinney, Michael D
> > > > >>>> <michael.d.kinney@intel.com>; Michael Kubacki
> > > > >>>> <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com)
> > > > >>>> <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>
> > > > >>>> *Subject:* Re: [edk2-devel] Uncrustify Conversion Detailed Plan and
> > > > >>>> Extended Hard Freeze Update #4
> > > > >>>>
> > > > >>>> Hey Mike,
> > > > >>>>
> > > > >>>> While most of the changes related to fixing coding style violations,
> > > > >>>> there are a couple of changes to NetworkPkg in that PR that make
> the
> > > > >>>> code less readable. Examples below.
> > > > >>>>
> > > > >>>> Example 1:
> > > > >>>>
> > > > >>>>
> > > > >>>>
> > > > >>>> Example 2:
> > > > >>>>
> > > > >>>>
> > > > >>>> Example 3:
> > > > >>>>
> > > > >>>>
> > > > >>>> Example 4:
> > > > >>>>
> > > > >>>> On 30-Nov-21 23:34, Michael D Kinney wrote:
> > > > >>>>
> > > > >>>> Hello,
> > > > >>>>
> > > > >>>> Thank you for your patience during this extended hard freeze.
> > > > >>>>
> > > > >>>> Just one more step to go.There has been a delay in the review of
> > > > >>>>
> > > > >>>> the patch series with the uncrustify source changes.PR(6).This
> > > > >>>>
> > > > >>>> patch series was not sent out as patch review email because of its
> > > > >>>>
> > > > >>>> very large size.It only contains source style changes and the
> > > > >>>>
> > > > >>>> CompareBuild tool and GitHub action has shown there are no
> binary
> > > > >>>>
> > > > >>>> differences introduced with these source style changes.
> > > > >>>>
> > > > >>>> If you are a package maintainer, then please review the following
> > > > >>>>
> > > > >>>> branch/PR for your package contents and review the EDK II CI
> results
> > > > >>>>
> > > > >>>> and BuildCompare results.I do not expect a line by line review
> > > > >>>>
> > > > >>>> because we already had time to provide feedback on the source
> style
> > > > >>>>
> > > > >>>> performed by uncrustify.Instead, a Reviewed-by for your package
> > > > >>>>
> > > > >>>> indicates that you have reviewed the EDK II CI results and
> CompareBuild
> > > > >>>>
> > > > >>>> tool functionality and results and you accept the source style
> > > > >>>>
> > > > >>>> changes to your package.
> > > > >>>>
> > > > >>>>
> *https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyCha
> nges_V5
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyCha
> nges_V5>
> > > > >>>>
> > > > >>>> *https://github.com/tianocore/edk2/pull/2229
> <https://github.com/tianocore/edk2/pull/2229>
> > > > >>>>
> > > > >>>> *https://github.com/mdkinney/edk2/actions/runs/1521618836
> > > > >>> <https://github.com/mdkinney/edk2/actions/runs/1521618836>
> > > > >>>>
> > > > >>>> Additional details on this update below.
> > > > >>>>
> > > > >>>> Thank you,
> > > > >>>>
> > > > >>>> Mike
> > > > >>>>
> > > > >>>> Changes from Update #3
> > > > >>>>
> > > > >>>> ----------------------------------------------------------------------------
> > > > >>>>
> > > > >>>> * Pushed PR (5)
> > > > >>>>
> > > > >>>> * Added link to PR(6). EDK II CI Status is PASS. Build Compare PASS.
> > > > >>>>
> > > > >>>> * Waiting for review of PR (6)
> > > > >>>>
> > > > >>>> * Review of PR (7) completed and waiting for review of PR (6)
> > > > >>>>
> > > > >>>> ----------------------------------------------------------------------------
> > > > >>>>
> > > > >>>> Changes from Update #2
> > > > >>>>
> > > > >>>> ----------------------------------------------------------------------------
> > > > >>>>
> > > > >>>> * Changed order of PRs swapping (4) and (5).The PR that activates
> > > > >>>>
> > > > >>>> increases the max CI agent job time is independent of all the other
> > > > >>>>
> > > > >>>> PRs and its review is complete, so it can be committed now.
> > > > >>>>
> > > > >>>> * Pushed PRs (1), (2), (3), (4).
> > > > >>>>
> > > > >>>> * Waiting for review to complete for PRs (5) and (6)
> > > > >>>>
> > > > >>>> * Reviews complete for PR (7)
> > > > >>>>
> > > > >>>> * Identifies steps using git filter-branch to apply uncrustify changes
> to a
> > > > >>>>
> > > > >>>> code review patch series that was generated before the uncrustify
> changes
> > > > >>>>
> > > > >>>> avoiding manual merge.
> > > > >>>>
> > > > >>>> * Identified steps using git filter-repo to generate an alternate
> history of
> > > > >>>>
> > > > >>>> the edk2 repo with uncrustify changes applied on every
> commit.This may
> > > > >>>>
> > > > >>>> be useful when evaluating changes to files using tools like git
> blame
> > > > >>>>
> > > > >>>> without the large uncrustify patch series.
> > > > >>>>
> > > > >>>> ---------------------------------------------------------------------------
> > > > >>>>
> > > > >>>> Changes from Update #1
> > > > >>>>
> > > > >>>> ----------------------------------------------------------------------------
> > > > >>>>
> > > > >>>> * Changed order of PRs swapping (6) and (7).The PR that activates
> > > > >>>>
> > > > >>>> EDK II CI check UncrustifyCheck has to be last because it
> unconditionally
> > > > >>>>
> > > > >>>> checks all C/H files in all packages.Not just files that have been
> > > > >>>>
> > > > >>>> modified like some of the other checkers.
> > > > >>>>
> > > > >>>> * Updated link to the branch with the UncrustifyCheck plugin that
> has been
> > > > >>>>
> > > > >>>> updated with a one line change and Reviewed-by and Tested-by
> tags.
> > > > >>>>
> > > > >>>>
> https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v
> 6
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_
> v6>
> > > > >>>>
> > > > >>>> * Reviews complete for (1), (2), (3), (5), and (7)
> > > > >>>>
> > > > >>>> ---------------------------------------------------------------------------
> > > > >>>>
> > > > >>>> Michael Kubacki and I have prepared the patches required to apply
> the
> > > > >>>>
> > > > >>>> uncrustify changes and enable EDK II CI to check all submitted
> > > > >>>>
> > > > >>>> patches have been run through uncrustify.
> > > > >>>>
> > > > >>>> We have verified through the CompareBuild GitHub Action that the
> > > > >>>>
> > > > >>>> format changes performed by uncrustify have no functional
> changes.
> > > > >>>>
> > > > >>>> All of the OBJ, LIB, DLL, EFI, FFS, FV, and FD files match 100%
> > > > >>>>
> > > > >>>> across 70 VS2019/GCC5 builds of all package/platform DSC files in
> > > > >>>>
> > > > >>>> the edk2 repo.
> > > > >>>>
> > > > >>>> The hard freeze will be extended after the edk2-stable202111 tag
> until
> > > > >>>>
> > > > >>>> all uncrustify related changes are committed.We do not expect
> this
> > > > >>>>
> > > > >>>> to take more than a few days.Do not push any PRs until the hard
> > > > >>>>
> > > > >>>> freeze is lifted.
> > > > >>>>
> > > > >>>> The changes are broken up into 7 patch series/PRs.The PRs are
> ordered
> > > > >>>>
> > > > >>>> so they can be submitted using the normal submission process and
> EDK II
> > > > >>>>
> > > > >>>> CI will pass for each one.Details are listed below.
> > > > >>>>
> > > > >>>> Uncrustify 73.0.3 for EDK II
> > > > >>>>
> > > > >>>> =============================
> > > > >>>>
> > > > >>>> * Sources:https://dev.azure.com/projectmu/_git/Uncrustify
> <https://dev.azure.com/projectmu/_git/Uncrustify>
> > > > >>>>
> > > > >>>> *
> Documentation:https://dev.azure.com/projectmu/Uncrustify/_wiki/wikis/Uncru
> stify.wiki/1/Project-Mu-(EDK-
> > II)-
> > > > Fork-
> > > > >>> Readme
> <https://dev.azure.com/projectmu/Uncrustify/_wiki/wikis/Uncrustify.wiki/1/Pr
> oject-Mu-(EDK-II)-Fork-Readme>
> > > > >>>>
> > > > >>>> *
> Download:https://dev.azure.com/projectmu/Uncrustify/_packaging?_a=packag
> e&feed=mu_uncrustify&package=mu-
> > > > >>> uncrustify-release&protocolType=NuGet&version=73.0.3
> > > > >>>
> <https://dev.azure.com/projectmu/Uncrustify/_packaging?_a=package&feed=
> mu_uncrustify&package=mu-uncrustify-
> > > > >>> release&protocolType=NuGet&version=73.0.3>
> > > > >>>>
> > > > >>>> Installing Uncrustify
> > > > >>>>
> > > > >>>> ======================
> > > > >>>>
> > > > >>>> The Uncrustify tool is installed automatically when the Pytools
> > > > >>>>
> > > > >>>> environment is used and the stuart* commands are run to
> complete the
> > > > >>>>
> > > > >>>> environment setup.Please see:
> > > > >>>>
> > > > >>>> https://github.com/tianocore/edk2/tree/master/.pytool#running-
> ci-locally
> > > > >>> <https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-
> locally>
> > > > >>>>
> > > > >>>> Uncrustify can also be installed from the download page listed
> above
> > > > >>>>
> > > > >>>> or built from sources from the source link above.
> > > > >>>>
> > > > >>>> The Documentation link provides instruction on how to run
> uncrustify from
> > > > >>>>
> > > > >>>> the command line or install as a Visual Studio Code plugin.The
> main
> > > > >>>>
> > > > >>>> uncrustify documentation also describes how to integrate with a
> few other
> > > > >>>>
> > > > >>>> editors.
> > > > >>>>
> > > > >>>> We have also discussed a client side githook.That effort has not
> started.
> > > > >>>>
> > > > >>>> Let us know if that is a feature you would find useful.
> > > > >>>>
> > > > >>>> Developer impact for new code reviews
> > > > >>>>
> > > > >>>> ======================================
> > > > >>>>
> > > > >>>> Once the uncrustify checker is active in EDK II CI, developers must
> > > > >>>>
> > > > >>>> make sure their patches are run through the uncrustify tool before
> > > > >>>>
> > > > >>>> sending the patches for review.
> > > > >>>>
> > > > >>>> Developers must install and run uncrustify against changes files
> before
> > > > >>>>
> > > > >>>> sending patch review emails or submitting PR for EDK II CI.If EDK II
> CI
> > > > >>>>
> > > > >>>> detects and differences in source formatting, then EDK II CI will fail
> > > > >>>>
> > > > >>>> and the developer must run uncrustify and resubmit the patches.
> > > > >>>>
> > > > >>>> Developer impact to patch series/PRs reviewed during edk2-
> stable201121 soft/hard freeze
> > > > >>>>
> > > > >>>>
> =================================================================
> ======================
> > > > >>>>
> > > > >>>> Developers must rebase their changes after the uncrustify source
> changes are
> > > > >>>>
> > > > >>>> committed.The branch with a preview of the uncrustify changes
> can be used
> > > > >>>>
> > > > >>>> to start this rebase work.
> > > > >>>>
> > > > >>>>
> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChan
> ges_V5
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyCha
> nges_V5>
> > > > >>>>
> > > > >>>> The following steps can be used to update an existing branch with
> the
> > > > >>>>
> > > > >>>> required uncrustify format.This is the Windows version.I will add
> > > > >>>>
> > > > >>>> the Linux version soon.
> > > > >>>>
> > > > >>>> 1) Fetch and checkout and rebase to latest edk2/master
> > > > >>>>
> > > > >>>> git fetch origin
> > > > >>>>
> > > > >>>> git checkout master
> > > > >>>>
> > > > >>>> git rebase origin/master
> > > > >>>>
> > > > >>>> 2) Make a backup copy of plugin UncrustifyCheck outside
> WORKSPACE.
> > > > >>>>
> > > > >>>> (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable
> and
> > > > >>>>
> > > > >>>> EDK II specific uncrustify configuration file available when working
> > > > >>>>
> > > > >>>> with a branch that does not have those tools in its scope.
> > > > >>>>
> > > > >>>> xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
> > > > >>>>
> > > > >>>> 3) Check out the patch series branch (e.g. MyBranch)
> > > > >>>>
> > > > >>>> git checkout MyBranch
> > > > >>>>
> > > > >>>> 4) Rebase patch series against edk2-stable202111
> > > > >>>>
> > > > >>>>
> > > > >>>>
> > > > >>>> git rebase edk2-stable202111
> > > > >>>>
> > > > >>>> 5) Create new branch for the uncrustifed version (e.g.
> MyBranch_Uncrustified)
> > > > >>>>
> > > > >>>> git checkout -b MyBranch_Uncrustified
> > > > >>>>
> > > > >>>> 6) Use git filter-branch to uncrustify all the commits in the series
> > > > >>>>
> > > > >>>> between the rebase target from (2) and HEAD of the branch.A filter
> > > > >>>>
> > > > >>>> can be used to scope the uncrustify operations to only the C/H files
> > > > >>>>
> > > > >>>> in the specific package the patch series is against. (e.g.
> DynamicTablesPkg).
> > > > >>>>
> > > > >>>> BaseTools should always be excluded.If the package scoped filter
> is
> > > > >>>>
> > > > >>>> not used, it will still work, but will take longer to run because
> > > > >>>>
> > > > >>>> uncrustify will rescan every C/H files in the whole repo.
> > > > >>>>
> > > > >>>> git filter-branch --tree-filter "git ls-files DynamicTablesPkg*.c
> DynamicTablesPkg*.h :!BaseTools/* |
> > > > >>> c:\\Temp\\UncrustifyCheck\\mu-uncrustify-
> release_extdep\\Windows-x86\\uncrustify.exe -c
> > > > >>> c:\\Temp\\UncrustifyCheck\\uncrustify.cfg -F - --replace --no-backup -
> -if-changed" edk2-stable202111..HEAD
> > > > >>>>
> > > > >>>> 7) Now that all the individual patches in the branch are uncrustified,
> > > > >>>>
> > > > >>>> rebase against latest edk2/master that is already uncrustified.
> > > > >>>>
> > > > >>>> git rebase master
> > > > >>>>
> > > > >>>> 8) Verify the patches in this new branch.
> > > > >>>>
> > > > >>>> Impacts to tracing history across the uncrusity changes
> > > > >>>>
> > > > >>>>
> =======================================================
> > > > >>>>
> > > > >>>> Tools the view file and line history do work with the large
> uncrustify
> > > > >>>>
> > > > >>>> patch series.One impact is that the operations can be very slow
> due
> > > > >>>>
> > > > >>>> to the large uncrustify patches.
> > > > >>>>
> > > > >>>> One option to provide a faster experience is to provide an
> alternate
> > > > >>>>
> > > > >>>> version of the edk2 repository as "documentation" that has the
> > > > >>>>
> > > > >>>> entire history re-written with uncrustify run on every commit.
> > > > >>>>
> > > > >>>> The tool called git-filter-repo can be used to perform this
> > > > >>>>
> > > > >>>> transformation and runs in a reasonable period of time (a few
> hours)
> > > > >>>>
> > > > >>>> https://github.com/newren/git-filter-repo
> <https://github.com/newren/git-filter-repo>
> > > > >>>>
> > > > >>>> https://github.com/newren/git-filter-
> repo/blob/main/contrib/filter-repo-demos/lint-history
> > > > >>> <https://github.com/newren/git-filter-repo/blob/main/contrib/filter-
> repo-demos/lint-history>
> > > > >>>>
> > > > >>>> The following steps can be used to perform this transformation.
> > > > >>>>
> > > > >>>> This is the Windows version. I will add the Linux version soon.
> > > > >>>>
> > > > >>>> ** WARNING **This operation modifies(rewrites) all the commits
> > > > >>>>
> > > > >>>> in the local copy of the repo.Do not perform
> > > > >>>>
> > > > >>>> these steps on a local repo you are using for
> > > > >>>>
> > > > >>>> active development.
> > > > >>>>
> > > > >>>> 1) Clone edk2 into a new directory (see **WARNING**)
> > > > >>>>
> > > > >>>> git clonehttps://github.com/tianocore/edk2.git
> <https://github.com/tianocore/edk2.git> edk2-uncrustified
> > > > >>>>
> > > > >>>> cd edk2-uncrustified
> > > > >>>>
> > > > >>>> 2) Setup python virtual env, install pytools, and run stuart
> commands
> > > > >>>>
> > > > >>>> to setup build environment which includes installing uncrustify
> tools.
> > > > >>>>
> > > > >>>> https://github.com/tianocore/edk2/tree/master/.pytool#running-
> ci-locally
> > > > >>> <https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-
> locally>
> > > > >>>>
> > > > >>>> 3) Make a backup copy of plugin UncrustifyCheck outside
> WORKSPACE.
> > > > >>>>
> > > > >>>> (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable
> and
> > > > >>>>
> > > > >>>> EDK II specific uncrustify configuration file available when working
> > > > >>>>
> > > > >>>> with a branch that does not have those tools in its scope.
> > > > >>>>
> > > > >>>> xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
> > > > >>>>
> > > > >>>> 4) Use lint-history.py from git-filter-repo examples
> > > > >>>>
> > > > >>>> https://github.com/newren/git-filter-repo
> <https://github.com/newren/git-filter-repo>
> > > > >>>>
> > > > >>>> https://github.com/newren/git-filter-
> repo/blob/main/contrib/filter-repo-demos/lint-history
> > > > >>> <https://github.com/newren/git-filter-repo/blob/main/contrib/filter-
> repo-demos/lint-history>
> > > > >>>>
> > > > >>>> Line #127 - Add try except around subprocess.check_call() with
> except
> > > > >>>>
> > > > >>>> being pass.This is required because there are a few commits of C
> > > > >>>>
> > > > >>>> files in the edk2 repo that have incorrect C syntax and do not
> > > > >>>>
> > > > >>>> build with a C compiler and break the uncrustify parser.Skip
> reformat
> > > > >>>>
> > > > >>>> of C files that can not be parsed by uncrustify.These rare instances
> > > > >>>>
> > > > >>>> are addressed in the commit that fixes the C syntax error.
> > > > >>>>
> > > > >>>> Run this slightly modified version of lint-history.Include only
> > > > >>>>
> > > > >>>> C/H files and exclude directories that start with 'Tools' or
> 'BaseTools'.
> > > > >>>>
> > > > >>>> This step took about 2.2 hours on a laptop.
> > > > >>>>
> > > > >>>> lint-history.py
> > > > >>>>
> > > > >>>> --relevant "return (not filename.startswith(b'Tools') and not
> filename.startswith(b'BaseTools') and
> > > > >>> (filename.endswith(b'.c') or filename.endswith(b'.h')))"
> > > > >>>>
> > > > >>>> c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\mu-
> uncrustify-release_extdep\\Windows-x86\\uncrustify.exe
> > -
> > > c
> > > > >>> c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\uncrustify.cfg --
> replace --no-backup --if-changed
> > > > >>>>
> > > > >>>> Order of PRs to apply during extended hard freeze
> > > > >>>>
> > > > >>>> ==================================================
> > > > >>>>
> > > > >>>> 1) Update EmulatorPkg Win Host [BuildOptions] MSFT CC_FLAGS
> to not force debug information
> > > > >>>>
> > > > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3747
> <https://bugzilla.tianocore.org/show_bug.cgi?id=3747>
> > > > >>>>
> > > > >>>>
> *https://github.com/mdkinney/edk2/tree/Bug_3747_EmulatorPkg_WinHost_Re
> producibleBuild
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_3747_EmulatorPkg_WinHost_Re
> producibleBuild>
> > > > >>>>
> > > > >>>> *https://github.com/tianocore/edk2/pull/2215
> <https://github.com/tianocore/edk2/pull/2215>
> > > > >>>>
> > > > >>>> * Required for EmulatorPkg to pass CompareBuild for VS2019
> IA32/X64 builds.
> > > > >>>>
> > > > >>>> * Status: Review complete.PR pushed.
> > > > >>>>
> > > > >>>> 2) EccCheck should not revert staged and local changes
> > > > >>>>
> > > > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=2986
> <https://bugzilla.tianocore.org/show_bug.cgi?id=2986>
> > > > >>>>
> > > > >>>>
> *https://github.com/mdkinney/edk2/tree/Bug_2986_EccCheckRemoveGitRever
> t_V2
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_2986_EccCheckRemoveGitRever
> t_V2>
> > > > >>>>
> > > > >>>> *https://github.com/tianocore/edk2/pull/2216
> <https://github.com/tianocore/edk2/pull/2216>
> > > > >>>>
> > > > >>>> * Required for EDK II CI to complete in a reasonable period of time
> when
> > > > >>>>
> > > > >>>> processing the 4000+ source file style changes made by uncrustify.
> > > > >>>>
> > > > >>>> * Also fixes critical bugs that can potentially corrupt git state when
> > > > >>>>
> > > > >>>> EccCheck is run locally.
> > > > >>>>
> > > > >>>> * Status: Review complete.PR pushed.
> > > > >>>>
> > > > >>>> 3) Update pytool LicenseCheck plugin to use temp directory for diff
> output file
> > > > >>>>
> > > > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3746
> <https://bugzilla.tianocore.org/show_bug.cgi?id=3746>
> > > > >>>>
> > > > >>>>
> *https://github.com/mdkinney/edk2/tree/Bug_3746_LicenseCheckUseDiffOutp
> utFile_V2
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_3746_LicenseCheckUseDiffOutp
> utFile_V2>
> > > > >>>>
> > > > >>>> *https://github.com/tianocore/edk2/pull/2217
> <https://github.com/tianocore/edk2/pull/2217>
> > > > >>>>
> > > > >>>> * Required to reduce EDK II CI build times.
> > > > >>>>
> > > > >>>> * Status: Review complete.PR pushed.
> > > > >>>>
> > > > >>>> 4) Update max job time from 60 min to 120 minutes
> in .azurepipelines/templates
> > > > >>>>
> > > > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3750
> <https://bugzilla.tianocore.org/show_bug.cgi?id=3750>
> > > > >>>>
> > > > >>>>
> *https://github.com/mdkinney/edk2/tree/Bug_3750_IncreaseAzurePipelinesTim
> eout
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_3750_IncreaseAzurePipelinesTim
> eout>
> > > > >>>>
> > > > >>>> *https://github.com/tianocore/edk2/pull/2219
> <https://github.com/tianocore/edk2/pull/2219>
> > > > >>>>
> > > > >>>> * Required to allow EccCheck of uncrustify changes to complete
> on Azure
> > > > >>>>
> > > > >>>> Pipelines CI agents without timing out.
> > > > >>>>
> > > > >>>> * Status: Review complete.PR pushed.
> > > > >>>>
> > > > >>>> 5) Update Package YAML to ignore specific ECC files/errors
> > > > >>>>
> > > > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3749
> <https://bugzilla.tianocore.org/show_bug.cgi?id=3749>
> > > > >>>>
> > > > >>>>
> *https://github.com/mdkinney/edk2/tree/Bug_3749_EccCheckIgnoreFilesErrors
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_3749_EccCheckIgnoreFilesErrors>
> > > > >>>>
> > > > >>>> *https://github.com/tianocore/edk2/pull/2218
> <https://github.com/tianocore/edk2/pull/2218>
> > > > >>>>
> > > > >>>> * Required to pass EccCheck
> > > > >>>>
> > > > >>>> * Status: Review complete. PR pushed
> > > > >>>>
> > > > >>>> 6) Uncrustify Source Changes
> > > > >>>>
> > > > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3737
> <https://bugzilla.tianocore.org/show_bug.cgi?id=3737>
> > > > >>>>
> > > > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3739
> <https://bugzilla.tianocore.org/show_bug.cgi?id=3739>
> > > > >>>>
> > > > >>>>
> *https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyCha
> nges_V5
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyCha
> nges_V5>
> > > > >>>>
> > > > >>>> *https://github.com/tianocore/edk2/pull/2229
> <https://github.com/tianocore/edk2/pull/2229>
> > > > >>>>
> > > > >>>> * Build comparison result
> PASS:https://github.com/mdkinney/edk2/actions/runs/1521618836
> > > > >>> <https://github.com/mdkinney/edk2/actions/runs/1521618836>
> > > > >>>>
> > > > >>>> * EFI_D_ -> DEBUG changes required to pass PatchCheck
> > > > >>>>
> > > > >>>> * Uncrustify format changes required to pass UncrustifyCheck
> > > > >>>>
> > > > >>>> * Status:
> > > > >>>>
> > > > >>>> Waiting for review
> > > > >>>>
> > > > >>>> 7) UncrustifyCheck EDK II CI Plugin
> > > > >>>>
> > > > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3748
> <https://bugzilla.tianocore.org/show_bug.cgi?id=3748>
> > > > >>>>
> > > > >>>>
> *https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_
> v6
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_
> v6>
> > > > >>>>
> > > > >>>> * Required to enforce all PRs submitted to EDK II CI match
> uncrustify format.
> > > > >>>>
> > > > >>>> * Unconditionally checks all packages.Can not be committed until
> all C/H
> > > > >>>>
> > > > >>>> source files have been updated.
> > > > >>>>
> > > > >>>> * Status: Review complete
> > > > >>>>
> > > > >>>> Combined Branch/PR for Review/Test
> > > > >>>>
> > > > >>>> ==================================
> > > > >>>>
> > > > >>>> * Build Comparison results must pass 100% across the full set of
> PRs before
> > > > >>>>
> > > > >>>> the individual PRs can be pushed in the order listed above.
> > > > >>>>
> > > > >>>> *
> Branch:https://github.com/mdkinney/edk2/tree/TestOnly_Uncrustify_PR_Series
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/TestOnly_Uncrustify_PR_Series>
> > > > >>>>
> > > > >>>> * PR:https://github.com/tianocore/edk2/pull/2229
> <https://github.com/tianocore/edk2/pull/2229>
> > > > >>>>
> > > > >>>> Status = PASS
> > > > >>>>
> > > > >>>> * CompareBuild:
> > > > >>>>
> > > > >>>>
> Branch:https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrusti
> fyChanges_V5
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyCha
> nges_V5>
> > > > >>>>
> > > > >>>> --ref1:ef9a059cdb15844fe52a49af2bf7d86b9dd3e9bf
> > > > >>>>
> > > > >>>> --ref2:Bug_3737_3739_ApplyUncrustifyChanges_V5
> > > > >>>>
> > > > >>>> Extra Options: -n 4 --quiet
> > > > >>>>
> > > > >>>>
> Results:https://github.com/mdkinney/edk2/actions/runs/1521618836
> > > > >>> <https://github.com/mdkinney/edk2/actions/runs/1521618836>
> > > > >>>>
> > > > >>>> 30 VS2019 build comparisons PASS
> > > > >>>>
> > > > >>>> 40 GCC5 build comparisons PASS
> > > > >>>>
> > > > >>>> 100% PASS
> > > > >>>>
> > > > >>>> The following git log shows the set of patches from --ref1 to --ref
> 2across
> > > > >>>>
> > > > >>>> which there are no differences in any of the
> OBJ/LIB/DLL/EFI/FFS/FV/FD files.
> > > > >>>>
> > > > >>>> --ref2
> > > > >>>>
> > > > >>>> b7d4bf0675b7 (HEAD ->
> Bug_3737_3739_ApplyUncrustifyChanges_V5) UnitTestFrameworkPkg: Apply
> uncrusitify
> > changes
> > > > >>>>
> > > > >>>> 7f03d25f60e7 UefiPayloadPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 0bfd8d9b5ac9 UefiCpuPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> e1cd9bfb9dea StandaloneMmPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 5da2f65be378 SourceLevelDebugPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 95b86de07e5d SignedCapsulePkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> fe71d97246c4 ShellPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 54c21c952992 SecurityPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 187a3785f12b RedfishPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 810100002a46 PcAtChipsetPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 276a695c0cf2 OvmfPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 303c0a91ab07 NetworkPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> bc80792cd1b1 MdePkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 3ea86be17a2a MdeModulePkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> c70ef11ed0cd IntelFsp2WrapperPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> c0291221f252 IntelFsp2Pkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 6a479952a690 FmpDevicePkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 3a7c05b7070d FatPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> b789f98c8959 EmulatorPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 952d7a1c9220 EmbeddedPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> a1cc9881bab6 DynamicTablesPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 50654dfe5785 CryptoPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> ed965a02dfa1 ArmVirtPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 9744023fbc46 ArmPlatformPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 7a1cde5f5bba ArmPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 19d17e0913e8 UefiCpuPkg: Change use of EFI_D_* to DEBUG_*
> > > > >>>>
> > > > >>>> ffa718b4f994 SourceLevelDebugPkg: Change use of EFI_D_* to
> DEBUG_*
> > > > >>>>
> > > > >>>> b86cb3c5e5b4 ShellPkg: Change use of EFI_D_* to DEBUG_*
> > > > >>>>
> > > > >>>> c7c42204dc07 SecurityPkg: Change use of EFI_D_* to DEBUG_*
> > > > >>>>
> > > > >>>> 16b8e6f958e4 PcAtChipsetPkg: Change use of EFI_D_* to
> DEBUG_*
> > > > >>>>
> > > > >>>> 0ac3f8b2dac5 OvmfPkg: Change use of EFI_D_* to DEBUG_*
> > > > >>>>
> > > > >>>> bc5004b8d294 NetworkPkg: Change use of EFI_D_* to DEBUG_*
> > > > >>>>
> > > > >>>> 6f671a8e2377 MdePkg: Change use of EFI_D_* to DEBUG_*
> > > > >>>>
> > > > >>>> a10c610ff9a3 MdeModulePkg: Change use of EFI_D_* to
> DEBUG_*
> > > > >>>>
> > > > >>>> 09a3bddba390 FatPkg: Change use of EFI_D_* to DEBUG_*
> > > > >>>>
> > > > >>>> 59c61318246a EmulatorPkg: Change use of EFI_D_* to DEBUG_*
> > > > >>>>
> > > > >>>> 3a80367dda3b EmbeddedPkg: Change use of EFI_D_* to DEBUG_*
> > > > >>>>
> > > > >>>> 23eb1aaf80ca ArmVirtPkg: Change use of EFI_D_* to DEBUG_*
> > > > >>>>
> > > > >>>> 875914b45c54 ArmPlatformPkg: Change use of EFI_D_* to
> DEBUG_*
> > > > >>>>
> > > > >>>> eb2eca82b451 ArmPkg: Change use of EFI_D_* to DEBUG_*
> > > > >>>>
> > > > >>>> f0f3f5aae7c4 (origin/master, origin/HEAD, master)
> UnitTestFrameworkPkg: Update YAML to ignore specific ECC
> > > > >>> files/errors
> > > > >>>>
> > > > >>>> c05734797790 UefiPayloadPkg: Update YAML to ignore specific
> ECC files/errors
> > > > >>>>
> > > > >>>> c30c40d6c63d StandaloneMmPkg: Update YAML to ignore specific
> ECC files/errors
> > > > >>>>
> > > > >>>> 9944508e85f1 ShellPkg: Update YAML to ignore specific ECC
> files/errors
> > > > >>>>
> > > > >>>> 60fa40be458d SecurityPkg: Update YAML to ignore specific ECC
> files/errors
> > > > >>>>
> > > > >>>> df790cd6b37e MdePkg: Update YAML to ignore specific ECC
> files/errors
> > > > >>>>
> > > > >>>> 9deb9370766e MdeModulePkg: Update YAML to ignore specific
> ECC files/errors
> > > > >>>>
> > > > >>>> d7d30e8f219f EmulatorPkg: Update YAML to ignore specific ECC
> files/errors
> > > > >>>>
> > > > >>>> d5744ecba813 CryptoPkg: Update YAML to ignore specific ECC
> files/errors
> > > > >>>>
> > > > >>>> c97fee87f0f9 ArmVirtPkg: Update YAML to ignore specific ECC
> files/errors
> > > > >>>>
> > > > >>>> 1939fc9569f2 ArmPlatformPkg: Update YAML to ignore specific
> ECC files/errors
> > > > >>>>
> > > > >>>> 365dced2c37a ArmPkg: Update YAML to ignore specific ECC
> files/errors
> > > > >>>>
> > > > >>>> 76a1ce4d5fec .azurepipelines/templates: Update max pipeline job
> time to 2 hours
> > > > >>>>
> > > > >>>> 99f84ff47390 .pytools/Plugin/LicenseCheck: Use temp directory
> for git diff output
> > > > >>>>
> > > > >>>> 3019f1bbabf1 .pytool/Plugin/EccCheck: Add performance
> optimizations
> > > > >>>>
> > > > >>>> 854462bd3479 .pytool/Plugin/EccCheck: Remove temp directory
> on exception
> > > > >>>>
> > > > >>>> 69877614fdee .pytool/Plugin/EccCheck: Remove RevertCode()
> > > > >>>>
> > > > >>>> --ref1
> > > > >>>>
> > > > >>>> ef9a059cdb15 EmulatorPkg/Win/Host: Update CC_FLAGS
> > > > >>>>
> > > > >>>> bb1bba3d7767 (tag: edk2-stable202111) NetworkPkg: Fix invalid
> pointer for DNS response token on error
> > > > >>>>
> > > > >>>> Best regards,
> > > > >>>>
> > > > >>>> Mike
> > > > >>>>
> > > > >>>>
>
>
>
>
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
2021-12-06 1:17 ` Michael D Kinney
` (5 preceding siblings ...)
2021-12-07 3:47 ` Wang, Jian J
@ 2021-12-07 4:21 ` Chiu, Chasel
2021-12-07 10:02 ` Sami Mujawar
7 siblings, 0 replies; 39+ messages in thread
From: Chiu, Chasel @ 2021-12-07 4:21 UTC (permalink / raw)
To: devel@edk2.groups.io, Kinney, Michael D, Michael Kubacki,
maciej.rabeda@linux.intel.com, Michael Kubacki,
Andrew Fish (afish@apple.com), Leif Lindholm
For below 2 packages, Reviewed-by: Chasel Chiu <chasel.chiu@intel.com>
Thanks,
Chasel
* IntelFsp2Pkg
* IntelFsp2WrapperPkg
> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Michael D
> Kinney
> Sent: Monday, December 6, 2021 9:18 AM
> To: Michael Kubacki <mikuback@linux.microsoft.com>; devel@edk2.groups.io;
> maciej.rabeda@linux.intel.com; Michael Kubacki
> <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com)
> <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>; Kinney, Michael D
> <michael.d.kinney@intel.com>
> Subject: Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended
> Hard Freeze Update #4
>
> Hello EDK II Maintainers,
>
> A detailed evaluation of the DEBUG_CODE() formatting issue has been
> completed.
> The reason DEBUG_CODE() is a challenge is that this looks like a macro from a
> C parsing perspective, but the EDK II usage places C statements or blocks of
> C code as the parameter to this macro.
>
> There are actually 2 methods available to mark a statement or a block of code
> to be included when the DEBUG_PROPERTY_DEBUG_CODE_ENABLED bit is
> enabled in
> PcdDebugPropertyMask. One is DEBUG_CODE() and the other is to mark the
> beginning
> and end of a code block with DEBUG_CODE_BEGIN() and DEBUG_CODE_END().
> In fact,
> DEBUG_CODE() is implemented using DEBUG_CODE_BEGIN() and
> DEBUG_CODE_END() macros.
>
> #define DEBUG_CODE(Expression) \
> DEBUG_CODE_BEGIN (); \
> Expression \
> DEBUG_CODE_END ()
>
> A complete review for the use of these DEBUG_CODE macros was performed on
> the
> edk2 repo. Uncrustify performs good formatting for code blocks between
> DEBUG_CODE_BEGIN() and DEBUG_CODE_END(). This is because these look
> like simple
> macros calls with no parameters and the lines of C code between these 2
> macros
> is formatted correctly.
>
> The uncrustify formatting issues are only present with the use of DEBUG_CODE().
> Simple use cases of DEBUG_CODE(Expression) where Expression is a single C
> statement also look correct. A medium complexity use case where Expression is
> a code block of simple statements or even some local variables and simple
> statements also look correct. It is only complex code blocks that use C
> statements such as if/for/while/case that include the use of braces {} does
> uncrustify perform incorrect formatting.
>
> The recommended solution to this issue is to convert the use of DEBUG_CODE()
> to DEBUG_CODE_BEGIN() / DEBUG_CODE_END() for cases where the
> Expression
> passed to DEBUG_CODE() is the complex use case that contains statements that
> use braces {}. There are 57 instances of this pattern across 40 files in the
> edk2 repo.
>
> I have posted a branch with these additional patches:
>
>
> https://github.com/mdkinney/edk2/tree/Bug_3737_3760_3739_ApplyUncrustif
> yChanges_V7
>
> I have performed CompareBuild tests with this revised patch series with
> the DEBUG_CODE changes. It passes 100% showing no binary differences.
>
> https://github.com/mdkinney/edk2/actions/runs/1542454606
>
> I have opened a PR to run this patch series through EDK II CI. It also passes 100%.
>
> https://github.com/tianocore/edk2/pull/2236
>
> The summary of changes made since the V6 review are:
>
> 1) Change uncrustify configuration assignment alignment threshold to 0
>
> align_assign_thresh = 0
>
> 2) Replace "<param>, OPTIONAL" with "<param> OPTIONAL,"
>
> 3) Replace DEBUG_CODE(Expression) with
>
> DEBUG_CODE_BEGIN();
> Expression
> DEBUG_CODE_END()
>
> if Expression is complex and contains braces {}.
>
> 4) No changes to uncrustify tool required.
>
> Please review the differences between the following 2 branches and provide
> feedback or a Series Reviewed-by if you agree with these additional changes.
>
>
> https://github.com/mdkinney/edk2/tree/Bug_3737_3760_3739_ApplyUncrustif
> yChanges_V6
>
> https://github.com/mdkinney/edk2/tree/Bug_3737_3760_3739_ApplyUncrustif
> yChanges_V7
>
> The goal is to complete the review and get the uncrustify change committed
> tomorrow so the extended hard freeze can be lifted.
>
> Thanks,
>
> Mike
>
> > -----Original Message-----
> > From: Kinney, Michael D <michael.d.kinney@intel.com>
> > Sent: Thursday, December 2, 2021 6:23 PM
> > To: Michael Kubacki <mikuback@linux.microsoft.com>; devel@edk2.groups.io;
> maciej.rabeda@linux.intel.com; Michael Kubacki
> > <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com)
> <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>;
> > Kinney, Michael D <michael.d.kinney@intel.com>
> > Subject: RE: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended
> Hard Freeze Update #4
> >
> > Hello EDK II Maintainers,
> >
> > I have entered BZ 3760 to make the use of the OPTIONAL keyword style
> consistent for all of edk2 repo
> > and to be compatible with uncrustify.
> >
> > I have posted the following V6 branch that does the EFI_D_* to DEBUG_*
> changes, the OPTIONAL keyword
> > style changes, and the uncrustify changes with the one configuration change
> for assignment alignment.
> >
> >
> https://github.com/mdkinney/edk2/tree/Bug_3737_3760_3739_ApplyUncrustif
> yChanges_V6
> >
> > Please provide feedback on the code style in this branch with the known
> DEBUG_CODE() issue still
> > present.
> >
> > If we are able to quickly update uncrustify to handle DEBUG_CODE(), I will
> generate a V7.
> >
> > Thanks,
> >
> > Mike
> >
> > > -----Original Message-----
> > > From: Kinney, Michael D <michael.d.kinney@intel.com>
> > > Sent: Thursday, December 2, 2021 4:53 PM
> > > To: Michael Kubacki <mikuback@linux.microsoft.com>;
> devel@edk2.groups.io; maciej.rabeda@linux.intel.com; Michael Kubacki
> > > <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com)
> <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>;
> > > Kinney, Michael D <michael.d.kinney@intel.com>
> > > Subject: RE: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended
> Hard Freeze Update #4
> > >
> > > Michael,
> > >
> > > Yes. Please update the patch series that adds UncrustifyCheck with those
> changes.
> > >
> > > Thanks,
> > >
> > > Mike
> > >
> > > > -----Original Message-----
> > > > From: Michael Kubacki <mikuback@linux.microsoft.com>
> > > > Sent: Thursday, December 2, 2021 4:31 PM
> > > > To: Kinney, Michael D <michael.d.kinney@intel.com>;
> devel@edk2.groups.io; maciej.rabeda@linux.intel.com; Michael
> > Kubacki
> > > > <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com)
> <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>
> > > > Subject: Re: [edk2-devel] Uncrustify Conversion Detailed Plan and
> Extended Hard Freeze Update #4
> > > >
> > > > Hi Mike,
> > > >
> > > > Thank you for the detailed analysis and recommendations.
> > > >
> > > > I agree with the recommendations and I will try to have an Uncrustify
> > > > tool update by tomorrow for (4).
> > > >
> > > > That will require an update in uncrustify_ext_dep.yaml to pull in the
> > > > new version. I'm assuming I should also update uncrustify.cfg to set
> > > > align_assign_thresh = 0 in that new patch series?
> > > >
> > > > Regards,
> > > > Michael
> > > >
> > > > On 12/2/2021 7:18 PM, Kinney, Michael D wrote:
> > > > > Hi Michael,
> > > > >
> > > > > CORRECTION: set align_assign_threshold to 0.
> > > > >
> > > > > Reponses inline below.
> > > > >
> > > > > I would like to summarize the 4 issues raised in the past day along with
> the recommendations.
> > > > >
> > > > > 1) Exclusion feature for UncrustifyCheck. There are 2 directories with 8
> files total that
> > > > > maintainers have noted they would like to see not go through
> uncrustify formatting. Today
> > > > > the only content that is skipped is BaseTools and submodules.
> > > > >
> > > > > Adding a general purpose exclusion feature would then require all
> developers to make
> > > > > sure their method of using uncrustify also excludes those same areas.
> This requires
> > > > > extra steps for all developers and maintainers.
> > > > >
> > > > > If we do not add the exclusion feature, then the 8 files will require an
> extra step
> > > > > to sync with the original source of those files. The rate of changes of
> these 8 files
> > > > > is very low today.
> > > > >
> > > > > RECOMMENDATION: Do not add exclusion feature at this time. Revisit
> if the extra work
> > > > > to maintain the files that would be candidates for exclusions increases
> significantly.
> > > > >
> > > > > 2) Alignment of assignments. The threshold of 4 characters appears to
> be too low and causes
> > > > > source files that are already aligned to become unaligned.
> > > > >
> > > > > RECOMMENDATION: Change threshold to the default value of 0 which
> means no limit.
> > > > >
> > > > > align_assign_thresh= 0
> > > > >
> > > > > 3) Alignment of parameters in function declaration not correct. The root
> cause of this
> > > > > is the use of the OPTIONAL keyword. If the OPTIONAL keyword is
> removed, then the
> > > > > alignment is correct. The alignment is also correct if the OPTIONAL
> keyword appears
> > > > > before the ','. If the OPTIONAL keyword appears after the ',', then the
> format is
> > > > > not correct. The OPTIONAL keyword indicates that the parameter in
> the function is
> > > > > not required and may be passed in as NULL or 0 or some other default
> value defined by
> > > > > the API. It makes more sense for this OPTIONAL keyword that follows
> the parameter
> > > > > names to appear before the ',' so it is scoped to the parameter on that
> line. If it
> > > > > appears after the ',', then C parsers thinks it is a prefix (IN, OUT, CONST,
> volatile,
> > > > > static) for the next parameter in the function.
> > > > >
> > > > > RECOMMENDATION: Update patch series with a global search and
> replace so OPTIONAL
> > > > > keyword always appears before the ',' on the same line.
> > > > >
> > > > > RegEx search string: ',( *)OPTIONAL( *)'
> > > > > RegEx replace string: ' $1OPTIONAL,$2'
> > > > >
> > > > > 4) Format issues with complex blocks in DEBUG_CODE(), or between
> DEBUG_CODE_BEGIN() and
> > > > > DEBUG_CODE_END(). Uncrustify treats these as macros and is not
> aware that the
> > > > > parameter passed into the macro call is a block of C code that needs to
> be formatted.
> > > > > Complex blocks with if/while/for/case statements are impacted the
> most.
> > > > >
> > > > > RECOMMENDATION: Update the uncrustify with an edk2 specific
> extension to treat these
> > > > > macros as a block of code as if they were surrounded by an extra set of
> braces {}.
> > > > >
> > > > >
> > > > > I have posted a branch for testing purposes that implements (2) and (3).
> > > > >
> > > > > Branch:
> https://github.com/mdkinney/edk2/tree/TestOnly_Bug_3737_3739_ApplyUncru
> stifyChanges_V6_OPTIONAL_Keyword_Fix
> > > > > PR: https://github.com/tianocore/edk2/pull/2233
> > > > > Status: PASS
> > > > > CompareBuild:
> https://github.com/mdkinney/edk2/actions/runs/1532855914
> > > > > Status: PASS
> > > > >
> > > > > You can see what changed by fetching and comparing the following 2
> branches:
> > > > >
> > > > >
> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChan
> ges_V5
> > > > >
> https://github.com/mdkinney/edk2/tree/TestOnly_Bug_3737_3739_ApplyUncru
> stifyChanges_V6_OPTIONAL_Keyword_Fix
> > > > >
> > > > > Please provide feedback on the RECOMMENDATIONS above. I will go
> ahead and prepare of V6 version of
> > > > > the patch series now that that test results are all PASS.
> > > > >
> > > > > Best regards,
> > > > >
> > > > > Mike
> > > > >
> > > > >
> > > > >> -----Original Message-----
> > > > >> From: Kinney, Michael D <michael.d.kinney@intel.com>
> > > > >> Sent: Thursday, December 2, 2021 4:15 PM
> > > > >> To: Michael Kubacki <mikuback@linux.microsoft.com>;
> devel@edk2.groups.io; maciej.rabeda@linux.intel.com; Michael
> > > > Kubacki
> > > > >> <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com)
> <afish@apple.com>; Leif Lindholm
> > <leif@nuviainc.com>;
> > > > >> Kinney, Michael D <michael.d.kinney@intel.com>
> > > > >> Subject: RE: [edk2-devel] Uncrustify Conversion Detailed Plan and
> Extended Hard Freeze Update #4
> > > > >>
> > > > >> Hi Michael,
> > > > >>
> > > > >> Reponses inline below.
> > > > >>
> > > > >> I would like to summarize the 4 issues raised in the past day along with
> the recommendations.
> > > > >>
> > > > >> 1) Exclusion feature for UncrustifyCheck. There are 2 directories with 8
> files total that
> > > > >> maintainers have noted they would like to see not go through
> uncrustify formatting. Today
> > > > >> the only content that is skipped is BaseTools and submodules.
> > > > >>
> > > > >> Adding a general purpose exclusion feature would then require all
> developers to make
> > > > >> sure their method of using uncrustify also excludes those same areas.
> This requires
> > > > >> extra steps for all developers and maintainers.
> > > > >>
> > > > >> If we do not add the exclusion feature, then the 8 files will require an
> extra step
> > > > >> to sync with the original source of those files. The rate of changes of
> these 8 files
> > > > >> is very low today.
> > > > >>
> > > > >> RECOMMENDATION: Do not add exclusion feature at this time.
> Revisit if the extra work
> > > > >> to maintain the files that would be candidates for exclusions increases
> significantly.
> > > > >>
> > > > >> 2) Alignment of assignments. The threshold of 4 characters appears to
> be too low and causes
> > > > >> source files that are already aligned to become unaligned.
> > > > >>
> > > > >> RECOMMENDATION: Change threshold to the default value of 0
> which means no limit.
> > > > >>
> > > > >> align_assign_thresh= 4
> > > > >>
> > > > >> 3) Alignment of parameters in function declaration not correct. The
> root cause of this
> > > > >> is the use of the OPTIONAL keyword. If the OPTIONAL keyword is
> removed, then the
> > > > >> alignment is correct. The alignment is also correct if the OPTIONAL
> keyword appears
> > > > >> before the ','. If the OPTIONAL keyword appears after the ',', then the
> format is
> > > > >> not correct. The OPTIONAL keyword indicates that the parameter in
> the function is
> > > > >> not required and may be passed in as NULL or 0 or some other default
> value defined by
> > > > >> the API. It makes more sense for this OPTIONAL keyword that follows
> the parameter
> > > > >> names to appear before the ',' so it is scoped to the parameter on
> that line. If it
> > > > >> appears after the ',', then C parsers thinks it is a prefix (IN, OUT,
> CONST, volatile,
> > > > >> static) for the next parameter in the function.
> > > > >>
> > > > >> RECOMMENDATION: Update patch series with a global search and
> replace so OPTIONAL
> > > > >> keyword always appears before the ',' on the same line.
> > > > >>
> > > > >> RegEx search string: ',( *)OPTIONAL( *)'
> > > > >> RegEx replace string: ' $1OPTIONAL,$2'
> > > > >>
> > > > >> 4) Format issues with complex blocks in DEBUG_CODE(), or between
> DEBUG_CODE_BEGIN() and
> > > > >> DEBUG_CODE_END(). Uncrustify treats these as macros and is not
> aware that the
> > > > >> parameter passed into the macro call is a block of C code that needs
> to be formatted.
> > > > >> Complex blocks with if/while/for/case statements are impacted the
> most.
> > > > >>
> > > > >> RECOMMENDATION: Update the uncrustify with an edk2 specific
> extension to treat these
> > > > >> macros as a block of code as if they were surrounded by an extra set
> of braces {}.
> > > > >>
> > > > >>
> > > > >> I have posted a branch for testing purposes that implements (2) and (3).
> > > > >>
> > > > >> Branch:
> https://github.com/mdkinney/edk2/tree/TestOnly_Bug_3737_3739_ApplyUncru
> stifyChanges_V6_OPTIONAL_Keyword_Fix
> > > > >> PR: https://github.com/tianocore/edk2/pull/2233
> > > > >> Status: PASS
> > > > >> CompareBuild:
> https://github.com/mdkinney/edk2/actions/runs/1532855914
> > > > >> Status: PASS
> > > > >>
> > > > >> You can see what changed by fetching and comparing the following 2
> branches:
> > > > >>
> > > > >>
> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChan
> ges_V5
> > > > >>
> https://github.com/mdkinney/edk2/tree/TestOnly_Bug_3737_3739_ApplyUncru
> stifyChanges_V6_OPTIONAL_Keyword_Fix
> > > > >>
> > > > >> Please provide feedback on the RECOMMENDATIONS above. I will go
> ahead and prepare of V6 version of
> > > > >> the patch series now that that test results are all PASS.
> > > > >>
> > > > >> Best regards,
> > > > >>
> > > > >> Mike
> > > > >>
> > > > >>
> > > > >>> -----Original Message-----
> > > > >>> From: Michael Kubacki <mikuback@linux.microsoft.com>
> > > > >>> Sent: Thursday, December 2, 2021 1:57 PM
> > > > >>> To: devel@edk2.groups.io; Kinney, Michael D
> <michael.d.kinney@intel.com>; maciej.rabeda@linux.intel.com; Michael
> > > > Kubacki
> > > > >>> <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com)
> <afish@apple.com>; Leif Lindholm
> > <leif@nuviainc.com>
> > > > >>> Subject: Re: [edk2-devel] Uncrustify Conversion Detailed Plan and
> Extended Hard Freeze Update #4
> > > > >>>
> > > > >>> My reply is inline.
> > > > >>>
> > > > >>> Regards,
> > > > >>> Michael
> > > > >>>
> > > > >>> On 12/2/2021 2:45 PM, Michael D Kinney wrote:
> > > > >>>> Hi Maciej,
> > > > >>>>
> > > > >>>> Thanks for the feedback.
> > > > >>>>
> > > > >>>> * Example #1.This appears to be caused by the following uncrustify
> > > > >>>> settings:
> > > > >>>>
> > > > >>>> # The threshold for aligning on '=' in assignments.
> > > > >>>>
> > > > >>>> # Use a negative number for absolute thresholds.
> > > > >>>>
> > > > >>>> #
> > > > >>>>
> > > > >>>> # 0: No limit (default).
> > > > >>>>
> > > > >>>> align_assign_thresh= 0# number
> > > > >>>>
> > > > >>>> The edk2 setting for this is:
> > > > >>>>
> > > > >>>> align_assign_thresh= 4
> > > > >>>>
> > > > >>>> This means blocks of assignments that are different than more than 4
> > > > >>>> spaces will be considered a new block.
> > > > >>>>
> > > > >>>> ‘HwAddreLen’ and ‘Xid’ are more than 4 characters in length
> > > > >>>> different.Same for ‘Xid’ and ‘Reserved’.So
> > > > >>>>
> > > > >>>> uncrustify treats these as 3 different assignment blocks.
> > > > >>>>
> > > > >>>> If we change to the default value of 0: No limit, this example is
> > > > >>>> treated as a single block and all ‘=’ are aligned.
> > > > >>>>
> > > > >>>> Is there a reason ‘4’ was selected?Is there is any harm in using the
> > > > >>>> default of 0?
> > > > >>>>
> > > > >>> We can certainly change the threshold. '4' was derived by
> > > > >>> experimentation. This is an area that is somewhat subjective and every
> > > > >>> case is difficult to cover well. Please feel free to suggest any changes.
> > > > >>>
> > > > >>
> > > > >> I recommend we use the default value of 0. That way, any code that
> choose to
> > > > >> align assignments will still be aligned. If a developer does not like
> > > > >> short assignments and long assignments in the same code block to use
> the
> > > > >> long assignment column, they can always break the block up into
> multiple
> > > > >> blocks by adding a carriage return.
> > > > >>
> > > > >>> For the benefit of others, this file can be a useful refernce
> > > > >>> understanding spans, gaps, and thresholds.
> > > > >>>
> > > > >>>
> https://github.com/uncrustify/uncrustify/blob/master/documentation/htdocs/c
> onfiguration.txt
> > > > >>>
> > > > >>>> * Example #2: Uncruistfy is confused by the DEBUG_CODE()
> macro.This is
> > > > >>>> not a traditional macro because the
> > > > >>>>
> > > > >>>> contents of the macro is a block of C code.I do not know how to
> convince
> > > > >>>> uncrustify that the contents of a
> > > > >>>>
> > > > >>>> macro like function call to be treated as a code block from an indent
> > > > >>>> perspective.
> > > > >>>>
> > > > >>> I believe this would have to be overridden in the Uncrustify fork since
> > > > >>> DEBUG_CODE() is being treated as a macro function call and code
> blocks
> > > > >>> are not expected there. In addition, some special treatment might be
> > > > >>> needed for alignment in between
> DEBUG_CODE_BEGIN()/DEBUG_CODE_END().
> > > > >>>
> > > > >>> I'm happy to look at this. However, regression validation can take a
> > > > >>> while so I'd like to make sure we need this now. In cases that do not
> > > > >>> have additional code blocks, it seems to format fairly well. Is this
> > > > >>> prevalent and impactful enough it must be fixed now? Or, could we
> > > > >>> revisit it with a follow up patch?
> > > > >>
> > > > >> How long does regression testing take? I see about 20 instances of
> > > > >> DEBUG_CODE() that are producing bad formatting. If the content inside
> > > > >> DEBUG_CODE() is a single line of a single block of statements without
> > > > >> any if/while/for/case statements that require further indent, then the
> > > > >> format looks ok.
> > > > >>
> > > > >> DEBUG_CODE_BEGIN() and DEBUG_CODE_END() would look better if
> the
> > > > >> contents between are also indented one level.
> > > > >>
> > > > >>>
> > > > >>>> * Example #3/#4: Uncrustify is confused by the OPTIONAL
> keyword.The
> > > > >>>> edk2 config declares it as a QUALIFIER
> > > > >>>>
> > > > >>>> like IN and OUT.However, OPTIONAL only appears at the end of the
> line
> > > > >>>> that declares a parameter to a
> > > > >>>>
> > > > >>>> function.There are 3 forms. One with comma after OPTIONAL.One
> with comma
> > > > >>>> before OPTIONAL, and
> > > > >>>>
> > > > >>>> one with no comma if the parameter is the last parameter in the
> function
> > > > >>>> declaration.
> > > > >>>>
> > > > >>>> TYPE ParamName OPTIONAL,
> > > > >>>>
> > > > >>>> TYPEParamName, OPTIONAL
> > > > >>>>
> > > > >>>> TYPEParamName OPTIONAL
> > > > >>>>
> > > > >>>> OPTIONAL is defined to nothing in edk2 builds.From a uncrustify
> > > > >>>> perspective, we really want is to be
> > > > >>>>
> > > > >>>> ignored or more correctly treated it as a token that is attached to
> > > > >>>> ParamName and the combination of
> > > > >>>>
> > > > >>>> ParamName and OPTIONAL treated as one unit to determine indents.
> > > > >>>>
> > > > >>> "TYPE ParamName, OPTIONAL" seems especially problematic and
> presents an
> > > > >>> inconsistency with the other formats. Mike, can you please let me
> know
> > > > >>> if you have the same observation and your thoughts on a consistent
> pattern?
> > > > >>>
> > > > >>>> Thanks,
> > > > >>>>
> > > > >>>> Mike
> > > > >>>>
> > > > >>>> *From:*devel@edk2.groups.io <devel@edk2.groups.io> *On Behalf
> Of *Maciej
> > > > >>>> Rabeda
> > > > >>>> *Sent:* Thursday, December 2, 2021 10:27 AM
> > > > >>>> *To:* devel@edk2.groups.io; Kinney, Michael D
> > > > >>>> <michael.d.kinney@intel.com>; Michael Kubacki
> > > > >>>> <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com)
> > > > >>>> <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>
> > > > >>>> *Subject:* Re: [edk2-devel] Uncrustify Conversion Detailed Plan and
> > > > >>>> Extended Hard Freeze Update #4
> > > > >>>>
> > > > >>>> Hey Mike,
> > > > >>>>
> > > > >>>> While most of the changes related to fixing coding style violations,
> > > > >>>> there are a couple of changes to NetworkPkg in that PR that make
> the
> > > > >>>> code less readable. Examples below.
> > > > >>>>
> > > > >>>> Example 1:
> > > > >>>>
> > > > >>>>
> > > > >>>>
> > > > >>>> Example 2:
> > > > >>>>
> > > > >>>>
> > > > >>>> Example 3:
> > > > >>>>
> > > > >>>>
> > > > >>>> Example 4:
> > > > >>>>
> > > > >>>> On 30-Nov-21 23:34, Michael D Kinney wrote:
> > > > >>>>
> > > > >>>> Hello,
> > > > >>>>
> > > > >>>> Thank you for your patience during this extended hard freeze.
> > > > >>>>
> > > > >>>> Just one more step to go.There has been a delay in the review of
> > > > >>>>
> > > > >>>> the patch series with the uncrustify source changes.PR(6).This
> > > > >>>>
> > > > >>>> patch series was not sent out as patch review email because of its
> > > > >>>>
> > > > >>>> very large size.It only contains source style changes and the
> > > > >>>>
> > > > >>>> CompareBuild tool and GitHub action has shown there are no
> binary
> > > > >>>>
> > > > >>>> differences introduced with these source style changes.
> > > > >>>>
> > > > >>>> If you are a package maintainer, then please review the following
> > > > >>>>
> > > > >>>> branch/PR for your package contents and review the EDK II CI
> results
> > > > >>>>
> > > > >>>> and BuildCompare results.I do not expect a line by line review
> > > > >>>>
> > > > >>>> because we already had time to provide feedback on the source
> style
> > > > >>>>
> > > > >>>> performed by uncrustify.Instead, a Reviewed-by for your package
> > > > >>>>
> > > > >>>> indicates that you have reviewed the EDK II CI results and
> CompareBuild
> > > > >>>>
> > > > >>>> tool functionality and results and you accept the source style
> > > > >>>>
> > > > >>>> changes to your package.
> > > > >>>>
> > > > >>>>
> *https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyCha
> nges_V5
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyCha
> nges_V5>
> > > > >>>>
> > > > >>>> *https://github.com/tianocore/edk2/pull/2229
> <https://github.com/tianocore/edk2/pull/2229>
> > > > >>>>
> > > > >>>> *https://github.com/mdkinney/edk2/actions/runs/1521618836
> > > > >>> <https://github.com/mdkinney/edk2/actions/runs/1521618836>
> > > > >>>>
> > > > >>>> Additional details on this update below.
> > > > >>>>
> > > > >>>> Thank you,
> > > > >>>>
> > > > >>>> Mike
> > > > >>>>
> > > > >>>> Changes from Update #3
> > > > >>>>
> > > > >>>> ----------------------------------------------------------------------------
> > > > >>>>
> > > > >>>> * Pushed PR (5)
> > > > >>>>
> > > > >>>> * Added link to PR(6). EDK II CI Status is PASS. Build Compare PASS.
> > > > >>>>
> > > > >>>> * Waiting for review of PR (6)
> > > > >>>>
> > > > >>>> * Review of PR (7) completed and waiting for review of PR (6)
> > > > >>>>
> > > > >>>> ----------------------------------------------------------------------------
> > > > >>>>
> > > > >>>> Changes from Update #2
> > > > >>>>
> > > > >>>> ----------------------------------------------------------------------------
> > > > >>>>
> > > > >>>> * Changed order of PRs swapping (4) and (5).The PR that activates
> > > > >>>>
> > > > >>>> increases the max CI agent job time is independent of all the other
> > > > >>>>
> > > > >>>> PRs and its review is complete, so it can be committed now.
> > > > >>>>
> > > > >>>> * Pushed PRs (1), (2), (3), (4).
> > > > >>>>
> > > > >>>> * Waiting for review to complete for PRs (5) and (6)
> > > > >>>>
> > > > >>>> * Reviews complete for PR (7)
> > > > >>>>
> > > > >>>> * Identifies steps using git filter-branch to apply uncrustify changes
> to a
> > > > >>>>
> > > > >>>> code review patch series that was generated before the uncrustify
> changes
> > > > >>>>
> > > > >>>> avoiding manual merge.
> > > > >>>>
> > > > >>>> * Identified steps using git filter-repo to generate an alternate
> history of
> > > > >>>>
> > > > >>>> the edk2 repo with uncrustify changes applied on every
> commit.This may
> > > > >>>>
> > > > >>>> be useful when evaluating changes to files using tools like git
> blame
> > > > >>>>
> > > > >>>> without the large uncrustify patch series.
> > > > >>>>
> > > > >>>> ---------------------------------------------------------------------------
> > > > >>>>
> > > > >>>> Changes from Update #1
> > > > >>>>
> > > > >>>> ----------------------------------------------------------------------------
> > > > >>>>
> > > > >>>> * Changed order of PRs swapping (6) and (7).The PR that activates
> > > > >>>>
> > > > >>>> EDK II CI check UncrustifyCheck has to be last because it
> unconditionally
> > > > >>>>
> > > > >>>> checks all C/H files in all packages.Not just files that have been
> > > > >>>>
> > > > >>>> modified like some of the other checkers.
> > > > >>>>
> > > > >>>> * Updated link to the branch with the UncrustifyCheck plugin that
> has been
> > > > >>>>
> > > > >>>> updated with a one line change and Reviewed-by and Tested-by
> tags.
> > > > >>>>
> > > > >>>>
> https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v
> 6
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_
> v6>
> > > > >>>>
> > > > >>>> * Reviews complete for (1), (2), (3), (5), and (7)
> > > > >>>>
> > > > >>>> ---------------------------------------------------------------------------
> > > > >>>>
> > > > >>>> Michael Kubacki and I have prepared the patches required to apply
> the
> > > > >>>>
> > > > >>>> uncrustify changes and enable EDK II CI to check all submitted
> > > > >>>>
> > > > >>>> patches have been run through uncrustify.
> > > > >>>>
> > > > >>>> We have verified through the CompareBuild GitHub Action that the
> > > > >>>>
> > > > >>>> format changes performed by uncrustify have no functional
> changes.
> > > > >>>>
> > > > >>>> All of the OBJ, LIB, DLL, EFI, FFS, FV, and FD files match 100%
> > > > >>>>
> > > > >>>> across 70 VS2019/GCC5 builds of all package/platform DSC files in
> > > > >>>>
> > > > >>>> the edk2 repo.
> > > > >>>>
> > > > >>>> The hard freeze will be extended after the edk2-stable202111 tag
> until
> > > > >>>>
> > > > >>>> all uncrustify related changes are committed.We do not expect
> this
> > > > >>>>
> > > > >>>> to take more than a few days.Do not push any PRs until the hard
> > > > >>>>
> > > > >>>> freeze is lifted.
> > > > >>>>
> > > > >>>> The changes are broken up into 7 patch series/PRs.The PRs are
> ordered
> > > > >>>>
> > > > >>>> so they can be submitted using the normal submission process and
> EDK II
> > > > >>>>
> > > > >>>> CI will pass for each one.Details are listed below.
> > > > >>>>
> > > > >>>> Uncrustify 73.0.3 for EDK II
> > > > >>>>
> > > > >>>> =============================
> > > > >>>>
> > > > >>>> * Sources:https://dev.azure.com/projectmu/_git/Uncrustify
> <https://dev.azure.com/projectmu/_git/Uncrustify>
> > > > >>>>
> > > > >>>> *
> Documentation:https://dev.azure.com/projectmu/Uncrustify/_wiki/wikis/Uncru
> stify.wiki/1/Project-Mu-(EDK-
> > II)-
> > > > Fork-
> > > > >>> Readme
> <https://dev.azure.com/projectmu/Uncrustify/_wiki/wikis/Uncrustify.wiki/1/Pr
> oject-Mu-(EDK-II)-Fork-Readme>
> > > > >>>>
> > > > >>>> *
> Download:https://dev.azure.com/projectmu/Uncrustify/_packaging?_a=packag
> e&feed=mu_uncrustify&package=mu-
> > > > >>> uncrustify-release&protocolType=NuGet&version=73.0.3
> > > > >>>
> <https://dev.azure.com/projectmu/Uncrustify/_packaging?_a=package&feed=
> mu_uncrustify&package=mu-uncrustify-
> > > > >>> release&protocolType=NuGet&version=73.0.3>
> > > > >>>>
> > > > >>>> Installing Uncrustify
> > > > >>>>
> > > > >>>> ======================
> > > > >>>>
> > > > >>>> The Uncrustify tool is installed automatically when the Pytools
> > > > >>>>
> > > > >>>> environment is used and the stuart* commands are run to
> complete the
> > > > >>>>
> > > > >>>> environment setup.Please see:
> > > > >>>>
> > > > >>>> https://github.com/tianocore/edk2/tree/master/.pytool#running-
> ci-locally
> > > > >>> <https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-
> locally>
> > > > >>>>
> > > > >>>> Uncrustify can also be installed from the download page listed
> above
> > > > >>>>
> > > > >>>> or built from sources from the source link above.
> > > > >>>>
> > > > >>>> The Documentation link provides instruction on how to run
> uncrustify from
> > > > >>>>
> > > > >>>> the command line or install as a Visual Studio Code plugin.The
> main
> > > > >>>>
> > > > >>>> uncrustify documentation also describes how to integrate with a
> few other
> > > > >>>>
> > > > >>>> editors.
> > > > >>>>
> > > > >>>> We have also discussed a client side githook.That effort has not
> started.
> > > > >>>>
> > > > >>>> Let us know if that is a feature you would find useful.
> > > > >>>>
> > > > >>>> Developer impact for new code reviews
> > > > >>>>
> > > > >>>> ======================================
> > > > >>>>
> > > > >>>> Once the uncrustify checker is active in EDK II CI, developers must
> > > > >>>>
> > > > >>>> make sure their patches are run through the uncrustify tool before
> > > > >>>>
> > > > >>>> sending the patches for review.
> > > > >>>>
> > > > >>>> Developers must install and run uncrustify against changes files
> before
> > > > >>>>
> > > > >>>> sending patch review emails or submitting PR for EDK II CI.If EDK II
> CI
> > > > >>>>
> > > > >>>> detects and differences in source formatting, then EDK II CI will fail
> > > > >>>>
> > > > >>>> and the developer must run uncrustify and resubmit the patches.
> > > > >>>>
> > > > >>>> Developer impact to patch series/PRs reviewed during edk2-
> stable201121 soft/hard freeze
> > > > >>>>
> > > > >>>>
> =================================================================
> ======================
> > > > >>>>
> > > > >>>> Developers must rebase their changes after the uncrustify source
> changes are
> > > > >>>>
> > > > >>>> committed.The branch with a preview of the uncrustify changes
> can be used
> > > > >>>>
> > > > >>>> to start this rebase work.
> > > > >>>>
> > > > >>>>
> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChan
> ges_V5
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyCha
> nges_V5>
> > > > >>>>
> > > > >>>> The following steps can be used to update an existing branch with
> the
> > > > >>>>
> > > > >>>> required uncrustify format.This is the Windows version.I will add
> > > > >>>>
> > > > >>>> the Linux version soon.
> > > > >>>>
> > > > >>>> 1) Fetch and checkout and rebase to latest edk2/master
> > > > >>>>
> > > > >>>> git fetch origin
> > > > >>>>
> > > > >>>> git checkout master
> > > > >>>>
> > > > >>>> git rebase origin/master
> > > > >>>>
> > > > >>>> 2) Make a backup copy of plugin UncrustifyCheck outside
> WORKSPACE.
> > > > >>>>
> > > > >>>> (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable
> and
> > > > >>>>
> > > > >>>> EDK II specific uncrustify configuration file available when working
> > > > >>>>
> > > > >>>> with a branch that does not have those tools in its scope.
> > > > >>>>
> > > > >>>> xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
> > > > >>>>
> > > > >>>> 3) Check out the patch series branch (e.g. MyBranch)
> > > > >>>>
> > > > >>>> git checkout MyBranch
> > > > >>>>
> > > > >>>> 4) Rebase patch series against edk2-stable202111
> > > > >>>>
> > > > >>>>
> > > > >>>>
> > > > >>>> git rebase edk2-stable202111
> > > > >>>>
> > > > >>>> 5) Create new branch for the uncrustifed version (e.g.
> MyBranch_Uncrustified)
> > > > >>>>
> > > > >>>> git checkout -b MyBranch_Uncrustified
> > > > >>>>
> > > > >>>> 6) Use git filter-branch to uncrustify all the commits in the series
> > > > >>>>
> > > > >>>> between the rebase target from (2) and HEAD of the branch.A filter
> > > > >>>>
> > > > >>>> can be used to scope the uncrustify operations to only the C/H files
> > > > >>>>
> > > > >>>> in the specific package the patch series is against. (e.g.
> DynamicTablesPkg).
> > > > >>>>
> > > > >>>> BaseTools should always be excluded.If the package scoped filter
> is
> > > > >>>>
> > > > >>>> not used, it will still work, but will take longer to run because
> > > > >>>>
> > > > >>>> uncrustify will rescan every C/H files in the whole repo.
> > > > >>>>
> > > > >>>> git filter-branch --tree-filter "git ls-files DynamicTablesPkg*.c
> DynamicTablesPkg*.h :!BaseTools/* |
> > > > >>> c:\\Temp\\UncrustifyCheck\\mu-uncrustify-
> release_extdep\\Windows-x86\\uncrustify.exe -c
> > > > >>> c:\\Temp\\UncrustifyCheck\\uncrustify.cfg -F - --replace --no-backup -
> -if-changed" edk2-stable202111..HEAD
> > > > >>>>
> > > > >>>> 7) Now that all the individual patches in the branch are uncrustified,
> > > > >>>>
> > > > >>>> rebase against latest edk2/master that is already uncrustified.
> > > > >>>>
> > > > >>>> git rebase master
> > > > >>>>
> > > > >>>> 8) Verify the patches in this new branch.
> > > > >>>>
> > > > >>>> Impacts to tracing history across the uncrusity changes
> > > > >>>>
> > > > >>>>
> =======================================================
> > > > >>>>
> > > > >>>> Tools the view file and line history do work with the large
> uncrustify
> > > > >>>>
> > > > >>>> patch series.One impact is that the operations can be very slow
> due
> > > > >>>>
> > > > >>>> to the large uncrustify patches.
> > > > >>>>
> > > > >>>> One option to provide a faster experience is to provide an
> alternate
> > > > >>>>
> > > > >>>> version of the edk2 repository as "documentation" that has the
> > > > >>>>
> > > > >>>> entire history re-written with uncrustify run on every commit.
> > > > >>>>
> > > > >>>> The tool called git-filter-repo can be used to perform this
> > > > >>>>
> > > > >>>> transformation and runs in a reasonable period of time (a few
> hours)
> > > > >>>>
> > > > >>>> https://github.com/newren/git-filter-repo
> <https://github.com/newren/git-filter-repo>
> > > > >>>>
> > > > >>>> https://github.com/newren/git-filter-
> repo/blob/main/contrib/filter-repo-demos/lint-history
> > > > >>> <https://github.com/newren/git-filter-repo/blob/main/contrib/filter-
> repo-demos/lint-history>
> > > > >>>>
> > > > >>>> The following steps can be used to perform this transformation.
> > > > >>>>
> > > > >>>> This is the Windows version. I will add the Linux version soon.
> > > > >>>>
> > > > >>>> ** WARNING **This operation modifies(rewrites) all the commits
> > > > >>>>
> > > > >>>> in the local copy of the repo.Do not perform
> > > > >>>>
> > > > >>>> these steps on a local repo you are using for
> > > > >>>>
> > > > >>>> active development.
> > > > >>>>
> > > > >>>> 1) Clone edk2 into a new directory (see **WARNING**)
> > > > >>>>
> > > > >>>> git clonehttps://github.com/tianocore/edk2.git
> <https://github.com/tianocore/edk2.git> edk2-uncrustified
> > > > >>>>
> > > > >>>> cd edk2-uncrustified
> > > > >>>>
> > > > >>>> 2) Setup python virtual env, install pytools, and run stuart
> commands
> > > > >>>>
> > > > >>>> to setup build environment which includes installing uncrustify
> tools.
> > > > >>>>
> > > > >>>> https://github.com/tianocore/edk2/tree/master/.pytool#running-
> ci-locally
> > > > >>> <https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-
> locally>
> > > > >>>>
> > > > >>>> 3) Make a backup copy of plugin UncrustifyCheck outside
> WORKSPACE.
> > > > >>>>
> > > > >>>> (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable
> and
> > > > >>>>
> > > > >>>> EDK II specific uncrustify configuration file available when working
> > > > >>>>
> > > > >>>> with a branch that does not have those tools in its scope.
> > > > >>>>
> > > > >>>> xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
> > > > >>>>
> > > > >>>> 4) Use lint-history.py from git-filter-repo examples
> > > > >>>>
> > > > >>>> https://github.com/newren/git-filter-repo
> <https://github.com/newren/git-filter-repo>
> > > > >>>>
> > > > >>>> https://github.com/newren/git-filter-
> repo/blob/main/contrib/filter-repo-demos/lint-history
> > > > >>> <https://github.com/newren/git-filter-repo/blob/main/contrib/filter-
> repo-demos/lint-history>
> > > > >>>>
> > > > >>>> Line #127 - Add try except around subprocess.check_call() with
> except
> > > > >>>>
> > > > >>>> being pass.This is required because there are a few commits of C
> > > > >>>>
> > > > >>>> files in the edk2 repo that have incorrect C syntax and do not
> > > > >>>>
> > > > >>>> build with a C compiler and break the uncrustify parser.Skip
> reformat
> > > > >>>>
> > > > >>>> of C files that can not be parsed by uncrustify.These rare instances
> > > > >>>>
> > > > >>>> are addressed in the commit that fixes the C syntax error.
> > > > >>>>
> > > > >>>> Run this slightly modified version of lint-history.Include only
> > > > >>>>
> > > > >>>> C/H files and exclude directories that start with 'Tools' or
> 'BaseTools'.
> > > > >>>>
> > > > >>>> This step took about 2.2 hours on a laptop.
> > > > >>>>
> > > > >>>> lint-history.py
> > > > >>>>
> > > > >>>> --relevant "return (not filename.startswith(b'Tools') and not
> filename.startswith(b'BaseTools') and
> > > > >>> (filename.endswith(b'.c') or filename.endswith(b'.h')))"
> > > > >>>>
> > > > >>>> c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\mu-
> uncrustify-release_extdep\\Windows-x86\\uncrustify.exe
> > -
> > > c
> > > > >>> c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\uncrustify.cfg --
> replace --no-backup --if-changed
> > > > >>>>
> > > > >>>> Order of PRs to apply during extended hard freeze
> > > > >>>>
> > > > >>>> ==================================================
> > > > >>>>
> > > > >>>> 1) Update EmulatorPkg Win Host [BuildOptions] MSFT CC_FLAGS
> to not force debug information
> > > > >>>>
> > > > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3747
> <https://bugzilla.tianocore.org/show_bug.cgi?id=3747>
> > > > >>>>
> > > > >>>>
> *https://github.com/mdkinney/edk2/tree/Bug_3747_EmulatorPkg_WinHost_Re
> producibleBuild
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_3747_EmulatorPkg_WinHost_Re
> producibleBuild>
> > > > >>>>
> > > > >>>> *https://github.com/tianocore/edk2/pull/2215
> <https://github.com/tianocore/edk2/pull/2215>
> > > > >>>>
> > > > >>>> * Required for EmulatorPkg to pass CompareBuild for VS2019
> IA32/X64 builds.
> > > > >>>>
> > > > >>>> * Status: Review complete.PR pushed.
> > > > >>>>
> > > > >>>> 2) EccCheck should not revert staged and local changes
> > > > >>>>
> > > > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=2986
> <https://bugzilla.tianocore.org/show_bug.cgi?id=2986>
> > > > >>>>
> > > > >>>>
> *https://github.com/mdkinney/edk2/tree/Bug_2986_EccCheckRemoveGitRever
> t_V2
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_2986_EccCheckRemoveGitRever
> t_V2>
> > > > >>>>
> > > > >>>> *https://github.com/tianocore/edk2/pull/2216
> <https://github.com/tianocore/edk2/pull/2216>
> > > > >>>>
> > > > >>>> * Required for EDK II CI to complete in a reasonable period of time
> when
> > > > >>>>
> > > > >>>> processing the 4000+ source file style changes made by uncrustify.
> > > > >>>>
> > > > >>>> * Also fixes critical bugs that can potentially corrupt git state when
> > > > >>>>
> > > > >>>> EccCheck is run locally.
> > > > >>>>
> > > > >>>> * Status: Review complete.PR pushed.
> > > > >>>>
> > > > >>>> 3) Update pytool LicenseCheck plugin to use temp directory for diff
> output file
> > > > >>>>
> > > > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3746
> <https://bugzilla.tianocore.org/show_bug.cgi?id=3746>
> > > > >>>>
> > > > >>>>
> *https://github.com/mdkinney/edk2/tree/Bug_3746_LicenseCheckUseDiffOutp
> utFile_V2
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_3746_LicenseCheckUseDiffOutp
> utFile_V2>
> > > > >>>>
> > > > >>>> *https://github.com/tianocore/edk2/pull/2217
> <https://github.com/tianocore/edk2/pull/2217>
> > > > >>>>
> > > > >>>> * Required to reduce EDK II CI build times.
> > > > >>>>
> > > > >>>> * Status: Review complete.PR pushed.
> > > > >>>>
> > > > >>>> 4) Update max job time from 60 min to 120 minutes
> in .azurepipelines/templates
> > > > >>>>
> > > > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3750
> <https://bugzilla.tianocore.org/show_bug.cgi?id=3750>
> > > > >>>>
> > > > >>>>
> *https://github.com/mdkinney/edk2/tree/Bug_3750_IncreaseAzurePipelinesTim
> eout
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_3750_IncreaseAzurePipelinesTim
> eout>
> > > > >>>>
> > > > >>>> *https://github.com/tianocore/edk2/pull/2219
> <https://github.com/tianocore/edk2/pull/2219>
> > > > >>>>
> > > > >>>> * Required to allow EccCheck of uncrustify changes to complete
> on Azure
> > > > >>>>
> > > > >>>> Pipelines CI agents without timing out.
> > > > >>>>
> > > > >>>> * Status: Review complete.PR pushed.
> > > > >>>>
> > > > >>>> 5) Update Package YAML to ignore specific ECC files/errors
> > > > >>>>
> > > > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3749
> <https://bugzilla.tianocore.org/show_bug.cgi?id=3749>
> > > > >>>>
> > > > >>>>
> *https://github.com/mdkinney/edk2/tree/Bug_3749_EccCheckIgnoreFilesErrors
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_3749_EccCheckIgnoreFilesErrors
> >
> > > > >>>>
> > > > >>>> *https://github.com/tianocore/edk2/pull/2218
> <https://github.com/tianocore/edk2/pull/2218>
> > > > >>>>
> > > > >>>> * Required to pass EccCheck
> > > > >>>>
> > > > >>>> * Status: Review complete. PR pushed
> > > > >>>>
> > > > >>>> 6) Uncrustify Source Changes
> > > > >>>>
> > > > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3737
> <https://bugzilla.tianocore.org/show_bug.cgi?id=3737>
> > > > >>>>
> > > > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3739
> <https://bugzilla.tianocore.org/show_bug.cgi?id=3739>
> > > > >>>>
> > > > >>>>
> *https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyCha
> nges_V5
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyCha
> nges_V5>
> > > > >>>>
> > > > >>>> *https://github.com/tianocore/edk2/pull/2229
> <https://github.com/tianocore/edk2/pull/2229>
> > > > >>>>
> > > > >>>> * Build comparison result
> PASS:https://github.com/mdkinney/edk2/actions/runs/1521618836
> > > > >>> <https://github.com/mdkinney/edk2/actions/runs/1521618836>
> > > > >>>>
> > > > >>>> * EFI_D_ -> DEBUG changes required to pass PatchCheck
> > > > >>>>
> > > > >>>> * Uncrustify format changes required to pass UncrustifyCheck
> > > > >>>>
> > > > >>>> * Status:
> > > > >>>>
> > > > >>>> Waiting for review
> > > > >>>>
> > > > >>>> 7) UncrustifyCheck EDK II CI Plugin
> > > > >>>>
> > > > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3748
> <https://bugzilla.tianocore.org/show_bug.cgi?id=3748>
> > > > >>>>
> > > > >>>>
> *https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_
> v6
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_
> v6>
> > > > >>>>
> > > > >>>> * Required to enforce all PRs submitted to EDK II CI match
> uncrustify format.
> > > > >>>>
> > > > >>>> * Unconditionally checks all packages.Can not be committed until
> all C/H
> > > > >>>>
> > > > >>>> source files have been updated.
> > > > >>>>
> > > > >>>> * Status: Review complete
> > > > >>>>
> > > > >>>> Combined Branch/PR for Review/Test
> > > > >>>>
> > > > >>>> ==================================
> > > > >>>>
> > > > >>>> * Build Comparison results must pass 100% across the full set of
> PRs before
> > > > >>>>
> > > > >>>> the individual PRs can be pushed in the order listed above.
> > > > >>>>
> > > > >>>> *
> Branch:https://github.com/mdkinney/edk2/tree/TestOnly_Uncrustify_PR_Series
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/TestOnly_Uncrustify_PR_Series>
> > > > >>>>
> > > > >>>> * PR:https://github.com/tianocore/edk2/pull/2229
> <https://github.com/tianocore/edk2/pull/2229>
> > > > >>>>
> > > > >>>> Status = PASS
> > > > >>>>
> > > > >>>> * CompareBuild:
> > > > >>>>
> > > > >>>>
> Branch:https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrusti
> fyChanges_V5
> > > > >>>
> <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyCha
> nges_V5>
> > > > >>>>
> > > > >>>> --ref1:ef9a059cdb15844fe52a49af2bf7d86b9dd3e9bf
> > > > >>>>
> > > > >>>> --ref2:Bug_3737_3739_ApplyUncrustifyChanges_V5
> > > > >>>>
> > > > >>>> Extra Options: -n 4 --quiet
> > > > >>>>
> > > > >>>>
> Results:https://github.com/mdkinney/edk2/actions/runs/1521618836
> > > > >>> <https://github.com/mdkinney/edk2/actions/runs/1521618836>
> > > > >>>>
> > > > >>>> 30 VS2019 build comparisons PASS
> > > > >>>>
> > > > >>>> 40 GCC5 build comparisons PASS
> > > > >>>>
> > > > >>>> 100% PASS
> > > > >>>>
> > > > >>>> The following git log shows the set of patches from --ref1 to --ref
> 2across
> > > > >>>>
> > > > >>>> which there are no differences in any of the
> OBJ/LIB/DLL/EFI/FFS/FV/FD files.
> > > > >>>>
> > > > >>>> --ref2
> > > > >>>>
> > > > >>>> b7d4bf0675b7 (HEAD ->
> Bug_3737_3739_ApplyUncrustifyChanges_V5) UnitTestFrameworkPkg: Apply
> uncrusitify
> > changes
> > > > >>>>
> > > > >>>> 7f03d25f60e7 UefiPayloadPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 0bfd8d9b5ac9 UefiCpuPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> e1cd9bfb9dea StandaloneMmPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 5da2f65be378 SourceLevelDebugPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 95b86de07e5d SignedCapsulePkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> fe71d97246c4 ShellPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 54c21c952992 SecurityPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 187a3785f12b RedfishPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 810100002a46 PcAtChipsetPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 276a695c0cf2 OvmfPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 303c0a91ab07 NetworkPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> bc80792cd1b1 MdePkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 3ea86be17a2a MdeModulePkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> c70ef11ed0cd IntelFsp2WrapperPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> c0291221f252 IntelFsp2Pkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 6a479952a690 FmpDevicePkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 3a7c05b7070d FatPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> b789f98c8959 EmulatorPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 952d7a1c9220 EmbeddedPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> a1cc9881bab6 DynamicTablesPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 50654dfe5785 CryptoPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> ed965a02dfa1 ArmVirtPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 9744023fbc46 ArmPlatformPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 7a1cde5f5bba ArmPkg: Apply uncrusitify changes
> > > > >>>>
> > > > >>>> 19d17e0913e8 UefiCpuPkg: Change use of EFI_D_* to DEBUG_*
> > > > >>>>
> > > > >>>> ffa718b4f994 SourceLevelDebugPkg: Change use of EFI_D_* to
> DEBUG_*
> > > > >>>>
> > > > >>>> b86cb3c5e5b4 ShellPkg: Change use of EFI_D_* to DEBUG_*
> > > > >>>>
> > > > >>>> c7c42204dc07 SecurityPkg: Change use of EFI_D_* to DEBUG_*
> > > > >>>>
> > > > >>>> 16b8e6f958e4 PcAtChipsetPkg: Change use of EFI_D_* to
> DEBUG_*
> > > > >>>>
> > > > >>>> 0ac3f8b2dac5 OvmfPkg: Change use of EFI_D_* to DEBUG_*
> > > > >>>>
> > > > >>>> bc5004b8d294 NetworkPkg: Change use of EFI_D_* to DEBUG_*
> > > > >>>>
> > > > >>>> 6f671a8e2377 MdePkg: Change use of EFI_D_* to DEBUG_*
> > > > >>>>
> > > > >>>> a10c610ff9a3 MdeModulePkg: Change use of EFI_D_* to
> DEBUG_*
> > > > >>>>
> > > > >>>> 09a3bddba390 FatPkg: Change use of EFI_D_* to DEBUG_*
> > > > >>>>
> > > > >>>> 59c61318246a EmulatorPkg: Change use of EFI_D_* to DEBUG_*
> > > > >>>>
> > > > >>>> 3a80367dda3b EmbeddedPkg: Change use of EFI_D_* to DEBUG_*
> > > > >>>>
> > > > >>>> 23eb1aaf80ca ArmVirtPkg: Change use of EFI_D_* to DEBUG_*
> > > > >>>>
> > > > >>>> 875914b45c54 ArmPlatformPkg: Change use of EFI_D_* to
> DEBUG_*
> > > > >>>>
> > > > >>>> eb2eca82b451 ArmPkg: Change use of EFI_D_* to DEBUG_*
> > > > >>>>
> > > > >>>> f0f3f5aae7c4 (origin/master, origin/HEAD, master)
> UnitTestFrameworkPkg: Update YAML to ignore specific ECC
> > > > >>> files/errors
> > > > >>>>
> > > > >>>> c05734797790 UefiPayloadPkg: Update YAML to ignore specific
> ECC files/errors
> > > > >>>>
> > > > >>>> c30c40d6c63d StandaloneMmPkg: Update YAML to ignore specific
> ECC files/errors
> > > > >>>>
> > > > >>>> 9944508e85f1 ShellPkg: Update YAML to ignore specific ECC
> files/errors
> > > > >>>>
> > > > >>>> 60fa40be458d SecurityPkg: Update YAML to ignore specific ECC
> files/errors
> > > > >>>>
> > > > >>>> df790cd6b37e MdePkg: Update YAML to ignore specific ECC
> files/errors
> > > > >>>>
> > > > >>>> 9deb9370766e MdeModulePkg: Update YAML to ignore specific
> ECC files/errors
> > > > >>>>
> > > > >>>> d7d30e8f219f EmulatorPkg: Update YAML to ignore specific ECC
> files/errors
> > > > >>>>
> > > > >>>> d5744ecba813 CryptoPkg: Update YAML to ignore specific ECC
> files/errors
> > > > >>>>
> > > > >>>> c97fee87f0f9 ArmVirtPkg: Update YAML to ignore specific ECC
> files/errors
> > > > >>>>
> > > > >>>> 1939fc9569f2 ArmPlatformPkg: Update YAML to ignore specific
> ECC files/errors
> > > > >>>>
> > > > >>>> 365dced2c37a ArmPkg: Update YAML to ignore specific ECC
> files/errors
> > > > >>>>
> > > > >>>> 76a1ce4d5fec .azurepipelines/templates: Update max pipeline job
> time to 2 hours
> > > > >>>>
> > > > >>>> 99f84ff47390 .pytools/Plugin/LicenseCheck: Use temp directory
> for git diff output
> > > > >>>>
> > > > >>>> 3019f1bbabf1 .pytool/Plugin/EccCheck: Add performance
> optimizations
> > > > >>>>
> > > > >>>> 854462bd3479 .pytool/Plugin/EccCheck: Remove temp directory
> on exception
> > > > >>>>
> > > > >>>> 69877614fdee .pytool/Plugin/EccCheck: Remove RevertCode()
> > > > >>>>
> > > > >>>> --ref1
> > > > >>>>
> > > > >>>> ef9a059cdb15 EmulatorPkg/Win/Host: Update CC_FLAGS
> > > > >>>>
> > > > >>>> bb1bba3d7767 (tag: edk2-stable202111) NetworkPkg: Fix invalid
> pointer for DNS response token on error
> > > > >>>>
> > > > >>>> Best regards,
> > > > >>>>
> > > > >>>> Mike
> > > > >>>>
> > > > >>>>
>
>
>
>
^ permalink raw reply [flat|nested] 39+ messages in thread
* Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
2021-12-06 1:17 ` Michael D Kinney
` (6 preceding siblings ...)
2021-12-07 4:21 ` Chiu, Chasel
@ 2021-12-07 10:02 ` Sami Mujawar
7 siblings, 0 replies; 39+ messages in thread
From: Sami Mujawar @ 2021-12-07 10:02 UTC (permalink / raw)
To: devel@edk2.groups.io, michael.d.kinney@intel.com, Michael Kubacki,
maciej.rabeda@linux.intel.com, Michael Kubacki,
Andrew Fish (afish@apple.com), Leif Lindholm
Cc: nd
Hi Mike,
Thanks a lot for these changes.
Reviewed-by: Sami Mujawar <sami.mujawar@arm.com>
Regards,
Sami Mujawar
On 06/12/2021, 01:18, "devel@edk2.groups.io on behalf of Michael D Kinney via groups.io" <devel@edk2.groups.io on behalf of michael.d.kinney=intel.com@groups.io> wrote:
Hello EDK II Maintainers,
A detailed evaluation of the DEBUG_CODE() formatting issue has been completed.
The reason DEBUG_CODE() is a challenge is that this looks like a macro from a
C parsing perspective, but the EDK II usage places C statements or blocks of
C code as the parameter to this macro.
There are actually 2 methods available to mark a statement or a block of code
to be included when the DEBUG_PROPERTY_DEBUG_CODE_ENABLED bit is enabled in
PcdDebugPropertyMask. One is DEBUG_CODE() and the other is to mark the beginning
and end of a code block with DEBUG_CODE_BEGIN() and DEBUG_CODE_END(). In fact,
DEBUG_CODE() is implemented using DEBUG_CODE_BEGIN() and DEBUG_CODE_END() macros.
#define DEBUG_CODE(Expression) \
DEBUG_CODE_BEGIN (); \
Expression \
DEBUG_CODE_END ()
A complete review for the use of these DEBUG_CODE macros was performed on the
edk2 repo. Uncrustify performs good formatting for code blocks between
DEBUG_CODE_BEGIN() and DEBUG_CODE_END(). This is because these look like simple
macros calls with no parameters and the lines of C code between these 2 macros
is formatted correctly.
The uncrustify formatting issues are only present with the use of DEBUG_CODE().
Simple use cases of DEBUG_CODE(Expression) where Expression is a single C
statement also look correct. A medium complexity use case where Expression is
a code block of simple statements or even some local variables and simple
statements also look correct. It is only complex code blocks that use C
statements such as if/for/while/case that include the use of braces {} does
uncrustify perform incorrect formatting.
The recommended solution to this issue is to convert the use of DEBUG_CODE()
to DEBUG_CODE_BEGIN() / DEBUG_CODE_END() for cases where the Expression
passed to DEBUG_CODE() is the complex use case that contains statements that
use braces {}. There are 57 instances of this pattern across 40 files in the
edk2 repo.
I have posted a branch with these additional patches:
https://github.com/mdkinney/edk2/tree/Bug_3737_3760_3739_ApplyUncrustifyChanges_V7
I have performed CompareBuild tests with this revised patch series with
the DEBUG_CODE changes. It passes 100% showing no binary differences.
https://github.com/mdkinney/edk2/actions/runs/1542454606
I have opened a PR to run this patch series through EDK II CI. It also passes 100%.
https://github.com/tianocore/edk2/pull/2236
The summary of changes made since the V6 review are:
1) Change uncrustify configuration assignment alignment threshold to 0
align_assign_thresh = 0
2) Replace "<param>, OPTIONAL" with "<param> OPTIONAL,"
3) Replace DEBUG_CODE(Expression) with
DEBUG_CODE_BEGIN();
Expression
DEBUG_CODE_END()
if Expression is complex and contains braces {}.
4) No changes to uncrustify tool required.
Please review the differences between the following 2 branches and provide
feedback or a Series Reviewed-by if you agree with these additional changes.
https://github.com/mdkinney/edk2/tree/Bug_3737_3760_3739_ApplyUncrustifyChanges_V6
https://github.com/mdkinney/edk2/tree/Bug_3737_3760_3739_ApplyUncrustifyChanges_V7
The goal is to complete the review and get the uncrustify change committed
tomorrow so the extended hard freeze can be lifted.
Thanks,
Mike
> -----Original Message-----
> From: Kinney, Michael D <michael.d.kinney@intel.com>
> Sent: Thursday, December 2, 2021 6:23 PM
> To: Michael Kubacki <mikuback@linux.microsoft.com>; devel@edk2.groups.io; maciej.rabeda@linux.intel.com; Michael Kubacki
> <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com) <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>;
> Kinney, Michael D <michael.d.kinney@intel.com>
> Subject: RE: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
>
> Hello EDK II Maintainers,
>
> I have entered BZ 3760 to make the use of the OPTIONAL keyword style consistent for all of edk2 repo
> and to be compatible with uncrustify.
>
> I have posted the following V6 branch that does the EFI_D_* to DEBUG_* changes, the OPTIONAL keyword
> style changes, and the uncrustify changes with the one configuration change for assignment alignment.
>
> https://github.com/mdkinney/edk2/tree/Bug_3737_3760_3739_ApplyUncrustifyChanges_V6
>
> Please provide feedback on the code style in this branch with the known DEBUG_CODE() issue still
> present.
>
> If we are able to quickly update uncrustify to handle DEBUG_CODE(), I will generate a V7.
>
> Thanks,
>
> Mike
>
> > -----Original Message-----
> > From: Kinney, Michael D <michael.d.kinney@intel.com>
> > Sent: Thursday, December 2, 2021 4:53 PM
> > To: Michael Kubacki <mikuback@linux.microsoft.com>; devel@edk2.groups.io; maciej.rabeda@linux.intel.com; Michael Kubacki
> > <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com) <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>;
> > Kinney, Michael D <michael.d.kinney@intel.com>
> > Subject: RE: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
> >
> > Michael,
> >
> > Yes. Please update the patch series that adds UncrustifyCheck with those changes.
> >
> > Thanks,
> >
> > Mike
> >
> > > -----Original Message-----
> > > From: Michael Kubacki <mikuback@linux.microsoft.com>
> > > Sent: Thursday, December 2, 2021 4:31 PM
> > > To: Kinney, Michael D <michael.d.kinney@intel.com>; devel@edk2.groups.io; maciej.rabeda@linux.intel.com; Michael
> Kubacki
> > > <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com) <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>
> > > Subject: Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
> > >
> > > Hi Mike,
> > >
> > > Thank you for the detailed analysis and recommendations.
> > >
> > > I agree with the recommendations and I will try to have an Uncrustify
> > > tool update by tomorrow for (4).
> > >
> > > That will require an update in uncrustify_ext_dep.yaml to pull in the
> > > new version. I'm assuming I should also update uncrustify.cfg to set
> > > align_assign_thresh = 0 in that new patch series?
> > >
> > > Regards,
> > > Michael
> > >
> > > On 12/2/2021 7:18 PM, Kinney, Michael D wrote:
> > > > Hi Michael,
> > > >
> > > > CORRECTION: set align_assign_threshold to 0.
> > > >
> > > > Reponses inline below.
> > > >
> > > > I would like to summarize the 4 issues raised in the past day along with the recommendations.
> > > >
> > > > 1) Exclusion feature for UncrustifyCheck. There are 2 directories with 8 files total that
> > > > maintainers have noted they would like to see not go through uncrustify formatting. Today
> > > > the only content that is skipped is BaseTools and submodules.
> > > >
> > > > Adding a general purpose exclusion feature would then require all developers to make
> > > > sure their method of using uncrustify also excludes those same areas. This requires
> > > > extra steps for all developers and maintainers.
> > > >
> > > > If we do not add the exclusion feature, then the 8 files will require an extra step
> > > > to sync with the original source of those files. The rate of changes of these 8 files
> > > > is very low today.
> > > >
> > > > RECOMMENDATION: Do not add exclusion feature at this time. Revisit if the extra work
> > > > to maintain the files that would be candidates for exclusions increases significantly.
> > > >
> > > > 2) Alignment of assignments. The threshold of 4 characters appears to be too low and causes
> > > > source files that are already aligned to become unaligned.
> > > >
> > > > RECOMMENDATION: Change threshold to the default value of 0 which means no limit.
> > > >
> > > > align_assign_thresh= 0
> > > >
> > > > 3) Alignment of parameters in function declaration not correct. The root cause of this
> > > > is the use of the OPTIONAL keyword. If the OPTIONAL keyword is removed, then the
> > > > alignment is correct. The alignment is also correct if the OPTIONAL keyword appears
> > > > before the ','. If the OPTIONAL keyword appears after the ',', then the format is
> > > > not correct. The OPTIONAL keyword indicates that the parameter in the function is
> > > > not required and may be passed in as NULL or 0 or some other default value defined by
> > > > the API. It makes more sense for this OPTIONAL keyword that follows the parameter
> > > > names to appear before the ',' so it is scoped to the parameter on that line. If it
> > > > appears after the ',', then C parsers thinks it is a prefix (IN, OUT, CONST, volatile,
> > > > static) for the next parameter in the function.
> > > >
> > > > RECOMMENDATION: Update patch series with a global search and replace so OPTIONAL
> > > > keyword always appears before the ',' on the same line.
> > > >
> > > > RegEx search string: ',( *)OPTIONAL( *)'
> > > > RegEx replace string: ' $1OPTIONAL,$2'
> > > >
> > > > 4) Format issues with complex blocks in DEBUG_CODE(), or between DEBUG_CODE_BEGIN() and
> > > > DEBUG_CODE_END(). Uncrustify treats these as macros and is not aware that the
> > > > parameter passed into the macro call is a block of C code that needs to be formatted.
> > > > Complex blocks with if/while/for/case statements are impacted the most.
> > > >
> > > > RECOMMENDATION: Update the uncrustify with an edk2 specific extension to treat these
> > > > macros as a block of code as if they were surrounded by an extra set of braces {}.
> > > >
> > > >
> > > > I have posted a branch for testing purposes that implements (2) and (3).
> > > >
> > > > Branch: https://github.com/mdkinney/edk2/tree/TestOnly_Bug_3737_3739_ApplyUncrustifyChanges_V6_OPTIONAL_Keyword_Fix
> > > > PR: https://github.com/tianocore/edk2/pull/2233
> > > > Status: PASS
> > > > CompareBuild: https://github.com/mdkinney/edk2/actions/runs/1532855914
> > > > Status: PASS
> > > >
> > > > You can see what changed by fetching and comparing the following 2 branches:
> > > >
> > > > https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
> > > > https://github.com/mdkinney/edk2/tree/TestOnly_Bug_3737_3739_ApplyUncrustifyChanges_V6_OPTIONAL_Keyword_Fix
> > > >
> > > > Please provide feedback on the RECOMMENDATIONS above. I will go ahead and prepare of V6 version of
> > > > the patch series now that that test results are all PASS.
> > > >
> > > > Best regards,
> > > >
> > > > Mike
> > > >
> > > >
> > > >> -----Original Message-----
> > > >> From: Kinney, Michael D <michael.d.kinney@intel.com>
> > > >> Sent: Thursday, December 2, 2021 4:15 PM
> > > >> To: Michael Kubacki <mikuback@linux.microsoft.com>; devel@edk2.groups.io; maciej.rabeda@linux.intel.com; Michael
> > > Kubacki
> > > >> <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com) <afish@apple.com>; Leif Lindholm
> <leif@nuviainc.com>;
> > > >> Kinney, Michael D <michael.d.kinney@intel.com>
> > > >> Subject: RE: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
> > > >>
> > > >> Hi Michael,
> > > >>
> > > >> Reponses inline below.
> > > >>
> > > >> I would like to summarize the 4 issues raised in the past day along with the recommendations.
> > > >>
> > > >> 1) Exclusion feature for UncrustifyCheck. There are 2 directories with 8 files total that
> > > >> maintainers have noted they would like to see not go through uncrustify formatting. Today
> > > >> the only content that is skipped is BaseTools and submodules.
> > > >>
> > > >> Adding a general purpose exclusion feature would then require all developers to make
> > > >> sure their method of using uncrustify also excludes those same areas. This requires
> > > >> extra steps for all developers and maintainers.
> > > >>
> > > >> If we do not add the exclusion feature, then the 8 files will require an extra step
> > > >> to sync with the original source of those files. The rate of changes of these 8 files
> > > >> is very low today.
> > > >>
> > > >> RECOMMENDATION: Do not add exclusion feature at this time. Revisit if the extra work
> > > >> to maintain the files that would be candidates for exclusions increases significantly.
> > > >>
> > > >> 2) Alignment of assignments. The threshold of 4 characters appears to be too low and causes
> > > >> source files that are already aligned to become unaligned.
> > > >>
> > > >> RECOMMENDATION: Change threshold to the default value of 0 which means no limit.
> > > >>
> > > >> align_assign_thresh= 4
> > > >>
> > > >> 3) Alignment of parameters in function declaration not correct. The root cause of this
> > > >> is the use of the OPTIONAL keyword. If the OPTIONAL keyword is removed, then the
> > > >> alignment is correct. The alignment is also correct if the OPTIONAL keyword appears
> > > >> before the ','. If the OPTIONAL keyword appears after the ',', then the format is
> > > >> not correct. The OPTIONAL keyword indicates that the parameter in the function is
> > > >> not required and may be passed in as NULL or 0 or some other default value defined by
> > > >> the API. It makes more sense for this OPTIONAL keyword that follows the parameter
> > > >> names to appear before the ',' so it is scoped to the parameter on that line. If it
> > > >> appears after the ',', then C parsers thinks it is a prefix (IN, OUT, CONST, volatile,
> > > >> static) for the next parameter in the function.
> > > >>
> > > >> RECOMMENDATION: Update patch series with a global search and replace so OPTIONAL
> > > >> keyword always appears before the ',' on the same line.
> > > >>
> > > >> RegEx search string: ',( *)OPTIONAL( *)'
> > > >> RegEx replace string: ' $1OPTIONAL,$2'
> > > >>
> > > >> 4) Format issues with complex blocks in DEBUG_CODE(), or between DEBUG_CODE_BEGIN() and
> > > >> DEBUG_CODE_END(). Uncrustify treats these as macros and is not aware that the
> > > >> parameter passed into the macro call is a block of C code that needs to be formatted.
> > > >> Complex blocks with if/while/for/case statements are impacted the most.
> > > >>
> > > >> RECOMMENDATION: Update the uncrustify with an edk2 specific extension to treat these
> > > >> macros as a block of code as if they were surrounded by an extra set of braces {}.
> > > >>
> > > >>
> > > >> I have posted a branch for testing purposes that implements (2) and (3).
> > > >>
> > > >> Branch: https://github.com/mdkinney/edk2/tree/TestOnly_Bug_3737_3739_ApplyUncrustifyChanges_V6_OPTIONAL_Keyword_Fix
> > > >> PR: https://github.com/tianocore/edk2/pull/2233
> > > >> Status: PASS
> > > >> CompareBuild: https://github.com/mdkinney/edk2/actions/runs/1532855914
> > > >> Status: PASS
> > > >>
> > > >> You can see what changed by fetching and comparing the following 2 branches:
> > > >>
> > > >> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
> > > >> https://github.com/mdkinney/edk2/tree/TestOnly_Bug_3737_3739_ApplyUncrustifyChanges_V6_OPTIONAL_Keyword_Fix
> > > >>
> > > >> Please provide feedback on the RECOMMENDATIONS above. I will go ahead and prepare of V6 version of
> > > >> the patch series now that that test results are all PASS.
> > > >>
> > > >> Best regards,
> > > >>
> > > >> Mike
> > > >>
> > > >>
> > > >>> -----Original Message-----
> > > >>> From: Michael Kubacki <mikuback@linux.microsoft.com>
> > > >>> Sent: Thursday, December 2, 2021 1:57 PM
> > > >>> To: devel@edk2.groups.io; Kinney, Michael D <michael.d.kinney@intel.com>; maciej.rabeda@linux.intel.com; Michael
> > > Kubacki
> > > >>> <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com) <afish@apple.com>; Leif Lindholm
> <leif@nuviainc.com>
> > > >>> Subject: Re: [edk2-devel] Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4
> > > >>>
> > > >>> My reply is inline.
> > > >>>
> > > >>> Regards,
> > > >>> Michael
> > > >>>
> > > >>> On 12/2/2021 2:45 PM, Michael D Kinney wrote:
> > > >>>> Hi Maciej,
> > > >>>>
> > > >>>> Thanks for the feedback.
> > > >>>>
> > > >>>> * Example #1.This appears to be caused by the following uncrustify
> > > >>>> settings:
> > > >>>>
> > > >>>> # The threshold for aligning on '=' in assignments.
> > > >>>>
> > > >>>> # Use a negative number for absolute thresholds.
> > > >>>>
> > > >>>> #
> > > >>>>
> > > >>>> # 0: No limit (default).
> > > >>>>
> > > >>>> align_assign_thresh= 0# number
> > > >>>>
> > > >>>> The edk2 setting for this is:
> > > >>>>
> > > >>>> align_assign_thresh= 4
> > > >>>>
> > > >>>> This means blocks of assignments that are different than more than 4
> > > >>>> spaces will be considered a new block.
> > > >>>>
> > > >>>> ‘HwAddreLen’ and ‘Xid’ are more than 4 characters in length
> > > >>>> different.Same for ‘Xid’ and ‘Reserved’.So
> > > >>>>
> > > >>>> uncrustify treats these as 3 different assignment blocks.
> > > >>>>
> > > >>>> If we change to the default value of 0: No limit, this example is
> > > >>>> treated as a single block and all ‘=’ are aligned.
> > > >>>>
> > > >>>> Is there a reason ‘4’ was selected?Is there is any harm in using the
> > > >>>> default of 0?
> > > >>>>
> > > >>> We can certainly change the threshold. '4' was derived by
> > > >>> experimentation. This is an area that is somewhat subjective and every
> > > >>> case is difficult to cover well. Please feel free to suggest any changes.
> > > >>>
> > > >>
> > > >> I recommend we use the default value of 0. That way, any code that choose to
> > > >> align assignments will still be aligned. If a developer does not like
> > > >> short assignments and long assignments in the same code block to use the
> > > >> long assignment column, they can always break the block up into multiple
> > > >> blocks by adding a carriage return.
> > > >>
> > > >>> For the benefit of others, this file can be a useful refernce
> > > >>> understanding spans, gaps, and thresholds.
> > > >>>
> > > >>> https://github.com/uncrustify/uncrustify/blob/master/documentation/htdocs/configuration.txt
> > > >>>
> > > >>>> * Example #2: Uncruistfy is confused by the DEBUG_CODE() macro.This is
> > > >>>> not a traditional macro because the
> > > >>>>
> > > >>>> contents of the macro is a block of C code.I do not know how to convince
> > > >>>> uncrustify that the contents of a
> > > >>>>
> > > >>>> macro like function call to be treated as a code block from an indent
> > > >>>> perspective.
> > > >>>>
> > > >>> I believe this would have to be overridden in the Uncrustify fork since
> > > >>> DEBUG_CODE() is being treated as a macro function call and code blocks
> > > >>> are not expected there. In addition, some special treatment might be
> > > >>> needed for alignment in between DEBUG_CODE_BEGIN()/DEBUG_CODE_END().
> > > >>>
> > > >>> I'm happy to look at this. However, regression validation can take a
> > > >>> while so I'd like to make sure we need this now. In cases that do not
> > > >>> have additional code blocks, it seems to format fairly well. Is this
> > > >>> prevalent and impactful enough it must be fixed now? Or, could we
> > > >>> revisit it with a follow up patch?
> > > >>
> > > >> How long does regression testing take? I see about 20 instances of
> > > >> DEBUG_CODE() that are producing bad formatting. If the content inside
> > > >> DEBUG_CODE() is a single line of a single block of statements without
> > > >> any if/while/for/case statements that require further indent, then the
> > > >> format looks ok.
> > > >>
> > > >> DEBUG_CODE_BEGIN() and DEBUG_CODE_END() would look better if the
> > > >> contents between are also indented one level.
> > > >>
> > > >>>
> > > >>>> * Example #3/#4: Uncrustify is confused by the OPTIONAL keyword.The
> > > >>>> edk2 config declares it as a QUALIFIER
> > > >>>>
> > > >>>> like IN and OUT.However, OPTIONAL only appears at the end of the line
> > > >>>> that declares a parameter to a
> > > >>>>
> > > >>>> function.There are 3 forms. One with comma after OPTIONAL.One with comma
> > > >>>> before OPTIONAL, and
> > > >>>>
> > > >>>> one with no comma if the parameter is the last parameter in the function
> > > >>>> declaration.
> > > >>>>
> > > >>>> TYPE ParamName OPTIONAL,
> > > >>>>
> > > >>>> TYPEParamName, OPTIONAL
> > > >>>>
> > > >>>> TYPEParamName OPTIONAL
> > > >>>>
> > > >>>> OPTIONAL is defined to nothing in edk2 builds.From a uncrustify
> > > >>>> perspective, we really want is to be
> > > >>>>
> > > >>>> ignored or more correctly treated it as a token that is attached to
> > > >>>> ParamName and the combination of
> > > >>>>
> > > >>>> ParamName and OPTIONAL treated as one unit to determine indents.
> > > >>>>
> > > >>> "TYPE ParamName, OPTIONAL" seems especially problematic and presents an
> > > >>> inconsistency with the other formats. Mike, can you please let me know
> > > >>> if you have the same observation and your thoughts on a consistent pattern?
> > > >>>
> > > >>>> Thanks,
> > > >>>>
> > > >>>> Mike
> > > >>>>
> > > >>>> *From:*devel@edk2.groups.io <devel@edk2.groups.io> *On Behalf Of *Maciej
> > > >>>> Rabeda
> > > >>>> *Sent:* Thursday, December 2, 2021 10:27 AM
> > > >>>> *To:* devel@edk2.groups.io; Kinney, Michael D
> > > >>>> <michael.d.kinney@intel.com>; Michael Kubacki
> > > >>>> <michael.kubacki@microsoft.com>; Andrew Fish (afish@apple.com)
> > > >>>> <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>
> > > >>>> *Subject:* Re: [edk2-devel] Uncrustify Conversion Detailed Plan and
> > > >>>> Extended Hard Freeze Update #4
> > > >>>>
> > > >>>> Hey Mike,
> > > >>>>
> > > >>>> While most of the changes related to fixing coding style violations,
> > > >>>> there are a couple of changes to NetworkPkg in that PR that make the
> > > >>>> code less readable. Examples below.
> > > >>>>
> > > >>>> Example 1:
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>> Example 2:
> > > >>>>
> > > >>>>
> > > >>>> Example 3:
> > > >>>>
> > > >>>>
> > > >>>> Example 4:
> > > >>>>
> > > >>>> On 30-Nov-21 23:34, Michael D Kinney wrote:
> > > >>>>
> > > >>>> Hello,
> > > >>>>
> > > >>>> Thank you for your patience during this extended hard freeze.
> > > >>>>
> > > >>>> Just one more step to go.There has been a delay in the review of
> > > >>>>
> > > >>>> the patch series with the uncrustify source changes.PR(6).This
> > > >>>>
> > > >>>> patch series was not sent out as patch review email because of its
> > > >>>>
> > > >>>> very large size.It only contains source style changes and the
> > > >>>>
> > > >>>> CompareBuild tool and GitHub action has shown there are no binary
> > > >>>>
> > > >>>> differences introduced with these source style changes.
> > > >>>>
> > > >>>> If you are a package maintainer, then please review the following
> > > >>>>
> > > >>>> branch/PR for your package contents and review the EDK II CI results
> > > >>>>
> > > >>>> and BuildCompare results.I do not expect a line by line review
> > > >>>>
> > > >>>> because we already had time to provide feedback on the source style
> > > >>>>
> > > >>>> performed by uncrustify.Instead, a Reviewed-by for your package
> > > >>>>
> > > >>>> indicates that you have reviewed the EDK II CI results and CompareBuild
> > > >>>>
> > > >>>> tool functionality and results and you accept the source style
> > > >>>>
> > > >>>> changes to your package.
> > > >>>>
> > > >>>> *https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
> > > >>> <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5>
> > > >>>>
> > > >>>> *https://github.com/tianocore/edk2/pull/2229 <https://github.com/tianocore/edk2/pull/2229>
> > > >>>>
> > > >>>> *https://github.com/mdkinney/edk2/actions/runs/1521618836
> > > >>> <https://github.com/mdkinney/edk2/actions/runs/1521618836>
> > > >>>>
> > > >>>> Additional details on this update below.
> > > >>>>
> > > >>>> Thank you,
> > > >>>>
> > > >>>> Mike
> > > >>>>
> > > >>>> Changes from Update #3
> > > >>>>
> > > >>>> ----------------------------------------------------------------------------
> > > >>>>
> > > >>>> * Pushed PR (5)
> > > >>>>
> > > >>>> * Added link to PR(6). EDK II CI Status is PASS. Build Compare PASS.
> > > >>>>
> > > >>>> * Waiting for review of PR (6)
> > > >>>>
> > > >>>> * Review of PR (7) completed and waiting for review of PR (6)
> > > >>>>
> > > >>>> ----------------------------------------------------------------------------
> > > >>>>
> > > >>>> Changes from Update #2
> > > >>>>
> > > >>>> ----------------------------------------------------------------------------
> > > >>>>
> > > >>>> * Changed order of PRs swapping (4) and (5).The PR that activates
> > > >>>>
> > > >>>> increases the max CI agent job time is independent of all the other
> > > >>>>
> > > >>>> PRs and its review is complete, so it can be committed now.
> > > >>>>
> > > >>>> * Pushed PRs (1), (2), (3), (4).
> > > >>>>
> > > >>>> * Waiting for review to complete for PRs (5) and (6)
> > > >>>>
> > > >>>> * Reviews complete for PR (7)
> > > >>>>
> > > >>>> * Identifies steps using git filter-branch to apply uncrustify changes to a
> > > >>>>
> > > >>>> code review patch series that was generated before the uncrustify changes
> > > >>>>
> > > >>>> avoiding manual merge.
> > > >>>>
> > > >>>> * Identified steps using git filter-repo to generate an alternate history of
> > > >>>>
> > > >>>> the edk2 repo with uncrustify changes applied on every commit.This may
> > > >>>>
> > > >>>> be useful when evaluating changes to files using tools like git blame
> > > >>>>
> > > >>>> without the large uncrustify patch series.
> > > >>>>
> > > >>>> ---------------------------------------------------------------------------
> > > >>>>
> > > >>>> Changes from Update #1
> > > >>>>
> > > >>>> ----------------------------------------------------------------------------
> > > >>>>
> > > >>>> * Changed order of PRs swapping (6) and (7).The PR that activates
> > > >>>>
> > > >>>> EDK II CI check UncrustifyCheck has to be last because it unconditionally
> > > >>>>
> > > >>>> checks all C/H files in all packages.Not just files that have been
> > > >>>>
> > > >>>> modified like some of the other checkers.
> > > >>>>
> > > >>>> * Updated link to the branch with the UncrustifyCheck plugin that has been
> > > >>>>
> > > >>>> updated with a one line change and Reviewed-by and Tested-by tags.
> > > >>>>
> > > >>>> https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v6
> > > >>> <https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v6>
> > > >>>>
> > > >>>> * Reviews complete for (1), (2), (3), (5), and (7)
> > > >>>>
> > > >>>> ---------------------------------------------------------------------------
> > > >>>>
> > > >>>> Michael Kubacki and I have prepared the patches required to apply the
> > > >>>>
> > > >>>> uncrustify changes and enable EDK II CI to check all submitted
> > > >>>>
> > > >>>> patches have been run through uncrustify.
> > > >>>>
> > > >>>> We have verified through the CompareBuild GitHub Action that the
> > > >>>>
> > > >>>> format changes performed by uncrustify have no functional changes.
> > > >>>>
> > > >>>> All of the OBJ, LIB, DLL, EFI, FFS, FV, and FD files match 100%
> > > >>>>
> > > >>>> across 70 VS2019/GCC5 builds of all package/platform DSC files in
> > > >>>>
> > > >>>> the edk2 repo.
> > > >>>>
> > > >>>> The hard freeze will be extended after the edk2-stable202111 tag until
> > > >>>>
> > > >>>> all uncrustify related changes are committed.We do not expect this
> > > >>>>
> > > >>>> to take more than a few days.Do not push any PRs until the hard
> > > >>>>
> > > >>>> freeze is lifted.
> > > >>>>
> > > >>>> The changes are broken up into 7 patch series/PRs.The PRs are ordered
> > > >>>>
> > > >>>> so they can be submitted using the normal submission process and EDK II
> > > >>>>
> > > >>>> CI will pass for each one.Details are listed below.
> > > >>>>
> > > >>>> Uncrustify 73.0.3 for EDK II
> > > >>>>
> > > >>>> =============================
> > > >>>>
> > > >>>> * Sources:https://dev.azure.com/projectmu/_git/Uncrustify <https://dev.azure.com/projectmu/_git/Uncrustify>
> > > >>>>
> > > >>>> * Documentation:https://dev.azure.com/projectmu/Uncrustify/_wiki/wikis/Uncrustify.wiki/1/Project-Mu-(EDK-
> II)-
> > > Fork-
> > > >>> Readme <https://dev.azure.com/projectmu/Uncrustify/_wiki/wikis/Uncrustify.wiki/1/Project-Mu-(EDK-II)-Fork-Readme>
> > > >>>>
> > > >>>> * Download:https://dev.azure.com/projectmu/Uncrustify/_packaging?_a=package&feed=mu_uncrustify&package=mu-
> > > >>> uncrustify-release&protocolType=NuGet&version=73.0.3
> > > >>> <https://dev.azure.com/projectmu/Uncrustify/_packaging?_a=package&feed=mu_uncrustify&package=mu-uncrustify-
> > > >>> release&protocolType=NuGet&version=73.0.3>
> > > >>>>
> > > >>>> Installing Uncrustify
> > > >>>>
> > > >>>> ======================
> > > >>>>
> > > >>>> The Uncrustify tool is installed automatically when the Pytools
> > > >>>>
> > > >>>> environment is used and the stuart* commands are run to complete the
> > > >>>>
> > > >>>> environment setup.Please see:
> > > >>>>
> > > >>>> https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally
> > > >>> <https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally>
> > > >>>>
> > > >>>> Uncrustify can also be installed from the download page listed above
> > > >>>>
> > > >>>> or built from sources from the source link above.
> > > >>>>
> > > >>>> The Documentation link provides instruction on how to run uncrustify from
> > > >>>>
> > > >>>> the command line or install as a Visual Studio Code plugin.The main
> > > >>>>
> > > >>>> uncrustify documentation also describes how to integrate with a few other
> > > >>>>
> > > >>>> editors.
> > > >>>>
> > > >>>> We have also discussed a client side githook.That effort has not started.
> > > >>>>
> > > >>>> Let us know if that is a feature you would find useful.
> > > >>>>
> > > >>>> Developer impact for new code reviews
> > > >>>>
> > > >>>> ======================================
> > > >>>>
> > > >>>> Once the uncrustify checker is active in EDK II CI, developers must
> > > >>>>
> > > >>>> make sure their patches are run through the uncrustify tool before
> > > >>>>
> > > >>>> sending the patches for review.
> > > >>>>
> > > >>>> Developers must install and run uncrustify against changes files before
> > > >>>>
> > > >>>> sending patch review emails or submitting PR for EDK II CI.If EDK II CI
> > > >>>>
> > > >>>> detects and differences in source formatting, then EDK II CI will fail
> > > >>>>
> > > >>>> and the developer must run uncrustify and resubmit the patches.
> > > >>>>
> > > >>>> Developer impact to patch series/PRs reviewed during edk2-stable201121 soft/hard freeze
> > > >>>>
> > > >>>> =======================================================================================
> > > >>>>
> > > >>>> Developers must rebase their changes after the uncrustify source changes are
> > > >>>>
> > > >>>> committed.The branch with a preview of the uncrustify changes can be used
> > > >>>>
> > > >>>> to start this rebase work.
> > > >>>>
> > > >>>> https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
> > > >>> <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5>
> > > >>>>
> > > >>>> The following steps can be used to update an existing branch with the
> > > >>>>
> > > >>>> required uncrustify format.This is the Windows version.I will add
> > > >>>>
> > > >>>> the Linux version soon.
> > > >>>>
> > > >>>> 1) Fetch and checkout and rebase to latest edk2/master
> > > >>>>
> > > >>>> git fetch origin
> > > >>>>
> > > >>>> git checkout master
> > > >>>>
> > > >>>> git rebase origin/master
> > > >>>>
> > > >>>> 2) Make a backup copy of plugin UncrustifyCheck outside WORKSPACE.
> > > >>>>
> > > >>>> (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable and
> > > >>>>
> > > >>>> EDK II specific uncrustify configuration file available when working
> > > >>>>
> > > >>>> with a branch that does not have those tools in its scope.
> > > >>>>
> > > >>>> xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
> > > >>>>
> > > >>>> 3) Check out the patch series branch (e.g. MyBranch)
> > > >>>>
> > > >>>> git checkout MyBranch
> > > >>>>
> > > >>>> 4) Rebase patch series against edk2-stable202111
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>> git rebase edk2-stable202111
> > > >>>>
> > > >>>> 5) Create new branch for the uncrustifed version (e.g. MyBranch_Uncrustified)
> > > >>>>
> > > >>>> git checkout -b MyBranch_Uncrustified
> > > >>>>
> > > >>>> 6) Use git filter-branch to uncrustify all the commits in the series
> > > >>>>
> > > >>>> between the rebase target from (2) and HEAD of the branch.A filter
> > > >>>>
> > > >>>> can be used to scope the uncrustify operations to only the C/H files
> > > >>>>
> > > >>>> in the specific package the patch series is against. (e.g. DynamicTablesPkg).
> > > >>>>
> > > >>>> BaseTools should always be excluded.If the package scoped filter is
> > > >>>>
> > > >>>> not used, it will still work, but will take longer to run because
> > > >>>>
> > > >>>> uncrustify will rescan every C/H files in the whole repo.
> > > >>>>
> > > >>>> git filter-branch --tree-filter "git ls-files DynamicTablesPkg*.c DynamicTablesPkg*.h :!BaseTools/* |
> > > >>> c:\\Temp\\UncrustifyCheck\\mu-uncrustify-release_extdep\\Windows-x86\\uncrustify.exe -c
> > > >>> c:\\Temp\\UncrustifyCheck\\uncrustify.cfg -F - --replace --no-backup --if-changed" edk2-stable202111..HEAD
> > > >>>>
> > > >>>> 7) Now that all the individual patches in the branch are uncrustified,
> > > >>>>
> > > >>>> rebase against latest edk2/master that is already uncrustified.
> > > >>>>
> > > >>>> git rebase master
> > > >>>>
> > > >>>> 8) Verify the patches in this new branch.
> > > >>>>
> > > >>>> Impacts to tracing history across the uncrusity changes
> > > >>>>
> > > >>>> =======================================================
> > > >>>>
> > > >>>> Tools the view file and line history do work with the large uncrustify
> > > >>>>
> > > >>>> patch series.One impact is that the operations can be very slow due
> > > >>>>
> > > >>>> to the large uncrustify patches.
> > > >>>>
> > > >>>> One option to provide a faster experience is to provide an alternate
> > > >>>>
> > > >>>> version of the edk2 repository as "documentation" that has the
> > > >>>>
> > > >>>> entire history re-written with uncrustify run on every commit.
> > > >>>>
> > > >>>> The tool called git-filter-repo can be used to perform this
> > > >>>>
> > > >>>> transformation and runs in a reasonable period of time (a few hours)
> > > >>>>
> > > >>>> https://github.com/newren/git-filter-repo <https://github.com/newren/git-filter-repo>
> > > >>>>
> > > >>>> https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-demos/lint-history
> > > >>> <https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-demos/lint-history>
> > > >>>>
> > > >>>> The following steps can be used to perform this transformation.
> > > >>>>
> > > >>>> This is the Windows version. I will add the Linux version soon.
> > > >>>>
> > > >>>> ** WARNING **This operation modifies(rewrites) all the commits
> > > >>>>
> > > >>>> in the local copy of the repo.Do not perform
> > > >>>>
> > > >>>> these steps on a local repo you are using for
> > > >>>>
> > > >>>> active development.
> > > >>>>
> > > >>>> 1) Clone edk2 into a new directory (see **WARNING**)
> > > >>>>
> > > >>>> git clonehttps://github.com/tianocore/edk2.git <https://github.com/tianocore/edk2.git> edk2-uncrustified
> > > >>>>
> > > >>>> cd edk2-uncrustified
> > > >>>>
> > > >>>> 2) Setup python virtual env, install pytools, and run stuart commands
> > > >>>>
> > > >>>> to setup build environment which includes installing uncrustify tools.
> > > >>>>
> > > >>>> https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally
> > > >>> <https://github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally>
> > > >>>>
> > > >>>> 3) Make a backup copy of plugin UncrustifyCheck outside WORKSPACE.
> > > >>>>
> > > >>>> (e.g. C:\Temp\UncrustifyCheck) so the uncrustify tool executable and
> > > >>>>
> > > >>>> EDK II specific uncrustify configuration file available when working
> > > >>>>
> > > >>>> with a branch that does not have those tools in its scope.
> > > >>>>
> > > >>>> xcopy .pytool\Plugin\UncrustifyCheck C:\Temp\UncrustifyCheck
> > > >>>>
> > > >>>> 4) Use lint-history.py from git-filter-repo examples
> > > >>>>
> > > >>>> https://github.com/newren/git-filter-repo <https://github.com/newren/git-filter-repo>
> > > >>>>
> > > >>>> https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-demos/lint-history
> > > >>> <https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-demos/lint-history>
> > > >>>>
> > > >>>> Line #127 - Add try except around subprocess.check_call() with except
> > > >>>>
> > > >>>> being pass.This is required because there are a few commits of C
> > > >>>>
> > > >>>> files in the edk2 repo that have incorrect C syntax and do not
> > > >>>>
> > > >>>> build with a C compiler and break the uncrustify parser.Skip reformat
> > > >>>>
> > > >>>> of C files that can not be parsed by uncrustify.These rare instances
> > > >>>>
> > > >>>> are addressed in the commit that fixes the C syntax error.
> > > >>>>
> > > >>>> Run this slightly modified version of lint-history.Include only
> > > >>>>
> > > >>>> C/H files and exclude directories that start with 'Tools' or 'BaseTools'.
> > > >>>>
> > > >>>> This step took about 2.2 hours on a laptop.
> > > >>>>
> > > >>>> lint-history.py
> > > >>>>
> > > >>>> --relevant "return (not filename.startswith(b'Tools') and not filename.startswith(b'BaseTools') and
> > > >>> (filename.endswith(b'.c') or filename.endswith(b'.h')))"
> > > >>>>
> > > >>>> c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\mu-uncrustify-release_extdep\\Windows-x86\\uncrustify.exe
> -
> > c
> > > >>> c:\\work\\GitHub\\tianocore\\foo\\UncrustifyCheck\\uncrustify.cfg --replace --no-backup --if-changed
> > > >>>>
> > > >>>> Order of PRs to apply during extended hard freeze
> > > >>>>
> > > >>>> ==================================================
> > > >>>>
> > > >>>> 1) Update EmulatorPkg Win Host [BuildOptions] MSFT CC_FLAGS to not force debug information
> > > >>>>
> > > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3747 <https://bugzilla.tianocore.org/show_bug.cgi?id=3747>
> > > >>>>
> > > >>>> *https://github.com/mdkinney/edk2/tree/Bug_3747_EmulatorPkg_WinHost_ReproducibleBuild
> > > >>> <https://github.com/mdkinney/edk2/tree/Bug_3747_EmulatorPkg_WinHost_ReproducibleBuild>
> > > >>>>
> > > >>>> *https://github.com/tianocore/edk2/pull/2215 <https://github.com/tianocore/edk2/pull/2215>
> > > >>>>
> > > >>>> * Required for EmulatorPkg to pass CompareBuild for VS2019 IA32/X64 builds.
> > > >>>>
> > > >>>> * Status: Review complete.PR pushed.
> > > >>>>
> > > >>>> 2) EccCheck should not revert staged and local changes
> > > >>>>
> > > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=2986 <https://bugzilla.tianocore.org/show_bug.cgi?id=2986>
> > > >>>>
> > > >>>> *https://github.com/mdkinney/edk2/tree/Bug_2986_EccCheckRemoveGitRevert_V2
> > > >>> <https://github.com/mdkinney/edk2/tree/Bug_2986_EccCheckRemoveGitRevert_V2>
> > > >>>>
> > > >>>> *https://github.com/tianocore/edk2/pull/2216 <https://github.com/tianocore/edk2/pull/2216>
> > > >>>>
> > > >>>> * Required for EDK II CI to complete in a reasonable period of time when
> > > >>>>
> > > >>>> processing the 4000+ source file style changes made by uncrustify.
> > > >>>>
> > > >>>> * Also fixes critical bugs that can potentially corrupt git state when
> > > >>>>
> > > >>>> EccCheck is run locally.
> > > >>>>
> > > >>>> * Status: Review complete.PR pushed.
> > > >>>>
> > > >>>> 3) Update pytool LicenseCheck plugin to use temp directory for diff output file
> > > >>>>
> > > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3746 <https://bugzilla.tianocore.org/show_bug.cgi?id=3746>
> > > >>>>
> > > >>>> *https://github.com/mdkinney/edk2/tree/Bug_3746_LicenseCheckUseDiffOutputFile_V2
> > > >>> <https://github.com/mdkinney/edk2/tree/Bug_3746_LicenseCheckUseDiffOutputFile_V2>
> > > >>>>
> > > >>>> *https://github.com/tianocore/edk2/pull/2217 <https://github.com/tianocore/edk2/pull/2217>
> > > >>>>
> > > >>>> * Required to reduce EDK II CI build times.
> > > >>>>
> > > >>>> * Status: Review complete.PR pushed.
> > > >>>>
> > > >>>> 4) Update max job time from 60 min to 120 minutes in .azurepipelines/templates
> > > >>>>
> > > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3750 <https://bugzilla.tianocore.org/show_bug.cgi?id=3750>
> > > >>>>
> > > >>>> *https://github.com/mdkinney/edk2/tree/Bug_3750_IncreaseAzurePipelinesTimeout
> > > >>> <https://github.com/mdkinney/edk2/tree/Bug_3750_IncreaseAzurePipelinesTimeout>
> > > >>>>
> > > >>>> *https://github.com/tianocore/edk2/pull/2219 <https://github.com/tianocore/edk2/pull/2219>
> > > >>>>
> > > >>>> * Required to allow EccCheck of uncrustify changes to complete on Azure
> > > >>>>
> > > >>>> Pipelines CI agents without timing out.
> > > >>>>
> > > >>>> * Status: Review complete.PR pushed.
> > > >>>>
> > > >>>> 5) Update Package YAML to ignore specific ECC files/errors
> > > >>>>
> > > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3749 <https://bugzilla.tianocore.org/show_bug.cgi?id=3749>
> > > >>>>
> > > >>>> *https://github.com/mdkinney/edk2/tree/Bug_3749_EccCheckIgnoreFilesErrors
> > > >>> <https://github.com/mdkinney/edk2/tree/Bug_3749_EccCheckIgnoreFilesErrors>
> > > >>>>
> > > >>>> *https://github.com/tianocore/edk2/pull/2218 <https://github.com/tianocore/edk2/pull/2218>
> > > >>>>
> > > >>>> * Required to pass EccCheck
> > > >>>>
> > > >>>> * Status: Review complete. PR pushed
> > > >>>>
> > > >>>> 6) Uncrustify Source Changes
> > > >>>>
> > > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3737 <https://bugzilla.tianocore.org/show_bug.cgi?id=3737>
> > > >>>>
> > > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3739 <https://bugzilla.tianocore.org/show_bug.cgi?id=3739>
> > > >>>>
> > > >>>> *https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
> > > >>> <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5>
> > > >>>>
> > > >>>> *https://github.com/tianocore/edk2/pull/2229 <https://github.com/tianocore/edk2/pull/2229>
> > > >>>>
> > > >>>> * Build comparison result PASS:https://github.com/mdkinney/edk2/actions/runs/1521618836
> > > >>> <https://github.com/mdkinney/edk2/actions/runs/1521618836>
> > > >>>>
> > > >>>> * EFI_D_ -> DEBUG changes required to pass PatchCheck
> > > >>>>
> > > >>>> * Uncrustify format changes required to pass UncrustifyCheck
> > > >>>>
> > > >>>> * Status:
> > > >>>>
> > > >>>> Waiting for review
> > > >>>>
> > > >>>> 7) UncrustifyCheck EDK II CI Plugin
> > > >>>>
> > > >>>> *https://bugzilla.tianocore.org/show_bug.cgi?id=3748 <https://bugzilla.tianocore.org/show_bug.cgi?id=3748>
> > > >>>>
> > > >>>> *https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v6
> > > >>> <https://github.com/mdkinney/edk2/tree/Bug_3748_add_uncrustify_ci_plugin_v6>
> > > >>>>
> > > >>>> * Required to enforce all PRs submitted to EDK II CI match uncrustify format.
> > > >>>>
> > > >>>> * Unconditionally checks all packages.Can not be committed until all C/H
> > > >>>>
> > > >>>> source files have been updated.
> > > >>>>
> > > >>>> * Status: Review complete
> > > >>>>
> > > >>>> Combined Branch/PR for Review/Test
> > > >>>>
> > > >>>> ==================================
> > > >>>>
> > > >>>> * Build Comparison results must pass 100% across the full set of PRs before
> > > >>>>
> > > >>>> the individual PRs can be pushed in the order listed above.
> > > >>>>
> > > >>>> * Branch:https://github.com/mdkinney/edk2/tree/TestOnly_Uncrustify_PR_Series
> > > >>> <https://github.com/mdkinney/edk2/tree/TestOnly_Uncrustify_PR_Series>
> > > >>>>
> > > >>>> * PR:https://github.com/tianocore/edk2/pull/2229 <https://github.com/tianocore/edk2/pull/2229>
> > > >>>>
> > > >>>> Status = PASS
> > > >>>>
> > > >>>> * CompareBuild:
> > > >>>>
> > > >>>> Branch:https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5
> > > >>> <https://github.com/mdkinney/edk2/tree/Bug_3737_3739_ApplyUncrustifyChanges_V5>
> > > >>>>
> > > >>>> --ref1:ef9a059cdb15844fe52a49af2bf7d86b9dd3e9bf
> > > >>>>
> > > >>>> --ref2:Bug_3737_3739_ApplyUncrustifyChanges_V5
> > > >>>>
> > > >>>> Extra Options: -n 4 --quiet
> > > >>>>
> > > >>>> Results:https://github.com/mdkinney/edk2/actions/runs/1521618836
> > > >>> <https://github.com/mdkinney/edk2/actions/runs/1521618836>
> > > >>>>
> > > >>>> 30 VS2019 build comparisons PASS
> > > >>>>
> > > >>>> 40 GCC5 build comparisons PASS
> > > >>>>
> > > >>>> 100% PASS
> > > >>>>
> > > >>>> The following git log shows the set of patches from --ref1 to --ref 2across
> > > >>>>
> > > >>>> which there are no differences in any of the OBJ/LIB/DLL/EFI/FFS/FV/FD files.
> > > >>>>
> > > >>>> --ref2
> > > >>>>
> > > >>>> b7d4bf0675b7 (HEAD -> Bug_3737_3739_ApplyUncrustifyChanges_V5) UnitTestFrameworkPkg: Apply uncrusitify
> changes
> > > >>>>
> > > >>>> 7f03d25f60e7 UefiPayloadPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> 0bfd8d9b5ac9 UefiCpuPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> e1cd9bfb9dea StandaloneMmPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> 5da2f65be378 SourceLevelDebugPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> 95b86de07e5d SignedCapsulePkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> fe71d97246c4 ShellPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> 54c21c952992 SecurityPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> 187a3785f12b RedfishPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> 810100002a46 PcAtChipsetPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> 276a695c0cf2 OvmfPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> 303c0a91ab07 NetworkPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> bc80792cd1b1 MdePkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> 3ea86be17a2a MdeModulePkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> c70ef11ed0cd IntelFsp2WrapperPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> c0291221f252 IntelFsp2Pkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> 6a479952a690 FmpDevicePkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> 3a7c05b7070d FatPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> b789f98c8959 EmulatorPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> 952d7a1c9220 EmbeddedPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> a1cc9881bab6 DynamicTablesPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> 50654dfe5785 CryptoPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> ed965a02dfa1 ArmVirtPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> 9744023fbc46 ArmPlatformPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> 7a1cde5f5bba ArmPkg: Apply uncrusitify changes
> > > >>>>
> > > >>>> 19d17e0913e8 UefiCpuPkg: Change use of EFI_D_* to DEBUG_*
> > > >>>>
> > > >>>> ffa718b4f994 SourceLevelDebugPkg: Change use of EFI_D_* to DEBUG_*
> > > >>>>
> > > >>>> b86cb3c5e5b4 ShellPkg: Change use of EFI_D_* to DEBUG_*
> > > >>>>
> > > >>>> c7c42204dc07 SecurityPkg: Change use of EFI_D_* to DEBUG_*
> > > >>>>
> > > >>>> 16b8e6f958e4 PcAtChipsetPkg: Change use of EFI_D_* to DEBUG_*
> > > >>>>
> > > >>>> 0ac3f8b2dac5 OvmfPkg: Change use of EFI_D_* to DEBUG_*
> > > >>>>
> > > >>>> bc5004b8d294 NetworkPkg: Change use of EFI_D_* to DEBUG_*
> > > >>>>
> > > >>>> 6f671a8e2377 MdePkg: Change use of EFI_D_* to DEBUG_*
> > > >>>>
> > > >>>> a10c610ff9a3 MdeModulePkg: Change use of EFI_D_* to DEBUG_*
> > > >>>>
> > > >>>> 09a3bddba390 FatPkg: Change use of EFI_D_* to DEBUG_*
> > > >>>>
> > > >>>> 59c61318246a EmulatorPkg: Change use of EFI_D_* to DEBUG_*
> > > >>>>
> > > >>>> 3a80367dda3b EmbeddedPkg: Change use of EFI_D_* to DEBUG_*
> > > >>>>
> > > >>>> 23eb1aaf80ca ArmVirtPkg: Change use of EFI_D_* to DEBUG_*
> > > >>>>
> > > >>>> 875914b45c54 ArmPlatformPkg: Change use of EFI_D_* to DEBUG_*
> > > >>>>
> > > >>>> eb2eca82b451 ArmPkg: Change use of EFI_D_* to DEBUG_*
> > > >>>>
> > > >>>> f0f3f5aae7c4 (origin/master, origin/HEAD, master) UnitTestFrameworkPkg: Update YAML to ignore specific ECC
> > > >>> files/errors
> > > >>>>
> > > >>>> c05734797790 UefiPayloadPkg: Update YAML to ignore specific ECC files/errors
> > > >>>>
> > > >>>> c30c40d6c63d StandaloneMmPkg: Update YAML to ignore specific ECC files/errors
> > > >>>>
> > > >>>> 9944508e85f1 ShellPkg: Update YAML to ignore specific ECC files/errors
> > > >>>>
> > > >>>> 60fa40be458d SecurityPkg: Update YAML to ignore specific ECC files/errors
> > > >>>>
> > > >>>> df790cd6b37e MdePkg: Update YAML to ignore specific ECC files/errors
> > > >>>>
> > > >>>> 9deb9370766e MdeModulePkg: Update YAML to ignore specific ECC files/errors
> > > >>>>
> > > >>>> d7d30e8f219f EmulatorPkg: Update YAML to ignore specific ECC files/errors
> > > >>>>
> > > >>>> d5744ecba813 CryptoPkg: Update YAML to ignore specific ECC files/errors
> > > >>>>
> > > >>>> c97fee87f0f9 ArmVirtPkg: Update YAML to ignore specific ECC files/errors
> > > >>>>
> > > >>>> 1939fc9569f2 ArmPlatformPkg: Update YAML to ignore specific ECC files/errors
> > > >>>>
> > > >>>> 365dced2c37a ArmPkg: Update YAML to ignore specific ECC files/errors
> > > >>>>
> > > >>>> 76a1ce4d5fec .azurepipelines/templates: Update max pipeline job time to 2 hours
> > > >>>>
> > > >>>> 99f84ff47390 .pytools/Plugin/LicenseCheck: Use temp directory for git diff output
> > > >>>>
> > > >>>> 3019f1bbabf1 .pytool/Plugin/EccCheck: Add performance optimizations
> > > >>>>
> > > >>>> 854462bd3479 .pytool/Plugin/EccCheck: Remove temp directory on exception
> > > >>>>
> > > >>>> 69877614fdee .pytool/Plugin/EccCheck: Remove RevertCode()
> > > >>>>
> > > >>>> --ref1
> > > >>>>
> > > >>>> ef9a059cdb15 EmulatorPkg/Win/Host: Update CC_FLAGS
> > > >>>>
> > > >>>> bb1bba3d7767 (tag: edk2-stable202111) NetworkPkg: Fix invalid pointer for DNS response token on error
> > > >>>>
> > > >>>> Best regards,
> > > >>>>
> > > >>>> Mike
> > > >>>>
> > > >>>>
^ permalink raw reply [flat|nested] 39+ messages in thread
end of thread, other threads:[~2021-12-07 10:03 UTC | newest]
Thread overview: 39+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2021-11-30 22:34 Uncrustify Conversion Detailed Plan and Extended Hard Freeze Update #4 Michael D Kinney
2021-12-01 1:42 ` Wu, Hao A
2021-12-01 2:10 ` 回复: [edk2-devel] " gaoliming
2021-12-01 2:26 ` Ni, Ray
2021-12-01 2:45 ` Chiu, Chasel
2021-12-01 5:34 ` Wang, Jian J
2021-12-01 6:38 ` Wang, Jian J
2021-12-01 7:33 ` [edk2-devel] " Abner Chang
2021-12-01 16:43 ` Michael D Kinney
2021-12-01 17:05 ` Michael Kubacki
2021-12-01 17:39 ` Michael D Kinney
2021-12-02 5:48 ` Abner Chang
2021-12-02 11:00 ` Gerd Hoffmann
2021-12-02 15:33 ` Michael Kubacki
2021-12-02 16:23 ` Gerd Hoffmann
2021-12-02 17:15 ` Michael Kubacki
2021-12-02 17:33 ` Michael D Kinney
2021-12-01 12:59 ` Sami Mujawar
2021-12-01 18:52 ` Bret Barkelew
2021-12-02 18:27 ` [edk2-devel] " Maciej Rabeda
2021-12-02 19:45 ` Michael D Kinney
2021-12-02 21:56 ` Michael Kubacki
2021-12-03 0:14 ` Michael D Kinney
2021-12-03 0:18 ` Michael D Kinney
2021-12-03 0:31 ` Michael Kubacki
2021-12-03 0:53 ` Michael D Kinney
2021-12-03 2:22 ` Michael D Kinney
2021-12-06 1:17 ` Michael D Kinney
2021-12-06 1:23 ` Wu, Hao A
2021-12-06 2:29 ` Ni, Ray
2021-12-06 9:31 ` 回复: " gaoliming
2021-12-06 19:45 ` Bret Barkelew
2021-12-06 20:28 ` Maciej Rabeda
2021-12-07 0:26 ` Abner Chang
2021-12-07 3:47 ` Wang, Jian J
2021-12-07 4:21 ` Chiu, Chasel
2021-12-07 10:02 ` Sami Mujawar
2021-12-03 8:56 ` Gerd Hoffmann
2021-12-03 16:25 ` Michael D Kinney
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox