From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (NAM10-MW2-obe.outbound.protection.outlook.com [40.92.42.101]) by mx.groups.io with SMTP id smtpd.web11.1788.1639101712595035163 for ; Thu, 09 Dec 2021 18:01:52 -0800 Authentication-Results: mx.groups.io; dkim=fail reason="body hash did not verify" header.i=@outlook.com header.s=selector1 header.b=XmQ3ruJz; spf=pass (domain: outlook.com, ip: 40.92.42.101, mailfrom: michael.kubacki@outlook.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Q/GGJVI+4UGi+tAHPqOdDBmUB/fGLb56U/aYtOEnUwv/298rlerT5z7rmY5IyJuPLJqUXB33tslIpLdYipJSRjhYyDm1Ne5+dwdtvHCUsyhUyqtipHLJTzNY1751obUymKFS9zRIwoTBMxYbWD5dIT3QGd+Um/qn0MffnZ/B8WqnzJi1ZS3UZVBTlt++QSeULZ0hYefUROquf6Va5mlde0Vu3cYKZPiGXJVlRXxFYYKAxCViZ5LDJrvarDjgRXpMs4vxPgBinrVKEN3EaE9TytdlHoPFxvydekxQB8Bn67ZvhqyemFzrtAjkOtH4DZNnggIm7h4xmuMH+a8JPDD4xQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=IPVjgRXoQs1cQPVtrQOHvP4FU5ZHMWJ9o2VqCNB2SJI=; b=gW9rctuZvnbKYZq8XEMzj+oo9WOs0iWfWdiMdbHr0ZRQ9W9J9bDS6g/JlWR99vurYrS9auEJOxUurBFljwBUyUXgMRJTK3sFf+66qpR+bsWOBjcAUxzPVBxO7r4KX8EP9mt/CQb4PaiDwFAlbje059iCdY1MZjU2AWs/9mMYrC5v9snv5BHucdhTHDYtlORUYSL8n9bE9fcgzOomSPWeJBpyrp8yHZJMy1oSNIMZQNjEOtWNbJQhOutg+2d+daciQKxT7N7UDZZIERDJAtnGvmkf16kaPOLXhR8Wy9zlm0fg3CrBWAfUpcQ45QYWQPpYWrqXVewHaqNGQUyhW+Ty9w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=IPVjgRXoQs1cQPVtrQOHvP4FU5ZHMWJ9o2VqCNB2SJI=; b=XmQ3ruJzIKjFbjUyoIulGSop7wq0uXDAPVOgK839ph8O3SkQDDY8BehKY2tdECt9FBE10bOkyAqF4N/f2pEQnot/9xQfv3HCPG9iEPvuMt9wHZiZz0RcCKYtHrQOtzt0AF4JzzlEZxBucMBVBdeV6dzDBlmPaRR0smDt+2AySpiEW2HWU6+zZ6Q77Ml0HUutwlXVbhUK6uHdxB9Ro/XkSR3bX0PwHLz6Ykm650dkD80E+n2P+6bzekK/frl0UoNd6lFFYuk5jgzMLolE4tR62wKH1o7f5NsAq4FhHNFI13KprJJUIOyLE9HbyRzKccEwQAesIhABjeu0MdqpZQvkPQ== Received: from MWHPR07MB3440.namprd07.prod.outlook.com (2603:10b6:301:69::28) by MW2PR07MB3994.namprd07.prod.outlook.com (2603:10b6:907:4::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4755.21; Fri, 10 Dec 2021 02:01:51 +0000 Received: from MWHPR07MB3440.namprd07.prod.outlook.com ([fe80::75d8:b961:c6c3:e97e]) by MWHPR07MB3440.namprd07.prod.outlook.com ([fe80::75d8:b961:c6c3:e97e%6]) with mapi id 15.20.4755.025; Fri, 10 Dec 2021 02:01:51 +0000 From: "Michael Kubacki" To: devel@edk2.groups.io CC: Andrew Fish , Leif Lindholm , Michael D Kinney , Liming Gao Subject: [edk2-wiki][PATCH v1 1/1] Add more details to EDK II Code Formatting (Uncrustify) Date: Thu, 9 Dec 2021 21:01:14 -0500 Message-ID: X-Mailer: git-send-email 2.28.0.windows.1 X-TMN: [nEy3tIWPBvyIeT/pMSNo8kDsIJN1Vb2NoOqJwCJK4iuKErmtso2kVDImEO70Ck8r] X-ClientProxiedBy: BN8PR04CA0010.namprd04.prod.outlook.com (2603:10b6:408:70::23) To MWHPR07MB3440.namprd07.prod.outlook.com (2603:10b6:301:69::28) Return-Path: michael.kubacki@outlook.com X-Microsoft-Original-Message-ID: <20211210020114.3832-1-michael.kubacki@outlook.com> MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from localhost.localdomain (2601:703:0:bb0:fd10:2167:3606:945b) by BN8PR04CA0010.namprd04.prod.outlook.com (2603:10b6:408:70::23) with Microsoft SMTP Server (version=TLS1_2, cipher=) via Frontend Transport; Fri, 10 Dec 2021 02:01:49 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: b293ef03-f4dd-4f0d-ae1a-08d9bb8107ce X-MS-TrafficTypeDiagnostic: MW2PR07MB3994:EE_ X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: t2FztRttdKBxUrHHsrj2iGFINu8gEh78ly+cGQkXUt0HXvyD/w7RHCiGsdiCOb7X7QYFbUVK6OajIGnq0vMiYaMslByXuTPUmmALm+sviF8mYICkknxkngGhhWkP7luLzjNM0bxQ5fF1+J1qcsZPX0Ul41yEQaaxPM5MOU54qywuK2ZPps+iEGIpA8tBmp6oWhKZHdteHjgMEGwf8t7yvPlELvYAw00NNhndZfxfxgif7HWzNyAZTubZqwtQ5Ctlzf8AxEdPllOspXvdcB+jbg0/OO40t1lb1rtQX3wkhpwS9R4zXFbmCTESqrCoSFcvalT83SasGsCkZnVZtQpuwSKIVj0f7rC9EpDbQsdu2KzxPIp2Y7JB/1+y8IwZWtqVVXLDjt9ukvEg/M6wjKDHBNMOfoBVUkSrIpkhfL42DVIBXpWjLXiV4sZ869uBUHsSn2/OEIzKpDlUJkEYiIlo3OdFwPSp58qhlXzbfG8bRFzsYRqyg4VTGyntCBqJDiyl01i6PGL0UaGOJbt61Q3+4SZ95SIzGQtmint5y0yM51MrFn3ETeokEeRXf1Pj18qSvMOJPyWJdeeb6t6Xf+ZBd940PFRv8+ovQYhVBOiogGJDOWJaSOB26K7+14WCT9TXviuJeb1qK2bJedIXdR/e5nsFGYJUoDV/qBPU70fboUI= X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?kLVPA4/+UkWi5AMZ/8cLdg3NKuB/gmp+fsJctCo8lSH8/Q9Ad8v44nvvDpcN?= =?us-ascii?Q?9x47xup815JXy9GRg6A8dhTjNmDr4lDnieR8/fIXrDtjmEYyHRer8MkASLDw?= =?us-ascii?Q?gp9dEZYq2smD/tF5OnqboQXj08HDhX69t9imhcD/r9/5PQAuZrBFPPs3Ahe1?= =?us-ascii?Q?7dzDZEJOS6QYSZtA0yZJamgxoMrC4OQStTEGneeYTTtJ4d9/9NdJuLx/sKya?= =?us-ascii?Q?PEunvX+d8C2cYpSyCwFHb5CsUyQ27+JEIEMynIQ8x6G9oE9QFD/gzMYzDvw0?= =?us-ascii?Q?ujxxMO82aQHcW/eMfSGWqGBMzLq/Rk3amjiqCMvyeI4qF8KqzttTEFNcib/4?= =?us-ascii?Q?2jmuTIw55Z4+0cY3rVE6SpAuBJI41dAPwZL+Q6zWfmBI87nXaVwfoJcefPt2?= =?us-ascii?Q?cRZqqy3UnugVfjwQ5Zqi4UHmcRMynjUic8h3oWVUtoJFgQ2WmeWzSC/f5hXO?= =?us-ascii?Q?XhGdL/A2/jX+Jb3bTX16p6L8NlHU+GLt5O6llvAAhjy6iZf9YWOn45ZNveZm?= =?us-ascii?Q?ClQ8h9yORdjUkw41J7ot4P45bBaFGQ9Fuae90jdwO0Y6gNguyIczwTV/PYKT?= =?us-ascii?Q?JMoMy6u6CyZkDk6DRBCJv34PgNrVJAnCSVvjA93SAgoQM+xavzKyMM50LEA1?= =?us-ascii?Q?T2TyY+Mns6Tv/FL+6iPWmfnEXIKchmCyJcKqX99lhtPngsTYVc0J2/JRRk6q?= =?us-ascii?Q?tDteyjrqpjTjw72iXHllaY3RLwA7bn07ja4E+1GVbIkCAxyBM5ebySmkhwZw?= =?us-ascii?Q?q67PSXEUPcRJBSXB4F+uA57DqHj/IJYBU2neBifkzWdBNFFE6nR6BTe0zbno?= =?us-ascii?Q?pfTPWicILUpbtg45H7Bbbe5/NL8DCbm4XpoP6GBWmgaI0GOPX7PvEPTlO1we?= =?us-ascii?Q?Yv8oRE/qXJLWvLNTRtfvaao1fALgzA/VIpVPhu++kWBki9Lyv3Jc8xCU2VDK?= =?us-ascii?Q?l1BOmH4fZqv7wHBXicF/wS93Js1Lp8ejdntFW6ejKqdu9mSCZa/7XQ0ogKe+?= =?us-ascii?Q?69hWEAP7XQKrZ9Rsovj5nnZeuA=3D=3D?= X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: b293ef03-f4dd-4f0d-ae1a-08d9bb8107ce X-MS-Exchange-CrossTenant-AuthSource: MWHPR07MB3440.namprd07.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Dec 2021 02:01:51.1108 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW2PR07MB3994 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain From: Michael Kubacki Updates the wiki page to include: 1. Minor formatting updates 2. Additional links to critical files 3. Increased details in a few important sections 4. More detailed manual usage instructions 5. More Linux examples Cc: Andrew Fish Cc: Leif Lindholm Cc: Michael D Kinney Cc: Liming Gao Signed-off-by: Michael Kubacki --- Notes: A rendered version of the markdown document in this patch is available here: =20 https://github.com/makubacki/tianocore.github.io/blob/update_uncrustify= _instructions/EDK-II-Code-Formatting.md EDK-II-Code-Formatting.md | 86 +++++++++++++++----- 1 file changed, 67 insertions(+), 19 deletions(-) diff --git a/EDK-II-Code-Formatting.md b/EDK-II-Code-Formatting.md index f634193cb608..8071e0863e62 100644 --- a/EDK-II-Code-Formatting.md +++ b/EDK-II-Code-Formatting.md @@ -1,6 +1,6 @@ # EDK II Code Formatting =20 -To better achieve the goals of the [EDK II C Coding Standards Specificatio= n](https://edk2-docs.gitbook.io/edk-ii-c-coding-standards-specification/), +To better realize the goals of the [EDK II C Coding Standards Specificatio= n](https://edk2-docs.gitbook.io/edk-ii-c-coding-standards-specification/), EDK II code formatting is automated using a source code beautifier called = Uncrustify. Uncrustify is compatible with C/C++ in addition to other languages. In EDK II, it is used to format C la= nguage source code. =20 @@ -20,11 +20,11 @@ standard: header. * **default_function_header.txt** - A text file containing a template that= is placed above functions that are missing a function header. -* **Readme.md** - A file that contains details about how the plugin works = and how to use it. +* [**Readme.md**](https://github.com/tianocore/edk2/blob/master/.pytool/Pl= ugin/UncrustifyCheck/Readme.md) - A file that contains details about how th= e plugin works and how to use it. * **uncrustify_ext_dep.yml** - An "external dependency" file that is used = to get the current version of the Uncrustify application used by the plugin. This file contains the NuGet feed URL an= d the version currently used. * **uncrustify_plug_in.yaml** - A file that contains information to descri= be the plugin to the build environment. -* **uncrustify.cfg** - A file used by the Uncrustify application to contro= l how it formats code. If you want to tweak +* [**uncrustify.cfg**](https://github.com/tianocore/edk2/blob/master/.pyto= ol/Plugin/UncrustifyCheck/uncrustify.cfg) - A file used by the Uncrustify a= pplication to control how it formats code. If you want to tweak particular formatting details, this is the place to start. * **UncrustifyCheck.py** - The actual Python file that is the CI plugin. L= ike all CI plugins, this plugin can be run in local CI and server CI. @@ -32,7 +32,7 @@ standard: ## EDK II Uncrustify Fork =20 Due to nuances in the way EDK II formats code, some changes were made to t= he upstream Uncrustify application. These -were changes that could not be made purely through the Uncrustify configur= ation file. For more details about the fork, +were changes that could not be controlled purely through the Uncrustify co= nfiguration file. For more details about the fork, please visit that project overview in the link below. =20 * Uncrustify upstream repository: https://github.com/uncrustify/uncrustify @@ -52,8 +52,9 @@ The recommended flow is: 2. Use the `stuart*` commands to pull the Uncrustify application into the = edk2 workspace 3. Set up the ability to run Uncrustify locally (for example, using the Vi= sual Studio Code Uncrustify plugin) 4. Make and test code changes -5. Run EDK II CI locally to verify UncrustifyCheck passes -6. Send the code patch to the EDK II mailing list +5. Format code locally using Uncrustify (for example, using the Visual Stu= dio Code Uncrustify plugin) +6. Run EDK II CI locally to verify UncrustifyCheck passes +7. Send the code patch to the EDK II mailing list =20 ## Installing Uncrustify =20 @@ -65,7 +66,7 @@ and ultimately published into a NuGet feed in that fork p= roject. It is strongly recommended to follow this flow. It sets up the workspace t= o work with local CI and automatically gets the current supported version of the application. =20 -The Uncrustify tool is installed automatically when the Pytools environmen= t is used and the `stuart*` commands are run +The Uncrustify tool is installed automatically when the pytools environmen= t is used and the `stuart*` commands are run to complete environment setup. Review the edk2 [.pytool/Readme.md](https:/= /github.com/tianocore/edk2/tree/master/.pytool#running-ci-locally) file for details on `stuart` and this overall flow. =20 @@ -76,9 +77,9 @@ application. =20 ### Manual Installation: Download from the fork project =20 -The release pipeline in the EDK II Uncrustify fork project contains the bu= ild information for each release. Each build -in this pipeline represents a release. By going to a specific build, the d= etails mapping the build to source code -(such as the branch and commit) are present. +The [release pipeline](https://dev.azure.com/projectmu/Uncrustify/_build?d= efinitionId=3D89) in the EDK II Uncrustify fork +project contains the build information for each release. Each build in thi= s pipeline represents a release. By going to +a specific build, the details mapping the build to source code (such as th= e branch and commit) are present. =20 The build content is published as a NuGet package to a NuGet feed. This is= the same feed, the recommended installation instructions automatically pull from. The NuGet feed is available in the [= "Artifacts"](https://dev.azure.com/projectmu/Uncrustify/_packaging?_a=3Dfee= d&feed=3Dmu_uncrustify) @@ -132,13 +133,48 @@ the configuration details are set once in the editor = configuration file. These instructions are written for Windows 10. These activities could be f= urther automated into a high-level script but that has not been done yet. =20 +#### Manual Usage - Generate File List + +Uncrustify must be given a list of files to run against. This can be done = by redirecting the list to stdin or by +providing a text file. Examples of how perform both approaches are given b= elow. + +##### Manual Usage - Generate File List via stdin + +This is the recommended way to manually run Uncrustify. It is works across= Linux and Windows and reduces the number of +overall steps. + +Example to run against all .c and .h files in `DynamicTablesPkg` executed = from the root of the edk2 workspace. + +Linux: + +```shell +git ls-files DynamicTablesPkg*.c DynamicTablesPkg*.h | ./.pytool/Plugin/Un= crustifyCheck/mu-uncrustify-release_extdep/Linux-x86/uncrustify -c ./.pytoo= l/Plugin/UncrustifyCheck/uncrustify.cfg -F - --replace --no-backup --if-cha= nged +``` + +Windows: + +```shell +git ls-files DynamicTablesPkg*.c DynamicTablesPkg*.h | .\.pytool\Plugin\Un= crustifyCheck\mu-uncrustify-release_extdep\Windows-x86\uncrustify.exe -c .\= .pytool\Plugin\UncrustifyCheck\uncrustify.cfg -F - --replace --no-backup --= if-changed +``` + +* The `git ls-files` command is used to gather the list of .c and .h files= in `DynamicTablesPkg` +* The output from `git ls-files` is redirected to `uncrustify` +* The following options are given to the Uncrustify application: + * `-c`: The path to the Uncrustify configuration file + * `-F`: Read the files one per line. `-` indicates the list should be re= ad from stdin. + * `--replace`: Replace the source files in place (convenient to diff for= matting with git) + * `--no-backup`: Replace files with no backup (again, useful to diff for= matting with git) + * `--if-changed`: Only produce output if a change is detected. + +##### Manual Usage - Generate File List via Text File + 1. Generate a list of the files to run against. This example generates a r= ecursive list of all .c and .h files. =20 * It is recommended to run this in cleanly cloned edk2 repo without sub= modules to prevent submodule files (such as Brotli files in MdeModulePkg) from getting included in the f= ile list (if you are running against all files). Including all files will significantly increase the amount of= time Uncrustify takes to run. =20 - * Sample Powershell command to recursively write all .c and .h files in= a given package to a text file: + * Sample Powershell command to recursively write all .c and .h files in= a given package to a text file (this can of course be done with other lang= uages/commands): =20 ```powershell Get-ChildItem -Path .\MdePkg\* -Include *.c, *.h -Recurse -Force | %{= $_.fullname} | Out-File @@ -147,29 +183,39 @@ but that has not been done yet. =20 > **WARNING** Powershell will put the UTF-8 BOM at the beginning of the= output file. Uncrustify does not recognize the BOM and it should be removed before passing the file as input to Uncrustify= . If it is not removed, Uncrustify will - not read the first file path in the text file properly which will cause= the file to not be formatted. + not read the first file path in the text file properly which will cause= the file to not be formatted. **Keep this in mind regardless of the method= used for generating the text file.** =20 2. Run Uncrustify using the generated text file as input =20 - The following assume you move the EDK II Uncrustify configuration file = to the directory .uncrustify in your edk2 - workspace. +Example to run against all .c and .h files in `MdePkg` executed from the r= oot of the edk2 workspace. + + Windows: =20 ```shell - uncrustify.exe -c .\.pytool\Plugin\UncrustifyCheck\uncrustify.cfg -F Md= ePkgFiles.txt --replace --no-backup --if-changed + .\.pytool\Plugin\UncrustifyCheck\mu-uncrustify-release_extdep\Windows-x= 86\uncrustify.exe -c .\.pytool\Plugin\UncrustifyCheck\uncrustify.cfg -F Mde= PkgFiles.txt --replace --no-backup --if-changed ``` =20 +* The following options are given to the Uncrustify application: + * `-c`: The path to the Uncrustify configuration file + * `-F`: Read the files one per line from the file `MdePkgFiles.txt` + * `--replace`: Replace the source files in place (convenient to diff for= matting with git) + * `--no-backup`: Replace files with no backup (again, useful to diff for= matting with git) + * `--if-changed`: Only produce output if a change is detected. + > *Note:* When testing a configuration change, it is sometimes useful t= o run Uncrustify against a particular file and check the debug output to understand what rule was applied and wh= y it was applied. The command shows an example of how to run the configuration file `uncrustify.cfg` against= the source file `VariableSmm.c` where the file is forced to be treated as C, the debug output is written to `un= crustify_debug.txt` and the log severity level is set to "all". =20 + Windows: + ```shell - uncrustify.exe -c .\.pytool\Plugin\UncrustifyCheck\uncrustify.cfg -f .\= MdeModulePkg\Universal\Variable\RuntimeDxe\VariableSmm.c -o output.c -l C -= p uncrustify_debug.txt -L A 2>verbose_debug.txt + .\.pytool\Plugin\UncrustifyCheck\mu-uncrustify-release_extdep\Windows-x= 86\uncrustify.exe -c .\.pytool\Plugin\UncrustifyCheck\uncrustify.cfg -f .\M= deModulePkg\Universal\Variable\RuntimeDxe\VariableSmm.c -o output.c -l C -p= uncrustify_debug.txt -L A 2>verbose_debug.txt ``` =20 Uncrustify will update the source files in-place (with the commands given)= . This allows you to diff the results with -git. From here, you can iteratively tweak the configuration file and check= the results until your satisfied with the +git. From here, you can iteratively tweak the configuration file and check= the results until you are satisfied with the outcome. =20 ## Uncrustify in CI @@ -192,7 +238,9 @@ terminal output in local CI and the build output log in= server CI. Read the [UncrustifyCheck Readme.md](https://github.com/tianocore/edk2/blo= b/master/.pytool/Plugin/UncrustifyCheck/Readme.md) to understand more about how the plugin can be configured for CI. =20 -## Extra Reading: Tracing History Across the Uncrustify Changes +## Extra Reading: Tracing History Across the Uncrustify Transition in edk + +Note: Most users do not need to read this section. =20 It might be helpful to view the entire history rewritten with Uncrustify f= ormatting on every commit. For example, an alternate version of the edk2 repository that serves as "documentation" wi= th the entire history re-written. @@ -206,7 +254,7 @@ A tool called git-filter-repo can be used to perform th= is transformation and run The following steps can be used to perform this transformation. This is th= e Windows process. A Linux process will be added in the future. =20 - > **WARNING** This operation modifies (rewrites) all the commits in the= local copy of the repo. Do not perform + > **WARNING** This operation modifies (rewrites) all the commits in the= local copy of the repo. Do not perform these steps on a local repo you are using for active deve= lopment. =20 1. Clone edk2 into a new directory (see **WARNING**) --=20 2.28.0.windows.1