Re: [PATCH 00/47] edk2: centralize and adopt ARRAY_SIZE()

From: Laszlo Ersek <lersek@redhat.com>
To: Jordan Justen <jordan.l.justen@intel.com>,
	edk2-devel-01 <edk2-devel@ml01.01.org>
Cc: Ruiyu Ni <ruiyu.ni@intel.com>, Tim He <tim.he@intel.com>,
	Feng Tian <feng.tian@intel.com>,
	Michael D Kinney <michael.d.kinney@intel.com>,
	Eric Dong <eric.dong@intel.com>,
	Cecil Sheng <cecil.sheng@hpe.com>,
	Ard Biesheuvel <ard.biesheuvel@linaro.org>,
	Liming Gao <liming.gao@intel.com>,
	Dandan Bi <dandan.bi@intel.com>, Jiaxin Wu <jiaxin.wu@intel.com>,
	Gary Lin <glin@suse.com>, Star Zeng <star.zeng@intel.com>,
	Daryl McDaniel <edk2-lists@mc2research.org>,
	Jaben Carsey <jaben.carsey@intel.com>,
	Siyuan Fu <siyuan.fu@intel.com>, Jeff Fan <jeff.fan@intel.com>,
	Chao Zhang <chao.b.zhang@intel.com>,
	David Wei <david.wei@intel.com>
Subject: Re: [PATCH 00/47] edk2: centralize and adopt ARRAY_SIZE()
Date: Thu, 27 Oct 2016 10:49:20 +0200	[thread overview]
Message-ID: <44954933-cebe-a5f3-2466-c5e92547611e@redhat.com> (raw)
In-Reply-To: <147754834759.8915.16019621269123697387@jljusten-ivb>

On 10/27/16 08:05, Jordan Justen wrote:
> On 2016-10-26 12:04:17, Laszlo Ersek wrote:
>> I'm sure most of us have wished at some point that ARRAY_SIZE()
>> had existed in a central header file, like it does in many other
>> open source projects.
> 
> Indeed!
> 
>> This series implements that.
>>
>> Patches 01-04 round up the modules that #define ARRAY_SIZE() just
>> for themselves, and wrap their definitions in #ifndef ARRAY_SIZE.
>>
>> Patch 05 defines ARRAY_SIZE() in "MdePkg/Include/Base.h". This
>> flips the previously modified modules to MdePkg's macro definition.
>>
>> Patches 06-09 modify the same set of modules as patches 01-04 do; this
>> time the (now unused) module-local ARRAY_SIZE() macro definitions are
>> eliminated.
>>
>> The rest of the patches (10-47) migrate the edk2 tree to ARRAY_SIZE(),
>> covering the most obvious open-coded ARRAY_SIZE() expressions. These
>> expressions were located and converted with the following shell
>> pipeline:
>>
>>   git ls-files \
>>   | xargs --no-run-if-empty -- \
>>       sed --regexp-extended --in-place \
>>         --expression='s,sizeof \(([a-zA-Z0-9_]+)\) / sizeof \((\*\1|\1\[0\])\),ARRAY_SIZE (\1),g' \
>>         --
>>
>> This pipeline lists all tracked files with "git". The pathnames are then
>> composed with "xargs" into maximal length "sed" command lines, and those
>> "sed" commands are run.
>>
>> Here the "sed" stream editor filters and modifies all the listed files
>> in place. The extended regular expression replaces the following two
>> patterns:
>>
>>   sizeof (ArrayName) / sizeof (*ArrayName)
>>   sizeof (ArrayName) / sizeof (ArrayName[0])
>>
>> with
>>
>>   ARRAY_SIZE (ArrayName)
>>
>> The regexp uses a backreference in the match, ensuring that the sizeof
>> operator in the dividend and the sizeof operator in the divisor both
>> refer to the same ArrayName. If there are multiple occurrences on the
>> same line, the sed command replaces all of them.
>>
>> I collected the changes into patches manually, and I reviewed them all.
>>
>> I test-built all of the modified packages, repeating the command
>>
>>   build -a X64 -t GCC48 -b DEBUG -p FooPkg/FooPkg.dsc
>>
>> as many times as necessary. Two packages were impossible to build like
>> this: Nt32Pkg and Vlv2TbltDevicePkg, so I couldn't build-test them.
>>
>> I did some light-weight testing with OVMF: boot with Secure Boot
>> enabled, S3 suspend/resume with and without SMM. I also booted
>> ArmVirtQemu once.
>>
>> Repository: https://github.com/lersek/edk2.git
>> Branch:     array_size
>>
>> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
>> Cc: Cecil Sheng <cecil.sheng@hpe.com>
>> Cc: Chao Zhang <chao.b.zhang@intel.com>
>> Cc: Dandan Bi <dandan.bi@intel.com>
>> Cc: Daryl McDaniel <edk2-lists@mc2research.org>
>> Cc: David Wei <david.wei@intel.com>
>> Cc: Eric Dong <eric.dong@intel.com>
>> Cc: Feng Tian <feng.tian@intel.com>
>> Cc: Gary Lin <glin@suse.com>
>> Cc: Jaben Carsey <jaben.carsey@intel.com>
>> Cc: Jeff Fan <jeff.fan@intel.com>
>> Cc: Jiaxin Wu <jiaxin.wu@intel.com>
>> Cc: Jordan Justen <jordan.l.justen@intel.com>
>> Cc: Liming Gao <liming.gao@intel.com>
>> Cc: Michael D Kinney <michael.d.kinney@intel.com>
>> Cc: Ruiyu Ni <ruiyu.ni@intel.com>
>> Cc: Siyuan Fu <siyuan.fu@intel.com>
>> Cc: Star Zeng <star.zeng@intel.com>
>> Cc: Tim He <tim.he@intel.com>
>>
>> Thanks
>> Laszlo
>>
>> Laszlo Ersek (47):
>>   MdeModulePkg/RegularExpressionDxe: guard the definition of ARRAY_SIZE
>>   NetworkPkg/IpsecConfig: guard the definition of ARRAY_SIZE
>>   OvmfPkg/PlatformBootManagerLib: guard the definition of ARRAY_SIZE
>>   OvmfPkg/XenBusDxe: guard the definition of ARRAY_SIZE
>>   MdePkg/Include/Base.h: introduce the ARRAY_SIZE() function-like macro
>>   MdeModulePkg/RegularExpressionDxe: remove module-local ARRAY_SIZE
>>     macro
>>   NetworkPkg/IpsecConfig: remove module-local ARRAY_SIZE macro
>>   OvmfPkg/PlatformBootManagerLib: remove module-local ARRAY_SIZE macro
>>   OvmfPkg/XenBusDxe: remove module-local ARRAY_SIZE macro
>>   ArmVirtPkg/NorFlashQemuLib: rebase to ARRAY_SIZE()
> 
> Maybe "Make use of" or something like that might be preferable to
> "rebase to", but it is not a big deal.

Yes, "utilize" and "adopt" crossed my mind at some point, but then I had
been quite into the middle of the series. Oh well. :)

> 
> I think I also saw some 'after' lines longer than 80 columns, but sed
> can only invoke a limited level of magic after all. :) I think you
> fixed *many* overly long lines in this series. Thanks for that as
> well! :)

Right, I was happy to see that as well. The shortening seemed especially
useful for:
- the new safe string functions, where ARRAY_SIZE() got applied to a
CHAR16 array,
- controlling expressions of "if" and "for" statements,
- assertions.

I saw that some lines remained that were still too long, but one wants
to get only so clever in a SED regexp, like you say :)

> Series Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>

Thank you!
Laszlo

> 
>>   DuetPkg/DuetBdsLib: rebase to ARRAY_SIZE()
>>   EdkCompatibilityPkg/FrameworkHiiOnUefiHiiThunk: rebase to ARRAY_SIZE()
>>   EdkCompatibilityPkg/Sample/Tools: rebase to ARRAY_SIZE()
>>   FatPkg/EnhancedFatDxe: rebase to ARRAY_SIZE()
>>   IntelFrameworkModulePkg/LegacyBootManagerLib: rebase to ARRAY_SIZE()
>>   IntelFrameworkModulePkg/BdsDxe: rebase to ARRAY_SIZE()
>>   MdeModulePkg/MemoryProfileInfo: rebase to ARRAY_SIZE()
>>   MdeModulePkg/PciBusDxe: rebase to ARRAY_SIZE()
>>   MdeModulePkg/PciHostBridgeDxe: rebase to ARRAY_SIZE()
>>   MdeModulePkg/UsbBusDxe: rebase to ARRAY_SIZE()
>>   MdeModulePkg/Core/Dxe: rebase to ARRAY_SIZE()
>>   MdeModulePkg/Core/PiSmmCore: rebase to ARRAY_SIZE()
>>   MdeModulePkg/BootMaintenanceManagerUiLib: rebase to ARRAY_SIZE()
>>   MdeModulePkg/BootManagerUiLib: rebase to ARRAY_SIZE()
>>   MdeModulePkg/UefiBootManagerLib: rebase to ARRAY_SIZE()
>>   MdeModulePkg/VarCheckHiiLib: rebase to ARRAY_SIZE()
>>   MdeModulePkg/Logo: rebase to ARRAY_SIZE()
>>   MdeModulePkg/BdsDxe: rebase to ARRAY_SIZE()
>>   MdeModulePkg/DisplayEngineDxe: rebase to ARRAY_SIZE()
>>   MdeModulePkg/EbcDxe: rebase to ARRAY_SIZE()
>>   MdeModulePkg/Tcp4Dxe: rebase to ARRAY_SIZE()
>>   MdeModulePkg/Variable/RuntimeDxe: rebase to ARRAY_SIZE()
>>   NetworkPkg/IScsiDxe: rebase to ARRAY_SIZE()
>>   NetworkPkg/TcpDxe: rebase to ARRAY_SIZE()
>>   Nt32Pkg/WinNtSerialIoDxe: rebase to ARRAY_SIZE()
>>   OptionRomPkg/AtapiPassThruDxe: rebase to ARRAY_SIZE()
>>   OvmfPkg/QemuBootOrderLib: rebase to ARRAY_SIZE()
>>   OvmfPkg/QemuVideoDxe: rebase to ARRAY_SIZE()
>>   PerformancePkg/Dp_App: rebase to ARRAY_SIZE()
>>   SecurityPkg/AuthVariableLib: rebase to ARRAY_SIZE()
>>   ShellPkg/UefiDpLib: rebase to ARRAY_SIZE()
>>   ShellPkg/UefiShellLevel2CommandsLib: rebase to ARRAY_SIZE()
>>   Vlv2TbltDevicePkg/FirmwareUpdate: rebase to ARRAY_SIZE()
>>   Vlv2TbltDevicePkg/PlatformInitPei: rebase to ARRAY_SIZE()
>>   Vlv2TbltDevicePkg/PlatformPei: rebase to ARRAY_SIZE()
>>   Vlv2TbltDevicePkg/PlatformSetupDxe: rebase to ARRAY_SIZE()
>>   Vlv2TbltDevicePkg/SmBiosMiscDxe: rebase to ARRAY_SIZE()
>>
>>  ArmVirtPkg/Library/NorFlashQemuLib/NorFlashQemuLib.c                          |  2 +-
>>  DuetPkg/Library/DuetBdsLib/BdsPlatform.c                                      |  2 +-
>>  EdkCompatibilityPkg/Compatibility/FrameworkHiiOnUefiHiiThunk/OpcodeCreation.c |  2 +-
>>  EdkCompatibilityPkg/Sample/Tools/Source/EfiRom/EfiRom.c                       |  4 ++--
>>  EdkCompatibilityPkg/Sample/Tools/Source/HiiPack/IfrParse.c                    |  2 +-
>>  FatPkg/EnhancedFatDxe/DirectoryManage.c                                       |  4 ++--
>>  FatPkg/EnhancedFatDxe/FileName.c                                              |  4 ++--
>>  FatPkg/EnhancedFatDxe/Hash.c                                                  |  4 ++--
>>  IntelFrameworkModulePkg/Library/LegacyBootManagerLib/LegacyBm.c               |  2 +-
>>  IntelFrameworkModulePkg/Universal/BdsDxe/BdsEntry.c                           |  2 +-
>>  IntelFrameworkModulePkg/Universal/BdsDxe/BootMaint/BBSsupport.c               |  4 ++--
>>  IntelFrameworkModulePkg/Universal/BdsDxe/BootMaint/BootMaint.c                |  8 ++++----
>>  IntelFrameworkModulePkg/Universal/BdsDxe/BootMaint/BootOption.c               |  2 +-
>>  IntelFrameworkModulePkg/Universal/BdsDxe/BootMaint/ConsoleOption.c            |  6 +++---
>>  IntelFrameworkModulePkg/Universal/BdsDxe/BootMaint/UpdatePage.c               | 10 +++++-----
>>  IntelFrameworkModulePkg/Universal/BdsDxe/BootMaint/Variable.c                 |  2 +-
>>  IntelFrameworkModulePkg/Universal/BdsDxe/BootMngr/BootManager.c               |  2 +-
>>  MdeModulePkg/Application/MemoryProfileInfo/MemoryProfileInfo.c                |  6 +++---
>>  MdeModulePkg/Bus/Pci/PciBusDxe/PciLib.c                                       |  2 +-
>>  MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c                         |  2 +-
>>  MdeModulePkg/Bus/Usb/UsbBusDxe/UsbHub.c                                       |  4 ++--
>>  MdeModulePkg/Core/Dxe/Mem/Pool.c                                              |  2 +-
>>  MdeModulePkg/Core/PiSmmCore/Pool.c                                            |  2 +-
>>  MdeModulePkg/Core/PiSmmCore/SmramProfileRecord.c                              |  6 +++---
>>  MdeModulePkg/Library/BootMaintenanceManagerUiLib/BootMaintenance.c            |  8 ++++----
>>  MdeModulePkg/Library/BootMaintenanceManagerUiLib/ConsoleOption.c              |  8 ++++----
>>  MdeModulePkg/Library/BootMaintenanceManagerUiLib/UpdatePage.c                 | 12 ++++++------
>>  MdeModulePkg/Library/BootMaintenanceManagerUiLib/Variable.c                   |  2 +-
>>  MdeModulePkg/Library/BootManagerUiLib/BootManager.c                           |  6 +++---
>>  MdeModulePkg/Library/UefiBootManagerLib/BmBootDescription.c                   |  2 +-
>>  MdeModulePkg/Library/UefiBootManagerLib/BmConsole.c                           |  2 +-
>>  MdeModulePkg/Library/UefiBootManagerLib/BmHotkey.c                            |  2 +-
>>  MdeModulePkg/Library/UefiBootManagerLib/BmLoadOption.c                        |  4 ++--
>>  MdeModulePkg/Library/VarCheckHiiLib/VarCheckHiiGen.c                          |  4 ++--
>>  MdeModulePkg/Library/VarCheckHiiLib/VarCheckHiiLibNullClass.c                 |  2 +-
>>  MdeModulePkg/Logo/Logo.c                                                      |  2 +-
>>  MdeModulePkg/Universal/BdsDxe/BdsEntry.c                                      |  2 +-
>>  MdeModulePkg/Universal/DisplayEngineDxe/FormDisplay.c                         |  4 ++--
>>  MdeModulePkg/Universal/DisplayEngineDxe/InputHandler.c                        |  6 +++---
>>  MdeModulePkg/Universal/EbcDxe/EbcExecute.c                                    |  2 +-
>>  MdeModulePkg/Universal/Network/Tcp4Dxe/Tcp4Output.c                           |  2 +-
>>  MdeModulePkg/Universal/RegularExpressionDxe/RegularExpressionDxe.h            |  2 --
>>  MdeModulePkg/Universal/Variable/RuntimeDxe/Variable.c                         |  2 +-
>>  MdePkg/Include/Base.h                                                         | 13 +++++++++++++
>>  NetworkPkg/Application/IpsecConfig/IpSecConfig.h                              |  2 --
>>  NetworkPkg/IScsiDxe/IScsiConfig.c                                             |  4 ++--
>>  NetworkPkg/IScsiDxe/IScsiDriver.c                                             |  2 +-
>>  NetworkPkg/IScsiDxe/IScsiMisc.c                                               |  6 +++---
>>  NetworkPkg/TcpDxe/TcpOutput.c                                                 |  2 +-
>>  Nt32Pkg/WinNtSerialIoDxe/WinNtSerialIo.c                                      |  2 +-
>>  OptionRomPkg/AtapiPassThruDxe/AtapiPassThru.c                                 |  4 ++--
>>  OvmfPkg/Library/PlatformBootManagerLib/BdsPlatform.c                          |  5 -----
>>  OvmfPkg/Library/QemuBootOrderLib/QemuBootOrderLib.c                           | 12 ++++++------
>>  OvmfPkg/QemuVideoDxe/Initialize.c                                             |  4 ++--
>>  OvmfPkg/XenBusDxe/XenStore.c                                                  |  1 -
>>  PerformancePkg/Dp_App/Dp.c                                                    |  2 +-
>>  PerformancePkg/Dp_App/DpTrace.c                                               | 10 +++++-----
>>  SecurityPkg/Library/AuthVariableLib/AuthVariableLib.c                         |  4 ++--
>>  ShellPkg/Library/UefiDpLib/DpTrace.c                                          | 10 +++++-----
>>  ShellPkg/Library/UefiShellLevel2CommandsLib/TimeDate.c                        |  2 +-
>>  Vlv2TbltDevicePkg/Application/FirmwareUpdate/FirmwareUpdate.c                 |  2 +-
>>  Vlv2TbltDevicePkg/PlatformInitPei/FlashMap.c                                  |  2 +-
>>  Vlv2TbltDevicePkg/PlatformPei/BootMode.c                                      |  6 +++---
>>  Vlv2TbltDevicePkg/PlatformSetupDxe/SetupInfoRecords.c                         |  2 +-
>>  Vlv2TbltDevicePkg/SmBiosMiscDxe/MiscOemType0x94Function.c                     |  2 +-
>>  65 files changed, 131 insertions(+), 128 deletions(-)
>>
>> -- 
>> 2.9.2
>>
> _______________________________________________
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel
> 