I was testing out dumping a raw OVMF_VARS.fd into an FDF data section and noticed that my rebuilds of OVMF with no code changes went from 15 seconds to over 1 minute. The only change was the data in the NV_VARIABLE_STORE data section in OvmfPkg/Include/Fdf/VarStore.fdf.inc which changed the size of the file from about 4 KiB to about 256 KiB. I was rather curious as to why my build times changed so much and profiled the build process with cProfile. Specifically
https://github.com/tianocore/edk2/blob/master/BaseTools/Source/Python/GenFds/FdfParser.py#L279 takes the vast majority of the time.
You can reproduce this by adding 256 KiB of "#" characters to the end of OvmfPkg/Include/Fdf/VarStore.fdf.inc and building OVMF, which produced this result for me:
Build total time: 00:01:31
34883442 function calls (34368868 primitive calls) in 91.191 seconds
Ordered by: internal time
ncalls tottime percall cumtime percall filename:lineno(function)
18769 73.501 0.004 75.942 0.004 FdfParser.py:276(_SkipWhiteSpace)
728 5.738 0.008 5.738 0.008 {method 'acquire' of '_thread.lock' objects}
12 1.569 0.131 1.569 0.131 {method 'poll' of 'select.poll' objects}
2849255 1.248 0.000 1.439 0.000 FdfParser.py:354(_GetOneChar)
2305959 0.873 0.000 1.131 0.000 FdfParser.py:293(_EndOfFile)
5374641 0.831 0.000 0.831 0.000 FdfParser.py:368(_CurrentChar)
2 0.526 0.263 1.278 0.639 FdfParser.py:497(PreprocessFile)
Changing _SkippedChars from a string to StringIO reduced my build time to 19 seconds:
Build total time: 00:00:19
36552029 function calls (36037563 primitive calls) in 18.618 seconds
Ordered by: internal time
ncalls tottime percall cumtime percall filename:lineno(function)
728 5.391 0.007 5.391 0.007 {method 'acquire' of '_thread.lock' objects}
18769 1.593 0.000 3.452 0.000 FdfParser.py:277(_SkipWhiteSpace)
12 1.551 0.129 1.551 0.129 {method 'poll' of 'select.poll' objects}
2849255 0.850 0.000 0.979 0.000 FdfParser.py:355(_GetOneChar)
5374641 0.742 0.000 0.742 0.000 FdfParser.py:369(_CurrentChar)
2305959 0.741 0.000 0.951 0.000 FdfParser.py:294(_EndOfFile)
2 0.511 0.256 1.237 0.618 FdfParser.py:498(PreprocessFile)
This seems like an easy change to make builds a tiny bit faster.
Thanks,
David