Re: [edk2-devel] FW: Discussion: Basetools a separate repo

["Matthew Carlson" <macarl@microsoft.com>]

[-- Attachment #1: Type: text/plain, Size: 3249 bytes --]

Hey Lazlo,

This is a great point of discussion. Just to make sure we're on the same page, let me paraphrase your scenario. Two features at the same time or a improperly tested new feature were committed in BaseTools and a new release was spun out. EDK2 was updated to have it's pip-requirements.txt to point to the new version committed. In an ideal world, CI should catch the build break for MdeModulePkg but let's say it didn't catch it. Your scenario is, how do you go back to the previous version of basetools that EDK2 used or perhaps go the the version of BaseTools that last worked.

There are several great ways to do this. The release cadence of a pip based BaseTools hasn't really been discussed but one option was every feature in BaseTools would be a new release (this would be easy to do via pipelines). The exact versioning would need to be discussed but there would likely be some notion of breaking changes with major and minor version. This means that you would be able to easily pip install the basetools 1:1 with the feature commit history. In the scenario you mentioned, it would be easy to roll back to C0, just "pip install -r the previous requirements file". If a virtual environment is setup, this pip install would be entirely localized to the workspace the user is operating in. The git show command you mentioned would show the pip-requirements file. We would likely tag the pip_requirements file to a specific release of BaseTools and there might be another file like pip_suggestions that would allow for a looser version such as 0.3.x were any minor version in 0.3 is fine. That said, determining what exact basetools was used in a given build would likely be best accomplished through build artifacts. We have a BuildToolsReport generator that specifically does this ( https://github.com/tianocore/edk2/tree/master/BaseTools/Plugin/BuildToolsReport ) and it would generate a report that would detail the exact pip version used in that build. Without build artifacts, it grows a little bit more fuzzy, but you can have reasonable confidence since it would need to at least conform to pip suggestions and likely was on pip_requirements. To summarize, moving back in time is simple and even trivial with minimal delay to a developer (pip installs are usually sub-second procedures). There might even be a part of edksetup that might do the pip install for you if that's what the community wants.

Going to the next step, if you wanted to debug the BaseTools at C1, you might want to generate a diff of the basetools to try and see where the error and compare the diff. This I think better addresses your question of bisectability (not sure if that's a word). You could clone the basetools repo and do a pip install -e {path to basetools}. You would checkout the version in C1 pip_requirements and generate the diff between it and C0. Then you could apply changes or revert changes and test them locally as the basetools used in your edk2 repo will be the local repo you cloned. Once you're done, simply do a pip install of your pip_requirements file and the symlinks will be overwritten and things will go back to the way they were before.

Hopefully that addresses your concerns.

Thanks,
- Matthew Carlson

[-- Attachment #2: Type: text/html, Size: 3432 bytes --]