From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: redhat.com, ip: 209.132.183.28, mailfrom: lersek@redhat.com) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by groups.io with SMTP; Fri, 03 May 2019 01:45:38 -0700 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 775922C976E; Fri, 3 May 2019 08:45:37 +0000 (UTC) Received: from lacos-laptop-7.usersys.redhat.com (ovpn-120-45.rdu2.redhat.com [10.10.120.45]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5522DF6E6; Fri, 3 May 2019 08:45:36 +0000 (UTC) Subject: Re: [edk2-devel] TianoCore Community Design Meeting Minutes To: devel@edk2.groups.io, sean.brogan@microsoft.com, "Brian J. Johnson" References: <70d2f499-bc28-058a-8675-069beee5835e@hpe.com> <31264.1556825609503060272@groups.io> From: "Laszlo Ersek" Message-ID: <461d9e97-729d-cee9-fa98-79c515f7727d@redhat.com> Date: Fri, 3 May 2019 10:45:35 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: <31264.1556825609503060272@groups.io> X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Fri, 03 May 2019 08:45:37 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit On 05/02/19 21:33, Sean via Groups.Io wrote: > Laszlo, (please add me to Cc: or To: directly when the email is directed at me (as well); I could have easily missed this message, as it is only in my list folder) > Except for a very few platforms that are in the current edk2 repo > today, everyone building products has to deal with this "split repo" > reality. It gets even harder when you account for different business > units, different silicon partners, IBVs, ODMs, open source, closed > source, secret projects, and so on. I don't contest the problem statement. We can look at what other high-profile open source projects do with this problem. For one example, the Linux kernel says, if it's not upstream, you're on your own. And Linux carries loads of platform code. I'm making an effort not to take such a hard-liner stance. We need to find a balance as to how much we want to harm the *truly* open source platforms, for the sake of proprietary downstreams of edk2. My understanding is that the maintainers of edk2-platforms (the open source ones anyway) would prefer an opposite code movement; i.e. integrate everything in one upstream repository, for better bisectability & tracking. > The reason I want to bring forth the Project Mu tools to Edk2 is that > everyone is forced to reinvent this process and tooling every time and > it doesn't work very well. If edk2 continues to ignore the challenge > of most downstream consumers it continues to lead to the results of > today. Very high cost of integration. Very low quality of > integrations and very few integrations into products (meaning products > don't get updates). The last couple of years has brought significant > light to the challenges of updating platform firmware and has shown > that the majority of products don't/can't get updates resulting in > customers not being fully protected. > > Regarding submodules and boundaries. I completely agree except that I > believe there are numerous boundaries within a UEFI code base. As > mentioned above one of our goals with splitting the code repositories > is to have all code within a repository owned/supported by a single > entity. This is 100% counter to Open Source / Open Development / Distributed Development. I didn't invent the idea that edk2 *should* follow Open Development, i.e. that it should be a good Open Source citizen. That decision had not been made by me, it was in place when I started contributing. I'm just saying that the direction you describe is *incompatible* with Open Source / Open Development. A real open source project is community-oriented, and the comminuty has shared ownership, as a whole. If tomorrow we decide that FooPkg is now officially owned/supported by the "single entity" called Company Bar (who happen to be a competitor to Company Baz), will help Baz's contributions to FooPkg? What you describe is a real problem of course. The open development answer is that, if one prefers to keep things downstream, then the rebase and integration burden stays with them, downstream, as well. Maybe edk2 doesn't *want* to practice Open Development that much. If we adopted "single entity ownership per repo", would that stay aligned with the current mission statement? https://www.tianocore.org/ Welcome to TianoCore, the community supporting an open source implementation of the Unified Extensible Firmware Interface (UEFI). EDK II is a modern, feature-rich, cross-platform firmware development environment for the UEFI and UEFI Platform Initialization (PI) specifications. [...] > Another point to splitting is attempting to get code with business > logic separated from core/common code. Business logic often is > different across products and if intermixed with core logic it adds > significantly to the cost of maintaining the product. Along your same > thinking these different repositories do have different development > models. Many are closed source and have proprietary change request > process. They all release on different cadences, different > dependencies and very different testing expectations so without a > framework that provides some support this leads to challenging and > complicated integration processes. I certainly don't intend to dismiss these use cases. In my opinion, the "upstream first" principle would help here too. Basically, don't ship anything until at least the critical parts of it are merged upstream. But, I don't want to get accused of being out of touch with reality. I guess I'll have to defer to others with more experience in such environments, and I should comment on specific edk2 proposals then. > Single repo: > > It is not possible for most products. Again when integrating large > amounts of code from numerous places all with different restrictions > it is not practical to have a single bisectable repository with good > history tracking. Some entities still deliver by zip files with > absolutely no src control history. Wouldn't you say that that is *their* problem? Absolutely deplorable source code management? Should we really destroy bisectability in upstream edk2 for their sake, especially given that you can always extract sub-repositories (mechanically, at that) from a unified repo, but not the other way around? > Many entities mix open and closed source code and make > hundreds/thousands of in-line changes. I just don't see a path where > a product can have 1 git-merged repo and still be able to efficiently > integrate from their upstream sources and track history. Normally this is why someone tries to push as much code upstream as possible; then downstream rebases are manageable. > These tools are just a first step down a path to reshaping tianocore > edk2 to be easier to consume (and update) by the ecosystem that > depends on Edk2 for products. Put differently, the first step to abandon the Open Development model, for the sake of proprietary downstreams. But, I can see myself becoming non-constructive here. I guess I should withdraw from the general discussion, and comment on specific proposals for edk2, when they appear. (Speaking for Red Hat in this sentence: we certainly depend on edk2 for products.) > These tools also have solutions for Ci builds, binary dependencies, > plugins, testing, and other features that edk2 will need for some of > the practical next steps. A side question here: what development model would you follow with these tools themselves? (Because... the "willingness" we've witnessed, for extending the email notifications sent by GitHub.com, is not reassuring.) Laszlo