From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from us-smtp-1.mimecast.com (us-smtp-1.mimecast.com [207.211.31.120]) by mx.groups.io with SMTP id smtpd.web11.4368.1583883296985305340 for ; Tue, 10 Mar 2020 16:34:57 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=gdPjtKeR; spf=pass (domain: redhat.com, ip: 207.211.31.120, mailfrom: lersek@redhat.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1583883296; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UjND/QsJBb67VaUl27jQIQN0ub6AB5XlMnBkR/LQL5c=; b=gdPjtKeR3Wn391qIpS2coK/Cf6gXKP2ZpnTpUbKccLxrjKYbNCwgKxwYhyk9Ll5RvDYRzk nSIbHe0tPqs7AJmz/Uj3OBq0l9KHxOxyjEGMM+uNcnaDztqlVU+ycUCWVVaRHe9YTdJniq CIuv2XOcWO5K8oVhDykdPGV0EIVJ2n4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-377-TWdav4FHMPe61geEQIY4Qg-1; Tue, 10 Mar 2020 19:34:54 -0400 X-MC-Unique: TWdav4FHMPe61geEQIY4Qg-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 17E59184C810; Tue, 10 Mar 2020 23:34:53 +0000 (UTC) Received: from lacos-laptop-7.usersys.redhat.com (ovpn-116-246.ams2.redhat.com [10.36.116.246]) by smtp.corp.redhat.com (Postfix) with ESMTP id 42C865D9C5; Tue, 10 Mar 2020 23:34:52 +0000 (UTC) Subject: Re: [edk2-devel] Adding Bhyve support into upstream EDK2 To: Sean , devel@edk2.groups.io References: <6ef172e5-4999-f1bc-0d08-43685b03f5ce@redhat.com> <21546.1583861100642932695@groups.io> From: "Laszlo Ersek" Message-ID: Date: Wed, 11 Mar 2020 00:34:51 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: <21546.1583861100642932695@groups.io> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 03/10/20 18:25, sean.brogan via [] wrote: > I don't see the difference besides the mechanics of the operation (which > you have described clearly).=C2=A0 To guarantee a repo or repos is > "git-bisectable" you need to build and test every commit on your > platform.=C2=A0 For example in the recent ArmMmuLib patchset, you were ab= le > to build every commit in the patch to identify which one caused the > break.=C2=A0 There isn't an enforced process in place to ensure that happ= ens > within Edk2.=C2=A0 Thankfully the review process and the developers knowl= edge > allowed the commits to be made in such a way that this was possible.=C2= =A0 > That doesn't have to change when you move to a submodule.=C2=A0 Also you > could put automation in place to enforce and/or test for this scenario.= =C2=A0 > You can put automation in place to "integrate" into your super project > at every commit if you really wanted to and had the resources to run > tests on every one of those commits.=C2=A0 Is this type of CI done today = for > OVMF?=C2=A0=C2=A0 Consider a patch series that needs to modify at least one core module and at least one platform, in order to reach a desired result. Such patch series are not infrequent: one of my recent contributions (commit range 61d3b2d4279e..1158fc8e2c7b) modified MdeModulePkg, UefiCpuPkg, and OvmfPkg. When I work on a patch series (any patch series, really) I not only test that the work builds at every stage, but also that it *functions* at every stage (in other words that no stage in the series regresses existing functionality, and that it indeed implements the desired step of the feature or bugfix). This is done specifically so that, in case I miss a use case completely, and regress it, another developer can return to any stage across my work (regardless of "core module" vs "platform code" distinction), via bisection, and pinpoint my mistake. So: this kind of CI is not being done (it's not enforced), but I take it seriously. If we split platforms to different repos from the core, I can't even propose a patch set, in the first place, that modifies both kinds of modules in one logical go. >=20 > Again this is what nearly all platforms have to do today and we have a > lot of experience with bisecting within the submodule to find the > error.=C2=A0 The longer you wait between integrations the more costly the > bisect is if you have to do it, but this is a choice of the super > project owner / platform owner.=C2=A0 Today I assume you make those choic= es > too, they just happen to be within the same repo.=C2=A0 I also assume tha= t if > you found the MmuLib bug in a few days you probably wouldn't bisect all > the changes but you would review the history to intelligently guess at > the most likely candidates and bisect within those commits. My thinking is the inverse. Bisection cost grows logarithmically -- covering a range of 256 commits takes just one build more than covering a range of 128 commits. To me, bisection comes first, exactly so I don't have to think (or intelligently guess), before the issue is narrowed down to a single patch. I have seen such a huge amount of incredible / stunning bisection results ("I would have never thought of this!") that I don't even try to guess, any more. Regarding the six patches that I bisected a few days ago -- those hadn't even been upstreamed, I was preparing to merge them, for Ard. I collected them from three separate series (posted & reviewed on the list), in the logical order explained by the cover letters. They were super quirky and over my head (aarch64 MMU programming? thanks but no thanks); I would have had zero chance to guess which one caused the crash with "grubaa64.efi". The bisection allowed me at least to tell Ard which one of his patches looked risky. ... I've got almost entirely mechanical Linux kernel bisections behind me that took 5-8 hours and pinpointed very unexpected culprit patches, from multiple thousands of commits between releases. I could have never tracked those bisections myself, if I had had to dig into submodules manually. I understand that I'm appearing too stubborn. I promise it's not just for the sake of arguing. Thanks, Laszlo >=20 > In the end I just don't see the big difference to the platform (OVMF in > this case) but I do see the reduced size/noise/content helping all > platforms.=C2=A0 Success still relies on good development practices, regu= lar > builds, and testing.=C2=A0 =C2=A0=C2=A0 >=20 > Thanks > Sean >=20 > =C2=A0=C2=A0