From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-qt1-f172.google.com (mail-qt1-f172.google.com [209.85.160.172]) by mx.groups.io with SMTP id smtpd.web10.287.1614879924118670939 for ; Thu, 04 Mar 2021 09:45:24 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@nuviainc-com.20150623.gappssmtp.com header.s=20150623 header.b=DjLz7I/Y; spf=pass (domain: nuviainc.com, ip: 209.85.160.172, mailfrom: rebecca@nuviainc.com) Received: by mail-qt1-f172.google.com with SMTP id 18so18415054qty.3 for ; Thu, 04 Mar 2021 09:45:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nuviainc-com.20150623.gappssmtp.com; s=20150623; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=Nr2hHORlFEwI7SJqtGc16Yf3nl/n5zTwpqC6e+ZLn1A=; b=DjLz7I/YxSNv5tu0/XQbj+IuP4D6n476mnY3ugCwb1Qi3K7dJD/ssdljgDY6jd8gBL uegQ9SyuhYtkNXSbqnr7HGaxiq1C8F4ZeEsC2OYg4H6e2mupk25Enaj32s2rWT2Nre30 UOA58ZEKpxcmmkPbDHw3naFjII4mKIPFaAdMTep4mjLKAAfU6T+NsNz3NnEHNHeEMd1n FmoOtP8ZuSJAn8fJrSKtycEhRzwTuhfqwZSgvC7tGgC2rsZtpHB9KADx+K59Uj42ZI0+ XMjOmugs8jcb6T5n2GbXWbeRS0zsL5dFsmrSjD+GZciCmfo8SXzVQeOgA2eWz5+l7VRF y3YA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=Nr2hHORlFEwI7SJqtGc16Yf3nl/n5zTwpqC6e+ZLn1A=; b=qaG0oF8WCQvN7qEKmW9MFPPIeeXxKWyFp+FhZ0G/Y9+g2D78um7M4p8D+TiQGUhfqv 6aa8v4JyOCqgNt7v27GDQXsV5Yj1O1gMVmsEVt5cAcp68t9IhzPyGlI8jCxn26CIBWY2 Kd8OeMJWb7A9cQ1L4Cm1wucWtKIAowyqjqmKkXhwXCu6ogyio7Lt616nZ2fgbksEvLa+ txQvOX7pursg3ViZw82po3NdlKRAzIMieVvqMmEG3f96In0Z4Md1wXipesvwncLnR9MY Brrd5nZNMd/aB6qdmj0nz61ftqlncWZsQUHCyrUZWRcVGJVKkS4L3lGUZYUAVTI14KAb DcVg== X-Gm-Message-State: AOAM530QYJEDH1GNKlbE03281VvH4CBZRGFStIG+t1/Hq9Tx4gHsFPZc q/qfThn28U1g26AaWHqWy1inLw== X-Google-Smtp-Source: ABdhPJw9KK+loN0Wgx7+2Pkd1EPSMJxNdLoXvwQegRw7MDo8Gmy8jt7cP5GEs4+WeFcKrRC21zoH8g== X-Received: by 2002:ac8:4e18:: with SMTP id c24mr4917454qtw.147.1614879923323; Thu, 04 Mar 2021 09:45:23 -0800 (PST) Return-Path: Received: from [10.0.10.142] (c-174-52-16-57.hsd1.ut.comcast.net. [174.52.16.57]) by smtp.gmail.com with ESMTPSA id o30sm19585690qkl.17.2021.03.04.09.45.22 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 04 Mar 2021 09:45:23 -0800 (PST) Subject: Re: [edk2-devel] generating the test matrix for local CI runs To: devel@edk2.groups.io, lersek@redhat.com Cc: Sean Brogan , Bret Barkelew , Michael Kinney References: <95b5738d-6af2-15ea-e69d-759b1599b00d@redhat.com> From: "Rebecca Cran" Message-ID: Date: Thu, 4 Mar 2021 10:45:20 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.7.1 MIME-Version: 1.0 In-Reply-To: <95b5738d-6af2-15ea-e69d-759b1599b00d@redhat.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit On 3/4/21 10:04 AM, Laszlo Ersek wrote: > (Sigh, my address book pulled one over me -- resending to the correct > list address now. Please ignore the previous posting that you may have > gotten "in private".) > > Hi All, > > while I can run specific CI tests locally, using the commands listed at: > > https://github.com/tianocore/edk2/tree/master/.pytool > https://github.com/tianocore/edk2/tree/master/ArmVirtPkg/PlatformCI > https://github.com/tianocore/edk2/tree/master/OvmfPkg/PlatformCI > > I don't know how I can determine and drive the whole "test plan" locally. > > When I submit a PR to github, "something" generates (say) 83 test cases. > How can I run that "something" locally? > > Assume I have the following: > - a Linux builder VM, with up-to-date tools, > - a Windows builder VM, with up-to-date tools, > - a topic branch, on top of "master". > > How do I generate and run those same 83 tests, locally? > > It has happened to me that all my local CI steps succeeded (meaning both > the Windows and the Linux builder), but github still rejected the PR -- > the reason was that I failed to think of a particular test that github > would run. Composing 83 "stuart" command lines isn't something I'd like > to do manually. FWIW it looks like people have been looking for similar functionality of testing a pipeline locally, and not found it. e.g.: https://github.com/microsoft/azure-pipelines-agent/issues/1764 https://developercommunity.visualstudio.com/t/yaml-how-to-test-yaml-locally-before-commit/1302759 -- Rebecca Cran