From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail02.groups.io (mail02.groups.io [66.175.222.108]) by spool.mail.gandi.net (Postfix) with ESMTPS id 7D9E67803CC for ; Tue, 13 Feb 2024 19:28:11 +0000 (UTC) DKIM-Signature: a=rsa-sha256; bh=NvQJd0Oahlz4pqA+FhewoKfCxeT8M8tvcborOOy2Y10=; c=relaxed/simple; d=groups.io; h=Feedback-ID:Message-ID:Date:MIME-Version:User-Agent:Subject:To:References:From:In-Reply-To:Precedence:List-Subscribe:List-Help:Sender:List-Id:Mailing-List:Delivered-To:Reply-To:List-Unsubscribe-Post:List-Unsubscribe:Content-Language:Content-Type:Content-Transfer-Encoding; s=20140610; t=1707852490; v=1; b=tJZSflCS5vBJjZQnjb0Ry2mv1UkAmj55Y0YVGnNXbkb50vUT9Lmlf5UFuQUXuBnhKbSq0HhZ zakM7M2uJ1t1rS0VRpAMPXFZNUaAIglnqzKBHsgU9W/qJ1pLDbUVCHUUr0asRSroChQqVBINeQC EZSuSFpw522+34u/YcXAkbnc= X-Received: by 127.0.0.2 with SMTP id lxQTYY7687511xAQljzXSTHY; Tue, 13 Feb 2024 11:28:10 -0800 X-Received: from fout7-smtp.messagingengine.com (fout7-smtp.messagingengine.com [103.168.172.150]) by mx.groups.io with SMTP id smtpd.web11.22690.1707852489463239787 for ; Tue, 13 Feb 2024 11:28:09 -0800 X-Received: from compute7.internal (compute7.nyi.internal [10.202.2.48]) by mailfout.nyi.internal (Postfix) with ESMTP id A1C9013800C8; Tue, 13 Feb 2024 14:28:08 -0500 (EST) X-Received: from mailfrontend1 ([10.202.2.162]) by compute7.internal (MEProxy); Tue, 13 Feb 2024 14:28:08 -0500 X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrudehgdduvdegucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepkfffgggfuffvfhfhjggtgfesthejredttddvjeenucfhrhhomheptfgvsggv tggtrgcuvehrrghnuceorhgvsggvtggtrgessghsughiohdrtghomheqnecuggftrfgrth htvghrnhepjeehtdeuudefjedvuddvueehtedvheevudduhfdtgfejfefggeffkefhffej lefhnecuffhomhgrihhnpehgihhthhhusgdrtghomhenucevlhhushhtvghrufhiiigvpe dtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehrvggsvggttggrsegsshguihhordgtohhm X-ME-Proxy: Feedback-ID: i5b994698:Fastmail X-Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 13 Feb 2024 14:28:07 -0500 (EST) Message-ID: Date: Tue, 13 Feb 2024 12:27:58 -0700 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [edk2-devel] Fixing edk2-basetools CI To: Joey Vagedes , Rebecca Cran , "devel@edk2.groups.io" , "Kinney, Michael D" , Sean , Michael Kubacki References: <2fe83108-e19b-4c95-8c45-08f43594e8cd@bsdio.com> <2c52fbac-dbe1-423a-a9d7-0bbe473cd9e9@os.amperecomputing.com> From: "Rebecca Cran" In-Reply-To: Precedence: Bulk List-Subscribe: List-Help: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,rebecca@bsdio.com List-Unsubscribe-Post: List-Unsubscribe=One-Click List-Unsubscribe: X-Gm-Message-State: Odp7AQpxDKxmInkWScJSwhKFx7686176AA= Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable X-GND-Status: LEGIT Authentication-Results: spool.mail.gandi.net; dkim=pass header.d=groups.io header.s=20140610 header.b=tJZSflCS; dmarc=none; spf=pass (spool.mail.gandi.net: domain of bounce@groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce@groups.io Thanks, I've updated=20 https://github.com/tianocore/edk2-basetools/pull/116 with the changes=20 you suggested. Unfortunately several of the tools don't work with the projects.scripts=20 section because they don't have Main etc. It turns out even Ecc is=20 currently broken in the edk2-basetools PyPI package because PYTHONPATH=20 doesn't contain the parent directory. I'll plan to fix the issues in the=20 near future, once I've worked through the other PRs. --=20 Rebecca Cran On 2/12/24 10:06, Joey Vagedes wrote: > Hello! Regarding flake8, I see. That will take some time to work through = - I completely understand. > > When ready to work through that, I suggest switching to ruff. It can reso= lve some errors automatically for you and allows you to customize it more t= han flake8 (it is also faster). > > I just ran ruff on the project, showing about 11k errors. One nice thing = is that a large majority of it is regarding importing using *. By fixing th= ose, you can reduce the errors by almost 70% (to 3400 errors), so it is not= *quite* as bad as it seems (though 3400 is still A LOT). > > -------------------------------------------------------------------------= -------------------------------------------------- > > Regarding the setup tools question, the build-backend part of pytoml (L3)= is the entry point that is invoked - so we are still invoking setuptools. = However, it pulls the necessary metadata from pyproject.toml rather than se= tup.py's setup() function. The main thing that is different is the "get_ver= sion" functionality, which we replicate in the pyproject.toml with setuptoo= ls-scm[toml] which simply uses the latest tag to set the version. > > -------------------------------------------------------------------------= -------------------------------------------------- > > Additionally, I wanted to comment on the wrappers that you mentioned. > > Using the [projects.scripts] section (i.e. CLI Commands feature) is in my= opinion the correct way to move forward as users will be able to invoke co= mmands themselves easily (i.e. just call Bin2Pcd, and it will work). Howeve= r, the wrappers will continue to be necessary until all python source is re= moved from Basetools, so that users can toggle between pip or local. Howeve= r, once source is removed form BaseTools, removing the wrappers is the next= step forward. > > In the meantime, you will be able to update the binpipwrappers to somethi= ng akin to the following (untested, so I don't garuntee it will be exactly = this: > > @setlocal > Ecc %* > > Rather than > > @setlocal > @set ToolName=3D%~n0% > @%PYTHON_COMMAND% -m edk2basetools.%ToolName%.EccMain %* > > -------------------------------------------------------------------------= -------------------------------------------------- > > One thing to keep in mind once Python is completely removed from BaseTool= s, is that consumers will need to get in the habit of verifying the correct= version of basetools is installed for the commit they are on (or automate = this checking) particularly for anyone that regularly switches between stab= le branches. > > Thanks, > Joey > > -----Original Message----- > From: Rebecca Cran > Sent: Monday, February 12, 2024 8:42 AM > To: devel@edk2.groups.io; Joey Vagedes ; Rebec= ca Cran ; Kinney, Michael D = ; Sean ; Michael Kubacki > Subject: [EXTERNAL] Re: [edk2-devel] Fixing edk2-basetools CI > > [You don't often get email from rebecca@os.amperecomputing.com. Learn why= this is important at https://aka.ms/LearnAboutSenderIdentification ] > > On 2/12/2024 9:08 AM, Joey Vagedes via groups.io wrote: >> Hello. It can be simplified - You can view the pyproject.toml for edk2-p= ytool-library and edk2-pytool-extensions: >> >> https://nam06.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fgith >> ub.com%2Ftianocore%2Fedk2-pytool-library%2Fblob%2Fmaster%2Fpyproject.t >> oml&data=3D05%7C02%7Cjoeyvagedes%40microsoft.com%7Ca37eecc569d54b5dbbf30 >> 8dc2be996f3%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6384335294831 >> 52477%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJB >> TiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=3D8RSQNcjgH02JyPiGCKmyf2u7 >> Ep6BC9ldwiND0ULj9Zs%3D&reserved=3D0 >> >> Also, Why disable flake8 rather than fix the changes? I also suggest swi= tching to ruff, as it does the same thing but better, and with way more rul= es / configuration options. > Thanks. I'm not sure that looks so much simpler than my pyproject.toml bu= t I'll update it to follow the same layout. Also, pyproject.toml says it's = using the setuptools build backend but you don't have a setup.py: > how does that work? > > I decided to disable flake8 for now because the BaseTools Python code is = very non-compliant: the log file has over 41,000 lines. In most places it l= ooks like people have been trying to write it as C code. > > A few of the lines: > > .\edk2basetools\Common\TargetTxtClassObject.py:44:80: E203 whitespace bef= ore ':' > .\edk2basetools\Common\TargetTxtClassObject.py:61:14: E111 indentation is= not a multiple of 4 > .\edk2basetools\Common\TargetTxtClassObject.py:61:14: E117 over-indented > .\edk2basetools\Common\TargetTxtClassObject.py:63:50: F405 'FILE_NOT_FOUN= D' may be undefined, or defined from star imports: > .BuildToolError > .\edk2basetools\Common\TargetTxtClassObject.py:73:121: E501 line too long= (123 > 120 characters) > .\edk2basetools\Common\TargetTxtClassObject.py:84:38: F405 'FILE_OPEN_FAI= LURE' may be undefined, or defined from star imports: > .BuildToolError > .\edk2basetools\Common\TargetTxtClassObject.py:100:108: E502 the backslas= h is redundant between brackets > .\edk2basetools\Common\TargetTxtClassObject.py:108:121: E501 line too lon= g (139 > 120 characters) > .\edk2basetools\Common\TargetTxtClassObject.py:118:121: E501 line too lon= g (140 > 120 characters) > .\edk2basetools\Common\TargetTxtClassObject.py:123:97: E502 the backslash= is redundant between brackets > .\edk2basetools\Common\TargetTxtClassObject.py:128:21: F841 local variabl= e 'V' is assigned to but never used > > > -- > Rebecca Cran -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#115405): https://edk2.groups.io/g/devel/message/115405 Mute This Topic: https://groups.io/mt/104306226/7686176 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-