From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail02.groups.io (mail02.groups.io [66.175.222.108]) by spool.mail.gandi.net (Postfix) with ESMTPS id B6B8174003B for ; Thu, 29 Feb 2024 20:08:55 +0000 (UTC) DKIM-Signature: a=rsa-sha256; bh=cuzhBd8m66t3icpzYXXzhupq078J0swcMhqMAZvDHLk=; c=relaxed/simple; d=groups.io; h=Message-ID:Date:MIME-Version:User-Agent:Subject:To:Cc:References:From:Autocrypt:In-Reply-To:Precedence:List-Subscribe:List-Help:Sender:List-Id:Mailing-List:Delivered-To:Reply-To:List-Unsubscribe-Post:List-Unsubscribe:Content-Language:Content-Type:Content-Transfer-Encoding; s=20140610; t=1709237334; v=1; b=IlvLQwxb0vHVUHvR6go3TAK4wY5HmUOjb4LmpJcj/Oq8i+iPCsvq5UYlVQJzfKw44nDpHu77 xSQLKS2d+2q5Jwk50zCeQdpGel5jWI54gB1tlHRu/XfC5Ggcz62SN2zvdWW0/mkrlvc6vSffIqf tOwvpWBC1ivIVsB1QV1wGkio= X-Received: by 127.0.0.2 with SMTP id HuWHYY7687511xGFzmNKWQdv; Thu, 29 Feb 2024 12:08:54 -0800 X-Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.groups.io with SMTP id smtpd.web11.5330.1709237333588547677 for ; Thu, 29 Feb 2024 12:08:53 -0800 X-Received: from mail-ej1-f69.google.com (mail-ej1-f69.google.com [209.85.218.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-12-Zn2ykSSQMqSvF87ef9rPuA-1; Thu, 29 Feb 2024 15:08:51 -0500 X-MC-Unique: Zn2ykSSQMqSvF87ef9rPuA-1 X-Received: by mail-ej1-f69.google.com with SMTP id a640c23a62f3a-a44460e6c06so51181666b.1 for ; Thu, 29 Feb 2024 12:08:51 -0800 (PST) X-Forwarded-Encrypted: i=1; AJvYcCWMYmQzYb8BChsr6VTIBpb+REOatCrFMhHs9mYr6bU61F7jqqd8wjrp9mO9UPUj9a2lc2MbYIFo3J12umhNvAX2AdA/Ew== X-Gm-Message-State: I3OPcOY1VKOLQd3YCULFjuvdx7686176AA= X-Received: by 2002:a17:906:6cf:b0:a43:d6a9:ced8 with SMTP id v15-20020a17090606cf00b00a43d6a9ced8mr6294ejb.56.1709237330347; Thu, 29 Feb 2024 12:08:50 -0800 (PST) X-Google-Smtp-Source: AGHT+IGTbIuypKyoC6iPxz7XxUpu6ERDdT4iNoQRtFZJ0xJiRpQvC8guoCYnHy8qb3vSxpT4MgZlGA== X-Received: by 2002:a17:906:6cf:b0:a43:d6a9:ced8 with SMTP id v15-20020a17090606cf00b00a43d6a9ced8mr6273ejb.56.1709237329950; Thu, 29 Feb 2024 12:08:49 -0800 (PST) X-Received: from ?IPV6:2001:b07:6468:f312:5e2c:eb9a:a8b6:fd3e? ([2001:b07:6468:f312:5e2c:eb9a:a8b6:fd3e]) by smtp.googlemail.com with ESMTPSA id hw16-20020a170907a0d000b00a3ed5479b8bsm994344ejc.21.2024.02.29.12.08.49 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 29 Feb 2024 12:08:49 -0800 (PST) Message-ID: <77bbc006-7a5d-478a-9ba5-398c8db1699c@redhat.com> Date: Thu, 29 Feb 2024 21:08:48 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [edk2-devel] [PATCH 2/2] MdeModulePkg/DxeCore: Fix stack overflow issue due to nested interrupts To: "Kinney, Michael D" , "Ni, Ray" , "devel@edk2.groups.io" Cc: Liming Gao , Laszlo Ersek , Michael Brown References: <20240229130246.3-1-ray.ni@intel.com> <20240229130246.3-3-ray.ni@intel.com> <6ce2acb8-08a2-417c-9f4b-5f96befb412c@redhat.com> From: "Paolo Bonzini" Autocrypt: addr=pbonzini@redhat.com; keydata= xsEhBFRCcBIBDqDGsz4K0zZun3jh+U6Z9wNGLKQ0kSFyjN38gMqU1SfP+TUNQepFHb/Gc0E2 CxXPkIBTvYY+ZPkoTh5xF9oS1jqI8iRLzouzF8yXs3QjQIZ2SfuCxSVwlV65jotcjD2FTN04 hVopm9llFijNZpVIOGUTqzM4U55sdsCcZUluWM6x4HSOdw5F5Utxfp1wOjD/v92Lrax0hjiX DResHSt48q+8FrZzY+AUbkUS+Jm34qjswdrgsC5uxeVcLkBgWLmov2kMaMROT0YmFY6A3m1S P/kXmHDXxhe23gKb3dgwxUTpENDBGcfEzrzilWueOeUWiOcWuFOed/C3SyijBx3Av/lbCsHU Vx6pMycNTdzU1BuAroB+Y3mNEuW56Yd44jlInzG2UOwt9XjjdKkJZ1g0P9dwptwLEgTEd3Fo UdhAQyRXGYO8oROiuh+RZ1lXp6AQ4ZjoyH8WLfTLf5g1EKCTc4C1sy1vQSdzIRu3rBIjAvnC tGZADei1IExLqB3uzXKzZ1BZ+Z8hnt2og9hb7H0y8diYfEk2w3R7wEr+Ehk5NQsT2MPI2QBd wEv1/Aj1DgUHZAHzG1QN9S8wNWQ6K9DqHZTBnI1hUlkp22zCSHK/6FwUCuYp1zcAEQEAAc0j UGFvbG8gQm9uemluaSA8cGJvbnppbmlAcmVkaGF0LmNvbT7CwU0EEwECACMFAlRCcBICGwMH CwkIBwMCAQYVCAIJCgsEFgIDAQIeAQIXgAAKCRB+FRAMzTZpsbceDp9IIN6BIA0Ol7MoB15E 11kRz/ewzryFY54tQlMnd4xxfH8MTQ/mm9I482YoSwPMdcWFAKnUX6Yo30tbLiNB8hzaHeRj jx12K+ptqYbg+cevgOtbLAlL9kNgLLcsGqC2829jBCUTVeMSZDrzS97ole/YEez2qFpPnTV0 VrRWClWVfYh+JfzpXmgyhbkuwUxNFk421s4Ajp3d8nPPFUGgBG5HOxzkAm7xb1cjAuJ+oi/K CHfkuN+fLZl/u3E/fw7vvOESApLU5o0icVXeakfSz0LsygEnekDbxPnE5af/9FEkXJD5EoYG SEahaEtgNrR4qsyxyAGYgZlS70vkSSYJ+iT2rrwEiDlo31MzRo6Ba2FfHBSJ7lcYdPT7bbk9 AO3hlNMhNdUhoQv7M5HsnqZ6unvSHOKmReNaS9egAGdRN0/GPDWr9wroyJ65ZNQsHl9nXBqE AukZNr5oJO5vxrYiAuuTSd6UI/xFkjtkzltG3mw5ao2bBpk/V/YuePrJsnPFHG7NhizrxttB nTuOSCMo45pfHQ+XYd5K1+Cv/NzZFNWscm5htJ0HznY+oOsZvHTyGz3v91pn51dkRYN0otqr bQ4tlFFuVjArBZcapSIe6NV8C4cEiSTOwE0EVEJx7gEIAMeHcVzuv2bp9HlWDp6+RkZe+vtl KwAHplb/WH59j2wyG8V6i33+6MlSSJMOFnYUCCL77bucx9uImI5nX24PIlqT+zasVEEVGSRF m8dgkcJDB7Tps0IkNrUi4yof3B3shR+vMY3i3Ip0e41zKx0CvlAhMOo6otaHmcxr35sWq1Jk tLkbn3wG+fPQCVudJJECvVQ//UAthSSEklA50QtD2sBkmQ14ZryEyTHQ+E42K3j2IUmOLriF dNr9NvE1QGmGyIcbw2NIVEBOK/GWxkS5+dmxM2iD4Jdaf2nSn3jlHjEXoPwpMs0KZsgdU0pP JQzMUMwmB1wM8JxovFlPYrhNT9MAEQEAAcLBMwQYAQIACQUCVEJx7gIbDAAKCRB+FRAMzTZp sadRDqCctLmYICZu4GSnie4lKXl+HqlLanpVMOoFNnWs9oRP47MbE2wv8OaYh5pNR9VVgyhD OG0AU7oidG36OeUlrFDTfnPYYSF/mPCxHttosyt8O5kabxnIPv2URuAxDByz+iVbL+RjKaGM GDph56ZTswlx75nZVtIukqzLAQ5fa8OALSGum0cFi4ptZUOhDNz1onz61klD6z3MODi0sBZN Aj6guB2L/+2ZwElZEeRBERRd/uommlYuToAXfNRdUwrwl9gRMiA0WSyTb190zneRRDfpSK5d usXnM/O+kr3Dm+Ui+UioPf6wgbn3T0o6I5BhVhs4h4hWmIW7iNhPjX1iybXfmb1gAFfjtHfL xRUr64svXpyfJMScIQtBAm0ihWPltXkyITA92ngCmPdHa6M1hMh4RDX+Jf1fiWubzp1voAg0 JBrdmNZSQDz0iKmSrx8xkoXYfA3bgtFN8WJH2xgFL28XnqY4M6dLhJwV3z08tPSRqYFm4NMP dRsn0/7oymhneL8RthIvjDDQ5ktUjMe8LtHr70OZE/TT88qvEdhiIVUogHdo4qBrk41+gGQh b906Dudw5YhTJFU3nC6bbF2nrLlB4C/XSiH76ZvqzV0Z/cAMBo5NF/w= In-Reply-To: X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Precedence: Bulk List-Subscribe: List-Help: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,pbonzini@redhat.com List-Unsubscribe-Post: List-Unsubscribe=One-Click List-Unsubscribe: Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable X-GND-Status: LEGIT Authentication-Results: spool.mail.gandi.net; dkim=pass header.d=groups.io header.s=20140610 header.b=IlvLQwxb; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=redhat.com (policy=none); spf=pass (spool.mail.gandi.net: domain of bounce@groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce@groups.io On 2/29/24 20:16, Kinney, Michael D wrote: >=20 >=20 >> -----Original Message----- >> From: Paolo Bonzini >> Sent: Thursday, February 29, 2024 11:04 AM >> To: Ni, Ray ; devel@edk2.groups.io >> Cc: Kinney, Michael D ; Liming Gao >> ; Laszlo Ersek ; Michael >> Brown >> Subject: Re: [PATCH 2/2] MdeModulePkg/DxeCore: Fix stack overflow issue >> due to nested interrupts >> >> On 2/29/24 14:02, Ray Ni wrote: >>> In the end, it will lower the TPL to TPL_APPLICATION with interrupt >> enabled. >>> >>> However, it's possible that another timer interrupt happens just in >> the end >>> of RestoreTPL() function when TPL is TPL_APPLICATION. >> >> How do non-OVMF platforms solve the issue? Do they just have the same >> bug as in https://bugzilla.tianocore.org/show_bug.cgi?id=3D4162 ? >=20 > Yes. This same issue can be reproduced on non-OVMF platforms. >=20 > This proposal here is an attempt to integrate a common fix into the DXE C= ore. >=20 > I would agree conceptually that integrating the NestedInterruptTplLib wor= k > into the DXE Core is another option. >=20 > I believe the root cause of all of these scenarios is enabling interrupts > in RestoreTPL() when processing a timer interrupt between the last proces= sed > event and the return from the interrupt handler. Ther are some instances > of the Timer Arch Protocol implementation that call Raise/Restore TPL, so > we want a DXE Core change that is compatible with the DXE Core doing Rais= e/Restore > when processing a timer interrupt and the Timer Arch Protocol implementat= ion > also doing the Raise/Restore TPL. Ok, now I understand better. The reason why the NestedInterruptTplLib was introduced (as opposed to=20 doing it in core DXE) was to enable returning with disabled interrupts=20 from the nested interrupt handler, but I think it can be done with a=20 function like the CoreRestoreTplInternal() I outlined in the previous=20 email, which is the same as current CoreRestoreTpl() but finishes with if (!DesiredInterruptState) { CoreSetInterruptState (FALSE); } gEfiCurrentTpl =3D NewTpl; if (DesiredInterruptState) { ASSERT (gEfiCurrentTpl < TPL_HIGH_LEVEL); CoreSetInterruptState (TRUE); } The new CoreRaiseTpl would be the same as in Ray and your patch, while=20 the CoreRestoreTpl would be something like this: if (NewTpl =3D=3D HighBitSet64 (mInterruptedTplMask)) { static NESTED_INTERRUPT_STATE NestedInterruptState; mInterruptedTplMask &=3D ~(UINTN)(1 << NewTpl); // // Use the deferred invocation logic that is currently // in NestedInterruptTplLib. // // But unlike current NestedInterruptRestoreTPL(), if the logic // is part of core DXE, the // // gBS->RestoreTPL (InterruptedTPL); // DisableInterrupts (); // // pair that requires "disable interrupts on IRET" logic can // be done without ever enabling interrupts, with // CoreRestoreTplInternal(InterruptedTPL, FALSE) // // As an aside, NestedInterruptState might as well become a // pair of globals. // NestedInterruptRestoreTPL (NewTpl, &NestedInterruptState); } else { CoreRestoreTplInternal(NewTpl, NewTpl < TPL_HIGH_LEVEL); } Requiring matching raise/restore pairs is a bit scary. It can be=20 avoided by changing the "if" to a while (NewTpl >=3D HighBitSet64 (mInterruptedTplMask)) mInterruptedTplMask &=3D ~(UINTN)(1 << HighBitSet64 (mInterruptedTplMask)); Then, if inlining NestedInterruptRestoreTPL() allows simplifications,=20 they can be done on top after the merge of NestedInterruptTplLib. In=20 particular, I suspect that the while loop above can be unified with the=20 loop in NestedInterruptRestoreTPL(). But again, that would be best=20 reviewed as a separate change. All this, as Michael said, is however conditional on being able to deal=20 with the TPL_HIGH_LEVEL+STI shenanigans that Windows does. Paolo >> >> The design of NestedInterruptTplLib is that each nested interrupt must >> increase the TPL, but if I understand correctly there is a hole here: >> >> // >> // Call RestoreTPL() to allow event notifications to be >> // dispatched. This will implicitly re-enable interrupts. >> // >> gBS->RestoreTPL (InterruptedTPL); >> >> // >> // Re-disable interrupts after the call to RestoreTPL() to ensure >> // that we have exclusive access to the shared state. >> // >> DisableInterrupts (); >> >> because gBS->RestoreTPL will unconditionally enable interrupts if >> InterruptedTPL < TPL_HIGH_LEVEL. >> >> >> If possible, the easiest solution would be to merge >> NestedInterruptTplLib into Core DXE. This way, instead of calling >> gBS->RestoreTPL, NestedInterruptTplLib can call a custom version of >> CoreRestoreTpl that exits with interrupts disabled. That is, something >> like >> >> VOID EFIAPI CoreRestoreTplInternal(IN EFI_TPL NewTpl, >> IN BOOLEAN InterruptState) >> { >> // >> // The caller can request disabled interrupts to access shared >> // state, but TPL_HIGH_LEVEL must *not* have them enabled. >> // >> ASSERT(!(NewTpl =3D=3D TPL_HIGH_LEVEL && InterruptState)); >> >> // ... >> >> gEfiCurrentTpl =3D NewTpl; >> CoreSetInterruptState (InterruptState); >> } >> >> Now, CoreRestoreTpl is just >> >> // >> // If lowering below HIGH_LEVEL, make sure >> // interrupts are enabled >> // >> CoreRestoreTplInternal(NewTpl, NewTpl < TPL_HIGH_LEVEL); >> >> whereas NestedInterruptRestoreTPL can do >> >> // >> // Call RestoreTPL() to allow event notifications to be >> // dispatched. This will implicitly re-enable interrupts, >> // but only if events have to be dispatched. >> // >> CoreRestoreTplInternal(InterruptedTPL, FALSE); >> >> // >> // Interrupts are now disabled, so we can access shared state. >> // >> >> This avoids the unlimited nesting of interrupts because each stack >> frame >> will indeed have a higher TPL than the outer version. >> >> Paolo >=20 -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#116189): https://edk2.groups.io/g/devel/message/116189 Mute This Topic: https://groups.io/mt/104642317/7686176 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-