From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: insyde.com, ip: 210.71.195.44, mailfrom: tim.lewis@insyde.com) Received: from out02.hibox.biz (out02.hibox.biz [210.71.195.44]) by groups.io with SMTP; Wed, 21 Aug 2019 17:47:16 -0700 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: =?us-ascii?q?A2BAAABs5V1d/ws0GKxhAx0BAQUBBwU?= =?us-ascii?q?BgVMIAQsBgRVYKm1yEiqEH4gcHYhVfo17ihqBewkBAQEBAQEBAQEIJQcBAgE?= =?us-ascii?q?BhD8CglwjNAkOAgUBAQUBAQEBAQYEbYRlQgyFSgEBAgMIAhkEBiE4BQYDDQQ?= =?us-ascii?q?EAQEOEwEGAwIbGBUDAQUIAgQBEgsFDQSCNkyBHW0Pqhl/MxoChBgBgRSFEIE?= =?us-ascii?q?0AYFig3WILYERgl01PoJhAgIYgUwDBBUVEYJEglgEjxOGUoc/jjEHAokHjVE?= =?us-ascii?q?bgjFthkOEBQOKXYw2gSWHY5BTgVA4gVhwgzwJCiyCDxeDT4JkhR2CciIwAYE?= =?us-ascii?q?gCBUBjAsBAQ?= X-IronPort-AV: E=Sophos;i="5.64,414,1559491200"; d="scan'208,217";a="16295410" Received: from unknown (HELO hb3-BKT201.hibox.biz) ([172.24.52.11]) by out02.hibox.biz with ESMTP; 22 Aug 2019 08:47:12 +0800 Received: from unknown (HELO hb3-BKT102.hibox.biz) ([172.24.51.12]) by hb3-BKT201.hibox.biz with ESMTP; 22 Aug 2019 08:47:12 +0800 Received: from unknown (HELO hb3-IN01.hibox.biz) ([172.24.12.11]) by hb3-BKT102.hibox.biz with ESMTP; 22 Aug 2019 08:47:12 +0800 X-Remote-IP: 63.117.101.130 X-Remote-Host: No hostname X-SBRS: None X-MID: 26472615 X-Auth-ID: tim.lewis@insyde.com X-EnvelopeFrom: tim.lewis@insyde.com hiBox-Sender: 1 Received: from unknown (HELO DESKTOPHG9V3E8) ([63.117.101.130]) by hb3-IN01.hibox.biz with ESMTP/TLS/AES256-SHA; 22 Aug 2019 08:47:10 +0800 From: "Tim Lewis" To: , , In-Reply-To: Subject: Re: [edk2-devel] EmulatorPkg does not unload DLL after exit Date: Wed, 21 Aug 2019 17:47:07 -0700 Message-ID: <0c2001d55883$232ed710$698c8530$@insyde.com> MIME-Version: 1.0 X-Mailer: Microsoft Outlook 16.0 Thread-index: AQL3RaYZ4yYK4SurS9JVyTHkOKYb5KTCeuYw Content-Type: multipart/alternative; boundary="----=_NextPart_000_0C21_01D55848.76D492F0" Content-language: en-us ------=_NextPart_000_0C21_01D55848.76D492F0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Mike =E2=80=93 Ok, it is bug 2104. =20 From: devel@edk2.groups.io On Behalf Of Michael D K= inney Sent: Wednesday, August 21, 2019 3:08 PM To: devel@edk2.groups.io; afish@apple.com; tim.lewis@insyde.com; Kinney, M= ichael D Subject: Re: [edk2-devel] EmulatorPkg does not unload DLL after exit =20 Tim, =20 Thanks for the analysis. =20 Looks like we need to fix PeCoffLoaderUnloadImageExtraAction() for winhost to unload the DLL to keep compatibility with Nt32. This is im= portant for both apps and unloadable drivers. =20 Please enter a BZ. This looks like a critical enough regression bug that = we may want to=20 consider fixing it before stable tag release. =20 There are some limitations to the DLL approach. It is good for source lev= el debug using VS. The limitation is when the same driver is loaded more than once= . The first one is loaded as a DLL and can be debugged. The 2nd one is loaded i= nto UEFI memory, but is not registered so all you get is assembly code debug. =20 The method that Andrew describes supports multiple instances of the same d= river. =20 Thanks, =20 Mike =20 From: devel@edk2.groups.io [mailto:devel@ed= k2.groups.io] On Behalf Of Andrew Fish via Groups.Io Sent: Wednesday, August 21, 2019 2:45 PM To: devel@edk2.groups.io ; tim.lewis@insyde.= com =20 Subject: Re: [edk2-devel] EmulatorPkg does not unload DLL after exit =20 Tim, =20 Sounds like you are on to something.... =20 The Unix side [1] does a dlclose() to undo a dlsym().=20 =20 I seem to remember this code flow is very very old, I wonder if it is poss= ible to do better? For lldb (Xcode) I skipped the LoadLibraryEx()/dlsym() a= ction and wrote a debugger script to load symbols. So for Xcode all the cod= e is running from the emulator memory, not in an OS context. For lldb I add= ed a breakpoint script on SecGdbScriptBreak() that loads symbols. Since the= SecGdbScriptBreak() is part of the App you get symbols for free, and the b= reak point function can access the arguments to SecGdbScriptBreak() to load= symbols [3]. Seems like this would easily work for gdb too? I'm not sure h= ow easy this is to do in VC++, but it seems like we can leverage what ever = scripts folks use to load symbols on real hardware.=20 =20 I'm not 100% sure what is happening on Linux as if you build the code as E= FIAPI it seems like the dlopen would fail since it is not x86_64 Sys V ABI,= and I guess it could be falling back to the use the gdb script to load sym= bols. This would Imply for X64 Windows is the odd person out.=20 =20 The only downside to "making it real" is you may need to launch pointing a= t a debugger script to get source level debug.=20 =20 I know the lldb symbol loading is working as I help Mike test his recent p= atches.=20 =20 [1] https://github.com/tianocore/edk2/blob/master/EmulatorPkg/Unix/Host/Ho= st.c#L1263 [2] https://github.com/tianocore/edk2/blob/master/EmulatorPkg/Unix/Host/Ho= st.c#L1120 [3] https://github.com/tianocore/edk2/blob/master/EmulatorPkg/Unix/lldbefi= .py#L357 =20 Thanks, =20 Andrew Fish On Aug 21, 2019, at 2:13 PM, Tim Lewis > wrote: =20 When running a shell app twice, I ran into an interesting problem: global = variables that had initializers were not initialized to the defaults specif= ied in the source. Running the same shell app under the old NT32 seems to w= ork. It turns out that the shell app was not being reloaded, but rather bei= ng relaunched. I deduced this from the following behavior: =20 1. The value of the global variables was the same as the last known value = from the previous invocation. 2. The following line in WinHost.c was returning the exact same value for = the DLL being loaded Library =3D LoadLibraryEx (DllFileName, NULL, DONT_RESOLVE_DLL_REFERENCES)= ; (around line 901) 3. The corresponding code in Nt32=E2=80=99s PeCoffExtractionLib had the fo= llowing lines in PeCoffLoaderUnloadImageExtraAction VOID *ModHandle; =20 ASSERT (ImageContext !=3D NULL); ModHandle =3D RemoveModeHandle (ImageContext); if (ModHandle !=3D NULL) { mWinNt->FreeLibrary (ModHandle); } =20 However, the same function in EmulatorPkg=E2=80=99s WinHost.c has: =20 ASSERT (ImageContext !=3D NULL); =20 So it appears that the DLL is never being unloaded and the subsequent invo= cation of the shell app uses the same instance of the DLL, leaving the glob= al variables with the previous values. There are two related functions: Add= ModHandle and RemoveModHandle. =20 Am I missing something? Or heading in the right direction?=20 =20 Thanks, Tim =20 From: devel@edk2.groups.io < devel@edk2.groups.io>=20 Sent: Saturday, August 17, 2019 6:30 PM To: devel@edk2.groups.io Subject: [edk2-devel] Upcoming Event: TianoCore Design Meeting - APAC/NAMO= - Thu, 08/22/2019 6:30pm-7:30pm #cal-reminder =20 Reminder: TianoCore Design Meeting - APAC/NAMO When: Thursday, 22 August 2019, 6:30pm to 7:30pm, (GMT-07:00) America/Los = Angeles=20 Where: https://zoom.us/j/969264410 View Event Organizer: Stephano Cetola stephano.cetol= a@intel.com Description: Join Zoom Meeting https://zoom.us/j/969264410 =20 One tap mobile +16465588656,,969264410# US (New York) +17207072699,,969264410# US =20 Dial by your location +1 646 558 8656 US (New York) +1 720 707 2699 US Meeting ID: 969 264 410 Find your local number: https://zoom.us/u/= abOtdJckxL =20 ------=_NextPart_000_0C21_01D55848.76D492F0 Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: quoted-printable

Mike =E2=80=93

Ok, it is bug 2104.

 

From:<= /b> devel= @edk2.groups.io <devel@edk2.groups.io> On Behalf Of Michael D = Kinney
Sent: Wednesday, August 21, 2019 3:08 PM
To: dev= el@edk2.groups.io; afish@apple.com; tim.lewis@insyde.com; Kinney, Michael D= <michael.d.kinney@intel.com>
Subject: Re: [edk2-devel] Emu= latorPkg does not unload DLL after exit

 

Tim,=

 

Thanks for the analysis.

 

Lo= oks like we need to fix PeCoffLoaderUnloadImageExtraAction()

for winhost to unload the DLL to keep comp= atibility with Nt32.=C2=A0 This is important

for both apps and unloadable drivers.<= /p>

 

Please enter a BZ.=C2=A0 This looks like a critical enough regressi= on bug that we may want to

= consider fixing it before stable tag release.

 

Th= ere are some limitations to the DLL approach.=C2=A0 It is good for source l= evel debug

using VS.=C2=A0 T= he limitation is when the same driver is loaded more than once.=C2=A0 The

first one is loaded as a DLL = and can be debugged.=C2=A0 The 2nd one is loaded into=

UEFI memory, but is not registered so = all you get is assembly code debug.

 

The method th= at Andrew describes supports multiple instances of the same driver.

 

Thanks,

<= o:p> 

Mike

 

From: devel@edk2.groups= .io [mailto:devel@edk2.groups.i= o] On Behalf Of Andrew Fish via Groups.Io
Sent: Wednes= day, August 21, 2019 2:45 PM
To: devel@edk2.groups.io; t= im.lewis@insyde.com
Subject: Re: [edk2-devel] EmulatorPkg doe= s not unload DLL after exit

 

Tim,

 

Sou= nds like you are on to something....

 

The Unix side = [1] does a dlclose() to undo a dlsym(). 

 

I see= m to remember this code flow is very very old, I wonder if it is possible t= o do better? For lldb (Xcode) I skipped the LoadLibraryEx()/dlsym() action = and wrote a debugger script to load symbols. So for Xcode all the code is r= unning from the emulator memory, not in an OS context. For lldb I added a b= reakpoint script on SecGdbScriptBreak() that loads symbols. Since the = SecGdbScriptBreak() is part of the App you get symbols for free, and the br= eak point function can access the arguments to SecGdbScriptBreak() to load = symbols [3]. Seems like this would easily work for gdb too? I'm not sure ho= w easy this is to do in VC++, but it seems like we can leverage what ever s= cripts folks use to load symbols on real hardware. 

 

I'm not 100% sure what is happening on Linux as if you build the code= as EFIAPI it seems like the dlopen would fail since it is not x86_64 Sys V= ABI, and I guess it could be falling back to the use the gdb script to loa= d symbols. This would Imply for X64 Windows is the odd person out. 

 

The only downside to "making it real" = is you may need to launch pointing at a debugger script to get source level= debug. 

 

I know the lldb symbol loading = is working as I help Mike test his recent patches. 

 

[3] https://github.com/tianocore= /edk2/blob/master/EmulatorPkg/Unix/lldbefi.py#L357

<= p class=3DMsoNormal> 

Th= anks,

 

Andrew Fish



On Aug 21, 2019, at 2:13 PM,= Tim Lewis <tim.lewis@insyde.com= > wrote:

 <= /p>

When running a shell app twice, I ran into an inte= resting problem: global variables that had initializers were not initialize= d to the defaults specified in the source. Running the same shell app under= the old NT32 seems to work. It turns out that the shell app was not being = reloaded, but rather being relaunched. I deduced this from the following be= havior:

 

  1. The value of the global variables was the sa= me as the last known value from the previous invocation.<= /li>
  2. The following line in Wi= nHost.c was returning the exact same value for the DLL being loaded

Library =3D&nbs= p;LoadLibraryEx (DllFileName, NULL, DONT_R= ESOLVE_DLL_REFERENCES); (around line 901)

<= ol style=3D'margin-top:0in' start=3D3 type=3D1>
  • The corresponding code in Nt32=E2=80=99s PeCoffExtract= ionLib had the following lines in PeCoffLoaderUnloadImageExtraAction

    •   VOID *ModHandle;

     

      ASSERT (ImageContext !=3D NULL);

    <= div>

      ModHandle =3D RemoveModeHandle (ImageContext);<= /o:p>

      if (ModHandle !=3D NULL) {

        mWinNt->Fre= eLibrary (ModHandle);

    =   }<= o:p>

     

    However, the same function in EmulatorPkg=E2=80=99s Win= Host.c has:

     

    ASSERT (ImageContext !=3D NULL);

     

    So it appears that the DLL is never being unloaded and the subsequent = invocation of the shell app uses the same instance of the DLL, leaving the = global variables with the previous values. There are two related functions:= AddModHandle and RemoveModHandle.

     

    Am I missing someth= ing? Or heading in the right direction?=  

     =

    Thanks,

    <= p class=3DMsoNormal>Tim

     

    From: devel@edk2.groups.io <devel@edk2.groups.io> 
    Sent:=  Saturday, August 17, 2019 6:30 PM
    To: <= span style=3D'color:purple'>devel@edk2.groups.io
    Subject:<= /b> [edk2-devel] Upcoming E= vent: TianoCore Design Meeting - APAC/NAMO - Thu, 08/22/2019 6:30pm-7:30pm = #cal-reminder

     =

    Reminder:=  TianoCore Design Meeting - APAC/NAMO

    = When: Thursday, 22 A= ugust 2019, 6:30pm to 7:30pm, (GMT-07:00) America/Los Angeles 

    Where:= https= ://zoom.us/j/969264410

    View Event

    Organizer: Stephano Cetola stephano.cetola@intel.com

    Description:

    Join Zoom Meeting

    &nbs= p;

    One tap mobile

    +16465588656,,969264410# US (New York)<= /o:p>

    +17207072699,,969264410# US

     

    =

    Dial by your location

            +1 646 558 8656 US (New Yor= k)

        &nbs= p;   +1 720 707 2699 US

    Meeting ID: 969 264 410

    Fi= nd your local number: https://zo= om.us/u/abOtdJckxL

     

    ------=_NextPart_000_0C21_01D55848.76D492F0--