From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: mx.groups.io; dkim=pass header.i=@apple.com header.s=20180706 header.b=szdZsW6Y; spf=pass (domain: apple.com, ip: 17.151.62.67, mailfrom: afish@apple.com) Received: from nwk-aaemail-lapp02.apple.com (nwk-aaemail-lapp02.apple.com [17.151.62.67]) by groups.io with SMTP; Wed, 21 Aug 2019 14:45:57 -0700 Received: from pps.filterd (nwk-aaemail-lapp02.apple.com [127.0.0.1]) by nwk-aaemail-lapp02.apple.com (8.16.0.27/8.16.0.27) with SMTP id x7LLbM7S058346; Wed, 21 Aug 2019 14:45:55 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=apple.com; h=sender : from : content-type : mime-version : subject : date : references : to : in-reply-to : message-id; s=20180706; bh=TJDElSYwtsYs0w6rwwA+wSpejgUQ+ur1YdkvJthnmAw=; b=szdZsW6Yt292ue7PLuQR/83r/mp8VRcRRC0C0tl516aEFoOcE0sXx3Xd8AqQEAmvcHe+ RiUCPKdOe/m/BCwmNR3/SPAdO9C1YsSCaGeO1FJhEtzwtCLs1kI/tW1hoEh1LULBybXB lmJIxCyZudipr8gUlSovJxJyk7h1w3FLi2ZbqgEgJesKJKdMAGE7AGGWqM3cIsMcFM56 NH5IlMHFSz41UZZZ9v2Vox3qnKIPWgNCksJuTha3ZFvzllMnEXPUaLphwbPG0sX+FrJI oUj4CfSzj3CpfFYhFkb6CHeejW5NK4y8OM3q89Vda8lz2jcw0d0BsBV9FbwuPJY54tym rg== Received: from mr2-mtap-s02.rno.apple.com (mr2-mtap-s02.rno.apple.com [17.179.226.134]) by nwk-aaemail-lapp02.apple.com with ESMTP id 2uee5pya5a-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO); Wed, 21 Aug 2019 14:45:55 -0700 Received: from nwk-mmpp-sz11.apple.com (nwk-mmpp-sz11.apple.com [17.128.115.155]) by mr2-mtap-s02.rno.apple.com (Oracle Communications Messaging Server 8.0.2.4.20190507 64bit (built May 7 2019)) with ESMTPS id <0PWL008H0WGIY8C0@mr2-mtap-s02.rno.apple.com>; Wed, 21 Aug 2019 14:45:54 -0700 (PDT) Received: from process_milters-daemon.nwk-mmpp-sz11.apple.com by nwk-mmpp-sz11.apple.com (Oracle Communications Messaging Server 8.0.2.4.20190507 64bit (built May 7 2019)) id <0PWL00H00WC5JQ00@nwk-mmpp-sz11.apple.com>; Wed, 21 Aug 2019 14:45:53 -0700 (PDT) X-Va-A: X-Va-T-CD: fc4de0ab1b28b90c1989f0db93b04441 X-Va-E-CD: aa16dc9728786108e339ae3abd3b96eb X-Va-R-CD: e2ed1727495e3fee0497f525da1086ad X-Va-CD: 0 X-Va-ID: ff551d2a-d3c3-4985-8d45-f46603f5cc99 X-V-A: X-V-T-CD: fc4de0ab1b28b90c1989f0db93b04441 X-V-E-CD: aa16dc9728786108e339ae3abd3b96eb X-V-R-CD: e2ed1727495e3fee0497f525da1086ad X-V-CD: 0 X-V-ID: 80040394-153c-4f46-ba1a-cef1522725e8 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2019-08-21_07:,, signatures=0 Received: from [17.226.41.186] (unknown [17.226.41.186]) by nwk-mmpp-sz11.apple.com (Oracle Communications Messaging Server 8.0.2.4.20190507 64bit (built May 7 2019)) with ESMTPSA id <0PWL00CXJWFT1830@nwk-mmpp-sz11.apple.com>; Wed, 21 Aug 2019 14:45:29 -0700 (PDT) Sender: afish@apple.com From: "Andrew Fish" MIME-version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) Subject: Re: [edk2-devel] EmulatorPkg does not unload DLL after exit Date: Wed, 21 Aug 2019 14:45:28 -0700 References: <0ba301d55865$53151750$f93f45f0$@insyde.com> To: devel@edk2.groups.io, tim.lewis@insyde.com In-reply-to: <0ba301d55865$53151750$f93f45f0$@insyde.com> Message-id: X-Mailer: Apple Mail (2.3445.104.11) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2019-08-21_07:,, signatures=0 Content-type: multipart/alternative; boundary="Apple-Mail=_26826DD0-1D14-4C9B-B285-0E112D11156A" --Apple-Mail=_26826DD0-1D14-4C9B-B285-0E112D11156A Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Tim, Sounds like you are on to something.... The Unix side [1] does a dlclose() to undo a dlsym().=20 I seem to remember this code flow is very very old, I wonder if it is poss= ible to do better? For lldb (Xcode) I skipped the LoadLibraryEx()/dlsym() a= ction and wrote a debugger script to load symbols. So for Xcode all the cod= e is running from the emulator memory, not in an OS context. For lldb I add= ed a breakpoint script on SecGdbScriptBreak() that loads symbols. Since the= SecGdbScriptBreak() is part of the App you get symbols for free, and the b= reak point function can access the arguments to SecGdbScriptBreak() to load= symbols [3]. Seems like this would easily work for gdb too? I'm not sure h= ow easy this is to do in VC++, but it seems like we can leverage what ever = scripts folks use to load symbols on real hardware.=20 I'm not 100% sure what is happening on Linux as if you build the code as E= FIAPI it seems like the dlopen would fail since it is not x86_64 Sys V ABI,= and I guess it could be falling back to the use the gdb script to load sym= bols. This would Imply for X64 Windows is the odd person out.=20 The only downside to "making it real" is you may need to launch pointing a= t a debugger script to get source level debug.=20 I know the lldb symbol loading is working as I help Mike test his recent p= atches.=20 [1] https://github.com/tianocore/edk2/blob/master/EmulatorPkg/Unix/Host/Ho= st.c#L1263 [2] https://github.com/tianocore/edk2/blob/master/EmulatorPkg/Unix/Host/Ho= st.c#L1120 [3] https://github.com/tianocore/edk2/blob/master/EmulatorPkg/Unix/lldbefi= .py#L357 Thanks, Andrew Fish > On Aug 21, 2019, at 2:13 PM, Tim Lewis wrote: >=20 > When running a shell app twice, I ran into an interesting problem: globa= l variables that had initializers were not initialized to the defaults spec= ified in the source. Running the same shell app under the old NT32 seems to= work. It turns out that the shell app was not being reloaded, but rather b= eing relaunched. I deduced this from the following behavior: > > The value of the global variables was the same as the last known value f= rom the previous invocation. > The following line in WinHost.c was returning the exact same value for t= he DLL being loaded > Library =3D LoadLibraryEx (DllFileName, NULL, DONT_RESOLVE_DLL_REFERENCE= S); (around line 901) > The corresponding code in Nt32=E2=80=99s PeCoffExtractionLib had the fol= lowing lines in PeCoffLoaderUnloadImageExtraAction > VOID *ModHandle; > > ASSERT (ImageContext !=3D NULL); > ModHandle =3D RemoveModeHandle (ImageContext); > if (ModHandle !=3D NULL) { > mWinNt->FreeLibrary (ModHandle); > } > > However, the same function in EmulatorPkg=E2=80=99s WinHost.c has: > > ASSERT (ImageContext !=3D NULL); > > So it appears that the DLL is never being unloaded and the subsequent in= vocation of the shell app uses the same instance of the DLL, leaving the gl= obal variables with the previous values. There are two related functions: A= ddModHandle and RemoveModHandle. > > Am I missing something? Or heading in the right direction?=20 > > Thanks, > Tim > > From: devel@edk2.groups.io >=20 > Sent: Saturday, August 17, 2019 6:30 PM > To: devel@edk2.groups.io > Subject: [edk2-devel] Upcoming Event: TianoCore Design Meeting - APAC/NA= MO - Thu, 08/22/2019 6:30pm-7:30pm #cal-reminder > > Reminder: TianoCore Design Meeting - APAC/NAMO >=20 > When: Thursday, 22 August 2019, 6:30pm to 7:30pm, (GMT-07:00) America/Lo= s Angeles=20 >=20 > Where:https://zoom.us/j/969264410 > View Event > Organizer: Stephano Cetola stephano.cetola@intel.com > Description: >=20 > Join Zoom Meeting > https://zoom.us/j/969264410 > > One tap mobile > +16465588656,,969264410# US (New York) > +17207072699,,969264410# US > > Dial by your location > +1 646 558 8656 US (New York) > +1 720 707 2699 US > Meeting ID: 969 264 410 > Find your local number: https://zoom.us/u/abOtdJckxL >=20 --Apple-Mail=_26826DD0-1D14-4C9B-B285-0E112D11156A Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 Tim,

Sounds like you are on to something....

The Unix side [1] do= es a dlclose() to undo a dlsym(). 

I seem to remember this code flow is very very old, = I wonder if it is possible to do better? For lldb (Xcode) I skipped the Loa= dLibraryEx()/dlsym() action and wrote a debugger script to load symbols. So= for Xcode all the code is running from the emulator memory, not in an OS c= ontext. For lldb I added a breakpoint script on SecGdbScriptBreak() th= at loads symbols. Since the SecGdbScriptBreak() is part of the App you get = symbols for free, and the break point function can access the arguments to = SecGdbScriptBreak() to load symbols [3]. Seems like this would easily work = for gdb too? I'm not sure how easy this is to do in VC++, but it seems like= we can leverage what ever scripts folks use to load symbols on real hardwa= re. 

I'm not= 100% sure what is happening on Linux as if you build the code as EFIAPI it= seems like the dlopen would fail since it is not x86_64 Sys V ABI, and I g= uess it could be falling back to the use the gdb script to load symbols. Th= is would Imply for X64 Windows is the odd person out. 

The only downsi= de to "making it real" is you may need to launch pointing at a debugger scr= ipt to get source level debug. 

I know the lldb symbol loading is working as I he= lp Mike test his recent patches. 

=
[1] https://github.= com/tianocore/edk2/blob/master/EmulatorPkg/Unix/Host/Host.c#L1263
=
[2] https://github.com/ti= anocore/edk2/blob/master/EmulatorPkg/Unix/Host/Host.c#L1120
[3] https://github.com/tianocore/= edk2/blob/master/EmulatorPkg/Unix/lldbefi.py#L357
Thanks,

Andrew = Fish

On Aug 21, 2019, at 2:13 PM, Tim Lewis <tim.lewis@insyde.com> wrote:

W= hen running a shell app twice, I ran into an interesting problem: global va= riables that had initializers were not initialized to the defaults specifie= d in the source. Running the same shell app under the old NT32 seems to wor= k. It turns out that the shell app was not being reloaded, but rather being= relaunched. I deduced this from the following behavior:
 
  1. The value of the global v= ariables was the same as the last known value from the previous invocation.=
  2. The f= ollowing line in WinHost.c was returning the exact same value for the DLL b= eing loaded
Library =3D=  LoadLibraryEx (DllFileNa= me, NULL, DONT_RESOLVE_DLL_REFERENCES); (around line 901)
  1. Th= e corresponding code in Nt32=E2=80=99s PeCoffExtractionLib had the followin= g lines in PeCoffLoaderUnloadImageExtraAction
    2.   VOID *ModHandle;=
     
      ASSERT (ImageContext !=3D NULL);
      ModHandle =3D RemoveModeHan= dle (ImageContext);
    =   if (ModHandle !=3D NULL) {
        mWinNt->FreeLibrary (ModHandle);
      }
     
    However, the same function in EmulatorPkg=E2=80=99= s WinHost.c has:
 
ASSERT (ImageContext != =3D NULL);
 
So it appea= rs that the DLL is never being unloaded and the subsequent invocation of th= e shell app uses the same instance of the DLL, leaving the global variables= with the previous values. There are two related functions: AddModHandle an= d RemoveModHandle.
<= o:p class=3D""> 
Am I missing s= omething? Or heading in the right direction? 
 
Thanks,
Tim
 
From: devel@edk2.groups.io<= span class=3D"Apple-converted-space"> <devel@edk2.groups.io>&= nbsp;
Sent: Saturday, August 17, 2019 6:30 PM
<= b class=3D"">To: devel@edk2.groups.io
Subject: [edk2-dev= el] Upcoming Event: TianoCore Design Meeting - APAC/NAMO - Thu, 08/22/2019 = 6:30pm-7:30pm #cal-reminder
 

= Reminder: TianoCore Des= ign Meeting - APAC/NAMO

When: Thursday, = 22 August 2019, 6:30pm to 7:30pm, (GMT-07:00) America/Los Angeles 

Where:https://zoom= .us/j/969264410

View Event

Organizer: Stephano Cetola stephano.cetola@intel.co= m

Description:

Join Zoom Meeting=
https://zoom.us/j/969264410
 
One tap mobile
+16465588656,,969264410# US (New York)
+17207072699,,969264410# US=
 
Dial by your location
        +1 646 55= 8 8656 US (New York)
        +1 720 707 2699 US
Meeting ID: 969 264 410
Find your local number: https://zoom.us/u/abOtdJckxL


--Apple-Mail=_26826DD0-1D14-4C9B-B285-0E112D11156A--