From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: mx.groups.io; dkim=pass header.i=@apple.com header.s=20180706 header.b=wNQLzzFN; spf=pass (domain: apple.com, ip: 17.151.62.67, mailfrom: afish@apple.com) Received: from nwk-aaemail-lapp02.apple.com (nwk-aaemail-lapp02.apple.com [17.151.62.67]) by groups.io with SMTP; Fri, 16 Aug 2019 11:36:00 -0700 Received: from pps.filterd (nwk-aaemail-lapp02.apple.com [127.0.0.1]) by nwk-aaemail-lapp02.apple.com (8.16.0.27/8.16.0.27) with SMTP id x7GIVu2E002793; Fri, 16 Aug 2019 11:35:58 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=apple.com; h=sender : from : message-id : content-type : mime-version : subject : date : in-reply-to : cc : to : references; s=20180706; bh=OLw70+uKH7zsFUW0ltBTVVCxTNTsJLJGscoQb3sk2q4=; b=wNQLzzFNRsSxXMA6pcvqQUOjVMZnDZoQA2Z23PjwuvDUycvxbIggPed5PSlq36w14mak NSw958dg56zv2uHvnuBr36zNd8XcfRwuUcqchspEkWmvI1oaHgXgySwn6+HpTOszGjsD yA7CYBLbyMROuGNZjPpXCLGcHPxPkkzjB1CvCNSEablEa+mr8JcDSdFh7kovQoTAtOEQ fczlXyCe/GcJoOSR4fp+ZIEouR5SzOEtCzh6In76xzQ3uTpBccEaQCMNS38jbybjp+ju UiSFVchFgchb8B7yRL7e6ij/ZNAKWQdZ4/gwgHnXZ+jdmiAsL6Ri7t/Y6N4v91SuhxrN Rg== Received: from ma1-mtap-s03.corp.apple.com (ma1-mtap-s03.corp.apple.com [17.40.76.7]) by nwk-aaemail-lapp02.apple.com with ESMTP id 2u9tgn0xfh-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO); Fri, 16 Aug 2019 11:35:58 -0700 Received: from nwk-mmpp-sz11.apple.com (nwk-mmpp-sz11.apple.com [17.128.115.155]) by ma1-mtap-s03.corp.apple.com (Oracle Communications Messaging Server 8.0.2.4.20190507 64bit (built May 7 2019)) with ESMTPS id <0PWC007ZSEBWCL20@ma1-mtap-s03.corp.apple.com>; Fri, 16 Aug 2019 11:35:57 -0700 (PDT) Received: from process_milters-daemon.nwk-mmpp-sz11.apple.com by nwk-mmpp-sz11.apple.com (Oracle Communications Messaging Server 8.0.2.4.20190507 64bit (built May 7 2019)) id <0PWC00300DV90S00@nwk-mmpp-sz11.apple.com>; Fri, 16 Aug 2019 11:35:57 -0700 (PDT) X-Va-A: X-Va-T-CD: e65e81296e009a2c770ee052678c11ce X-Va-E-CD: 0ff67942e24c10226e1835dfb7e409d0 X-Va-R-CD: d8157d875cc43f345e6dd4d26f1e9320 X-Va-CD: 0 X-Va-ID: 139aff9d-d6dc-4857-9926-76276fa9e183 X-V-A: X-V-T-CD: e65e81296e009a2c770ee052678c11ce X-V-E-CD: 0ff67942e24c10226e1835dfb7e409d0 X-V-R-CD: d8157d875cc43f345e6dd4d26f1e9320 X-V-CD: 0 X-V-ID: b5df2611-bd73-4196-a663-85fe7801a17b X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2019-08-16_08:,, signatures=0 Received: from [17.235.18.156] (unknown [17.235.18.156]) by nwk-mmpp-sz11.apple.com (Oracle Communications Messaging Server 8.0.2.4.20190507 64bit (built May 7 2019)) with ESMTPSA id <0PWC00DALEBVKJ10@nwk-mmpp-sz11.apple.com>; Fri, 16 Aug 2019 11:35:57 -0700 (PDT) Sender: afish@apple.com From: "Andrew Fish" Message-id: <1E44B076-EDC9-492C-9214-D4FC1D3AEAFB@apple.com> MIME-version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) Subject: Re: [edk2-devel] Determining TSC frequency programmatically Date: Fri, 16 Aug 2019 11:35:55 -0700 In-reply-to: <8EC14D0D-DFA5-412D-A4E1-4D641576D58E@protonmail.com> Cc: jiewen.yao@intel.com To: devel@edk2.groups.io, vit9696@protonmail.com References: <8EC14D0D-DFA5-412D-A4E1-4D641576D58E@protonmail.com> X-Mailer: Apple Mail (2.3445.104.11) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2019-08-16_08:,, signatures=0 Content-type: multipart/alternative; boundary="Apple-Mail=_D9886371-786C-43A3-BEE3-A7940F322F56" --Apple-Mail=_D9886371-786C-43A3-BEE3-A7940F322F56 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Vitaly, As Mike mentioned platforms can know more info about how they are construc= ted thus you may not want to have a lot of generic discovery code floating = about if you don't really need it.=20 One option could be to pass up the TSC Frequency/Period via some EFI mecha= nism so generic code can be told by platform specific code.=20 The PI spec already has an abstraction for a CPU based timer that is archi= tecture neutral. The CPU Architectural Protocol has a GetTimerValue() membe= r function.=20 https://github.com/tianocore/edk2/blob/master/MdePkg/Include/Protocol/Cpu.= h#L220 For X86 it returns TSC https://github.com/tianocore/edk2/blob/master/UefiCpuPkg/CpuDxe/CpuDxe.c#L= 289 EFI Systems are not required to implement PI so we usually don't encourage= generic EFI code to go after PI APIs.=20 I'd also point out that using TSC can break in things like the EmulatorPkg= as you end up running in ring 0 and TSC access is blocked.=20 https://github.com/tianocore/edk2/blob/master/EmulatorPkg/CpuRuntimeDxe/Cp= u.c#L352 https://github.com/tianocore/edk2/blob/master/EmulatorPkg/Unix/Host/EmuThu= nk.c#L250 I would point that a library that did TSC frequency discovery would likely= be useful for the UefiCpuPkg CpuDxe driver.=20 Thanks, Andrew Fish > On Aug 15, 2019, at 2:10 PM, Vitaly Cheptsov via Groups.Io wrote: >=20 > Hello, >=20 > I initially raised this question in a new TimerLib patch[1], but as the = discussion was getting more distracted, I decided to create a separate thre= ad in hopes new people could join. >=20 > The issue is that our UEFI bootloader needs to obtain TSC frequency to p= ass it to our specialised operating system that uses TSC for scheduling on = x86. >=20 > For a while we went with ACPI power management timer to measure the freq= uency, but as modern Intel CPUs support CPUID 15H leaf (CPUID_TIME_STAMP_CO= UNTER) we try to use where possible for better accuracy. The issue with thi= s CPUID leaf is that the crystal clock frequency returned in ECX register i= s optional and therefore can be 0. Intel SDM suggests to use a static value= in this case[2], but it is completely opaque on how to match the running C= PU with its static value from SDM. >=20 > Initially we went with CPUID model checking, but this failed badly for X= eon Scalable and Xeon W, as they share the CPUID (06_55H) but have differen= t crystal clock frequencies (25 MHz vs 24 MHz accordingly). Donald Kuo gave= a good hint in the previous thread that client CPUs usually get 24 MHz cry= stal clock, server CPUs have 25, and Atoms have 19.2. This, however, does n= ot make the situation easier as we do not see a way to determine CPU vertic= al segment without e.g. parsing the CPUID brand string. >=20 > Apparently, we are not alone, and different open-source operating system= s have different workarounds to this issue. For example, Linux kernel went = with using marketing frequency from CPUID 16H leaf (CPUID_PROCESSOR_FREQUEN= CY)[3], and BSD flavours fallback to older methods when neither crystal clo= ck frequency can be obtained through CPUID 15H, nor unambiguous CPUID model= s exist to be able to use static values. >=20 > Another issue we see with EDK II TimerLib implementations for x86 is tha= t they are very model specific. As Michael Kinney said, the situation is no= t a problem when you use TimerLib for BSP bringup, as you already know the = CPU family you target, however, it is not great for other uses, although th= ey may be discouraged. In our opinion, regardless of the use, this approach= has a number of design issues. >=20 > All modern Intel x86 CPUs have virtual TSC counter that has fixed freque= ncy. In fact, this is true for most, if not all, modern x86 CPUs by other v= endors as well. This makes us believe that EDK II should have generic TscTi= merLib for x86, which will work anywhere when given valid TSC frequency, an= d TscFrequencyLib, which would determine TSC frequency in a vendor-specific= method. Ideally there exists GenericTscFrequencyLib that can do this for a= wide majority of CPUs through different methods at runtime, including CPUI= D 15H, ACPI power management, vendor-specific extensions, etc. >=20 > To summarise: > - We need to know how to match current running CPU with its crystal cloc= k frequency when CPUID 15H reports 0 ECX. > - We would like to suggest to split TSC-based TimerLib in two libraries:= generic with timer implementation and platform-specific with TSC frequency= discovery. > - We wonder whether a generic vendor-supported TSC frequency discovery l= ibrary working on a wide range of CPUs is feasible to have in EDK II mainli= ne. >=20 > Best regards, > Vitaly >=20 > [1] https://edk2.groups.io/g/devel/topic/patch_ueficpupkg_adding_a/32839= 184?p=3D,,,20,0,0,0::recentpostdate%2Fsticky,,,20,2,0,32839184 > [PATCH] UefiCpuPkg: Adding a new TSC library by using CPUID(0x15) TS= C leaf > [2] 18.7.3 Determining the Processor Base Frequency > Table 18-75. Nominal Core Crystal Clock Frequency > [3] https://lore.kernel.org/lkml/20190509055417.13152-1-drake@endlessm.c= om/ >=20 >=20 --Apple-Mail=_D9886371-786C-43A3-BEE3-A7940F322F56 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii
Vitaly,

As Mike mentioned platforms can know more info about how they are co= nstructed thus you may not want to have a lot of generic discovery code flo= ating about if you don't really need it. 

One option could be to pass up the TSC Freque= ncy/Period via some EFI mechanism so generic code can be told by platform s= pecific code. 

The PI spec already has an abstraction for a CPU based timer that is arc= hitecture neutral. The CPU Architectural Protocol has a GetTimerValue() mem= ber function. 
https= ://github.com/tianocore/edk2/blob/master/MdePkg/Include/Protocol/Cpu.h#L220=

For X86 it r= eturns TSC
https://github.co= m/tianocore/edk2/blob/master/UefiCpuPkg/CpuDxe/CpuDxe.c#L289

EFI Systems are not require= d to implement PI so we usually don't encourage generic EFI code to go afte= r PI APIs. 

= I'd also point out that using TSC can break in things like the EmulatorPkg = as you end up running in ring 0 and TSC access is blocked. 
https://github.com/tianocore/edk2= /blob/master/EmulatorPkg/CpuRuntimeDxe/Cpu.c#L352
= https://github.com/tianocore/edk2/blob/mast= er/EmulatorPkg/Unix/Host/EmuThunk.c#L250


I would p= oint that a library that did TSC frequency discovery would likely be useful= for the UefiCpuPkg CpuDxe driver. 

Thanks,

<= div class=3D"">Andrew Fish

On Aug 15, 2019, at 2:10 PM, Vitaly = Cheptsov via Groups.Io <vit9696=3Dprotonmail.com@groups.io> wrote:
Hello,

I initially raised this question in a new TimerLib patch[1], but as = the discussion was getting more distracted, I decided to create a separate = thread in hopes new people could join.

=
The issue is that our UEFI bootloader needs to obtain= TSC frequency to pass it to our specialised operating system that uses TSC= for scheduling on x86.

For a while we went with ACPI power management timer to measure the = frequency, but as modern Intel CPUs support CPUID 15H leaf (CPUID_TIME_STAM= P_COUNTER) we try to use where possible for better accuracy. The issue with= this CPUID leaf is that the crystal clock frequency returned in ECX regist= er is optional and therefore can be 0. Intel SDM suggests to use a static v= alue in this case[2], but it is completely opaque on how to match the runni= ng CPU with its static value from SDM.

=
Initially we went with CPUID model checking, but this= failed badly for Xeon Scalable and Xeon W, as they share the CPUID (06_55H= ) but have different crystal clock frequencies (25 MHz vs 24 MHz accordingl= y). Donald Kuo gave a good hint in the previous thread that client CPUs usu= ally get 24 MHz crystal clock, server CPUs have 25, and Atoms have 19.2. Th= is, however, does not make the situation easier as we do not see a way to d= etermine CPU vertical segment without e.g. parsing the CPUID brand string.<= /div>

Apparently, we ar= e not alone, and different open-source operating systems have different wor= karounds to this issue. For example, Linux kernel went with using marketing= frequency from CPUID 16H leaf (CPUID_PROCESSOR_FREQUENCY)[3], and BSD flav= ours fallback to older methods when neither crystal clock frequency can be = obtained through CPUID 15H, nor unambiguous CPUID models exist to be able t= o use static values.

Another issue we see with EDK II TimerLib implementations for x86 is= that they are very model specific. As Michael Kinney said, the situation i= s not a problem when you use TimerLib for BSP bringup, as you already know = the CPU family you target, however, it is not great for other uses, althoug= h they may be discouraged. In our opinion, regardless of the use, this appr= oach has a number of design issues.

All modern Intel x86 CPUs have virtual TSC counter that = has fixed frequency. In fact, this is true for most, if not all, modern x86= CPUs by other vendors as well. This makes us believe that EDK II should ha= ve generic TscTimerLib for x86, which will work anywhere when given valid T= SC frequency, and TscFrequencyLib, which would determine TSC frequency in a= vendor-specific method. Ideally there exists GenericTscFrequencyLib that c= an do this for a wide majority of CPUs through different methods at runtime= , including CPUID 15H, ACPI power management, vendor-specific extensions, e= tc.

To summarise:=
- We need to know how to match current running CPU wi= th its crystal clock frequency when CPUID 15H reports 0 ECX.
- We would like to suggest to split TSC-based TimerLib in two librar= ies: generic with timer implementation and platform-specific with TSC frequ= ency discovery.
- We wonder whether a generic vendor-s= upported TSC frequency discovery library working on a wide range of CPUs is= feasible to have in EDK II mainline.

<= /div>
Best regards,
Vitaly

[1] https://edk2.group= s.io/g/devel/topic/patch_ueficpupkg_adding_a/32839184?p=3D,,,20,0,0,0::rece= ntpostdate%2Fsticky,,,20,2,0,32839184
   = ; [PATCH] UefiCpuPkg: Adding a new TSC library by using CPUID(0x15) TSC lea= f
[2] 18.7.3 Determining the Processor Base Frequ= ency
Table 18-75. Nominal Core Crystal Clock Frequency=
[3] https://lore.kernel.org/lkml/20190509055417.13152-1-drake@endlessm.c= om/


--Apple-Mail=_D9886371-786C-43A3-BEE3-A7940F322F56--