[Patch 0/2] Separate semaphore container.

[Patch 0/2] Separate semaphore container.

[Eric Dong]

In current implementation, core level semaphore use same container
with package level semaphore. This design will let the core level
semaphore not works as expected in below case:
1. Feature A has CPU_FEATURE_CORE_BEFORE dependence with Feature B.
2. Feature C has CPU_FEATURE_PACKAGE_AFTER dependence with Feature B.
in this case an core level semaphore will be add between A and B, and
an package level semaphore will be add between B and C.

For a CPU has one package, two cores and 4 threads. Execute like below:

  Thread 1          Thread 2    .....     Thread 4
ReleaseSemaph(1,2)  -|
WaitForSemaph(1(2)) -|<-----------------------These two are Core Semaph
                  ReleaseSemaph(1,2) -|
                  WaitForSemaph(2)   -| <---  Core Semaph

ReleaseSemaph (1,2,3,4) -|
WaitForSemaph (1(4))    -| <----------------  Package Semaph

                                      ReleaseSemaph(3,4)
                                      WaitForSemaph(4(2)) <- Core Semaph

In above case, for thread 4, when it executes a core semaphore, i will
found WaitForSemaph(4(2)) is met because Thread 1 has execute a package
semaphore and ReleaseSemaph(4) for it before. This is not an expect
behavior. Thread 4 should wait for thread 3 to do this.

Fix this issue by separate the semaphore container for core level and
package level.

Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Ruiyu Ni <ruiyu.ni@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Eric Dong <eric.dong@intel.com>

Eric Dong (2):
  UefiCpuPkg/RegisterCpuFeaturesLib: Separate semaphore container.
  UefiCpuPkg/PiSmmCpuDxeSmm: Separate semaphore container.

 .../RegisterCpuFeaturesLib/CpuFeaturesInitialize.c  |  9 ++++++---
 .../RegisterCpuFeaturesLib/RegisterCpuFeatures.h    |  7 ++++---
 UefiCpuPkg/PiSmmCpuDxeSmm/CpuS3.c                   | 21 ++++++++++++++-------
 3 files changed, 24 insertions(+), 13 deletions(-)

-- 
2.15.0.windows.1

[Patch 1/2] UefiCpuPkg/RegisterCpuFeaturesLib: Separate semaphore container.

[Eric Dong]

In current implementation, core level semaphore use same container
with package level semaphore. This design will let the core level
semaphore not works as expected in below case:
1. Feature A has CPU_FEATURE_CORE_BEFORE dependence with Feature B.
2. Feature C has CPU_FEATURE_PACKAGE_AFTER dependence with Feature B.
in this case an core level semaphore will be add between A and B, and
an package level semaphore will be add between B and C.

For a CPU has one package, two cores and 4 threads. Execute like below:

  Thread 1          Thread 2    .....     Thread 4
ReleaseSemaph(1,2)  -|
WaitForSemaph(1(2)) -|<-----------------------These two are Core Semaph
                  ReleaseSemaph(1,2) -|
                  WaitForSemaph(2)   -| <---  Core Semaph

ReleaseSemaph (1,2,3,4) -|
WaitForSemaph (1(4))    -| <----------------  Package Semaph

                                      ReleaseSemaph(3,4)
                                      WaitForSemaph(4(2)) <- Core Semaph

In above case, for thread 4, when it executes a core semaphore, i will
found WaitForSemaph(4(2)) is met because Thread 1 has execute a package
semaphore and ReleaseSemaph(4) for it before. This is not an expect
behavior. Thread 4 should wait for thread 3 to do this.

Fix this issue by separate the semaphore container for core level and
package level.

Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Ruiyu Ni <ruiyu.ni@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Eric Dong <eric.dong@intel.com>
---
 .../Library/RegisterCpuFeaturesLib/CpuFeaturesInitialize.c       | 9 ++++++---
 UefiCpuPkg/Library/RegisterCpuFeaturesLib/RegisterCpuFeatures.h  | 7 ++++---
 2 files changed, 10 insertions(+), 6 deletions(-)

diff --git a/UefiCpuPkg/Library/RegisterCpuFeaturesLib/CpuFeaturesInitialize.c b/UefiCpuPkg/Library/RegisterCpuFeaturesLib/CpuFeaturesInitialize.c
index 7f208dbe6a..4bed0ce3a4 100644
--- a/UefiCpuPkg/Library/RegisterCpuFeaturesLib/CpuFeaturesInitialize.c
+++ b/UefiCpuPkg/Library/RegisterCpuFeaturesLib/CpuFeaturesInitialize.c
@@ -269,8 +269,10 @@ CpuInitDataInitialize (
     DEBUG ((DEBUG_INFO, "Package: %d, Valid Core : %d\n", Index, ValidCoreCountPerPackage[Index]));
   }
 
-  CpuFeaturesData->CpuFlags.SemaphoreCount = AllocateZeroPool (sizeof (UINT32) * CpuStatus->PackageCount * CpuStatus->MaxCoreCount * CpuStatus->MaxThreadCount);
-  ASSERT (CpuFeaturesData->CpuFlags.SemaphoreCount != NULL);
+  CpuFeaturesData->CpuFlags.CoreSemaphoreCount = AllocateZeroPool (sizeof (UINT32) * CpuStatus->PackageCount * CpuStatus->MaxCoreCount * CpuStatus->MaxThreadCount);
+  ASSERT (CpuFeaturesData->CpuFlags.CoreSemaphoreCount != NULL);
+  CpuFeaturesData->CpuFlags.PackageSemaphoreCount = AllocateZeroPool (sizeof (UINT32) * CpuStatus->PackageCount * CpuStatus->MaxCoreCount * CpuStatus->MaxThreadCount);
+  ASSERT (CpuFeaturesData->CpuFlags.PackageSemaphoreCount != NULL);
 
   //
   // Get support and configuration PCDs
@@ -933,9 +935,9 @@ ProgramProcessorRegister (
       //  V(0...n)       V(0...n)      ...           V(0...n)
       //  n * P(0)       n * P(1)      ...           n * P(n)
       //
-      SemaphorePtr = CpuFlags->SemaphoreCount;
       switch (RegisterTableEntry->Value) {
       case CoreDepType:
+        SemaphorePtr = CpuFlags->CoreSemaphoreCount;
         //
         // Get Offset info for the first thread in the core which current thread belongs to.
         //
@@ -956,6 +958,7 @@ ProgramProcessorRegister (
         break;
 
       case PackageDepType:
+        SemaphorePtr = CpuFlags->PackageSemaphoreCount;
         ValidCoreCountPerPackage = (UINT32 *)(UINTN)CpuStatus->ValidCoreCountPerPackage;
         //
         // Get Offset info for the first thread in the package which current thread belongs to.
diff --git a/UefiCpuPkg/Library/RegisterCpuFeaturesLib/RegisterCpuFeatures.h b/UefiCpuPkg/Library/RegisterCpuFeaturesLib/RegisterCpuFeatures.h
index b4c8ab777e..4898a80827 100644
--- a/UefiCpuPkg/Library/RegisterCpuFeaturesLib/RegisterCpuFeatures.h
+++ b/UefiCpuPkg/Library/RegisterCpuFeaturesLib/RegisterCpuFeatures.h
@@ -60,9 +60,10 @@ typedef struct {
 // Flags used when program the register.
 //
 typedef struct {
-  volatile UINTN           ConsoleLogLock;       // Spinlock used to control console.
-  volatile UINTN           MemoryMappedLock;     // Spinlock used to program mmio
-  volatile UINT32          *SemaphoreCount;      // Semaphore used to program semaphore.
+  volatile UINTN           ConsoleLogLock;          // Spinlock used to control console.
+  volatile UINTN           MemoryMappedLock;        // Spinlock used to program mmio
+  volatile UINT32          *CoreSemaphoreCount;     // Semaphore containers used to program Core semaphore.
+  volatile UINT32          *PackageSemaphoreCount;  // Semaphore containers used to program Package semaphore.
 } PROGRAM_CPU_REGISTER_FLAGS;
 
 typedef struct {
-- 
2.15.0.windows.1

[Patch 2/2] UefiCpuPkg/PiSmmCpuDxeSmm: Separate semaphore container.

[Eric Dong]

In current implementation, core level semaphore use same container
with package level semaphore. This design will let the core level
semaphore not works as expected in below case:
1. Feature A has CPU_FEATURE_CORE_BEFORE dependence with Feature B.
2. Feature C has CPU_FEATURE_PACKAGE_AFTER dependence with Feature B.
in this case an core level semaphore will be add between A and B, and
an package level semaphore will be add between B and C.

For a CPU has one package, two cores and 4 threads. Execute like below:

  Thread 1          Thread 2    .....     Thread 4
ReleaseSemaph(1,2)  -|
WaitForSemaph(1(2)) -|<-----------------------These two are Core Semaph
                  ReleaseSemaph(1,2) -|
                  WaitForSemaph(2)   -| <---  Core Semaph

ReleaseSemaph (1,2,3,4) -|
WaitForSemaph (1(4))    -| <----------------  Package Semaph

                                      ReleaseSemaph(3,4)
                                      WaitForSemaph(4(2)) <- Core Semaph

In above case, for thread 4, when it executes a core semaphore, i will
found WaitForSemaph(4(2)) is met because Thread 1 has execute a package
semaphore and ReleaseSemaph(4) for it before. This is not an expect
behavior. Thread 4 should wait for thread 3 to do this.

Fix this issue by separate the semaphore container for core level and
package level.

Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Ruiyu Ni <ruiyu.ni@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Eric Dong <eric.dong@intel.com>
---
 UefiCpuPkg/PiSmmCpuDxeSmm/CpuS3.c | 21 ++++++++++++++-------
 1 file changed, 14 insertions(+), 7 deletions(-)

diff --git a/UefiCpuPkg/PiSmmCpuDxeSmm/CpuS3.c b/UefiCpuPkg/PiSmmCpuDxeSmm/CpuS3.c
index a45e2dd3d7..65461485a4 100644
--- a/UefiCpuPkg/PiSmmCpuDxeSmm/CpuS3.c
+++ b/UefiCpuPkg/PiSmmCpuDxeSmm/CpuS3.c
@@ -41,9 +41,10 @@ typedef struct {
 // Flags used when program the register.
 //
 typedef struct {
-  volatile UINTN           ConsoleLogLock;       // Spinlock used to control console.
-  volatile UINTN           MemoryMappedLock;     // Spinlock used to program mmio
-  volatile UINT32          *SemaphoreCount;      // Semaphore used to program semaphore.
+  volatile UINTN           ConsoleLogLock;          // Spinlock used to control console.
+  volatile UINTN           MemoryMappedLock;        // Spinlock used to program mmio
+  volatile UINT32          *CoreSemaphoreCount;     // Semaphore used to program semaphore.
+  volatile UINT32          *PackageSemaphoreCount;  // Semaphore used to program semaphore.
 } PROGRAM_CPU_REGISTER_FLAGS;
 
 //
@@ -348,11 +349,12 @@ ProgramProcessorRegister (
       ASSERT (
         (ApLocation != NULL) &&
         (CpuStatus->ValidCoreCountPerPackage != 0) &&
-        (CpuFlags->SemaphoreCount) != NULL
+        (CpuFlags->CoreSemaphoreCount != NULL) &&
+        (CpuFlags->PackageSemaphoreCount != NULL)
         );
-      SemaphorePtr = CpuFlags->SemaphoreCount;
       switch (RegisterTableEntry->Value) {
       case CoreDepType:
+        SemaphorePtr = CpuFlags->CoreSemaphoreCount;
         //
         // Get Offset info for the first thread in the core which current thread belongs to.
         //
@@ -373,6 +375,7 @@ ProgramProcessorRegister (
         break;
 
       case PackageDepType:
+        SemaphorePtr = CpuFlags->PackageSemaphoreCount;
         ValidCoreCountPerPackage = (UINT32 *)(UINTN)CpuStatus->ValidCoreCountPerPackage;
         //
         // Get Offset info for the first thread in the package which current thread belongs to.
@@ -1037,10 +1040,14 @@ GetAcpiCpuData (
     ASSERT (mAcpiCpuData.ApLocation != 0);
   }
   if (CpuStatus->PackageCount != 0) {
-    mCpuFlags.SemaphoreCount = AllocateZeroPool (
+    mCpuFlags.CoreSemaphoreCount = AllocateZeroPool (
                                  sizeof (UINT32) * CpuStatus->PackageCount *
                                  CpuStatus->MaxCoreCount * CpuStatus->MaxThreadCount);
-    ASSERT (mCpuFlags.SemaphoreCount != NULL);
+    ASSERT (mCpuFlags.CoreSemaphoreCount != NULL);
+    mCpuFlags.PackageSemaphoreCount = AllocateZeroPool (
+                                 sizeof (UINT32) * CpuStatus->PackageCount *
+                                 CpuStatus->MaxCoreCount * CpuStatus->MaxThreadCount);
+    ASSERT (mCpuFlags.PackageSemaphoreCount != NULL);
   }
   InitializeSpinLock((SPIN_LOCK*) &mCpuFlags.MemoryMappedLock);
   InitializeSpinLock((SPIN_LOCK*) &mCpuFlags.ConsoleLogLock);
-- 
2.15.0.windows.1

Re: [Patch 2/2] UefiCpuPkg/PiSmmCpuDxeSmm: Separate semaphore container.

[Laszlo Ersek]

On 11/08/18 03:58, Eric Dong wrote:
> In current implementation, core level semaphore use same container
> with package level semaphore. This design will let the core level
> semaphore not works as expected in below case:
> 1. Feature A has CPU_FEATURE_CORE_BEFORE dependence with Feature B.
> 2. Feature C has CPU_FEATURE_PACKAGE_AFTER dependence with Feature B.
> in this case an core level semaphore will be add between A and B, and
> an package level semaphore will be add between B and C.
> 
> For a CPU has one package, two cores and 4 threads. Execute like below:
> 
>   Thread 1          Thread 2    .....     Thread 4
> ReleaseSemaph(1,2)  -|
> WaitForSemaph(1(2)) -|<-----------------------These two are Core Semaph
>                   ReleaseSemaph(1,2) -|
>                   WaitForSemaph(2)   -| <---  Core Semaph
> 
> ReleaseSemaph (1,2,3,4) -|
> WaitForSemaph (1(4))    -| <----------------  Package Semaph
> 
>                                       ReleaseSemaph(3,4)
>                                       WaitForSemaph(4(2)) <- Core Semaph
> 
> In above case, for thread 4, when it executes a core semaphore, i will
> found WaitForSemaph(4(2)) is met because Thread 1 has execute a package
> semaphore and ReleaseSemaph(4) for it before. This is not an expect
> behavior. Thread 4 should wait for thread 3 to do this.
> 
> Fix this issue by separate the semaphore container for core level and
> package level.
> 
> Cc: Laszlo Ersek <lersek@redhat.com>
> Cc: Ruiyu Ni <ruiyu.ni@intel.com>
> Contributed-under: TianoCore Contribution Agreement 1.1
> Signed-off-by: Eric Dong <eric.dong@intel.com>
> ---
>  UefiCpuPkg/PiSmmCpuDxeSmm/CpuS3.c | 21 ++++++++++++++-------
>  1 file changed, 14 insertions(+), 7 deletions(-)
> 
> diff --git a/UefiCpuPkg/PiSmmCpuDxeSmm/CpuS3.c b/UefiCpuPkg/PiSmmCpuDxeSmm/CpuS3.c
> index a45e2dd3d7..65461485a4 100644
> --- a/UefiCpuPkg/PiSmmCpuDxeSmm/CpuS3.c
> +++ b/UefiCpuPkg/PiSmmCpuDxeSmm/CpuS3.c
> @@ -41,9 +41,10 @@ typedef struct {
>  // Flags used when program the register.
>  //
>  typedef struct {
> -  volatile UINTN           ConsoleLogLock;       // Spinlock used to control console.
> -  volatile UINTN           MemoryMappedLock;     // Spinlock used to program mmio
> -  volatile UINT32          *SemaphoreCount;      // Semaphore used to program semaphore.
> +  volatile UINTN           ConsoleLogLock;          // Spinlock used to control console.
> +  volatile UINTN           MemoryMappedLock;        // Spinlock used to program mmio
> +  volatile UINT32          *CoreSemaphoreCount;     // Semaphore used to program semaphore.
> +  volatile UINT32          *PackageSemaphoreCount;  // Semaphore used to program semaphore.
>  } PROGRAM_CPU_REGISTER_FLAGS;
>  
>  //
> @@ -348,11 +349,12 @@ ProgramProcessorRegister (
>        ASSERT (
>          (ApLocation != NULL) &&
>          (CpuStatus->ValidCoreCountPerPackage != 0) &&
> -        (CpuFlags->SemaphoreCount) != NULL
> +        (CpuFlags->CoreSemaphoreCount != NULL) &&
> +        (CpuFlags->PackageSemaphoreCount != NULL)
>          );
> -      SemaphorePtr = CpuFlags->SemaphoreCount;
>        switch (RegisterTableEntry->Value) {
>        case CoreDepType:
> +        SemaphorePtr = CpuFlags->CoreSemaphoreCount;
>          //
>          // Get Offset info for the first thread in the core which current thread belongs to.
>          //
> @@ -373,6 +375,7 @@ ProgramProcessorRegister (
>          break;
>  
>        case PackageDepType:
> +        SemaphorePtr = CpuFlags->PackageSemaphoreCount;
>          ValidCoreCountPerPackage = (UINT32 *)(UINTN)CpuStatus->ValidCoreCountPerPackage;
>          //
>          // Get Offset info for the first thread in the package which current thread belongs to.
> @@ -1037,10 +1040,14 @@ GetAcpiCpuData (
>      ASSERT (mAcpiCpuData.ApLocation != 0);
>    }
>    if (CpuStatus->PackageCount != 0) {
> -    mCpuFlags.SemaphoreCount = AllocateZeroPool (
> +    mCpuFlags.CoreSemaphoreCount = AllocateZeroPool (
>                                   sizeof (UINT32) * CpuStatus->PackageCount *
>                                   CpuStatus->MaxCoreCount * CpuStatus->MaxThreadCount);
> -    ASSERT (mCpuFlags.SemaphoreCount != NULL);
> +    ASSERT (mCpuFlags.CoreSemaphoreCount != NULL);
> +    mCpuFlags.PackageSemaphoreCount = AllocateZeroPool (
> +                                 sizeof (UINT32) * CpuStatus->PackageCount *
> +                                 CpuStatus->MaxCoreCount * CpuStatus->MaxThreadCount);
> +    ASSERT (mCpuFlags.PackageSemaphoreCount != NULL);
>    }
>    InitializeSpinLock((SPIN_LOCK*) &mCpuFlags.MemoryMappedLock);
>    InitializeSpinLock((SPIN_LOCK*) &mCpuFlags.ConsoleLogLock);
> 

The patch looks OK, superficially speaking.

Also this looks like a bugfix to a new feature already committed in this
development cycle, so I think it may go in during the hard feature freeze.

I have some requests (no need to repost):

(1) Please make sure there is a TianoCore BZ for this issue.

(2) Please reference said BZ in the commit message.

(For example, commit c60d36b4d1, for
<https://bugzilla.tianocore.org/show_bug.cgi?id=1307> missed the
reference to BZ#1307.)

(3) Before pushing, please fix up the indentation of the
AllocateZeroPool() arguments (both calls).

(4) Can you please file the bugzilla now about unifying the
implementation between RegisterCpuFeaturesLib and PiSmmCpuDxeSmm?

(That's for the next development cycle, but we should report the BZ for
it at this point, I believe.)

Thanks!
Laszlo

Re: [Patch 2/2] UefiCpuPkg/PiSmmCpuDxeSmm: Separate semaphore container.

[Laszlo Ersek]

On 11/08/18 14:33, Laszlo Ersek wrote:
> On 11/08/18 03:58, Eric Dong wrote:
>> In current implementation, core level semaphore use same container
>> with package level semaphore. This design will let the core level
>> semaphore not works as expected in below case:
>> 1. Feature A has CPU_FEATURE_CORE_BEFORE dependence with Feature B.
>> 2. Feature C has CPU_FEATURE_PACKAGE_AFTER dependence with Feature B.
>> in this case an core level semaphore will be add between A and B, and
>> an package level semaphore will be add between B and C.
>>
>> For a CPU has one package, two cores and 4 threads. Execute like below:
>>
>>   Thread 1          Thread 2    .....     Thread 4
>> ReleaseSemaph(1,2)  -|
>> WaitForSemaph(1(2)) -|<-----------------------These two are Core Semaph
>>                   ReleaseSemaph(1,2) -|
>>                   WaitForSemaph(2)   -| <---  Core Semaph
>>
>> ReleaseSemaph (1,2,3,4) -|
>> WaitForSemaph (1(4))    -| <----------------  Package Semaph
>>
>>                                       ReleaseSemaph(3,4)
>>                                       WaitForSemaph(4(2)) <- Core Semaph
>>
>> In above case, for thread 4, when it executes a core semaphore, i will
>> found WaitForSemaph(4(2)) is met because Thread 1 has execute a package
>> semaphore and ReleaseSemaph(4) for it before. This is not an expect
>> behavior. Thread 4 should wait for thread 3 to do this.
>>
>> Fix this issue by separate the semaphore container for core level and
>> package level.
>>
>> Cc: Laszlo Ersek <lersek@redhat.com>
>> Cc: Ruiyu Ni <ruiyu.ni@intel.com>
>> Contributed-under: TianoCore Contribution Agreement 1.1
>> Signed-off-by: Eric Dong <eric.dong@intel.com>
>> ---
>>  UefiCpuPkg/PiSmmCpuDxeSmm/CpuS3.c | 21 ++++++++++++++-------
>>  1 file changed, 14 insertions(+), 7 deletions(-)
>>
>> diff --git a/UefiCpuPkg/PiSmmCpuDxeSmm/CpuS3.c b/UefiCpuPkg/PiSmmCpuDxeSmm/CpuS3.c
>> index a45e2dd3d7..65461485a4 100644
>> --- a/UefiCpuPkg/PiSmmCpuDxeSmm/CpuS3.c
>> +++ b/UefiCpuPkg/PiSmmCpuDxeSmm/CpuS3.c
>> @@ -41,9 +41,10 @@ typedef struct {
>>  // Flags used when program the register.
>>  //
>>  typedef struct {
>> -  volatile UINTN           ConsoleLogLock;       // Spinlock used to control console.
>> -  volatile UINTN           MemoryMappedLock;     // Spinlock used to program mmio
>> -  volatile UINT32          *SemaphoreCount;      // Semaphore used to program semaphore.
>> +  volatile UINTN           ConsoleLogLock;          // Spinlock used to control console.
>> +  volatile UINTN           MemoryMappedLock;        // Spinlock used to program mmio
>> +  volatile UINT32          *CoreSemaphoreCount;     // Semaphore used to program semaphore.
>> +  volatile UINT32          *PackageSemaphoreCount;  // Semaphore used to program semaphore.
>>  } PROGRAM_CPU_REGISTER_FLAGS;
>>  
>>  //
>> @@ -348,11 +349,12 @@ ProgramProcessorRegister (
>>        ASSERT (
>>          (ApLocation != NULL) &&
>>          (CpuStatus->ValidCoreCountPerPackage != 0) &&
>> -        (CpuFlags->SemaphoreCount) != NULL
>> +        (CpuFlags->CoreSemaphoreCount != NULL) &&
>> +        (CpuFlags->PackageSemaphoreCount != NULL)
>>          );
>> -      SemaphorePtr = CpuFlags->SemaphoreCount;
>>        switch (RegisterTableEntry->Value) {
>>        case CoreDepType:
>> +        SemaphorePtr = CpuFlags->CoreSemaphoreCount;
>>          //
>>          // Get Offset info for the first thread in the core which current thread belongs to.
>>          //
>> @@ -373,6 +375,7 @@ ProgramProcessorRegister (
>>          break;
>>  
>>        case PackageDepType:
>> +        SemaphorePtr = CpuFlags->PackageSemaphoreCount;
>>          ValidCoreCountPerPackage = (UINT32 *)(UINTN)CpuStatus->ValidCoreCountPerPackage;
>>          //
>>          // Get Offset info for the first thread in the package which current thread belongs to.
>> @@ -1037,10 +1040,14 @@ GetAcpiCpuData (
>>      ASSERT (mAcpiCpuData.ApLocation != 0);
>>    }
>>    if (CpuStatus->PackageCount != 0) {
>> -    mCpuFlags.SemaphoreCount = AllocateZeroPool (
>> +    mCpuFlags.CoreSemaphoreCount = AllocateZeroPool (
>>                                   sizeof (UINT32) * CpuStatus->PackageCount *
>>                                   CpuStatus->MaxCoreCount * CpuStatus->MaxThreadCount);
>> -    ASSERT (mCpuFlags.SemaphoreCount != NULL);
>> +    ASSERT (mCpuFlags.CoreSemaphoreCount != NULL);
>> +    mCpuFlags.PackageSemaphoreCount = AllocateZeroPool (
>> +                                 sizeof (UINT32) * CpuStatus->PackageCount *
>> +                                 CpuStatus->MaxCoreCount * CpuStatus->MaxThreadCount);
>> +    ASSERT (mCpuFlags.PackageSemaphoreCount != NULL);
>>    }
>>    InitializeSpinLock((SPIN_LOCK*) &mCpuFlags.MemoryMappedLock);
>>    InitializeSpinLock((SPIN_LOCK*) &mCpuFlags.ConsoleLogLock);
>>
> 
> The patch looks OK, superficially speaking.
> 
> Also this looks like a bugfix to a new feature already committed in this
> development cycle, so I think it may go in during the hard feature freeze.
> 
> I have some requests (no need to repost):
> 
> (1) Please make sure there is a TianoCore BZ for this issue.
> 
> (2) Please reference said BZ in the commit message.
> 
> (For example, commit c60d36b4d1, for
> <https://bugzilla.tianocore.org/show_bug.cgi?id=1307> missed the
> reference to BZ#1307.)
> 
> (3) Before pushing, please fix up the indentation of the
> AllocateZeroPool() arguments (both calls).
> 
> (4) Can you please file the bugzilla now about unifying the
> implementation between RegisterCpuFeaturesLib and PiSmmCpuDxeSmm?
> 
> (That's for the next development cycle, but we should report the BZ for
> it at this point, I believe.)

Oh, sorry, I thought I had added my ACK, but it turns out I forgot about
it. So, with the above points,

Acked-by: Laszlo Ersek <lersek@redhat.com>

Thanks
Laszlo

Re: [Patch 2/2] UefiCpuPkg/PiSmmCpuDxeSmm: Separate semaphore container.

[Dong, Eric]

Hi Laszlo,


> -----Original Message-----
> From: Laszlo Ersek [mailto:lersek@redhat.com]
> Sent: Friday, November 9, 2018 1:51 AM
> To: Dong, Eric <eric.dong@intel.com>; edk2-devel@lists.01.org
> Cc: Ni, Ruiyu <ruiyu.ni@intel.com>
> Subject: Re: [edk2] [Patch 2/2] UefiCpuPkg/PiSmmCpuDxeSmm: Separate
> semaphore container.
> 
> On 11/08/18 14:33, Laszlo Ersek wrote:
> > On 11/08/18 03:58, Eric Dong wrote:
> >> In current implementation, core level semaphore use same container
> >> with package level semaphore. This design will let the core level
> >> semaphore not works as expected in below case:
> >> 1. Feature A has CPU_FEATURE_CORE_BEFORE dependence with Feature
> B.
> >> 2. Feature C has CPU_FEATURE_PACKAGE_AFTER dependence with
> Feature B.
> >> in this case an core level semaphore will be add between A and B, and
> >> an package level semaphore will be add between B and C.
> >>
> >> For a CPU has one package, two cores and 4 threads. Execute like below:
> >>
> >>   Thread 1          Thread 2    .....     Thread 4
> >> ReleaseSemaph(1,2)  -|
> >> WaitForSemaph(1(2)) -|<-----------------------These two are Core Semaph
> >>                   ReleaseSemaph(1,2) -|
> >>                   WaitForSemaph(2)   -| <---  Core Semaph
> >>
> >> ReleaseSemaph (1,2,3,4) -|
> >> WaitForSemaph (1(4))    -| <----------------  Package Semaph
> >>
> >>                                       ReleaseSemaph(3,4)
> >>                                       WaitForSemaph(4(2)) <- Core
> >> Semaph
> >>
> >> In above case, for thread 4, when it executes a core semaphore, i
> >> will found WaitForSemaph(4(2)) is met because Thread 1 has execute a
> >> package semaphore and ReleaseSemaph(4) for it before. This is not an
> >> expect behavior. Thread 4 should wait for thread 3 to do this.
> >>
> >> Fix this issue by separate the semaphore container for core level and
> >> package level.
> >>
> >> Cc: Laszlo Ersek <lersek@redhat.com>
> >> Cc: Ruiyu Ni <ruiyu.ni@intel.com>
> >> Contributed-under: TianoCore Contribution Agreement 1.1
> >> Signed-off-by: Eric Dong <eric.dong@intel.com>
> >> ---
> >>  UefiCpuPkg/PiSmmCpuDxeSmm/CpuS3.c | 21 ++++++++++++++-------
> >>  1 file changed, 14 insertions(+), 7 deletions(-)
> >>
> >> diff --git a/UefiCpuPkg/PiSmmCpuDxeSmm/CpuS3.c
> >> b/UefiCpuPkg/PiSmmCpuDxeSmm/CpuS3.c
> >> index a45e2dd3d7..65461485a4 100644
> >> --- a/UefiCpuPkg/PiSmmCpuDxeSmm/CpuS3.c
> >> +++ b/UefiCpuPkg/PiSmmCpuDxeSmm/CpuS3.c
> >> @@ -41,9 +41,10 @@ typedef struct {
> >>  // Flags used when program the register.
> >>  //
> >>  typedef struct {
> >> -  volatile UINTN           ConsoleLogLock;       // Spinlock used to control
> console.
> >> -  volatile UINTN           MemoryMappedLock;     // Spinlock used to
> program mmio
> >> -  volatile UINT32          *SemaphoreCount;      // Semaphore used to
> program semaphore.
> >> +  volatile UINTN           ConsoleLogLock;          // Spinlock used to control
> console.
> >> +  volatile UINTN           MemoryMappedLock;        // Spinlock used to
> program mmio
> >> +  volatile UINT32          *CoreSemaphoreCount;     // Semaphore used to
> program semaphore.
> >> +  volatile UINT32          *PackageSemaphoreCount;  // Semaphore used to
> program semaphore.
> >>  } PROGRAM_CPU_REGISTER_FLAGS;
> >>
> >>  //
> >> @@ -348,11 +349,12 @@ ProgramProcessorRegister (
> >>        ASSERT (
> >>          (ApLocation != NULL) &&
> >>          (CpuStatus->ValidCoreCountPerPackage != 0) &&
> >> -        (CpuFlags->SemaphoreCount) != NULL
> >> +        (CpuFlags->CoreSemaphoreCount != NULL) &&
> >> +        (CpuFlags->PackageSemaphoreCount != NULL)
> >>          );
> >> -      SemaphorePtr = CpuFlags->SemaphoreCount;
> >>        switch (RegisterTableEntry->Value) {
> >>        case CoreDepType:
> >> +        SemaphorePtr = CpuFlags->CoreSemaphoreCount;
> >>          //
> >>          // Get Offset info for the first thread in the core which current thread
> belongs to.
> >>          //
> >> @@ -373,6 +375,7 @@ ProgramProcessorRegister (
> >>          break;
> >>
> >>        case PackageDepType:
> >> +        SemaphorePtr = CpuFlags->PackageSemaphoreCount;
> >>          ValidCoreCountPerPackage = (UINT32 *)(UINTN)CpuStatus-
> >ValidCoreCountPerPackage;
> >>          //
> >>          // Get Offset info for the first thread in the package which current
> thread belongs to.
> >> @@ -1037,10 +1040,14 @@ GetAcpiCpuData (
> >>      ASSERT (mAcpiCpuData.ApLocation != 0);
> >>    }
> >>    if (CpuStatus->PackageCount != 0) {
> >> -    mCpuFlags.SemaphoreCount = AllocateZeroPool (
> >> +    mCpuFlags.CoreSemaphoreCount = AllocateZeroPool (
> >>                                   sizeof (UINT32) * CpuStatus->PackageCount *
> >>                                   CpuStatus->MaxCoreCount * CpuStatus-
> >MaxThreadCount);
> >> -    ASSERT (mCpuFlags.SemaphoreCount != NULL);
> >> +    ASSERT (mCpuFlags.CoreSemaphoreCount != NULL);
> >> +    mCpuFlags.PackageSemaphoreCount = AllocateZeroPool (
> >> +                                 sizeof (UINT32) * CpuStatus->PackageCount *
> >> +                                 CpuStatus->MaxCoreCount * CpuStatus-
> >MaxThreadCount);
> >> +    ASSERT (mCpuFlags.PackageSemaphoreCount != NULL);
> >>    }
> >>    InitializeSpinLock((SPIN_LOCK*) &mCpuFlags.MemoryMappedLock);
> >>    InitializeSpinLock((SPIN_LOCK*) &mCpuFlags.ConsoleLogLock);
> >>
> >
> > The patch looks OK, superficially speaking.
> >
> > Also this looks like a bugfix to a new feature already committed in
> > this development cycle, so I think it may go in during the hard feature
> freeze.
> >
> > I have some requests (no need to repost):
> >
> > (1) Please make sure there is a TianoCore BZ for this issue.
> >
> > (2) Please reference said BZ in the commit message.

Bugz link: https://bugzilla.tianocore.org/show_bug.cgi?id=1311
Will include it when I push the change.

> >
> > (For example, commit c60d36b4d1, for
> > <https://bugzilla.tianocore.org/show_bug.cgi?id=1307> missed the
> > reference to BZ#1307.)
> >
> > (3) Before pushing, please fix up the indentation of the
> > AllocateZeroPool() arguments (both calls).

Will adjust it when I push the change.

> >
> > (4) Can you please file the bugzilla now about unifying the
> > implementation between RegisterCpuFeaturesLib and PiSmmCpuDxeSmm?
> >

File Bugz for it, link: https://bugzilla.tianocore.org/show_bug.cgi?id=1313

Thanks,
Eric
> > (That's for the next development cycle, but we should report the BZ
> > for it at this point, I believe.)
> 
> Oh, sorry, I thought I had added my ACK, but it turns out I forgot about it. So,
> with the above points,
> 
> Acked-by: Laszlo Ersek <lersek@redhat.com>
> 
> Thanks
> Laszlo

Re: [Patch 1/2] UefiCpuPkg/RegisterCpuFeaturesLib: Separate semaphore container.

[Ni, Ruiyu]

Eric,
Patch is good.
Reviewed-by: Ruiyu Ni <ruiyu.ni@intel.com>

Some modification to commit message to help understanding the changes.
Please rewording if you think some of them is not proper.

You can push the changes if no concerns about the new commit message.

-----

In current implementation, core and package level sync uses same 
semaphores. Sharing the semaphore may cause wrong execution order.
For example:
1. Feature A has CPU_FEATURE_CORE_BEFORE dependency with Feature B.
2. Feature C has CPU_FEATURE_PACKAGE_AFTER dependency with Feature B.
The expected feature initialization order is A B C:
A ---- (Core Depends) ----> B ---- (Package Depends) ----> C

For a CPU has 1 package, 2 cores and 4 threads. The feature 
initialization order may like below:

   Thread#1             Thread#2    Thread#3   Thread#4
   [A.Init]             [A.Init]               [A.Init]
Release(S1, S2)        Release(S1, S2)        Release(S3, S4)
Wait(S1) * 2           Wait(S2) * 2                        <- Core sync
   [B.Init]             [B.Init]
Release (S1,S2,S3,S4)
Wait (S1) * 4  <------------------------------------------ Package sync
                                               Wait(S4 * 2) <- Core sync
                                               [B.Init]

In above case, for thread#4, when it syncs in core level, Wait(S4) * 2 
isn't blocked and [B.Init] runs. But [A.Init] hasn't run in thread#3. 
It's wrong! Thread#4 should execute [B.Init] after thread#3 executes 
[A.Init] because B core level depends on A.

The reason is of the wrong execution order is that S4 is released in 
thread#1 by calling Release (S1, S2, S3, S4) and in thread #4 by calling 
Release (S3, S4).

To fix this issue, core level sync and package level sync should use 
separate semaphores.

In above example, the S4 released in Release (S1, S2, S3, S4) should not 
be the same semaphore as that in Release (S3, S4).

Thanks,
Ray

Re: [Patch 2/2] UefiCpuPkg/PiSmmCpuDxeSmm: Separate semaphore container.

[Ni, Ruiyu]

Eric,
Similar comments suggestion to this patch.
Reviewed-by: Ruiyu Ni <ruiyu.ni@intel.com>

-- 
Thanks,
Ray

Re: [Patch 0/2] Separate semaphore container.

[Dong, Eric]

Hi Stewards:
 
Since this is a bug fix, and the risk for this release is small. I plan to push this serial changes before edk2-stable201811 tag.

If you have any concern, please raise here. 

Thanks,
Eric

> -----Original Message-----
> From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of Eric
> Dong
> Sent: Thursday, November 8, 2018 10:58 AM
> To: edk2-devel@lists.01.org
> Cc: Ni, Ruiyu <ruiyu.ni@intel.com>; Laszlo Ersek <lersek@redhat.com>
> Subject: [edk2] [Patch 0/2] Separate semaphore container.
> 
> In current implementation, core level semaphore use same container with
> package level semaphore. This design will let the core level semaphore not
> works as expected in below case:
> 1. Feature A has CPU_FEATURE_CORE_BEFORE dependence with Feature B.
> 2. Feature C has CPU_FEATURE_PACKAGE_AFTER dependence with Feature B.
> in this case an core level semaphore will be add between A and B, and an
> package level semaphore will be add between B and C.
> 
> For a CPU has one package, two cores and 4 threads. Execute like below:
> 
>   Thread 1          Thread 2    .....     Thread 4
> ReleaseSemaph(1,2)  -|
> WaitForSemaph(1(2)) -|<-----------------------These two are Core Semaph
>                   ReleaseSemaph(1,2) -|
>                   WaitForSemaph(2)   -| <---  Core Semaph
> 
> ReleaseSemaph (1,2,3,4) -|
> WaitForSemaph (1(4))    -| <----------------  Package Semaph
> 
>                                       ReleaseSemaph(3,4)
>                                       WaitForSemaph(4(2)) <- Core Semaph
> 
> In above case, for thread 4, when it executes a core semaphore, i will found
> WaitForSemaph(4(2)) is met because Thread 1 has execute a package
> semaphore and ReleaseSemaph(4) for it before. This is not an expect behavior.
> Thread 4 should wait for thread 3 to do this.
> 
> Fix this issue by separate the semaphore container for core level and package
> level.
> 
> Cc: Laszlo Ersek <lersek@redhat.com>
> Cc: Ruiyu Ni <ruiyu.ni@intel.com>
> Contributed-under: TianoCore Contribution Agreement 1.1
> Signed-off-by: Eric Dong <eric.dong@intel.com>
> 
> Eric Dong (2):
>   UefiCpuPkg/RegisterCpuFeaturesLib: Separate semaphore container.
>   UefiCpuPkg/PiSmmCpuDxeSmm: Separate semaphore container.
> 
>  .../RegisterCpuFeaturesLib/CpuFeaturesInitialize.c  |  9 ++++++---
>  .../RegisterCpuFeaturesLib/RegisterCpuFeatures.h    |  7 ++++---
>  UefiCpuPkg/PiSmmCpuDxeSmm/CpuS3.c                   | 21 ++++++++++++++-------
>  3 files changed, 24 insertions(+), 13 deletions(-)
> 
> --
> 2.15.0.windows.1
> 
> _______________________________________________
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel

Re: [Patch 0/2] Separate semaphore container.

[Laszlo Ersek]

On 11/10/18 04:19, Dong, Eric wrote:
> Hi Stewards:
>  
> Since this is a bug fix, and the risk for this release is small. I plan to push this serial changes before edk2-stable201811 tag.
> 
> If you have any concern, please raise here. 

I'm fine with pushing this, as I explained elsewhere in this thread:

https://www.mail-archive.com/edk2-devel@lists.01.org/msg46898.html

"Also this looks like a bugfix to a new feature already committed in
this development cycle, so I think it may go in during the hard feature
freeze."

Thanks
Laszlo

>> -----Original Message-----
>> From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of Eric
>> Dong
>> Sent: Thursday, November 8, 2018 10:58 AM
>> To: edk2-devel@lists.01.org
>> Cc: Ni, Ruiyu <ruiyu.ni@intel.com>; Laszlo Ersek <lersek@redhat.com>
>> Subject: [edk2] [Patch 0/2] Separate semaphore container.
>>
>> In current implementation, core level semaphore use same container with
>> package level semaphore. This design will let the core level semaphore not
>> works as expected in below case:
>> 1. Feature A has CPU_FEATURE_CORE_BEFORE dependence with Feature B.
>> 2. Feature C has CPU_FEATURE_PACKAGE_AFTER dependence with Feature B.
>> in this case an core level semaphore will be add between A and B, and an
>> package level semaphore will be add between B and C.
>>
>> For a CPU has one package, two cores and 4 threads. Execute like below:
>>
>>   Thread 1          Thread 2    .....     Thread 4
>> ReleaseSemaph(1,2)  -|
>> WaitForSemaph(1(2)) -|<-----------------------These two are Core Semaph
>>                   ReleaseSemaph(1,2) -|
>>                   WaitForSemaph(2)   -| <---  Core Semaph
>>
>> ReleaseSemaph (1,2,3,4) -|
>> WaitForSemaph (1(4))    -| <----------------  Package Semaph
>>
>>                                       ReleaseSemaph(3,4)
>>                                       WaitForSemaph(4(2)) <- Core Semaph
>>
>> In above case, for thread 4, when it executes a core semaphore, i will found
>> WaitForSemaph(4(2)) is met because Thread 1 has execute a package
>> semaphore and ReleaseSemaph(4) for it before. This is not an expect behavior.
>> Thread 4 should wait for thread 3 to do this.
>>
>> Fix this issue by separate the semaphore container for core level and package
>> level.
>>
>> Cc: Laszlo Ersek <lersek@redhat.com>
>> Cc: Ruiyu Ni <ruiyu.ni@intel.com>
>> Contributed-under: TianoCore Contribution Agreement 1.1
>> Signed-off-by: Eric Dong <eric.dong@intel.com>
>>
>> Eric Dong (2):
>>   UefiCpuPkg/RegisterCpuFeaturesLib: Separate semaphore container.
>>   UefiCpuPkg/PiSmmCpuDxeSmm: Separate semaphore container.
>>
>>  .../RegisterCpuFeaturesLib/CpuFeaturesInitialize.c  |  9 ++++++---
>>  .../RegisterCpuFeaturesLib/RegisterCpuFeatures.h    |  7 ++++---
>>  UefiCpuPkg/PiSmmCpuDxeSmm/CpuS3.c                   | 21 ++++++++++++++-------
>>  3 files changed, 24 insertions(+), 13 deletions(-)
>>
>> --
>> 2.15.0.windows.1
>>
>> _______________________________________________
>> edk2-devel mailing list
>> edk2-devel@lists.01.org
>> https://lists.01.org/mailman/listinfo/edk2-devel