SCT Test crashes After HTTPS boot success.

SCT Test crashes After HTTPS boot success.

[Karunakar P]

Hello All,

When I do SCT test, get the below failure
GenericTest\EFICompliantTest 0 0 CB6F7B77-0B15-43F7-A95B-8C7F9FD70B21 FAIL UEFI Compliant - TLS support is required

The reason for the failure is that the platform doesn't have TLS related protocols installed.

->if the platform supports TLS feature then, EFI_TLS_PROTOCOL, EFI_TLS_SERVICE_BINDING_PROTOCOL, EFI_TLS_CONFIGURATION_PROTOCOL must be existed.
-> According UEFI2.7 Spec - 28.10 EFI TLS Protocols(page-1787)
The TLS consumer need locate EFI_TLS_SERVICE_BINDING_PROTOCOL and call CreateChild() to create a new child of EFI_TLS_PROTOCOL instance. Then use EFI_TLS_PROTOCOL to start TLS session. After use, the TLS consumer need call DestroyChild() to destroy it.
-> Network Stack follows same in HTTPS boot.
While doing IPv4/6 HTTPS boot , will locate gEfiTlsServiceBindingProtocolGuid and call TlsServiceBindingCreateChild, So that EFI_TLS_PROTOCOL and EFI_TLS_CONFIGURATION_PROTOCOL will be installed.

So once HTTPS boot is success then TLS supported protocols exist.

And if we do SCT test after HTTPS boot is success, Then TLS related failures should NOT happen in SCT.

I've tried SCT test after HTTPS boot success, But SCT test Crashes.

[Steps to reproduce]

1.       Configure the HTTPS Server with EFI Shell as NBP file.

2.       Connect test machine and HTTPS server with LAN cable.

3.       Perform HTTPS boot in test machine

4.       Once HTTPS boot is success, It will launch Shell.

5.       Run SCT test


[Observations]

1.       SCT test was crashed and unable to continue the test.

Could you please provide your comments/Suggestion on this?


Thanks,
Karunakar

Re: SCT Test crashes After HTTPS boot success.

[Wu, Jiaxin]

Hi Karunakar,

Can you provide more detailed info for the SCT test steps? The crash can happen after "Sct.efi -u" or need run any specific test case?

Thanks,
Jiaxin

From: Karunakar P [mailto:karunakarp@amiindia.co.in]
Sent: Thursday, November 16, 2017 4:48 PM
To: 'edk2-devel@lists.01.org' <edk2-devel@lists.01.org>
Cc: Wu, Jiaxin <jiaxin.wu@intel.com>; Fu, Siyuan <siyuan.fu@intel.com>; Ye, Ting <ting.ye@intel.com>
Subject: SCT Test crashes After HTTPS boot success.

Hello All,

When I do SCT test, get the below failure
GenericTest\EFICompliantTest 0 0 CB6F7B77-0B15-43F7-A95B-8C7F9FD70B21 FAIL UEFI Compliant - TLS support is required

The reason for the failure is that the platform doesn't have TLS related protocols installed.

->if the platform supports TLS feature then, EFI_TLS_PROTOCOL, EFI_TLS_SERVICE_BINDING_PROTOCOL, EFI_TLS_CONFIGURATION_PROTOCOL must be existed.
-> According UEFI2.7 Spec - 28.10 EFI TLS Protocols(page-1787)
The TLS consumer need locate EFI_TLS_SERVICE_BINDING_PROTOCOL and call CreateChild() to create a new child of EFI_TLS_PROTOCOL instance. Then use EFI_TLS_PROTOCOL to start TLS session. After use, the TLS consumer need call DestroyChild() to destroy it.
-> Network Stack follows same in HTTPS boot.
While doing IPv4/6 HTTPS boot , will locate gEfiTlsServiceBindingProtocolGuid and call TlsServiceBindingCreateChild, So that EFI_TLS_PROTOCOL and EFI_TLS_CONFIGURATION_PROTOCOL will be installed.

So once HTTPS boot is success then TLS supported protocols exist.

And if we do SCT test after HTTPS boot is success, Then TLS related failures should NOT happen in SCT.

I've tried SCT test after HTTPS boot success, But SCT test Crashes.

[Steps to reproduce]

1.       Configure the HTTPS Server with EFI Shell as NBP file.

2.       Connect test machine and HTTPS server with LAN cable.

3.       Perform HTTPS boot in test machine

4.       Once HTTPS boot is success, It will launch Shell.

5.       Run SCT test


[Observations]

1.       SCT test was crashed and unable to continue the test.

Could you please provide your comments/Suggestion on this?


Thanks,
Karunakar

Re: SCT Test crashes After HTTPS boot success.

[Karunakar P]

[-- Attachment #1: Type: text/plain, Size: 2923 bytes --]

Hi Jiaxin,

Below are the detailed steps for SCT test

[Steps to reproduce]
.
  5. Run SCT test

    a.  Execute the following commands to run SCT test on SUT
             sct -r

  sct -u
   b.  In Test Case Management page, Enable GenericTest
   c.  Press F9 to run the selected test case

[Observation]
synchronous exception occurred in SCT ,Attached the Log for reference.

Thanks,
Karunakar

From: Wu, Jiaxin [mailto:jiaxin.wu@intel.com]
Sent: Friday, November 17, 2017 6:01 AM
To: Karunakar P; 'edk2-devel@lists.01.org'
Cc: Fu, Siyuan; Ye, Ting
Subject: RE: SCT Test crashes After HTTPS boot success.

Hi Karunakar,

Can you provide more detailed info for the SCT test steps? The crash can happen after "Sct.efi -u" or need run any specific test case?

Thanks,
Jiaxin

From: Karunakar P [mailto:karunakarp@amiindia.co.in]
Sent: Thursday, November 16, 2017 4:48 PM
To: 'edk2-devel@lists.01.org' <edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>>
Cc: Wu, Jiaxin <jiaxin.wu@intel.com<mailto:jiaxin.wu@intel.com>>; Fu, Siyuan <siyuan.fu@intel.com<mailto:siyuan.fu@intel.com>>; Ye, Ting <ting.ye@intel.com<mailto:ting.ye@intel.com>>
Subject: SCT Test crashes After HTTPS boot success.

Hello All,

When I do SCT test, get the below failure
GenericTest\EFICompliantTest 0 0 CB6F7B77-0B15-43F7-A95B-8C7F9FD70B21 FAIL UEFI Compliant - TLS support is required

The reason for the failure is that the platform doesn't have TLS related protocols installed.

->if the platform supports TLS feature then, EFI_TLS_PROTOCOL, EFI_TLS_SERVICE_BINDING_PROTOCOL, EFI_TLS_CONFIGURATION_PROTOCOL must be existed.
-> According UEFI2.7 Spec - 28.10 EFI TLS Protocols(page-1787)
The TLS consumer need locate EFI_TLS_SERVICE_BINDING_PROTOCOL and call CreateChild() to create a new child of EFI_TLS_PROTOCOL instance. Then use EFI_TLS_PROTOCOL to start TLS session. After use, the TLS consumer need call DestroyChild() to destroy it.
-> Network Stack follows same in HTTPS boot.
While doing IPv4/6 HTTPS boot , will locate gEfiTlsServiceBindingProtocolGuid and call TlsServiceBindingCreateChild, So that EFI_TLS_PROTOCOL and EFI_TLS_CONFIGURATION_PROTOCOL will be installed.

So once HTTPS boot is success then TLS supported protocols exist.

And if we do SCT test after HTTPS boot is success, Then TLS related failures should NOT happen in SCT.

I've tried SCT test after HTTPS boot success, But SCT test Crashes.

[Steps to reproduce]

1.       Configure the HTTPS Server with EFI Shell as NBP file.

2.       Connect test machine and HTTPS server with LAN cable.

3.       Perform HTTPS boot in test machine

4.       Once HTTPS boot is success, It will launch Shell.

5.       Run SCT test


[Observations]

1.       SCT test was crashed and unable to continue the test.

Could you please provide your comments/Suggestion on this?


Thanks,
Karunakar

[-- Attachment #2: SCTHandLog.txt --]
[-- Type: text/plain, Size: 997 bytes --]


------------------------------------------------------------
      *********RECOVERY*********
------------------------------------------------------------
System hangs or stops abnormally. -- FAILURE
DE687A18-0BBD-4396-8509-498FF23234F1
/home/supven01/upstream_edk2/edk2/SctPkg/TestInfrastructure/SCT/Framework/Execute/Execute.c:2104


Returned Status Code: Unsupported

PlatformSpecificElements: [FAILED]
  Passes........... 18
  Warnings......... 0
  Errors........... 8
------------------------------------------------------------
UEFI 2.6
Revision 0x00010001
Test Entry Point GUID: A0A8BED3-3D6F-4AD8-907A-84D52EE1543B
------------------------------------------------------------
Logfile: "\UEFISCT_2.6\UEFISCT\SctPackageAARCH64\AARCH64\Log\GenericTest\EFICompliantTest0\PlatformSpecificElements_0_0_A0A8BED3-3D6F-4AD8-907A-84D52EE1543B.log"
Test Finished: 09/28/80  12:41p
Elapsed Time: 00 Days 00:00:02

------------------------------------------------------------

Re: SCT Test crashes After HTTPS boot success.

[Wu, Jiaxin]

I will try to reproduce the issue with EDK2 trunk code, then feedback to you. Thanks the report.

Jiaxin

From: Karunakar P [mailto:karunakarp@amiindia.co.in]
Sent: Friday, November 17, 2017 2:32 PM
To: Wu, Jiaxin <jiaxin.wu@intel.com>; 'edk2-devel@lists.01.org' <edk2-devel@lists.01.org>
Cc: Fu, Siyuan <siyuan.fu@intel.com>; Ye, Ting <ting.ye@intel.com>
Subject: RE: SCT Test crashes After HTTPS boot success.

Hi Jiaxin,

Below are the detailed steps for SCT test

[Steps to reproduce]
.
  5. Run SCT test

    a.  Execute the following commands to run SCT test on SUT
             sct -r

  sct -u
   b.  In Test Case Management page, Enable GenericTest
   c.  Press F9 to run the selected test case

[Observation]
synchronous exception occurred in SCT ,Attached the Log for reference.

Thanks,
Karunakar

From: Wu, Jiaxin [mailto:jiaxin.wu@intel.com]
Sent: Friday, November 17, 2017 6:01 AM
To: Karunakar P; 'edk2-devel@lists.01.org'
Cc: Fu, Siyuan; Ye, Ting
Subject: RE: SCT Test crashes After HTTPS boot success.

Hi Karunakar,

Can you provide more detailed info for the SCT test steps? The crash can happen after "Sct.efi -u" or need run any specific test case?

Thanks,
Jiaxin

From: Karunakar P [mailto:karunakarp@amiindia.co.in]
Sent: Thursday, November 16, 2017 4:48 PM
To: 'edk2-devel@lists.01.org' <edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>>
Cc: Wu, Jiaxin <jiaxin.wu@intel.com<mailto:jiaxin.wu@intel.com>>; Fu, Siyuan <siyuan.fu@intel.com<mailto:siyuan.fu@intel.com>>; Ye, Ting <ting.ye@intel.com<mailto:ting.ye@intel.com>>
Subject: SCT Test crashes After HTTPS boot success.

Hello All,

When I do SCT test, get the below failure
GenericTest\EFICompliantTest 0 0 CB6F7B77-0B15-43F7-A95B-8C7F9FD70B21 FAIL UEFI Compliant - TLS support is required

The reason for the failure is that the platform doesn't have TLS related protocols installed.

->if the platform supports TLS feature then, EFI_TLS_PROTOCOL, EFI_TLS_SERVICE_BINDING_PROTOCOL, EFI_TLS_CONFIGURATION_PROTOCOL must be existed.
-> According UEFI2.7 Spec - 28.10 EFI TLS Protocols(page-1787)
The TLS consumer need locate EFI_TLS_SERVICE_BINDING_PROTOCOL and call CreateChild() to create a new child of EFI_TLS_PROTOCOL instance. Then use EFI_TLS_PROTOCOL to start TLS session. After use, the TLS consumer need call DestroyChild() to destroy it.
-> Network Stack follows same in HTTPS boot.
While doing IPv4/6 HTTPS boot , will locate gEfiTlsServiceBindingProtocolGuid and call TlsServiceBindingCreateChild, So that EFI_TLS_PROTOCOL and EFI_TLS_CONFIGURATION_PROTOCOL will be installed.

So once HTTPS boot is success then TLS supported protocols exist.

And if we do SCT test after HTTPS boot is success, Then TLS related failures should NOT happen in SCT.

I've tried SCT test after HTTPS boot success, But SCT test Crashes.

[Steps to reproduce]

1.       Configure the HTTPS Server with EFI Shell as NBP file.

2.       Connect test machine and HTTPS server with LAN cable.

3.       Perform HTTPS boot in test machine

4.       Once HTTPS boot is success, It will launch Shell.

5.       Run SCT test


[Observations]

1.       SCT test was crashed and unable to continue the test.

Could you please provide your comments/Suggestion on this?


Thanks,
Karunakar

Re: SCT Test crashes After HTTPS boot success.

[Karunakar P]

Hi Jiaxin,

It happens in DEBUG MODE

Thanks,
Karunakar

From: Wu, Jiaxin [mailto:jiaxin.wu@intel.com]
Sent: Friday, November 17, 2017 12:05 PM
To: Karunakar P; 'edk2-devel@lists.01.org'
Cc: Fu, Siyuan; Ye, Ting
Subject: RE: SCT Test crashes After HTTPS boot success.

I will try to reproduce the issue with EDK2 trunk code, then feedback to you. Thanks the report.

Jiaxin

From: Karunakar P [mailto:karunakarp@amiindia.co.in]
Sent: Friday, November 17, 2017 2:32 PM
To: Wu, Jiaxin <jiaxin.wu@intel.com<mailto:jiaxin.wu@intel.com>>; 'edk2-devel@lists.01.org' <edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>>
Cc: Fu, Siyuan <siyuan.fu@intel.com<mailto:siyuan.fu@intel.com>>; Ye, Ting <ting.ye@intel.com<mailto:ting.ye@intel.com>>
Subject: RE: SCT Test crashes After HTTPS boot success.

Hi Jiaxin,

Below are the detailed steps for SCT test

[Steps to reproduce]
.
  5. Run SCT test

    a.  Execute the following commands to run SCT test on SUT
             sct -r

  sct -u
   b.  In Test Case Management page, Enable GenericTest
   c.  Press F9 to run the selected test case

[Observation]
synchronous exception occurred in SCT ,Attached the Log for reference.

Thanks,
Karunakar

From: Wu, Jiaxin [mailto:jiaxin.wu@intel.com]
Sent: Friday, November 17, 2017 6:01 AM
To: Karunakar P; 'edk2-devel@lists.01.org'
Cc: Fu, Siyuan; Ye, Ting
Subject: RE: SCT Test crashes After HTTPS boot success.

Hi Karunakar,

Can you provide more detailed info for the SCT test steps? The crash can happen after "Sct.efi -u" or need run any specific test case?

Thanks,
Jiaxin

From: Karunakar P [mailto:karunakarp@amiindia.co.in]
Sent: Thursday, November 16, 2017 4:48 PM
To: 'edk2-devel@lists.01.org' <edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>>
Cc: Wu, Jiaxin <jiaxin.wu@intel.com<mailto:jiaxin.wu@intel.com>>; Fu, Siyuan <siyuan.fu@intel.com<mailto:siyuan.fu@intel.com>>; Ye, Ting <ting.ye@intel.com<mailto:ting.ye@intel.com>>
Subject: SCT Test crashes After HTTPS boot success.

Hello All,

When I do SCT test, get the below failure
GenericTest\EFICompliantTest 0 0 CB6F7B77-0B15-43F7-A95B-8C7F9FD70B21 FAIL UEFI Compliant - TLS support is required

The reason for the failure is that the platform doesn't have TLS related protocols installed.

->if the platform supports TLS feature then, EFI_TLS_PROTOCOL, EFI_TLS_SERVICE_BINDING_PROTOCOL, EFI_TLS_CONFIGURATION_PROTOCOL must be existed.
-> According UEFI2.7 Spec - 28.10 EFI TLS Protocols(page-1787)
The TLS consumer need locate EFI_TLS_SERVICE_BINDING_PROTOCOL and call CreateChild() to create a new child of EFI_TLS_PROTOCOL instance. Then use EFI_TLS_PROTOCOL to start TLS session. After use, the TLS consumer need call DestroyChild() to destroy it.
-> Network Stack follows same in HTTPS boot.
While doing IPv4/6 HTTPS boot , will locate gEfiTlsServiceBindingProtocolGuid and call TlsServiceBindingCreateChild, So that EFI_TLS_PROTOCOL and EFI_TLS_CONFIGURATION_PROTOCOL will be installed.

So once HTTPS boot is success then TLS supported protocols exist.

And if we do SCT test after HTTPS boot is success, Then TLS related failures should NOT happen in SCT.

I've tried SCT test after HTTPS boot success, But SCT test Crashes.

[Steps to reproduce]

1.       Configure the HTTPS Server with EFI Shell as NBP file.

2.       Connect test machine and HTTPS server with LAN cable.

3.       Perform HTTPS boot in test machine

4.       Once HTTPS boot is success, It will launch Shell.

5.       Run SCT test


[Observations]

1.       SCT test was crashed and unable to continue the test.

Could you please provide your comments/Suggestion on this?


Thanks,
Karunakar

Re: SCT Test crashes After HTTPS boot success.

[Wu, Jiaxin]

Hi Karunakar,

I didn't see the SCT crash issue but also see the TLS test failure you attached.

For the failure, it's related to the EFI compliant test. According the UEFI Spec, section of 2.6.2:

If the network environment requires TLS features,
EFI_TLS_SERVICE_BINDING_PROTOCOL,EFI_TLS_PROTOCOL and
EFI_TLS_CONFIGURATION_PROTOCOL are required.

So, the SCT will check the EFI_TLS_SERVICE_BINDING_PROTOCOL, EFI_TLS_PROTOCOL and EFI_TLS_CONFIGURATION_PROTOCOL, but it didn't find the EFI_TLS_PROTOCOL and EFI_TLS_CONFIGURATION_PROTOCOL because no available TLS instance is created by default since it's no driver dependency. Even you tried the HTTPS, the TLS instance will be destroyed after finishing the HTTPS boot process, that's the reason why it still fail after HTTPS boot.
  Status = gtBS->LocateProtocol (
                   &gEfiTlsConfigurationProtocolGuid,
                   NULL,
                   (VOID **) &Interface
                   );
  if (!EFI_ERROR (Status)) {
    ValueB = TRUE;
  } else {
    ValueB = FALSE;
  }

After talk with SCT expert (Jin, Eric <eric.jin@intel.com>) , we agree the Spec may need to be updated since it can mislead the caller.

Thanks,
Jiaxin


From: Karunakar P [mailto:karunakarp@amiindia.co.in]
Sent: Tuesday, November 21, 2017 7:55 PM
To: Wu, Jiaxin <jiaxin.wu@intel.com>; 'edk2-devel@lists.01.org' <edk2-devel@lists.01.org>
Cc: Fu, Siyuan <siyuan.fu@intel.com>; Ye, Ting <ting.ye@intel.com>
Subject: RE: SCT Test crashes After HTTPS boot success.


Hi Jiaxin,

I've done SCT Test After HTTPS Boot success(Release Mode), I'm facing the below failures


1.       GenericTest\EFICompliantTest - EFI Compliant - PXE_BC protocols and one of UNDI/SNP/MNP must be implemented if a platform supports to boot from a network device.

2.       GenericTest\EFICompliantTest - UEFI-Compliant - EFI_NVM_EXPRESS_PASS_THRU_PROTOCOL must be implemented if a platform includes an NVM Express controller.

3.       GenericTest\EFICompliantTest - UEFI-Compliant - EFI_BLOCK_IO_PROTOCOL must be existed if the platform supports booting from a block-oriented NVM Express controller. EFI_NVM_EXPRESS_PASS_THRU_PROTOCOL may be required.

4.       GenericTest\EFICompliantTest - EFI Compliant - SCSI_PASS_THRU protocol must be implemented if a platform includes an I/O system that uses SCSI command packets.

5.       GenericTest\EFICompliantTest - UEFI-Compliant \x13EFI_SCSI IO_PROTOCOL, EFI_Block IO_PROTOCOL and EFI_EXT_SCSI_PASS_THRU_PROTOCOL must be implemented if a platform supports booting from a SCSI peripheral device.

6.       GenericTest\EFICompliantTest - EFI Compliant - DEBUG_SUPPORT and DEBUG_PORT protocols must be implemented if a platform supports debugging capabilities.

7.       GenericTest\EFICompliantTest - UEFI-Compliant - EFI_ATA_PASS_THRU_PROTOCOL must be implemented if a platform includes an I/O subsystem that utilizes ATA command packets.

8.       GenericTest\EFICompliantTest - UEFI-Compliant - EFI_TLS_PROTOCOL, EFI_TLS_SERVICE_BINDING_PROTOCOL, EFI_TLS_CONFIGURATION_PROTOCOL must be existed if the platform supports TLS feature.

9.       GenericTest\EFICompliantTest - UEFI-Compliant - EFI_EAP_PROTOCOL, EFI_EAP_CONFIGURATION_PROTOCOL, EFI_EAP_MANAGEMENT2_PROTOCOL must be existed if the platform includes the ability to perform a wireless boot from a network device with EAP feature, and if this platform provides

10.   GenericTest\EFICompliantTest - UEFI-Compliant - EFI_BLUETOOTH_HC_PROTOCOL, EFI_BLUETOOTH_IO_PROTOCOL, EFI_BLUETOOTH_CONFIG_PROTOCOL must be existed if the platform supports classic Bluetooth.

I've attached the Test Report for reference.
Could you please provide your comments/suggestions on the failures?

Thanks,
Karunakar

From: Wu, Jiaxin [mailto:jiaxin.wu@intel.com]
Sent: Friday, November 17, 2017 12:05 PM
To: Karunakar P; 'edk2-devel@lists.01.org'
Cc: Fu, Siyuan; Ye, Ting
Subject: RE: SCT Test crashes After HTTPS boot success.

I will try to reproduce the issue with EDK2 trunk code, then feedback to you. Thanks the report.

Jiaxin

From: Karunakar P [mailto:karunakarp@amiindia.co.in]
Sent: Friday, November 17, 2017 2:32 PM
To: Wu, Jiaxin <jiaxin.wu@intel.com<mailto:jiaxin.wu@intel.com>>; 'edk2-devel@lists.01.org' <edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>>
Cc: Fu, Siyuan <siyuan.fu@intel.com<mailto:siyuan.fu@intel.com>>; Ye, Ting <ting.ye@intel.com<mailto:ting.ye@intel.com>>
Subject: RE: SCT Test crashes After HTTPS boot success.

Hi Jiaxin,

Below are the detailed steps for SCT test

[Steps to reproduce]
.
  5. Run SCT test

    a.  Execute the following commands to run SCT test on SUT
             sct -r

  sct -u
   b.  In Test Case Management page, Enable GenericTest
   c.  Press F9 to run the selected test case

[Observation]
synchronous exception occurred in SCT ,Attached the Log for reference.

Thanks,
Karunakar

From: Wu, Jiaxin [mailto:jiaxin.wu@intel.com]
Sent: Friday, November 17, 2017 6:01 AM
To: Karunakar P; 'edk2-devel@lists.01.org'
Cc: Fu, Siyuan; Ye, Ting
Subject: RE: SCT Test crashes After HTTPS boot success.

Hi Karunakar,

Can you provide more detailed info for the SCT test steps? The crash can happen after "Sct.efi -u" or need run any specific test case?

Thanks,
Jiaxin

From: Karunakar P [mailto:karunakarp@amiindia.co.in]
Sent: Thursday, November 16, 2017 4:48 PM
To: 'edk2-devel@lists.01.org' <edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>>
Cc: Wu, Jiaxin <jiaxin.wu@intel.com<mailto:jiaxin.wu@intel.com>>; Fu, Siyuan <siyuan.fu@intel.com<mailto:siyuan.fu@intel.com>>; Ye, Ting <ting.ye@intel.com<mailto:ting.ye@intel.com>>
Subject: SCT Test crashes After HTTPS boot success.

Hello All,

When I do SCT test, get the below failure
GenericTest\EFICompliantTest 0 0 CB6F7B77-0B15-43F7-A95B-8C7F9FD70B21 FAIL UEFI Compliant - TLS support is required

The reason for the failure is that the platform doesn't have TLS related protocols installed.

->if the platform supports TLS feature then, EFI_TLS_PROTOCOL, EFI_TLS_SERVICE_BINDING_PROTOCOL, EFI_TLS_CONFIGURATION_PROTOCOL must be existed.
-> According UEFI2.7 Spec - 28.10 EFI TLS Protocols(page-1787)
The TLS consumer need locate EFI_TLS_SERVICE_BINDING_PROTOCOL and call CreateChild() to create a new child of EFI_TLS_PROTOCOL instance. Then use EFI_TLS_PROTOCOL to start TLS session. After use, the TLS consumer need call DestroyChild() to destroy it.
-> Network Stack follows same in HTTPS boot.
While doing IPv4/6 HTTPS boot , will locate gEfiTlsServiceBindingProtocolGuid and call TlsServiceBindingCreateChild, So that EFI_TLS_PROTOCOL and EFI_TLS_CONFIGURATION_PROTOCOL will be installed.

So once HTTPS boot is success then TLS supported protocols exist.

And if we do SCT test after HTTPS boot is success, Then TLS related failures should NOT happen in SCT.

I've tried SCT test after HTTPS boot success, But SCT test Crashes.

[Steps to reproduce]

1.       Configure the HTTPS Server with EFI Shell as NBP file.

2.       Connect test machine and HTTPS server with LAN cable.

3.       Perform HTTPS boot in test machine

4.       Once HTTPS boot is success, It will launch Shell.

5.       Run SCT test


[Observations]

1.       SCT test was crashed and unable to continue the test.

Could you please provide your comments/Suggestion on this?


Thanks,
Karunakar

Re: SCT Test crashes After HTTPS boot success.

[Wu, Jiaxin]

Hi Karunakar,

We're going to update the below description in UEFI Spec section of 2.6.2, not SCT Spec:
If the network environment requires TLS features,
EFI_TLS_SERVICE_BINDING_PROTOCOL,EFI_TLS_PROTOCOL and
EFI_TLS_CONFIGURATION_PROTOCOL are required.

The SCT test case was drafted by according to the above sentence that all of the TlsSB/TlsProtocol/TlsConfigProtocol must exist, if not, the failure happen.

For other cases, we will review them, then feedback to you.

Thanks,
Jiaxin




From: Karunakar P [mailto:karunakarp@amiindia.co.in]
Sent: Wednesday, November 22, 2017 6:25 PM
To: Wu, Jiaxin <jiaxin.wu@intel.com>; 'edk2-devel@lists.01.org' <edk2-devel@lists.01.org>
Cc: Fu, Siyuan <siyuan.fu@intel.com>; Ye, Ting <ting.ye@intel.com>; Jin, Eric <eric.jin@intel.com>
Subject: RE: SCT Test crashes After HTTPS boot success.

Hi Jiaxin,

Thank you very much for your detailed explanation.

After talk with SCT expert (Jin, Eric <eric.jin@intel.com<mailto:eric.jin@intel.com>>) , we agree the Spec may need to be updated since it can mislead the caller.
Sorry, I didn't get the point here, whether we're going to update SCT spec to resolve this TLS failure?

And regards other failures

? The platform have SCSI module, PXE_BC protocols and one of UNDI/SNP/MNP are present, But still I'm getting SCT failures related to this.

? Also there are some failures in HiiTest

Could you please provide your comments/Suggestions on other failures too.

Thank You.
Karunakar

From: Wu, Jiaxin [mailto:jiaxin.wu@intel.com]
Sent: Wednesday, November 22, 2017 2:13 PM
To: Karunakar P; 'edk2-devel@lists.01.org'
Cc: Fu, Siyuan; Ye, Ting; Jin, Eric
Subject: RE: SCT Test crashes After HTTPS boot success.

Hi Karunakar,

I didn't see the SCT crash issue but also see the TLS test failure you attached.

For the failure, it's related to the EFI compliant test. According the UEFI Spec, section of 2.6.2:

If the network environment requires TLS features,
EFI_TLS_SERVICE_BINDING_PROTOCOL,EFI_TLS_PROTOCOL and
EFI_TLS_CONFIGURATION_PROTOCOL are required.

So, the SCT will check the EFI_TLS_SERVICE_BINDING_PROTOCOL, EFI_TLS_PROTOCOL and EFI_TLS_CONFIGURATION_PROTOCOL, but it didn't find the EFI_TLS_PROTOCOL and EFI_TLS_CONFIGURATION_PROTOCOL because no available TLS instance is created by default since it's no driver dependency. Even you tried the HTTPS, the TLS instance will be destroyed after finishing the HTTPS boot process, that's the reason why it still fail after HTTPS boot.
  Status = gtBS->LocateProtocol (
                   &gEfiTlsConfigurationProtocolGuid,
                   NULL,
                   (VOID **) &Interface
                   );
  if (!EFI_ERROR (Status)) {
    ValueB = TRUE;
  } else {
    ValueB = FALSE;
  }

After talk with SCT expert (Jin, Eric <eric.jin@intel.com<mailto:eric.jin@intel.com>>) , we agree the Spec may need to be updated since it can mislead the caller.

Thanks,
Jiaxin


From: Karunakar P [mailto:karunakarp@amiindia.co.in]
Sent: Tuesday, November 21, 2017 7:55 PM
To: Wu, Jiaxin <jiaxin.wu@intel.com<mailto:jiaxin.wu@intel.com>>; 'edk2-devel@lists.01.org' <edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>>
Cc: Fu, Siyuan <siyuan.fu@intel.com<mailto:siyuan.fu@intel.com>>; Ye, Ting <ting.ye@intel.com<mailto:ting.ye@intel.com>>
Subject: RE: SCT Test crashes After HTTPS boot success.


Hi Jiaxin,

I've done SCT Test After HTTPS Boot success(Release Mode), I'm facing the below failures


1.       GenericTest\EFICompliantTest - EFI Compliant - PXE_BC protocols and one of UNDI/SNP/MNP must be implemented if a platform supports to boot from a network device.

2.       GenericTest\EFICompliantTest - UEFI-Compliant - EFI_NVM_EXPRESS_PASS_THRU_PROTOCOL must be implemented if a platform includes an NVM Express controller.

3.       GenericTest\EFICompliantTest - UEFI-Compliant - EFI_BLOCK_IO_PROTOCOL must be existed if the platform supports booting from a block-oriented NVM Express controller. EFI_NVM_EXPRESS_PASS_THRU_PROTOCOL may be required.

4.       GenericTest\EFICompliantTest - EFI Compliant - SCSI_PASS_THRU protocol must be implemented if a platform includes an I/O system that uses SCSI command packets.

5.       GenericTest\EFICompliantTest - UEFI-Compliant \x13EFI_SCSI IO_PROTOCOL, EFI_Block IO_PROTOCOL and EFI_EXT_SCSI_PASS_THRU_PROTOCOL must be implemented if a platform supports booting from a SCSI peripheral device.

6.       GenericTest\EFICompliantTest - EFI Compliant - DEBUG_SUPPORT and DEBUG_PORT protocols must be implemented if a platform supports debugging capabilities.

7.       GenericTest\EFICompliantTest - UEFI-Compliant - EFI_ATA_PASS_THRU_PROTOCOL must be implemented if a platform includes an I/O subsystem that utilizes ATA command packets.

8.       GenericTest\EFICompliantTest - UEFI-Compliant - EFI_TLS_PROTOCOL, EFI_TLS_SERVICE_BINDING_PROTOCOL, EFI_TLS_CONFIGURATION_PROTOCOL must be existed if the platform supports TLS feature.

9.       GenericTest\EFICompliantTest - UEFI-Compliant - EFI_EAP_PROTOCOL, EFI_EAP_CONFIGURATION_PROTOCOL, EFI_EAP_MANAGEMENT2_PROTOCOL must be existed if the platform includes the ability to perform a wireless boot from a network device with EAP feature, and if this platform provides

10.   GenericTest\EFICompliantTest - UEFI-Compliant - EFI_BLUETOOTH_HC_PROTOCOL, EFI_BLUETOOTH_IO_PROTOCOL, EFI_BLUETOOTH_CONFIG_PROTOCOL must be existed if the platform supports classic Bluetooth.

I've attached the Test Report for reference.
Could you please provide your comments/suggestions on the failures?

Thanks,
Karunakar

From: Wu, Jiaxin [mailto:jiaxin.wu@intel.com]
Sent: Friday, November 17, 2017 12:05 PM
To: Karunakar P; 'edk2-devel@lists.01.org'
Cc: Fu, Siyuan; Ye, Ting
Subject: RE: SCT Test crashes After HTTPS boot success.

I will try to reproduce the issue with EDK2 trunk code, then feedback to you. Thanks the report.

Jiaxin

From: Karunakar P [mailto:karunakarp@amiindia.co.in]
Sent: Friday, November 17, 2017 2:32 PM
To: Wu, Jiaxin <jiaxin.wu@intel.com<mailto:jiaxin.wu@intel.com>>; 'edk2-devel@lists.01.org' <edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>>
Cc: Fu, Siyuan <siyuan.fu@intel.com<mailto:siyuan.fu@intel.com>>; Ye, Ting <ting.ye@intel.com<mailto:ting.ye@intel.com>>
Subject: RE: SCT Test crashes After HTTPS boot success.

Hi Jiaxin,

Below are the detailed steps for SCT test

[Steps to reproduce]
.
  5. Run SCT test

    a.  Execute the following commands to run SCT test on SUT
             sct -r

  sct -u
   b.  In Test Case Management page, Enable GenericTest
   c.  Press F9 to run the selected test case

[Observation]
synchronous exception occurred in SCT ,Attached the Log for reference.

Thanks,
Karunakar

From: Wu, Jiaxin [mailto:jiaxin.wu@intel.com]
Sent: Friday, November 17, 2017 6:01 AM
To: Karunakar P; 'edk2-devel@lists.01.org'
Cc: Fu, Siyuan; Ye, Ting
Subject: RE: SCT Test crashes After HTTPS boot success.

Hi Karunakar,

Can you provide more detailed info for the SCT test steps? The crash can happen after "Sct.efi -u" or need run any specific test case?

Thanks,
Jiaxin

From: Karunakar P [mailto:karunakarp@amiindia.co.in]
Sent: Thursday, November 16, 2017 4:48 PM
To: 'edk2-devel@lists.01.org' <edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>>
Cc: Wu, Jiaxin <jiaxin.wu@intel.com<mailto:jiaxin.wu@intel.com>>; Fu, Siyuan <siyuan.fu@intel.com<mailto:siyuan.fu@intel.com>>; Ye, Ting <ting.ye@intel.com<mailto:ting.ye@intel.com>>
Subject: SCT Test crashes After HTTPS boot success.

Hello All,

When I do SCT test, get the below failure
GenericTest\EFICompliantTest 0 0 CB6F7B77-0B15-43F7-A95B-8C7F9FD70B21 FAIL UEFI Compliant - TLS support is required

The reason for the failure is that the platform doesn't have TLS related protocols installed.

->if the platform supports TLS feature then, EFI_TLS_PROTOCOL, EFI_TLS_SERVICE_BINDING_PROTOCOL, EFI_TLS_CONFIGURATION_PROTOCOL must be existed.
-> According UEFI2.7 Spec - 28.10 EFI TLS Protocols(page-1787)
The TLS consumer need locate EFI_TLS_SERVICE_BINDING_PROTOCOL and call CreateChild() to create a new child of EFI_TLS_PROTOCOL instance. Then use EFI_TLS_PROTOCOL to start TLS session. After use, the TLS consumer need call DestroyChild() to destroy it.
-> Network Stack follows same in HTTPS boot.
While doing IPv4/6 HTTPS boot , will locate gEfiTlsServiceBindingProtocolGuid and call TlsServiceBindingCreateChild, So that EFI_TLS_PROTOCOL and EFI_TLS_CONFIGURATION_PROTOCOL will be installed.

So once HTTPS boot is success then TLS supported protocols exist.

And if we do SCT test after HTTPS boot is success, Then TLS related failures should NOT happen in SCT.

I've tried SCT test after HTTPS boot success, But SCT test Crashes.

[Steps to reproduce]

1.       Configure the HTTPS Server with EFI Shell as NBP file.

2.       Connect test machine and HTTPS server with LAN cable.

3.       Perform HTTPS boot in test machine

4.       Once HTTPS boot is success, It will launch Shell.

5.       Run SCT test


[Observations]

1.       SCT test was crashed and unable to continue the test.

Could you please provide your comments/Suggestion on this?


Thanks,
Karunakar

Re: SCT Test crashes After HTTPS boot success.

[Wu, Jiaxin]

Hi Karunakar,

I think most cases failure are related to the lack of modules in your platform, detailed see below:

c:\work\sct\utwg\uefi-sct\SctPkg\TestCase\UEFI\EFI\Generic\EfiCompliant\BlackBoxTest\EfiCompliantBBTestPlatform_uefi.c:1428:SetupMode equal zero - No
Jiaxin: That's because you set ValidateBootImageThruNet to yes in EfiCompliant.ini file but the SetupMode in your platform is 1, which means the platform doesn't support to validate a boot image.

c:\work\sct\utwg\uefi-sct\SctPkg\TestCase\UEFI\EFI\Generic\EfiCompliant\BlackBoxTest\EfiCompliantBBTestPlatform_uefi.c:2185:NVM Express Pass Thru protocol - No
c:\work\sct\utwg\uefi-sct\SctPkg\TestCase\UEFI\EFI\Generic\EfiCompliant\BlackBoxTest\EfiCompliantBBTestPlatform_uefi.c:2306:NVMExpressPassThru - No
Jiaxin: No NvmExpressDxe Module in your platform.

c:\work\sct\utwg\uefi-sct\SctPkg\TestCase\UEFI\EFI\Generic\EfiCompliant\BlackBoxTest\EfiCompliantBBTestPlatform_uefi.c:2379:Ext SCSI Pass Thru - No
c:\work\sct\utwg\uefi-sct\SctPkg\TestCase\UEFI\EFI\Generic\EfiCompliant\BlackBoxTest\EfiCompliantBBTestPlatform_uefi.c:2478:SCSI IO - No, Block IO - Yes
Jiaxin: No driver produce the ExtScsiPassThruProtocol, so your platform doesn't include an I/O system that uses SCSI command packets. For this case, you can setup ISCSI connection, then the failure should be gone. Or set the value of ExtScsiPassThru to no in EfiCompliant.ini file if you don't care about it.

c:\work\sct\utwg\uefi-sct\SctPkg\TestCase\UEFI\EFI\Generic\EfiCompliant\BlackBoxTest\EfiCompliantBBTestPlatform_uefi.c:2690:Debug Support - Yes, Debug Port - No
Jiaxin: No DebugPortDxe Module in your platform.

c:\work\sct\utwg\uefi-sct\SctPkg\TestCase\UEFI\EFI\Generic\EfiCompliant\BlackBoxTest\EfiCompliantBBTestPlatform_uefi.c:2829:Ata Pass Thru - No
Jiaxin: No AtaBusDxe Module in your platform.

c:\work\sct\utwg\uefi-sct\SctPkg\TestCase\UEFI\EFI\Generic\EfiCompliant\BlackBoxTest\EfiCompliantBBTestPlatform_uefi.c:3558:EAP - No, EAP Config - No, EAP Management2 - No
c:\work\sct\utwg\uefi-sct\SctPkg\TestCase\UEFI\EFI\Generic\EfiCompliant\BlackBoxTest\EfiCompliantBBTestPlatform_uefi.c:3653:BLUETOOTH HC - No, BLUETOOTH Service Binding - No, BLUETOOTH Config - No
Jiaxin: No EAP/ BLUETOOTH modules in your platform.

For the Hii part, I can't reproduce the failure with UDK2017 code base. I think it's much related to your platform.

Thanks,
Jiaxin

From: Wu, Jiaxin
Sent: Thursday, November 23, 2017 12:51 PM
To: Karunakar P <karunakarp@amiindia.co.in>; 'edk2-devel@lists.01.org' <edk2-devel@lists.01.org>
Cc: Fu, Siyuan <siyuan.fu@intel.com>; Ye, Ting <ting.ye@intel.com>; Jin, Eric <eric.jin@intel.com>
Subject: RE: SCT Test crashes After HTTPS boot success.

Hi Karunakar,

We're going to update the below description in UEFI Spec section of 2.6.2, not SCT Spec:
If the network environment requires TLS features,
EFI_TLS_SERVICE_BINDING_PROTOCOL,EFI_TLS_PROTOCOL and
EFI_TLS_CONFIGURATION_PROTOCOL are required.

The SCT test case was drafted by according to the above sentence that all of the TlsSB/TlsProtocol/TlsConfigProtocol must exist, if not, the failure happen.

For other cases, we will review them, then feedback to you.

Thanks,
Jiaxin




From: Karunakar P [mailto:karunakarp@amiindia.co.in]
Sent: Wednesday, November 22, 2017 6:25 PM
To: Wu, Jiaxin <jiaxin.wu@intel.com<mailto:jiaxin.wu@intel.com>>; 'edk2-devel@lists.01.org' <edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>>
Cc: Fu, Siyuan <siyuan.fu@intel.com<mailto:siyuan.fu@intel.com>>; Ye, Ting <ting.ye@intel.com<mailto:ting.ye@intel.com>>; Jin, Eric <eric.jin@intel.com<mailto:eric.jin@intel.com>>
Subject: RE: SCT Test crashes After HTTPS boot success.

Hi Jiaxin,

Thank you very much for your detailed explanation.

After talk with SCT expert (Jin, Eric <eric.jin@intel.com<mailto:eric.jin@intel.com>>) , we agree the Spec may need to be updated since it can mislead the caller.
Sorry, I didn't get the point here, whether we're going to update SCT spec to resolve this TLS failure?

And regards other failures

? The platform have SCSI module, PXE_BC protocols and one of UNDI/SNP/MNP are present, But still I'm getting SCT failures related to this.

? Also there are some failures in HiiTest

Could you please provide your comments/Suggestions on other failures too.

Thank You.
Karunakar

From: Wu, Jiaxin [mailto:jiaxin.wu@intel.com]
Sent: Wednesday, November 22, 2017 2:13 PM
To: Karunakar P; 'edk2-devel@lists.01.org'
Cc: Fu, Siyuan; Ye, Ting; Jin, Eric
Subject: RE: SCT Test crashes After HTTPS boot success.

Hi Karunakar,

I didn't see the SCT crash issue but also see the TLS test failure you attached.

For the failure, it's related to the EFI compliant test. According the UEFI Spec, section of 2.6.2:

If the network environment requires TLS features,
EFI_TLS_SERVICE_BINDING_PROTOCOL,EFI_TLS_PROTOCOL and
EFI_TLS_CONFIGURATION_PROTOCOL are required.

So, the SCT will check the EFI_TLS_SERVICE_BINDING_PROTOCOL, EFI_TLS_PROTOCOL and EFI_TLS_CONFIGURATION_PROTOCOL, but it didn't find the EFI_TLS_PROTOCOL and EFI_TLS_CONFIGURATION_PROTOCOL because no available TLS instance is created by default since it's no driver dependency. Even you tried the HTTPS, the TLS instance will be destroyed after finishing the HTTPS boot process, that's the reason why it still fail after HTTPS boot.
  Status = gtBS->LocateProtocol (
                   &gEfiTlsConfigurationProtocolGuid,
                   NULL,
                   (VOID **) &Interface
                   );
  if (!EFI_ERROR (Status)) {
    ValueB = TRUE;
  } else {
    ValueB = FALSE;
  }

After talk with SCT expert (Jin, Eric <eric.jin@intel.com<mailto:eric.jin@intel.com>>) , we agree the Spec may need to be updated since it can mislead the caller.

Thanks,
Jiaxin


From: Karunakar P [mailto:karunakarp@amiindia.co.in]
Sent: Tuesday, November 21, 2017 7:55 PM
To: Wu, Jiaxin <jiaxin.wu@intel.com<mailto:jiaxin.wu@intel.com>>; 'edk2-devel@lists.01.org' <edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>>
Cc: Fu, Siyuan <siyuan.fu@intel.com<mailto:siyuan.fu@intel.com>>; Ye, Ting <ting.ye@intel.com<mailto:ting.ye@intel.com>>
Subject: RE: SCT Test crashes After HTTPS boot success.


Hi Jiaxin,

I've done SCT Test After HTTPS Boot success(Release Mode), I'm facing the below failures


1.       GenericTest\EFICompliantTest - EFI Compliant - PXE_BC protocols and one of UNDI/SNP/MNP must be implemented if a platform supports to boot from a network device.

2.       GenericTest\EFICompliantTest - UEFI-Compliant - EFI_NVM_EXPRESS_PASS_THRU_PROTOCOL must be implemented if a platform includes an NVM Express controller.

3.       GenericTest\EFICompliantTest - UEFI-Compliant - EFI_BLOCK_IO_PROTOCOL must be existed if the platform supports booting from a block-oriented NVM Express controller. EFI_NVM_EXPRESS_PASS_THRU_PROTOCOL may be required.

4.       GenericTest\EFICompliantTest - EFI Compliant - SCSI_PASS_THRU protocol must be implemented if a platform includes an I/O system that uses SCSI command packets.

5.       GenericTest\EFICompliantTest - UEFI-Compliant \x13EFI_SCSI IO_PROTOCOL, EFI_Block IO_PROTOCOL and EFI_EXT_SCSI_PASS_THRU_PROTOCOL must be implemented if a platform supports booting from a SCSI peripheral device.

6.       GenericTest\EFICompliantTest - EFI Compliant - DEBUG_SUPPORT and DEBUG_PORT protocols must be implemented if a platform supports debugging capabilities.

7.       GenericTest\EFICompliantTest - UEFI-Compliant - EFI_ATA_PASS_THRU_PROTOCOL must be implemented if a platform includes an I/O subsystem that utilizes ATA command packets.

8.       GenericTest\EFICompliantTest - UEFI-Compliant - EFI_TLS_PROTOCOL, EFI_TLS_SERVICE_BINDING_PROTOCOL, EFI_TLS_CONFIGURATION_PROTOCOL must be existed if the platform supports TLS feature.

9.       GenericTest\EFICompliantTest - UEFI-Compliant - EFI_EAP_PROTOCOL, EFI_EAP_CONFIGURATION_PROTOCOL, EFI_EAP_MANAGEMENT2_PROTOCOL must be existed if the platform includes the ability to perform a wireless boot from a network device with EAP feature, and if this platform provides

10.   GenericTest\EFICompliantTest - UEFI-Compliant - EFI_BLUETOOTH_HC_PROTOCOL, EFI_BLUETOOTH_IO_PROTOCOL, EFI_BLUETOOTH_CONFIG_PROTOCOL must be existed if the platform supports classic Bluetooth.

I've attached the Test Report for reference.
Could you please provide your comments/suggestions on the failures?

Thanks,
Karunakar

From: Wu, Jiaxin [mailto:jiaxin.wu@intel.com]
Sent: Friday, November 17, 2017 12:05 PM
To: Karunakar P; 'edk2-devel@lists.01.org'
Cc: Fu, Siyuan; Ye, Ting
Subject: RE: SCT Test crashes After HTTPS boot success.

I will try to reproduce the issue with EDK2 trunk code, then feedback to you. Thanks the report.

Jiaxin

From: Karunakar P [mailto:karunakarp@amiindia.co.in]
Sent: Friday, November 17, 2017 2:32 PM
To: Wu, Jiaxin <jiaxin.wu@intel.com<mailto:jiaxin.wu@intel.com>>; 'edk2-devel@lists.01.org' <edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>>
Cc: Fu, Siyuan <siyuan.fu@intel.com<mailto:siyuan.fu@intel.com>>; Ye, Ting <ting.ye@intel.com<mailto:ting.ye@intel.com>>
Subject: RE: SCT Test crashes After HTTPS boot success.

Hi Jiaxin,

Below are the detailed steps for SCT test

[Steps to reproduce]
.
  5. Run SCT test

    a.  Execute the following commands to run SCT test on SUT
             sct -r

  sct -u
   b.  In Test Case Management page, Enable GenericTest
   c.  Press F9 to run the selected test case

[Observation]
synchronous exception occurred in SCT ,Attached the Log for reference.

Thanks,
Karunakar

From: Wu, Jiaxin [mailto:jiaxin.wu@intel.com]
Sent: Friday, November 17, 2017 6:01 AM
To: Karunakar P; 'edk2-devel@lists.01.org'
Cc: Fu, Siyuan; Ye, Ting
Subject: RE: SCT Test crashes After HTTPS boot success.

Hi Karunakar,

Can you provide more detailed info for the SCT test steps? The crash can happen after "Sct.efi -u" or need run any specific test case?

Thanks,
Jiaxin

From: Karunakar P [mailto:karunakarp@amiindia.co.in]
Sent: Thursday, November 16, 2017 4:48 PM
To: 'edk2-devel@lists.01.org' <edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>>
Cc: Wu, Jiaxin <jiaxin.wu@intel.com<mailto:jiaxin.wu@intel.com>>; Fu, Siyuan <siyuan.fu@intel.com<mailto:siyuan.fu@intel.com>>; Ye, Ting <ting.ye@intel.com<mailto:ting.ye@intel.com>>
Subject: SCT Test crashes After HTTPS boot success.

Hello All,

When I do SCT test, get the below failure
GenericTest\EFICompliantTest 0 0 CB6F7B77-0B15-43F7-A95B-8C7F9FD70B21 FAIL UEFI Compliant - TLS support is required

The reason for the failure is that the platform doesn't have TLS related protocols installed.

->if the platform supports TLS feature then, EFI_TLS_PROTOCOL, EFI_TLS_SERVICE_BINDING_PROTOCOL, EFI_TLS_CONFIGURATION_PROTOCOL must be existed.
-> According UEFI2.7 Spec - 28.10 EFI TLS Protocols(page-1787)
The TLS consumer need locate EFI_TLS_SERVICE_BINDING_PROTOCOL and call CreateChild() to create a new child of EFI_TLS_PROTOCOL instance. Then use EFI_TLS_PROTOCOL to start TLS session. After use, the TLS consumer need call DestroyChild() to destroy it.
-> Network Stack follows same in HTTPS boot.
While doing IPv4/6 HTTPS boot , will locate gEfiTlsServiceBindingProtocolGuid and call TlsServiceBindingCreateChild, So that EFI_TLS_PROTOCOL and EFI_TLS_CONFIGURATION_PROTOCOL will be installed.

So once HTTPS boot is success then TLS supported protocols exist.

And if we do SCT test after HTTPS boot is success, Then TLS related failures should NOT happen in SCT.

I've tried SCT test after HTTPS boot success, But SCT test Crashes.

[Steps to reproduce]

1.       Configure the HTTPS Server with EFI Shell as NBP file.

2.       Connect test machine and HTTPS server with LAN cable.

3.       Perform HTTPS boot in test machine

4.       Once HTTPS boot is success, It will launch Shell.

5.       Run SCT test


[Observations]

1.       SCT test was crashed and unable to continue the test.

Could you please provide your comments/Suggestion on this?


Thanks,
Karunakar

Re: SCT Test crashes After HTTPS boot success.

[Wu, Jiaxin]

For #1, I have response as below:
That's because you set ValidateBootImageThruNet to yes in EfiCompliant.ini file but the SetupMode in your platform is 1, which means the platform doesn't support to validate a boot image. You need set the SetupMode to zero if you want to pass the ValidateBootImageThruNet test (in Secure Boot UI page).

For #2, according your test log, I responded it's caused by the lack of ExtScsiPassThruProtocol, so your platform doesn't include an I/O system that uses SCSI command packets.
But it looks strange since you have setup the ISCSI connection, then the ExtScsiPassThruProtocol should have been created. Maybe ExtScsiPassThruProtocol has been destroyed anywhere if ISCSI any error happen?

Thanks,
Jiaxin

From: Karunakar P [mailto:karunakarp@amiindia.co.in]
Sent: Thursday, November 23, 2017 4:27 PM
To: Wu, Jiaxin <jiaxin.wu@intel.com>; 'edk2-devel@lists.01.org' <edk2-devel@lists.01.org>
Cc: Fu, Siyuan <siyuan.fu@intel.com>; Ye, Ting <ting.ye@intel.com>; Jin, Eric <eric.jin@intel.com>
Subject: RE: SCT Test crashes After HTTPS boot success.

Hi Jiaxin,

Thanks for your detailed explanation.
Yes I agree that few failures are because of the lack of modules.
But few cases are still fails even though related modules are present like below.

1.       GenericTest\EFICompliantTest - EFI Compliant - PXE_BC protocols and one of UNDI/SNP/MNP must be implemented if a platform supports to boot from a network device.

? Could you please comment on this.

2.       GenericTest\EFICompliantTest - EFI Compliant - SCSI_PASS_THRU protocol must be implemented if a platform includes an I/O system that uses SCSI command packets.

? I will try to setup ISCSI connection and then perform SCT Test.


Thanks for your support
-Karunakar.

From: Wu, Jiaxin [mailto:jiaxin.wu@intel.com]
Sent: Thursday, November 23, 2017 1:03 PM
To: Karunakar P; 'edk2-devel@lists.01.org'
Cc: Fu, Siyuan; Ye, Ting; Jin, Eric
Subject: RE: SCT Test crashes After HTTPS boot success.

Hi Karunakar,

I think most cases failure are related to the lack of modules in your platform, detailed see below:

c:\work\sct\utwg\uefi-sct\SctPkg\TestCase\UEFI\EFI\Generic\EfiCompliant\BlackBoxTest\EfiCompliantBBTestPlatform_uefi.c:1428:SetupMode equal zero - No
Jiaxin: That's because you set ValidateBootImageThruNet to yes in EfiCompliant.ini file but the SetupMode in your platform is 1, which means the platform doesn't support to validate a boot image.

c:\work\sct\utwg\uefi-sct\SctPkg\TestCase\UEFI\EFI\Generic\EfiCompliant\BlackBoxTest\EfiCompliantBBTestPlatform_uefi.c:2185:NVM Express Pass Thru protocol - No
c:\work\sct\utwg\uefi-sct\SctPkg\TestCase\UEFI\EFI\Generic\EfiCompliant\BlackBoxTest\EfiCompliantBBTestPlatform_uefi.c:2306:NVMExpressPassThru - No
Jiaxin: No NvmExpressDxe Module in your platform.

c:\work\sct\utwg\uefi-sct\SctPkg\TestCase\UEFI\EFI\Generic\EfiCompliant\BlackBoxTest\EfiCompliantBBTestPlatform_uefi.c:2379:Ext SCSI Pass Thru - No
c:\work\sct\utwg\uefi-sct\SctPkg\TestCase\UEFI\EFI\Generic\EfiCompliant\BlackBoxTest\EfiCompliantBBTestPlatform_uefi.c:2478:SCSI IO - No, Block IO - Yes
Jiaxin: No driver produce the ExtScsiPassThruProtocol, so your platform doesn't include an I/O system that uses SCSI command packets. For this case, you can setup ISCSI connection, then the failure should be gone. Or set the value of ExtScsiPassThru to no in EfiCompliant.ini file if you don't care about it.

c:\work\sct\utwg\uefi-sct\SctPkg\TestCase\UEFI\EFI\Generic\EfiCompliant\BlackBoxTest\EfiCompliantBBTestPlatform_uefi.c:2690:Debug Support - Yes, Debug Port - No
Jiaxin: No DebugPortDxe Module in your platform.

c:\work\sct\utwg\uefi-sct\SctPkg\TestCase\UEFI\EFI\Generic\EfiCompliant\BlackBoxTest\EfiCompliantBBTestPlatform_uefi.c:2829:Ata Pass Thru - No
Jiaxin: No AtaBusDxe Module in your platform.

c:\work\sct\utwg\uefi-sct\SctPkg\TestCase\UEFI\EFI\Generic\EfiCompliant\BlackBoxTest\EfiCompliantBBTestPlatform_uefi.c:3558:EAP - No, EAP Config - No, EAP Management2 - No
c:\work\sct\utwg\uefi-sct\SctPkg\TestCase\UEFI\EFI\Generic\EfiCompliant\BlackBoxTest\EfiCompliantBBTestPlatform_uefi.c:3653:BLUETOOTH HC - No, BLUETOOTH Service Binding - No, BLUETOOTH Config - No
Jiaxin: No EAP/ BLUETOOTH modules in your platform.

For the Hii part, I can't reproduce the failure with UDK2017 code base. I think it's much related to your platform.

Thanks,
Jiaxin

From: Wu, Jiaxin
Sent: Thursday, November 23, 2017 12:51 PM
To: Karunakar P <karunakarp@amiindia.co.in<mailto:karunakarp@amiindia.co.in>>; 'edk2-devel@lists.01.org' <edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>>
Cc: Fu, Siyuan <siyuan.fu@intel.com<mailto:siyuan.fu@intel.com>>; Ye, Ting <ting.ye@intel.com<mailto:ting.ye@intel.com>>; Jin, Eric <eric.jin@intel.com<mailto:eric.jin@intel.com>>
Subject: RE: SCT Test crashes After HTTPS boot success.

Hi Karunakar,

We're going to update the below description in UEFI Spec section of 2.6.2, not SCT Spec:
If the network environment requires TLS features,
EFI_TLS_SERVICE_BINDING_PROTOCOL,EFI_TLS_PROTOCOL and
EFI_TLS_CONFIGURATION_PROTOCOL are required.

The SCT test case was drafted by according to the above sentence that all of the TlsSB/TlsProtocol/TlsConfigProtocol must exist, if not, the failure happen.

For other cases, we will review them, then feedback to you.

Thanks,
Jiaxin




From: Karunakar P [mailto:karunakarp@amiindia.co.in]
Sent: Wednesday, November 22, 2017 6:25 PM
To: Wu, Jiaxin <jiaxin.wu@intel.com<mailto:jiaxin.wu@intel.com>>; 'edk2-devel@lists.01.org' <edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>>
Cc: Fu, Siyuan <siyuan.fu@intel.com<mailto:siyuan.fu@intel.com>>; Ye, Ting <ting.ye@intel.com<mailto:ting.ye@intel.com>>; Jin, Eric <eric.jin@intel.com<mailto:eric.jin@intel.com>>
Subject: RE: SCT Test crashes After HTTPS boot success.

Hi Jiaxin,

Thank you very much for your detailed explanation.

After talk with SCT expert (Jin, Eric <eric.jin@intel.com<mailto:eric.jin@intel.com>>) , we agree the Spec may need to be updated since it can mislead the caller.
Sorry, I didn't get the point here, whether we're going to update SCT spec to resolve this TLS failure?

And regards other failures

? The platform have SCSI module, PXE_BC protocols and one of UNDI/SNP/MNP are present, But still I'm getting SCT failures related to this.

? Also there are some failures in HiiTest

Could you please provide your comments/Suggestions on other failures too.

Thank You.
Karunakar

From: Wu, Jiaxin [mailto:jiaxin.wu@intel.com]
Sent: Wednesday, November 22, 2017 2:13 PM
To: Karunakar P; 'edk2-devel@lists.01.org'
Cc: Fu, Siyuan; Ye, Ting; Jin, Eric
Subject: RE: SCT Test crashes After HTTPS boot success.

Hi Karunakar,

I didn't see the SCT crash issue but also see the TLS test failure you attached.

For the failure, it's related to the EFI compliant test. According the UEFI Spec, section of 2.6.2:

If the network environment requires TLS features,
EFI_TLS_SERVICE_BINDING_PROTOCOL,EFI_TLS_PROTOCOL and
EFI_TLS_CONFIGURATION_PROTOCOL are required.

So, the SCT will check the EFI_TLS_SERVICE_BINDING_PROTOCOL, EFI_TLS_PROTOCOL and EFI_TLS_CONFIGURATION_PROTOCOL, but it didn't find the EFI_TLS_PROTOCOL and EFI_TLS_CONFIGURATION_PROTOCOL because no available TLS instance is created by default since it's no driver dependency. Even you tried the HTTPS, the TLS instance will be destroyed after finishing the HTTPS boot process, that's the reason why it still fail after HTTPS boot.
  Status = gtBS->LocateProtocol (
                   &gEfiTlsConfigurationProtocolGuid,
                   NULL,
                   (VOID **) &Interface
                   );
  if (!EFI_ERROR (Status)) {
    ValueB = TRUE;
  } else {
    ValueB = FALSE;
  }

After talk with SCT expert (Jin, Eric <eric.jin@intel.com<mailto:eric.jin@intel.com>>) , we agree the Spec may need to be updated since it can mislead the caller.

Thanks,
Jiaxin


From: Karunakar P [mailto:karunakarp@amiindia.co.in]
Sent: Tuesday, November 21, 2017 7:55 PM
To: Wu, Jiaxin <jiaxin.wu@intel.com<mailto:jiaxin.wu@intel.com>>; 'edk2-devel@lists.01.org' <edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>>
Cc: Fu, Siyuan <siyuan.fu@intel.com<mailto:siyuan.fu@intel.com>>; Ye, Ting <ting.ye@intel.com<mailto:ting.ye@intel.com>>
Subject: RE: SCT Test crashes After HTTPS boot success.


Hi Jiaxin,

I've done SCT Test After HTTPS Boot success(Release Mode), I'm facing the below failures


1.       GenericTest\EFICompliantTest - EFI Compliant - PXE_BC protocols and one of UNDI/SNP/MNP must be implemented if a platform supports to boot from a network device.

2.       GenericTest\EFICompliantTest - UEFI-Compliant - EFI_NVM_EXPRESS_PASS_THRU_PROTOCOL must be implemented if a platform includes an NVM Express controller.

3.       GenericTest\EFICompliantTest - UEFI-Compliant - EFI_BLOCK_IO_PROTOCOL must be existed if the platform supports booting from a block-oriented NVM Express controller. EFI_NVM_EXPRESS_PASS_THRU_PROTOCOL may be required.

4.       GenericTest\EFICompliantTest - EFI Compliant - SCSI_PASS_THRU protocol must be implemented if a platform includes an I/O system that uses SCSI command packets.

5.       GenericTest\EFICompliantTest - UEFI-Compliant \x13EFI_SCSI IO_PROTOCOL, EFI_Block IO_PROTOCOL and EFI_EXT_SCSI_PASS_THRU_PROTOCOL must be implemented if a platform supports booting from a SCSI peripheral device.

6.       GenericTest\EFICompliantTest - EFI Compliant - DEBUG_SUPPORT and DEBUG_PORT protocols must be implemented if a platform supports debugging capabilities.

7.       GenericTest\EFICompliantTest - UEFI-Compliant - EFI_ATA_PASS_THRU_PROTOCOL must be implemented if a platform includes an I/O subsystem that utilizes ATA command packets.

8.       GenericTest\EFICompliantTest - UEFI-Compliant - EFI_TLS_PROTOCOL, EFI_TLS_SERVICE_BINDING_PROTOCOL, EFI_TLS_CONFIGURATION_PROTOCOL must be existed if the platform supports TLS feature.

9.       GenericTest\EFICompliantTest - UEFI-Compliant - EFI_EAP_PROTOCOL, EFI_EAP_CONFIGURATION_PROTOCOL, EFI_EAP_MANAGEMENT2_PROTOCOL must be existed if the platform includes the ability to perform a wireless boot from a network device with EAP feature, and if this platform provides

10.   GenericTest\EFICompliantTest - UEFI-Compliant - EFI_BLUETOOTH_HC_PROTOCOL, EFI_BLUETOOTH_IO_PROTOCOL, EFI_BLUETOOTH_CONFIG_PROTOCOL must be existed if the platform supports classic Bluetooth.

I've attached the Test Report for reference.
Could you please provide your comments/suggestions on the failures?

Thanks,
Karunakar

From: Wu, Jiaxin [mailto:jiaxin.wu@intel.com]
Sent: Friday, November 17, 2017 12:05 PM
To: Karunakar P; 'edk2-devel@lists.01.org'
Cc: Fu, Siyuan; Ye, Ting
Subject: RE: SCT Test crashes After HTTPS boot success.

I will try to reproduce the issue with EDK2 trunk code, then feedback to you. Thanks the report.

Jiaxin

From: Karunakar P [mailto:karunakarp@amiindia.co.in]
Sent: Friday, November 17, 2017 2:32 PM
To: Wu, Jiaxin <jiaxin.wu@intel.com<mailto:jiaxin.wu@intel.com>>; 'edk2-devel@lists.01.org' <edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>>
Cc: Fu, Siyuan <siyuan.fu@intel.com<mailto:siyuan.fu@intel.com>>; Ye, Ting <ting.ye@intel.com<mailto:ting.ye@intel.com>>
Subject: RE: SCT Test crashes After HTTPS boot success.

Hi Jiaxin,

Below are the detailed steps for SCT test

[Steps to reproduce]
.
  5. Run SCT test

    a.  Execute the following commands to run SCT test on SUT
             sct -r

  sct -u
   b.  In Test Case Management page, Enable GenericTest
   c.  Press F9 to run the selected test case

[Observation]
synchronous exception occurred in SCT ,Attached the Log for reference.

Thanks,
Karunakar

From: Wu, Jiaxin [mailto:jiaxin.wu@intel.com]
Sent: Friday, November 17, 2017 6:01 AM
To: Karunakar P; 'edk2-devel@lists.01.org'
Cc: Fu, Siyuan; Ye, Ting
Subject: RE: SCT Test crashes After HTTPS boot success.

Hi Karunakar,

Can you provide more detailed info for the SCT test steps? The crash can happen after "Sct.efi -u" or need run any specific test case?

Thanks,
Jiaxin

From: Karunakar P [mailto:karunakarp@amiindia.co.in]
Sent: Thursday, November 16, 2017 4:48 PM
To: 'edk2-devel@lists.01.org' <edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.org>>
Cc: Wu, Jiaxin <jiaxin.wu@intel.com<mailto:jiaxin.wu@intel.com>>; Fu, Siyuan <siyuan.fu@intel.com<mailto:siyuan.fu@intel.com>>; Ye, Ting <ting.ye@intel.com<mailto:ting.ye@intel.com>>
Subject: SCT Test crashes After HTTPS boot success.

Hello All,

When I do SCT test, get the below failure
GenericTest\EFICompliantTest 0 0 CB6F7B77-0B15-43F7-A95B-8C7F9FD70B21 FAIL UEFI Compliant - TLS support is required

The reason for the failure is that the platform doesn't have TLS related protocols installed.

->if the platform supports TLS feature then, EFI_TLS_PROTOCOL, EFI_TLS_SERVICE_BINDING_PROTOCOL, EFI_TLS_CONFIGURATION_PROTOCOL must be existed.
-> According UEFI2.7 Spec - 28.10 EFI TLS Protocols(page-1787)
The TLS consumer need locate EFI_TLS_SERVICE_BINDING_PROTOCOL and call CreateChild() to create a new child of EFI_TLS_PROTOCOL instance. Then use EFI_TLS_PROTOCOL to start TLS session. After use, the TLS consumer need call DestroyChild() to destroy it.
-> Network Stack follows same in HTTPS boot.
While doing IPv4/6 HTTPS boot , will locate gEfiTlsServiceBindingProtocolGuid and call TlsServiceBindingCreateChild, So that EFI_TLS_PROTOCOL and EFI_TLS_CONFIGURATION_PROTOCOL will be installed.

So once HTTPS boot is success then TLS supported protocols exist.

And if we do SCT test after HTTPS boot is success, Then TLS related failures should NOT happen in SCT.

I've tried SCT test after HTTPS boot success, But SCT test Crashes.

[Steps to reproduce]

1.       Configure the HTTPS Server with EFI Shell as NBP file.

2.       Connect test machine and HTTPS server with LAN cable.

3.       Perform HTTPS boot in test machine

4.       Once HTTPS boot is success, It will launch Shell.

5.       Run SCT test


[Observations]

1.       SCT test was crashed and unable to continue the test.

Could you please provide your comments/Suggestion on this?


Thanks,
Karunakar