From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (NAM11-BN8-obe.outbound.protection.outlook.com [40.107.236.61]) by mx.groups.io with SMTP id smtpd.web10.12429.1650055907848739225 for ; Fri, 15 Apr 2022 13:51:48 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@amd.com header.s=selector1 header.b=HstBB7PD; spf=permerror, err=parse error for token &{10 18 %{i}._ip.%{h}._ehlo.%{d}._spf.vali.email}: invalid domain name (domain: amd.com, ip: 40.107.236.61, mailfrom: thomas.lendacky@amd.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=cTYuHiK5EvZy4BLmWaxyyeb9Elz+GqvuY6OUP+Q3IUDNUxCLKrsVKa2WNt7Bb1HToL6DEQPdcUFCMsu5rOubRIuxVvl6e928ju90qeEkkI+pg+PDIQJFE5gkz4hozBNNtFBNQ9L8hWEn/1o2Qoz60JTRgGEpFq7yhIZAUi6xg8UvUYMYcc6QU0YmXDPYMuLGgJ3AWSu6+fZwXgb+JawyB8owWFjFH5FItg5ZXyGMdjDi+JXvxLEDEtAuNcM60Qcn+7QcKp8c9yZMWJRegVV1V/PYbAyWsSvCFB1t4nEGqyt8Zbe9+poQ7TKCIFE9J2lNZcsDKJusRTZyGuFZ9kO/IA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=K376TdH7/L/9gs3JN2JSWHt97l59OXVeGI5gzoOqnl8=; b=GDIXQ2x6uNH1Zx4dQs2oTwEteOh/lQsShKiveibEuUkFGGmhpaiLaPVT0KV0gKSvML1KrOXX0GDTC3RXj5Q45asZ3Dqe14atF9WL0Ju59vJprtWRg2918WTS+1b5j1I7rlvzuw3OLMN+S66230jIp3nO1BsTnX6AW1NUH1cNXUbNBmvNlWmIKPwiZR1r6MCwm0YlXEc/qw/9IZBJKITkWQcQGoIs9gBR8hK8TzzA96e1Y3Z45C3efcoVIB8ogW0oS8+iQZxI319zt63bCB5limkVWegkshvbM+W2z99MT0UvPlCUybUphkFzdevD/PWhCoPH6RsQv9jSL2rVFoOV7g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=K376TdH7/L/9gs3JN2JSWHt97l59OXVeGI5gzoOqnl8=; b=HstBB7PDhJDwKSuEX6F0PXHijzJoGzkRbWoSTVfJVCs5hYu9EjwwdzUFovL2+k8XWPqX9gCRpXfYA6V7Am34WB+EYgSOL9VXxT4/qtrkqB0N1Z448oCeGtP/Cau6VSmcbtv6CHeGKnsqqoBdDidUJhlkWVgWwf8EYvM1KqMEnfU= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=amd.com; Received: from DM4PR12MB5229.namprd12.prod.outlook.com (2603:10b6:5:398::12) by DM6PR12MB4250.namprd12.prod.outlook.com (2603:10b6:5:21a::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5164.20; Fri, 15 Apr 2022 20:51:45 +0000 Received: from DM4PR12MB5229.namprd12.prod.outlook.com ([fe80::781d:15d6:8f63:a4e7]) by DM4PR12MB5229.namprd12.prod.outlook.com ([fe80::781d:15d6:8f63:a4e7%5]) with mapi id 15.20.5164.020; Fri, 15 Apr 2022 20:51:44 +0000 Message-ID: <5832e647-63d5-6bbe-4daa-f2f5591a7324@amd.com> Date: Fri, 15 Apr 2022 15:51:42 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.5.0 To: Min Xu , devel@edk2.groups.io Cc: Ard Biesheuvel , Jordan Justen , Brijesh Singh , Erdem Aktas , James Bottomley , Jiewen Yao , Gerd Hoffmann References: <4c3aa6915fe7aac06940bea0f9bc5fdd3c539121.1648555175.git.min.m.xu@intel.com> From: "Lendacky, Thomas" Subject: Re: [PATCH V12 42/47] OvmfPkg: Add TdxDxe driver In-Reply-To: <4c3aa6915fe7aac06940bea0f9bc5fdd3c539121.1648555175.git.min.m.xu@intel.com> X-ClientProxiedBy: SN6PR05CA0005.namprd05.prod.outlook.com (2603:10b6:805:de::18) To DM4PR12MB5229.namprd12.prod.outlook.com (2603:10b6:5:398::12) Return-Path: Thomas.Lendacky@amd.com MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 320f9d22-71c6-4d1c-08c7-08da1f21c02d X-MS-TrafficTypeDiagnostic: DM6PR12MB4250:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: WM6sVSjjCAdVNpFA/DVdi0+kTyeHMjt/f+FebbOzzI1iGDe7RhVUhDD0rWOzCeqWkvpLHbjc/PhPu1VCnX38zzEHVRUMKMS8ttdCbKnBS8lBmq/ax4CfRrTdDIQJ/erMv876yziy6KlD2NdiruskxxhadfNvT42NwemtGPqcsz3UPseWb6Ev1lhNr8OscbeeVKCO93k0B8JPHVMbSl8fKBX8j9ZHiJb9arTkRsCwvnFR97pR7NuhNZ00u0Q9AjWWpxUDg1WH0C5FN9cqTqBgx6IBPiTQFwtfgxsZSQZePZNcBvMoGvgG+7n5RK1dXGy7FLpQETgxHWrbcwXQflPw1Whmbj916VCr9+jM52vcRPnIusuOBBtF0q8RarbcP4i+wgIwpcpxIOP+7oyT2I68tOjyzHw92t/WUwEmyi+Iqq4lNDwL2Vmt1DbRMLq+KEq31Ev4bm0O+7eEANtiPPi1EG5jEkBtnXxPlpKZYbnuzO1bqJ04Y8GefNEUK791PJC4BUzPgpcwBn/dFyy6KRFhAnfxcZkJjnvi8qwMMax+02g1uhnSmZYxWS86PF5p5ys/e2s1R7d+9xtOjyJLfeXiZPyNmjuKsEOJNdgrfundsqpT9KULMFJ6JgTmiEnus1nvUhTHpu/tyFiZJnxGwYCaUi6NxM074qcxlBGPA/Cirw2jhJfl4UnMl3fVi8rR/+X6c3NCVNVFKvdQOmjGuOM5W+9TPyeY4GTbofXvqrEWt7PNiIGykjbIQbP6bvVyR26ma7pdeA7B1k1XNyjHkCF2D2h8wVEZRqy1LBxZjNcc0nSEj7RMHIO3bHVjzkNeq0QbEeDRtXp9U2doXFFKDHN0OAKDSO3vqY7N1dbT9qWlpXXiEtNgk9z/PnHlLkX65o1w X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR12MB5229.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(31696002)(6512007)(186003)(6506007)(26005)(2616005)(508600001)(53546011)(38100700002)(31686004)(19627235002)(66556008)(4326008)(66946007)(8676002)(66476007)(83380400001)(966005)(6486002)(54906003)(86362001)(316002)(8936002)(2906002)(36756003)(5660300002)(43740500002)(45980500001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?RzVRVzVuVFRDVEtIeGJZdFhwU3psRHE3TkQ2dTkrZXdXYUxiamIxOFpSTWVr?= =?utf-8?B?UkcxVkJtNE12K1ZESG5nUXFrSmdlM0ZLbFNJeSs0cUlNZEw2MTV5ajhsMDEr?= =?utf-8?B?VGpaR1hxWDlJWHJrYXR0cjBpV0lKenVaRzdxR1BQVFpRSFJMVWdDem5BUHQ5?= =?utf-8?B?ZFF5REVrZ2ZJQXdkQTJKS1hLL3U0R0xGQmZST21JcHNUc0dpR1JqdkV4Kytk?= =?utf-8?B?Q21kY1Z4VEVsVUNxUmkxQ29uV2t2WmNvYm1kVERkSm5KTzNtMHp4YllFNDlt?= =?utf-8?B?WGRpVVN0RkVsVllCWUZEZmtJeEMyYXRQVVk4eEVmL3kreVlkdWJKVkE3UC9I?= =?utf-8?B?ZVlWd1hRLzRxVzU2SVR1YVErTldtS0VINEx0RUY5QlYwc2hiQnI0WFZhUkdG?= =?utf-8?B?S1hXcnhFTnVxKzNLdTcveVJGR2lhNmQ4TXZ2RUova3hKaWVHVTUxZ3hyNlY0?= =?utf-8?B?YTBZZ2ZmSUZaVDA5L2k5ZWxXbFNyZ21IdVBjcVFOOFBwTENtZ3lVZGVZcnVt?= =?utf-8?B?Z0pFbXdGdGNpMzhaT1NUMTI4L2RDQS9DNmFKSUxPc0hnbEk5Nm5xby9oSDYr?= =?utf-8?B?VzBvaks4eGh1ZWFMK1BJRFJHQ0dOZzg0MW1NU05xa2xKOVJMb1EweFRuRi9G?= =?utf-8?B?NnNsWklsQml6RGwrYWh5TTRTZTQ0MFJZU01VQzFOOUFGRTVQRHlJcGMvTDRJ?= =?utf-8?B?dE9tdG9PWFdJajVqeWNUY3kzbldXQ2QxZi9GMHMyMkE3Y1B5RnpCc2xPNXdv?= =?utf-8?B?TWNCejBaV1BOWVpTeU1UMlUzUlE0S1J0RmUyWmlIVXVaTmJzRkhKWU92U0N2?= =?utf-8?B?MTlQZkxiTlB6MXgrQnNSRWJXbUtiOHdBNFJnTnhFbHY3RzF4Mld4TjZ1MWp4?= =?utf-8?B?c3d2QXZNWWxpRlhybE4vd1M5V01CZVRoQm1GU1A3eC9LTkJXQUFxTERQeFRy?= =?utf-8?B?K3QrbXViS2d0ME5KdG8zUWJtbHUxWjl2eVdvUktZNDBrMEtXb3lDcjBwMjB1?= =?utf-8?B?Q1RIWWhEc082dzdXVnNvUXduOWlUTUlkS0dLVmFFVVBJYjBVNmtxWXRLSzRm?= =?utf-8?B?ZFZzL1dnemsvM2RvWkFyVzNsN3AyczVuOFh3VGgwQmVSeUV4R3ZRbkJpMDA2?= =?utf-8?B?N1IvT0lzU3Q5TlhxTk5SSDdNOFYvdGppS1N1N0Qzb3VtWGtaRGdVdTJyb2FF?= =?utf-8?B?Zk5ONTVsbUxHdmpqd1pwR1REYUdhQnliQVQ2NHdTdjVGc2Q3dUtycE10QWJ0?= =?utf-8?B?VllaQ0crZ20vZTZFVlByMDNMTUtFWFpHMVBoTXN4dkRDREZnODlPSXB6SXVl?= =?utf-8?B?Ly9NUVYwYlEva3pCbkU4NGt2ZWduSTFCZXlsTFZaV0RrcEEvd0NNU2txdXF3?= =?utf-8?B?dzkxUmhORUVwZVZOQnF6OXAvNzdndm1iamVsaWUzS3RqcXBNb3NrdTU4OWpj?= =?utf-8?B?bTcwNjZJOEtVdTNHakJDR2xhZ2J0U21nNDUzcVFSVWdlSzBKTjRtdk9GQ3Zw?= =?utf-8?B?aUdac3JFMjkxWVp3ZVZXdlRHeUFkRFJRZ0FtZXM1NXJJWEthUS9weGFMZk54?= =?utf-8?B?WDVJL2I1cWlKNGp4R3pHcmxjZ2wvQUxHSHYxZ3ZFWGhxMnpZdUtIY3hiL1Bm?= =?utf-8?B?bUdDSk1CeDdydXl2RHFQYlBRZ1pQUVk1MFV4T3hTdERGWHkweHJFQnBUc2lq?= =?utf-8?B?ZHVld0YycWJYRHBHMk9qYVRHSTVieUxKWHlrdzBxTWRBTDR4VkJoZzlHQ3Ry?= =?utf-8?B?ejBIVVJlU1ZVanRzT0hIMk1PME90WWFIQXFOTk5aMnBvOTVxYnlkZCtuUm8y?= =?utf-8?B?ck13YlFZeW9hbndxUHkxL3dRbEFVQXplUm53YXg1RU5vRDV4UVhJWWVjNWlj?= =?utf-8?B?MUpDZDJOVURCTzA3ZHVvMmloRXQwRFlQUGw4NFUrR2trakJOT1RIdnJONUNz?= =?utf-8?B?ZW5vMy8wdityR2s4YWUxTXdKd1lkdmNjZTV4aWtjTm5hZE5FOWdCV2ppNys3?= =?utf-8?B?WW9KQXRDTTl5c3FmWFBhWS9tRks1aFk5TGNYUHlDdFBJTGZzN3J6ZkpJVVYz?= =?utf-8?B?aEN3L1grY1h3b3BmajFxUWRRcllZYUZ3SW1Cb2V2Y1FRRnMvdVlnbit5WG1S?= =?utf-8?B?RXFmTlR2N3ZMUk1aRlNLTDRlQUhSY0tzTmtubkUySlEwYk5XQ1BuWnhydlo0?= =?utf-8?B?dDBkZkhIUUZ3WXJWZWRnU1FpSnY3dnAvVjZWeFBUSnhkMnhNcU8yMitHbXZZ?= =?utf-8?B?UnByWThNaUFZZTdzQ2JqaUU3ZXVhRGN0eWlQYjJzVlVoYjQwV3I1RS9Pdkp5?= =?utf-8?B?U2s5MW1vazB0NkVyMXFuUHBnMjh2K1dnbzAxcGR0T28yT0liUG9iQT09?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 320f9d22-71c6-4d1c-08c7-08da1f21c02d X-MS-Exchange-CrossTenant-AuthSource: DM4PR12MB5229.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Apr 2022 20:51:44.6623 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: y0vP5kvv6Q5VcKTT9Y9P3UJaK2IEfhfG7dofXk8e1quN/atsfhhyFGJzCgqLlneIwJ23WaTUHKfprhmsQGuhdg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4250 Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit On 3/29/22 18:46, Min Xu wrote: > RFC: https://bugzilla.tianocore.org/show_bug.cgi?id=3429 > > TdxDxe driver is dispatched early in DXE, due to being list in APRIORI. > This module is responsible for below features: > - Sets max logical cpus based on TDINFO > - Sets PCI PCDs based on resource hobs > - Set shared bit in MMIO region > - Relocate Td mailbox and set its address in MADT table. > > 1. Set shared bit in MMIO region > > Qemu allows a ROM device to set to ROMD mode (default) or MMIO mode. > When it is in ROMD mode, the device is mapped to guest memory and > satisfies read access directly. > > In EDK2 Option ROM is treated as MMIO region. So Tdx guest access > Option ROM via TDVMCALL(MMIO). But as explained above, since Qemu set > the Option ROM to ROMD mode, the call of TDVMCALL(MMIO) always return > INVALID_OPERAND. Tdvf then falls back to direct access. This requires > to set the shared bit to corresponding PageTable entry. Otherwise it > triggers GP fault. > > TdxDxe's entry point is the right place to set the shared bit in MMIO > region because Option ROM has not been discoverd yet. > > 2. Relocate Td mailbox and set the new address in MADT Mutiprocessor > Wakeup Table. > > In TDX the guest firmware is designed to publish a multiprocessor-wakeup > structure to let the guest-bootstrap processor wake up guest-application > processors with a mailbox. The mailbox is memory that the guest firmware > can reserve so each guest virtual processor can have the guest OS send > a message to them. The address of the mailbox is recorded in the MADT > table. See [ACPI]. > > TdxDxe registers for protocol notification > (gQemuAcpiTableNotifyProtocolGuid) to call the AlterAcpiTable(), in > which MADT table is altered by the above Mailbox address. The protocol > will be installed in AcpiPlatformDxe when the MADT table provided by > Qemu is ready. This is to maintain the simplicity of the AcpiPlatformDxe. > > AlterAcpiTable is the registered function which traverses the ACPI > table list to find the original MADT from Qemu. After the new MADT is > configured and installed, the original one will be uninstalled. > > [ACPI] https://uefi.org/specs/ACPI/6.4/05_ACPI_Software_Programming_Model > /ACPI_Software_Programming_Model.html#multiprocessor-wakeup-structure > > Cc: Ard Biesheuvel > Cc: Jordan Justen > Cc: Brijesh Singh > Cc: Erdem Aktas > Cc: James Bottomley > Cc: Jiewen Yao > Cc: Tom Lendacky > Cc: Gerd Hoffmann > Acked-by: Gerd Hoffmann > Signed-off-by: Min Xu Unfortunately, this driver also breaks SEV-ES. I bypassed the TDX code in the SEC library, but then hit an issue because this driver is loaded before the AmdSevDxe driver. The AmdSevDxe driver performs a MemEncryptSevClearMmioPageEncMask() call against the PcdPciExpressBaseAddress range to mark it shared/unencrypted. However, the TdxDxe driver is loaded before the AmdSevDxe driver, and it appears the dependencies result in an MMIO being performed to an address in the PcdPciExpressBaseAddress range. Since the range has not been marked shared/unencrypted, the #VC handler terminates the guest for trying to do MMIO to an encrypted region. Here is the error information: Loading driver at 0x0003F1A5000 EntryPoint=0x0003F1A6BF2 TdxDxe.efi InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 3F1AFA98 ProtectUefiImageCommon - 0x3F1AFB40 - 0x000000003F1A5000 - 0x0000000000004300 MMIO using encrypted memory: B00F8040 !!!! X64 Exception Type - 0D(#GP - General Protection) CPU Apic ID - 00000000 !!!! ExceptionData - 0000000000000000 RIP - 000000003F1A6E86, CS - 0000000000000018, RFLAGS - 0000000000010002 RAX - 00000000B0000000, RCX - 000000003F4EAD18, RDX - 000000003F4EAD01 RBX - 0000000000000001, RSP - 000000003FBA5B60, RBP - 0000000000000000 RSI - 0000000000000000, RDI - 000000003F1AFB18 R8 - 00000000B00F8040, R9 - 00000000000029C0, R10 - 0000000000000000 R11 - 0000000000000000, R12 - 0000000000000000, R13 - 0000000000000000 R14 - 000000003FBCC6D0, R15 - 000000003FBC43E6 DS - 0000000000000008, ES - 0000000000000008, FS - 0000000000000008 GS - 0000000000000008, SS - 0000000000000008 CR0 - 0000000080010033, CR2 - 0000000000000000, CR3 - 000000003F801000 CR4 - 0000000000000668, CR8 - 0000000000000000 DR0 - 0000000000000000, DR1 - 0000000000000000, DR2 - 0000000000000000 DR3 - 0000000000000000, DR6 - 0000000000000000, DR7 - 0000000000000000 GDTR - 000000003FBFE000 0000000000000027, LDTR - 0000000000000000 IDTR - 000000003BF95D70 000000000000021F, TR - 0000000000000000 FXSAVE_STATE - 000000003FBA57C0 !!!! Find image based on IP(0x3F1A6E86) /root/kernels/ovmf-build-X64/Build/OvmfX64/DEBUG_GCC5/X64/OvmfPkg/TdxDxe/TdxDxe/DEBUG/TdxDxe.dll (ImageBase=000000003F1A5000, EntryPoint=000000003F1A6BF2) !!!! Not sure what the best way forward is on this. Thanks, Tom > ---