[edk2-devel] Python368.efi failed to run in shell environment

[edk2-devel] Python368.efi failed to run in shell environment

[Yoshinoya]

[-- Attachment #1: Type: text/plain, Size: 284 bytes --]

Hi
I tried to run Python368.efi in shell environment.
but failed, the tips was;
Fatal Python error: Py_Initialize: can't initialize sys standard streams


Does anybody have ever met this error?


Python368.efi is a sample python app in AppPkg\Applications\Python\Python-3.6.8


Thanks

[-- Attachment #2: Type: text/html, Size: 656 bytes --]

Re: [edk2-devel] Python368.efi failed to run in shell environment

[Michael D Kinney]

[-- Attachment #1: Type: text/plain, Size: 525 bytes --]

+JP

Mike

From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Yoshinoya
Sent: Wednesday, October 5, 2022 4:32 AM
To: devel@edk2.groups.io
Subject: [edk2-devel] Python368.efi failed to run in shell environment

Hi
I tried to run Python368.efi in shell environment.
but failed, the tips was;
Fatal Python error: Py_Initialize: can't initialize sys standard streams

Does anybody have ever met this error?

Python368.efi is a sample python app in AppPkg\Applications\Python\Python-3.6.8

Thanks


[-- Attachment #2: Type: text/html, Size: 41372 bytes --]

Re: [edk2-devel] Python368.efi failed to run in shell environment

[Yoshinoya]

[-- Attachment #1: Type: text/plain, Size: 751 bytes --]

Hi, JP:
It seems PyImport_ImportModule("encoding.utf_8") failed.
Do you have any suggestions?


THank you very much!




best wishes,










At 2022-10-06 00:28:46, "Kinney, Michael D" <michael.d.kinney@intel.com> wrote:

+JP

 

Mike

 

From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Yoshinoya
Sent: Wednesday, October 5, 2022 4:32 AM
To: devel@edk2.groups.io
Subject: [edk2-devel] Python368.efi failed to run in shell environment

 

Hi

I tried to run Python368.efi in shell environment.

but failed, the tips was;

Fatal Python error: Py_Initialize: can't initialize sys standard streams

 

Does anybody have ever met this error?

 

Python368.efi is a sample python app in AppPkg\Applications\Python\Python-3.6.8

 

Thanks


[-- Attachment #2: Type: text/html, Size: 4246 bytes --]

Re: [edk2-devel] Python368.efi failed to run in shell environment

[Jayaprakash, N]

[-- Attachment #1: Type: text/plain, Size: 1415 bytes --]

Hi Yoshinoya,

Could you provide some details about the platform on which you are trying to run?
What tools you used for building the Python interpreter?

If you could raise a bug with relevant details, we can take a look at this issue.

Regards,
JP

From: Yoshinoya <yoshinoyatoko@163.com>
Sent: 11 October 2022 15:51
To: Kinney, Michael D <michael.d.kinney@intel.com>
Cc: devel@edk2.groups.io; Jayaprakash, N <n.jayaprakash@intel.com>
Subject: Re:RE: [edk2-devel] Python368.efi failed to run in shell environment

Hi, JP:
It seems PyImport_ImportModule("encoding.utf_8") failed.
Do you have any suggestions?

THank you very much!



best wishes,







At 2022-10-06 00:28:46, "Kinney, Michael D" <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>> wrote:
+JP

Mike

From: devel@edk2.groups.io<mailto:devel@edk2.groups.io> <devel@edk2.groups.io<mailto:devel@edk2.groups.io>> On Behalf Of Yoshinoya
Sent: Wednesday, October 5, 2022 4:32 AM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>
Subject: [edk2-devel] Python368.efi failed to run in shell environment

Hi
I tried to run Python368.efi in shell environment.
but failed, the tips was;
Fatal Python error: Py_Initialize: can't initialize sys standard streams

Does anybody have ever met this error?

Python368.efi is a sample python app in AppPkg\Applications\Python\Python-3.6.8

Thanks


[-- Attachment #2: Type: text/html, Size: 11282 bytes --]

Re: [edk2-devel] Python368.efi failed to run in shell environment

[Yoshinoya]

[-- Attachment #1: Type: text/plain, Size: 1939 bytes --]

Hi, JP:
I followed Py368ReadMe.txt to compile Python368 app in Ubuntu 22.04 environment.
The steps is:
1. execute srcprep.py
2. add this module to AppPkg/AppPkg.dsc
3. execute "build -a X64 -p AppPkg\AppPkg.dsc"


but it failed, the tips is:
In file included from /home/yoshi/edk2/AppPkg/Applications/Python/Python-3.6.8/Modules/_bisectmodule.c:7:
/home/yoshi/edk2/AppPkg/Applications/Python/Python-3.6.8/Include/Python.h:44:10: fatal error: crypt.h: No such file or directory
   44 | #include <crypt.h>
      |          ^~~~~~~~~
compilation terminated.






Thanks










At 2022-10-18 17:04:42, "Jayaprakash, N" <n.jayaprakash@intel.com> wrote:

Hi Yoshinoya,

 

Could you provide some details about the platform on which you are trying to run?

What tools you used for building the Python interpreter?

 

If you could raise a bug with relevant details, we can take a look at this issue.

 

Regards,

JP

 

From: Yoshinoya <yoshinoyatoko@163.com>
Sent: 11 October 2022 15:51
To: Kinney, Michael D <michael.d.kinney@intel.com>
Cc: devel@edk2.groups.io; Jayaprakash, N <n.jayaprakash@intel.com>
Subject: Re:RE: [edk2-devel] Python368.efi failed to run in shell environment

 

Hi, JP:

It seems PyImport_ImportModule("encoding.utf_8") failed.

Do you have any suggestions?

 

THank you very much!

 

best wishes,

 

 

 

At 2022-10-06 00:28:46, "Kinney, Michael D" <michael.d.kinney@intel.com> wrote:

+JP

 

Mike

 

From:devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Yoshinoya
Sent: Wednesday, October 5, 2022 4:32 AM
To:devel@edk2.groups.io
Subject: [edk2-devel] Python368.efi failed to run in shell environment

 

Hi

I tried to run Python368.efi in shell environment.

but failed, the tips was;

Fatal Python error: Py_Initialize: can't initialize sys standard streams

 

Does anybody have ever met this error?

 

Python368.efi is a sample python app in AppPkg\Applications\Python\Python-3.6.8

 

Thanks


[-- Attachment #2: Type: text/html, Size: 10858 bytes --]

Re: [edk2-devel] Python368.efi failed to run in shell environment

[Jayaprakash, N]

[-- Attachment #1: Type: text/plain, Size: 2696 bytes --]

You may follow instructions provided in https://github.com/tianocore/edk2-libc/blob/master/AppPkg/Applications/Python/Python-3.6.8/GCCCompilationBKMs.rst

Regards,
JP

From: Yoshinoya <yoshinoyatoko@163.com>
Sent: 19 October 2022 14:01
To: Jayaprakash, N <n.jayaprakash@intel.com>
Cc: Kinney, Michael D <michael.d.kinney@intel.com>; devel@edk2.groups.io
Subject: Re:RE: Re:RE: [edk2-devel] Python368.efi failed to run in shell environment

Hi, JP:
I followed Py368ReadMe.txt to compile Python368 app in Ubuntu 22.04 environment.
The steps is:
1. execute srcprep.py
2. add this module to AppPkg/AppPkg.dsc
3. execute "build -a X64 -p AppPkg\AppPkg.dsc"

but it failed, the tips is:
In file included from /home/yoshi/edk2/AppPkg/Applications/Python/Python-3.6.8/Modules/_bisectmodule.c:7:
/home/yoshi/edk2/AppPkg/Applications/Python/Python-3.6.8/Include/Python.h:44:10: fatal error: crypt.h: No such file or directory
   44 | #include <crypt.h>
      |          ^~~~~~~~~
compilation terminated.




Thanks







At 2022-10-18 17:04:42, "Jayaprakash, N" <n.jayaprakash@intel.com<mailto:n.jayaprakash@intel.com>> wrote:
Hi Yoshinoya,

Could you provide some details about the platform on which you are trying to run?
What tools you used for building the Python interpreter?

If you could raise a bug with relevant details, we can take a look at this issue.

Regards,
JP

From: Yoshinoya <yoshinoyatoko@163.com<mailto:yoshinoyatoko@163.com>>
Sent: 11 October 2022 15:51
To: Kinney, Michael D <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>
Cc: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Jayaprakash, N <n.jayaprakash@intel.com<mailto:n.jayaprakash@intel.com>>
Subject: Re:RE: [edk2-devel] Python368.efi failed to run in shell environment

Hi, JP:
It seems PyImport_ImportModule("encoding.utf_8") failed.
Do you have any suggestions?

THank you very much!



best wishes,







At 2022-10-06 00:28:46, "Kinney, Michael D" <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>> wrote:
+JP

Mike

From: devel@edk2.groups.io<mailto:devel@edk2.groups.io> <devel@edk2.groups.io<mailto:devel@edk2.groups.io>> On Behalf Of Yoshinoya
Sent: Wednesday, October 5, 2022 4:32 AM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>
Subject: [edk2-devel] Python368.efi failed to run in shell environment

Hi
I tried to run Python368.efi in shell environment.
but failed, the tips was;
Fatal Python error: Py_Initialize: can't initialize sys standard streams

Does anybody have ever met this error?

Python368.efi is a sample python app in AppPkg\Applications\Python\Python-3.6.8

Thanks


[-- Attachment #2: Type: text/html, Size: 19203 bytes --]

Re: [edk2-devel] Python368.efi failed to run in shell environment

[Yoshinoya]

[-- Attachment #1: Type: text/plain, Size: 2623 bytes --]

Hi, JP:
Thanks a lot!
It works.


And using create_python368_pkg.sh to create U disk image, and it runs ok in shell.







best wishes,










At 2022-10-19 21:16:47, "Jayaprakash, N" <n.jayaprakash@intel.com> wrote:

You may follow instructions provided in https://github.com/tianocore/edk2-libc/blob/master/AppPkg/Applications/Python/Python-3.6.8/GCCCompilationBKMs.rst

 

Regards,

JP

 

From: Yoshinoya <yoshinoyatoko@163.com>
Sent: 19 October 2022 14:01
To: Jayaprakash, N <n.jayaprakash@intel.com>
Cc: Kinney, Michael D <michael.d.kinney@intel.com>; devel@edk2.groups.io
Subject: Re:RE: Re:RE: [edk2-devel] Python368.efi failed to run in shell environment

 

Hi, JP:

I followed Py368ReadMe.txt to compile Python368 app in Ubuntu 22.04 environment.

The steps is:

1. execute srcprep.py

2. add this module to AppPkg/AppPkg.dsc

3. execute "build -a X64 -p AppPkg\AppPkg.dsc"

 

but it failed, the tips is:

In file included from /home/yoshi/edk2/AppPkg/Applications/Python/Python-3.6.8/Modules/_bisectmodule.c:7:

/home/yoshi/edk2/AppPkg/Applications/Python/Python-3.6.8/Include/Python.h:44:10: fatal error: crypt.h: No such file or directory

   44 | #include <crypt.h>

      |          ^~~~~~~~~

compilation terminated.

 

 

Thanks

 

 

 

At 2022-10-18 17:04:42, "Jayaprakash, N" <n.jayaprakash@intel.com> wrote:

Hi Yoshinoya,

 

Could you provide some details about the platform on which you are trying to run?

What tools you used for building the Python interpreter?

 

If you could raise a bug with relevant details, we can take a look at this issue.

 

Regards,

JP

 

From: Yoshinoya <yoshinoyatoko@163.com>
Sent: 11 October 2022 15:51
To: Kinney, Michael D <michael.d.kinney@intel.com>
Cc:devel@edk2.groups.io; Jayaprakash, N <n.jayaprakash@intel.com>
Subject: Re:RE: [edk2-devel] Python368.efi failed to run in shell environment

 

Hi, JP:

It seems PyImport_ImportModule("encoding.utf_8") failed.

Do you have any suggestions?

 

THank you very much!

 

best wishes,

 

 

 

At 2022-10-06 00:28:46, "Kinney, Michael D" <michael.d.kinney@intel.com> wrote:

+JP

 

Mike

 

From:devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Yoshinoya
Sent: Wednesday, October 5, 2022 4:32 AM
To:devel@edk2.groups.io
Subject: [edk2-devel] Python368.efi failed to run in shell environment

 

Hi

I tried to run Python368.efi in shell environment.

but failed, the tips was;

Fatal Python error: Py_Initialize: can't initialize sys standard streams

 

Does anybody have ever met this error?

 

Python368.efi is a sample python app in AppPkg\Applications\Python\Python-3.6.8

 

Thanks


[-- Attachment #2: Type: text/html, Size: 18045 bytes --]

Re: [edk2-devel] Python368.efi failed to run in shell environment

[Jayaprakash, N]

[-- Attachment #1: Type: text/plain, Size: 5240 bytes --]

That's perfect. Great to know that you were able to build and run it successfully.

Please let's know if you are interested in contributing to any of the open bugs as given below:

3781<https://bugzilla.tianocore.org/show_bug.cgi?id=3781>
EDK2
Code
unassigned@tianocore.org<mailto:unassigned@tianocore.org>
UNCO
---
edk2-libc: Enhance Py3 UEFI interpreter with the ARM architecture support<https://bugzilla.tianocore.org/show_bug.cgi?id=3781>
2022-01-04
3782<https://bugzilla.tianocore.org/show_bug.cgi?id=3782>
EDK2
Code
unassigned@tianocore.org<mailto:unassigned@tianocore.org>
UNCO
---
edk2-libc: Enhance Py3 UEFI interpreter with the AARCH 64 architecture support<https://bugzilla.tianocore.org/show_bug.cgi?id=3782>
2022-01-04
3783<https://bugzilla.tianocore.org/show_bug.cgi?id=3783>
EDK2
Code
unassigned@tianocore.org<mailto:unassigned@tianocore.org>
UNCO
---
edk2-libc: Enhance Py3 UEFI interpreter with the RISC V 64 architecture support<https://bugzilla.tianocore.org/show_bug.cgi?id=3783>
2022-02-22
3784<https://bugzilla.tianocore.org/show_bug.cgi?id=3784>
EDK2
Code
n.jayaprakash@intel.com<mailto:n.jayaprakash@intel.com>
CONF
---
edk2-libc : Upstreaming UEFI changes to Python project<https://bugzilla.tianocore.org/show_bug.cgi?id=3784>
2022-01-04
3785<https://bugzilla.tianocore.org/show_bug.cgi?id=3785>
Tianocor
Code
n.jayaprakash@intel.com<mailto:n.jayaprakash@intel.com>
CONF
---
edk2-libc : Py3 UEFI port add pip install support<https://bugzilla.tianocore.org/show_bug.cgi?id=3785>
2022-02-15
3786<https://bugzilla.tianocore.org/show_bug.cgi?id=3786>
EDK2
Code
n.jayaprakash@intel.com<mailto:n.jayaprakash@intel.com>
CONF
---
edk2-libc : Enhance Py3 UEFI port to provide access to System Firmware Tables from Python scripts<https://bugzilla.tianocore.org/show_bug.cgi?id=3786>


Regards,
JP

From: Yoshinoya <yoshinoyatoko@163.com>
Sent: 20 October 2022 08:36
To: Jayaprakash, N <n.jayaprakash@intel.com>
Cc: Kinney, Michael D <michael.d.kinney@intel.com>; devel@edk2.groups.io
Subject: Re:RE: Re:RE: Re:RE: [edk2-devel] Python368.efi failed to run in shell environment

Hi, JP:
Thanks a lot!
It works.

And using create_python368_pkg.sh to create U disk image, and it runs ok in shell.




best wishes,







At 2022-10-19 21:16:47, "Jayaprakash, N" <n.jayaprakash@intel.com<mailto:n.jayaprakash@intel.com>> wrote:
You may follow instructions provided in https://github.com/tianocore/edk2-libc/blob/master/AppPkg/Applications/Python/Python-3.6.8/GCCCompilationBKMs.rst

Regards,
JP

From: Yoshinoya <yoshinoyatoko@163.com<mailto:yoshinoyatoko@163.com>>
Sent: 19 October 2022 14:01
To: Jayaprakash, N <n.jayaprakash@intel.com<mailto:n.jayaprakash@intel.com>>
Cc: Kinney, Michael D <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>; devel@edk2.groups.io<mailto:devel@edk2.groups.io>
Subject: Re:RE: Re:RE: [edk2-devel] Python368.efi failed to run in shell environment

Hi, JP:
I followed Py368ReadMe.txt to compile Python368 app in Ubuntu 22.04 environment.
The steps is:
1. execute srcprep.py
2. add this module to AppPkg/AppPkg.dsc
3. execute "build -a X64 -p AppPkg\AppPkg.dsc"

but it failed, the tips is:
In file included from /home/yoshi/edk2/AppPkg/Applications/Python/Python-3.6.8/Modules/_bisectmodule.c:7:
/home/yoshi/edk2/AppPkg/Applications/Python/Python-3.6.8/Include/Python.h:44:10: fatal error: crypt.h: No such file or directory
   44 | #include <crypt.h>
      |          ^~~~~~~~~
compilation terminated.




Thanks







At 2022-10-18 17:04:42, "Jayaprakash, N" <n.jayaprakash@intel.com<mailto:n.jayaprakash@intel.com>> wrote:
Hi Yoshinoya,

Could you provide some details about the platform on which you are trying to run?
What tools you used for building the Python interpreter?

If you could raise a bug with relevant details, we can take a look at this issue.

Regards,
JP

From: Yoshinoya <yoshinoyatoko@163.com<mailto:yoshinoyatoko@163.com>>
Sent: 11 October 2022 15:51
To: Kinney, Michael D <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>
Cc: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Jayaprakash, N <n.jayaprakash@intel.com<mailto:n.jayaprakash@intel.com>>
Subject: Re:RE: [edk2-devel] Python368.efi failed to run in shell environment

Hi, JP:
It seems PyImport_ImportModule("encoding.utf_8") failed.
Do you have any suggestions?

THank you very much!



best wishes,







At 2022-10-06 00:28:46, "Kinney, Michael D" <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>> wrote:
+JP

Mike

From: devel@edk2.groups.io<mailto:devel@edk2.groups.io> <devel@edk2.groups.io<mailto:devel@edk2.groups.io>> On Behalf Of Yoshinoya
Sent: Wednesday, October 5, 2022 4:32 AM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>
Subject: [edk2-devel] Python368.efi failed to run in shell environment

Hi
I tried to run Python368.efi in shell environment.
but failed, the tips was;
Fatal Python error: Py_Initialize: can't initialize sys standard streams

Does anybody have ever met this error?

Python368.efi is a sample python app in AppPkg\Applications\Python\Python-3.6.8

Thanks


[-- Attachment #2: Type: text/html, Size: 38238 bytes --]

[edk2-devel] Access 64bit address space in 32bit mode

[Yoshinoya]

[-- Attachment #1: Type: text/plain, Size: 267 bytes --]

Hello
Is it possible to access 64bit address space in 32bit mode?


For example, opcode prefix 0x66/67 could let code running in 16bit mode to access 32bit data/address.


Or, establishing page table is a must requirement for accessing 64bit address space.


Thanks
 

[-- Attachment #2: Type: text/html, Size: 625 bytes --]

Re: [edk2-devel] Access 64bit address space in 32bit mode

[Andrew Fish]

[-- Attachment #1: Type: text/plain, Size: 2343 bytes --]


> On Nov 7, 2022, at 7:16 PM, Yoshinoya <yoshinoyatoko@163.com> wrote:
> 
> Hello
> Is it possible to access 64bit address space in 32bit mode?
> 

I assume you are talking about x86?


> For example, opcode prefix 0x66/67 could let code running in 16bit mode to access 32bit data/address.
> 

This is more complex than just instruction prefix. You need the CPU to be setup in big real mode via the GDT, so you basically have a 32-bit environment setup via GDT etc. Also the prefix opcodes have different meaning in different modes. I don’t think there is a way to make 32-bit code access 64-bit data via instruction prefix even if a 64-bit GDT was setup with paging enabled.  

> Or, establishing page table is a must requirement for accessing 64bit address space.
> 

For x86 you have to have 64-bit versions of the IDT, GDT, and you need to enable paging to enter 64-bit Long Mode. 

In a 32-bit x86 world you can access up to 64 GB of physical memory via using 32-bit page table using PAE [1]. PAE is a 32-bit virtual address space, but with support for a 36-bit physical address. I think in the olden days of 32-bit x86 EFI servers would have custom EFI code that enabled paging in 32-bit and carved out a chunk of the 32-bit memory space that could be mapped to 36-bit physical addresses. I think this was platform specific code and I don’t know of any open source version. The 32-bit Long Mode EFI does not have paging enabled, so adding PAE means enabling paging yourself. 

The edk2 has the opposite version of this code so you can call 16-bit really mode (Legacy BIOS) from 32-bit Protected mode, or 64-bit Long Mode. This is the code to Thunk for 32-bit/64-bit mode to 16-bit code [2]/ 

[1] https://en.wikipedia.org/wiki/Physical_Address_Extensio <https://en.wikipedia.org/wiki/Physical_Address_Extension>n <https://en.wikipedia.org/wiki/Physical_Address_Extension>
[2] https://github.com/tianocore/edk2/blob/master/MdePkg/Library/BaseLib/X86Thunk.c
https://github.com/tianocore/edk2/blob/master/MdePkg/Library/BaseLib/X64/Thunk16.nasm
edk2/Thunk16.nasm at master · tianocore/edk2
github.com
https://github.com/tianocore/edk2/blob/master/MdePkg/Library/BaseLib/Ia32/Thunk16.nasm
edk2/Thunk16.nasm at master · tianocore/edk2
github.com

Thanks,

Andrew Fish


> Thanks
>  
> 


[-- Attachment #2.1: Type: text/html, Size: 8768 bytes --]

[-- Attachment #2.2: edk2.png --]
[-- Type: image/png, Size: 70276 bytes --]

[-- Attachment #2.3: edk2.png --]
[-- Type: image/png, Size: 70276 bytes --]

Re: [edk2-devel] Access 64bit address space in 32bit mode

[vincent zimmer]

[-- Attachment #1.1.1: Type: text/plain, Size: 3364 bytes --]

we have the challenge of 32-bit PEI needing to access 64-bit addresses to
support 64-bit DXE/UEFI OS's in the capsule use-case scenario.  This is
described in
https://raw.githubusercontent.com/tianocore-docs/Docs/master/White_Papers/A_Tour_Beyond_BIOS_Capsule_Update_and_Recovery_in_EDK_II.pdf
page 22 with code
https://github.com/tianocore/edk2/tree/master/MdeModulePkg/Universal/CapsulePei

On Wed, Nov 9, 2022 at 9:21 AM Andrew Fish via groups.io <afish=
apple.com@groups.io> wrote:

>
> On Nov 7, 2022, at 7:16 PM, Yoshinoya <yoshinoyatoko@163.com> wrote:
>
> Hello
> Is it possible to access 64bit address space in 32bit mode?
>
>
> I assume you are talking about x86?
>
>
> For example, opcode prefix 0x66/67 could let code running in 16bit mode to
> access 32bit data/address.
>
>
> This is more complex than just instruction prefix. You need the CPU to be
> setup in big real mode via the GDT, so you basically have a 32-bit
> environment setup via GDT etc. Also the prefix opcodes have different
> meaning in different modes. I don’t think there is a way to make 32-bit
> code access 64-bit data via instruction prefix even if a 64-bit GDT was
> setup with paging enabled.
>
> Or, establishing page table is a must requirement for accessing 64bit
> address space.
>
>
> For x86 you have to have 64-bit versions of the IDT, GDT, and you need to
> enable paging to enter 64-bit Long Mode.
>
> In a 32-bit x86 world you can access up to 64 GB of physical memory via
> using 32-bit page table using PAE [1]. PAE is a 32-bit virtual address
> space, but with support for a 36-bit physical address. I think in the olden
> days of 32-bit x86 EFI servers would have custom EFI code that enabled
> paging in 32-bit and carved out a chunk of the 32-bit memory space that
> could be mapped to 36-bit physical addresses. I think this was platform
> specific code and I don’t know of any open source version. The 32-bit Long
> Mode EFI does not have paging enabled, so adding PAE means enabling paging
> yourself.
>
> The edk2 has the opposite version of this code so you can call 16-bit
> really mode (Legacy BIOS) from 32-bit Protected mode, or 64-bit Long Mode.
> This is the code to Thunk for 32-bit/64-bit mode to 16-bit code [2]/
>
> [1] https://en.wikipedia.org/wiki/Physical_Address_Extensio
> <https://en.wikipedia.org/wiki/Physical_Address_Extension>n
> <https://en.wikipedia.org/wiki/Physical_Address_Extension>
> [2]
> https://github.com/tianocore/edk2/blob/master/MdePkg/Library/BaseLib/X86Thunk.c
> [image: edk2.png]
>
> edk2/Thunk16.nasm at master · tianocore/edk2
> <https://github.com/tianocore/edk2/blob/master/MdePkg/Library/BaseLib/X64/Thunk16.nasm>
> github.com
> <https://github.com/tianocore/edk2/blob/master/MdePkg/Library/BaseLib/X64/Thunk16.nasm>
>
> <https://github.com/tianocore/edk2/blob/master/MdePkg/Library/BaseLib/X64/Thunk16.nasm>
> [image: edk2.png]
>
> edk2/Thunk16.nasm at master · tianocore/edk2
> <https://github.com/tianocore/edk2/blob/master/MdePkg/Library/BaseLib/Ia32/Thunk16.nasm>
> github.com
> <https://github.com/tianocore/edk2/blob/master/MdePkg/Library/BaseLib/Ia32/Thunk16.nasm>
>
> <https://github.com/tianocore/edk2/blob/master/MdePkg/Library/BaseLib/Ia32/Thunk16.nasm>
>
> Thanks,
>
> Andrew Fish
>
>
> Thanks
>
>
>
> 
>
>

[-- Attachment #1.1.2: Type: text/html, Size: 8039 bytes --]

[-- Attachment #1.2: edk2.png --]
[-- Type: image/png, Size: 70276 bytes --]

[-- Attachment #2: edk2.png --]
[-- Type: image/png, Size: 70276 bytes --]

Re: [edk2-devel] Access 64bit address space in 32bit mode

[Andrew Fish]

[-- Attachment #1: Type: text/plain, Size: 3797 bytes --]

Vincent,

Thanks! I’d forgotten about that path. 

The other answer is defer the work to a DXE driver that runs 64-bit x86.

Thanks,

Andrew Fish

> On Nov 9, 2022, at 10:58 AM, vincent zimmer <vincent.zimmer@gmail.com> wrote:
> 
> we have the challenge of 32-bit PEI needing to access 64-bit addresses to support 64-bit DXE/UEFI OS's in the capsule use-case scenario.  This is described in https://raw.githubusercontent.com/tianocore-docs/Docs/master/White_Papers/A_Tour_Beyond_BIOS_Capsule_Update_and_Recovery_in_EDK_II.pdf page 22 with code https://github.com/tianocore/edk2/tree/master/MdeModulePkg/Universal/CapsulePei
> 
> On Wed, Nov 9, 2022 at 9:21 AM Andrew Fish via groups.io <http://groups.io/> <afish=apple.com@groups.io <mailto:apple.com@groups.io>> wrote:
>> 
>>> On Nov 7, 2022, at 7:16 PM, Yoshinoya <yoshinoyatoko@163.com <mailto:yoshinoyatoko@163.com>> wrote:
>>> 
>>> Hello
>>> Is it possible to access 64bit address space in 32bit mode?
>>> 
>> 
>> I assume you are talking about x86?
>> 
>> 
>>> For example, opcode prefix 0x66/67 could let code running in 16bit mode to access 32bit data/address.
>>> 
>> 
>> This is more complex than just instruction prefix. You need the CPU to be setup in big real mode via the GDT, so you basically have a 32-bit environment setup via GDT etc. Also the prefix opcodes have different meaning in different modes. I don’t think there is a way to make 32-bit code access 64-bit data via instruction prefix even if a 64-bit GDT was setup with paging enabled.  
>> 
>>> Or, establishing page table is a must requirement for accessing 64bit address space.
>>> 
>> 
>> For x86 you have to have 64-bit versions of the IDT, GDT, and you need to enable paging to enter 64-bit Long Mode. 
>> 
>> In a 32-bit x86 world you can access up to 64 GB of physical memory via using 32-bit page table using PAE [1]. PAE is a 32-bit virtual address space, but with support for a 36-bit physical address. I think in the olden days of 32-bit x86 EFI servers would have custom EFI code that enabled paging in 32-bit and carved out a chunk of the 32-bit memory space that could be mapped to 36-bit physical addresses. I think this was platform specific code and I don’t know of any open source version. The 32-bit Long Mode EFI does not have paging enabled, so adding PAE means enabling paging yourself. 
>> 
>> The edk2 has the opposite version of this code so you can call 16-bit really mode (Legacy BIOS) from 32-bit Protected mode, or 64-bit Long Mode. This is the code to Thunk for 32-bit/64-bit mode to 16-bit code [2]/ 
>> 
>> [1] https://en.wikipedia.org/wiki/Physical_Address_Extensio <https://en.wikipedia.org/wiki/Physical_Address_Extension>n <https://en.wikipedia.org/wiki/Physical_Address_Extension>
>> [2] https://github.com/tianocore/edk2/blob/master/MdePkg/Library/BaseLib/X86Thunk.c
>> <edk2.png>
>> edk2/Thunk16.nasm at master · tianocore/edk2
>> github.com
>>  <https://github.com/tianocore/edk2/blob/master/MdePkg/Library/BaseLib/X64/Thunk16.nasm>edk2/Thunk16.nasm at master · tianocore/edk2 <https://github.com/tianocore/edk2/blob/master/MdePkg/Library/BaseLib/X64/Thunk16.nasm>
>> github.com <https://github.com/tianocore/edk2/blob/master/MdePkg/Library/BaseLib/X64/Thunk16.nasm>
>> 
>> edk2/Thunk16.nasm at master · tianocore/edk2
>> github.com
>>  <https://github.com/tianocore/edk2/blob/master/MdePkg/Library/BaseLib/Ia32/Thunk16.nasm>edk2/Thunk16.nasm at master · tianocore/edk2 <https://github.com/tianocore/edk2/blob/master/MdePkg/Library/BaseLib/Ia32/Thunk16.nasm>
>> github.com <https://github.com/tianocore/edk2/blob/master/MdePkg/Library/BaseLib/Ia32/Thunk16.nasm>
>> 
>> Thanks,
>> 
>> Andrew Fish
>> 
>> 
>>> Thanks
>>>  
>> 
>> 
>> 
> 
> 
> <edk2.png>


[-- Attachment #2: Type: text/html, Size: 10145 bytes --]

Re: [edk2-devel] Access 64bit address space in 32bit mode

[Brian J. Johnson]

[-- Attachment #1: Type: text/plain, Size: 3462 bytes --]

Slight correction:  PAE paging can access up to 52 physical address 
bits, for 4 PBytes of memory.  Section 4.4 of the Intel® 64 and IA-32 
Architectures Software Developer’s Manual Volume 3 (3A, 3B, 3C & 3D): 
System Programming Guide covers it.

Brian J. Johnson

------------------------------------------------------------------------
*From:* Andrew Fish via groups.io [mailto:afish=apple.com@groups.io]
*Sent:* Wednesday, November 9, 2022 at 11:20 AM
*To:* devel@edk2.groups.io, yoshinoyatoko@163.com
*Subject:* [edk2-devel] Access 64bit address space in 32bit mode

>
>> On Nov 7, 2022, at 7:16 PM, Yoshinoya <yoshinoyatoko@163.com> wrote:
>>
>> Hello
>> Is it possible to access 64bit address space in 32bit mode?
>>
>
> I assume you are talking about x86?
>
>
>> For example, opcode prefix 0x66/67 could let code running in 16bit 
>> mode to access 32bit data/address.
>>
>
> This is more complex than just instruction prefix. You need the CPU to 
> be setup in big real mode via the GDT, so you basically have a 32-bit 
> environment setup via GDT etc. Also the prefix opcodes have different 
> meaning in different modes. I don’t think there is a way to make 
> 32-bit code access 64-bit data via instruction prefix even if a 64-bit 
> GDT was setup with paging enabled.
>
>> Or, establishing page table is a must requirement for accessing 64bit 
>> address space.
>>
>
> For x86 you have to have 64-bit versions of the IDT, GDT, and you need 
> to enable paging to enter 64-bit Long Mode.
>
> In a 32-bit x86 world you can access up to 64 GB of physical memory 
> via using 32-bit page table using PAE [1]. PAE is a 32-bit virtual 
> address space, but with support for a 36-bit physical address. I think 
> in the olden days of 32-bit x86 EFI servers would have custom EFI code 
> that enabled paging in 32-bit and carved out a chunk of the 32-bit 
> memory space that could be mapped to 36-bit physical addresses. I 
> think this was platform specific code and I don’t know of any open 
> source version. The 32-bit Long Mode EFI does not have paging enabled, 
> so adding PAE means enabling paging yourself.
>
> The edk2 has the opposite version of this code so you can call 16-bit 
> really mode (Legacy BIOS) from 32-bit Protected mode, or 64-bit Long 
> Mode. This is the code to Thunk for 32-bit/64-bit mode to 16-bit code 
> [2]/
>
> [1] https://en.wikipedia.org/wiki/Physical_Address_Extensio 
> <https://en.wikipedia.org/wiki/Physical_Address_Extension>n 
> <https://en.wikipedia.org/wiki/Physical_Address_Extension>
> [2] 
> https://github.com/tianocore/edk2/blob/master/MdePkg/Library/BaseLib/X86Thunk.c
> edk2.png
> edk2/Thunk16.nasm at master · tianocore/edk2 
> <https://github.com/tianocore/edk2/blob/master/MdePkg/Library/BaseLib/X64/Thunk16.nasm>
> github.com 
> <https://github.com/tianocore/edk2/blob/master/MdePkg/Library/BaseLib/X64/Thunk16.nasm>
>
> <https://github.com/tianocore/edk2/blob/master/MdePkg/Library/BaseLib/X64/Thunk16.nasm>
> edk2.png
> edk2/Thunk16.nasm at master · tianocore/edk2 
> <https://github.com/tianocore/edk2/blob/master/MdePkg/Library/BaseLib/Ia32/Thunk16.nasm>
> github.com 
> <https://github.com/tianocore/edk2/blob/master/MdePkg/Library/BaseLib/Ia32/Thunk16.nasm>
>
> <https://github.com/tianocore/edk2/blob/master/MdePkg/Library/BaseLib/Ia32/Thunk16.nasm>
>
> Thanks,
>
> Andrew Fish
>
>
>> Thanks
>
> 

[-- Attachment #2.1: Type: text/html, Size: 15577 bytes --]

[-- Attachment #2.2: edk2.png --]
[-- Type: image/png, Size: 70276 bytes --]

[edk2-devel] PciBus driver: support pcie 4.0 bus/device enumeration ?

[Yoshinoya]

[-- Attachment #1: Type: text/plain, Size: 208 bytes --]

Hello:
Currently, more and more platforms support PCIE 4.0 or above bus.
So, udk pci bus driver supports pcie 4.0 new added features?
Such as emergency power reduction, extended msi data field.


best wishes,

[-- Attachment #2: Type: text/html, Size: 885 bytes --]

[edk2-devel] PciBus scan: Does it support scan from EndBusNum to StartBusNum?

[Yoshinoya]

[-- Attachment #1: Type: text/plain, Size: 130 bytes --]

Hi
Usually PciBus scan is from low bus num to high bus num.


So, does it support scan from end bus num to start bus num?


Thanks

[-- Attachment #2: Type: text/html, Size: 509 bytes --]

[edk2-devel] PciBus scan: Does it support scan from EndBusNum to StartBusNum?

[Yoshinoya]

[-- Attachment #1: Type: text/plain, Size: 130 bytes --]

Hi
Usually PciBus scan is from low bus num to high bus num.


So, does it support scan from end bus num to start bus num?


Thanks

[-- Attachment #2: Type: text/html, Size: 718 bytes --]

[edk2-devel] PciBus scan: Does it support scan from EndBusNum to StartBusNum?

[Yoshinoya]

[-- Attachment #1: Type: text/plain, Size: 130 bytes --]

Hi
Usually PciBus scan is from low bus num to high bus num.


So, does it support scan from end bus num to start bus num?


Thanks

[-- Attachment #2: Type: text/html, Size: 718 bytes --]

[edk2-devel] PciBus scan: Does it support scan from EndBusNum to StartBusNum?

[Yoshinoya]

[-- Attachment #1: Type: text/plain, Size: 134 bytes --]

Hi
Usually PciBus scan is from low bus num to high bus num.
     
So, does it support scan from end bus num to start bus num?


Thanks

[-- Attachment #2: Type: text/html, Size: 943 bytes --]

Re: [edk2-devel] PciBus scan: Does it support scan from EndBusNum to StartBusNum?

[Pedro Falcato]

On Tue, Feb 21, 2023 at 3:08 AM Yoshinoya <yoshinoyatoko@163.com> wrote:
>
> Hi
> Usually PciBus scan is from low bus num to high bus num.
>
> So, does it support scan from end bus num to start bus num?
>
> Thanks
Hi,

What do you mean? Why do you want this?

Do you want to number buses in reverse? That is technically doable
(apart from probably the host bridge itself) but absolutely useless.

If you mean "order of enumeration", then no. It is not possible. As
PCI(e) topology works, you always start at the host bridge(s)
(00:00.0) and its bus (bus 0, by definition), and then walk through
whatever bridges you find. You cannot really start at any other place,
unless you do some silly bruteforce method where you probe every bus
for a device; and that method would not allow you to configure the
buses and their BARs, as you need to.

-- 
Pedro

Re: [edk2-devel] PciBus scan: Does it support scan from EndBusNum to StartBusNum?

[Tiger Liu(BJ-RD)]

Hi, Pedro:
Thanks for your reply!

For example:
There are 4 root bridges in a server motherboard, named them as : RP-A / RP-B / RP-C / RP-D
RP-A : PCI Bus num range is 0 ~0x40 (hardware pre-configured)
RP-B : PCI Bus num range is 0x40 ~0x80 (hardware pre-configured)
RP-C : PCI Bus num range is 0x80 ~0xA0 (hardware pre-configured)
RP-D : PCI Bus num range is 0xA0 ~0xE0 (hardware pre-configured)

Usually, we create pci root bridge instances with sequence RP-A / RP-B / RP-C / RP-D.
So, Pci bus driver scan pci root bridge sequence is RP-A/RP-B/RP-C/RP-D.

So, could I rewrite pci root bridge driver with different root bridge report sequence?
For example:
I let pci root bridge driver create RP-D's pci root bridge io protocol first, and then RP-C / RP-B / RP-A, and so pci bus briver could scan RP-D's pci bus range first?


Thanks
-----邮件原件-----
发件人: devel@edk2.groups.io <devel@edk2.groups.io> 代表 Pedro Falcato
发送时间: 2023年2月22日 3:18
收件人: devel@edk2.groups.io; yoshinoyatoko@163.com
主题: Re: [edk2-devel] PciBus scan: Does it support scan from EndBusNum to StartBusNum?

On Tue, Feb 21, 2023 at 3:08 AM Yoshinoya <yoshinoyatoko@163.com> wrote:
>
> Hi
> Usually PciBus scan is from low bus num to high bus num.
>
> So, does it support scan from end bus num to start bus num?
>
> Thanks
Hi,

What do you mean? Why do you want this?

Do you want to number buses in reverse? That is technically doable (apart from probably the host bridge itself) but absolutely useless.

If you mean "order of enumeration", then no. It is not possible. As
PCI(e) topology works, you always start at the host bridge(s)
(00:00.0) and its bus (bus 0, by definition), and then walk through whatever bridges you find. You cannot really start at any other place, unless you do some silly bruteforce method where you probe every bus for a device; and that method would not allow you to configure the buses and their BARs, as you need to.

--
Pedro







保密声明：
本邮件含有保密或专有信息，仅供指定收件人使用。严禁对本邮件或其内容做任何未经授权的查阅、使用、复制或转发。
CONFIDENTIAL NOTE:
This email contains confidential or legally privileged information and is for the sole use of its intended recipient. Any unauthorized review, use, copying or forwarding of this email or the content of this email is strictly prohibited.

Re: [edk2-devel] PciBus scan: Does it support scan from EndBusNum to StartBusNum?

[Pedro Falcato]

On Wed, Feb 22, 2023 at 1:10 AM Tiger Liu(BJ-RD) <TigerLiu@zhaoxin.com> wrote:
>
> Hi, Pedro:
> Thanks for your reply!
>
> For example:
> There are 4 root bridges in a server motherboard, named them as : RP-A / RP-B / RP-C / RP-D
> RP-A : PCI Bus num range is 0 ~0x40 (hardware pre-configured)
> RP-B : PCI Bus num range is 0x40 ~0x80 (hardware pre-configured)
> RP-C : PCI Bus num range is 0x80 ~0xA0 (hardware pre-configured)
> RP-D : PCI Bus num range is 0xA0 ~0xE0 (hardware pre-configured)
>
> Usually, we create pci root bridge instances with sequence RP-A / RP-B / RP-C / RP-D.
> So, Pci bus driver scan pci root bridge sequence is RP-A/RP-B/RP-C/RP-D.
>
> So, could I rewrite pci root bridge driver with different root bridge report sequence?
> For example:
> I let pci root bridge driver create RP-D's pci root bridge io protocol first, and then RP-C / RP-B / RP-A, and so pci bus briver could scan RP-D's pci bus range first?

Hi,

>From a brief look at
MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c, it does seem
that it should enumerate in the order the root bridges were given by
PciHostBridgeGetRootBridges.
I don't know if this reflects your internal code. If so, I would try
just swapping the bridges in your PciHostBridgeGetRootBridges
implementation.

Although since I'm lacking details, I'll assume you really shouldn't
do this? I'm failing to see what purpose this can have, apart from
some other code that needs to be fixed.

-- 
Pedro

[edk2-devel] PciHostBridge: dynamic pcie bus limit assignment

[Yoshinoya]

[-- Attachment #1: Type: text/plain, Size: 346 bytes --]

Hello,
Does PciHostBridge driver support dynamic pcie bus limit assignement?


For example, Xeon chip supports allocating pci bus range dynamically.
So, on a dedicated motherboard, during BIOS Post, user could change every cpu socket's bus limit assignment dynamically ,
and so user could influene some pcie devices enumeration order.


Thanks 


[-- Attachment #2: Type: text/html, Size: 833 bytes --]