question about uefi shell pipe.

question about uefi shell pipe.

[krishnaLee]

Hi,
I wrote this shell application(smalltest.efi),did some pipe function test,find some strange result.
boot into shell:


# test-1,the follow two command should has the same output ,but infact not the same in QEMU,and the second command failed WriteFile in real machine(AMI bios uefi 2.6):
ls | smalltest.efi
ls | smalltest.efi | smalltest.efi             # the ls command directory only has one file(this tool),so the tool's buffer 80x25*4 won't overflow.


#test-2
run smalltest.efi,
just key in  some chars and Enter,nothing output,why?


my test environment:
UDK2018 + vs2015
QEMU emulator version 2.10.95
OVMF_X64.fd( x64,release build)


the tool's build command:
D:\edk2-vUDK2018>build -p ShellPkg\ShellPkg.dsc -m ShellPkg\Application2\smalltest\smalltest.inf -a X64 -b RELEASE


//--------code---smalltest.c-------------


#include <Uefi.h>
#include <Library/UefiApplicationEntryPoint.h>
#include <Library/UefiLib.h>
#include <Library/UefiBootServicesTableLib.h>//global gST gBS gImageHandle
#include <Library/BaseMemoryLib.h>
#include <Library/MemoryAllocationLib.h>


#include <Protocol/Shell.h>
#include <Protocol/ShellParameters.h>



EFI_STATUS
EFIAPI
UefiMain (
IN EFI_HANDLE ImageHandle,
IN EFI_SYSTEM_TABLE *SystemTable
)
{
EFI_STATUS status;
EFI_SHELL_PROTOCOL* gShell;
EFI_SHELL_PARAMETERS_PROTOCOL*gParameters;
UINT16 Buffer[80*25*4]; //tool's buffer;
UINTN BufferSize=sizeof(Buffer);
UINTN BufferByteSize=BufferSize*sizeof(UINT16);


status=gBS->HandleProtocol(gImageHandle,&gEfiShellParametersProtocolGuid,&gParameters);
if(status!=EFI_SUCCESS)
{
Print(L"locate gEfiShellParametersProtocolGuid failed.\n");
return status;
}


status=gBS->LocateProtocol(&gEfiShellProtocolGuid,NULL,&gShell);
if(status!=EFI_SUCCESS)
{
Print(L"locate gEfiShellProtocolGuid failed.\n");
return status;
}


status=gShell->ReadFile(gParameters->StdIn,&BufferByteSize,(VOID*)Buffer);
if(status!=EFI_SUCCESS)
{
Print(L"read from gParameters->StdIn failed.\n");
return status;
}


status=gShell->WriteFile(gParameters->StdOut,&BufferByteSize,(VOID*)Buffer);
if(status!=EFI_SUCCESS)
{
Print(L"wirte gParameters->StdOut failed\n");
return status;
}


gShell->FlushFile(gParameters->StdOut);


return EFI_SUCCESS;
}


//--------code---smalltest.inf-------------


[Defines]
  INF_VERSION                    = 0x00010005
  BASE_NAME                      = smalltest
  FILE_GUID                      = 8F7D7B1D-0E1C-4c98-B12E-4EC99C400704
  MODULE_TYPE                    = UEFI_APPLICATION
  VERSION_STRING                 = 1.0
  ENTRY_POINT                    = UefiMain


[Sources]
  smalltest.c


[Packages]
  MdePkg/MdePkg.dec


[Protocols]
gEfiShellProtocolGuid
gEfiShellParametersProtocolGuid


[LibraryClasses]
  UefiApplicationEntryPoint
  UefiLib
//--------code---end----------------------------------------------------------






thank you, 
by krishna.

Re: question about uefi shell pipe.

[Andrew Fish]

Krishnal,

Thanks for including the code. You are overflowing the buffer. In C sizeof(Buffer) will be 80*25*4*2 since sizeof(UINT16) is 2. Your code is passing in double the size of the buffer. 

UINT16 Buffer[80*25*4]; //tool's buffer;
UINTN BufferSize=sizeof(Buffer);
UINTN BufferByteSize=BufferSize*sizeof(UINT16);

This should work:

UINT16 Buffer[80*25*4]; //tool's buffer;
UINTN BufferByteSize = sizeof(Buffer);

Thanks,

Andrew Fish

> On Jul 4, 2018, at 2:15 AM, krishnaLee <sssky307@163.com> wrote:
> 
> Hi,
> I wrote this shell application(smalltest.efi),did some pipe function test,find some strange result.
> boot into shell:
> 
> 
> # test-1,the follow two command should has the same output ,but infact not the same in QEMU,and the second command failed WriteFile in real machine(AMI bios uefi 2.6):
> ls | smalltest.efi
> ls | smalltest.efi | smalltest.efi             # the ls command directory only has one file(this tool),so the tool's buffer 80x25*4 won't overflow.
> 
> 
> #test-2
> run smalltest.efi,
> just key in  some chars and Enter,nothing output,why?
> 
> 
> my test environment:
> UDK2018 + vs2015
> QEMU emulator version 2.10.95
> OVMF_X64.fd( x64,release build)
> 
> 
> the tool's build command:
> D:\edk2-vUDK2018>build -p ShellPkg\ShellPkg.dsc -m ShellPkg\Application2\smalltest\smalltest.inf -a X64 -b RELEASE
> 
> 
> //--------code---smalltest.c-------------
> 
> 
> #include <Uefi.h>
> #include <Library/UefiApplicationEntryPoint.h>
> #include <Library/UefiLib.h>
> #include <Library/UefiBootServicesTableLib.h>//global gST gBS gImageHandle
> #include <Library/BaseMemoryLib.h>
> #include <Library/MemoryAllocationLib.h>
> 
> 
> #include <Protocol/Shell.h>
> #include <Protocol/ShellParameters.h>
> 
> 
> 
> EFI_STATUS
> EFIAPI
> UefiMain (
> IN EFI_HANDLE ImageHandle,
> IN EFI_SYSTEM_TABLE *SystemTable
> )
> {
> EFI_STATUS status;
> EFI_SHELL_PROTOCOL* gShell;
> EFI_SHELL_PARAMETERS_PROTOCOL*gParameters;
> UINT16 Buffer[80*25*4]; //tool's buffer;
> UINTN BufferSize=sizeof(Buffer);
> UINTN BufferByteSize=BufferSize*sizeof(UINT16);
> 
> 
> status=gBS->HandleProtocol(gImageHandle,&gEfiShellParametersProtocolGuid,&gParameters);
> if(status!=EFI_SUCCESS)
> {
> Print(L"locate gEfiShellParametersProtocolGuid failed.\n");
> return status;
> }
> 
> 
> status=gBS->LocateProtocol(&gEfiShellProtocolGuid,NULL,&gShell);
> if(status!=EFI_SUCCESS)
> {
> Print(L"locate gEfiShellProtocolGuid failed.\n");
> return status;
> }
> 
> 
> status=gShell->ReadFile(gParameters->StdIn,&BufferByteSize,(VOID*)Buffer);
> if(status!=EFI_SUCCESS)
> {
> Print(L"read from gParameters->StdIn failed.\n");
> return status;
> }
> 
> 
> status=gShell->WriteFile(gParameters->StdOut,&BufferByteSize,(VOID*)Buffer);
> if(status!=EFI_SUCCESS)
> {
> Print(L"wirte gParameters->StdOut failed\n");
> return status;
> }
> 
> 
> gShell->FlushFile(gParameters->StdOut);
> 
> 
> return EFI_SUCCESS;
> }
> 
> 
> //--------code---smalltest.inf-------------
> 
> 
> [Defines]
>  INF_VERSION                    = 0x00010005
>  BASE_NAME                      = smalltest
>  FILE_GUID                      = 8F7D7B1D-0E1C-4c98-B12E-4EC99C400704
>  MODULE_TYPE                    = UEFI_APPLICATION
>  VERSION_STRING                 = 1.0
>  ENTRY_POINT                    = UefiMain
> 
> 
> [Sources]
>  smalltest.c
> 
> 
> [Packages]
>  MdePkg/MdePkg.dec
> 
> 
> [Protocols]
> gEfiShellProtocolGuid
> gEfiShellParametersProtocolGuid
> 
> 
> [LibraryClasses]
>  UefiApplicationEntryPoint
>  UefiLib
> //--------code---end----------------------------------------------------------
> 
> 
> 
> 
> 
> 
> thank you, 
> by krishna.
> _______________________________________________
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel

Re: question about uefi shell pipe.

[krishnaLee]

Andrew Fish,
Yes,some times I am work in C++,thanks for correct this,but if I use this:
UINT16 Buffer[80*25*4]; //tool's buffer;
UINTN BufferByteSize = sizeof(Buffer);
it still cannot work.
I had checked the read size( gShell->ReadFile(gParameters->StdIn,...)) before I post my question,it just read  some hundred of bytes,It seems the question is not about overflow...
so I want to know if I'm wrong or it is a bug?


thank you,
by krishna.







At 2018-07-04 22:55:27, "Andrew Fish" <afish@apple.com> wrote:
>Krishnal,
>
>Thanks for including the code. You are overflowing the buffer. In C sizeof(Buffer) will be 80*25*4*2 since sizeof(UINT16) is 2. Your code is passing in double the size of the buffer. 
>
>UINT16 Buffer[80*25*4]; //tool's buffer;
>UINTN BufferSize=sizeof(Buffer);
>UINTN BufferByteSize=BufferSize*sizeof(UINT16);
>
>This should work:
>
>UINT16 Buffer[80*25*4]; //tool's buffer;
>UINTN BufferByteSize = sizeof(Buffer);
>
>Thanks,
>
>Andrew Fish
>
>> On Jul 4, 2018, at 2:15 AM, krishnaLee <sssky307@163.com> wrote:
>> 
>> Hi,
>> I wrote this shell application(smalltest.efi),did some pipe function test,find some strange result.
>> boot into shell:
>> 
>> 
>> # test-1,the follow two command should has the same output ,but infact not the same in QEMU,and the second command failed WriteFile in real machine(AMI bios uefi 2.6):
>> ls | smalltest.efi
>> ls | smalltest.efi | smalltest.efi             # the ls command directory only has one file(this tool),so the tool's buffer 80x25*4 won't overflow.
>> 
>> 
>> #test-2
>> run smalltest.efi,
>> just key in  some chars and Enter,nothing output,why?
>> 
>> 
>> my test environment:
>> UDK2018 + vs2015
>> QEMU emulator version 2.10.95
>> OVMF_X64.fd( x64,release build)
>> 
>> 
>> the tool's build command:
>> D:\edk2-vUDK2018>build -p ShellPkg\ShellPkg.dsc -m ShellPkg\Application2\smalltest\smalltest.inf -a X64 -b RELEASE
>> 
>> 
>> //--------code---smalltest.c-------------
>> 
>> 
>> #include <Uefi.h>
>> #include <Library/UefiApplicationEntryPoint.h>
>> #include <Library/UefiLib.h>
>> #include <Library/UefiBootServicesTableLib.h>//global gST gBS gImageHandle
>> #include <Library/BaseMemoryLib.h>
>> #include <Library/MemoryAllocationLib.h>
>> 
>> 
>> #include <Protocol/Shell.h>
>> #include <Protocol/ShellParameters.h>
>> 
>> 
>> 
>> EFI_STATUS
>> EFIAPI
>> UefiMain (
>> IN EFI_HANDLE ImageHandle,
>> IN EFI_SYSTEM_TABLE *SystemTable
>> )
>> {
>> EFI_STATUS status;
>> EFI_SHELL_PROTOCOL* gShell;
>> EFI_SHELL_PARAMETERS_PROTOCOL*gParameters;
>> UINT16 Buffer[80*25*4]; //tool's buffer;
>> UINTN BufferSize=sizeof(Buffer);
>> UINTN BufferByteSize=BufferSize*sizeof(UINT16);
>> 
>> 
>> status=gBS->HandleProtocol(gImageHandle,&gEfiShellParametersProtocolGuid,&gParameters);
>> if(status!=EFI_SUCCESS)
>> {
>> Print(L"locate gEfiShellParametersProtocolGuid failed.\n");
>> return status;
>> }
>> 
>> 
>> status=gBS->LocateProtocol(&gEfiShellProtocolGuid,NULL,&gShell);
>> if(status!=EFI_SUCCESS)
>> {
>> Print(L"locate gEfiShellProtocolGuid failed.\n");
>> return status;
>> }
>> 
>> 
>> status=gShell->ReadFile(gParameters->StdIn,&BufferByteSize,(VOID*)Buffer);
>> if(status!=EFI_SUCCESS)
>> {
>> Print(L"read from gParameters->StdIn failed.\n");
>> return status;
>> }
>> 
>> 
>> status=gShell->WriteFile(gParameters->StdOut,&BufferByteSize,(VOID*)Buffer);
>> if(status!=EFI_SUCCESS)
>> {
>> Print(L"wirte gParameters->StdOut failed\n");
>> return status;
>> }
>> 
>> 
>> gShell->FlushFile(gParameters->StdOut);
>> 
>> 
>> return EFI_SUCCESS;
>> }
>> 
>> 
>> //--------code---smalltest.inf-------------
>> 
>> 
>> [Defines]
>>  INF_VERSION                    = 0x00010005
>>  BASE_NAME                      = smalltest
>>  FILE_GUID                      = 8F7D7B1D-0E1C-4c98-B12E-4EC99C400704
>>  MODULE_TYPE                    = UEFI_APPLICATION
>>  VERSION_STRING                 = 1.0
>>  ENTRY_POINT                    = UefiMain
>> 
>> 
>> [Sources]
>>  smalltest.c
>> 
>> 
>> [Packages]
>>  MdePkg/MdePkg.dec
>> 
>> 
>> [Protocols]
>> gEfiShellProtocolGuid
>> gEfiShellParametersProtocolGuid
>> 
>> 
>> [LibraryClasses]
>>  UefiApplicationEntryPoint
>>  UefiLib
>> //--------code---end----------------------------------------------------------
>> 
>> 
>> 
>> 
>> 
>> 
>> thank you, 
>> by krishna.
>> _______________________________________________
>> edk2-devel mailing list
>> edk2-devel@lists.01.org
>> https://lists.01.org/mailman/listinfo/edk2-devel

Re: question about uefi shell pipe.

[krishnaLee]

Hi,All:
may be I should reply this to EDK2 group,I had attached my file again.


After many test,my right key of "ls | grep -trim-last 0 | grep -trim-last 0" is:
  1,before pipe out the buffer,trim off all in-visible wchars at end of this buffer, (the trimed of wchar  is "like ASCII's range of" 0x0000~0x0020. )
  2,always make sure outputBuffer[lastTwoWchar] = { 0x000A,0x000D },it will also bypass many other strange result.


thank you,
by krishna.





At 2018-07-06 11:07:05, "krishnaLee" <sssky307@163.com> wrote:

Jim,
I attached the smalltest.efi if you have not compiled the test code.


I am in developing of  this grep.efi tool,I can now bypass the test-1 question by follow command:
ls | grep -trim-last 0
ls | grep -trim-last 0 | grep -trim-last 0


I also attached this grep.efi  tool, it almost ok :)


//draft usage guide-------------------------------------------------


//if find ok,returncode 1,Print(string);else returncode 0;
grep.efi -find string


//get first word,
//the front and the end in-visible chars of this word will be trimed off;
grep.efi -first


//get back part after the string.

grep.efi -begin-at string


//get front part before the string
grep.efi -end-at string


//trim off first n chars at front
//before start -trim-first,the front in-visible chars will be trimed off;
grep.efi -trim-first n


//trim off back n chars at end,
//before start -trim-last,the end in-visible chars will be trimed off;
grep.efi -trim-last n


//trim off whitespace at front and end,
//trim off in-visible chars infact;
grep.efi -trim-space




thank you,
by krishna.

Re: question about uefi shell pipe.

[Gao, Liming]

Krishna:
 Sorry, I am not clear what's your question here? Would you like to share this grep.efi tool? Or you meet with the problem in this grep.efi tool?

Thanks
Liming
>-----Original Message-----
>From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of
>krishnaLee
>Sent: Friday, July 06, 2018 11:35 AM
>To: krishnaLee <sssky307@163.com>
>Cc: edk2-devel@lists.01.org
>Subject: Re: [edk2] question about uefi shell pipe.
>
>
>
>Hi,All:
>may be I should reply this to EDK2 group,I had attached my file again.
>
>
>After many test,my right key of "ls | grep -trim-last 0 | grep -trim-last 0" is:
>  1,before pipe out the buffer,trim off all in-visible wchars at end of this buffer,
>(the trimed of wchar  is "like ASCII's range of" 0x0000~0x0020. )
>  2,always make sure outputBuffer[lastTwoWchar] = { 0x000A,0x000D },it will
>also bypass many other strange result.
>
>
>thank you,
>by krishna.
>
>
>
>
>
>At 2018-07-06 11:07:05, "krishnaLee" <sssky307@163.com> wrote:
>
>Jim,
>I attached the smalltest.efi if you have not compiled the test code.
>
>
>I am in developing of  this grep.efi tool,I can now bypass the test-1 question
>by follow command:
>ls | grep -trim-last 0
>ls | grep -trim-last 0 | grep -trim-last 0
>
>
>I also attached this grep.efi  tool, it almost ok :)
>
>
>//draft usage guide-------------------------------------------------
>
>
>//if find ok,returncode 1,Print(string);else returncode 0;
>grep.efi -find string
>
>
>//get first word,
>//the front and the end in-visible chars of this word will be trimed off;
>grep.efi -first
>
>
>//get back part after the string.
>
>grep.efi -begin-at string
>
>
>//get front part before the string
>grep.efi -end-at string
>
>
>//trim off first n chars at front
>//before start -trim-first,the front in-visible chars will be trimed off;
>grep.efi -trim-first n
>
>
>//trim off back n chars at end,
>//before start -trim-last,the end in-visible chars will be trimed off;
>grep.efi -trim-last n
>
>
>//trim off whitespace at front and end,
>//trim off in-visible chars infact;
>grep.efi -trim-space
>
>
>
>
>thank you,
>by krishna.
>_______________________________________________
>edk2-devel mailing list
>edk2-devel@lists.01.org
>https://lists.01.org/mailman/listinfo/edk2-devel

Re: question about uefi shell pipe.

[krishnaLee]

Hi,liming:
 the files attached to this mail may be missed,so I had made a copy here:
https://github.com/testcheng307/UEFI-TEST


the question is in first mail,I repeat it here:


# test-1,the follow two command should has the same output ,but infact not the same in QEMU,and the second command failed WriteFile in real machine(AMI bios uefi 2.6):
# the smalltest.efi 's fullsource code is in my first mail;
ls | smalltest.efi
ls | smalltest.efi | smalltest.efi       


#test-2
run smalltest.efi,
just key in  some chars, and I can not end input by press Enter  or[Ctrl+C] or [Ctrl+Z],  so nothing output,why?


#test-3,the follow two command will be the same.
ls | grep -trim-last 0
ls | grep -trim-last 0 | grep -trim-last 0
# my grep.efi tool will process input buffer like this:
1,before pipe out the buffer,trim off all in-visible wchars at end of this buffer,
(the trimed of wchar  is "like ASCII's range of" 0x0000~0x0020. )
 2,always make sure outputBuffer[lastTwoWchar] = { 0x000A,0x000D },it will
also bypass many other strange result.




thank you,
by krishna.









At 2018-07-09 15:17:56, "Gao, Liming" <liming.gao@intel.com> wrote:
>Krishna:
> Sorry, I am not clear what's your question here? Would you like to share this grep.efi tool? Or you meet with the problem in this grep.efi tool?
>
>Thanks
>Liming
>>-----Original Message-----
>>From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of
>>krishnaLee
>>Sent: Friday, July 06, 2018 11:35 AM
>>To: krishnaLee <sssky307@163.com>
>>Cc: edk2-devel@lists.01.org
>>Subject: Re: [edk2] question about uefi shell pipe.
>>
>>
>>
>>Hi,All:
>>may be I should reply this to EDK2 group,I had attached my file again.
>>
>>
>>After many test,my right key of "ls | grep -trim-last 0 | grep -trim-last 0" is:
>>  1,before pipe out the buffer,trim off all in-visible wchars at end of this buffer,
>>(the trimed of wchar  is "like ASCII's range of" 0x0000~0x0020. )
>>  2,always make sure outputBuffer[lastTwoWchar] = { 0x000A,0x000D },it will
>>also bypass many other strange result.
>>
>>
>>thank you,
>>by krishna.
>>
>>
>>
>>
>>
>>At 2018-07-06 11:07:05, "krishnaLee" <sssky307@163.com> wrote:
>>
>>Jim,
>>I attached the smalltest.efi if you have not compiled the test code.
>>
>>
>>I am in developing of  this grep.efi tool,I can now bypass the test-1 question
>>by follow command:
>>ls | grep -trim-last 0
>>ls | grep -trim-last 0 | grep -trim-last 0
>>
>>
>>I also attached this grep.efi  tool, it almost ok :)
>>
>>
>>//draft usage guide-------------------------------------------------
>>
>>
>>//if find ok,returncode 1,Print(string);else returncode 0;
>>grep.efi -find string
>>
>>
>>//get first word,
>>//the front and the end in-visible chars of this word will be trimed off;
>>grep.efi -first
>>
>>
>>//get back part after the string.
>>
>>grep.efi -begin-at string
>>
>>
>>//get front part before the string
>>grep.efi -end-at string
>>
>>
>>//trim off first n chars at front
>>//before start -trim-first,the front in-visible chars will be trimed off;
>>grep.efi -trim-first n
>>
>>
>>//trim off back n chars at end,
>>//before start -trim-last,the end in-visible chars will be trimed off;
>>grep.efi -trim-last n
>>
>>
>>//trim off whitespace at front and end,
>>//trim off in-visible chars infact;
>>grep.efi -trim-space
>>
>>
>>
>>
>>thank you,
>>by krishna.
>>_______________________________________________
>>edk2-devel mailing list
>>edk2-devel@lists.01.org
>>https://lists.01.org/mailman/listinfo/edk2-devel
\x16&#65533;&

Re: question about uefi shell pipe.

[krishnaLee]

Hi,All:
may be I should reply this to EDK2 group,sorry for this mail-size>600K,I had trim the picture and attached my file again,and send it again.


After many test,my right key of "ls | grep -trim-last 0 | grep -trim-last 0" is:
  1,before pipe out the buffer,trim off all in-visible wchars at end of this buffer, (the trimed of wchar  is "like ASCII's range of" 0x0000~0x0020. )
  2,always make sure outputBuffer[lastTwoWchar] = { 0x000A,0x000D },it will also bypass many other strange result.


thank you,
by krishna.





At 2018-07-06 11:07:05, "krishnaLee" <sssky307@163.com> wrote:

Jim,
I attached the smalltest.efi if you have not compiled the test code.


I am in developing of  this grep.efi tool,I can now bypass the test-1 question by follow command:
ls | grep -trim-last 0
ls | grep -trim-last 0 | grep -trim-last 0


I also attached this grep.efi  tool, it almost ok :)


//draft usage guide-------------------------------------------------


//if find ok,returncode 1,Print(string);else returncode 0;
grep.efi -find string


//get first word,
//the front and the end in-visible chars of this word will be trimed off;
grep.efi -first


//get back part after the string.

grep.efi -begin-at string


//get front part before the string
grep.efi -end-at string


//trim off first n chars at front
//before start -trim-first,the front in-visible chars will be trimed off;
grep.efi -trim-first n


//trim off back n chars at end,
//before start -trim-last,the end in-visible chars will be trimed off;
grep.efi -trim-last n


//trim off whitespace at front and end,
//trim off in-visible chars infact;
grep.efi -trim-space




thank you,
by krishna.