Unit Test and sanitizers

Unit Test and sanitizers

[Andrew Fish]

[-- Attachment #1: Type: text/plain, Size: 4768 bytes --]

Just throwing out an idea for the edk2 unit tests. At least for clang you can turn on the sanitizer via a simple command line flag to the compiler. So seems it would make sense to turn on it for unit tests? I’m not sure if the Linux clang, and maybe even some versions of gcc support this too?  Not clear how it works on VC++ or other compilers.

Here is a stupid example from the Xcode clang on macOS of what I’m talking about. In the 1st case the write to NULL crashes the test app with a seg fault. With the sanitizers the buffer overflow and UB  is detected. So the sanitizer gives you better test coverage for free and makes it much easier to root cause the failure.

~/work/Compiler/sanitize>cat t.c                                                          
int
main(int argc, char **argv)
{
  char test[1] = { 0 };
  char *ptr = &test[0];

#ifndef SKIP_OVERFLOW
  ptr += 2;
  *ptr = 1;
#endif
  
  ptr = (char *)0;
  *ptr = 2;
  return 0;
}
~/work/Compiler/sanitize>clang -g t.c && ./a.out                                          
zsh: segmentation fault  ./a.out
~/work/Compiler/sanitize>clang -g -fsanitize=address -fsanitize=undefined  t.c && ./a.out                 
=================================================================
==5302==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7ff7b066f7c2 at pc 0x00010f893db0 bp 0x7ff7b066f790 sp 0x7ff7b066f788
WRITE of size 1 at 0x7ff7b066f7c2 thread T0
    #0 0x10f893daf in main t.c:9
    #1 0x1129754fd in start dyldMain.cpp:879

Address 0x7ff7b066f7c2 is located in stack of thread T0 at offset 34 in frame
    #0 0x10f893baf in main t.c:3

  This frame has 1 object(s):
    [32, 33) 'test' (line 4) <== Memory access at offset 34 overflows this variable
HINT: this may be a false positive if your program uses some custom stack unwind mechanism, swapcontext or vfork
      (longjmp and C++ exceptions *are* supported)
SUMMARY: AddressSanitizer: stack-buffer-overflow t.c:9 in main
Shadow bytes around the buggy address:
  0x1ffef60cdea0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x1ffef60cdeb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x1ffef60cdec0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x1ffef60cded0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x1ffef60cdee0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x1ffef60cdef0: 00 00 00 00 f1 f1 f1 f1[01]f3 f3 f3 00 00 00 00
  0x1ffef60cdf00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x1ffef60cdf10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x1ffef60cdf20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x1ffef60cdf30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x1ffef60cdf40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
  Shadow gap:              cc
==5302==ABORTING
zsh: abort      ./a.out
~/work/Compiler/sanitize>clang -g -fsanitize=address -fsanitize=undefined -DSKIP_OVERFLOW  t.c && ./a.out 
t.c:13:3: runtime error: store to null pointer of type 'char'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior t.c:13:3 in 
AddressSanitizer:DEADLYSIGNAL
=================================================================
==5312==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x0001020cde3f bp 0x7ff7bde358c0 sp 0x7ff7bde357c0 T0)
==5312==The signal is caused by a WRITE memory access.
==5312==Hint: address points to the zero page.
    #0 0x1020cde3f in main t.c:13
    #1 0x108d414fd in start dyldMain.cpp:879

==5312==Register values:
rax = 0x0000000000000000  rbx = 0x00007ff7bde35800  rcx = 0x00007ff7bde357c0  rdx = 0x00001ffef7bc6af8  
rdi = 0x00007ff7bde352f1  rsi = 0x0000000000000000  rbp = 0x00007ff7bde358c0  rsp = 0x00007ff7bde357c0  
 r8 = 0x0000000102580480   r9 = 0x00007ff7bde34a90  r10 = 0x0000000000000000  r11 = 0x0000000000000206  
r12 = 0x0000000108db43a0  r13 = 0x00007ff7bde35978  r14 = 0x00000001020cdc80  r15 = 0x0000000108da8010  
AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV t.c:13 in main
==5312==ABORTING
zsh: abort      ./a.out

Thanks,

Andrew Fish

[-- Attachment #2: Type: text/html, Size: 26194 bytes --]

Re: [edk2-devel] Unit Test and sanitizers

[Marvin Häuser]

Hey Andrew,

+1

Linux Clang sanitizers not only work but actually have more advanced 
techniques here and there over macOS and Windows. GCC definitely has 
some level of support, but if I recall correctly it was not as nice for 
some reason. I think MSVC only has ASan and it's not very nice. I'm not 
sure people want this, but to get the most out of it, I could totally 
see a Docker container running unit tests with Linux Clang and ASan/UBSan.

Best regards,
Marvin

On 24.02.22 00:18, Andrew Fish via groups.io wrote:
> Just throwing out an idea for the edk2 unit tests. At least for clang 
> you can turn on the sanitizer via a simple command line flag to the 
> compiler. So seems it would make sense to turn on it for unit tests? 
> I’m not sure if the Linux clang, and maybe even some versions of gcc 
> support this too?  Not clear how it works on VC++ or other compilers.
>
> Here is a stupid example from the Xcode clang on macOS of what I’m 
> talking about. In the 1st case the write to NULL crashes the test app 
> with a seg fault. With the sanitizers the buffer overflow and UB  is 
> detected. So the sanitizer gives you better test coverage for free and 
> makes it much easier to root cause the failure.
>
> ~/work/Compiler/sanitize*>*cat t.c
> int
> main(int argc, char **argv)
> {
>   char test[1] = { 0 };
>   char *ptr = &test[0];
>
> #ifndef SKIP_OVERFLOW
>   ptr += 2;
>   *ptr = 1;
> #endif
>
>
>   ptr = (char *)0;
>   *ptr = 2;
>   return 0;
> }
> ~/work/Compiler/sanitize*>*clang -g t.c && ./a.out
> zsh: segmentation fault  ./a.out
> ~/work/Compiler/sanitize*>*clang -g -fsanitize=address 
> -fsanitize=undefined t.c && ./a.out
> =================================================================
> *==5302==ERROR: AddressSanitizer: stack-buffer-overflow on address 
> 0x7ff7b066f7c2 at pc 0x00010f893db0 bp 0x7ff7b066f790 sp 0x7ff7b066f788*
> *WRITE of size 1 at 0x7ff7b066f7c2 thread T0*
>     #0 0x10f893daf in main t.c:9
>     #1 0x1129754fd in start dyldMain.cpp:879
>
> *Address 0x7ff7b066f7c2 is located in stack of thread T0 at offset 34 
> in frame*
>     #0 0x10f893baf in main t.c:3
>
>   This frame has 1 object(s):
>     [32, 33) 'test' (line 4)*<== Memory access at offset 34 overflows 
> this variable*
> HINT: this may be a false positive if your program uses some custom 
> stack unwind mechanism, swapcontext or vfork
>       (longjmp and C++ exceptions *are* supported)
> SUMMARY: AddressSanitizer: stack-buffer-overflow t.c:9 in main
> Shadow bytes around the buggy address:
>   0x1ffef60cdea0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>   0x1ffef60cdeb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>   0x1ffef60cdec0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>   0x1ffef60cded0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>   0x1ffef60cdee0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> =>0x1ffef60cdef0: 00 00 00 00 *f1**f1**f1**f1*[01]*f3**f3**f3*00 00 00 00
>   0x1ffef60cdf00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>   0x1ffef60cdf10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>   0x1ffef60cdf20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>   0x1ffef60cdf30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>   0x1ffef60cdf40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> Shadow byte legend (one shadow byte represents 8 application bytes):
>   Addressable:           00
>   Partially addressable: 01 02 03 04 05 06 07
>   Heap left redzone: *fa*
>   Freed heap region: *fd*
>   Stack left redzone: *f1*
>   Stack mid redzone: *f2*
>   Stack right redzone: *f3*
>   Stack after return: *f5*
>   Stack use after scope: *f8*
>   Global redzone: *f9*
>   Global init order: *f6*
>   Poisoned by user: *f7*
>   Container overflow: *fc*
>   Array cookie: *ac*
>   Intra object redzone: *bb*
>   ASan internal: *fe*
>   Left alloca redzone: *ca*
>   Right alloca redzone: *cb*
>   Shadow gap:              cc
> ==5302==ABORTING
> zsh: abort      ./a.out
> ~/work/Compiler/sanitize*>*clang -g -fsanitize=address 
> -fsanitize=undefined -DSKIP_OVERFLOW  t.c && ./a.out
> *t.c:13:3:**runtime error: **store to null pointer of type 'char'*
> SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior t.c:13:3 in
> AddressSanitizer:DEADLYSIGNAL
> =================================================================
> *==5312==ERROR: AddressSanitizer: SEGV on unknown address 
> 0x000000000000 (pc 0x0001020cde3f bp 0x7ff7bde358c0 sp 0x7ff7bde357c0 T0)*
> ==5312==The signal is caused by a WRITE memory access.
> ==5312==Hint: address points to the zero page.
>     #0 0x1020cde3f in main t.c:13
>     #1 0x108d414fd in start dyldMain.cpp:879
>
> ==5312==Register values:
> rax = 0x0000000000000000  rbx = 0x00007ff7bde35800  rcx = 
> 0x00007ff7bde357c0  rdx = 0x00001ffef7bc6af8
> rdi = 0x00007ff7bde352f1  rsi = 0x0000000000000000  rbp = 
> 0x00007ff7bde358c0  rsp = 0x00007ff7bde357c0
>  r8 = 0x0000000102580480   r9 = 0x00007ff7bde34a90  r10 = 
> 0x0000000000000000  r11 = 0x0000000000000206
> r12 = 0x0000000108db43a0  r13 = 0x00007ff7bde35978  r14 = 
> 0x00000001020cdc80  r15 = 0x0000000108da8010
> AddressSanitizer can not provide additional info.
> SUMMARY: AddressSanitizer: SEGV t.c:13 in main
> ==5312==ABORTING
> zsh: abort      ./a.out
>
> Thanks,
>
> Andrew Fish
>