public inbox for devel@edk2.groups.io
 help / color / mirror / Atom feed
From: Pete Batard <pete@akeo.ie>
To: edk2-devel@lists.01.org
Subject: [PATCH 5/5] BaseTools: add scripts to generate EBC call signatures
Date: Tue, 24 Jan 2017 12:30:26 +0000	[thread overview]
Message-ID: <20170124123026.5204-5-pete@akeo.ie> (raw)
In-Reply-To: <20170124123026.5204-1-pete@akeo.ie>

* EBC binaries meant to be compatible with the ARM EBC VM
  require the insertion of call signatures at the locations
  where the BREAK 5 offsets are stored.
* This patch adds 2 new Python applications, as well as the
  required modifications for build_rule and tools_def templates:
  - GenEbcSignature is a script that parses a preprocessed C
    source, along with a binary object file, as generated by the
    intel EBC compiler, to create the signature data, which it
    stores into a .sig file.
  - PatchEbcSignature is a script that processes the .sig files
    as well as the .map data and the .efi binary, to insert the
    signatures at the required locations.
* Note that the insertion of signatures into EBC binaries has
  absolutely no impact for EBC execution on non ARM platforms,
  as it applies to data parts that are ignored on these VMs.
* Also note the GenEbcSignature is designed to be very basic
  with regards to the detection of call signatures that have
  straight 64-bit parameters, as it is meant to be used as a
  stopgap solution until the intel EBC compiler (which does
  has easy access to the data required to do so, and is best
  placed for such processing) is updated to generate .sig files
  on its own.

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Pete Batard <pete@akeo.ie>
---
 .../BinWrappers/WindowsLike/GenEbcSignature.bat    |   3 +
 .../BinWrappers/WindowsLike/PatchEbcSignature.bat  |   3 +
 BaseTools/Conf/build_rule.template                 |  72 +++--
 BaseTools/Conf/tools_def.template                  |  12 +
 .../Python/GenEbcSignature/GenEbcSignature.py      | 306 +++++++++++++++++++++
 .../Source/Python/GenEbcSignature/__init__.py      |  15 +
 .../Python/PatchEbcSignature/PatchEbcSignature.py  | 226 +++++++++++++++
 .../Source/Python/PatchEbcSignature/__init__.py    |  15 +
 8 files changed, 633 insertions(+), 19 deletions(-)
 create mode 100644 BaseTools/BinWrappers/WindowsLike/GenEbcSignature.bat
 create mode 100644 BaseTools/BinWrappers/WindowsLike/PatchEbcSignature.bat
 create mode 100644 BaseTools/Source/Python/GenEbcSignature/GenEbcSignature.py
 create mode 100644 BaseTools/Source/Python/GenEbcSignature/__init__.py
 create mode 100644 BaseTools/Source/Python/PatchEbcSignature/PatchEbcSignature.py
 create mode 100644 BaseTools/Source/Python/PatchEbcSignature/__init__.py

diff --git a/BaseTools/BinWrappers/WindowsLike/GenEbcSignature.bat b/BaseTools/BinWrappers/WindowsLike/GenEbcSignature.bat
new file mode 100644
index 000000000000..9fbb704a6eb0
--- /dev/null
+++ b/BaseTools/BinWrappers/WindowsLike/GenEbcSignature.bat
@@ -0,0 +1,3 @@
+@setlocal
+@set ToolName=%~n0%
+@%PYTHON_HOME%\python.exe %BASE_TOOLS_PATH%\Source\Python\%ToolName%\%ToolName%.py %*
diff --git a/BaseTools/BinWrappers/WindowsLike/PatchEbcSignature.bat b/BaseTools/BinWrappers/WindowsLike/PatchEbcSignature.bat
new file mode 100644
index 000000000000..9fbb704a6eb0
--- /dev/null
+++ b/BaseTools/BinWrappers/WindowsLike/PatchEbcSignature.bat
@@ -0,0 +1,3 @@
+@setlocal
+@set ToolName=%~n0%
+@%PYTHON_HOME%\python.exe %BASE_TOOLS_PATH%\Source\Python\%ToolName%\%ToolName%.py %*
diff --git a/BaseTools/Conf/build_rule.template b/BaseTools/Conf/build_rule.template
index 1db94b696f89..3408f507e78b 100755
--- a/BaseTools/Conf/build_rule.template
+++ b/BaseTools/Conf/build_rule.template
@@ -161,6 +161,27 @@
         "$(CC)" $(CC_FLAGS) -c -o ${dst} $(INC) ${src}
         "$(SYMRENAME)" $(SYMRENAME_FLAGS) ${dst}
 
+[C-Code-File.COMMON.EBC]
+    <InputFile>
+        ?.c
+        ?.C
+        ?.cc
+        ?.CC
+        ?.cpp
+        ?.Cpp
+        ?.CPP
+
+    <ExtraDependency>
+        $(MAKE_FILE)
+
+    <OutputFile>
+        $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.obj
+
+    <Command.MSFT, Command.INTEL>
+        "$(CC)" /Fo${dst} $(CC_FLAGS) $(INC) ${src}
+        # Parse preprocessed sources to generate the EBC call signatures
+        "$(CC)" /E /EP $(CC_FLAGS) $(INC) ${src} | "$(GENEBC)" -v -o ${dst} -s $(OUTPUT_DIR)(+)$(MODULE_NAME).sig
+
 [C-Code-File.BASE.AARCH64,C-Code-File.SEC.AARCH64,C-Code-File.PEI_CORE.AARCH64,C-Code-File.PEIM.AARCH64,C-Code-File.BASE.ARM,C-Code-File.SEC.ARM,C-Code-File.PEI_CORE.ARM,C-Code-File.PEIM.ARM]
     <InputFile>
         ?.c
@@ -267,10 +288,10 @@
 
     <Command.GCC, Command.GCCLD>
         "$(SLINK)" cr ${dst} $(SLINK_FLAGS) @$(OBJECT_FILES_LIST)
-    
+
     <Command.RVCT>
         "$(SLINK)" $(SLINK_FLAGS) ${dst} --via $(OBJECT_FILES_LIST)
-    
+
     <Command.RVCTCYGWIN>
         # $(OBJECT_FILES_LIST) has wrong paths for cygwin
         "$(SLINK)" $(SLINK_FLAGS) ${dst} $(OBJECT_FILES)
@@ -308,8 +329,8 @@
 
     <Command.XCODE>
         "$(DLINK)" $(DLINK_FLAGS) -o ${dst} $(DLINK_SPATH) -filelist $(STATIC_LIBRARY_FILES_LIST)  $(DLINK2_FLAGS)
-    
-    
+
+
 [Static-Library-File.SEC.AARCH64, Static-Library-File.PEI_CORE.AARCH64, Static-Library-File.PEIM.AARCH64,Static-Library-File.SEC.ARM, Static-Library-File.PEI_CORE.ARM, Static-Library-File.PEIM.ARM]
     <InputFile>
         *.lib
@@ -353,8 +374,8 @@
 
     <Command.XCODE>
         "$(DLINK)" -o ${dst} $(DLINK_FLAGS)  $(DLINK_SPATH) -filelist $(STATIC_LIBRARY_FILES_LIST)  $(DLINK2_FLAGS)
-      
-      
+
+
 [Dynamic-Library-File]
     <InputFile>
         ?.dll
@@ -367,7 +388,7 @@
         $(CP) ${dst} $(OUTPUT_DIR)
         $(CP) ${dst} $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi
         -$(CP) $(DEBUG_DIR)(+)*.map $(OUTPUT_DIR)
-        -$(CP) $(DEBUG_DIR)(+)*.pdb $(OUTPUT_DIR) 
+        -$(CP) $(DEBUG_DIR)(+)*.pdb $(OUTPUT_DIR)
     <Command.GCC, Command.GCCLD>
         $(CP) ${src} $(DEBUG_DIR)(+)$(MODULE_NAME).debug
         $(OBJCOPY) --strip-unneeded -R .eh_frame ${src}
@@ -382,7 +403,7 @@
         $(CP) ${dst} $(OUTPUT_DIR)
         $(CP) ${dst} $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi
         -$(CP) $(DEBUG_DIR)(+)*.map $(OUTPUT_DIR)
-        
+
     <Command.XCODE>
         # tool to convert Mach-O to PE/COFF
         "$(MTOC)" -subsystem $(MODULE_TYPE)  $(MTOC_FLAGS)  ${src}  $(DEBUG_DIR)(+)$(MODULE_NAME).pecoff
@@ -393,6 +414,21 @@
         $(CP) ${dst} $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi
         -$(CP) $(DEBUG_DIR)(+)*.map $(OUTPUT_DIR)
 
+[Dynamic-Library-File.COMMON.EBC]
+    <InputFile>
+        ?.dll
+
+    <OutputFile>
+        $(DEBUG_DIR)(+)$(MODULE_NAME).efi
+
+    <Command.MSFT, Command.INTEL, Command.RVCT>
+        "$(GENFW)" -e $(MODULE_TYPE) -o ${dst} ${src} $(GENFW_FLAGS)
+        "$(PATCHEBC)" -v -m $(DEBUG_DIR)(+)$(MODULE_NAME).map -e ${dst} $(OUTPUT_DIR)(+)$(MODULE_NAME).sig $(STATIC_LIBRARY_FILES)
+        $(CP) ${dst} $(OUTPUT_DIR)
+        $(CP) ${dst} $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi
+        -$(CP) $(DEBUG_DIR)(+)*.map $(OUTPUT_DIR)
+        -$(CP) $(DEBUG_DIR)(+)*.pdb $(OUTPUT_DIR)
+
 [Dependency-Expression-File]
     <InputFile>
         ?.dxs, ?.Dxs, ?.DXS
@@ -421,13 +457,13 @@
     <Command.MSFT, Command.INTEL>
         Trim --asl-file -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.i -i $(INC_LIST) ${src}
         "$(ASLPP)" $(ASLPP_FLAGS) $(INC) /I${s_path} $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.i > $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.iii
-        Trim --source-code -l -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.iiii $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.iii 
+        Trim --source-code -l -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.iiii $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.iii
         "$(ASL)" $(ASL_FLAGS) $(ASL_OUTFLAGS)${dst} $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.iiii
 
     <Command.GCC, Command.GCCLD>
         Trim --asl-file -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.i -i $(INC_LIST) ${src}
         "$(ASLPP)" $(ASLPP_FLAGS) $(INC) -I${s_path} $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.i > $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.iii
-        Trim --source-code -l -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.iiii $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.iii 
+        Trim --source-code -l -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.iiii $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.iii
         "$(ASL)" $(ASL_FLAGS) $(ASL_OUTFLAGS)${dst} $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.iiii
 
 [C-Code-File.AcpiTable]
@@ -469,14 +505,14 @@
         "$(ASLCC)" -c -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.obj $(CC_FLAGS) $(ASLCC_FLAGS) $(INC) ${src}
         "$(ASLDLINK)" -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.dll $(ASLDLINK_FLAGS) $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.obj
         "$(GENFW)" -o ${dst} -c $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.dll $(GENFW_FLAGS)
-        
-    <Command.XCODE>        
+
+    <Command.XCODE>
         "$(ASLCC)" -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.obj  $(ASLCC_FLAGS) $(INC) ${src}
         "$(ASLDLINK)" -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.dll $(ASLDLINK_FLAGS) $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.obj
         "$(MTOC)" -subsystem $(MODULE_TYPE)  $(MTOC_FLAGS) $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.dll $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.efi
         "$(GENFW)" -o ${dst} -c $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.efi $(GENFW_FLAGS)
-      
-      
+
+
 [Masm16-Code-File]
     <InputFile>
         ?.asm16, ?.Asm16, ?.ASM16, ?.s16, ?.S16
@@ -499,14 +535,14 @@
       Trim --source-code -o ${d_path}(+)${s_base}.iii ${d_path}(+)${s_base}.i
       "$(ASM)" -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.obj $(ASM_FLAGS) $(INC) ${d_path}(+)${s_base}.iii
       "$(DLINK)" -o ${dst} $(DLINK_FLAGS) --start-group $(DLINK_SPATH) $(LIBS) $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.obj --end-group
-     
+
     <Command.XCODE>
       "$(PP)" $(PP_FLAGS) $(INC) ${src} > ${d_path}(+)${s_base}.i
       Trim --source-code -o ${d_path}(+)${s_base}.iii ${d_path}(+)${s_base}.i
       "$(ASM)" -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.obj $(ASM_FLAGS) $(INC) ${d_path}(+)${s_base}.iii
       "$(SLINK)" $(SLINK_FLAGS) $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.slib $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.obj
       otool -t $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.slib | hex2bin.py ${dst}
-      
+
 
 [Nasm-to-Binary-Code-File]
     <InputFile>
@@ -642,8 +678,6 @@
     <Command.GCC, Command.GCCLD>
         "$(GENFW)" -o $(OUTPUT_DIR)(+)$(MODULE_NAME)hii.rc -g $(MODULE_GUID) --hiibinpackage $(HII_BINARY_PACKAGES) $(GENFW_FLAGS)
         "$(RC)" $(RC_FLAGS) $(OUTPUT_DIR)(+)$(MODULE_NAME)hii.rc ${dst}
-        
+
     <Command.XCODE, Command.RVCT>
         GenFw -o $(OUTPUT_DIR)(+)$(MODULE_NAME)hii.rc -g $(MODULE_GUID) --hiibinpackage $(HII_BINARY_PACKAGES)
-        
-        
diff --git a/BaseTools/Conf/tools_def.template b/BaseTools/Conf/tools_def.template
index aaae4fcd2916..fa0e1ac23211 100755
--- a/BaseTools/Conf/tools_def.template
+++ b/BaseTools/Conf/tools_def.template
@@ -7651,6 +7651,18 @@ RELEASE_RVCTCYGWIN_ARM_CC_FLAGS  = "$(CCPATH_FLAG)" $(ARCHCC_FLAGS) $(PLATFORM_F
 *_*_*_GENFW_FLAGS                  =
 
 ##################
+# GenEbc tool definitions
+##################
+*_*_EBC_GENEBC_PATH                = GenEbcSignature
+*_*_EBC_GENEBC_FLAGS               =
+
+##################
+# PatchEbc tool definitions
+##################
+*_*_EBC_PATCHEBC_PATH              = PatchEbcSignature
+*_*_EBC_PATCHEBC_FLAGS             =
+
+##################
 # Asl Compiler definitions
 ##################
 *_*_*_ASLCC_FLAGS                  = /nologo /c /FIAutoGen.h /TC /Dmain=ReferenceAcpiTable
diff --git a/BaseTools/Source/Python/GenEbcSignature/GenEbcSignature.py b/BaseTools/Source/Python/GenEbcSignature/GenEbcSignature.py
new file mode 100644
index 000000000000..0a509d8d971e
--- /dev/null
+++ b/BaseTools/Source/Python/GenEbcSignature/GenEbcSignature.py
@@ -0,0 +1,306 @@
+## @file
+# Generate a list of EBC call signatures
+#
+# Copyright (c) 2008 - 2016, Intel Corporation. All rights reserved.<BR>
+# Copyright (c) 2016, Pete Batard. All rights reserved.<BR>
+# This program and the accompanying materials
+# are licensed and made available under the terms and conditions of the BSD License
+# which accompanies this distribution.  The full text of the license may be found at
+# http://opensource.org/licenses/bsd-license.php
+#
+# THE PROGRAM IS DISTRIBUTED UNDER THE BSD LICENSE ON AN "AS IS" BASIS,
+# WITHOUT WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED.
+#
+#
+
+#======================================  External Libraries ========================================
+import optparse
+import Common.LongFilePathOs as os
+import re
+import sys
+import array
+import struct
+import pickle
+
+from Common.BuildToolError import *
+import Common.EdkLogger as EdkLogger
+from Common.Misc import PeImageClass
+from Common.BuildVersion import gBUILD_VERSION
+from Common.LongFilePathSupport import OpenLongFilePath as open
+
+# Version and Copyright
+__version_number__ = ("0.5" + " " + gBUILD_VERSION)
+__version__ = "%prog Version " + __version_number__
+__copyright__ = "Copyright (c) 2016, Intel Corporation. All rights reserved."
+
+#======================================  Internal Libraries ========================================
+
+#============================================== Code ===============================================
+EBC_CALL_SIGNATURE = 0x2EBC0000
+
+functionStartRe = re.compile('^\s*(\w+)\s+\(\s*', re.UNICODE)
+param64BitRe = re.compile('\\bU*INT64\\b(?!\s*\*)\s*\w+\\b', re.UNICODE)
+
+def ParseSource(CallList):
+    """ Parse preprocessed source to generate a function signature.
+
+    Note that this is a very basic parser, that expects all function definitions to follow the
+    UEFI coding standard, i.e. with function name and each parameters on individual lines.
+    It also expects 64 bit parameter to be explicitly passed as INT64 or UINT64.
+
+    @return the processed function call list with their signatures set
+    """
+    global Options
+    Status = 0
+    Signature = 0
+    FuncName = ""
+
+    for Line in sys.stdin:
+        Line = Line.strip()
+
+        if Status == 0:
+            m = functionStartRe.match(Line)
+            if m != None:
+                FuncName, = m.groups(0)
+                if FuncName in CallList:
+                    Status = 1
+                    Mask = 1
+                    Signature = 0
+                    if Options.debug > 1:
+                        print FuncName + "("
+                continue
+
+        if Status == 1 and Line[0] == ')':
+            if Options.debug > 1:
+                print ")"
+            CallList[FuncName] = Signature
+            Status = 0
+        if Status == 1:
+            if Options.debug > 1:
+                print "     " + Line
+            m = param64BitRe.match(Line)
+            if m != None:
+                Signature = Signature | Mask
+            Mask = Mask << 1
+
+    return True
+
+def ParseObject(ObjPath):
+    """ Parse a COFF object file to identify function calls that require signature generation
+    @param ObjPath    COFF object absolute path
+
+    @return a list of function call names require signature generation
+    """
+    global Options
+    CallList = {}
+    RelocList = []
+    SymbolList = {}
+
+    with file(ObjPath, 'rb') as f:
+        # Parse the object header
+        Machine = struct.unpack('H', f.read(2))[0]
+        assert Machine == 0xEBC, "Not an EBC object file"
+        NumSections = struct.unpack('H', f.read(2))[0]
+        f.seek(4, 1)    # Skip timestamp
+        SymTabAddr = struct.unpack('I', f.read(4))[0]
+        NumSymbols = struct.unpack('I', f.read(4))[0]
+        OptHeaderSize = struct.unpack('H', f.read(2))[0]
+        assert OptHeaderSize == 0, "Unexpected object file header"
+        Characteristics = struct.unpack('H', f.read(2))[0]
+        StrTabAddr = SymTabAddr + NumSymbols * 18
+        # Optional debug output
+        if Options.debug > 0:
+            print "Number of Sections: " + str(NumSections)
+            print "Number of Symbols:  " + str(NumSymbols)
+            print "Symbol Table at: " + hex(SymTabAddr)
+            print "String Table at: " + hex(StrTabAddr)
+
+        # Parse the object sections, to identify reloc sections
+        for i in range (0, NumSections):
+            SectionName = struct.unpack('8s', f.read(8))[0]
+            SectionName = SectionName.strip()
+            VirtualSize = struct.unpack('I', f.read(4))[0]
+            assert VirtualSize == 0, "Unexpected Obj file section"
+            # Skip Virtual Address
+            f.seek(4, 1)
+            DataSize = struct.unpack('I', f.read(4))[0]
+            DataAddr = struct.unpack('I', f.read(4))[0]
+            RelocAddr = struct.unpack('I', f.read(4))[0]
+            # Skip Line Numbers
+            f.seek(4, 1)
+            RelocNum = struct.unpack('H', f.read(2))[0]
+            f.seek(6, 1)
+            if RelocNum > 0:
+                RelocList.append([RelocNum, RelocAddr])
+            if Options.debug > 1:
+                print "  Section " + str(i) + ": " + SectionName
+                print "    Size: " + hex(DataSize)
+                print "    Addr: " + hex(DataAddr)
+                if RelocNum > 0:
+                    print "    Rels: " + str(RelocNum) + " at " + hex(RelocAddr)
+
+        # Parse reloc sections
+        if RelocList != None:
+            # Copy the String Table into something we can access more easily
+            f.seek(StrTabAddr, 0)
+            StrTabSize = struct.unpack('I', f.read(4))[0]
+            StrData = f.read(StrTabSize)
+            if Options.debug > 0:
+                print "String Table Size: " + str(StrTabSize)
+                if Options.debug > 1:
+                    StrArray = StrData.split(b'\x00')
+                    for String in StrArray:
+                        print "    " + String
+
+            # Now build a full symbol table, that includes long names
+            f.seek(SymTabAddr, 0)
+            NumAuxSections = 0
+            if Options.debug > 0 and NumSymbols > 0:
+                print "Relocation symbols:"
+            for i in range(0, NumSymbols):
+                if NumAuxSections > 0:
+                    f.seek(18, 1)
+                    NumAuxSections = NumAuxSections - 1
+                    continue
+                SymName = f.read(8)
+                SymValue = struct.unpack('I', f.read(4))[0]
+                SymSecNum = struct.unpack('h', f.read(2))[0]
+                SymType = struct.unpack('H', f.read(2))[0]
+                SymClass = struct.unpack('B', f.read(1))[0]
+                NumAuxSections = struct.unpack('B', f.read(1))[0]
+                if SymName[0] != '\0':
+                   SymName = SymName.strip(' \0')
+                else:
+                   StrIndex = struct.unpack('I', SymName[4:8])[0]
+                   StrEnd = StrData.find('\0', StrIndex-3)
+                   SymName = StrData[StrIndex-4:StrEnd]
+                SymbolList[i] = SymName
+                if Options.debug > 1:
+                    print "    " + SymName + ":"
+                    print "      Index=" + str(i) + " SecNum=" + str(SymSecNum) + " Type=" + str(SymType) + " Class=" + str(SymClass) + " NbAux=" + str(NumAuxSections)
+
+            # Finally, go through each reloc symbol and identify the "_plabel" suffixed ones
+            for Entry in RelocList:
+                f.seek(Entry[1], 0)
+
+                for i in range(0, Entry[0]):
+                    # Skip Virtual Address
+                    f.seek(4, 1)
+                    SymbolIndex = struct.unpack('I', f.read(4))[0]
+                    assert SymbolIndex in SymbolList, "Symbol not found in list"
+                    RelocType = struct.unpack('H', f.read(2))[0]
+                    m = re.match('(.*)_plabel', SymbolList[SymbolIndex])
+                    if m != None:
+                        CallName, = m.groups(0)
+                        if CallName not in CallList:
+                            # Add this call, and set its signature to undefined (-1)
+                            CallList[CallName] = -1
+                    if Options.debug > 0:
+                         print "    " + SymbolList[SymbolIndex] + " (Index " + hex(SymbolIndex) + ", Type " + str(RelocType) + ")"
+
+    if len(CallList) == 0:
+        return None
+    return CallList
+
+def UpdateSignatures(CallList, SigFile):
+    """ Create or update a signature file
+    @param CallList   A dictionary containing the signatures to save/update
+    @param SigFile    A path to the signature file to create/update
+
+    @return True if successfull
+    """
+    SavedList = {}
+
+    if Options.verbose != None:
+        print "EBC signatures:"
+        Width = max(len(CallName) for CallName in CallList) + 3
+        for CallName in CallList:
+            print "  " + (CallName + "()").ljust(Width) + "0b" + "{:016b}".format(CallList[CallName])
+
+    try:
+        f = open(SigFile, 'rb')
+        SavedList = pickle.load(f)
+        f.close()
+
+    except:
+        pass
+
+    SavedList.update(CallList)
+    with open(Options.sigfile, 'wb') as f:
+        pickle.dump(SavedList, f, protocol=pickle.HIGHEST_PROTOCOL)
+
+    return True
+
+def ParseOptions():
+    Usage = "%prog [-v] -o <ObjFile> -s <SigFile>"
+    AdditionalNotes = "\nThis application currently only recognizes INT64 or UINT64 types as 64-bit arg."
+    Parser = optparse.OptionParser(usage=Usage, description=__copyright__, version="%prog " + __version_number__)
+    Parser.add_option('-o', '--objfile', action='store', dest='objfile', type="string", help='Path of the object file.')
+    Parser.add_option('-s', '--sigfile',  action='store', dest='sigfile', type="string", help='Path to the signature file.')
+    Parser.add_option("-v", "--verbose", action="store_true", type=None, help="Turn on verbose output with informational messages printed.")
+    Parser.add_option("-q", "--quiet", action="store_true", type=None, help="Disable all messages except FATAL ERRORS.")
+    Parser.add_option("-d", "--debug", action="store", type="int", help="Enable debug messages at specified level.")
+
+    (options, args) = Parser.parse_args()
+    return options
+
+def main():
+    global Options
+    AppName = os.path.basename(sys.argv[0])
+    Options = ParseOptions()
+    ReturnCode = 0
+
+    EdkLogger.Initialize()
+    try:
+        if Options.verbose != None:
+            EdkLogger.SetLevel(EdkLogger.VERBOSE)
+        if Options.quiet != None:
+            EdkLogger.SetLevel(EdkLogger.QUIET)
+        if Options.debug != None:
+            EdkLogger.SetLevel(Options.debug + 1)
+        else:
+            EdkLogger.SetLevel(EdkLogger.INFO)
+
+        if Options.objfile == None:
+            EdkLogger.error(AppName, OPTION_MISSING, "Object file not defined",
+                ExtraData="Please use '-o' switch to set the input Object file.")
+        if Options.sigfile == None:
+            EdkLogger.error(AppName, OPTION_MISSING, "Signature file not defined",
+                ExtraData="Please use '-s' switch to set the output Signature file.")
+
+        List = ParseObject(Options.objfile)
+        if List != None:
+            ParseSource(List)
+            # Check for missing signatures
+            for CallName in List:
+                assert List[CallName] != -1, "Could not set signature for function " + CallName
+            UpdateSignatures(List, Options.sigfile)
+        else:
+            # Still need to deplete stdin data
+            for Line in sys.stdin:
+                pass
+
+    except FatalError, X:
+        if Options.debug != None:
+            import traceback
+            EdkLogger.quiet(traceback.format_exc())
+        ReturnCode = X.args[0]
+
+    except:
+        import traceback
+        EdkLogger.error(
+            "\nPython",
+            CODE_ERROR,
+            "Tools code failure",
+            ExtraData="Please send email to edk2-devel@lists.01.org for help, attaching following call stack trace!\n",
+            RaiseError=False
+        )
+        EdkLogger.quiet(traceback.format_exc())
+        ReturnCode = CODE_ERROR
+    return ReturnCode
+
+if __name__ == '__main__':
+    r = main()
+    ## 0-127 is a safe return range, and 1 is a standard default error
+    if r < 0 or r > 127: r = 1
+    sys.exit(r)
diff --git a/BaseTools/Source/Python/GenEbcSignature/__init__.py b/BaseTools/Source/Python/GenEbcSignature/__init__.py
new file mode 100644
index 000000000000..58a5b1d630f0
--- /dev/null
+++ b/BaseTools/Source/Python/GenEbcSignature/__init__.py
@@ -0,0 +1,15 @@
+## @file
+# Python 'GenPatchPcdTable' package initialization file.
+#
+# This file is required to make Python interpreter treat the directory
+# as containing package.
+#
+# Copyright (c) 2010, Intel Corporation. All rights reserved.<BR>
+# This program and the accompanying materials
+# are licensed and made available under the terms and conditions of the BSD License
+# which accompanies this distribution.  The full text of the license may be found at
+# http://opensource.org/licenses/bsd-license.php
+#
+# THE PROGRAM IS DISTRIBUTED UNDER THE BSD LICENSE ON AN "AS IS" BASIS,
+# WITHOUT WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED.
+#
diff --git a/BaseTools/Source/Python/PatchEbcSignature/PatchEbcSignature.py b/BaseTools/Source/Python/PatchEbcSignature/PatchEbcSignature.py
new file mode 100644
index 000000000000..14f34641fb09
--- /dev/null
+++ b/BaseTools/Source/Python/PatchEbcSignature/PatchEbcSignature.py
@@ -0,0 +1,226 @@
+## @file
+# Insert EBC Call Signature data in an EFI executable
+#
+# Copyright (c) 2008 - 2016, Intel Corporation. All rights reserved.<BR>
+# Copyright (c) 2016, Pete Batard. All rights reserved.<BR>
+# This program and the accompanying materials
+# are licensed and made available under the terms and conditions of the BSD License
+# which accompanies this distribution.  The full text of the license may be found at
+# http://opensource.org/licenses/bsd-license.php
+#
+# THE PROGRAM IS DISTRIBUTED UNDER THE BSD LICENSE ON AN "AS IS" BASIS,
+# WITHOUT WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED.
+#
+#
+
+#======================================  External Libraries ========================================
+import optparse
+import Common.LongFilePathOs as os
+import re
+import sys
+import array
+import struct
+import pickle
+
+from Common.BuildToolError import *
+import Common.EdkLogger as EdkLogger
+from Common.Misc import PeImageClass
+from Common.BuildVersion import gBUILD_VERSION
+from Common.LongFilePathSupport import OpenLongFilePath as open
+
+# Version and Copyright
+__version_number__ = ("1.0" + " " + gBUILD_VERSION)
+__version__ = "%prog Version " + __version_number__
+__copyright__ = "Copyright (c) 2016, Intel Corporation. All rights reserved."
+
+#======================================  Internal Libraries ========================================
+
+#============================================== Code ===============================================
+EBC_CALL_SIGNATURE = 0x2EBC0000
+
+CallRegexp = re.compile('^[\da-fA-F]+:[\da-fA-F]+ +([\w@\$]+)_plabel +([\da-fA-F]+) f? +([\w]+):([\w]+)\.obj', re.UNICODE)
+LoadRegexp = re.compile('Preferred load address is +([\da-fA-F]+)', re.UNICODE)
+
+def ParseMap(MapPath):
+    """ Parse a map file and get the list and address of functions that need an EBC call signature
+    @param MapPath    Map file absolute path
+
+    @return a list of call names with their address and source location
+    """
+    Status = 0
+    CallList = []
+
+    with file(MapPath, 'r') as f:
+        for Line in f:
+            Line = Line.strip()
+
+            if Status == 0:
+                m = LoadRegexp.match(Line)
+                if m != None:
+                    LoadAddr, = m.groups(0)
+                    LoadAddr = int(LoadAddr, 16)
+                    Status = 1
+
+            if Status == 1:
+                m = CallRegexp.match(Line)
+                if m != None:
+                    FuncName, FuncAddr, ModName, SrcName = m.groups(0)
+                    FuncAddr = int(FuncAddr, 16)
+                    SrcName = SrcName + ".c"
+                    CallList.append([FuncName, FuncAddr - LoadAddr, ModName, SrcName])
+
+    assert Status != 0, "Failed to read load address"
+
+    if len(CallList) == 0:
+        return None
+    return CallList
+
+def BuildSignatureList(FileList):
+    """ Parse the list of files passed as arguments, and try to locate and open
+        the matching signature files to build a complete EBC signature list.
+        as the signature file to ensure that the signatures we need are present.
+    @param FileList     A list of .sig or .lib files
+
+    @return a list of function calls with their signature data
+    """
+    global Options
+    SigList = {}
+
+    for File in FileList:
+        File = os.path.splitext(File)[0]+'.sig'
+        try:
+            with open(File, 'rb') as f:
+                SavedList = pickle.load(f)
+                SigList.update(SavedList)
+                if Options.verbose != None:
+                    print "  Loaded " + str(len(SavedList)) + " EBC signature(s) from " + File
+        except:
+            pass
+
+    if len(SigList) == 0:
+        return None
+    return SigList
+
+def CheckSignatures(EfiPath, MapList, SigList):
+    """ Sanity check to ensures that the signatures' data and addresses are valid.
+    @param EfiPath      EFI binary absolute path
+    @param MapList      Function calls requiring signature, with their address
+    @param SigList      Function calls signature dictionary
+
+    @return True if the check passed
+    """
+
+    for Entry in MapList:
+        # Check for missing signatures
+        assert Entry[0] in SigList, Entry[0] + ": missing signature"
+        # Make sure the signature fits in 16 bits
+        assert SigList[Entry[0]] < 0x10000, Entry[0] + ": invalid signature"
+
+    with file(EfiPath, 'rb') as f:
+        for Entry in MapList:
+            f.seek(Entry[1] + 4)
+            Data = struct.unpack('I', f.read(4))[0]
+            # The 32 bit data should either be 0 or have the call signature marker
+            assert Data == 0 or Data & 0xFFFF0000 == EBC_CALL_SIGNATURE, "Unexpected data at address 0x%x" % Entry[1]
+
+    return True;
+
+def InsertSignatures(EfiPath, MapList, SigList):
+    """ Check the EFI binary to ensure we have the right signature addresses
+    @param EfiPath      EFI binary absolute path
+    @param Maplist      Function calls requiring signature, with their address
+    @param SigList      Function calls signature dictionary
+
+    @return True if the signatures were successfully patched
+    """
+    global Options
+
+    with file(EfiPath, 'r+b') as f:
+        Width = max(len(Entry[0]) for Entry in MapList) + 3
+        for Entry in MapList:
+            f.seek(Entry[1] + 4)
+            f.write(struct.pack('I', EBC_CALL_SIGNATURE + SigList[Entry[0]]))
+            if Options.verbose != None:
+                print "  Patched " + (Entry[0] + "()").ljust(Width) + "at 0x{:08X}".format(Entry[1]) + " with signature 0x{:04X}".format(SigList[Entry[0]]) + " (" + Entry[2] + ":" + Entry[3] + ")"
+
+    return True;
+
+def ParseOptions():
+    Usage = "%prog [-v] -m <MapFile> -e <EfiFile> -s <SigFile> [file1] [file2] [...]"
+    AdditionalNotes = "\nfile# can be either a .sig or .lib file. If the latter, its path is used to look for a matching .sig."
+    Parser = optparse.OptionParser(usage=Usage, description=__copyright__, version="%prog " + __version_number__)
+    Parser.add_option('-m', '--mapfile', action='store', dest='mapfile', type="string", help='Path of the module map file.')
+    Parser.add_option('-e', '--efifile', action='store', dest='efifile', type="string", help='Path of the EFI binary to patch.')
+    Parser.add_option("-v", "--verbose", action="store_true", type=None, help="Turn on verbose output with informational messages printed.")
+    Parser.add_option("-q", "--quiet", action="store_true", type=None, help="Disable all messages except FATAL ERRORS.")
+    Parser.add_option("-d", "--debug", action="store", type="int", help="Enable debug messages at specified level.")
+
+    return Parser.parse_args()
+
+def main():
+    global Options
+    AppName = os.path.basename(sys.argv[0])
+    (Options, ArgList) = ParseOptions()
+    ReturnCode = 0
+
+    EdkLogger.Initialize()
+
+    try:
+        if Options.verbose != None:
+            EdkLogger.SetLevel(EdkLogger.VERBOSE)
+        if Options.quiet != None:
+            EdkLogger.SetLevel(EdkLogger.QUIET)
+        if Options.debug != None:
+            EdkLogger.SetLevel(Options.debug + 1)
+        else:
+            EdkLogger.SetLevel(EdkLogger.INFO)
+
+        if Options.mapfile == None:
+            EdkLogger.error(AppName, OPTION_MISSING, "Map file not defined",
+                ExtraData="Please use '-m' switch to define a map file.")
+        if Options.efifile == None:
+            EdkLogger.error(AppName, OPTION_MISSING, "EFI file not defined",
+                ExtraData="Please use '-e' switch to set the EFI binary to patch.")
+        if ArgList == None:
+            EdkLogger.error(AppName, OPTION_MISSING, "No signature or library file provided",
+                ExtraData="At least one signature or library path must be provided as argument.")
+
+        MapList = ParseMap(Options.mapfile)
+        if MapList != None:
+            SigList = BuildSignatureList(ArgList)
+            assert SigList != None, "No signatures found"
+            CheckSignatures(Options.efifile, MapList, SigList)
+            if Options.debug != None:
+                print "EBC signatures:"
+                Width = max(len(CallName) for CallName in SigList) + 3
+                for CallName in SigList:
+                    print "  " + (CallName + "()").ljust(Width) + "0b" + "{:016b}".format(SigList[CallName])
+            InsertSignatures(Options.efifile, MapList, SigList)
+        elif Options.verbose:
+            print 'No EBC call signatures patching required'
+
+    except FatalError, X:
+        if Options.debug != None:
+            import traceback
+            EdkLogger.quiet(traceback.format_exc())
+        ReturnCode = X.args[0]
+
+    except:
+        import traceback
+        EdkLogger.error(
+            "\nPython",
+            CODE_ERROR,
+            "Tools code failure",
+            ExtraData="Please send email to edk2-devel@lists.01.org for help, attaching following call stack trace!\n",
+            RaiseError=False
+        )
+        EdkLogger.quiet(traceback.format_exc())
+        ReturnCode = CODE_ERROR
+
+    return ReturnCode
+
+if __name__ == '__main__':
+    r = main()
+    ## 0-127 is a safe return range, and 1 is a standard default error
+    if r < 0 or r > 127: r = 1
+    sys.exit(r)
diff --git a/BaseTools/Source/Python/PatchEbcSignature/__init__.py b/BaseTools/Source/Python/PatchEbcSignature/__init__.py
new file mode 100644
index 000000000000..58a5b1d630f0
--- /dev/null
+++ b/BaseTools/Source/Python/PatchEbcSignature/__init__.py
@@ -0,0 +1,15 @@
+## @file
+# Python 'GenPatchPcdTable' package initialization file.
+#
+# This file is required to make Python interpreter treat the directory
+# as containing package.
+#
+# Copyright (c) 2010, Intel Corporation. All rights reserved.<BR>
+# This program and the accompanying materials
+# are licensed and made available under the terms and conditions of the BSD License
+# which accompanies this distribution.  The full text of the license may be found at
+# http://opensource.org/licenses/bsd-license.php
+#
+# THE PROGRAM IS DISTRIBUTED UNDER THE BSD LICENSE ON AN "AS IS" BASIS,
+# WITHOUT WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED.
+#
-- 
2.9.3.windows.2



      parent reply	other threads:[~2017-01-24 12:30 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-01-24 12:30 [PATCH 1/5] MdeModulePkg/EbcDxe: allow VmReadIndex##() to return a decoded index Pete Batard
2017-01-24 12:30 ` [PATCH 2/5] MdeModulePkg/EbcDxe: add ARM support Pete Batard
2017-01-24 12:30 ` [PATCH 3/5] MdeModulePkg/EbcDxe: add a stack tracker for ARM EBC->native support Pete Batard
2017-01-24 12:30 ` [PATCH 4/5] MdeModulePkg/EbcDxe: add call signatures for ARM native->EBC support Pete Batard
2017-01-24 12:30 ` Pete Batard [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-list from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20170124123026.5204-5-pete@akeo.ie \
    --to=devel@edk2.groups.io \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox