From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) (using TLSv1 with cipher CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id E93051A1DEF for ; Thu, 27 Oct 2016 21:52:36 -0700 (PDT) Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga101.fm.intel.com with ESMTP; 27 Oct 2016 21:52:37 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.31,407,1473145200"; d="scan'208";a="24683647" Received: from fmsmsx103.amr.corp.intel.com ([10.18.124.201]) by fmsmga005.fm.intel.com with ESMTP; 27 Oct 2016 21:52:36 -0700 Received: from fmsmsx153.amr.corp.intel.com (10.18.125.6) by FMSMSX103.amr.corp.intel.com (10.18.124.201) with Microsoft SMTP Server (TLS) id 14.3.248.2; Thu, 27 Oct 2016 21:52:35 -0700 Received: from shsmsx151.ccr.corp.intel.com (10.239.6.50) by FMSMSX153.amr.corp.intel.com (10.18.125.6) with Microsoft SMTP Server (TLS) id 14.3.248.2; Thu, 27 Oct 2016 21:52:34 -0700 Received: from shsmsx103.ccr.corp.intel.com ([169.254.4.139]) by SHSMSX151.ccr.corp.intel.com ([169.254.3.96]) with mapi id 14.03.0248.002; Fri, 28 Oct 2016 12:52:22 +0800 From: "Zhu, Yonghong" To: "Gao, Liming" , "edk2-devel@lists.01.org" Thread-Topic: [Patch 1/3] BaseTools LzmaCompress: Update LZMA to new 16.04 version Thread-Index: AQHSMFeYpZg3bZceyUC964KYaZLFHaC9TT/A Date: Fri, 28 Oct 2016 04:52:21 +0000 Message-ID: References: <1477575568-18932-1-git-send-email-liming.gao@intel.com> <1477575568-18932-2-git-send-email-liming.gao@intel.com> In-Reply-To: <1477575568-18932-2-git-send-email-liming.gao@intel.com> Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiMGZlMGYwYTMtMTllNy00ODQ0LWIzMzktMGMyMzZiN2Y0NjU4IiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX0lDIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE1LjkuNi42IiwiVHJ1c3RlZExhYmVsSGFzaCI6IkY5SVFpNWV3NEFsOXdnQWRidmZ5OE9IUlpKRWl5NThkeE9JeHJnVGhidGc9In0= x-ctpclassification: CTP_IC x-originating-ip: [10.239.127.40] MIME-Version: 1.0 Subject: Re: [Patch 1/3] BaseTools LzmaCompress: Update LZMA to new 16.04 version X-BeenThere: edk2-devel@lists.01.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: EDK II Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Oct 2016 04:52:37 -0000 Content-Language: en-US Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Reviewed-by: Yonghong Zhu =20 Best Regards, Zhu Yonghong -----Original Message----- From: Gao, Liming=20 Sent: Thursday, October 27, 2016 9:39 PM To: edk2-devel@lists.01.org Cc: Zhu, Yonghong Subject: [Patch 1/3] BaseTools LzmaCompress: Update LZMA to new 16.04 versi= on New version LZMA SDK improves the compression performance on windows OS, and has no change on the compression ratio. I compress 8M FVMAIN image, the compression time is reduced from 2.590s to 1.419s. Cc: Yonghong Zhu Contributed-under: TianoCore Contribution Agreement 1.0 Signed-off-by: Liming Gao --- BaseTools/Source/C/LzmaCompress/GNUmakefile | 3 +- .../Source/C/LzmaCompress/LZMA-SDK-README.txt | 4 +- BaseTools/Source/C/LzmaCompress/LzmaCompress.c | 12 +- BaseTools/Source/C/LzmaCompress/Makefile | 6 +- BaseTools/Source/C/LzmaCompress/Sdk/C/7zFile.c | 31 +- BaseTools/Source/C/LzmaCompress/Sdk/C/7zFile.h | 13 +- BaseTools/Source/C/LzmaCompress/Sdk/C/7zStream.c | 12 +- BaseTools/Source/C/LzmaCompress/Sdk/C/7zTypes.h | 256 +++++++++ BaseTools/Source/C/LzmaCompress/Sdk/C/7zVersion.h | 24 +- BaseTools/Source/C/LzmaCompress/Sdk/C/Alloc.c | 15 +- BaseTools/Source/C/LzmaCompress/Sdk/C/Alloc.h | 13 +- BaseTools/Source/C/LzmaCompress/Sdk/C/Bra.h | 8 +- BaseTools/Source/C/LzmaCompress/Sdk/C/Bra86.c | 99 ++-- BaseTools/Source/C/LzmaCompress/Sdk/C/Compiler.h | 32 ++ BaseTools/Source/C/LzmaCompress/Sdk/C/CpuArch.h | 206 ++++++- BaseTools/Source/C/LzmaCompress/Sdk/C/LzFind.c | 515 ++++++++++++++---= - BaseTools/Source/C/LzmaCompress/Sdk/C/LzFind.h | 40 +- BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.c | 214 ++++---- BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.h | 14 +- BaseTools/Source/C/LzmaCompress/Sdk/C/LzHash.h | 47 +- BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaDec.c | 269 +++++++--- BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaDec.h | 12 +- BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.c | 512 ++++++++++-------= - BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.h | 14 +- BaseTools/Source/C/LzmaCompress/Sdk/C/Precomp.h | 10 + BaseTools/Source/C/LzmaCompress/Sdk/C/Threads.c | 116 ++-- BaseTools/Source/C/LzmaCompress/Sdk/C/Threads.h | 95 ++-- BaseTools/Source/C/LzmaCompress/Sdk/C/Types.h | 208 -------- .../Source/C/LzmaCompress/Sdk/DOC/lzma-history.txt | 363 +++++++++++++ .../Source/C/LzmaCompress/Sdk/DOC/lzma-sdk.txt | 357 +++++++++++++ BaseTools/Source/C/LzmaCompress/Sdk/history.txt | 236 -------- BaseTools/Source/C/LzmaCompress/Sdk/lzma.txt | 594 -----------------= ---- 32 files changed, 2504 insertions(+), 1846 deletions(-) create mode 100644 BaseTools/Source/C/LzmaCompress/Sdk/C/7zTypes.h create mode 100644 BaseTools/Source/C/LzmaCompress/Sdk/C/Compiler.h create mode 100644 BaseTools/Source/C/LzmaCompress/Sdk/C/Precomp.h delete mode 100644 BaseTools/Source/C/LzmaCompress/Sdk/C/Types.h create mode 100644 BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-history.tx= t create mode 100644 BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-sdk.txt delete mode 100644 BaseTools/Source/C/LzmaCompress/Sdk/history.txt delete mode 100644 BaseTools/Source/C/LzmaCompress/Sdk/lzma.txt diff --git a/BaseTools/Source/C/LzmaCompress/GNUmakefile b/BaseTools/Source= /C/LzmaCompress/GNUmakefile index 21c75ef..63d7ee0 100644 --- a/BaseTools/Source/C/LzmaCompress/GNUmakefile +++ b/BaseTools/Source/C/LzmaCompress/GNUmakefile @@ -1,7 +1,7 @@ ## @file # GNU/Linux makefile for 'LzmaCompress' module build. # -# Copyright (c) 2009 - 2012, Intel Corporation. All rights reserved.
+# Copyright (c) 2009 - 2016, Intel Corporation. All rights reserved.
# This program and the accompanying materials # are licensed and made available under the terms and conditions of the BS= D License # which accompanies this distribution. The full text of the license may b= e found at @@ -29,3 +29,4 @@ OBJECTS =3D \ =20 include $(MAKEROOT)/Makefiles/app.makefile =20 +BUILD_CFLAGS +=3D -D_7ZIP_ST diff --git a/BaseTools/Source/C/LzmaCompress/LZMA-SDK-README.txt b/BaseTool= s/Source/C/LzmaCompress/LZMA-SDK-README.txt index e0476ad..6882e74 100644 --- a/BaseTools/Source/C/LzmaCompress/LZMA-SDK-README.txt +++ b/BaseTools/Source/C/LzmaCompress/LZMA-SDK-README.txt @@ -1,3 +1,3 @@ -LzmaCompress is based on the LZMA SDK 4.65. LZMA SDK 4.65 -was placed in the public domain on 2009-02-03. It was +LzmaCompress is based on the LZMA SDK 16.04. LZMA SDK 16.04 +was placed in the public domain on 2016-10-04. It was released on the http://www.7-zip.org/sdk.html website. \ No newline at end of file diff --git a/BaseTools/Source/C/LzmaCompress/LzmaCompress.c b/BaseTools/Sou= rce/C/LzmaCompress/LzmaCompress.c index 1de07a3..aff0bed 100644 --- a/BaseTools/Source/C/LzmaCompress/LzmaCompress.c +++ b/BaseTools/Source/C/LzmaCompress/LzmaCompress.c @@ -1,11 +1,11 @@ /** @file LZMA Compress/Decompress tool (LzmaCompress) =20 - Based on LZMA SDK 4.65: + Based on LZMA SDK 16.04: LzmaUtil.c -- Test application for LZMA compression - 2008-11-23 : Igor Pavlov : Public domain + 2016-10-04 : Igor Pavlov : Public domain =20 - Copyright (c) 2006 - 2012, Intel Corporation. All rights reserved.
+ Copyright (c) 2006 - 2016, Intel Corporation. All rights reserved.
This program and the accompanying materials are licensed and made available under the terms and conditions of the BS= D License which accompanies this distribution. The full text of the license may b= e found at @@ -43,10 +43,6 @@ const char *kCantWriteMessage =3D "Can not write output = file"; const char *kCantAllocateMessage =3D "Can not allocate memory"; const char *kDataErrorMessage =3D "Data error"; =20 -static void *SzAlloc(void *p, size_t size) { (void)p; return MyAlloc(size)= ; } -static void SzFree(void *p, void *address) { (void)p; MyFree(address); } -static ISzAlloc g_Alloc =3D { SzAlloc, SzFree }; - static Bool mQuietMode =3D False; static CONVERTER_TYPE mConType =3D NoConverter; =20 @@ -54,7 +50,7 @@ static CONVERTER_TYPE mConType =3D NoConverter; #define UTILITY_MAJOR_VERSION 0 #define UTILITY_MINOR_VERSION 2 #define INTEL_COPYRIGHT \ - "Copyright (c) 2009-2012, Intel Corporation. All rights reserved." + "Copyright (c) 2009-2016, Intel Corporation. All rights reserved." void PrintHelp(char *buffer) { strcat(buffer, diff --git a/BaseTools/Source/C/LzmaCompress/Makefile b/BaseTools/Source/C/= LzmaCompress/Makefile index 872e899..48aca88 100644 --- a/BaseTools/Source/C/LzmaCompress/Makefile +++ b/BaseTools/Source/C/LzmaCompress/Makefile @@ -1,7 +1,7 @@ ## @file # Windows makefile for 'LzmaCompress' module build. # -# Copyright (c) 2009 - 2012, Intel Corporation. All rights reserved.
+# Copyright (c) 2009 - 2016, Intel Corporation. All rights reserved.
# This program and the accompanying materials # are licensed and made available under the terms and conditions of the BS= D License # which accompanies this distribution. The full text of the license may b= e found at @@ -26,7 +26,9 @@ OBJECTS =3D \ $(SDK_C)\LzmaEnc.obj \ $(SDK_C)\7zFile.obj \ $(SDK_C)\7zStream.obj \ - $(SDK_C)\Bra86.obj + $(SDK_C)\Bra86.obj \ + $(SDK_C)\LzFindMt.obj \ + $(SDK_C)\Threads.obj =20 !INCLUDE ..\Makefiles\ms.app =20 diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/7zFile.c b/BaseTools/Sou= rce/C/LzmaCompress/Sdk/C/7zFile.c index 2b605a5..98fe716 100644 --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/7zFile.c +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/7zFile.c @@ -1,15 +1,17 @@ /* 7zFile.c -- File IO -2008-11-22 : Igor Pavlov : Public domain */ +2009-11-24 : Igor Pavlov : Public domain */ + +#include "Precomp.h" =20 #include "7zFile.h" =20 #ifndef USE_WINDOWS_FILE =20 +#ifndef UNDER_CE #include - #endif =20 -#ifdef USE_WINDOWS_FILE +#else =20 /* ReadFile and WriteFile functions in Windows have BUG: @@ -34,6 +36,7 @@ void File_Construct(CSzFile *p) #endif } =20 +#if !defined(UNDER_CE) || !defined(USE_WINDOWS_FILE) static WRes File_Open(CSzFile *p, const char *name, int writeMode) { #ifdef USE_WINDOWS_FILE @@ -45,12 +48,32 @@ static WRes File_Open(CSzFile *p, const char *name, int= writeMode) return (p->handle !=3D INVALID_HANDLE_VALUE) ? 0 : GetLastError(); #else p->file =3D fopen(name, writeMode ? "wb+" : "rb"); - return (p->file !=3D 0) ? 0 : errno; + return (p->file !=3D 0) ? 0 : + #ifdef UNDER_CE + 2; /* ENOENT */ + #else + errno; + #endif #endif } =20 WRes InFile_Open(CSzFile *p, const char *name) { return File_Open(p, name,= 0); } WRes OutFile_Open(CSzFile *p, const char *name) { return File_Open(p, name= , 1); } +#endif + +#ifdef USE_WINDOWS_FILE +static WRes File_OpenW(CSzFile *p, const WCHAR *name, int writeMode) +{ + p->handle =3D CreateFileW(name, + writeMode ? GENERIC_WRITE : GENERIC_READ, + FILE_SHARE_READ, NULL, + writeMode ? CREATE_ALWAYS : OPEN_EXISTING, + FILE_ATTRIBUTE_NORMAL, NULL); + return (p->handle !=3D INVALID_HANDLE_VALUE) ? 0 : GetLastError(); +} +WRes InFile_OpenW(CSzFile *p, const WCHAR *name) { return File_OpenW(p, na= me, 0); } +WRes OutFile_OpenW(CSzFile *p, const WCHAR *name) { return File_OpenW(p, n= ame, 1); } +#endif =20 WRes File_Close(CSzFile *p) { diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/7zFile.h b/BaseTools/Sou= rce/C/LzmaCompress/Sdk/C/7zFile.h index 4fd2936..d62a192 100644 --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/7zFile.h +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/7zFile.h @@ -1,5 +1,5 @@ /* 7zFile.h -- File IO -2008-11-22 : Igor Pavlov : Public domain */ +2013-01-18 : Igor Pavlov : Public domain */ =20 #ifndef __7Z_FILE_H #define __7Z_FILE_H @@ -14,8 +14,9 @@ #include #endif =20 -#include "Types.h" +#include "7zTypes.h" =20 +EXTERN_C_BEGIN =20 /* ---------- File ---------- */ =20 @@ -29,8 +30,14 @@ typedef struct } CSzFile; =20 void File_Construct(CSzFile *p); +#if !defined(UNDER_CE) || !defined(USE_WINDOWS_FILE) WRes InFile_Open(CSzFile *p, const char *name); WRes OutFile_Open(CSzFile *p, const char *name); +#endif +#ifdef USE_WINDOWS_FILE +WRes InFile_OpenW(CSzFile *p, const WCHAR *name); +WRes OutFile_OpenW(CSzFile *p, const WCHAR *name); +#endif WRes File_Close(CSzFile *p); =20 /* reads max(*size, remain file's size) bytes */ @@ -71,4 +78,6 @@ typedef struct =20 void FileOutStream_CreateVTable(CFileOutStream *p); =20 +EXTERN_C_END + #endif diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/7zStream.c b/BaseTools/S= ource/C/LzmaCompress/Sdk/C/7zStream.c index a0a2405..5a92d53 100644 --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/7zStream.c +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/7zStream.c @@ -1,9 +1,11 @@ /* 7zStream.c -- 7z Stream functions -2008-11-23 : Igor Pavlov : Public domain */ +2013-11-12 : Igor Pavlov : Public domain */ + +#include "Precomp.h" =20 #include =20 -#include "Types.h" +#include "7zTypes.h" =20 SRes SeqInStream_Read2(ISeqInStream *stream, void *buf, size_t size, SRes = errorType) { @@ -39,7 +41,7 @@ SRes LookInStream_SeekTo(ILookInStream *stream, UInt64 of= fset) =20 SRes LookInStream_LookRead(ILookInStream *stream, void *buf, size_t *size) { - void *lookBuf; + const void *lookBuf; if (*size =3D=3D 0) return SZ_OK; RINOK(stream->Look(stream, &lookBuf, size)); @@ -66,7 +68,7 @@ SRes LookInStream_Read(ILookInStream *stream, void *buf, = size_t size) return LookInStream_Read2(stream, buf, size, SZ_ERROR_INPUT_EOF); } =20 -static SRes LookToRead_Look_Lookahead(void *pp, void **buf, size_t *size) +static SRes LookToRead_Look_Lookahead(void *pp, const void **buf, size_t *= size) { SRes res =3D SZ_OK; CLookToRead *p =3D (CLookToRead *)pp; @@ -84,7 +86,7 @@ static SRes LookToRead_Look_Lookahead(void *pp, void **bu= f, size_t *size) return res; } =20 -static SRes LookToRead_Look_Exact(void *pp, void **buf, size_t *size) +static SRes LookToRead_Look_Exact(void *pp, const void **buf, size_t *size= ) { SRes res =3D SZ_OK; CLookToRead *p =3D (CLookToRead *)pp; diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/7zTypes.h b/BaseTools/So= urce/C/LzmaCompress/Sdk/C/7zTypes.h new file mode 100644 index 0000000..903047b --- /dev/null +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/7zTypes.h @@ -0,0 +1,256 @@ +/* 7zTypes.h -- Basic types +2013-11-12 : Igor Pavlov : Public domain */ + +#ifndef __7Z_TYPES_H +#define __7Z_TYPES_H + +#ifdef _WIN32 +/* #include */ +#endif + +#include + +#ifndef EXTERN_C_BEGIN +#ifdef __cplusplus +#define EXTERN_C_BEGIN extern "C" { +#define EXTERN_C_END } +#else +#define EXTERN_C_BEGIN +#define EXTERN_C_END +#endif +#endif + +EXTERN_C_BEGIN + +#define SZ_OK 0 + +#define SZ_ERROR_DATA 1 +#define SZ_ERROR_MEM 2 +#define SZ_ERROR_CRC 3 +#define SZ_ERROR_UNSUPPORTED 4 +#define SZ_ERROR_PARAM 5 +#define SZ_ERROR_INPUT_EOF 6 +#define SZ_ERROR_OUTPUT_EOF 7 +#define SZ_ERROR_READ 8 +#define SZ_ERROR_WRITE 9 +#define SZ_ERROR_PROGRESS 10 +#define SZ_ERROR_FAIL 11 +#define SZ_ERROR_THREAD 12 + +#define SZ_ERROR_ARCHIVE 16 +#define SZ_ERROR_NO_ARCHIVE 17 + +typedef int SRes; + +#ifdef _WIN32 +/* typedef DWORD WRes; */ +typedef unsigned WRes; +#else +typedef int WRes; +#endif + +#ifndef RINOK +#define RINOK(x) { int __result__ =3D (x); if (__result__ !=3D 0) return _= _result__; } +#endif + +typedef unsigned char Byte; +typedef short Int16; +typedef unsigned short UInt16; + +#ifdef _LZMA_UINT32_IS_ULONG +typedef long Int32; +typedef unsigned long UInt32; +#else +typedef int Int32; +typedef unsigned int UInt32; +#endif + +#ifdef _SZ_NO_INT_64 + +/* define _SZ_NO_INT_64, if your compiler doesn't support 64-bit integers. + NOTES: Some code will work incorrectly in that case! */ + +typedef long Int64; +typedef unsigned long UInt64; + +#else + +#if defined(_MSC_VER) || defined(__BORLANDC__) +typedef __int64 Int64; +typedef unsigned __int64 UInt64; +#define UINT64_CONST(n) n +#else +typedef long long int Int64; +typedef unsigned long long int UInt64; +#define UINT64_CONST(n) n ## ULL +#endif + +#endif + +#ifdef _LZMA_NO_SYSTEM_SIZE_T +typedef UInt32 SizeT; +#else +typedef size_t SizeT; +#endif + +typedef int Bool; +#define True 1 +#define False 0 + + +#ifdef _WIN32 +#define MY_STD_CALL __stdcall +#else +#define MY_STD_CALL +#endif + +#ifdef _MSC_VER + +#if _MSC_VER >=3D 1300 +#define MY_NO_INLINE __declspec(noinline) +#else +#define MY_NO_INLINE +#endif + +#define MY_CDECL __cdecl +#define MY_FAST_CALL __fastcall + +#else + +#define MY_NO_INLINE +#define MY_CDECL +#define MY_FAST_CALL + +#endif + + +/* The following interfaces use first parameter as pointer to structure */ + +typedef struct +{ + Byte (*Read)(void *p); /* reads one byte, returns 0 in case of EOF or er= ror */ +} IByteIn; + +typedef struct +{ + void (*Write)(void *p, Byte b); +} IByteOut; + +typedef struct +{ + SRes (*Read)(void *p, void *buf, size_t *size); + /* if (input(*size) !=3D 0 && output(*size) =3D=3D 0) means end_of_str= eam. + (output(*size) < input(*size)) is allowed */ +} ISeqInStream; + +/* it can return SZ_ERROR_INPUT_EOF */ +SRes SeqInStream_Read(ISeqInStream *stream, void *buf, size_t size); +SRes SeqInStream_Read2(ISeqInStream *stream, void *buf, size_t size, SRes = errorType); +SRes SeqInStream_ReadByte(ISeqInStream *stream, Byte *buf); + +typedef struct +{ + size_t (*Write)(void *p, const void *buf, size_t size); + /* Returns: result - the number of actually written bytes. + (result < size) means error */ +} ISeqOutStream; + +typedef enum +{ + SZ_SEEK_SET =3D 0, + SZ_SEEK_CUR =3D 1, + SZ_SEEK_END =3D 2 +} ESzSeek; + +typedef struct +{ + SRes (*Read)(void *p, void *buf, size_t *size); /* same as ISeqInStream= ::Read */ + SRes (*Seek)(void *p, Int64 *pos, ESzSeek origin); +} ISeekInStream; + +typedef struct +{ + SRes (*Look)(void *p, const void **buf, size_t *size); + /* if (input(*size) !=3D 0 && output(*size) =3D=3D 0) means end_of_str= eam. + (output(*size) > input(*size)) is not allowed + (output(*size) < input(*size)) is allowed */ + SRes (*Skip)(void *p, size_t offset); + /* offset must be <=3D output(*size) of Look */ + + SRes (*Read)(void *p, void *buf, size_t *size); + /* reads directly (without buffer). It's same as ISeqInStream::Read */ + SRes (*Seek)(void *p, Int64 *pos, ESzSeek origin); +} ILookInStream; + +SRes LookInStream_LookRead(ILookInStream *stream, void *buf, size_t *size)= ; +SRes LookInStream_SeekTo(ILookInStream *stream, UInt64 offset); + +/* reads via ILookInStream::Read */ +SRes LookInStream_Read2(ILookInStream *stream, void *buf, size_t size, SRe= s errorType); +SRes LookInStream_Read(ILookInStream *stream, void *buf, size_t size); + +#define LookToRead_BUF_SIZE (1 << 14) + +typedef struct +{ + ILookInStream s; + ISeekInStream *realStream; + size_t pos; + size_t size; + Byte buf[LookToRead_BUF_SIZE]; +} CLookToRead; + +void LookToRead_CreateVTable(CLookToRead *p, int lookahead); +void LookToRead_Init(CLookToRead *p); + +typedef struct +{ + ISeqInStream s; + ILookInStream *realStream; +} CSecToLook; + +void SecToLook_CreateVTable(CSecToLook *p); + +typedef struct +{ + ISeqInStream s; + ILookInStream *realStream; +} CSecToRead; + +void SecToRead_CreateVTable(CSecToRead *p); + +typedef struct +{ + SRes (*Progress)(void *p, UInt64 inSize, UInt64 outSize); + /* Returns: result. (result !=3D SZ_OK) means break. + Value (UInt64)(Int64)-1 for size means unknown value. */ +} ICompressProgress; + +typedef struct +{ + void *(*Alloc)(void *p, size_t size); + void (*Free)(void *p, void *address); /* address can be 0 */ +} ISzAlloc; + +#define IAlloc_Alloc(p, size) (p)->Alloc((p), size) +#define IAlloc_Free(p, a) (p)->Free((p), a) + +#ifdef _WIN32 + +#define CHAR_PATH_SEPARATOR '\\' +#define WCHAR_PATH_SEPARATOR L'\\' +#define STRING_PATH_SEPARATOR "\\" +#define WSTRING_PATH_SEPARATOR L"\\" + +#else + +#define CHAR_PATH_SEPARATOR '/' +#define WCHAR_PATH_SEPARATOR L'/' +#define STRING_PATH_SEPARATOR "/" +#define WSTRING_PATH_SEPARATOR L"/" + +#endif + +EXTERN_C_END + +#endif diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/7zVersion.h b/BaseTools/= Source/C/LzmaCompress/Sdk/C/7zVersion.h index 5a6bcad..acb67a9 100644 --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/7zVersion.h +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/7zVersion.h @@ -1,7 +1,19 @@ -#define MY_VER_MAJOR 4 -#define MY_VER_MINOR 65 +#define MY_VER_MAJOR 16 +#define MY_VER_MINOR 04 #define MY_VER_BUILD 0 -#define MY_VERSION "4.65" -#define MY_DATE "2009-02-03" -#define MY_COPYRIGHT ": Igor Pavlov : Public domain" -#define MY_VERSION_COPYRIGHT_DATE MY_VERSION " " MY_COPYRIGHT " : " MY_DAT= E +#define MY_VERSION_NUMBERS "16.04" +#define MY_VERSION "16.04" +#define MY_DATE "2016-10-04" +#undef MY_COPYRIGHT +#undef MY_VERSION_COPYRIGHT_DATE +#define MY_AUTHOR_NAME "Igor Pavlov" +#define MY_COPYRIGHT_PD "Igor Pavlov : Public domain" +#define MY_COPYRIGHT_CR "Copyright (c) 1999-2016 Igor Pavlov" + +#ifdef USE_COPYRIGHT_CR + #define MY_COPYRIGHT MY_COPYRIGHT_CR +#else + #define MY_COPYRIGHT MY_COPYRIGHT_PD +#endif + +#define MY_VERSION_COPYRIGHT_DATE MY_VERSION " : " MY_COPYRIGHT " : " MY_D= ATE diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/Alloc.c b/BaseTools/Sour= ce/C/LzmaCompress/Sdk/C/Alloc.c index bb24a77..9f1d036 100644 --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/Alloc.c +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/Alloc.c @@ -1,7 +1,7 @@ /* Alloc.c -- Memory allocation functions -2008-09-24 -Igor Pavlov -Public domain */ +2015-02-21 : Igor Pavlov : Public domain */ + +#include "Precomp.h" =20 #ifdef _WIN32 #include @@ -125,3 +125,12 @@ void BigFree(void *address) } =20 #endif + + +static void *SzAlloc(void *p, size_t size) { UNUSED_VAR(p); return MyAlloc= (size); } +static void SzFree(void *p, void *address) { UNUSED_VAR(p); MyFree(address= ); } +ISzAlloc g_Alloc =3D { SzAlloc, SzFree }; + +static void *SzBigAlloc(void *p, size_t size) { UNUSED_VAR(p); return BigA= lloc(size); } +static void SzBigFree(void *p, void *address) { UNUSED_VAR(p); BigFree(add= ress); } +ISzAlloc g_BigAlloc =3D { SzBigAlloc, SzBigFree }; diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/Alloc.h b/BaseTools/Sour= ce/C/LzmaCompress/Sdk/C/Alloc.h index a396c6b..73b282a 100644 --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/Alloc.h +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/Alloc.h @@ -1,12 +1,12 @@ /* Alloc.h -- Memory allocation functions -2008-03-13 -Igor Pavlov -Public domain */ +2015-02-21 : Igor Pavlov : Public domain */ =20 #ifndef __COMMON_ALLOC_H #define __COMMON_ALLOC_H =20 -#include +#include "7zTypes.h" + +EXTERN_C_BEGIN =20 void *MyAlloc(size_t size); void MyFree(void *address); @@ -29,4 +29,9 @@ void BigFree(void *address); =20 #endif =20 +extern ISzAlloc g_Alloc; +extern ISzAlloc g_BigAlloc; + +EXTERN_C_END + #endif diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/Bra.h b/BaseTools/Source= /C/LzmaCompress/Sdk/C/Bra.h index b9018eb..aba8dce 100644 --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/Bra.h +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/Bra.h @@ -1,10 +1,12 @@ /* Bra.h -- Branch converters for executables -2008-10-04 : Igor Pavlov : Public domain */ +2013-01-18 : Igor Pavlov : Public domain */ =20 #ifndef __BRA_H #define __BRA_H =20 -#include "Types.h" +#include "7zTypes.h" + +EXTERN_C_BEGIN =20 /* These functions convert relative addresses to absolute addresses @@ -57,4 +59,6 @@ SizeT PPC_Convert(Byte *data, SizeT size, UInt32 ip, int = encoding); SizeT SPARC_Convert(Byte *data, SizeT size, UInt32 ip, int encoding); SizeT IA64_Convert(Byte *data, SizeT size, UInt32 ip, int encoding); =20 +EXTERN_C_END + #endif diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/Bra86.c b/BaseTools/Sour= ce/C/LzmaCompress/Sdk/C/Bra86.c index 93566cb..8dd3ed4 100644 --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/Bra86.c +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/Bra86.c @@ -1,85 +1,82 @@ /* Bra86.c -- Converter for x86 code (BCJ) -2008-10-04 : Igor Pavlov : Public domain */ +2013-11-12 : Igor Pavlov : Public domain */ =20 -#include "Bra.h" +#include "Precomp.h" =20 -#define Test86MSByte(b) ((b) =3D=3D 0 || (b) =3D=3D 0xFF) +#include "Bra.h" =20 -const Byte kMaskToAllowedStatus[8] =3D {1, 1, 1, 0, 1, 0, 0, 0}; -const Byte kMaskToBitNumber[8] =3D {0, 1, 2, 2, 3, 3, 3, 3}; +#define Test86MSByte(b) ((((b) + 1) & 0xFE) =3D=3D 0) =20 SizeT x86_Convert(Byte *data, SizeT size, UInt32 ip, UInt32 *state, int en= coding) { - SizeT bufferPos =3D 0, prevPosT; - UInt32 prevMask =3D *state & 0x7; + SizeT pos =3D 0; + UInt32 mask =3D *state & 7; if (size < 5) return 0; + size -=3D 4; ip +=3D 5; - prevPosT =3D (SizeT)0 - 1; =20 for (;;) { - Byte *p =3D data + bufferPos; - Byte *limit =3D data + size - 4; + Byte *p =3D data + pos; + const Byte *limit =3D data + size; for (; p < limit; p++) if ((*p & 0xFE) =3D=3D 0xE8) break; - bufferPos =3D (SizeT)(p - data); - if (p >=3D limit) - break; - prevPosT =3D bufferPos - prevPosT; - if (prevPosT > 3) - prevMask =3D 0; - else + { - prevMask =3D (prevMask << ((int)prevPosT - 1)) & 0x7; - if (prevMask !=3D 0) + SizeT d =3D (SizeT)(p - data - pos); + pos =3D (SizeT)(p - data); + if (p >=3D limit) { - Byte b =3D p[4 - kMaskToBitNumber[prevMask]]; - if (!kMaskToAllowedStatus[prevMask] || Test86MSByte(b)) + *state =3D (d > 2 ? 0 : mask >> (unsigned)d); + return pos; + } + if (d > 2) + mask =3D 0; + else + { + mask >>=3D (unsigned)d; + if (mask !=3D 0 && (mask > 4 || mask =3D=3D 3 || Test86MSByte(p[(m= ask >> 1) + 1]))) { - prevPosT =3D bufferPos; - prevMask =3D ((prevMask << 1) & 0x7) | 1; - bufferPos++; + mask =3D (mask >> 1) | 4; + pos++; continue; } } } - prevPosT =3D bufferPos; =20 if (Test86MSByte(p[4])) { - UInt32 src =3D ((UInt32)p[4] << 24) | ((UInt32)p[3] << 16) | ((UInt3= 2)p[2] << 8) | ((UInt32)p[1]); - UInt32 dest; - for (;;) + UInt32 v =3D ((UInt32)p[4] << 24) | ((UInt32)p[3] << 16) | ((UInt32)= p[2] << 8) | ((UInt32)p[1]); + UInt32 cur =3D ip + (UInt32)pos; + pos +=3D 5; + if (encoding) + v +=3D cur; + else + v -=3D cur; + if (mask !=3D 0) { - Byte b; - int index; - if (encoding) - dest =3D (ip + (UInt32)bufferPos) + src; - else - dest =3D src - (ip + (UInt32)bufferPos); - if (prevMask =3D=3D 0) - break; - index =3D kMaskToBitNumber[prevMask] * 8; - b =3D (Byte)(dest >> (24 - index)); - if (!Test86MSByte(b)) - break; - src =3D dest ^ ((1 << (32 - index)) - 1); + unsigned sh =3D (mask & 6) << 2; + if (Test86MSByte((Byte)(v >> sh))) + { + v ^=3D (((UInt32)0x100 << sh) - 1); + if (encoding) + v +=3D cur; + else + v -=3D cur; + } + mask =3D 0; } - p[4] =3D (Byte)(~(((dest >> 24) & 1) - 1)); - p[3] =3D (Byte)(dest >> 16); - p[2] =3D (Byte)(dest >> 8); - p[1] =3D (Byte)dest; - bufferPos +=3D 5; + p[1] =3D (Byte)v; + p[2] =3D (Byte)(v >> 8); + p[3] =3D (Byte)(v >> 16); + p[4] =3D (Byte)(0 - ((v >> 24) & 1)); } else { - prevMask =3D ((prevMask << 1) & 0x7) | 1; - bufferPos++; + mask =3D (mask >> 1) | 4; + pos++; } } - prevPosT =3D bufferPos - prevPosT; - *state =3D ((prevPosT > 3) ? 0 : ((prevMask << ((int)prevPosT - 1)) & 0x= 7)); - return bufferPos; } diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/Compiler.h b/BaseTools/S= ource/C/LzmaCompress/Sdk/C/Compiler.h new file mode 100644 index 0000000..de8fab3 --- /dev/null +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/Compiler.h @@ -0,0 +1,32 @@ +/* Compiler.h +2015-08-02 : Igor Pavlov : Public domain */ + +#ifndef __7Z_COMPILER_H +#define __7Z_COMPILER_H + +#ifdef _MSC_VER + + #ifdef UNDER_CE + #define RPC_NO_WINDOWS_H + /* #pragma warning(disable : 4115) // '_RPC_ASYNC_STATE' : named type = definition in parentheses */ + #pragma warning(disable : 4201) // nonstandard extension used : namele= ss struct/union + #pragma warning(disable : 4214) // nonstandard extension used : bit fi= eld types other than int + #endif + + #if _MSC_VER >=3D 1300 + #pragma warning(disable : 4996) // This function or variable may be un= safe + #else + #pragma warning(disable : 4511) // copy constructor could not be gener= ated + #pragma warning(disable : 4512) // assignment operator could not be ge= nerated + #pragma warning(disable : 4514) // unreferenced inline function has be= en removed + #pragma warning(disable : 4702) // unreachable code + #pragma warning(disable : 4710) // not inlined + #pragma warning(disable : 4786) // identifier was truncated to '255' c= haracters in the debug information + #endif + +#endif + +#define UNUSED_VAR(x) (void)x; +/* #define UNUSED_VAR(x) x=3Dx; */ + +#endif diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/CpuArch.h b/BaseTools/So= urce/C/LzmaCompress/Sdk/C/CpuArch.h index 006361f..ef6083c 100644 --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/CpuArch.h +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/CpuArch.h @@ -1,33 +1,119 @@ -/* CpuArch.h -2008-08-05 -Igor Pavlov -Public domain */ +/* CpuArch.h -- CPU specific code +2016-06-09: Igor Pavlov : Public domain */ =20 -#ifndef __CPUARCH_H -#define __CPUARCH_H +#ifndef __CPU_ARCH_H +#define __CPU_ARCH_H + +#include "7zTypes.h" + +EXTERN_C_BEGIN =20 /* -LITTLE_ENDIAN_UNALIGN means: - 1) CPU is LITTLE_ENDIAN - 2) it's allowed to make unaligned memory accesses -if LITTLE_ENDIAN_UNALIGN is not defined, it means that we don't know -about these properties of platform. +MY_CPU_LE means that CPU is LITTLE ENDIAN. +MY_CPU_BE means that CPU is BIG ENDIAN. +If MY_CPU_LE and MY_CPU_BE are not defined, we don't know about ENDIANNESS= of platform. + +MY_CPU_LE_UNALIGN means that CPU is LITTLE ENDIAN and CPU supports unalign= ed memory accesses. */ =20 -#if defined(_M_IX86) || defined(_M_X64) || defined(_M_AMD64) || defined(__= i386__) || defined(__x86_64__) -#define LITTLE_ENDIAN_UNALIGN +#if defined(_M_X64) \ + || defined(_M_AMD64) \ + || defined(__x86_64__) \ + || defined(__AMD64__) \ + || defined(__amd64__) + #define MY_CPU_AMD64 +#endif + +#if defined(MY_CPU_AMD64) \ + || defined(_M_IA64) \ + || defined(__AARCH64EL__) \ + || defined(__AARCH64EB__) + #define MY_CPU_64BIT +#endif + +#if defined(_M_IX86) || defined(__i386__) +#define MY_CPU_X86 +#endif + +#if defined(MY_CPU_X86) || defined(MY_CPU_AMD64) +#define MY_CPU_X86_OR_AMD64 +#endif + +#if defined(MY_CPU_X86) \ + || defined(_M_ARM) \ + || defined(__ARMEL__) \ + || defined(__THUMBEL__) \ + || defined(__ARMEB__) \ + || defined(__THUMBEB__) + #define MY_CPU_32BIT +#endif + +#if defined(_WIN32) && defined(_M_ARM) +#define MY_CPU_ARM_LE #endif =20 -#ifdef LITTLE_ENDIAN_UNALIGN +#if defined(_WIN32) && defined(_M_IA64) +#define MY_CPU_IA64_LE +#endif + +#if defined(MY_CPU_X86_OR_AMD64) \ + || defined(MY_CPU_ARM_LE) \ + || defined(MY_CPU_IA64_LE) \ + || defined(__LITTLE_ENDIAN__) \ + || defined(__ARMEL__) \ + || defined(__THUMBEL__) \ + || defined(__AARCH64EL__) \ + || defined(__MIPSEL__) \ + || defined(__MIPSEL) \ + || defined(_MIPSEL) \ + || defined(__BFIN__) \ + || (defined(__BYTE_ORDER__) && (__BYTE_ORDER__ =3D=3D __ORDER_LITTLE_E= NDIAN__)) + #define MY_CPU_LE +#endif + +#if defined(__BIG_ENDIAN__) \ + || defined(__ARMEB__) \ + || defined(__THUMBEB__) \ + || defined(__AARCH64EB__) \ + || defined(__MIPSEB__) \ + || defined(__MIPSEB) \ + || defined(_MIPSEB) \ + || defined(__m68k__) \ + || defined(__s390__) \ + || defined(__s390x__) \ + || defined(__zarch__) \ + || (defined(__BYTE_ORDER__) && (__BYTE_ORDER__ =3D=3D __ORDER_BIG_ENDI= AN__)) + #define MY_CPU_BE +#endif + +#if defined(MY_CPU_LE) && defined(MY_CPU_BE) +Stop_Compiling_Bad_Endian +#endif + + +#ifdef MY_CPU_LE + #if defined(MY_CPU_X86_OR_AMD64) \ + /* || defined(__AARCH64EL__) */ + #define MY_CPU_LE_UNALIGN + #endif +#endif + + +#ifdef MY_CPU_LE_UNALIGN =20 -#define GetUi16(p) (*(const UInt16 *)(p)) -#define GetUi32(p) (*(const UInt32 *)(p)) -#define GetUi64(p) (*(const UInt64 *)(p)) -#define SetUi32(p, d) *(UInt32 *)(p) =3D (d); +#define GetUi16(p) (*(const UInt16 *)(const void *)(p)) +#define GetUi32(p) (*(const UInt32 *)(const void *)(p)) +#define GetUi64(p) (*(const UInt64 *)(const void *)(p)) + +#define SetUi16(p, v) { *(UInt16 *)(p) =3D (v); } +#define SetUi32(p, v) { *(UInt32 *)(p) =3D (v); } +#define SetUi64(p, v) { *(UInt64 *)(p) =3D (v); } =20 #else =20 -#define GetUi16(p) (((const Byte *)(p))[0] | ((UInt16)((const Byte *)(p))[= 1] << 8)) +#define GetUi16(p) ( (UInt16) ( \ + ((const Byte *)(p))[0] | \ + ((UInt16)((const Byte *)(p))[1] << 8) )) =20 #define GetUi32(p) ( \ ((const Byte *)(p))[0] | \ @@ -37,21 +123,43 @@ about these properties of platform. =20 #define GetUi64(p) (GetUi32(p) | ((UInt64)GetUi32(((const Byte *)(p)) + 4)= << 32)) =20 -#define SetUi32(p, d) { UInt32 _x_ =3D (d); \ - ((Byte *)(p))[0] =3D (Byte)_x_; \ - ((Byte *)(p))[1] =3D (Byte)(_x_ >> 8); \ - ((Byte *)(p))[2] =3D (Byte)(_x_ >> 16); \ - ((Byte *)(p))[3] =3D (Byte)(_x_ >> 24); } +#define SetUi16(p, v) { Byte *_ppp_ =3D (Byte *)(p); UInt32 _vvv_ =3D (v);= \ + _ppp_[0] =3D (Byte)_vvv_; \ + _ppp_[1] =3D (Byte)(_vvv_ >> 8); } + +#define SetUi32(p, v) { Byte *_ppp_ =3D (Byte *)(p); UInt32 _vvv_ =3D (v);= \ + _ppp_[0] =3D (Byte)_vvv_; \ + _ppp_[1] =3D (Byte)(_vvv_ >> 8); \ + _ppp_[2] =3D (Byte)(_vvv_ >> 16); \ + _ppp_[3] =3D (Byte)(_vvv_ >> 24); } + +#define SetUi64(p, v) { Byte *_ppp2_ =3D (Byte *)(p); UInt64 _vvv2_ =3D (v= ); \ + SetUi32(_ppp2_ , (UInt32)_vvv2_); \ + SetUi32(_ppp2_ + 4, (UInt32)(_vvv2_ >> 32)); } =20 #endif =20 -#if defined(LITTLE_ENDIAN_UNALIGN) && defined(_WIN64) && (_MSC_VER >=3D 13= 00) + +#if defined(MY_CPU_LE_UNALIGN) && /* defined(_WIN64) && */ (_MSC_VER >=3D = 1300) + +/* Note: we use bswap instruction, that is unsupported in 386 cpu */ + +#include =20 #pragma intrinsic(_byteswap_ulong) #pragma intrinsic(_byteswap_uint64) #define GetBe32(p) _byteswap_ulong(*(const UInt32 *)(const Byte *)(p)) #define GetBe64(p) _byteswap_uint64(*(const UInt64 *)(const Byte *)(p)) =20 +#define SetBe32(p, v) (*(UInt32 *)(void *)(p)) =3D _byteswap_ulong(v) + +#elif defined(MY_CPU_LE_UNALIGN) && defined (__GNUC__) && (__GNUC__ > 4 ||= (__GNUC__ =3D=3D 4 && __GNUC_MINOR__ >=3D 3)) + +#define GetBe32(p) __builtin_bswap32(*(const UInt32 *)(const Byte *)(p)) +#define GetBe64(p) __builtin_bswap64(*(const UInt64 *)(const Byte *)(p)) + +#define SetBe32(p, v) (*(UInt32 *)(void *)(p)) =3D __builtin_bswap32(v) + #else =20 #define GetBe32(p) ( \ @@ -62,8 +170,54 @@ about these properties of platform. =20 #define GetBe64(p) (((UInt64)GetBe32(p) << 32) | GetBe32(((const Byte *)(p= )) + 4)) =20 +#define SetBe32(p, v) { Byte *_ppp_ =3D (Byte *)(p); UInt32 _vvv_ =3D (v);= \ + _ppp_[0] =3D (Byte)(_vvv_ >> 24); \ + _ppp_[1] =3D (Byte)(_vvv_ >> 16); \ + _ppp_[2] =3D (Byte)(_vvv_ >> 8); \ + _ppp_[3] =3D (Byte)_vvv_; } + +#endif + + +#define GetBe16(p) ( (UInt16) ( \ + ((UInt16)((const Byte *)(p))[0] << 8) | \ + ((const Byte *)(p))[1] )) + + + +#ifdef MY_CPU_X86_OR_AMD64 + +typedef struct +{ + UInt32 maxFunc; + UInt32 vendor[3]; + UInt32 ver; + UInt32 b; + UInt32 c; + UInt32 d; +} Cx86cpuid; + +enum +{ + CPU_FIRM_INTEL, + CPU_FIRM_AMD, + CPU_FIRM_VIA +}; + +void MyCPUID(UInt32 function, UInt32 *a, UInt32 *b, UInt32 *c, UInt32 *d); + +Bool x86cpuid_CheckAndRead(Cx86cpuid *p); +int x86cpuid_GetFirm(const Cx86cpuid *p); + +#define x86cpuid_GetFamily(ver) (((ver >> 16) & 0xFF0) | ((ver >> 8) & 0xF= )) +#define x86cpuid_GetModel(ver) (((ver >> 12) & 0xF0) | ((ver >> 4) & 0xF= )) +#define x86cpuid_GetStepping(ver) (ver & 0xF) + +Bool CPU_Is_InOrder(); +Bool CPU_Is_Aes_Supported(); + #endif =20 -#define GetBe16(p) (((UInt16)((const Byte *)(p))[0] << 8) | ((const Byte *= )(p))[1]) +EXTERN_C_END =20 #endif diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFind.c b/BaseTools/Sou= rce/C/LzmaCompress/Sdk/C/LzFind.c index e0ebe62..c335d36 100644 --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFind.c +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFind.c @@ -1,5 +1,7 @@ /* LzFind.c -- Match finder for LZ algorithms -2008-10-04 : Igor Pavlov : Public domain */ +2015-10-15 : Igor Pavlov : Public domain */ + +#include "Precomp.h" =20 #include =20 @@ -9,8 +11,8 @@ #define kEmptyHashValue 0 #define kMaxValForNormalize ((UInt32)0xFFFFFFFF) #define kNormalizeStepMin (1 << 10) /* it must be power of 2 */ -#define kNormalizeMask (~(kNormalizeStepMin - 1)) -#define kMaxHistorySize ((UInt32)3 << 30) +#define kNormalizeMask (~(UInt32)(kNormalizeStepMin - 1)) +#define kMaxHistorySize ((UInt32)7 << 29) =20 #define kStartMaxLen 3 =20 @@ -19,7 +21,7 @@ static void LzInWindow_Free(CMatchFinder *p, ISzAlloc *al= loc) if (!p->directInput) { alloc->Free(alloc, p->bufferBase); - p->bufferBase =3D 0; + p->bufferBase =3D NULL; } } =20 @@ -33,17 +35,16 @@ static int LzInWindow_Create(CMatchFinder *p, UInt32 ke= epSizeReserv, ISzAlloc *a p->blockSize =3D blockSize; return 1; } - if (p->bufferBase =3D=3D 0 || p->blockSize !=3D blockSize) + if (!p->bufferBase || p->blockSize !=3D blockSize) { LzInWindow_Free(p, alloc); p->blockSize =3D blockSize; p->bufferBase =3D (Byte *)alloc->Alloc(alloc, (size_t)blockSize); } - return (p->bufferBase !=3D 0); + return (p->bufferBase !=3D NULL); } =20 Byte *MatchFinder_GetPointerToCurrentPos(CMatchFinder *p) { return p->buff= er; } -Byte MatchFinder_GetIndexByte(CMatchFinder *p, Int32 index) { return p->bu= ffer[index]; } =20 UInt32 MatchFinder_GetNumAvailableBytes(CMatchFinder *p) { return p->strea= mPos - p->pos; } =20 @@ -58,12 +59,28 @@ static void MatchFinder_ReadBlock(CMatchFinder *p) { if (p->streamEndWasReached || p->result !=3D SZ_OK) return; + + /* We use (p->streamPos - p->pos) value. (p->streamPos < p->pos) is allo= wed. */ + + if (p->directInput) + { + UInt32 curSize =3D 0xFFFFFFFF - (p->streamPos - p->pos); + if (curSize > p->directInputRem) + curSize =3D (UInt32)p->directInputRem; + p->directInputRem -=3D curSize; + p->streamPos +=3D curSize; + if (p->directInputRem =3D=3D 0) + p->streamEndWasReached =3D 1; + return; + } + =20 for (;;) { Byte *dest =3D p->buffer + (p->streamPos - p->pos); size_t size =3D (p->bufferBase + p->blockSize - dest); if (size =3D=3D 0) return; + p->result =3D p->stream->Read(p->stream, dest, &size); if (p->result !=3D SZ_OK) return; @@ -81,13 +98,15 @@ static void MatchFinder_ReadBlock(CMatchFinder *p) void MatchFinder_MoveBlock(CMatchFinder *p) { memmove(p->bufferBase, - p->buffer - p->keepSizeBefore, - (size_t)(p->streamPos - p->pos + p->keepSizeBefore)); + p->buffer - p->keepSizeBefore, + (size_t)(p->streamPos - p->pos) + p->keepSizeBefore); p->buffer =3D p->bufferBase + p->keepSizeBefore; } =20 int MatchFinder_NeedMove(CMatchFinder *p) { + if (p->directInput) + return 0; /* if (p->streamEndWasReached) return 0; */ return ((size_t)(p->bufferBase + p->blockSize - p->buffer) <=3D p->keepS= izeAfter); } @@ -112,8 +131,6 @@ static void MatchFinder_SetDefaultSettings(CMatchFinder= *p) p->cutValue =3D 32; p->btMode =3D 1; p->numHashBytes =3D 4; - /* p->skipModeBits =3D 0; */ - p->directInput =3D 0; p->bigHash =3D 0; } =20 @@ -122,15 +139,15 @@ static void MatchFinder_SetDefaultSettings(CMatchFind= er *p) void MatchFinder_Construct(CMatchFinder *p) { UInt32 i; - p->bufferBase =3D 0; + p->bufferBase =3D NULL; p->directInput =3D 0; - p->hash =3D 0; + p->hash =3D NULL; MatchFinder_SetDefaultSettings(p); =20 for (i =3D 0; i < 256; i++) { UInt32 r =3D i; - int j; + unsigned j; for (j =3D 0; j < 8; j++) r =3D (r >> 1) ^ (kCrcPoly & ~((r & 1) - 1)); p->crc[i] =3D r; @@ -140,7 +157,7 @@ void MatchFinder_Construct(CMatchFinder *p) static void MatchFinder_FreeThisClassMemory(CMatchFinder *p, ISzAlloc *all= oc) { alloc->Free(alloc, p->hash); - p->hash =3D 0; + p->hash =3D NULL; } =20 void MatchFinder_Free(CMatchFinder *p, ISzAlloc *alloc) @@ -149,11 +166,11 @@ void MatchFinder_Free(CMatchFinder *p, ISzAlloc *allo= c) LzInWindow_Free(p, alloc); } =20 -static CLzRef* AllocRefs(UInt32 num, ISzAlloc *alloc) +static CLzRef* AllocRefs(size_t num, ISzAlloc *alloc) { size_t sizeInBytes =3D (size_t)num * sizeof(CLzRef); if (sizeInBytes / sizeof(CLzRef) !=3D num) - return 0; + return NULL; return (CLzRef *)alloc->Alloc(alloc, sizeInBytes); } =20 @@ -162,22 +179,27 @@ int MatchFinder_Create(CMatchFinder *p, UInt32 histor= ySize, ISzAlloc *alloc) { UInt32 sizeReserv; + =20 if (historySize > kMaxHistorySize) { MatchFinder_Free(p, alloc); return 0; } + =20 sizeReserv =3D historySize >> 1; - if (historySize > ((UInt32)2 << 30)) - sizeReserv =3D historySize >> 2; + if (historySize >=3D ((UInt32)3 << 30)) sizeReserv =3D historySize = >> 3; + else if (historySize >=3D ((UInt32)2 << 30)) sizeReserv =3D historySize = >> 2; + =20 sizeReserv +=3D (keepAddBufferBefore + matchMaxLen + keepAddBufferAfter)= / 2 + (1 << 19); =20 p->keepSizeBefore =3D historySize + keepAddBufferBefore + 1; p->keepSizeAfter =3D matchMaxLen + keepAddBufferAfter; + =20 /* we need one additional byte, since we use MoveBlock after pos++ and b= efore dictionary using */ + =20 if (LzInWindow_Create(p, sizeReserv, alloc)) { - UInt32 newCyclicBufferSize =3D (historySize /* >> p->skipModeBits */) = + 1; + UInt32 newCyclicBufferSize =3D historySize + 1; UInt32 hs; p->matchMaxLen =3D matchMaxLen; { @@ -192,7 +214,6 @@ int MatchFinder_Create(CMatchFinder *p, UInt32 historyS= ize, hs |=3D (hs >> 4); hs |=3D (hs >> 8); hs >>=3D 1; - /* hs >>=3D p->skipModeBits; */ hs |=3D 0xFFFF; /* don't change it! It's required for Deflate */ if (hs > (1 << 24)) { @@ -200,6 +221,7 @@ int MatchFinder_Create(CMatchFinder *p, UInt32 historyS= ize, hs =3D (1 << 24) - 1; else hs >>=3D 1; + /* if (bigHash) mode, GetHeads4b() in LzFindMt.c needs (hs >=3D = ((1 << 24) - 1))) */ } } p->hashMask =3D hs; @@ -211,24 +233,32 @@ int MatchFinder_Create(CMatchFinder *p, UInt32 histor= ySize, } =20 { - UInt32 prevSize =3D p->hashSizeSum + p->numSons; - UInt32 newSize; + size_t newSize; + size_t numSons; p->historySize =3D historySize; p->hashSizeSum =3D hs; p->cyclicBufferSize =3D newCyclicBufferSize; - p->numSons =3D (p->btMode ? newCyclicBufferSize * 2 : newCyclicBuffe= rSize); - newSize =3D p->hashSizeSum + p->numSons; - if (p->hash !=3D 0 && prevSize =3D=3D newSize) + =20 + numSons =3D newCyclicBufferSize; + if (p->btMode) + numSons <<=3D 1; + newSize =3D hs + numSons; + + if (p->hash && p->numRefs =3D=3D newSize) return 1; + =20 MatchFinder_FreeThisClassMemory(p, alloc); + p->numRefs =3D newSize; p->hash =3D AllocRefs(newSize, alloc); - if (p->hash !=3D 0) + =20 + if (p->hash) { p->son =3D p->hash + p->hashSizeSum; return 1; } } } + MatchFinder_Free(p, alloc); return 0; } @@ -237,9 +267,11 @@ static void MatchFinder_SetLimits(CMatchFinder *p) { UInt32 limit =3D kMaxValForNormalize - p->pos; UInt32 limit2 =3D p->cyclicBufferSize - p->cyclicBufferPos; + =20 if (limit2 < limit) limit =3D limit2; limit2 =3D p->streamPos - p->pos; + =20 if (limit2 <=3D p->keepSizeAfter) { if (limit2 > 0) @@ -247,8 +279,10 @@ static void MatchFinder_SetLimits(CMatchFinder *p) } else limit2 -=3D p->keepSizeAfter; + =20 if (limit2 < limit) limit =3D limit2; + =20 { UInt32 lenLimit =3D p->streamPos - p->pos; if (lenLimit > p->matchMaxLen) @@ -258,28 +292,39 @@ static void MatchFinder_SetLimits(CMatchFinder *p) p->posLimit =3D p->pos + limit; } =20 -void MatchFinder_Init(CMatchFinder *p) +void MatchFinder_Init_2(CMatchFinder *p, int readData) { UInt32 i; - for (i =3D 0; i < p->hashSizeSum; i++) - p->hash[i] =3D kEmptyHashValue; + UInt32 *hash =3D p->hash; + UInt32 num =3D p->hashSizeSum; + for (i =3D 0; i < num; i++) + hash[i] =3D kEmptyHashValue; + =20 p->cyclicBufferPos =3D 0; p->buffer =3D p->bufferBase; p->pos =3D p->streamPos =3D p->cyclicBufferSize; p->result =3D SZ_OK; p->streamEndWasReached =3D 0; - MatchFinder_ReadBlock(p); + =20 + if (readData) + MatchFinder_ReadBlock(p); + =20 MatchFinder_SetLimits(p); } =20 +void MatchFinder_Init(CMatchFinder *p) +{ + MatchFinder_Init_2(p, True); +} + =20 static UInt32 MatchFinder_GetSubValue(CMatchFinder *p) { return (p->pos - p->historySize - 1) & kNormalizeMask; } =20 -void MatchFinder_Normalize3(UInt32 subValue, CLzRef *items, UInt32 numItem= s) +void MatchFinder_Normalize3(UInt32 subValue, CLzRef *items, size_t numItem= s) { - UInt32 i; + size_t i; for (i =3D 0; i < numItems; i++) { UInt32 value =3D items[i]; @@ -294,7 +339,7 @@ void MatchFinder_Normalize3(UInt32 subValue, CLzRef *it= ems, UInt32 numItems) static void MatchFinder_Normalize(CMatchFinder *p) { UInt32 subValue =3D MatchFinder_GetSubValue(p); - MatchFinder_Normalize3(subValue, p->hash, p->hashSizeSum + p->numSons); + MatchFinder_Normalize3(subValue, p->hash, p->numRefs); MatchFinder_ReduceOffsets(p, subValue); } =20 @@ -455,7 +500,7 @@ static void SkipMatchesSpec(UInt32 lenLimit, UInt32 cur= Match, UInt32 pos, const static void MatchFinder_MovePos(CMatchFinder *p) { MOVE_POS; } =20 #define GET_MATCHES_HEADER2(minLen, ret_op) \ - UInt32 lenLimit; UInt32 hashValue; const Byte *cur; UInt32 curMatch; \ + UInt32 lenLimit; UInt32 hv; const Byte *cur; UInt32 curMatch; \ lenLimit =3D p->lenLimit; { if (lenLimit < minLen) { MatchFinder_MovePos= (p); ret_op; }} \ cur =3D p->buffer; =20 @@ -471,13 +516,20 @@ static void MatchFinder_MovePos(CMatchFinder *p) { MO= VE_POS; } #define SKIP_FOOTER \ SkipMatchesSpec(lenLimit, curMatch, MF_PARAMS(p)); MOVE_POS; =20 +#define UPDATE_maxLen { \ + ptrdiff_t diff =3D (ptrdiff_t)0 - d2; \ + const Byte *c =3D cur + maxLen; \ + const Byte *lim =3D cur + lenLimit; \ + for (; c !=3D lim; c++) if (*(c + diff) !=3D *c) break; \ + maxLen =3D (UInt32)(c - cur); } + static UInt32 Bt2_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distance= s) { UInt32 offset; GET_MATCHES_HEADER(2) HASH2_CALC; - curMatch =3D p->hash[hashValue]; - p->hash[hashValue] =3D p->pos; + curMatch =3D p->hash[hv]; + p->hash[hv] =3D p->pos; offset =3D 0; GET_MATCHES_FOOTER(offset, 1) } @@ -487,35 +539,38 @@ UInt32 Bt3Zip_MatchFinder_GetMatches(CMatchFinder *p,= UInt32 *distances) UInt32 offset; GET_MATCHES_HEADER(3) HASH_ZIP_CALC; - curMatch =3D p->hash[hashValue]; - p->hash[hashValue] =3D p->pos; + curMatch =3D p->hash[hv]; + p->hash[hv] =3D p->pos; offset =3D 0; GET_MATCHES_FOOTER(offset, 2) } =20 static UInt32 Bt3_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distance= s) { - UInt32 hash2Value, delta2, maxLen, offset; + UInt32 h2, d2, maxLen, offset, pos; + UInt32 *hash; GET_MATCHES_HEADER(3) =20 HASH3_CALC; =20 - delta2 =3D p->pos - p->hash[hash2Value]; - curMatch =3D p->hash[kFix3HashSize + hashValue]; - =20 - p->hash[hash2Value] =3D - p->hash[kFix3HashSize + hashValue] =3D p->pos; + hash =3D p->hash; + pos =3D p->pos; =20 + d2 =3D pos - hash[h2]; + + curMatch =3D hash[kFix3HashSize + hv]; + =20 + hash[h2] =3D pos; + hash[kFix3HashSize + hv] =3D pos; =20 maxLen =3D 2; offset =3D 0; - if (delta2 < p->cyclicBufferSize && *(cur - delta2) =3D=3D *cur) + + if (d2 < p->cyclicBufferSize && *(cur - d2) =3D=3D *cur) { - for (; maxLen !=3D lenLimit; maxLen++) - if (cur[(ptrdiff_t)maxLen - delta2] !=3D cur[maxLen]) - break; + UPDATE_maxLen distances[0] =3D maxLen; - distances[1] =3D delta2 - 1; + distances[1] =3D d2 - 1; offset =3D 2; if (maxLen =3D=3D lenLimit) { @@ -523,44 +578,51 @@ static UInt32 Bt3_MatchFinder_GetMatches(CMatchFinder= *p, UInt32 *distances) MOVE_POS_RET; } } + =20 GET_MATCHES_FOOTER(offset, maxLen) } =20 static UInt32 Bt4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distance= s) { - UInt32 hash2Value, hash3Value, delta2, delta3, maxLen, offset; + UInt32 h2, h3, d2, d3, maxLen, offset, pos; + UInt32 *hash; GET_MATCHES_HEADER(4) =20 HASH4_CALC; =20 - delta2 =3D p->pos - p->hash[ hash2Value]; - delta3 =3D p->pos - p->hash[kFix3HashSize + hash3Value]; - curMatch =3D p->hash[kFix4HashSize + hashValue]; - =20 - p->hash[ hash2Value] =3D - p->hash[kFix3HashSize + hash3Value] =3D - p->hash[kFix4HashSize + hashValue] =3D p->pos; + hash =3D p->hash; + pos =3D p->pos; + + d2 =3D pos - hash[ h2]; + d3 =3D pos - hash[kFix3HashSize + h3]; + + curMatch =3D hash[kFix4HashSize + hv]; =20 - maxLen =3D 1; + hash[ h2] =3D pos; + hash[kFix3HashSize + h3] =3D pos; + hash[kFix4HashSize + hv] =3D pos; + + maxLen =3D 0; offset =3D 0; - if (delta2 < p->cyclicBufferSize && *(cur - delta2) =3D=3D *cur) + =20 + if (d2 < p->cyclicBufferSize && *(cur - d2) =3D=3D *cur) { distances[0] =3D maxLen =3D 2; - distances[1] =3D delta2 - 1; + distances[1] =3D d2 - 1; offset =3D 2; } - if (delta2 !=3D delta3 && delta3 < p->cyclicBufferSize && *(cur - delta3= ) =3D=3D *cur) + =20 + if (d2 !=3D d3 && d3 < p->cyclicBufferSize && *(cur - d3) =3D=3D *cur) { maxLen =3D 3; - distances[offset + 1] =3D delta3 - 1; + distances[offset + 1] =3D d3 - 1; offset +=3D 2; - delta2 =3D delta3; + d2 =3D d3; } + =20 if (offset !=3D 0) { - for (; maxLen !=3D lenLimit; maxLen++) - if (cur[(ptrdiff_t)maxLen - delta2] !=3D cur[maxLen]) - break; + UPDATE_maxLen distances[offset - 2] =3D maxLen; if (maxLen =3D=3D lenLimit) { @@ -568,46 +630,131 @@ static UInt32 Bt4_MatchFinder_GetMatches(CMatchFinde= r *p, UInt32 *distances) MOVE_POS_RET; } } + =20 if (maxLen < 3) maxLen =3D 3; + =20 GET_MATCHES_FOOTER(offset, maxLen) } =20 +/* +static UInt32 Bt5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distance= s) +{ + UInt32 h2, h3, h4, d2, d3, d4, maxLen, offset, pos; + UInt32 *hash; + GET_MATCHES_HEADER(5) + + HASH5_CALC; + + hash =3D p->hash; + pos =3D p->pos; + + d2 =3D pos - hash[ h2]; + d3 =3D pos - hash[kFix3HashSize + h3]; + d4 =3D pos - hash[kFix4HashSize + h4]; + + curMatch =3D hash[kFix5HashSize + hv]; + + hash[ h2] =3D pos; + hash[kFix3HashSize + h3] =3D pos; + hash[kFix4HashSize + h4] =3D pos; + hash[kFix5HashSize + hv] =3D pos; + + maxLen =3D 0; + offset =3D 0; + + if (d2 < p->cyclicBufferSize && *(cur - d2) =3D=3D *cur) + { + distances[0] =3D maxLen =3D 2; + distances[1] =3D d2 - 1; + offset =3D 2; + if (*(cur - d2 + 2) =3D=3D cur[2]) + distances[0] =3D maxLen =3D 3; + else if (d3 < p->cyclicBufferSize && *(cur - d3) =3D=3D *cur) + { + distances[2] =3D maxLen =3D 3; + distances[3] =3D d3 - 1; + offset =3D 4; + d2 =3D d3; + } + } + else if (d3 < p->cyclicBufferSize && *(cur - d3) =3D=3D *cur) + { + distances[0] =3D maxLen =3D 3; + distances[1] =3D d3 - 1; + offset =3D 2; + d2 =3D d3; + } + =20 + if (d2 !=3D d4 && d4 < p->cyclicBufferSize + && *(cur - d4) =3D=3D *cur + && *(cur - d4 + 3) =3D=3D *(cur + 3)) + { + maxLen =3D 4; + distances[offset + 1] =3D d4 - 1; + offset +=3D 2; + d2 =3D d4; + } + =20 + if (offset !=3D 0) + { + UPDATE_maxLen + distances[offset - 2] =3D maxLen; + if (maxLen =3D=3D lenLimit) + { + SkipMatchesSpec(lenLimit, curMatch, MF_PARAMS(p)); + MOVE_POS_RET; + } + } + + if (maxLen < 4) + maxLen =3D 4; + =20 + GET_MATCHES_FOOTER(offset, maxLen) +} +*/ + static UInt32 Hc4_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distance= s) { - UInt32 hash2Value, hash3Value, delta2, delta3, maxLen, offset; + UInt32 h2, h3, d2, d3, maxLen, offset, pos; + UInt32 *hash; GET_MATCHES_HEADER(4) =20 HASH4_CALC; =20 - delta2 =3D p->pos - p->hash[ hash2Value]; - delta3 =3D p->pos - p->hash[kFix3HashSize + hash3Value]; - curMatch =3D p->hash[kFix4HashSize + hashValue]; + hash =3D p->hash; + pos =3D p->pos; + =20 + d2 =3D pos - hash[ h2]; + d3 =3D pos - hash[kFix3HashSize + h3]; + =20 + curMatch =3D hash[kFix4HashSize + hv]; =20 - p->hash[ hash2Value] =3D - p->hash[kFix3HashSize + hash3Value] =3D - p->hash[kFix4HashSize + hashValue] =3D p->pos; + hash[ h2] =3D pos; + hash[kFix3HashSize + h3] =3D pos; + hash[kFix4HashSize + hv] =3D pos; =20 - maxLen =3D 1; + maxLen =3D 0; offset =3D 0; - if (delta2 < p->cyclicBufferSize && *(cur - delta2) =3D=3D *cur) + + if (d2 < p->cyclicBufferSize && *(cur - d2) =3D=3D *cur) { distances[0] =3D maxLen =3D 2; - distances[1] =3D delta2 - 1; + distances[1] =3D d2 - 1; offset =3D 2; } - if (delta2 !=3D delta3 && delta3 < p->cyclicBufferSize && *(cur - delta3= ) =3D=3D *cur) + =20 + if (d2 !=3D d3 && d3 < p->cyclicBufferSize && *(cur - d3) =3D=3D *cur) { maxLen =3D 3; - distances[offset + 1] =3D delta3 - 1; + distances[offset + 1] =3D d3 - 1; offset +=3D 2; - delta2 =3D delta3; + d2 =3D d3; } + =20 if (offset !=3D 0) { - for (; maxLen !=3D lenLimit; maxLen++) - if (cur[(ptrdiff_t)maxLen - delta2] !=3D cur[maxLen]) - break; + UPDATE_maxLen distances[offset - 2] =3D maxLen; if (maxLen =3D=3D lenLimit) { @@ -615,22 +762,103 @@ static UInt32 Hc4_MatchFinder_GetMatches(CMatchFinde= r *p, UInt32 *distances) MOVE_POS_RET; } } + =20 if (maxLen < 3) maxLen =3D 3; + offset =3D (UInt32)(Hc_GetMatchesSpec(lenLimit, curMatch, MF_PARAMS(p), - distances + offset, maxLen) - (distances)); + distances + offset, maxLen) - (distances)); MOVE_POS_RET } =20 +/* +static UInt32 Hc5_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distance= s) +{ + UInt32 h2, h3, h4, d2, d3, d4, maxLen, offset, pos + UInt32 *hash; + GET_MATCHES_HEADER(5) + + HASH5_CALC; + + hash =3D p->hash; + pos =3D p->pos; + =20 + d2 =3D pos - hash[ h2]; + d3 =3D pos - hash[kFix3HashSize + h3]; + d4 =3D pos - hash[kFix4HashSize + h4]; + + curMatch =3D hash[kFix5HashSize + hv]; + + hash[ h2] =3D pos; + hash[kFix3HashSize + h3] =3D pos; + hash[kFix4HashSize + h4] =3D pos; + hash[kFix5HashSize + hv] =3D pos; + + maxLen =3D 0; + offset =3D 0; + + if (d2 < p->cyclicBufferSize && *(cur - d2) =3D=3D *cur) + { + distances[0] =3D maxLen =3D 2; + distances[1] =3D d2 - 1; + offset =3D 2; + if (*(cur - d2 + 2) =3D=3D cur[2]) + distances[0] =3D maxLen =3D 3; + else if (d3 < p->cyclicBufferSize && *(cur - d3) =3D=3D *cur) + { + distances[2] =3D maxLen =3D 3; + distances[3] =3D d3 - 1; + offset =3D 4; + d2 =3D d3; + } + } + else if (d3 < p->cyclicBufferSize && *(cur - d3) =3D=3D *cur) + { + distances[0] =3D maxLen =3D 3; + distances[1] =3D d3 - 1; + offset =3D 2; + d2 =3D d3; + } + =20 + if (d2 !=3D d4 && d4 < p->cyclicBufferSize + && *(cur - d4) =3D=3D *cur + && *(cur - d4 + 3) =3D=3D *(cur + 3)) + { + maxLen =3D 4; + distances[offset + 1] =3D d4 - 1; + offset +=3D 2; + d2 =3D d4; + } + =20 + if (offset !=3D 0) + { + UPDATE_maxLen + distances[offset - 2] =3D maxLen; + if (maxLen =3D=3D lenLimit) + { + p->son[p->cyclicBufferPos] =3D curMatch; + MOVE_POS_RET; + } + } + =20 + if (maxLen < 4) + maxLen =3D 4; + + offset =3D (UInt32)(Hc_GetMatchesSpec(lenLimit, curMatch, MF_PARAMS(p), + distances + offset, maxLen) - (distances)); + MOVE_POS_RET +} +*/ + UInt32 Hc3Zip_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances) { UInt32 offset; GET_MATCHES_HEADER(3) HASH_ZIP_CALC; - curMatch =3D p->hash[hashValue]; - p->hash[hashValue] =3D p->pos; + curMatch =3D p->hash[hv]; + p->hash[hv] =3D p->pos; offset =3D (UInt32)(Hc_GetMatchesSpec(lenLimit, curMatch, MF_PARAMS(p), - distances, 2) - (distances)); + distances, 2) - (distances)); MOVE_POS_RET } =20 @@ -640,8 +868,8 @@ static void Bt2_MatchFinder_Skip(CMatchFinder *p, UInt3= 2 num) { SKIP_HEADER(2) HASH2_CALC; - curMatch =3D p->hash[hashValue]; - p->hash[hashValue] =3D p->pos; + curMatch =3D p->hash[hv]; + p->hash[hv] =3D p->pos; SKIP_FOOTER } while (--num !=3D 0); @@ -653,8 +881,8 @@ void Bt3Zip_MatchFinder_Skip(CMatchFinder *p, UInt32 nu= m) { SKIP_HEADER(3) HASH_ZIP_CALC; - curMatch =3D p->hash[hashValue]; - p->hash[hashValue] =3D p->pos; + curMatch =3D p->hash[hv]; + p->hash[hv] =3D p->pos; SKIP_FOOTER } while (--num !=3D 0); @@ -664,12 +892,14 @@ static void Bt3_MatchFinder_Skip(CMatchFinder *p, UIn= t32 num) { do { - UInt32 hash2Value; + UInt32 h2; + UInt32 *hash; SKIP_HEADER(3) HASH3_CALC; - curMatch =3D p->hash[kFix3HashSize + hashValue]; - p->hash[hash2Value] =3D - p->hash[kFix3HashSize + hashValue] =3D p->pos; + hash =3D p->hash; + curMatch =3D hash[kFix3HashSize + hv]; + hash[h2] =3D + hash[kFix3HashSize + hv] =3D p->pos; SKIP_FOOTER } while (--num !=3D 0); @@ -679,34 +909,81 @@ static void Bt4_MatchFinder_Skip(CMatchFinder *p, UIn= t32 num) { do { - UInt32 hash2Value, hash3Value; + UInt32 h2, h3; + UInt32 *hash; SKIP_HEADER(4) HASH4_CALC; - curMatch =3D p->hash[kFix4HashSize + hashValue]; - p->hash[ hash2Value] =3D - p->hash[kFix3HashSize + hash3Value] =3D p->pos; - p->hash[kFix4HashSize + hashValue] =3D p->pos; + hash =3D p->hash; + curMatch =3D hash[kFix4HashSize + hv]; + hash[ h2] =3D + hash[kFix3HashSize + h3] =3D + hash[kFix4HashSize + hv] =3D p->pos; SKIP_FOOTER } while (--num !=3D 0); } =20 +/* +static void Bt5_MatchFinder_Skip(CMatchFinder *p, UInt32 num) +{ + do + { + UInt32 h2, h3, h4; + UInt32 *hash; + SKIP_HEADER(5) + HASH5_CALC; + hash =3D p->hash; + curMatch =3D hash[kFix5HashSize + hv]; + hash[ h2] =3D + hash[kFix3HashSize + h3] =3D + hash[kFix4HashSize + h4] =3D + hash[kFix5HashSize + hv] =3D p->pos; + SKIP_FOOTER + } + while (--num !=3D 0); +} +*/ + static void Hc4_MatchFinder_Skip(CMatchFinder *p, UInt32 num) { do { - UInt32 hash2Value, hash3Value; + UInt32 h2, h3; + UInt32 *hash; SKIP_HEADER(4) HASH4_CALC; - curMatch =3D p->hash[kFix4HashSize + hashValue]; - p->hash[ hash2Value] =3D - p->hash[kFix3HashSize + hash3Value] =3D - p->hash[kFix4HashSize + hashValue] =3D p->pos; + hash =3D p->hash; + curMatch =3D hash[kFix4HashSize + hv]; + hash[ h2] =3D + hash[kFix3HashSize + h3] =3D + hash[kFix4HashSize + hv] =3D p->pos; + p->son[p->cyclicBufferPos] =3D curMatch; + MOVE_POS + } + while (--num !=3D 0); +} + +/* +static void Hc5_MatchFinder_Skip(CMatchFinder *p, UInt32 num) +{ + do + { + UInt32 h2, h3, h4; + UInt32 *hash; + SKIP_HEADER(5) + HASH5_CALC; + hash =3D p->hash; + curMatch =3D p->hash[kFix5HashSize + hv]; + hash[ h2] =3D + hash[kFix3HashSize + h3] =3D + hash[kFix4HashSize + h4] =3D + hash[kFix5HashSize + hv] =3D p->pos; p->son[p->cyclicBufferPos] =3D curMatch; MOVE_POS } while (--num !=3D 0); } +*/ =20 void Hc3Zip_MatchFinder_Skip(CMatchFinder *p, UInt32 num) { @@ -714,8 +991,8 @@ void Hc3Zip_MatchFinder_Skip(CMatchFinder *p, UInt32 nu= m) { SKIP_HEADER(3) HASH_ZIP_CALC; - curMatch =3D p->hash[hashValue]; - p->hash[hashValue] =3D p->pos; + curMatch =3D p->hash[hv]; + p->hash[hv] =3D p->pos; p->son[p->cyclicBufferPos] =3D curMatch; MOVE_POS } @@ -725,13 +1002,22 @@ void Hc3Zip_MatchFinder_Skip(CMatchFinder *p, UInt32= num) void MatchFinder_CreateVTable(CMatchFinder *p, IMatchFinder *vTable) { vTable->Init =3D (Mf_Init_Func)MatchFinder_Init; - vTable->GetIndexByte =3D (Mf_GetIndexByte_Func)MatchFinder_GetIndexByte; vTable->GetNumAvailableBytes =3D (Mf_GetNumAvailableBytes_Func)MatchFind= er_GetNumAvailableBytes; vTable->GetPointerToCurrentPos =3D (Mf_GetPointerToCurrentPos_Func)Match= Finder_GetPointerToCurrentPos; if (!p->btMode) { - vTable->GetMatches =3D (Mf_GetMatches_Func)Hc4_MatchFinder_GetMatches; - vTable->Skip =3D (Mf_Skip_Func)Hc4_MatchFinder_Skip; + /* if (p->numHashBytes <=3D 4) */ + { + vTable->GetMatches =3D (Mf_GetMatches_Func)Hc4_MatchFinder_GetMatche= s; + vTable->Skip =3D (Mf_Skip_Func)Hc4_MatchFinder_Skip; + } + /* + else + { + vTable->GetMatches =3D (Mf_GetMatches_Func)Hc5_MatchFinder_GetMatche= s; + vTable->Skip =3D (Mf_Skip_Func)Hc5_MatchFinder_Skip; + } + */ } else if (p->numHashBytes =3D=3D 2) { @@ -743,9 +1029,16 @@ void MatchFinder_CreateVTable(CMatchFinder *p, IMatch= Finder *vTable) vTable->GetMatches =3D (Mf_GetMatches_Func)Bt3_MatchFinder_GetMatches; vTable->Skip =3D (Mf_Skip_Func)Bt3_MatchFinder_Skip; } - else + else /* if (p->numHashBytes =3D=3D 4) */ { vTable->GetMatches =3D (Mf_GetMatches_Func)Bt4_MatchFinder_GetMatches; vTable->Skip =3D (Mf_Skip_Func)Bt4_MatchFinder_Skip; } + /* + else + { + vTable->GetMatches =3D (Mf_GetMatches_Func)Bt5_MatchFinder_GetMatches; + vTable->Skip =3D (Mf_Skip_Func)Bt5_MatchFinder_Skip; + } + */ } diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFind.h b/BaseTools/Sou= rce/C/LzmaCompress/Sdk/C/LzFind.h index 423d67e..2ff6673 100644 --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFind.h +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFind.h @@ -1,10 +1,12 @@ /* LzFind.h -- Match finder for LZ algorithms -2008-10-04 : Igor Pavlov : Public domain */ +2015-10-15 : Igor Pavlov : Public domain */ =20 -#ifndef __LZFIND_H -#define __LZFIND_H +#ifndef __LZ_FIND_H +#define __LZ_FIND_H =20 -#include "Types.h" +#include "7zTypes.h" + +EXTERN_C_BEGIN =20 typedef UInt32 CLzRef; =20 @@ -19,6 +21,11 @@ typedef struct _CMatchFinder UInt32 cyclicBufferPos; UInt32 cyclicBufferSize; /* it must be =3D (historySize + 1) */ =20 + Byte streamEndWasReached; + Byte btMode; + Byte bigHash; + Byte directInput; + UInt32 matchMaxLen; CLzRef *hash; CLzRef *son; @@ -27,30 +34,30 @@ typedef struct _CMatchFinder =20 Byte *bufferBase; ISeqInStream *stream; - int streamEndWasReached; - + =20 UInt32 blockSize; UInt32 keepSizeBefore; UInt32 keepSizeAfter; =20 UInt32 numHashBytes; - int directInput; - int btMode; - /* int skipModeBits; */ - int bigHash; + size_t directInputRem; UInt32 historySize; UInt32 fixedHashSize; UInt32 hashSizeSum; - UInt32 numSons; SRes result; UInt32 crc[256]; + size_t numRefs; } CMatchFinder; =20 #define Inline_MatchFinder_GetPointerToCurrentPos(p) ((p)->buffer) -#define Inline_MatchFinder_GetIndexByte(p, index) ((p)->buffer[(Int32)(ind= ex)]) =20 #define Inline_MatchFinder_GetNumAvailableBytes(p) ((p)->streamPos - (p)->= pos) =20 +#define Inline_MatchFinder_IsFinishedOK(p) \ + ((p)->streamEndWasReached \ + && (p)->streamPos =3D=3D (p)->pos \ + && (!(p)->directInput || (p)->directInputRem =3D=3D 0)) + =20 int MatchFinder_NeedMove(CMatchFinder *p); Byte *MatchFinder_GetPointerToCurrentPos(CMatchFinder *p); void MatchFinder_MoveBlock(CMatchFinder *p); @@ -66,7 +73,7 @@ int MatchFinder_Create(CMatchFinder *p, UInt32 historySiz= e, UInt32 keepAddBufferBefore, UInt32 matchMaxLen, UInt32 keepAddBufferAf= ter, ISzAlloc *alloc); void MatchFinder_Free(CMatchFinder *p, ISzAlloc *alloc); -void MatchFinder_Normalize3(UInt32 subValue, CLzRef *items, UInt32 numItem= s); +void MatchFinder_Normalize3(UInt32 subValue, CLzRef *items, size_t numItem= s); void MatchFinder_ReduceOffsets(CMatchFinder *p, UInt32 subValue); =20 UInt32 * GetMatchesSpec1(UInt32 lenLimit, UInt32 curMatch, UInt32 pos, con= st Byte *buffer, CLzRef *son, @@ -80,7 +87,6 @@ Conditions: */ =20 typedef void (*Mf_Init_Func)(void *object); -typedef Byte (*Mf_GetIndexByte_Func)(void *object, Int32 index); typedef UInt32 (*Mf_GetNumAvailableBytes_Func)(void *object); typedef const Byte * (*Mf_GetPointerToCurrentPos_Func)(void *object); typedef UInt32 (*Mf_GetMatches_Func)(void *object, UInt32 *distances); @@ -89,7 +95,6 @@ typedef void (*Mf_Skip_Func)(void *object, UInt32); typedef struct _IMatchFinder { Mf_Init_Func Init; - Mf_GetIndexByte_Func GetIndexByte; Mf_GetNumAvailableBytes_Func GetNumAvailableBytes; Mf_GetPointerToCurrentPos_Func GetPointerToCurrentPos; Mf_GetMatches_Func GetMatches; @@ -98,10 +103,15 @@ typedef struct _IMatchFinder =20 void MatchFinder_CreateVTable(CMatchFinder *p, IMatchFinder *vTable); =20 +void MatchFinder_Init_2(CMatchFinder *p, int readData); void MatchFinder_Init(CMatchFinder *p); + UInt32 Bt3Zip_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances); UInt32 Hc3Zip_MatchFinder_GetMatches(CMatchFinder *p, UInt32 *distances); + void Bt3Zip_MatchFinder_Skip(CMatchFinder *p, UInt32 num); void Hc3Zip_MatchFinder_Skip(CMatchFinder *p, UInt32 num); =20 +EXTERN_C_END + #endif diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.c b/BaseTools/S= ource/C/LzmaCompress/Sdk/C/LzFindMt.c index 0ef1347..cb61e09 100644 --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.c +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.c @@ -1,11 +1,13 @@ /* LzFindMt.c -- multithreaded Match finder for LZ algorithms -2008-10-04 : Igor Pavlov : Public domain */ +2015-10-15 : Igor Pavlov : Public domain */ + +#include "Precomp.h" =20 #include "LzHash.h" =20 #include "LzFindMt.h" =20 -void MtSync_Construct(CMtSync *p) +static void MtSync_Construct(CMtSync *p) { p->wasCreated =3D False; p->csWasInitialized =3D False; @@ -18,7 +20,7 @@ void MtSync_Construct(CMtSync *p) Semaphore_Construct(&p->filledSemaphore); } =20 -void MtSync_GetNextBlock(CMtSync *p) +static void MtSync_GetNextBlock(CMtSync *p) { if (p->needStart) { @@ -46,7 +48,7 @@ void MtSync_GetNextBlock(CMtSync *p) =20 /* MtSync_StopWriting must be called if Writing was started */ =20 -void MtSync_StopWriting(CMtSync *p) +static void MtSync_StopWriting(CMtSync *p) { UInt32 myNumBlocks =3D p->numProcessedBlocks; if (!Thread_WasCreated(&p->thread) || p->needStart) @@ -69,7 +71,7 @@ void MtSync_StopWriting(CMtSync *p) p->needStart =3D True; } =20 -void MtSync_Destruct(CMtSync *p) +static void MtSync_Destruct(CMtSync *p) { if (Thread_WasCreated(&p->thread)) { @@ -97,7 +99,7 @@ void MtSync_Destruct(CMtSync *p) =20 #define RINOK_THREAD(x) { if ((x) !=3D 0) return SZ_ERROR_THREAD; } =20 -static SRes MtSync_Create2(CMtSync *p, unsigned (MY_STD_CALL *startAddress= )(void *), void *obj, UInt32 numBlocks) +static SRes MtSync_Create2(CMtSync *p, THREAD_FUNC_TYPE startAddress, void= *obj, UInt32 numBlocks) { if (p->wasCreated) return SZ_OK; @@ -119,7 +121,7 @@ static SRes MtSync_Create2(CMtSync *p, unsigned (MY_STD= _CALL *startAddress)(void return SZ_OK; } =20 -static SRes MtSync_Create(CMtSync *p, unsigned (MY_STD_CALL *startAddress)= (void *), void *obj, UInt32 numBlocks) +static SRes MtSync_Create(CMtSync *p, THREAD_FUNC_TYPE startAddress, void = *obj, UInt32 numBlocks) { SRes res =3D MtSync_Create2(p, startAddress, obj, numBlocks); if (res !=3D SZ_OK) @@ -132,20 +134,20 @@ void MtSync_Init(CMtSync *p) { p->needStart =3D True;= } #define kMtMaxValForNormalize 0xFFFFFFFF =20 #define DEF_GetHeads2(name, v, action) \ -static void GetHeads ## name(const Byte *p, UInt32 pos, \ -UInt32 *hash, UInt32 hashMask, UInt32 *heads, UInt32 numHeads, const UInt3= 2 *crc) \ -{ action; for (; numHeads !=3D 0; numHeads--) { \ -const UInt32 value =3D (v); p++; *heads++ =3D pos - hash[value]; hash[valu= e] =3D pos++; } } + static void GetHeads ## name(const Byte *p, UInt32 pos, \ + UInt32 *hash, UInt32 hashMask, UInt32 *heads, UInt32 numHeads, const= UInt32 *crc) \ + { action; for (; numHeads !=3D 0; numHeads--) { \ + const UInt32 value =3D (v); p++; *heads++ =3D pos - hash[value]; has= h[value] =3D pos++; } } =20 #define DEF_GetHeads(name, v) DEF_GetHeads2(name, v, ;) =20 -DEF_GetHeads2(2, (p[0] | ((UInt32)p[1] << 8)), hashMask =3D hashMask; crc= =3D crc; ) +DEF_GetHeads2(2, (p[0] | ((UInt32)p[1] << 8)), UNUSED_VAR(hashMask); UNUS= ED_VAR(crc); ) DEF_GetHeads(3, (crc[p[0]] ^ p[1] ^ ((UInt32)p[2] << 8)) & hashMask) DEF_GetHeads(4, (crc[p[0]] ^ p[1] ^ ((UInt32)p[2] << 8) ^ (crc[p[3]] << 5= )) & hashMask) DEF_GetHeads(4b, (crc[p[0]] ^ p[1] ^ ((UInt32)p[2] << 8) ^ ((UInt32)p[3] <= < 16)) & hashMask) -DEF_GetHeads(5, (crc[p[0]] ^ p[1] ^ ((UInt32)p[2] << 8) ^ (crc[p[3]] << 5= ) ^ (crc[p[4]] << 3)) & hashMask) +/* DEF_GetHeads(5, (crc[p[0]] ^ p[1] ^ ((UInt32)p[2] << 8) ^ (crc[p[3]] <= < 5) ^ (crc[p[4]] << 3)) & hashMask) */ =20 -void HashThreadFunc(CMatchFinderMt *mt) +static void HashThreadFunc(CMatchFinderMt *mt) { CMtSync *p =3D &mt->hashSync; for (;;) @@ -171,12 +173,12 @@ void HashThreadFunc(CMatchFinderMt *mt) CriticalSection_Enter(&mt->btSync.cs); CriticalSection_Enter(&mt->hashSync.cs); { - const Byte *beforePtr =3D MatchFinder_GetPointerToCurrentPos(m= f); - const Byte *afterPtr; + const Byte *beforePtr =3D Inline_MatchFinder_GetPointerToCurre= ntPos(mf); + ptrdiff_t offset; MatchFinder_MoveBlock(mf); - afterPtr =3D MatchFinder_GetPointerToCurrentPos(mf); - mt->pointerToCurPos -=3D beforePtr - afterPtr; - mt->buffer -=3D beforePtr - afterPtr; + offset =3D beforePtr - Inline_MatchFinder_GetPointerToCurrentP= os(mf); + mt->pointerToCurPos -=3D offset; + mt->buffer -=3D offset; } CriticalSection_Leave(&mt->btSync.cs); CriticalSection_Leave(&mt->hashSync.cs); @@ -190,7 +192,7 @@ void HashThreadFunc(CMatchFinderMt *mt) { UInt32 subValue =3D (mf->pos - mf->historySize - 1); MatchFinder_ReduceOffsets(mf, subValue); - MatchFinder_Normalize3(subValue, mf->hash + mf->fixedHashSize, m= f->hashMask + 1); + MatchFinder_Normalize3(subValue, mf->hash + mf->fixedHashSize, (= size_t)mf->hashMask + 1); } { UInt32 *heads =3D mt->hashBuf + ((numProcessedBlocks++) & kMtHas= hNumBlocksMask) * kMtHashBlockSize; @@ -215,7 +217,7 @@ void HashThreadFunc(CMatchFinderMt *mt) } } =20 -void MatchFinderMt_GetNextBlock_Hash(CMatchFinderMt *p) +static void MatchFinderMt_GetNextBlock_Hash(CMatchFinderMt *p) { MtSync_GetNextBlock(&p->hashSync); p->hashBufPosLimit =3D p->hashBufPos =3D ((p->hashSync.numProcessedBlock= s - 1) & kMtHashNumBlocksMask) * kMtHashBlockSize; @@ -231,7 +233,7 @@ void MatchFinderMt_GetNextBlock_Hash(CMatchFinderMt *p) =20 #define NO_INLINE MY_FAST_CALL =20 -Int32 NO_INLINE GetMatchesSpecN(UInt32 lenLimit, UInt32 pos, const Byte *c= ur, CLzRef *son, +static Int32 NO_INLINE GetMatchesSpecN(UInt32 lenLimit, UInt32 pos, const = Byte *cur, CLzRef *son, UInt32 _cyclicBufferPos, UInt32 _cyclicBufferSize, UInt32 _cutValue, UInt32 *_distances, UInt32 _maxLen, const UInt32 *hash, Int32 limit, U= Int32 size, UInt32 *posRes) { @@ -308,12 +310,14 @@ Int32 NO_INLINE GetMatchesSpecN(UInt32 lenLimit, UInt= 32 pos, const Byte *cur, CL =20 #endif =20 -void BtGetMatches(CMatchFinderMt *p, UInt32 *distances) +static void BtGetMatches(CMatchFinderMt *p, UInt32 *distances) { UInt32 numProcessed =3D 0; UInt32 curPos =3D 2; UInt32 limit =3D kMtBtBlockSize - (p->matchMaxLen * 2); + =20 distances[1] =3D p->hashNumAvail; + =20 while (curPos < limit) { if (p->hashBufPos =3D=3D p->hashBufPosLimit) @@ -322,9 +326,11 @@ void BtGetMatches(CMatchFinderMt *p, UInt32 *distances= ) distances[1] =3D numProcessed + p->hashNumAvail; if (p->hashNumAvail >=3D p->numHashBytes) continue; + distances[0] =3D curPos + p->hashNumAvail; + distances +=3D curPos; for (; p->hashNumAvail !=3D 0; p->hashNumAvail--) - distances[curPos++] =3D 0; - break; + *distances++ =3D 0; + return; } { UInt32 size =3D p->hashBufPosLimit - p->hashBufPos; @@ -341,13 +347,14 @@ void BtGetMatches(CMatchFinderMt *p, UInt32 *distance= s) if (size2 < size) size =3D size2; } + =20 #ifndef MFMT_GM_INLINE while (curPos < limit && size-- !=3D 0) { UInt32 *startDistances =3D distances + curPos; UInt32 num =3D (UInt32)(GetMatchesSpec1(lenLimit, pos - p->hashBuf= [p->hashBufPos++], - pos, p->buffer, p->son, cyclicBufferPos, p->cyclicBufferSize, p-= >cutValue, - startDistances + 1, p->numHashBytes - 1) - startDistances); + pos, p->buffer, p->son, cyclicBufferPos, p->cyclicBufferSize, = p->cutValue, + startDistances + 1, p->numHashBytes - 1) - startDistances); *startDistances =3D num - 1; curPos +=3D num; cyclicBufferPos++; @@ -358,7 +365,7 @@ void BtGetMatches(CMatchFinderMt *p, UInt32 *distances) { UInt32 posRes; curPos =3D limit - GetMatchesSpecN(lenLimit, pos, p->buffer, p->so= n, cyclicBufferPos, p->cyclicBufferSize, p->cutValue, - distances + curPos, p->numHashBytes - 1, p->hashBuf + p->hashBuf= Pos, (Int32)(limit - curPos) , size, &posRes); + distances + curPos, p->numHashBytes - 1, p->hashBuf + p->hashB= ufPos, (Int32)(limit - curPos), size, &posRes); p->hashBufPos +=3D posRes - pos; cyclicBufferPos +=3D posRes - pos; p->buffer +=3D posRes - pos; @@ -374,10 +381,11 @@ void BtGetMatches(CMatchFinderMt *p, UInt32 *distance= s) p->cyclicBufferPos =3D cyclicBufferPos; } } + =20 distances[0] =3D curPos; } =20 -void BtFillBlock(CMatchFinderMt *p, UInt32 globalBlockIndex) +static void BtFillBlock(CMatchFinderMt *p, UInt32 globalBlockIndex) { CMtSync *sync =3D &p->hashSync; if (!sync->needStart) @@ -391,7 +399,7 @@ void BtFillBlock(CMatchFinderMt *p, UInt32 globalBlockI= ndex) if (p->pos > kMtMaxValForNormalize - kMtBtBlockSize) { UInt32 subValue =3D p->pos - p->cyclicBufferSize; - MatchFinder_Normalize3(subValue, p->son, p->cyclicBufferSize * 2); + MatchFinder_Normalize3(subValue, p->son, (size_t)p->cyclicBufferSize *= 2); p->pos -=3D subValue; } =20 @@ -430,15 +438,15 @@ void BtThreadFunc(CMatchFinderMt *mt) =20 void MatchFinderMt_Construct(CMatchFinderMt *p) { - p->hashBuf =3D 0; + p->hashBuf =3D NULL; MtSync_Construct(&p->hashSync); MtSync_Construct(&p->btSync); } =20 -void MatchFinderMt_FreeMem(CMatchFinderMt *p, ISzAlloc *alloc) +static void MatchFinderMt_FreeMem(CMatchFinderMt *p, ISzAlloc *alloc) { alloc->Free(alloc, p->hashBuf); - p->hashBuf =3D 0; + p->hashBuf =3D NULL; } =20 void MatchFinderMt_Destruct(CMatchFinderMt *p, ISzAlloc *alloc) @@ -451,14 +459,15 @@ void MatchFinderMt_Destruct(CMatchFinderMt *p, ISzAll= oc *alloc) #define kHashBufferSize (kMtHashBlockSize * kMtHashNumBlocks) #define kBtBufferSize (kMtBtBlockSize * kMtBtNumBlocks) =20 -static unsigned MY_STD_CALL HashThreadFunc2(void *p) { HashThreadFunc((CMa= tchFinderMt *)p); return 0; } -static unsigned MY_STD_CALL BtThreadFunc2(void *p) +static THREAD_FUNC_RET_TYPE THREAD_FUNC_CALL_TYPE HashThreadFunc2(void *p)= { HashThreadFunc((CMatchFinderMt *)p); return 0; } +static THREAD_FUNC_RET_TYPE THREAD_FUNC_CALL_TYPE BtThreadFunc2(void *p) { Byte allocaDummy[0x180]; - int i =3D 0; + unsigned i =3D 0; for (i =3D 0; i < 16; i++) - allocaDummy[i] =3D (Byte)i; - BtThreadFunc((CMatchFinderMt *)p); + allocaDummy[i] =3D (Byte)0; + if (allocaDummy[0] =3D=3D 0) + BtThreadFunc((CMatchFinderMt *)p); return 0; } =20 @@ -469,10 +478,10 @@ SRes MatchFinderMt_Create(CMatchFinderMt *p, UInt32 h= istorySize, UInt32 keepAddB p->historySize =3D historySize; if (kMtBtBlockSize <=3D matchMaxLen * 4) return SZ_ERROR_PARAM; - if (p->hashBuf =3D=3D 0) + if (!p->hashBuf) { p->hashBuf =3D (UInt32 *)alloc->Alloc(alloc, (kHashBufferSize + kBtBuf= ferSize) * sizeof(UInt32)); - if (p->hashBuf =3D=3D 0) + if (!p->hashBuf) return SZ_ERROR_MEM; p->btBuf =3D p->hashBuf + kHashBufferSize; } @@ -492,8 +501,11 @@ void MatchFinderMt_Init(CMatchFinderMt *p) CMatchFinder *mf =3D p->MatchFinder; p->btBufPos =3D p->btBufPosLimit =3D 0; p->hashBufPos =3D p->hashBufPosLimit =3D 0; - MatchFinder_Init(mf); - p->pointerToCurPos =3D MatchFinder_GetPointerToCurrentPos(mf); + + /* Init without data reading. We don't want to read data in this thread = */ + MatchFinder_Init_2(mf, False); + =20 + p->pointerToCurPos =3D Inline_MatchFinder_GetPointerToCurrentPos(mf); p->btNumAvailBytes =3D 0; p->lzPos =3D p->historySize + 1; =20 @@ -518,13 +530,13 @@ void MatchFinderMt_ReleaseStream(CMatchFinderMt *p) /* p->MatchFinder->ReleaseStream(); */ } =20 -void MatchFinderMt_Normalize(CMatchFinderMt *p) +static void MatchFinderMt_Normalize(CMatchFinderMt *p) { MatchFinder_Normalize3(p->lzPos - p->historySize - 1, p->hash, p->fixedH= ashSize); p->lzPos =3D p->historySize + 1; } =20 -void MatchFinderMt_GetNextBlock_Bt(CMatchFinderMt *p) +static void MatchFinderMt_GetNextBlock_Bt(CMatchFinderMt *p) { UInt32 blockIndex; MtSync_GetNextBlock(&p->btSync); @@ -536,34 +548,29 @@ void MatchFinderMt_GetNextBlock_Bt(CMatchFinderMt *p) MatchFinderMt_Normalize(p); } =20 -const Byte * MatchFinderMt_GetPointerToCurrentPos(CMatchFinderMt *p) +static const Byte * MatchFinderMt_GetPointerToCurrentPos(CMatchFinderMt *p= ) { return p->pointerToCurPos; } =20 #define GET_NEXT_BLOCK_IF_REQUIRED if (p->btBufPos =3D=3D p->btBufPosLimit= ) MatchFinderMt_GetNextBlock_Bt(p); =20 -UInt32 MatchFinderMt_GetNumAvailableBytes(CMatchFinderMt *p) +static UInt32 MatchFinderMt_GetNumAvailableBytes(CMatchFinderMt *p) { GET_NEXT_BLOCK_IF_REQUIRED; return p->btNumAvailBytes; } =20 -Byte MatchFinderMt_GetIndexByte(CMatchFinderMt *p, Int32 index) +static UInt32 * MixMatches2(CMatchFinderMt *p, UInt32 matchMinPos, UInt32 = *distances) { - return p->pointerToCurPos[index]; -} - -UInt32 * MixMatches2(CMatchFinderMt *p, UInt32 matchMinPos, UInt32 *distan= ces) -{ - UInt32 hash2Value, curMatch2; + UInt32 h2, curMatch2; UInt32 *hash =3D p->hash; const Byte *cur =3D p->pointerToCurPos; UInt32 lzPos =3D p->lzPos; MT_HASH2_CALC =20 - curMatch2 =3D hash[hash2Value]; - hash[hash2Value] =3D lzPos; + curMatch2 =3D hash[h2]; + hash[h2] =3D lzPos; =20 if (curMatch2 >=3D matchMinPos) if (cur[(ptrdiff_t)curMatch2 - lzPos] =3D=3D cur[0]) @@ -571,23 +578,23 @@ UInt32 * MixMatches2(CMatchFinderMt *p, UInt32 matchM= inPos, UInt32 *distances) *distances++ =3D 2; *distances++ =3D lzPos - curMatch2 - 1; } + =20 return distances; } =20 -UInt32 * MixMatches3(CMatchFinderMt *p, UInt32 matchMinPos, UInt32 *distan= ces) +static UInt32 * MixMatches3(CMatchFinderMt *p, UInt32 matchMinPos, UInt32 = *distances) { - UInt32 hash2Value, hash3Value, curMatch2, curMatch3; + UInt32 h2, h3, curMatch2, curMatch3; UInt32 *hash =3D p->hash; const Byte *cur =3D p->pointerToCurPos; UInt32 lzPos =3D p->lzPos; MT_HASH3_CALC =20 - curMatch2 =3D hash[ hash2Value]; - curMatch3 =3D hash[kFix3HashSize + hash3Value]; + curMatch2 =3D hash[ h2]; + curMatch3 =3D hash[kFix3HashSize + h3]; =20 - hash[ hash2Value] =3D - hash[kFix3HashSize + hash3Value] =3D - lzPos; + hash[ h2] =3D lzPos; + hash[kFix3HashSize + h3] =3D lzPos; =20 if (curMatch2 >=3D matchMinPos && cur[(ptrdiff_t)curMatch2 - lzPos] =3D= =3D cur[0]) { @@ -600,43 +607,45 @@ UInt32 * MixMatches3(CMatchFinderMt *p, UInt32 matchM= inPos, UInt32 *distances) distances[0] =3D 2; distances +=3D 2; } + =20 if (curMatch3 >=3D matchMinPos && cur[(ptrdiff_t)curMatch3 - lzPos] =3D= =3D cur[0]) { *distances++ =3D 3; *distances++ =3D lzPos - curMatch3 - 1; } + =20 return distances; } =20 /* -UInt32 *MixMatches4(CMatchFinderMt *p, UInt32 matchMinPos, UInt32 *distanc= es) +static UInt32 *MixMatches4(CMatchFinderMt *p, UInt32 matchMinPos, UInt32 *= distances) { - UInt32 hash2Value, hash3Value, hash4Value, curMatch2, curMatch3, curMatc= h4; + UInt32 h2, h3, h4, curMatch2, curMatch3, curMatch4; UInt32 *hash =3D p->hash; const Byte *cur =3D p->pointerToCurPos; UInt32 lzPos =3D p->lzPos; MT_HASH4_CALC =20 - curMatch2 =3D hash[ hash2Value]; - curMatch3 =3D hash[kFix3HashSize + hash3Value]; - curMatch4 =3D hash[kFix4HashSize + hash4Value]; + curMatch2 =3D hash[ h2]; + curMatch3 =3D hash[kFix3HashSize + h3]; + curMatch4 =3D hash[kFix4HashSize + h4]; =20 - hash[ hash2Value] =3D - hash[kFix3HashSize + hash3Value] =3D - hash[kFix4HashSize + hash4Value] =3D - lzPos; + hash[ h2] =3D lzPos; + hash[kFix3HashSize + h3] =3D lzPos; + hash[kFix4HashSize + h4] =3D lzPos; =20 if (curMatch2 >=3D matchMinPos && cur[(ptrdiff_t)curMatch2 - lzPos] =3D= =3D cur[0]) { distances[1] =3D lzPos - curMatch2 - 1; if (cur[(ptrdiff_t)curMatch2 - lzPos + 2] =3D=3D cur[2]) { - distances[0] =3D (cur[(ptrdiff_t)curMatch2 - lzPos + 3] =3D=3D cur[= 3]) ? 4 : 3; + distances[0] =3D (cur[(ptrdiff_t)curMatch2 - lzPos + 3] =3D=3D cur[3= ]) ? 4 : 3; return distances + 2; } distances[0] =3D 2; distances +=3D 2; } + =20 if (curMatch3 >=3D matchMinPos && cur[(ptrdiff_t)curMatch3 - lzPos] =3D= =3D cur[0]) { distances[1] =3D lzPos - curMatch3 - 1; @@ -658,13 +667,14 @@ UInt32 *MixMatches4(CMatchFinderMt *p, UInt32 matchMi= nPos, UInt32 *distances) *distances++ =3D 4; *distances++ =3D lzPos - curMatch4 - 1; } + =20 return distances; } */ =20 #define INCREASE_LZ_POS p->lzPos++; p->pointerToCurPos++; =20 -UInt32 MatchFinderMt2_GetMatches(CMatchFinderMt *p, UInt32 *distances) +static UInt32 MatchFinderMt2_GetMatches(CMatchFinderMt *p, UInt32 *distanc= es) { const UInt32 *btBuf =3D p->btBuf + p->btBufPos; UInt32 len =3D *btBuf++; @@ -682,7 +692,7 @@ UInt32 MatchFinderMt2_GetMatches(CMatchFinderMt *p, UIn= t32 *distances) return len; } =20 -UInt32 MatchFinderMt_GetMatches(CMatchFinderMt *p, UInt32 *distances) +static UInt32 MatchFinderMt_GetMatches(CMatchFinderMt *p, UInt32 *distance= s) { const UInt32 *btBuf =3D p->btBuf + p->btBufPos; UInt32 len =3D *btBuf++; @@ -690,6 +700,7 @@ UInt32 MatchFinderMt_GetMatches(CMatchFinderMt *p, UInt= 32 *distances) =20 if (len =3D=3D 0) { + /* change for bt5 ! */ if (p->btNumAvailBytes-- >=3D 4) len =3D (UInt32)(p->MixMatchesFunc(p, p->lzPos - p->historySize, dis= tances) - (distances)); } @@ -705,64 +716,64 @@ UInt32 MatchFinderMt_GetMatches(CMatchFinderMt *p, UI= nt32 *distances) *distances2++ =3D *btBuf++; } while ((len -=3D 2) !=3D 0); - len =3D (UInt32)(distances2 - (distances)); + len =3D (UInt32)(distances2 - (distances)); } INCREASE_LZ_POS return len; } =20 -#define SKIP_HEADER2 do { GET_NEXT_BLOCK_IF_REQUIRED -#define SKIP_HEADER(n) SKIP_HEADER2 if (p->btNumAvailBytes-- >=3D (n)) { c= onst Byte *cur =3D p->pointerToCurPos; UInt32 *hash =3D p->hash; -#define SKIP_FOOTER } INCREASE_LZ_POS p->btBufPos +=3D p->btBuf[p->btBufPo= s] + 1; } while (--num !=3D 0); +#define SKIP_HEADER2_MT do { GET_NEXT_BLOCK_IF_REQUIRED +#define SKIP_HEADER_MT(n) SKIP_HEADER2_MT if (p->btNumAvailBytes-- >=3D (n= )) { const Byte *cur =3D p->pointerToCurPos; UInt32 *hash =3D p->hash; +#define SKIP_FOOTER_MT } INCREASE_LZ_POS p->btBufPos +=3D p->btBuf[p->btBu= fPos] + 1; } while (--num !=3D 0); =20 -void MatchFinderMt0_Skip(CMatchFinderMt *p, UInt32 num) +static void MatchFinderMt0_Skip(CMatchFinderMt *p, UInt32 num) { - SKIP_HEADER2 { p->btNumAvailBytes--; - SKIP_FOOTER + SKIP_HEADER2_MT { p->btNumAvailBytes--; + SKIP_FOOTER_MT } =20 -void MatchFinderMt2_Skip(CMatchFinderMt *p, UInt32 num) +static void MatchFinderMt2_Skip(CMatchFinderMt *p, UInt32 num) { - SKIP_HEADER(2) - UInt32 hash2Value; + SKIP_HEADER_MT(2) + UInt32 h2; MT_HASH2_CALC - hash[hash2Value] =3D p->lzPos; - SKIP_FOOTER + hash[h2] =3D p->lzPos; + SKIP_FOOTER_MT } =20 -void MatchFinderMt3_Skip(CMatchFinderMt *p, UInt32 num) +static void MatchFinderMt3_Skip(CMatchFinderMt *p, UInt32 num) { - SKIP_HEADER(3) - UInt32 hash2Value, hash3Value; + SKIP_HEADER_MT(3) + UInt32 h2, h3; MT_HASH3_CALC - hash[kFix3HashSize + hash3Value] =3D - hash[ hash2Value] =3D + hash[kFix3HashSize + h3] =3D + hash[ h2] =3D p->lzPos; - SKIP_FOOTER + SKIP_FOOTER_MT } =20 /* -void MatchFinderMt4_Skip(CMatchFinderMt *p, UInt32 num) +static void MatchFinderMt4_Skip(CMatchFinderMt *p, UInt32 num) { - SKIP_HEADER(4) - UInt32 hash2Value, hash3Value, hash4Value; + SKIP_HEADER_MT(4) + UInt32 h2, h3, h4; MT_HASH4_CALC - hash[kFix4HashSize + hash4Value] =3D - hash[kFix3HashSize + hash3Value] =3D - hash[ hash2Value] =3D + hash[kFix4HashSize + h4] =3D + hash[kFix3HashSize + h3] =3D + hash[ h2] =3D p->lzPos; - SKIP_FOOTER + SKIP_FOOTER_MT } */ =20 void MatchFinderMt_CreateVTable(CMatchFinderMt *p, IMatchFinder *vTable) { vTable->Init =3D (Mf_Init_Func)MatchFinderMt_Init; - vTable->GetIndexByte =3D (Mf_GetIndexByte_Func)MatchFinderMt_GetIndexByt= e; vTable->GetNumAvailableBytes =3D (Mf_GetNumAvailableBytes_Func)MatchFind= erMt_GetNumAvailableBytes; vTable->GetPointerToCurrentPos =3D (Mf_GetPointerToCurrentPos_Func)Match= FinderMt_GetPointerToCurrentPos; vTable->GetMatches =3D (Mf_GetMatches_Func)MatchFinderMt_GetMatches; - switch(p->MatchFinder->numHashBytes) + =20 + switch (p->MatchFinder->numHashBytes) { case 2: p->GetHeadsFunc =3D GetHeads2; @@ -778,7 +789,6 @@ void MatchFinderMt_CreateVTable(CMatchFinderMt *p, IMat= chFinder *vTable) default: /* case 4: */ p->GetHeadsFunc =3D p->MatchFinder->bigHash ? GetHeads4b : GetHeads4= ; - /* p->GetHeadsFunc =3D GetHeads4; */ p->MixMatchesFunc =3D (Mf_Mix_Matches)MixMatches3; vTable->Skip =3D (Mf_Skip_Func)MatchFinderMt3_Skip; break; diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.h b/BaseTools/S= ource/C/LzmaCompress/Sdk/C/LzFindMt.h index b7ead2d..46b6924 100644 --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.h +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzFindMt.h @@ -1,11 +1,13 @@ /* LzFindMt.h -- multithreaded Match finder for LZ algorithms -2008-10-04 : Igor Pavlov : Public domain */ +2015-05-03 : Igor Pavlov : Public domain */ =20 -#ifndef __LZFINDMT_H -#define __LZFINDMT_H +#ifndef __LZ_FIND_MT_H +#define __LZ_FIND_MT_H =20 -#include "Threads.h" #include "LzFind.h" +#include "Threads.h" + +EXTERN_C_BEGIN =20 #define kMtHashBlockSize (1 << 13) #define kMtHashNumBlocks (1 << 3) @@ -73,7 +75,7 @@ typedef struct _CMatchFinderMt UInt32 matchMaxLen; UInt32 numHashBytes; UInt32 pos; - Byte *buffer; + const Byte *buffer; UInt32 cyclicBufferPos; UInt32 cyclicBufferSize; /* it must be historySize + 1 */ UInt32 cutValue; @@ -94,4 +96,6 @@ SRes MatchFinderMt_Create(CMatchFinderMt *p, UInt32 histo= rySize, UInt32 keepAddB void MatchFinderMt_CreateVTable(CMatchFinderMt *p, IMatchFinder *vTable); void MatchFinderMt_ReleaseStream(CMatchFinderMt *p); =20 +EXTERN_C_END + #endif diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzHash.h b/BaseTools/Sou= rce/C/LzmaCompress/Sdk/C/LzHash.h index c923417..2191444 100644 --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzHash.h +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzHash.h @@ -1,8 +1,8 @@ /* LzHash.h -- HASH functions for LZ algorithms -2008-10-04 : Igor Pavlov : Public domain */ +2015-04-12 : Igor Pavlov : Public domain */ =20 -#ifndef __LZHASH_H -#define __LZHASH_H +#ifndef __LZ_HASH_H +#define __LZ_HASH_H =20 #define kHash2Size (1 << 10) #define kHash3Size (1 << 16) @@ -12,43 +12,46 @@ #define kFix4HashSize (kHash2Size + kHash3Size) #define kFix5HashSize (kHash2Size + kHash3Size + kHash4Size) =20 -#define HASH2_CALC hashValue =3D cur[0] | ((UInt32)cur[1] << 8); +#define HASH2_CALC hv =3D cur[0] | ((UInt32)cur[1] << 8); =20 #define HASH3_CALC { \ UInt32 temp =3D p->crc[cur[0]] ^ cur[1]; \ - hash2Value =3D temp & (kHash2Size - 1); \ - hashValue =3D (temp ^ ((UInt32)cur[2] << 8)) & p->hashMask; } + h2 =3D temp & (kHash2Size - 1); \ + hv =3D (temp ^ ((UInt32)cur[2] << 8)) & p->hashMask; } =20 #define HASH4_CALC { \ UInt32 temp =3D p->crc[cur[0]] ^ cur[1]; \ - hash2Value =3D temp & (kHash2Size - 1); \ - hash3Value =3D (temp ^ ((UInt32)cur[2] << 8)) & (kHash3Size - 1); \ - hashValue =3D (temp ^ ((UInt32)cur[2] << 8) ^ (p->crc[cur[3]] << 5)) & p= ->hashMask; } + h2 =3D temp & (kHash2Size - 1); \ + temp ^=3D ((UInt32)cur[2] << 8); \ + h3 =3D temp & (kHash3Size - 1); \ + hv =3D (temp ^ (p->crc[cur[3]] << 5)) & p->hashMask; } =20 #define HASH5_CALC { \ UInt32 temp =3D p->crc[cur[0]] ^ cur[1]; \ - hash2Value =3D temp & (kHash2Size - 1); \ - hash3Value =3D (temp ^ ((UInt32)cur[2] << 8)) & (kHash3Size - 1); \ - hash4Value =3D (temp ^ ((UInt32)cur[2] << 8) ^ (p->crc[cur[3]] << 5)); \ - hashValue =3D (hash4Value ^ (p->crc[cur[4]] << 3)) & p->hashMask; \ - hash4Value &=3D (kHash4Size - 1); } + h2 =3D temp & (kHash2Size - 1); \ + temp ^=3D ((UInt32)cur[2] << 8); \ + h3 =3D temp & (kHash3Size - 1); \ + temp ^=3D (p->crc[cur[3]] << 5); \ + h4 =3D temp & (kHash4Size - 1); \ + hv =3D (temp ^ (p->crc[cur[4]] << 3)) & p->hashMask; } =20 -/* #define HASH_ZIP_CALC hashValue =3D ((cur[0] | ((UInt32)cur[1] << 8)) ^= p->crc[cur[2]]) & 0xFFFF; */ -#define HASH_ZIP_CALC hashValue =3D ((cur[2] | ((UInt32)cur[0] << 8)) ^ p-= >crc[cur[1]]) & 0xFFFF; +/* #define HASH_ZIP_CALC hv =3D ((cur[0] | ((UInt32)cur[1] << 8)) ^ p->crc= [cur[2]]) & 0xFFFF; */ +#define HASH_ZIP_CALC hv =3D ((cur[2] | ((UInt32)cur[0] << 8)) ^ p->crc[cu= r[1]]) & 0xFFFF; =20 =20 #define MT_HASH2_CALC \ - hash2Value =3D (p->crc[cur[0]] ^ cur[1]) & (kHash2Size - 1); + h2 =3D (p->crc[cur[0]] ^ cur[1]) & (kHash2Size - 1); =20 #define MT_HASH3_CALC { \ UInt32 temp =3D p->crc[cur[0]] ^ cur[1]; \ - hash2Value =3D temp & (kHash2Size - 1); \ - hash3Value =3D (temp ^ ((UInt32)cur[2] << 8)) & (kHash3Size - 1); } + h2 =3D temp & (kHash2Size - 1); \ + h3 =3D (temp ^ ((UInt32)cur[2] << 8)) & (kHash3Size - 1); } =20 #define MT_HASH4_CALC { \ UInt32 temp =3D p->crc[cur[0]] ^ cur[1]; \ - hash2Value =3D temp & (kHash2Size - 1); \ - hash3Value =3D (temp ^ ((UInt32)cur[2] << 8)) & (kHash3Size - 1); \ - hash4Value =3D (temp ^ ((UInt32)cur[2] << 8) ^ (p->crc[cur[3]] << 5)) & = (kHash4Size - 1); } + h2 =3D temp & (kHash2Size - 1); \ + temp ^=3D ((UInt32)cur[2] << 8); \ + h3 =3D temp & (kHash3Size - 1); \ + h4 =3D (temp ^ (p->crc[cur[3]] << 5)) & (kHash4Size - 1); } =20 #endif diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaDec.c b/BaseTools/So= urce/C/LzmaCompress/Sdk/C/LzmaDec.c index 87bea86..64f1164 100644 --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaDec.c +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaDec.c @@ -1,5 +1,7 @@ /* LzmaDec.c -- LZMA Decoder -2008-11-06 : Igor Pavlov : Public domain */ +2016-05-16 : Igor Pavlov : Public domain */ + +#include "Precomp.h" =20 #include "LzmaDec.h" =20 @@ -44,6 +46,13 @@ i -=3D 0x40; } #endif =20 +#define NORMAL_LITER_DEC GET_BIT(prob + symbol, symbol) +#define MATCHED_LITER_DEC \ + matchByte <<=3D 1; \ + bit =3D (matchByte & offs); \ + probLit =3D prob + offs + bit + symbol; \ + GET_BIT2(probLit, symbol, offs &=3D ~bit, offs &=3D bit) + #define NORMALIZE_CHECK if (range < kTopValue) { if (buf >=3D bufLimit) re= turn DUMMY_ERROR; range <<=3D 8; code =3D (code << 8) | (*buf++); } =20 #define IF_BIT_0_CHECK(p) ttt =3D *(p); NORMALIZE_CHECK; bound =3D (range = >> kNumBitModelTotalBits) * ttt; if (code < bound) @@ -105,19 +114,13 @@ #define Literal (RepLenCoder + kNumLenProbs) =20 #define LZMA_BASE_SIZE 1846 -#define LZMA_LIT_SIZE 768 - -#define LzmaProps_GetNumProbs(p) ((UInt32)LZMA_BASE_SIZE + (LZMA_LIT_SIZE = << ((p)->lc + (p)->lp))) +#define LZMA_LIT_SIZE 0x300 =20 #if Literal !=3D LZMA_BASE_SIZE StopCompilingDueBUG #endif =20 -static const Byte kLiteralNextStates[kNumStates * 2] =3D -{ - 0, 0, 0, 0, 1, 2, 3, 4, 5, 6, 4, 5, - 7, 7, 7, 7, 7, 7, 7, 10, 10, 10, 10, 10 -}; +#define LzmaProps_GetNumProbs(p) (Literal + ((UInt32)LZMA_LIT_SIZE << ((p)= ->lc + (p)->lp))) =20 #define LZMA_DIC_MIN (1 << 12) =20 @@ -130,8 +133,8 @@ Out: p->remainLen: < kMatchSpecLenStart : normal remain =3D kMatchSpecLenStart : finished - =3D kMatchSpecLenStart + 1 : Flush marker - =3D kMatchSpecLenStart + 2 : State Init Marker + =3D kMatchSpecLenStart + 1 : Flush marker (unused now) + =3D kMatchSpecLenStart + 2 : State Init Marker (unused now) */ =20 static int MY_FAST_CALL LzmaDec_DecodeReal(CLzmaDec *p, SizeT limit, const= Byte *bufLimit) @@ -169,39 +172,62 @@ static int MY_FAST_CALL LzmaDec_DecodeReal(CLzmaDec *= p, SizeT limit, const Byte unsigned symbol; UPDATE_0(prob); prob =3D probs + Literal; - if (checkDicSize !=3D 0 || processedPos !=3D 0) - prob +=3D (LZMA_LIT_SIZE * (((processedPos & lpMask) << lc) + - (dic[(dicPos =3D=3D 0 ? dicBufSize : dicPos) - 1] >> (8 - lc)))); + if (processedPos !=3D 0 || checkDicSize !=3D 0) + prob +=3D ((UInt32)LZMA_LIT_SIZE * (((processedPos & lpMask) << lc= ) + + (dic[(dicPos =3D=3D 0 ? dicBufSize : dicPos) - 1] >> (8 - lc))= )); + processedPos++; =20 if (state < kNumLitStates) { + state -=3D (state < 4) ? state : 3; symbol =3D 1; - do { GET_BIT(prob + symbol, symbol) } while (symbol < 0x100); + #ifdef _LZMA_SIZE_OPT + do { NORMAL_LITER_DEC } while (symbol < 0x100); + #else + NORMAL_LITER_DEC + NORMAL_LITER_DEC + NORMAL_LITER_DEC + NORMAL_LITER_DEC + NORMAL_LITER_DEC + NORMAL_LITER_DEC + NORMAL_LITER_DEC + NORMAL_LITER_DEC + #endif } else { - unsigned matchByte =3D p->dic[(dicPos - rep0) + ((dicPos < rep0) ?= dicBufSize : 0)]; + unsigned matchByte =3D dic[dicPos - rep0 + (dicPos < rep0 ? dicBuf= Size : 0)]; unsigned offs =3D 0x100; + state -=3D (state < 10) ? 3 : 6; symbol =3D 1; + #ifdef _LZMA_SIZE_OPT do { unsigned bit; CLzmaProb *probLit; - matchByte <<=3D 1; - bit =3D (matchByte & offs); - probLit =3D prob + offs + bit + symbol; - GET_BIT2(probLit, symbol, offs &=3D ~bit, offs &=3D bit) + MATCHED_LITER_DEC } while (symbol < 0x100); + #else + { + unsigned bit; + CLzmaProb *probLit; + MATCHED_LITER_DEC + MATCHED_LITER_DEC + MATCHED_LITER_DEC + MATCHED_LITER_DEC + MATCHED_LITER_DEC + MATCHED_LITER_DEC + MATCHED_LITER_DEC + MATCHED_LITER_DEC + } + #endif } - dic[dicPos++] =3D (Byte)symbol; - processedPos++; =20 - state =3D kLiteralNextStates[state]; - /* if (state < 4) state =3D 0; else if (state < 10) state -=3D 3; el= se state -=3D 6; */ + dic[dicPos++] =3D (Byte)symbol; continue; } - else + =20 { UPDATE_1(prob); prob =3D probs + IsRep + state; @@ -224,7 +250,7 @@ static int MY_FAST_CALL LzmaDec_DecodeReal(CLzmaDec *p,= SizeT limit, const Byte IF_BIT_0(prob) { UPDATE_0(prob); - dic[dicPos] =3D dic[(dicPos - rep0) + ((dicPos < rep0) ? dicBu= fSize : 0)]; + dic[dicPos] =3D dic[dicPos - rep0 + (dicPos < rep0 ? dicBufSiz= e : 0)]; dicPos++; processedPos++; state =3D state < kNumLitStates ? 9 : 11; @@ -265,15 +291,17 @@ static int MY_FAST_CALL LzmaDec_DecodeReal(CLzmaDec *= p, SizeT limit, const Byte state =3D state < kNumLitStates ? 8 : 11; prob =3D probs + RepLenCoder; } + =20 + #ifdef _LZMA_SIZE_OPT { - unsigned offset; + unsigned lim, offset; CLzmaProb *probLen =3D prob + LenChoice; IF_BIT_0(probLen) { UPDATE_0(probLen); probLen =3D prob + LenLow + (posState << kLenNumLowBits); offset =3D 0; - limit =3D (1 << kLenNumLowBits); + lim =3D (1 << kLenNumLowBits); } else { @@ -284,19 +312,55 @@ static int MY_FAST_CALL LzmaDec_DecodeReal(CLzmaDec *= p, SizeT limit, const Byte UPDATE_0(probLen); probLen =3D prob + LenMid + (posState << kLenNumMidBits); offset =3D kLenNumLowSymbols; - limit =3D (1 << kLenNumMidBits); + lim =3D (1 << kLenNumMidBits); } else { UPDATE_1(probLen); probLen =3D prob + LenHigh; offset =3D kLenNumLowSymbols + kLenNumMidSymbols; - limit =3D (1 << kLenNumHighBits); + lim =3D (1 << kLenNumHighBits); } } - TREE_DECODE(probLen, limit, len); + TREE_DECODE(probLen, lim, len); len +=3D offset; } + #else + { + CLzmaProb *probLen =3D prob + LenChoice; + IF_BIT_0(probLen) + { + UPDATE_0(probLen); + probLen =3D prob + LenLow + (posState << kLenNumLowBits); + len =3D 1; + TREE_GET_BIT(probLen, len); + TREE_GET_BIT(probLen, len); + TREE_GET_BIT(probLen, len); + len -=3D 8; + } + else + { + UPDATE_1(probLen); + probLen =3D prob + LenChoice2; + IF_BIT_0(probLen) + { + UPDATE_0(probLen); + probLen =3D prob + LenMid + (posState << kLenNumMidBits); + len =3D 1; + TREE_GET_BIT(probLen, len); + TREE_GET_BIT(probLen, len); + TREE_GET_BIT(probLen, len); + } + else + { + UPDATE_1(probLen); + probLen =3D prob + LenHigh; + TREE_DECODE(probLen, (1 << kLenNumHighBits), len); + len +=3D kLenNumLowSymbols + kLenNumMidSymbols; + } + } + } + #endif =20 if (state >=3D kNumStates) { @@ -307,7 +371,7 @@ static int MY_FAST_CALL LzmaDec_DecodeReal(CLzmaDec *p,= SizeT limit, const Byte if (distance >=3D kStartPosModelIndex) { unsigned posSlot =3D (unsigned)distance; - int numDirectBits =3D (int)(((distance >> 1) - 1)); + unsigned numDirectBits =3D (unsigned)(((distance >> 1) - 1)); distance =3D (2 | (distance & 1)); if (posSlot < kEndPosModelIndex) { @@ -366,6 +430,7 @@ static int MY_FAST_CALL LzmaDec_DecodeReal(CLzmaDec *p,= SizeT limit, const Byte } } } + =20 rep3 =3D rep2; rep2 =3D rep1; rep1 =3D rep0; @@ -373,27 +438,39 @@ static int MY_FAST_CALL LzmaDec_DecodeReal(CLzmaDec *= p, SizeT limit, const Byte if (checkDicSize =3D=3D 0) { if (distance >=3D processedPos) + { + p->dicPos =3D dicPos; return SZ_ERROR_DATA; + } } else if (distance >=3D checkDicSize) + { + p->dicPos =3D dicPos; return SZ_ERROR_DATA; + } state =3D (state < kNumStates + kNumLitStates) ? kNumLitStates : k= NumLitStates + 3; - /* state =3D kLiteralNextStates[state]; */ } =20 len +=3D kMatchMinLen; =20 - if (limit =3D=3D dicPos) - return SZ_ERROR_DATA; { - SizeT rem =3D limit - dicPos; - unsigned curLen =3D ((rem < len) ? (unsigned)rem : len); - SizeT pos =3D (dicPos - rep0) + ((dicPos < rep0) ? dicBufSize : 0)= ; + SizeT rem; + unsigned curLen; + SizeT pos; + =20 + if ((rem =3D limit - dicPos) =3D=3D 0) + { + p->dicPos =3D dicPos; + return SZ_ERROR_DATA; + } + =20 + curLen =3D ((rem < len) ? (unsigned)rem : len); + pos =3D dicPos - rep0 + (dicPos < rep0 ? dicBufSize : 0); =20 processedPos +=3D curLen; =20 len -=3D curLen; - if (pos + curLen <=3D dicBufSize) + if (curLen <=3D dicBufSize - pos) { Byte *dest =3D dic + dicPos; ptrdiff_t src =3D (ptrdiff_t)pos - (ptrdiff_t)dicPos; @@ -417,7 +494,9 @@ static int MY_FAST_CALL LzmaDec_DecodeReal(CLzmaDec *p,= SizeT limit, const Byte } } while (dicPos < limit && buf < bufLimit); + NORMALIZE; + =20 p->buf =3D buf; p->range =3D range; p->code =3D code; @@ -441,18 +520,20 @@ static void MY_FAST_CALL LzmaDec_WriteRem(CLzmaDec *p= , SizeT limit) SizeT dicPos =3D p->dicPos; SizeT dicBufSize =3D p->dicBufSize; unsigned len =3D p->remainLen; - UInt32 rep0 =3D p->reps[0]; - if (limit - dicPos < len) - len =3D (unsigned)(limit - dicPos); + SizeT rep0 =3D p->reps[0]; /* we use SizeT to avoid the BUG of VC14 fo= r AMD64 */ + SizeT rem =3D limit - dicPos; + if (rem < len) + len =3D (unsigned)(rem); =20 if (p->checkDicSize =3D=3D 0 && p->prop.dicSize - p->processedPos <=3D= len) p->checkDicSize =3D p->prop.dicSize; =20 p->processedPos +=3D len; p->remainLen -=3D len; - while (len-- !=3D 0) + while (len !=3D 0) { - dic[dicPos] =3D dic[(dicPos - rep0) + ((dicPos < rep0) ? dicBufSize = : 0)]; + len--; + dic[dicPos] =3D dic[dicPos - rep0 + (dicPos < rep0 ? dicBufSize : 0)= ]; dicPos++; } p->dicPos =3D dicPos; @@ -470,17 +551,19 @@ static int MY_FAST_CALL LzmaDec_DecodeReal2(CLzmaDec = *p, SizeT limit, const Byte if (limit - p->dicPos > rem) limit2 =3D p->dicPos + rem; } + =20 RINOK(LzmaDec_DecodeReal(p, limit2, bufLimit)); - if (p->processedPos >=3D p->prop.dicSize) + =20 + if (p->checkDicSize =3D=3D 0 && p->processedPos >=3D p->prop.dicSize) p->checkDicSize =3D p->prop.dicSize; + =20 LzmaDec_WriteRem(p, limit); } while (p->dicPos < limit && p->buf < bufLimit && p->remainLen < kMatchSp= ecLenStart); =20 if (p->remainLen > kMatchSpecLenStart) - { p->remainLen =3D kMatchSpecLenStart; - } + return 0; } =20 @@ -497,12 +580,12 @@ static ELzmaDummy LzmaDec_TryDummy(const CLzmaDec *p,= const Byte *buf, SizeT inS UInt32 range =3D p->range; UInt32 code =3D p->code; const Byte *bufLimit =3D buf + inSize; - CLzmaProb *probs =3D p->probs; + const CLzmaProb *probs =3D p->probs; unsigned state =3D p->state; ELzmaDummy res; =20 { - CLzmaProb *prob; + const CLzmaProb *prob; UInt32 bound; unsigned ttt; unsigned posState =3D (p->processedPos) & ((1 << p->prop.pb) - 1); @@ -516,9 +599,9 @@ static ELzmaDummy LzmaDec_TryDummy(const CLzmaDec *p, c= onst Byte *buf, SizeT inS =20 prob =3D probs + Literal; if (p->checkDicSize !=3D 0 || p->processedPos !=3D 0) - prob +=3D (LZMA_LIT_SIZE * - ((((p->processedPos) & ((1 << (p->prop.lp)) - 1)) << p->prop.lc)= + - (p->dic[(p->dicPos =3D=3D 0 ? p->dicBufSize : p->dicPos) - 1] >>= (8 - p->prop.lc)))); + prob +=3D ((UInt32)LZMA_LIT_SIZE * + ((((p->processedPos) & ((1 << (p->prop.lp)) - 1)) << p->prop.l= c) + + (p->dic[(p->dicPos =3D=3D 0 ? p->dicBufSize : p->dicPos) - 1] = >> (8 - p->prop.lc)))); =20 if (state < kNumLitStates) { @@ -528,13 +611,13 @@ static ELzmaDummy LzmaDec_TryDummy(const CLzmaDec *p,= const Byte *buf, SizeT inS else { unsigned matchByte =3D p->dic[p->dicPos - p->reps[0] + - ((p->dicPos < p->reps[0]) ? p->dicBufSize : 0)]; + (p->dicPos < p->reps[0] ? p->dicBufSize : 0)]; unsigned offs =3D 0x100; unsigned symbol =3D 1; do { unsigned bit; - CLzmaProb *probLit; + const CLzmaProb *probLit; matchByte <<=3D 1; bit =3D (matchByte & offs); probLit =3D prob + offs + bit + symbol; @@ -604,7 +687,7 @@ static ELzmaDummy LzmaDec_TryDummy(const CLzmaDec *p, c= onst Byte *buf, SizeT inS } { unsigned limit, offset; - CLzmaProb *probLen =3D prob + LenChoice; + const CLzmaProb *probLen =3D prob + LenChoice; IF_BIT_0_CHECK(probLen) { UPDATE_0_CHECK; @@ -644,7 +727,7 @@ static ELzmaDummy LzmaDec_TryDummy(const CLzmaDec *p, c= onst Byte *buf, SizeT inS TREE_DECODE_CHECK(prob, 1 << kNumPosSlotBits, posSlot); if (posSlot >=3D kStartPosModelIndex) { - int numDirectBits =3D ((posSlot >> 1) - 1); + unsigned numDirectBits =3D ((posSlot >> 1) - 1); =20 /* if (bufLimit - buf >=3D 8) return DUMMY_MATCH; */ =20 @@ -683,13 +766,6 @@ static ELzmaDummy LzmaDec_TryDummy(const CLzmaDec *p, = const Byte *buf, SizeT inS } =20 =20 -static void LzmaDec_InitRc(CLzmaDec *p, const Byte *data) -{ - p->code =3D ((UInt32)data[1] << 24) | ((UInt32)data[2] << 16) | ((UInt32= )data[3] << 8) | ((UInt32)data[4]); - p->range =3D 0xFFFFFFFF; - p->needFlush =3D 0; -} - void LzmaDec_InitDicAndState(CLzmaDec *p, Bool initDic, Bool initState) { p->needFlush =3D 1; @@ -714,8 +790,8 @@ void LzmaDec_Init(CLzmaDec *p) =20 static void LzmaDec_InitStateReal(CLzmaDec *p) { - UInt32 numProbs =3D Literal + ((UInt32)LZMA_LIT_SIZE << (p->prop.lc + p-= >prop.lp)); - UInt32 i; + SizeT numProbs =3D LzmaProps_GetNumProbs(&p->prop); + SizeT i; CLzmaProb *probs =3D p->probs; for (i =3D 0; i < numProbs; i++) probs[i] =3D kBitModelTotal >> 1; @@ -737,7 +813,7 @@ SRes LzmaDec_DecodeToDic(CLzmaDec *p, SizeT dicLimit, c= onst Byte *src, SizeT *sr { int checkEndMarkNow; =20 - if (p->needFlush !=3D 0) + if (p->needFlush) { for (; inSize > 0 && p->tempBufSize < RC_INIT_SIZE; (*srcLen)++, i= nSize--) p->tempBuf[p->tempBufSize++] =3D *src++; @@ -748,8 +824,13 @@ SRes LzmaDec_DecodeToDic(CLzmaDec *p, SizeT dicLimit, = const Byte *src, SizeT *sr } if (p->tempBuf[0] !=3D 0) return SZ_ERROR_DATA; - - LzmaDec_InitRc(p, p->tempBuf); + p->code =3D + ((UInt32)p->tempBuf[1] << 24) + | ((UInt32)p->tempBuf[2] << 16) + | ((UInt32)p->tempBuf[3] << 8) + | ((UInt32)p->tempBuf[4]); + p->range =3D 0xFFFFFFFF; + p->needFlush =3D 0; p->tempBufSize =3D 0; } =20 @@ -833,7 +914,16 @@ SRes LzmaDec_DecodeToDic(CLzmaDec *p, SizeT dicLimit, = const Byte *src, SizeT *sr p->buf =3D p->tempBuf; if (LzmaDec_DecodeReal2(p, dicLimit, p->buf) !=3D 0) return SZ_ERROR_DATA; - lookAhead -=3D (rem - (unsigned)(p->buf - p->tempBuf)); + =20 + { + unsigned kkk =3D (unsigned)(p->buf - p->tempBuf); + if (rem < kkk) + return SZ_ERROR_FAIL; /* some internal error */ + rem -=3D kkk; + if (lookAhead < rem) + return SZ_ERROR_FAIL; /* some internal error */ + lookAhead -=3D rem; + } (*srcLen) +=3D lookAhead; src +=3D lookAhead; inSize -=3D lookAhead; @@ -888,13 +978,13 @@ SRes LzmaDec_DecodeToBuf(CLzmaDec *p, Byte *dest, Siz= eT *destLen, const Byte *sr void LzmaDec_FreeProbs(CLzmaDec *p, ISzAlloc *alloc) { alloc->Free(alloc, p->probs); - p->probs =3D 0; + p->probs =3D NULL; } =20 static void LzmaDec_FreeDict(CLzmaDec *p, ISzAlloc *alloc) { alloc->Free(alloc, p->dic); - p->dic =3D 0; + p->dic =3D NULL; } =20 void LzmaDec_Free(CLzmaDec *p, ISzAlloc *alloc) @@ -932,12 +1022,12 @@ SRes LzmaProps_Decode(CLzmaProps *p, const Byte *dat= a, unsigned size) static SRes LzmaDec_AllocateProbs2(CLzmaDec *p, const CLzmaProps *propNew,= ISzAlloc *alloc) { UInt32 numProbs =3D LzmaProps_GetNumProbs(propNew); - if (p->probs =3D=3D 0 || numProbs !=3D p->numProbs) + if (!p->probs || numProbs !=3D p->numProbs) { LzmaDec_FreeProbs(p, alloc); p->probs =3D (CLzmaProb *)alloc->Alloc(alloc, numProbs * sizeof(CLzmaP= rob)); p->numProbs =3D numProbs; - if (p->probs =3D=3D 0) + if (!p->probs) return SZ_ERROR_MEM; } return SZ_OK; @@ -958,12 +1048,22 @@ SRes LzmaDec_Allocate(CLzmaDec *p, const Byte *props= , unsigned propsSize, ISzAll SizeT dicBufSize; RINOK(LzmaProps_Decode(&propNew, props, propsSize)); RINOK(LzmaDec_AllocateProbs2(p, &propNew, alloc)); - dicBufSize =3D propNew.dicSize; - if (p->dic =3D=3D 0 || dicBufSize !=3D p->dicBufSize) + + { + UInt32 dictSize =3D propNew.dicSize; + SizeT mask =3D ((UInt32)1 << 12) - 1; + if (dictSize >=3D ((UInt32)1 << 30)) mask =3D ((UInt32)1 << 22) -= 1; + else if (dictSize >=3D ((UInt32)1 << 22)) mask =3D ((UInt32)1 << 20) -= 1;; + dicBufSize =3D ((SizeT)dictSize + mask) & ~mask; + if (dicBufSize < dictSize) + dicBufSize =3D dictSize; + } + + if (!p->dic || dicBufSize !=3D p->dicBufSize) { LzmaDec_FreeDict(p, alloc); p->dic =3D (Byte *)alloc->Alloc(alloc, dicBufSize); - if (p->dic =3D=3D 0) + if (!p->dic) { LzmaDec_FreeProbs(p, alloc); return SZ_ERROR_MEM; @@ -980,28 +1080,21 @@ SRes LzmaDecode(Byte *dest, SizeT *destLen, const By= te *src, SizeT *srcLen, { CLzmaDec p; SRes res; - SizeT inSize =3D *srcLen; - SizeT outSize =3D *destLen; - *srcLen =3D *destLen =3D 0; + SizeT outSize =3D *destLen, inSize =3D *srcLen; + *destLen =3D *srcLen =3D 0; + *status =3D LZMA_STATUS_NOT_SPECIFIED; if (inSize < RC_INIT_SIZE) return SZ_ERROR_INPUT_EOF; - LzmaDec_Construct(&p); - res =3D LzmaDec_AllocateProbs(&p, propData, propSize, alloc); - if (res !=3D 0) - return res; + RINOK(LzmaDec_AllocateProbs(&p, propData, propSize, alloc)); p.dic =3D dest; p.dicBufSize =3D outSize; - LzmaDec_Init(&p); - =20 *srcLen =3D inSize; res =3D LzmaDec_DecodeToDic(&p, outSize, src, srcLen, finishMode, status= ); - + *destLen =3D p.dicPos; if (res =3D=3D SZ_OK && *status =3D=3D LZMA_STATUS_NEEDS_MORE_INPUT) res =3D SZ_ERROR_INPUT_EOF; - - (*destLen) =3D p.dicPos; LzmaDec_FreeProbs(&p, alloc); return res; } diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaDec.h b/BaseTools/So= urce/C/LzmaCompress/Sdk/C/LzmaDec.h index ad7d705..2633abe 100644 --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaDec.h +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaDec.h @@ -1,10 +1,12 @@ /* LzmaDec.h -- LZMA Decoder -2008-10-04 : Igor Pavlov : Public domain */ +2013-01-18 : Igor Pavlov : Public domain */ =20 -#ifndef __LZMADEC_H -#define __LZMADEC_H +#ifndef __LZMA_DEC_H +#define __LZMA_DEC_H =20 -#include "Types.h" +#include "7zTypes.h" + +EXTERN_C_BEGIN =20 /* #define _LZMA_PROB32 */ /* _LZMA_PROB32 can increase the speed on some CPUs, @@ -220,4 +222,6 @@ SRes LzmaDecode(Byte *dest, SizeT *destLen, const Byte = *src, SizeT *srcLen, const Byte *propData, unsigned propSize, ELzmaFinishMode finishMode, ELzmaStatus *status, ISzAlloc *alloc); =20 +EXTERN_C_END + #endif diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.c b/BaseTools/So= urce/C/LzmaCompress/Sdk/C/LzmaEnc.c index 5f70ebd..462ca67 100644 --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.c +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.c @@ -1,18 +1,7 @@ -/** @file - Based on LZMA SDK 4.65: - LzmaEnc.c -- LZMA Encoder - 2009-02-02 : Igor Pavlov : Public domain +/* LzmaEnc.c -- LZMA Encoder +2016-05-16 : Igor Pavlov : Public domain */ =20 - Copyright (c) 2011 - 2016, Intel Corporation. All rights reserved.
- This program and the accompanying materials - are licensed and made available under the terms and conditions of the BS= D License - which accompanies this distribution. The full text of the license may b= e found at - http://opensource.org/licenses/bsd-license.php - - THE PROGRAM IS DISTRIBUTED UNDER THE BSD LICENSE ON AN "AS IS" BASIS, - WITHOUT WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMP= LIED. - -**/ +#include "Precomp.h" =20 #include =20 @@ -26,14 +15,17 @@ #include "LzmaEnc.h" =20 #include "LzFind.h" -#ifdef COMPRESS_MF_MT +#ifndef _7ZIP_ST #include "LzFindMt.h" #endif =20 #ifdef SHOW_STAT -static int ttt =3D 0; +static unsigned g_STAT_OFFSET =3D 0; #endif =20 +#define kMaxHistorySize ((UInt32)3 << 29) +/* #define kMaxHistorySize ((UInt32)7 << 29) */ + #define kBlockSizeMax ((1 << LZMA_NUM_BLOCK_SIZE_BITS) - 1) =20 #define kBlockSize (9 << 10) @@ -59,6 +51,7 @@ void LzmaEncProps_Init(CLzmaEncProps *p) { p->level =3D 5; p->dictSize =3D p->mc =3D 0; + p->reduceSize =3D (UInt64)(Int64)-1; p->lc =3D p->lp =3D p->pb =3D p->algo =3D p->fb =3D p->btMode =3D p->num= HashBytes =3D p->numThreads =3D -1; p->writeEndMark =3D 0; } @@ -68,18 +61,31 @@ void LzmaEncProps_Normalize(CLzmaEncProps *p) int level =3D p->level; if (level < 0) level =3D 5; p->level =3D level; + =20 if (p->dictSize =3D=3D 0) p->dictSize =3D (level <=3D 5 ? (1 << (level *= 2 + 14)) : (level =3D=3D 6 ? (1 << 25) : (1 << 26))); + if (p->dictSize > p->reduceSize) + { + unsigned i; + for (i =3D 11; i <=3D 30; i++) + { + if ((UInt32)p->reduceSize <=3D ((UInt32)2 << i)) { p->dictSize =3D (= (UInt32)2 << i); break; } + if ((UInt32)p->reduceSize <=3D ((UInt32)3 << i)) { p->dictSize =3D (= (UInt32)3 << i); break; } + } + } + if (p->lc < 0) p->lc =3D 3; if (p->lp < 0) p->lp =3D 0; if (p->pb < 0) p->pb =3D 2; + if (p->algo < 0) p->algo =3D (level < 5 ? 0 : 1); if (p->fb < 0) p->fb =3D (level < 7 ? 32 : 64); if (p->btMode < 0) p->btMode =3D (p->algo =3D=3D 0 ? 0 : 1); if (p->numHashBytes < 0) p->numHashBytes =3D 4; - if (p->mc =3D=3D 0) p->mc =3D (16 + (p->fb >> 1)) >> (p->btMode ? 0 : 1= ); + if (p->mc =3D=3D 0) p->mc =3D (16 + (p->fb >> 1)) >> (p->btMode ? 0 : 1)= ; + =20 if (p->numThreads < 0) p->numThreads =3D - #ifdef COMPRESS_MF_MT + #ifndef _7ZIP_ST ((p->btMode && p->algo) ? 2 : 1); #else 1; @@ -93,17 +99,18 @@ UInt32 LzmaEncProps_GetDictSize(const CLzmaEncProps *pr= ops2) return props.dictSize; } =20 +#if (_MSC_VER >=3D 1400) +/* BSR code is fast for some new CPUs */ /* #define LZMA_LOG_BSR */ -/* Define it for Intel's CPU */ - +#endif =20 #ifdef LZMA_LOG_BSR =20 -#define kDicLogSizeMaxCompress 30 +#define kDicLogSizeMaxCompress 32 =20 -#define BSR2_RET(pos, res) { unsigned long i; _BitScanReverse(&i, (pos)); = res =3D (i + i) + ((pos >> (i - 1)) & 1); } +#define BSR2_RET(pos, res) { unsigned long zz; _BitScanReverse(&zz, (pos))= ; res =3D (zz + zz) + ((pos >> (zz - 1)) & 1); } =20 -UInt32 GetPosSlot1(UInt32 pos) +static UInt32 GetPosSlot1(UInt32 pos) { UInt32 res; BSR2_RET(pos, res); @@ -114,27 +121,44 @@ UInt32 GetPosSlot1(UInt32 pos) =20 #else =20 -#define kNumLogBits (9 + (int)sizeof(size_t) / 2) +#define kNumLogBits (9 + sizeof(size_t) / 2) +/* #define kNumLogBits (11 + sizeof(size_t) / 8 * 3) */ + #define kDicLogSizeMaxCompress ((kNumLogBits - 1) * 2 + 7) =20 -void LzmaEnc_FastPosInit(Byte *g_FastPos) +static void LzmaEnc_FastPosInit(Byte *g_FastPos) { - int c =3D 2, slotFast; + unsigned slot; g_FastPos[0] =3D 0; g_FastPos[1] =3D 1; + g_FastPos +=3D 2; =20 - for (slotFast =3D 2; slotFast < kNumLogBits * 2; slotFast++) + for (slot =3D 2; slot < kNumLogBits * 2; slot++) { - UInt32 k =3D (1 << ((slotFast >> 1) - 1)); - UInt32 j; - for (j =3D 0; j < k; j++, c++) - g_FastPos[c] =3D (Byte)slotFast; + size_t k =3D ((size_t)1 << ((slot >> 1) - 1)); + size_t j; + for (j =3D 0; j < k; j++) + g_FastPos[j] =3D (Byte)slot; + g_FastPos +=3D k; } } =20 -#define BSR2_RET(pos, res) { UInt32 ij =3D 6 + ((kNumLogBits - 1) & \ +/* we can use ((limit - pos) >> 31) only if (pos < ((UInt32)1 << 31)) */ +/* +#define BSR2_RET(pos, res) { UInt32 zz =3D 6 + ((kNumLogBits - 1) & \ (0 - (((((UInt32)1 << (kNumLogBits + 6)) - 1) - pos) >> 31))); \ - res =3D p->g_FastPos[pos >> ij] + (ij * 2); } + res =3D p->g_FastPos[pos >> zz] + (zz * 2); } +*/ + +/* +#define BSR2_RET(pos, res) { UInt32 zz =3D 6 + ((kNumLogBits - 1) & \ + (0 - (((((UInt32)1 << (kNumLogBits)) - 1) - (pos >> 6)) >> 31))); \ + res =3D p->g_FastPos[pos >> zz] + (zz * 2); } +*/ + +#define BSR2_RET(pos, res) { UInt32 zz =3D (pos < (1 << (kNumLogBits + 6))= ) ? 6 : 6 + kNumLogBits - 1; \ + res =3D p->g_FastPos[pos >> zz] + (zz * 2); } + /* #define BSR2_RET(pos, res) { res =3D (pos < (1 << (kNumLogBits + 6))) ? \ p->g_FastPos[pos >> 6] + 12 : \ @@ -152,7 +176,7 @@ void LzmaEnc_FastPosInit(Byte *g_FastPos) =20 typedef unsigned CState; =20 -typedef struct _COptimal +typedef struct { UInt32 price; =20 @@ -185,7 +209,7 @@ typedef struct _COptimal #define kEndPosModelIndex 14 #define kNumPosModels (kEndPosModelIndex - kStartPosModelIndex) =20 -#define kNumFullDistances (1 << (kEndPosModelIndex / 2)) +#define kNumFullDistances (1 << (kEndPosModelIndex >> 1)) =20 #ifdef _LZMA_PROB32 #define CLzmaProb UInt32 @@ -214,6 +238,7 @@ typedef struct _COptimal =20 #define kNumStates 12 =20 + typedef struct { CLzmaProb choice; @@ -223,15 +248,17 @@ typedef struct CLzmaProb high[kLenNumHighSymbols]; } CLenEnc; =20 + typedef struct { CLenEnc p; - UInt32 prices[LZMA_NUM_PB_STATES_MAX][kLenNumSymbolsTotal]; UInt32 tableSize; + UInt32 prices[LZMA_NUM_PB_STATES_MAX][kLenNumSymbolsTotal]; UInt32 counters[LZMA_NUM_PB_STATES_MAX]; } CLenPriceEnc; =20 -typedef struct _CRangeEnc + +typedef struct { UInt32 range; Byte cache; @@ -245,30 +272,14 @@ typedef struct _CRangeEnc SRes res; } CRangeEnc; =20 -typedef struct _CSeqInStreamBuf -{ - ISeqInStream funcTable; - const Byte *data; - SizeT rem; -} CSeqInStreamBuf; - -static SRes MyRead(void *pp, void *data, size_t *size) -{ - size_t curSize =3D *size; - CSeqInStreamBuf *p =3D (CSeqInStreamBuf *)pp; - if (p->rem < curSize) - curSize =3D p->rem; - memcpy(data, p->data, curSize); - p->rem -=3D curSize; - p->data +=3D curSize; - *size =3D curSize; - return SZ_OK; -} =20 typedef struct { CLzmaProb *litProbs; =20 + UInt32 state; + UInt32 reps[LZMA_NUM_REPS]; + CLzmaProb isMatch[kNumStates][LZMA_NUM_PB_STATES_MAX]; CLzmaProb isRep[kNumStates]; CLzmaProb isRepG0[kNumStates]; @@ -282,33 +293,61 @@ typedef struct =20 CLenPriceEnc lenEnc; CLenPriceEnc repLenEnc; - - UInt32 reps[LZMA_NUM_REPS]; - UInt32 state; } CSaveState; =20 -typedef struct _CLzmaEnc + +typedef struct { - IMatchFinder matchFinder; void *matchFinderObj; + IMatchFinder matchFinder; + + UInt32 optimumEndIndex; + UInt32 optimumCurrentIndex; =20 - #ifdef COMPRESS_MF_MT + UInt32 longestMatchLength; + UInt32 numPairs; + UInt32 numAvail; + + UInt32 numFastBytes; + UInt32 additionalOffset; + UInt32 reps[LZMA_NUM_REPS]; + UInt32 state; + + unsigned lc, lp, pb; + unsigned lpMask, pbMask; + unsigned lclp; + + CLzmaProb *litProbs; + + Bool fastMode; + Bool writeEndMark; + Bool finished; + Bool multiThread; + Bool needInit; + + UInt64 nowPos64; + =20 + UInt32 matchPriceCount; + UInt32 alignPriceCount; + + UInt32 distTableSize; + + UInt32 dictSize; + SRes result; + + CRangeEnc rc; + + #ifndef _7ZIP_ST Bool mtMode; CMatchFinderMt matchFinderMt; #endif =20 CMatchFinder matchFinderBase; =20 - #ifdef COMPRESS_MF_MT + #ifndef _7ZIP_ST Byte pad[128]; #endif =20 - UInt32 optimumEndIndex; - UInt32 optimumCurrentIndex; - - UInt32 longestMatchLength; - UInt32 numPairs; - UInt32 numAvail; COptimal opt[kNumOpts]; =20 #ifndef LZMA_LOG_BSR @@ -317,22 +356,10 @@ typedef struct _CLzmaEnc =20 UInt32 ProbPrices[kBitModelTotal >> kNumMoveReducingBits]; UInt32 matches[LZMA_MATCH_LEN_MAX * 2 + 2 + 1]; - UInt32 numFastBytes; - UInt32 additionalOffset; - UInt32 reps[LZMA_NUM_REPS]; - UInt32 state; =20 UInt32 posSlotPrices[kNumLenToPosStates][kDistTableSizeMax]; UInt32 distancesPrices[kNumLenToPosStates][kNumFullDistances]; UInt32 alignPrices[kAlignTableSize]; - UInt32 alignPriceCount; - - UInt32 distTableSize; - - unsigned lc, lp, pb; - unsigned lpMask, pbMask; - - CLzmaProb *litProbs; =20 CLzmaProb isMatch[kNumStates][LZMA_NUM_PB_STATES_MAX]; CLzmaProb isRep[kNumStates]; @@ -348,28 +375,14 @@ typedef struct _CLzmaEnc CLenPriceEnc lenEnc; CLenPriceEnc repLenEnc; =20 - unsigned lclp; - - Bool fastMode; - =20 - CRangeEnc rc; - - Bool writeEndMark; - UInt64 nowPos64; - UInt32 matchPriceCount; - Bool finished; - Bool multiThread; - - SRes result; - UInt32 dictSize; - UInt32 matchFinderCycles; - - ISeqInStream *inStream; - CSeqInStreamBuf seqBufInStream; - CSaveState saveState; + + #ifndef _7ZIP_ST + Byte pad2[128]; + #endif } CLzmaEnc; =20 + void LzmaEnc_SaveState(CLzmaEncHandle pp) { CLzmaEnc *p =3D (CLzmaEnc *)pp; @@ -393,7 +406,7 @@ void LzmaEnc_SaveState(CLzmaEncHandle pp) memcpy(dest->posEncoders, p->posEncoders, sizeof(p->posEncoders)); memcpy(dest->posAlignEncoder, p->posAlignEncoder, sizeof(p->posAlignEnco= der)); memcpy(dest->reps, p->reps, sizeof(p->reps)); - memcpy(dest->litProbs, p->litProbs, (0x300 << p->lclp) * sizeof(CLzmaPro= b)); + memcpy(dest->litProbs, p->litProbs, ((UInt32)0x300 << p->lclp) * sizeof(= CLzmaProb)); } =20 void LzmaEnc_RestoreState(CLzmaEncHandle pp) @@ -419,7 +432,7 @@ void LzmaEnc_RestoreState(CLzmaEncHandle pp) memcpy(dest->posEncoders, p->posEncoders, sizeof(p->posEncoders)); memcpy(dest->posAlignEncoder, p->posAlignEncoder, sizeof(p->posAlignEnco= der)); memcpy(dest->reps, p->reps, sizeof(p->reps)); - memcpy(dest->litProbs, p->litProbs, (0x300 << dest->lclp) * sizeof(CLzma= Prob)); + memcpy(dest->litProbs, p->litProbs, ((UInt32)0x300 << dest->lclp) * size= of(CLzmaProb)); } =20 SRes LzmaEnc_SetProps(CLzmaEncHandle pp, const CLzmaEncProps *props2) @@ -428,11 +441,14 @@ SRes LzmaEnc_SetProps(CLzmaEncHandle pp, const CLzmaE= ncProps *props2) CLzmaEncProps props =3D *props2; LzmaEncProps_Normalize(&props); =20 - if (props.lc > LZMA_LC_MAX || props.lp > LZMA_LP_MAX || props.pb > LZMA_= PB_MAX || - props.dictSize > (1 << kDicLogSizeMaxCompress) || props.dictSize > (= 1 << 30)) + if (props.lc > LZMA_LC_MAX + || props.lp > LZMA_LP_MAX + || props.pb > LZMA_PB_MAX + || props.dictSize > ((UInt64)1 << kDicLogSizeMaxCompress) + || props.dictSize > kMaxHistorySize) return SZ_ERROR_PARAM; + p->dictSize =3D props.dictSize; - p->matchFinderCycles =3D props.mc; { unsigned fb =3D props.fb; if (fb < 5) @@ -445,7 +461,7 @@ SRes LzmaEnc_SetProps(CLzmaEncHandle pp, const CLzmaEnc= Props *props2) p->lp =3D props.lp; p->pb =3D props.pb; p->fastMode =3D (props.algo =3D=3D 0); - p->matchFinderBase.btMode =3D props.btMode; + p->matchFinderBase.btMode =3D (Byte)(props.btMode ? 1 : 0); { UInt32 numHashBytes =3D 4; if (props.btMode) @@ -462,7 +478,7 @@ SRes LzmaEnc_SetProps(CLzmaEncHandle pp, const CLzmaEnc= Props *props2) =20 p->writeEndMark =3D props.writeEndMark; =20 - #ifdef COMPRESS_MF_MT + #ifndef _7ZIP_ST /* if (newMultiThread !=3D _multiThread) { @@ -489,8 +505,8 @@ static const int kShortRepNextStates[kNumStates]=3D {9,= 9, 9, 9, 9, 9, 9, 11, 11, =20 static void RangeEnc_Construct(CRangeEnc *p) { - p->outStream =3D 0; - p->bufBase =3D 0; + p->outStream =3D NULL; + p->bufBase =3D NULL; } =20 #define RangeEnc_GetProcessed(p) ((p)->processed + ((p)->buf - (p)->bufBas= e) + (p)->cacheSize) @@ -498,10 +514,10 @@ static void RangeEnc_Construct(CRangeEnc *p) #define RC_BUF_SIZE (1 << 16) static int RangeEnc_Alloc(CRangeEnc *p, ISzAlloc *alloc) { - if (p->bufBase =3D=3D 0) + if (!p->bufBase) { p->bufBase =3D (Byte *)alloc->Alloc(alloc, RC_BUF_SIZE); - if (p->bufBase =3D=3D 0) + if (!p->bufBase) return 0; p->bufLim =3D p->bufBase + RC_BUF_SIZE; } @@ -542,7 +558,7 @@ static void RangeEnc_FlushStream(CRangeEnc *p) =20 static void MY_FAST_CALL RangeEnc_ShiftLow(CRangeEnc *p) { - if ((UInt32)p->low < (UInt32)0xFF000000 || (int)(p->low >> 32) !=3D 0) + if ((UInt32)p->low < (UInt32)0xFF000000 || (unsigned)(p->low >> 32) !=3D= 0) { Byte temp =3D p->cache; do @@ -568,7 +584,7 @@ static void RangeEnc_FlushData(CRangeEnc *p) RangeEnc_ShiftLow(p); } =20 -static void RangeEnc_EncodeDirectBits(CRangeEnc *p, UInt32 value, int numB= its) +static void RangeEnc_EncodeDirectBits(CRangeEnc *p, UInt32 value, unsigned= numBits) { do { @@ -631,7 +647,7 @@ static void LitEnc_EncodeMatched(CRangeEnc *p, CLzmaPro= b *probs, UInt32 symbol, while (symbol < 0x10000); } =20 -void LzmaEnc_InitPriceTables(UInt32 *ProbPrices) +static void LzmaEnc_InitPriceTables(UInt32 *ProbPrices) { UInt32 i; for (i =3D (1 << kNumMoveReducingBits) / 2; i < kBitModelTotal; i +=3D (= 1 << kNumMoveReducingBits)) @@ -667,7 +683,7 @@ void LzmaEnc_InitPriceTables(UInt32 *ProbPrices) #define GET_PRICE_0a(prob) ProbPrices[(prob) >> kNumMoveReducingBits] #define GET_PRICE_1a(prob) ProbPrices[((prob) ^ (kBitModelTotal - 1)) >> k= NumMoveReducingBits] =20 -static UInt32 LitEnc_GetPrice(const CLzmaProb *probs, UInt32 symbol, UInt3= 2 *ProbPrices) +static UInt32 LitEnc_GetPrice(const CLzmaProb *probs, UInt32 symbol, const= UInt32 *ProbPrices) { UInt32 price =3D 0; symbol |=3D 0x100; @@ -680,7 +696,7 @@ static UInt32 LitEnc_GetPrice(const CLzmaProb *probs, U= Int32 symbol, UInt32 *Pro return price; } =20 -static UInt32 LitEnc_GetPriceMatched(const CLzmaProb *probs, UInt32 symbol= , UInt32 matchByte, UInt32 *ProbPrices) +static UInt32 LitEnc_GetPriceMatched(const CLzmaProb *probs, UInt32 symbol= , UInt32 matchByte, const UInt32 *ProbPrices) { UInt32 price =3D 0; UInt32 offs =3D 0x100; @@ -724,7 +740,7 @@ static void RcTree_ReverseEncode(CRangeEnc *rc, CLzmaPr= ob *probs, int numBitLeve } } =20 -static UInt32 RcTree_GetPrice(const CLzmaProb *probs, int numBitLevels, UI= nt32 symbol, UInt32 *ProbPrices) +static UInt32 RcTree_GetPrice(const CLzmaProb *probs, int numBitLevels, UI= nt32 symbol, const UInt32 *ProbPrices) { UInt32 price =3D 0; symbol |=3D (1 << numBitLevels); @@ -736,7 +752,7 @@ static UInt32 RcTree_GetPrice(const CLzmaProb *probs, i= nt numBitLevels, UInt32 s return price; } =20 -static UInt32 RcTree_ReverseGetPrice(const CLzmaProb *probs, int numBitLev= els, UInt32 symbol, UInt32 *ProbPrices) +static UInt32 RcTree_ReverseGetPrice(const CLzmaProb *probs, int numBitLev= els, UInt32 symbol, const UInt32 *ProbPrices) { UInt32 price =3D 0; UInt32 m =3D 1; @@ -787,7 +803,7 @@ static void LenEnc_Encode(CLenEnc *p, CRangeEnc *rc, UI= nt32 symbol, UInt32 posSt } } =20 -static void LenEnc_SetPrices(CLenEnc *p, UInt32 posState, UInt32 numSymbol= s, UInt32 *prices, UInt32 *ProbPrices) +static void LenEnc_SetPrices(CLenEnc *p, UInt32 posState, UInt32 numSymbol= s, UInt32 *prices, const UInt32 *ProbPrices) { UInt32 a0 =3D GET_PRICE_0a(p->choice); UInt32 a1 =3D GET_PRICE_1a(p->choice); @@ -810,20 +826,20 @@ static void LenEnc_SetPrices(CLenEnc *p, UInt32 posSt= ate, UInt32 numSymbols, UIn prices[i] =3D b1 + RcTree_GetPrice(p->high, kLenNumHighBits, i - kLenN= umLowSymbols - kLenNumMidSymbols, ProbPrices); } =20 -static void MY_FAST_CALL LenPriceEnc_UpdateTable(CLenPriceEnc *p, UInt32 p= osState, UInt32 *ProbPrices) +static void MY_FAST_CALL LenPriceEnc_UpdateTable(CLenPriceEnc *p, UInt32 p= osState, const UInt32 *ProbPrices) { LenEnc_SetPrices(&p->p, posState, p->tableSize, p->prices[posState], Pro= bPrices); p->counters[posState] =3D p->tableSize; } =20 -static void LenPriceEnc_UpdateTables(CLenPriceEnc *p, UInt32 numPosStates,= UInt32 *ProbPrices) +static void LenPriceEnc_UpdateTables(CLenPriceEnc *p, UInt32 numPosStates,= const UInt32 *ProbPrices) { UInt32 posState; for (posState =3D 0; posState < numPosStates; posState++) LenPriceEnc_UpdateTable(p, posState, ProbPrices); } =20 -static void LenEnc_Encode2(CLenPriceEnc *p, CRangeEnc *rc, UInt32 symbol, = UInt32 posState, Bool updatePrice, UInt32 *ProbPrices) +static void LenEnc_Encode2(CLenPriceEnc *p, CRangeEnc *rc, UInt32 symbol, = UInt32 posState, Bool updatePrice, const UInt32 *ProbPrices) { LenEnc_Encode(&p->p, rc, symbol, posState); if (updatePrice) @@ -837,9 +853,10 @@ static void LenEnc_Encode2(CLenPriceEnc *p, CRangeEnc = *rc, UInt32 symbol, UInt32 static void MovePos(CLzmaEnc *p, UInt32 num) { #ifdef SHOW_STAT - ttt +=3D num; - printf("\n MovePos %d", num); + g_STAT_OFFSET +=3D num; + printf("\n MovePos %u", num); #endif + =20 if (num !=3D 0) { p->additionalOffset +=3D num; @@ -852,28 +869,32 @@ static UInt32 ReadMatchDistances(CLzmaEnc *p, UInt32 = *numDistancePairsRes) UInt32 lenRes =3D 0, numPairs; p->numAvail =3D p->matchFinder.GetNumAvailableBytes(p->matchFinderObj); numPairs =3D p->matchFinder.GetMatches(p->matchFinderObj, p->matches); + =20 #ifdef SHOW_STAT - printf("\n i =3D %d numPairs =3D %d ", ttt, numPairs / 2); - ttt++; + printf("\n i =3D %u numPairs =3D %u ", g_STAT_OFFSET, numPairs / 2); + g_STAT_OFFSET++; { UInt32 i; for (i =3D 0; i < numPairs; i +=3D 2) - printf("%2d %6d | ", p->matches[i], p->matches[i + 1]); + printf("%2u %6u | ", p->matches[i], p->matches[i + 1]); } #endif + =20 if (numPairs > 0) { lenRes =3D p->matches[numPairs - 2]; if (lenRes =3D=3D p->numFastBytes) { - const Byte *pby =3D p->matchFinder.GetPointerToCurrentPos(p->matchFi= nderObj) - 1; - UInt32 distance =3D p->matches[numPairs - 1] + 1; UInt32 numAvail =3D p->numAvail; if (numAvail > LZMA_MATCH_LEN_MAX) numAvail =3D LZMA_MATCH_LEN_MAX; { - const Byte *pby2 =3D pby - distance; - for (; lenRes < numAvail && pby[lenRes] =3D=3D pby2[lenRes]; lenRe= s++); + const Byte *pbyCur =3D p->matchFinder.GetPointerToCurrentPos(p->ma= tchFinderObj) - 1; + const Byte *pby =3D pbyCur + lenRes; + ptrdiff_t dif =3D (ptrdiff_t)-1 - p->matches[numPairs - 1]; + const Byte *pbyLim =3D pbyCur + numAvail; + for (; pby !=3D pbyLim && *pby =3D=3D pby[dif]; pby++); + lenRes =3D (UInt32)(pby - pbyCur); } } } @@ -958,16 +979,21 @@ static UInt32 Backward(CLzmaEnc *p, UInt32 *backRes, = UInt32 cur) return p->optimumCurrentIndex; } =20 -#define LIT_PROBS(pos, prevByte) (p->litProbs + ((((pos) & p->lpMask) << p= ->lc) + ((prevByte) >> (8 - p->lc))) * 0x300) +#define LIT_PROBS(pos, prevByte) (p->litProbs + ((((pos) & p->lpMask) << p= ->lc) + ((prevByte) >> (8 - p->lc))) * (UInt32)0x300) =20 static UInt32 GetOptimum(CLzmaEnc *p, UInt32 position, UInt32 *backRes) { - UInt32 numAvail, mainLen, numPairs, repMaxIndex, i, posState, lenEnd, le= n, cur; - UInt32 matchPrice, repMatchPrice, normalMatchPrice; + UInt32 lenEnd, cur; UInt32 reps[LZMA_NUM_REPS], repLens[LZMA_NUM_REPS]; UInt32 *matches; + + { + + UInt32 numAvail, mainLen, numPairs, repMaxIndex, i, posState, len; + UInt32 matchPrice, repMatchPrice, normalMatchPrice; const Byte *data; Byte curByte, matchByte; + if (p->optimumEndIndex !=3D p->optimumCurrentIndex) { const COptimal *opt =3D &p->opt[p->optimumCurrentIndex]; @@ -1002,7 +1028,7 @@ static UInt32 GetOptimum(CLzmaEnc *p, UInt32 position= , UInt32 *backRes) UInt32 lenTest; const Byte *data2; reps[i] =3D p->reps[i]; - data2 =3D data - (reps[i] + 1); + data2 =3D data - reps[i] - 1; if (data[0] !=3D data2[0] || data[1] !=3D data2[1]) { repLens[i] =3D 0; @@ -1146,20 +1172,25 @@ static UInt32 GetOptimum(CLzmaEnc *p, UInt32 positi= on, UInt32 *backRes) cur =3D 0; =20 #ifdef SHOW_STAT2 - if (position >=3D 0) + /* if (position >=3D 0) */ { - unsigned ii; + unsigned i; printf("\n pos =3D %4X", position); - for (ii =3D cur; ii <=3D lenEnd; ii++) - printf("\nprice[%4X] =3D %d", position - cur + ii, p->opt[ii].price)= ; + for (i =3D cur; i <=3D lenEnd; i++) + printf("\nprice[%4X] =3D %u", position - cur + i, p->opt[i].price); } #endif =20 + } + for (;;) { - UInt32 numAvailFull, newLen, posPrev, state, startLen; - UInt32 curPrice, curAnd1Price; + UInt32 numAvail; + UInt32 numAvailFull, newLen, numPairs, posPrev, state, posState, start= Len; + UInt32 curPrice, curAnd1Price, matchPrice, repMatchPrice; Bool nextIsChar; + Byte curByte, matchByte; + const Byte *data; COptimal *curOpt; COptimal *nextOpt; =20 @@ -1222,6 +1253,7 @@ static UInt32 GetOptimum(CLzmaEnc *p, UInt32 position= , UInt32 *backRes) prevOpt =3D &p->opt[posPrev]; if (pos < LZMA_NUM_REPS) { + UInt32 i; reps[0] =3D prevOpt->backs[pos]; for (i =3D 1; i <=3D pos; i++) reps[i] =3D prevOpt->backs[i - 1]; @@ -1230,6 +1262,7 @@ static UInt32 GetOptimum(CLzmaEnc *p, UInt32 position= , UInt32 *backRes) } else { + UInt32 i; reps[0] =3D (pos - LZMA_NUM_REPS); for (i =3D 1; i < LZMA_NUM_REPS; i++) reps[i] =3D prevOpt->backs[i - 1]; @@ -1299,7 +1332,7 @@ static UInt32 GetOptimum(CLzmaEnc *p, UInt32 position= , UInt32 *backRes) /* try Literal + rep0 */ UInt32 temp; UInt32 lenTest2; - const Byte *data2 =3D data - (reps[0] + 1); + const Byte *data2 =3D data - reps[0] - 1; UInt32 limit =3D p->numFastBytes + 1; if (limit > numAvailFull) limit =3D numAvailFull; @@ -1315,11 +1348,12 @@ static UInt32 GetOptimum(CLzmaEnc *p, UInt32 positi= on, UInt32 *backRes) GET_PRICE_1(p->isRep[state2]); /* for (; lenTest2 >=3D 2; lenTest2--) */ { + UInt32 curAndLenPrice; COptimal *opt; UInt32 offset =3D cur + 1 + lenTest2; while (lenEnd < offset) p->opt[++lenEnd].price =3D kInfinityPrice; - UInt32 curAndLenPrice =3D nextRepMatchPrice + GetRepPrice(p, 0, = lenTest2, state2, posStateNext); + curAndLenPrice =3D nextRepMatchPrice + GetRepPrice(p, 0, lenTest= 2, state2, posStateNext); opt =3D &p->opt[offset]; if (curAndLenPrice < opt->price) { @@ -1341,7 +1375,7 @@ static UInt32 GetOptimum(CLzmaEnc *p, UInt32 position= , UInt32 *backRes) UInt32 lenTest; UInt32 lenTestTemp; UInt32 price; - const Byte *data2 =3D data - (reps[repIndex] + 1); + const Byte *data2 =3D data - reps[repIndex] - 1; if (data[0] !=3D data2[0] || data[1] !=3D data2[1]) continue; for (lenTest =3D 2; lenTest < numAvail && data[lenTest] =3D=3D data2= [lenTest]; lenTest++); @@ -1366,20 +1400,18 @@ static UInt32 GetOptimum(CLzmaEnc *p, UInt32 positi= on, UInt32 *backRes) =20 if (repIndex =3D=3D 0) startLen =3D lenTest + 1; - -#ifndef _MSC_VER - if (1 /* _maxMode */) -#endif + =20 + /* if (_maxMode) */ { UInt32 lenTest2 =3D lenTest + 1; UInt32 limit =3D lenTest2 + p->numFastBytes; - UInt32 nextRepMatchPrice; if (limit > numAvailFull) limit =3D numAvailFull; for (; lenTest2 < limit && data[lenTest2] =3D=3D data2[lenTest2]= ; lenTest2++); lenTest2 -=3D lenTest + 1; if (lenTest2 >=3D 2) { + UInt32 nextRepMatchPrice; UInt32 state2 =3D kRepNextStates[state]; UInt32 posStateNext =3D (position + lenTest) & p->pbMask; UInt32 curAndLenCharPrice =3D @@ -1427,7 +1459,7 @@ static UInt32 GetOptimum(CLzmaEnc *p, UInt32 position= , UInt32 *backRes) } if (newLen >=3D startLen) { - normalMatchPrice =3D matchPrice + GET_PRICE_0(p->isRep[state]); + UInt32 normalMatchPrice =3D matchPrice + GET_PRICE_0(p->isRep[state]= ); UInt32 offs, curBack, posSlot; UInt32 lenTest; while (lenEnd < cur + newLen) @@ -1440,8 +1472,8 @@ static UInt32 GetOptimum(CLzmaEnc *p, UInt32 position= , UInt32 *backRes) GetPosSlot2(curBack, posSlot); for (lenTest =3D /*2*/ startLen; ; lenTest++) { - UInt32 curAndLenPrice; - curAndLenPrice =3D normalMatchPrice + p->lenEnc.prices[posState][l= enTest - LZMA_MATCH_LEN_MIN]; + UInt32 curAndLenPrice =3D normalMatchPrice + p->lenEnc.prices[posS= tate][lenTest - LZMA_MATCH_LEN_MIN]; + { UInt32 lenToPosState =3D GetLenToPosState(lenTest); COptimal *opt; if (curBack < kNumFullDistances) @@ -1457,20 +1489,21 @@ static UInt32 GetOptimum(CLzmaEnc *p, UInt32 positi= on, UInt32 *backRes) opt->backPrev =3D curBack + LZMA_NUM_REPS; opt->prev1IsChar =3D False; } + } =20 if (/*_maxMode && */lenTest =3D=3D matches[offs]) { /* Try Match + Literal + Rep0 */ - const Byte *data2 =3D data - (curBack + 1); + const Byte *data2 =3D data - curBack - 1; UInt32 lenTest2 =3D lenTest + 1; UInt32 limit =3D lenTest2 + p->numFastBytes; - UInt32 nextRepMatchPrice; if (limit > numAvailFull) limit =3D numAvailFull; for (; lenTest2 < limit && data[lenTest2] =3D=3D data2[lenTest2]= ; lenTest2++); lenTest2 -=3D lenTest + 1; if (lenTest2 >=3D 2) { + UInt32 nextRepMatchPrice; UInt32 state2 =3D kMatchNextStates[state]; UInt32 posStateNext =3D (position + lenTest) & p->pbMask; UInt32 curAndLenCharPrice =3D curAndLenPrice + @@ -1486,14 +1519,15 @@ static UInt32 GetOptimum(CLzmaEnc *p, UInt32 positi= on, UInt32 *backRes) /* for (; lenTest2 >=3D 2; lenTest2--) */ { UInt32 offset =3D cur + lenTest + 1 + lenTest2; - + UInt32 curAndLenPrice2; + COptimal *opt; while (lenEnd < offset) p->opt[++lenEnd].price =3D kInfinityPrice; - curAndLenPrice =3D nextRepMatchPrice + GetRepPrice(p, 0, len= Test2, state2, posStateNext); + curAndLenPrice2 =3D nextRepMatchPrice + GetRepPrice(p, 0, le= nTest2, state2, posStateNext); opt =3D &p->opt[offset]; - if (curAndLenPrice < opt->price) + if (curAndLenPrice2 < opt->price) { - opt->price =3D curAndLenPrice; + opt->price =3D curAndLenPrice2; opt->posPrev =3D cur + lenTest + 1; opt->backPrev =3D 0; opt->prev1IsChar =3D True; @@ -1543,7 +1577,7 @@ static UInt32 GetOptimumFast(CLzmaEnc *p, UInt32 *bac= kRes) for (i =3D 0; i < LZMA_NUM_REPS; i++) { UInt32 len; - const Byte *data2 =3D data - (p->reps[i] + 1); + const Byte *data2 =3D data - p->reps[i] - 1; if (data[0] !=3D data2[0] || data[1] !=3D data2[1]) continue; for (len =3D 2; len < numAvail && data[len] =3D=3D data2[len]; len++); @@ -1612,7 +1646,7 @@ static UInt32 GetOptimumFast(CLzmaEnc *p, UInt32 *bac= kRes) for (i =3D 0; i < LZMA_NUM_REPS; i++) { UInt32 len, limit; - const Byte *data2 =3D data - (p->reps[i] + 1); + const Byte *data2 =3D data - p->reps[i] - 1; if (data[0] !=3D data2[0] || data[1] !=3D data2[1]) continue; limit =3D mainLen - 1; @@ -1694,7 +1728,6 @@ static void FillDistancesPrices(CLzmaEnc *p) =20 { UInt32 *distancesPrices =3D p->distancesPrices[lenToPosState]; - for (i =3D 0; i < kStartPosModelIndex; i++) distancesPrices[i] =3D posSlotPrices[i]; for (; i < kNumFullDistances; i++) @@ -1708,7 +1741,8 @@ void LzmaEnc_Construct(CLzmaEnc *p) { RangeEnc_Construct(&p->rc); MatchFinder_Construct(&p->matchFinderBase); - #ifdef COMPRESS_MF_MT + =20 + #ifndef _7ZIP_ST MatchFinderMt_Construct(&p->matchFinderMt); p->matchFinderMt.MatchFinder =3D &p->matchFinderBase; #endif @@ -1724,15 +1758,15 @@ void LzmaEnc_Construct(CLzmaEnc *p) #endif =20 LzmaEnc_InitPriceTables(p->ProbPrices); - p->litProbs =3D 0; - p->saveState.litProbs =3D 0; + p->litProbs =3D NULL; + p->saveState.litProbs =3D NULL; } =20 CLzmaEncHandle LzmaEnc_Create(ISzAlloc *alloc) { void *p; p =3D alloc->Alloc(alloc, sizeof(CLzmaEnc)); - if (p !=3D 0) + if (p) LzmaEnc_Construct((CLzmaEnc *)p); return p; } @@ -1741,15 +1775,16 @@ void LzmaEnc_FreeLits(CLzmaEnc *p, ISzAlloc *alloc) { alloc->Free(alloc, p->litProbs); alloc->Free(alloc, p->saveState.litProbs); - p->litProbs =3D 0; - p->saveState.litProbs =3D 0; + p->litProbs =3D NULL; + p->saveState.litProbs =3D NULL; } =20 void LzmaEnc_Destruct(CLzmaEnc *p, ISzAlloc *alloc, ISzAlloc *allocBig) { - #ifdef COMPRESS_MF_MT + #ifndef _7ZIP_ST MatchFinderMt_Destruct(&p->matchFinderMt, allocBig); #endif + =20 MatchFinder_Free(&p->matchFinderBase, allocBig); LzmaEnc_FreeLits(p, alloc); RangeEnc_Free(&p->rc, alloc); @@ -1764,11 +1799,10 @@ void LzmaEnc_Destroy(CLzmaEncHandle p, ISzAlloc *al= loc, ISzAlloc *allocBig) static SRes LzmaEnc_CodeOneBlock(CLzmaEnc *p, Bool useLimits, UInt32 maxPa= ckSize, UInt32 maxUnpackSize) { UInt32 nowPos32, startPos32; - if (p->inStream !=3D 0) + if (p->needInit) { - p->matchFinderBase.stream =3D p->inStream; p->matchFinder.Init(p->matchFinderObj); - p->inStream =3D 0; + p->needInit =3D 0; } =20 if (p->finished) @@ -1787,7 +1821,7 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc *p, Bool us= eLimits, UInt32 maxPackSize ReadMatchDistances(p, &numPairs); RangeEnc_EncodeBit(&p->rc, &p->isMatch[p->state][0], 0); p->state =3D kLiteralNextStates[p->state]; - curByte =3D p->matchFinder.GetIndexByte(p->matchFinderObj, 0 - p->addi= tionalOffset); + curByte =3D *(p->matchFinder.GetPointerToCurrentPos(p->matchFinderObj)= - p->additionalOffset); LitEnc_Encode(&p->rc, p->litProbs, curByte); p->additionalOffset--; nowPos32++; @@ -1804,7 +1838,7 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc *p, Bool us= eLimits, UInt32 maxPackSize len =3D GetOptimum(p, nowPos32, &pos); =20 #ifdef SHOW_STAT2 - printf("\n pos =3D %4X, len =3D %d pos =3D %d", nowPos32, len, pos= ); + printf("\n pos =3D %4X, len =3D %u pos =3D %u", nowPos32, len, pos= ); #endif =20 posState =3D nowPos32 & p->pbMask; @@ -1913,7 +1947,7 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc *p, Bool us= eLimits, UInt32 maxPackSize RangeEnc_GetProcessed(&p->rc) + kNumOpts * 2 >=3D maxPackSize) break; } - else if (processed >=3D (1 << 15)) + else if (processed >=3D (1 << 17)) { p->nowPos64 +=3D nowPos32 - startPos32; return CheckErrors(p); @@ -1929,24 +1963,21 @@ static SRes LzmaEnc_CodeOneBlock(CLzmaEnc *p, Bool = useLimits, UInt32 maxPackSize static SRes LzmaEnc_Alloc(CLzmaEnc *p, UInt32 keepWindowSize, ISzAlloc *al= loc, ISzAlloc *allocBig) { UInt32 beforeSize =3D kNumOpts; - #ifdef COMPRESS_MF_MT - Bool btMode; - #endif if (!RangeEnc_Alloc(&p->rc, alloc)) return SZ_ERROR_MEM; - #ifdef COMPRESS_MF_MT - btMode =3D (p->matchFinderBase.btMode !=3D 0); - p->mtMode =3D (p->multiThread && !p->fastMode && btMode); + + #ifndef _7ZIP_ST + p->mtMode =3D (p->multiThread && !p->fastMode && (p->matchFinderBase.btM= ode !=3D 0)); #endif =20 { unsigned lclp =3D p->lc + p->lp; - if (p->litProbs =3D=3D 0 || p->saveState.litProbs =3D=3D 0 || p->lclp = !=3D lclp) + if (!p->litProbs || !p->saveState.litProbs || p->lclp !=3D lclp) { LzmaEnc_FreeLits(p, alloc); - p->litProbs =3D (CLzmaProb *)alloc->Alloc(alloc, (0x300 << lclp) * s= izeof(CLzmaProb)); - p->saveState.litProbs =3D (CLzmaProb *)alloc->Alloc(alloc, (0x300 <<= lclp) * sizeof(CLzmaProb)); - if (p->litProbs =3D=3D 0 || p->saveState.litProbs =3D=3D 0) + p->litProbs =3D (CLzmaProb *)alloc->Alloc(alloc, ((UInt32)0x300 << l= clp) * sizeof(CLzmaProb)); + p->saveState.litProbs =3D (CLzmaProb *)alloc->Alloc(alloc, ((UInt32)= 0x300 << lclp) * sizeof(CLzmaProb)); + if (!p->litProbs || !p->saveState.litProbs) { LzmaEnc_FreeLits(p, alloc); return SZ_ERROR_MEM; @@ -1955,12 +1986,12 @@ static SRes LzmaEnc_Alloc(CLzmaEnc *p, UInt32 keepW= indowSize, ISzAlloc *alloc, I } } =20 - p->matchFinderBase.bigHash =3D (p->dictSize > kBigHashDicLimit); + p->matchFinderBase.bigHash =3D (Byte)(p->dictSize > kBigHashDicLimit ? 1= : 0); =20 if (beforeSize + p->dictSize < keepWindowSize) beforeSize =3D keepWindowSize - p->dictSize; =20 - #ifdef COMPRESS_MF_MT + #ifndef _7ZIP_ST if (p->mtMode) { RINOK(MatchFinderMt_Create(&p->matchFinderMt, p->dictSize, beforeSize,= p->numFastBytes, LZMA_MATCH_LEN_MAX, allocBig)); @@ -1975,6 +2006,7 @@ static SRes LzmaEnc_Alloc(CLzmaEnc *p, UInt32 keepWin= dowSize, ISzAlloc *alloc, I p->matchFinderObj =3D &p->matchFinderBase; MatchFinder_CreateVTable(&p->matchFinderBase, &p->matchFinder); } + =20 return SZ_OK; } =20 @@ -2003,9 +2035,10 @@ void LzmaEnc_Init(CLzmaEnc *p) } =20 { - UInt32 num =3D 0x300 << (p->lp + p->lc); + UInt32 num =3D (UInt32)0x300 << (p->lp + p->lc); + CLzmaProb *probs =3D p->litProbs; for (i =3D 0; i < num; i++) - p->litProbs[i] =3D kProbInitValue; + probs[i] =3D kProbInitValue; } =20 { @@ -2068,11 +2101,12 @@ static SRes LzmaEnc_AllocAndInit(CLzmaEnc *p, UInt3= 2 keepWindowSize, ISzAlloc *a return SZ_OK; } =20 -static SRes LzmaEnc_Prepare(CLzmaEncHandle pp, ISeqInStream *inStream, ISe= qOutStream *outStream, +static SRes LzmaEnc_Prepare(CLzmaEncHandle pp, ISeqOutStream *outStream, I= SeqInStream *inStream, ISzAlloc *alloc, ISzAlloc *allocBig) { CLzmaEnc *p =3D (CLzmaEnc *)pp; - p->inStream =3D inStream; + p->matchFinderBase.stream =3D inStream; + p->needInit =3D 1; p->rc.outStream =3D outStream; return LzmaEnc_AllocAndInit(p, 0, alloc, allocBig); } @@ -2082,15 +2116,16 @@ SRes LzmaEnc_PrepareForLzma2(CLzmaEncHandle pp, ISzAlloc *alloc, ISzAlloc *allocBig) { CLzmaEnc *p =3D (CLzmaEnc *)pp; - p->inStream =3D inStream; + p->matchFinderBase.stream =3D inStream; + p->needInit =3D 1; return LzmaEnc_AllocAndInit(p, keepWindowSize, alloc, allocBig); } =20 static void LzmaEnc_SetInputBuf(CLzmaEnc *p, const Byte *src, SizeT srcLen= ) { - p->seqBufInStream.funcTable.Read =3D MyRead; - p->seqBufInStream.data =3D src; - p->seqBufInStream.rem =3D srcLen; + p->matchFinderBase.directInput =3D 1; + p->matchFinderBase.bufferBase =3D (Byte *)src; + p->matchFinderBase.directInputRem =3D srcLen; } =20 SRes LzmaEnc_MemPrepare(CLzmaEncHandle pp, const Byte *src, SizeT srcLen, @@ -2098,22 +2133,24 @@ SRes LzmaEnc_MemPrepare(CLzmaEncHandle pp, const By= te *src, SizeT srcLen, { CLzmaEnc *p =3D (CLzmaEnc *)pp; LzmaEnc_SetInputBuf(p, src, srcLen); - p->inStream =3D &p->seqBufInStream.funcTable; + p->needInit =3D 1; + return LzmaEnc_AllocAndInit(p, keepWindowSize, alloc, allocBig); } =20 void LzmaEnc_Finish(CLzmaEncHandle pp) { - #ifdef COMPRESS_MF_MT + #ifndef _7ZIP_ST CLzmaEnc *p =3D (CLzmaEnc *)pp; if (p->mtMode) MatchFinderMt_ReleaseStream(&p->matchFinderMt); #else - (void)pp; + UNUSED_VAR(pp); #endif } =20 -typedef struct _CSeqOutStreamBuf + +typedef struct { ISeqOutStream funcTable; Byte *data; @@ -2142,12 +2179,14 @@ UInt32 LzmaEnc_GetNumAvailableBytes(CLzmaEncHandle = pp) return p->matchFinder.GetNumAvailableBytes(p->matchFinderObj); } =20 + const Byte *LzmaEnc_GetCurBuf(CLzmaEncHandle pp) { const CLzmaEnc *p =3D (CLzmaEnc *)pp; return p->matchFinder.GetPointerToCurrentPos(p->matchFinderObj) - p->add= itionalOffset; } =20 + SRes LzmaEnc_CodeOneMemBlock(CLzmaEncHandle pp, Bool reInit, Byte *dest, size_t *destLen, UInt32 desiredPackSize, UInt32 *unpackSiz= e) { @@ -2182,27 +2221,23 @@ SRes LzmaEnc_CodeOneMemBlock(CLzmaEncHandle pp, Boo= l reInit, return res; } =20 -SRes LzmaEnc_Encode(CLzmaEncHandle pp, ISeqOutStream *outStream, ISeqInStr= eam *inStream, ICompressProgress *progress, - ISzAlloc *alloc, ISzAlloc *allocBig) + +static SRes LzmaEnc_Encode2(CLzmaEnc *p, ICompressProgress *progress) { - CLzmaEnc *p =3D (CLzmaEnc *)pp; SRes res =3D SZ_OK; =20 - #ifdef COMPRESS_MF_MT + #ifndef _7ZIP_ST Byte allocaDummy[0x300]; - int i =3D 0; - for (i =3D 0; i < 16; i++) - allocaDummy[i] =3D (Byte)i; + allocaDummy[0] =3D 0; + allocaDummy[1] =3D allocaDummy[0]; #endif =20 - RINOK(LzmaEnc_Prepare(pp, inStream, outStream, alloc, allocBig)); - for (;;) { res =3D LzmaEnc_CodeOneBlock(p, False, 0, 0); - if (res !=3D SZ_OK || p->finished !=3D 0) + if (res !=3D SZ_OK || p->finished) break; - if (progress !=3D 0) + if (progress) { res =3D progress->Progress(progress, p->nowPos64, RangeEnc_GetProces= sed(&p->rc)); if (res !=3D SZ_OK) @@ -2212,32 +2247,47 @@ SRes LzmaEnc_Encode(CLzmaEncHandle pp, ISeqOutStrea= m *outStream, ISeqInStream *i } } } - LzmaEnc_Finish(pp); + =20 + LzmaEnc_Finish(p); + + /* + if (res =3D=3D S_OK && !Inline_MatchFinder_IsFinishedOK(&p->matchFinderB= ase)) + res =3D SZ_ERROR_FAIL; + } + */ + return res; } =20 + +SRes LzmaEnc_Encode(CLzmaEncHandle pp, ISeqOutStream *outStream, ISeqInStr= eam *inStream, ICompressProgress *progress, + ISzAlloc *alloc, ISzAlloc *allocBig) +{ + RINOK(LzmaEnc_Prepare(pp, outStream, inStream, alloc, allocBig)); + return LzmaEnc_Encode2((CLzmaEnc *)pp, progress); +} + + SRes LzmaEnc_WriteProperties(CLzmaEncHandle pp, Byte *props, SizeT *size) { CLzmaEnc *p =3D (CLzmaEnc *)pp; - int i; + unsigned i; UInt32 dictSize =3D p->dictSize; if (*size < LZMA_PROPS_SIZE) return SZ_ERROR_PARAM; *size =3D LZMA_PROPS_SIZE; props[0] =3D (Byte)((p->pb * 5 + p->lp) * 9 + p->lc); =20 - for (i =3D 11; i <=3D 30; i++) + if (dictSize >=3D ((UInt32)1 << 22)) { - if (dictSize <=3D ((UInt32)2 << i)) - { - dictSize =3D (2 << i); - break; - } - if (dictSize <=3D ((UInt32)3 << i)) - { - dictSize =3D (3 << i); - break; - } + UInt32 kDictMask =3D ((UInt32)1 << 20) - 1; + if (dictSize < (UInt32)0xFFFFFFFF - kDictMask) + dictSize =3D (dictSize + kDictMask) & ~kDictMask; + } + else for (i =3D 11; i <=3D 30; i++) + { + if (dictSize <=3D ((UInt32)2 << i)) { dictSize =3D (2 << i); break; } + if (dictSize <=3D ((UInt32)3 << i)) { dictSize =3D (3 << i); break; } } =20 for (i =3D 0; i < 4; i++) @@ -2245,6 +2295,7 @@ SRes LzmaEnc_WriteProperties(CLzmaEncHandle pp, Byte = *props, SizeT *size) return SZ_OK; } =20 + SRes LzmaEnc_MemEncode(CLzmaEncHandle pp, Byte *dest, SizeT *destLen, cons= t Byte *src, SizeT srcLen, int writeEndMark, ICompressProgress *progress, ISzAlloc *alloc, ISzAll= oc *allocBig) { @@ -2253,16 +2304,22 @@ SRes LzmaEnc_MemEncode(CLzmaEncHandle pp, Byte *des= t, SizeT *destLen, const Byte =20 CSeqOutStreamBuf outStream; =20 - LzmaEnc_SetInputBuf(p, src, srcLen); - outStream.funcTable.Write =3D MyWrite; outStream.data =3D dest; outStream.rem =3D *destLen; outStream.overflow =3D False; =20 p->writeEndMark =3D writeEndMark; - res =3D LzmaEnc_Encode(pp, &outStream.funcTable, &p->seqBufInStream.func= Table, - progress, alloc, allocBig); + p->rc.outStream =3D &outStream.funcTable; + + res =3D LzmaEnc_MemPrepare(pp, src, srcLen, 0, alloc, allocBig); + =20 + if (res =3D=3D SZ_OK) + { + res =3D LzmaEnc_Encode2(p, progress); + if (res =3D=3D SZ_OK && p->nowPos64 !=3D srcLen) + res =3D SZ_ERROR_FAIL; + } =20 *destLen -=3D outStream.rem; if (outStream.overflow) @@ -2270,13 +2327,14 @@ SRes LzmaEnc_MemEncode(CLzmaEncHandle pp, Byte *des= t, SizeT *destLen, const Byte return res; } =20 + SRes LzmaEncode(Byte *dest, SizeT *destLen, const Byte *src, SizeT srcLen, const CLzmaEncProps *props, Byte *propsEncoded, SizeT *propsSize, int = writeEndMark, ICompressProgress *progress, ISzAlloc *alloc, ISzAlloc *allocBig) { CLzmaEnc *p =3D (CLzmaEnc *)LzmaEnc_Create(alloc); SRes res; - if (p =3D=3D 0) + if (!p) return SZ_ERROR_MEM; =20 res =3D LzmaEnc_SetProps(p, props); diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.h b/BaseTools/So= urce/C/LzmaCompress/Sdk/C/LzmaEnc.h index e3d84fa..c2806b4 100644 --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.h +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/LzmaEnc.h @@ -1,10 +1,12 @@ /* LzmaEnc.h -- LZMA Encoder -2008-10-04 : Igor Pavlov : Public domain */ +2013-01-18 : Igor Pavlov : Public domain */ =20 -#ifndef __LZMAENC_H -#define __LZMAENC_H +#ifndef __LZMA_ENC_H +#define __LZMA_ENC_H =20 -#include "Types.h" +#include "7zTypes.h" + +EXTERN_C_BEGIN =20 #define LZMA_PROPS_SIZE 5 =20 @@ -14,6 +16,8 @@ typedef struct _CLzmaEncProps UInt32 dictSize; /* (1 << 12) <=3D dictSize <=3D (1 << 27) for 32-bit ve= rsion (1 << 12) <=3D dictSize <=3D (1 << 30) for 64-bit ve= rsion default =3D (1 << 24) */ + UInt64 reduceSize; /* estimated size of data that will be compressed. de= fault =3D 0xFFFFFFFF. + Encoder uses this value to reduce dictionary size = */ int lc; /* 0 <=3D lc <=3D 8, default =3D 3 */ int lp; /* 0 <=3D lp <=3D 4, default =3D 0 */ int pb; /* 0 <=3D pb <=3D 4, default =3D 2 */ @@ -69,4 +73,6 @@ SRes LzmaEncode(Byte *dest, SizeT *destLen, const Byte *s= rc, SizeT srcLen, const CLzmaEncProps *props, Byte *propsEncoded, SizeT *propsSize, int = writeEndMark, ICompressProgress *progress, ISzAlloc *alloc, ISzAlloc *allocBig); =20 +EXTERN_C_END + #endif diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/Precomp.h b/BaseTools/So= urce/C/LzmaCompress/Sdk/C/Precomp.h new file mode 100644 index 0000000..edb5814 --- /dev/null +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/Precomp.h @@ -0,0 +1,10 @@ +/* Precomp.h -- StdAfx +2013-11-12 : Igor Pavlov : Public domain */ + +#ifndef __7Z_PRECOMP_H +#define __7Z_PRECOMP_H + +#include "Compiler.h" +/* #include "7zTypes.h" */ + +#endif diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/Threads.c b/BaseTools/So= urce/C/LzmaCompress/Sdk/C/Threads.c index c7d8b0a..ece07e6 100644 --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/Threads.c +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/Threads.c @@ -1,10 +1,13 @@ /* Threads.c -- multithreading library -2008-08-05 -Igor Pavlov -Public domain */ +2014-09-21 : Igor Pavlov : Public domain */ =20 -#include "Threads.h" +#include "Precomp.h" + +#ifndef UNDER_CE #include +#endif + +#include "Threads.h" =20 static WRes GetError() { @@ -15,95 +18,76 @@ static WRes GetError() WRes HandleToWRes(HANDLE h) { return (h !=3D 0) ? 0 : GetError(); } WRes BOOLToWRes(BOOL v) { return v ? 0 : GetError(); } =20 -static WRes MyCloseHandle(HANDLE *h) +WRes HandlePtr_Close(HANDLE *p) { - if (*h !=3D NULL) - if (!CloseHandle(*h)) + if (*p !=3D NULL) + if (!CloseHandle(*p)) return GetError(); - *h =3D NULL; + *p =3D NULL; return 0; } =20 -WRes Thread_Create(CThread *thread, THREAD_FUNC_RET_TYPE (THREAD_FUNC_CALL= _TYPE *startAddress)(void *), LPVOID parameter) -{ - unsigned threadId; /* Windows Me/98/95: threadId parameter may not be NU= LL in _beginthreadex/CreateThread functions */ - thread->handle =3D - /* CreateThread(0, 0, startAddress, parameter, 0, &threadId); */ - (HANDLE)_beginthreadex(NULL, 0, startAddress, parameter, 0, &threadId)= ; - /* maybe we must use errno here, but probably GetLastError() is also O= K. */ - return HandleToWRes(thread->handle); -} +WRes Handle_WaitObject(HANDLE h) { return (WRes)WaitForSingleObject(h, INF= INITE); } =20 -WRes WaitObject(HANDLE h) +WRes Thread_Create(CThread *p, THREAD_FUNC_TYPE func, LPVOID param) { - return (WRes)WaitForSingleObject(h, INFINITE); + /* Windows Me/98/95: threadId parameter may not be NULL in _beginthreade= x/CreateThread functions */ + =20 + #ifdef UNDER_CE + =20 + DWORD threadId; + *p =3D CreateThread(0, 0, func, param, 0, &threadId); + + #else + + unsigned threadId; + *p =3D (HANDLE)_beginthreadex(NULL, 0, func, param, 0, &threadId); + =20 + #endif + + /* maybe we must use errno here, but probably GetLastError() is also OK.= */ + return HandleToWRes(*p); } =20 -WRes Thread_Wait(CThread *thread) +WRes Event_Create(CEvent *p, BOOL manualReset, int signaled) { - if (thread->handle =3D=3D NULL) - return 1; - return WaitObject(thread->handle); + *p =3D CreateEvent(NULL, manualReset, (signaled ? TRUE : FALSE), NULL); + return HandleToWRes(*p); } =20 -WRes Thread_Close(CThread *thread) -{ - return MyCloseHandle(&thread->handle); -} - -WRes Event_Create(CEvent *p, BOOL manualReset, int initialSignaled) -{ - p->handle =3D CreateEvent(NULL, manualReset, (initialSignaled ? TRUE : F= ALSE), NULL); - return HandleToWRes(p->handle); -} +WRes Event_Set(CEvent *p) { return BOOLToWRes(SetEvent(*p)); } +WRes Event_Reset(CEvent *p) { return BOOLToWRes(ResetEvent(*p)); } =20 -WRes ManualResetEvent_Create(CManualResetEvent *p, int initialSignaled) - { return Event_Create(p, TRUE, initialSignaled); } -WRes ManualResetEvent_CreateNotSignaled(CManualResetEvent *p) - { return ManualResetEvent_Create(p, 0); } +WRes ManualResetEvent_Create(CManualResetEvent *p, int signaled) { return = Event_Create(p, TRUE, signaled); } +WRes AutoResetEvent_Create(CAutoResetEvent *p, int signaled) { return Even= t_Create(p, FALSE, signaled); } +WRes ManualResetEvent_CreateNotSignaled(CManualResetEvent *p) { return Man= ualResetEvent_Create(p, 0); } +WRes AutoResetEvent_CreateNotSignaled(CAutoResetEvent *p) { return AutoRes= etEvent_Create(p, 0); } =20 -WRes AutoResetEvent_Create(CAutoResetEvent *p, int initialSignaled) - { return Event_Create(p, FALSE, initialSignaled); } -WRes AutoResetEvent_CreateNotSignaled(CAutoResetEvent *p) - { return AutoResetEvent_Create(p, 0); } =20 -WRes Event_Set(CEvent *p) { return BOOLToWRes(SetEvent(p->handle)); } -WRes Event_Reset(CEvent *p) { return BOOLToWRes(ResetEvent(p->handle)); } -WRes Event_Wait(CEvent *p) { return WaitObject(p->handle); } -WRes Event_Close(CEvent *p) { return MyCloseHandle(&p->handle); } - - -WRes Semaphore_Create(CSemaphore *p, UInt32 initiallyCount, UInt32 maxCoun= t) -{ - p->handle =3D CreateSemaphore(NULL, (LONG)initiallyCount, (LONG)maxCount= , NULL); - return HandleToWRes(p->handle); -} - -WRes Semaphore_Release(CSemaphore *p, LONG releaseCount, LONG *previousCou= nt) +WRes Semaphore_Create(CSemaphore *p, UInt32 initCount, UInt32 maxCount) { - return BOOLToWRes(ReleaseSemaphore(p->handle, releaseCount, previousCoun= t)); -} -WRes Semaphore_ReleaseN(CSemaphore *p, UInt32 releaseCount) -{ - return Semaphore_Release(p, (LONG)releaseCount, NULL); -} -WRes Semaphore_Release1(CSemaphore *p) -{ - return Semaphore_ReleaseN(p, 1); + *p =3D CreateSemaphore(NULL, (LONG)initCount, (LONG)maxCount, NULL); + return HandleToWRes(*p); } =20 -WRes Semaphore_Wait(CSemaphore *p) { return WaitObject(p->handle); } -WRes Semaphore_Close(CSemaphore *p) { return MyCloseHandle(&p->handle); } +static WRes Semaphore_Release(CSemaphore *p, LONG releaseCount, LONG *prev= iousCount) + { return BOOLToWRes(ReleaseSemaphore(*p, releaseCount, previousCount)); = } +WRes Semaphore_ReleaseN(CSemaphore *p, UInt32 num) + { return Semaphore_Release(p, (LONG)num, NULL); } +WRes Semaphore_Release1(CSemaphore *p) { return Semaphore_ReleaseN(p, 1); = } =20 WRes CriticalSection_Init(CCriticalSection *p) { /* InitializeCriticalSection can raise only STATUS_NO_MEMORY exception *= / + #ifdef _MSC_VER __try + #endif { InitializeCriticalSection(p); /* InitializeCriticalSectionAndSpinCount(p, 0); */ } + #ifdef _MSC_VER __except (EXCEPTION_EXECUTE_HANDLER) { return 1; } + #endif return 0; } - diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/Threads.h b/BaseTools/So= urce/C/LzmaCompress/Sdk/C/Threads.h index 3d90729..e927208 100644 --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/Threads.h +++ b/BaseTools/Source/C/LzmaCompress/Sdk/C/Threads.h @@ -1,68 +1,67 @@ /* Threads.h -- multithreading library -2008-11-22 : Igor Pavlov : Public domain */ +2013-11-12 : Igor Pavlov : Public domain */ =20 -#ifndef __7Z_THRESDS_H -#define __7Z_THRESDS_H +#ifndef __7Z_THREADS_H +#define __7Z_THREADS_H =20 -#include "Types.h" - -typedef struct _CThread -{ - HANDLE handle; -} CThread; - -#define Thread_Construct(thread) (thread)->handle =3D NULL -#define Thread_WasCreated(thread) ((thread)->handle !=3D NULL) -=20 -typedef unsigned THREAD_FUNC_RET_TYPE; -#define THREAD_FUNC_CALL_TYPE MY_STD_CALL -#define THREAD_FUNC_DECL THREAD_FUNC_RET_TYPE THREAD_FUNC_CALL_TYPE - -WRes Thread_Create(CThread *thread, THREAD_FUNC_RET_TYPE (THREAD_FUNC_CALL= _TYPE *startAddress)(void *), LPVOID parameter); -WRes Thread_Wait(CThread *thread); -WRes Thread_Close(CThread *thread); - -typedef struct _CEvent -{ - HANDLE handle; -} CEvent; +#ifdef _WIN32 +#include +#endif =20 -typedef CEvent CAutoResetEvent; -typedef CEvent CManualResetEvent; +#include "7zTypes.h" =20 -#define Event_Construct(event) (event)->handle =3D NULL -#define Event_IsCreated(event) ((event)->handle !=3D NULL) +EXTERN_C_BEGIN =20 -WRes ManualResetEvent_Create(CManualResetEvent *event, int initialSignaled= ); -WRes ManualResetEvent_CreateNotSignaled(CManualResetEvent *event); -WRes AutoResetEvent_Create(CAutoResetEvent *event, int initialSignaled); -WRes AutoResetEvent_CreateNotSignaled(CAutoResetEvent *event); -WRes Event_Set(CEvent *event); -WRes Event_Reset(CEvent *event); -WRes Event_Wait(CEvent *event); -WRes Event_Close(CEvent *event); +WRes HandlePtr_Close(HANDLE *h); +WRes Handle_WaitObject(HANDLE h); =20 +typedef HANDLE CThread; +#define Thread_Construct(p) *(p) =3D NULL +#define Thread_WasCreated(p) (*(p) !=3D NULL) +#define Thread_Close(p) HandlePtr_Close(p) +#define Thread_Wait(p) Handle_WaitObject(*(p)) =20 -typedef struct _CSemaphore -{ - HANDLE handle; -} CSemaphore; +typedef +#ifdef UNDER_CE + DWORD +#else + unsigned +#endif + THREAD_FUNC_RET_TYPE; =20 -#define Semaphore_Construct(p) (p)->handle =3D NULL +#define THREAD_FUNC_CALL_TYPE MY_STD_CALL +#define THREAD_FUNC_DECL THREAD_FUNC_RET_TYPE THREAD_FUNC_CALL_TYPE +typedef THREAD_FUNC_RET_TYPE (THREAD_FUNC_CALL_TYPE * THREAD_FUNC_TYPE)(vo= id *); +WRes Thread_Create(CThread *p, THREAD_FUNC_TYPE func, LPVOID param); =20 -WRes Semaphore_Create(CSemaphore *p, UInt32 initiallyCount, UInt32 maxCoun= t); +typedef HANDLE CEvent; +typedef CEvent CAutoResetEvent; +typedef CEvent CManualResetEvent; +#define Event_Construct(p) *(p) =3D NULL +#define Event_IsCreated(p) (*(p) !=3D NULL) +#define Event_Close(p) HandlePtr_Close(p) +#define Event_Wait(p) Handle_WaitObject(*(p)) +WRes Event_Set(CEvent *p); +WRes Event_Reset(CEvent *p); +WRes ManualResetEvent_Create(CManualResetEvent *p, int signaled); +WRes ManualResetEvent_CreateNotSignaled(CManualResetEvent *p); +WRes AutoResetEvent_Create(CAutoResetEvent *p, int signaled); +WRes AutoResetEvent_CreateNotSignaled(CAutoResetEvent *p); + +typedef HANDLE CSemaphore; +#define Semaphore_Construct(p) (*p) =3D NULL +#define Semaphore_Close(p) HandlePtr_Close(p) +#define Semaphore_Wait(p) Handle_WaitObject(*(p)) +WRes Semaphore_Create(CSemaphore *p, UInt32 initCount, UInt32 maxCount); WRes Semaphore_ReleaseN(CSemaphore *p, UInt32 num); WRes Semaphore_Release1(CSemaphore *p); -WRes Semaphore_Wait(CSemaphore *p); -WRes Semaphore_Close(CSemaphore *p); - =20 typedef CRITICAL_SECTION CCriticalSection; - WRes CriticalSection_Init(CCriticalSection *p); #define CriticalSection_Delete(p) DeleteCriticalSection(p) #define CriticalSection_Enter(p) EnterCriticalSection(p) #define CriticalSection_Leave(p) LeaveCriticalSection(p) =20 -#endif +EXTERN_C_END =20 +#endif diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/C/Types.h b/BaseTools/Sour= ce/C/LzmaCompress/Sdk/C/Types.h deleted file mode 100644 index 2638196..0000000 --- a/BaseTools/Source/C/LzmaCompress/Sdk/C/Types.h +++ /dev/null @@ -1,208 +0,0 @@ -/* Types.h -- Basic types -2008-11-23 : Igor Pavlov : Public domain */ - -#ifndef __7Z_TYPES_H -#define __7Z_TYPES_H - -#include - -#ifdef _WIN32 -#include -#endif - -#define SZ_OK 0 - -#define SZ_ERROR_DATA 1 -#define SZ_ERROR_MEM 2 -#define SZ_ERROR_CRC 3 -#define SZ_ERROR_UNSUPPORTED 4 -#define SZ_ERROR_PARAM 5 -#define SZ_ERROR_INPUT_EOF 6 -#define SZ_ERROR_OUTPUT_EOF 7 -#define SZ_ERROR_READ 8 -#define SZ_ERROR_WRITE 9 -#define SZ_ERROR_PROGRESS 10 -#define SZ_ERROR_FAIL 11 -#define SZ_ERROR_THREAD 12 - -#define SZ_ERROR_ARCHIVE 16 -#define SZ_ERROR_NO_ARCHIVE 17 - -typedef int SRes; - -#ifdef _WIN32 -typedef DWORD WRes; -#else -typedef int WRes; -#endif - -#ifndef RINOK -#define RINOK(x) { int __result__ =3D (x); if (__result__ !=3D 0) return _= _result__; } -#endif - -typedef unsigned char Byte; -typedef short Int16; -typedef unsigned short UInt16; - -#ifdef _LZMA_UINT32_IS_ULONG -typedef long Int32; -typedef unsigned long UInt32; -#else -typedef int Int32; -typedef unsigned int UInt32; -#endif - -#ifdef _SZ_NO_INT_64 - -/* define _SZ_NO_INT_64, if your compiler doesn't support 64-bit integers. - NOTES: Some code will work incorrectly in that case! */ - -typedef long Int64; -typedef unsigned long UInt64; - -#else - -#if defined(_MSC_VER) || defined(__BORLANDC__) -typedef __int64 Int64; -typedef unsigned __int64 UInt64; -#else -typedef long long int Int64; -typedef unsigned long long int UInt64; -#endif - -#endif - -#ifdef _LZMA_NO_SYSTEM_SIZE_T -typedef UInt32 SizeT; -#else -typedef size_t SizeT; -#endif - -typedef int Bool; -#define True 1 -#define False 0 - - -#ifdef _MSC_VER - -#if _MSC_VER >=3D 1300 -#define MY_NO_INLINE __declspec(noinline) -#else -#define MY_NO_INLINE -#endif - -#define MY_CDECL __cdecl -#define MY_STD_CALL __stdcall -#define MY_FAST_CALL MY_NO_INLINE __fastcall - -#else - -#define MY_CDECL -#define MY_STD_CALL -#define MY_FAST_CALL - -#endif - - -/* The following interfaces use first parameter as pointer to structure */ - -typedef struct -{ - SRes (*Read)(void *p, void *buf, size_t *size); - /* if (input(*size) !=3D 0 && output(*size) =3D=3D 0) means end_of_str= eam. - (output(*size) < input(*size)) is allowed */ -} ISeqInStream; - -/* it can return SZ_ERROR_INPUT_EOF */ -SRes SeqInStream_Read(ISeqInStream *stream, void *buf, size_t size); -SRes SeqInStream_Read2(ISeqInStream *stream, void *buf, size_t size, SRes = errorType); -SRes SeqInStream_ReadByte(ISeqInStream *stream, Byte *buf); - -typedef struct -{ - size_t (*Write)(void *p, const void *buf, size_t size); - /* Returns: result - the number of actually written bytes. - (result < size) means error */ -} ISeqOutStream; - -typedef enum -{ - SZ_SEEK_SET =3D 0, - SZ_SEEK_CUR =3D 1, - SZ_SEEK_END =3D 2 -} ESzSeek; - -typedef struct -{ - SRes (*Read)(void *p, void *buf, size_t *size); /* same as ISeqInStream= ::Read */ - SRes (*Seek)(void *p, Int64 *pos, ESzSeek origin); -} ISeekInStream; - -typedef struct -{ - SRes (*Look)(void *p, void **buf, size_t *size); - /* if (input(*size) !=3D 0 && output(*size) =3D=3D 0) means end_of_str= eam. - (output(*size) > input(*size)) is not allowed - (output(*size) < input(*size)) is allowed */ - SRes (*Skip)(void *p, size_t offset); - /* offset must be <=3D output(*size) of Look */ - - SRes (*Read)(void *p, void *buf, size_t *size); - /* reads directly (without buffer). It's same as ISeqInStream::Read */ - SRes (*Seek)(void *p, Int64 *pos, ESzSeek origin); -} ILookInStream; - -SRes LookInStream_LookRead(ILookInStream *stream, void *buf, size_t *size)= ; -SRes LookInStream_SeekTo(ILookInStream *stream, UInt64 offset); - -/* reads via ILookInStream::Read */ -SRes LookInStream_Read2(ILookInStream *stream, void *buf, size_t size, SRe= s errorType); -SRes LookInStream_Read(ILookInStream *stream, void *buf, size_t size); - -#define LookToRead_BUF_SIZE (1 << 14) - -typedef struct -{ - ILookInStream s; - ISeekInStream *realStream; - size_t pos; - size_t size; - Byte buf[LookToRead_BUF_SIZE]; -} CLookToRead; - -void LookToRead_CreateVTable(CLookToRead *p, int lookahead); -void LookToRead_Init(CLookToRead *p); - -typedef struct -{ - ISeqInStream s; - ILookInStream *realStream; -} CSecToLook; - -void SecToLook_CreateVTable(CSecToLook *p); - -typedef struct -{ - ISeqInStream s; - ILookInStream *realStream; -} CSecToRead; - -void SecToRead_CreateVTable(CSecToRead *p); - -typedef struct -{ - SRes (*Progress)(void *p, UInt64 inSize, UInt64 outSize); - /* Returns: result. (result !=3D SZ_OK) means break. - Value (UInt64)(Int64)-1 for size means unknown value. */ -} ICompressProgress; - -typedef struct -{ - void *(*Alloc)(void *p, size_t size); - void (*Free)(void *p, void *address); /* address can be 0 */ -} ISzAlloc; - -#define IAlloc_Alloc(p, size) (p)->Alloc((p), size) -#define IAlloc_Free(p, a) (p)->Free((p), a) - -#endif diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-history.txt b/Bas= eTools/Source/C/LzmaCompress/Sdk/DOC/lzma-history.txt new file mode 100644 index 0000000..7aaeb07 --- /dev/null +++ b/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-history.txt @@ -0,0 +1,363 @@ +HISTORY of the LZMA SDK +----------------------- + +16.04 2016-10-04 +------------------------- +- The bug was fixed in DllSecur.c. + + +16.03 2016-09-28 +------------------------- +- SFX modules now use some protection against DLL preloading attack. +- Some bugs in 7z code were fixed. + + +16.02 2016-05-21 +------------------------- +- The BUG in 16.00 - 16.01 was fixed: + Split Handler (SplitHandler.cpp) returned incorrect=20 + total size value (kpidSize) for split archives. + + +16.01 2016-05-19 +-------------------------=09 +- Some internal changes to reduce the number of compiler warnings. + + +16.00 2016-05-10 +-------------------------=09 +- Some bugs were fixed. + + +15.12 2015-11-19 +-------------------------=09 +- The BUG in C version of 7z decoder was fixed: + 7zDec.c : SzDecodeLzma2() + 7z decoder could mistakenly report about decoding error for some 7z arch= ives + that use LZMA2 compression method. + The probability to get that mistaken decoding error report was about=20 + one error per 16384 solid blocks for solid blocks larger than 16 KB (com= pressed size).=20 +- The BUG (in 9.26-15.11) in C version of 7z decoder was fixed: + 7zArcIn.c : SzReadHeader2() + 7z decoder worked incorrectly for 7z archives that contain=20 + empty solid blocks, that can be placed to 7z archive, if some file is=20 + unavailable for reading during archive creation. + + +15.09 beta 2015-10-16 +-------------------------=09 +- The BUG in LZMA / LZMA2 encoding code was fixed. + The BUG in LzFind.c::MatchFinder_ReadBlock() function. + If input data size is larger than (4 GiB - dictionary_size), + the following code worked incorrectly: + - LZMA : LzmaEnc_MemEncode(), LzmaEncode() : LZMA encoding functions=20 + for compressing from memory to memory.=20 + That BUG is not related to LZMA encoder version that works via stream= s. + - LZMA2 : multi-threaded version of LZMA2 encoder worked incorrectly, i= f=20 + default value of chunk size (CLzma2EncProps::blockSize) is changed=20 + to value larger than (4 GiB - dictionary_size). + + +9.38 beta 2015-01-03 +-------------------------=09 +- The BUG in 9.31-9.37 was fixed: + IArchiveGetRawProps interface was disabled for 7z archives. +- The BUG in 9.26-9.36 was fixed: + Some code in CPP\7zip\Archive\7z\ worked correctly only under Windows. + + +9.36 beta 2014-12-26 +-------------------------=09 +- The BUG in command line version was fixed: + 7-Zip created temporary archive in current folder during update archive + operation, if -w{Path} switch was not specified.=20 + The fixed 7-Zip creates temporary archive in folder that contains update= d archive. +- The BUG in 9.33-9.35 was fixed: + 7-Zip silently ignored file reading errors during 7z or gz archive creat= ion, + and the created archive contained only part of file that was read before= error. + The fixed 7-Zip stops archive creation and it reports about error. + + +9.35 beta 2014-12-07 +-------------------------=09 +- 7zr.exe now support AES encryption. +- SFX mudules were added to LZMA SDK +- Some bugs were fixed. + + +9.21 beta 2011-04-11 +-------------------------=09 +- New class FString for file names at file systems. +- Speed optimization in CRC code for big-endian CPUs. +- The BUG in Lzma2Dec.c was fixed: + Lzma2Decode function didn't work. + + +9.18 beta 2010-11-02 +-------------------------=09 +- New small SFX module for installers (SfxSetup). + + +9.12 beta 2010-03-24 +------------------------- +- The BUG in LZMA SDK 9.* was fixed: LZMA2 codec didn't work, + if more than 10 threads were used (or more than 20 threads in some modes= ). + + +9.11 beta 2010-03-15 +------------------------- +- PPMd compression method support + =20 + +9.09 2009-12-12 +------------------------- +- The bug was fixed: + Utf16_To_Utf8 funstions in UTFConvert.cpp and 7zMain.c + incorrectly converted surrogate characters (the code >=3D 0x10000) to U= TF-8. +- Some bugs were fixed + + +9.06 2009-08-17 +------------------------- +- Some changes in ANSI-C 7z Decoder interfaces. + + +9.04 2009-05-30 +------------------------- +- LZMA2 compression method support +- xz format support + + +4.65 2009-02-03 +------------------------- +- Some minor fixes + + +4.63 2008-12-31 +------------------------- +- Some minor fixes + + +4.61 beta 2008-11-23 +------------------------- +- The bug in ANSI-C LZMA Decoder was fixed: + If encoded stream was corrupted, decoder could access memory=20 + outside of allocated range. +- Some changes in ANSI-C 7z Decoder interfaces. +- LZMA SDK is placed in the public domain. + + +4.60 beta 2008-08-19 +------------------------- +- Some minor fixes. + + +4.59 beta 2008-08-13 +------------------------- +- The bug was fixed: + LZMA Encoder in fast compression mode could access memory outside of=20 + allocated range in some rare cases. + + +4.58 beta 2008-05-05 +------------------------- +- ANSI-C LZMA Decoder was rewritten for speed optimizations. +- ANSI-C LZMA Encoder was included to LZMA SDK. +- C++ LZMA code now is just wrapper over ANSI-C code. + + +4.57 2007-12-12 +------------------------- +- Speed optimizations in C++ LZMA Decoder.=20 +- Small changes for more compatibility with some C/C++ compilers. + + +4.49 beta 2007-07-05 +------------------------- +- .7z ANSI-C Decoder: + - now it supports BCJ and BCJ2 filters + - now it supports files larger than 4 GB. + - now it supports "Last Write Time" field for files. +- C++ code for .7z archives compressing/decompressing from 7-zip=20 + was included to LZMA SDK. + =20 + +4.43 2006-06-04 +------------------------- +- Small changes for more compatibility with some C/C++ compilers. + =20 + +4.42 2006-05-15 +------------------------- +- Small changes in .h files in ANSI-C version. + =20 + +4.39 beta 2006-04-14 +------------------------- +- The bug in versions 4.33b:4.38b was fixed: + C++ version of LZMA encoder could not correctly compress=20 + files larger than 2 GB with HC4 match finder (-mfhc4). + =20 + +4.37 beta 2005-04-06 +------------------------- +- Fixes in C++ code: code could no be compiled if _NO_EXCEPTIONS was defin= ed.=20 + + +4.35 beta 2005-03-02 +------------------------- +- The bug was fixed in C++ version of LZMA Decoder: + If encoded stream was corrupted, decoder could access memory=20 + outside of allocated range. + + +4.34 beta 2006-02-27 +------------------------- +- Compressing speed and memory requirements for compressing were increased +- LZMA now can use only these match finders: HC4, BT2, BT3, BT4 + + +4.32 2005-12-09 +------------------------- +- Java version of LZMA SDK was included + + +4.30 2005-11-20 +------------------------- +- Compression ratio was improved in -a2 mode +- Speed optimizations for compressing in -a2 mode +- -fb switch now supports values up to 273 +- The bug in 7z_C (7zIn.c) was fixed: + It used Alloc/Free functions from different memory pools. + So if program used two memory pools, it worked incorrectly. +- 7z_C: .7z format supporting was improved +- LZMA# SDK (C#.NET version) was included + + +4.27 (Updated) 2005-09-21 +------------------------- +- Some GUIDs/interfaces in C++ were changed. + IStream.h: + ISequentialInStream::Read now works as old ReadPart + ISequentialOutStream::Write now works as old WritePart + + +4.27 2005-08-07 +------------------------- +- The bug in LzmaDecodeSize.c was fixed: + if _LZMA_IN_CB and _LZMA_OUT_READ were defined, + decompressing worked incorrectly. + + +4.26 2005-08-05 +------------------------- +- Fixes in 7z_C code and LzmaTest.c: + previous versions could work incorrectly, + if malloc(0) returns 0 + + +4.23 2005-06-29 +------------------------- +- Small fixes in C++ code + + +4.22 2005-06-10 +------------------------- +- Small fixes + + +4.21 2005-06-08 +------------------------- +- Interfaces for ANSI-C LZMA Decoder (LzmaDecode.c) were changed +- New additional version of ANSI-C LZMA Decoder with zlib-like interface: + - LzmaStateDecode.h + - LzmaStateDecode.c + - LzmaStateTest.c +- ANSI-C LZMA Decoder now can decompress files larger than 4 GB + + +4.17 2005-04-18 +------------------------- +- New example for RAM->RAM compressing/decompressing:=20 + LZMA + BCJ (filter for x86 code): + - LzmaRam.h + - LzmaRam.cpp + - LzmaRamDecode.h + - LzmaRamDecode.c + - -f86 switch for lzma.exe + + +4.16 2005-03-29 +------------------------- +- The bug was fixed in LzmaDecode.c (ANSI-C LZMA Decoder):=20 + If _LZMA_OUT_READ was defined, and if encoded stream was corrupted, + decoder could access memory outside of allocated range. +- Speed optimization of ANSI-C LZMA Decoder (now it's about 20% faster). + Old version of LZMA Decoder now is in file LzmaDecodeSize.c.=20 + LzmaDecodeSize.c can provide slightly smaller code than LzmaDecode.c +- Small speed optimization in LZMA C++ code +- filter for SPARC's code was added +- Simplified version of .7z ANSI-C Decoder was included + + +4.06 2004-09-05 +------------------------- +- The bug in v4.05 was fixed: + LZMA-Encoder didn't release output stream in some cases. + + +4.05 2004-08-25 +------------------------- +- Source code of filters for x86, IA-64, ARM, ARM-Thumb=20 + and PowerPC code was included to SDK +- Some internal minor changes + + +4.04 2004-07-28 +------------------------- +- More compatibility with some C++ compilers + + +4.03 2004-06-18 +------------------------- +- "Benchmark" command was added. It measures compressing=20 + and decompressing speed and shows rating values.=20 + Also it checks hardware errors. + + +4.02 2004-06-10 +------------------------- +- C++ LZMA Encoder/Decoder code now is more portable + and it can be compiled by GCC on Linux. + + +4.01 2004-02-15 +------------------------- +- Some detection of data corruption was enabled. + LzmaDecode.c / RangeDecoderReadByte + ..... + { + rd->ExtraBytes =3D 1; + return 0xFF; + } + + +4.00 2004-02-13 +------------------------- +- Original version of LZMA SDK + + + +HISTORY of the LZMA +------------------- + 2001-2008: Improvements to LZMA compressing/decompressing code,=20 + keeping compatibility with original LZMA format + 1996-2001: Development of LZMA compression format + + Some milestones: + + 2001-08-30: LZMA compression was added to 7-Zip + 1999-01-02: First version of 7-Zip was released + =20 + +End of document diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-sdk.txt b/BaseToo= ls/Source/C/LzmaCompress/Sdk/DOC/lzma-sdk.txt new file mode 100644 index 0000000..86fef24 --- /dev/null +++ b/BaseTools/Source/C/LzmaCompress/Sdk/DOC/lzma-sdk.txt @@ -0,0 +1,357 @@ +LZMA SDK 16.04 +-------------- + +LZMA SDK provides the documentation, samples, header files, +libraries, and tools you need to develop applications that=20 +use 7z / LZMA / LZMA2 / XZ compression. + +LZMA is an improved version of famous LZ77 compression algorithm.=20 +It was improved in way of maximum increasing of compression ratio, +keeping high decompression speed and low memory requirements for=20 +decompressing. + +LZMA2 is a LZMA based compression method. LZMA2 provides better=20 +multithreading support for compression than LZMA and some other improvemen= ts. + +7z is a file format for data compression and file archiving. +7z is a main file format for 7-Zip compression program (www.7-zip.org). +7z format supports different compression methods: LZMA, LZMA2 and others. +7z also supports AES-256 based encryption. + +XZ is a file format for data compression that uses LZMA2 compression. +XZ format provides additional features: SHA/CRC check, filters for=20 +improved compression ratio, splitting to blocks and streams, + + + +LICENSE +------- + +LZMA SDK is written and placed in the public domain by Igor Pavlov. + +Some code in LZMA SDK is based on public domain code from another develope= rs: + 1) PPMd var.H (2001): Dmitry Shkarin + 2) SHA-256: Wei Dai (Crypto++ library) + +Anyone is free to copy, modify, publish, use, compile, sell, or distribute= the=20 +original LZMA SDK code, either in source code form or as a compiled binary= , for=20 +any purpose, commercial or non-commercial, and by any means. + +LZMA SDK code is compatible with open source licenses, for example, you ca= n=20 +include it to GNU GPL or GNU LGPL code. + + +LZMA SDK Contents +----------------- + + Source code: + + - C / C++ / C# / Java - LZMA compression and decompression + - C / C++ - LZMA2 compression and decompression + - C / C++ - XZ compression and decompression + - C - 7z decompression + - C++ - 7z compression and decompression + - C - small SFXs for installers (7z decompression) + - C++ - SFXs and SFXs for installers (7z decompressi= on) + + Precomiled binaries: + + - console programs for lzma / 7z / xz compression and decompression + - SFX modules for installers. + + +UNIX/Linux version=20 +------------------ +To compile C++ version of file->file LZMA encoding, go to directory +CPP/7zip/Bundles/LzmaCon +and call make to recompile it: + make -f makefile.gcc clean all + +In some UNIX/Linux versions you must compile LZMA with static libraries. +To compile with static libraries, you can use=20 +LIB =3D -lm -static + +Also you can use p7zip (port of 7-Zip for POSIX systems like Unix or Linux= ): + =20 + http://p7zip.sourceforge.net/ + + +Files +----- + +DOC/7zC.txt - 7z ANSI-C Decoder description +DOC/7zFormat.txt - 7z Format description +DOC/installer.txt - information about 7-Zip for installers +DOC/lzma.txt - LZMA compression description +DOC/lzma-sdk.txt - LZMA SDK description (this file) +DOC/lzma-history.txt - history of LZMA SDK +DOC/lzma-specification.txt - Specification of LZMA +DOC/Methods.txt - Compression method IDs for .7z + +bin/installer/ - example script to create installer that uses SFX module= , + +bin/7zdec.exe - simplified 7z archive decoder +bin/7zr.exe - 7-Zip console program (reduced version) +bin/x64/7zr.exe - 7-Zip console program (reduced version) (x64 version) +bin/lzma.exe - file->file LZMA encoder/decoder for Windows +bin/7zS2.sfx - small SFX module for installers (GUI version) +bin/7zS2con.sfx - small SFX module for installers (Console version) +bin/7zSD.sfx - SFX module for installers. + + +7zDec.exe +--------- +7zDec.exe is simplified 7z archive decoder. +It supports only LZMA, LZMA2, and PPMd methods. +7zDec decodes whole solid block from 7z archive to RAM. +The RAM consumption can be high. + + + + +Source code structure +--------------------- + + +Asm/ - asm files (optimized code for CRC calculation and Intel-AES encrypt= ion) + +C/ - C files (compression / decompression and other) + Util/ + 7z - 7z decoder program (decoding 7z files) + Lzma - LZMA program (file->file LZMA encoder/decoder). + LzmaLib - LZMA library (.DLL for Windows) + SfxSetup - small SFX module for installers=20 + +CPP/ -- CPP files + + Common - common files for C++ projects + Windows - common files for Windows related code + + 7zip - files related to 7-Zip + + Archive - files related to archiving + + Common - common files for archive handling + 7z - 7z C++ Encoder/Decoder + + Bundles - Modules that are bundles of other modules (files) + =20 + Alone7z - 7zr.exe: Standalone 7-Zip console program (reduced v= ersion) + Format7zExtractR - 7zxr.dll: Reduced version of 7z DLL: extracting = from 7z/LZMA/BCJ/BCJ2. + Format7zR - 7zr.dll: Reduced version of 7z DLL: extracting/= compressing to 7z/LZMA/BCJ/BCJ2 + LzmaCon - lzma.exe: LZMA compression/decompression + LzmaSpec - example code for LZMA Specification + SFXCon - 7zCon.sfx: Console 7z SFX module + SFXSetup - 7zS.sfx: 7z SFX module for installers + SFXWin - 7z.sfx: GUI 7z SFX module + + Common - common files for 7-Zip + + Compress - files for compression/decompression + + Crypto - files for encryption / decompression + + UI - User Interface files + =20 + Client7z - Test application for 7za.dll, 7zr.dll, 7zxr.dll + Common - Common UI files + Console - Code for console program (7z.exe) + Explorer - Some code from 7-Zip Shell extension + FileManager - Some GUI code from 7-Zip File Manager + GUI - Some GUI code from 7-Zip + + +CS/ - C# files + 7zip + Common - some common files for 7-Zip + Compress - files related to compression/decompression + LZ - files related to LZ (Lempel-Ziv) compression algorithm + LZMA - LZMA compression/decompression + LzmaAlone - file->file LZMA compression/decompression + RangeCoder - Range Coder (special code of compression/decompressio= n) + +Java/ - Java files + SevenZip + Compression - files related to compression/decompression + LZ - files related to LZ (Lempel-Ziv) compression algorith= m + LZMA - LZMA compression/decompression + RangeCoder - Range Coder (special code of compression/decompressio= n) + + +Note:=20 + Asm / C / C++ source code of LZMA SDK is part of 7-Zip's source code. + 7-Zip's source code can be downloaded from 7-Zip's SourceForge page: + + http://sourceforge.net/projects/sevenzip/ + + + +LZMA features +------------- + - Variable dictionary size (up to 1 GB) + - Estimated compressing speed: about 2 MB/s on 2 GHz CPU + - Estimated decompressing speed:=20 + - 20-30 MB/s on modern 2 GHz cpu + - 1-2 MB/s on 200 MHz simple RISC cpu: (ARM, MIPS, PowerPC) + - Small memory requirements for decompressing (16 KB + DictionarySize) + - Small code size for decompressing: 5-8 KB + +LZMA decoder uses only integer operations and can be=20 +implemented in any modern 32-bit CPU (or on 16-bit CPU with some condition= s). + +Some critical operations that affect the speed of LZMA decompression: + 1) 32*16 bit integer multiply + 2) Mispredicted branches (penalty mostly depends from pipeline length) + 3) 32-bit shift and arithmetic operations + +The speed of LZMA decompressing mostly depends from CPU speed. +Memory speed has no big meaning. But if your CPU has small data cache,=20 +overall weight of memory speed will slightly increase. + + +How To Use +---------- + +Using LZMA encoder/decoder executable +-------------------------------------- + +Usage: LZMA inputFile outputFile [...] + + e: encode file + + d: decode file + + b: Benchmark. There are two tests: compressing and decompressing=20 + with LZMA method. Benchmark shows rating in MIPS (million=20 + instructions per second). Rating value is calculated from=20 + measured speed and it is normalized with Intel's Core 2 results. + Also Benchmark checks possible hardware errors (RAM=20 + errors in most cases). Benchmark uses these settings: + (-a1, -d21, -fb32, -mfbt4). You can change only -d parameter.=20 + Also you can change the number of iterations. Example for 30 iteratio= ns: + LZMA b 30 + Default number of iterations is 10. + + + =20 + + -a{N}: set compression mode 0 =3D fast, 1 =3D normal + default: 1 (normal) + + d{N}: Sets Dictionary size - [0, 30], default: 23 (8MB) + The maximum value for dictionary size is 1 GB =3D 2^30 bytes. + Dictionary size is calculated as DictionarySize =3D 2^N bytes.=20 + For decompressing file compressed by LZMA method with dictionary= =20 + size D =3D 2^N you need about D bytes of memory (RAM). + + -fb{N}: set number of fast bytes - [5, 273], default: 128 + Usually big number gives a little bit better compression ratio=20 + and slower compression process. + + -lc{N}: set number of literal context bits - [0, 8], default: 3 + Sometimes lc=3D4 gives gain for big files. + + -lp{N}: set number of literal pos bits - [0, 4], default: 0 + lp switch is intended for periodical data when period is=20 + equal 2^N. For example, for 32-bit (4 bytes)=20 + periodical data you can use lp=3D2. Often it's better to set lc0= ,=20 + if you change lp switch. + + -pb{N}: set number of pos bits - [0, 4], default: 2 + pb switch is intended for periodical data=20 + when period is equal 2^N. + + -mf{MF_ID}: set Match Finder. Default: bt4.=20 + Algorithms from hc* group doesn't provide good compression=20 + ratio, but they often works pretty fast in combination with= =20 + fast mode (-a0). + + Memory requirements depend from dictionary size=20 + (parameter "d" in table below).=20 + + MF_ID Memory Description + + bt2 d * 9.5 + 4MB Binary Tree with 2 bytes hashing. + bt3 d * 11.5 + 4MB Binary Tree with 3 bytes hashing. + bt4 d * 11.5 + 4MB Binary Tree with 4 bytes hashing. + hc4 d * 7.5 + 4MB Hash Chain with 4 bytes hashing. + + -eos: write End Of Stream marker. By default LZMA doesn't write=20 + eos marker, since LZMA decoder knows uncompressed size=20 + stored in .lzma file header. + + -si: Read data from stdin (it will write End Of Stream marker). + -so: Write data to stdout + + +Examples: + +1) LZMA e file.bin file.lzma -d16 -lc0=20 + +compresses file.bin to file.lzma with 64 KB dictionary (2^16=3D64K) =20 +and 0 literal context bits. -lc0 allows to reduce memory requirements=20 +for decompression. + + +2) LZMA e file.bin file.lzma -lc0 -lp2 + +compresses file.bin to file.lzma with settings suitable=20 +for 32-bit periodical data (for example, ARM or MIPS code). + +3) LZMA d file.lzma file.bin + +decompresses file.lzma to file.bin. + + +Compression ratio hints +----------------------- + +Recommendations +--------------- + +To increase the compression ratio for LZMA compressing it's desirable=20 +to have aligned data (if it's possible) and also it's desirable to locate +data in such order, where code is grouped in one place and data is=20 +grouped in other place (it's better than such mixing: code, data, code, +data, ...). + + +Filters +------- +You can increase the compression ratio for some data types, using +special filters before compressing. For example, it's possible to=20 +increase the compression ratio on 5-10% for code for those CPU ISAs:=20 +x86, IA-64, ARM, ARM-Thumb, PowerPC, SPARC. + +You can find C source code of such filters in C/Bra*.* files + +You can check the compression ratio gain of these filters with such=20 +7-Zip commands (example for ARM code): +No filter: + 7z a a1.7z a.bin -m0=3Dlzma + +With filter for little-endian ARM code: + 7z a a2.7z a.bin -m0=3Darm -m1=3Dlzma =20 + +It works in such manner: +Compressing =3D Filter_encoding + LZMA_encoding +Decompressing =3D LZMA_decoding + Filter_decoding + +Compressing and decompressing speed of such filters is very high, +so it will not increase decompressing time too much. +Moreover, it reduces decompression time for LZMA_decoding,=20 +since compression ratio with filtering is higher. + +These filters convert CALL (calling procedure) instructions=20 +from relative offsets to absolute addresses, so such data becomes more=20 +compressible. + +For some ISAs (for example, for MIPS) it's impossible to get gain from suc= h filter. + + + +--- + +http://www.7-zip.org +http://www.7-zip.org/sdk.html +http://www.7-zip.org/support.html diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/history.txt b/BaseTools/So= urce/C/LzmaCompress/Sdk/history.txt deleted file mode 100644 index 9bed5eb..0000000 --- a/BaseTools/Source/C/LzmaCompress/Sdk/history.txt +++ /dev/null @@ -1,236 +0,0 @@ -HISTORY of the LZMA SDK ------------------------ - -4.65 2009-02-03 -------------------------- -- Some minor fixes - - -4.63 2008-12-31 -------------------------- -- Some minor fixes - - -4.61 beta 2008-11-23 -------------------------- -- The bug in ANSI-C LZMA Decoder was fixed: - If encoded stream was corrupted, decoder could access memory=20 - outside of allocated range. -- Some changes in ANSI-C 7z Decoder interfaces. -- LZMA SDK is placed in the public domain. - - -4.60 beta 2008-08-19 -------------------------- -- Some minor fixes. - - -4.59 beta 2008-08-13 -------------------------- -- The bug was fixed: - LZMA Encoder in fast compression mode could access memory outside of=20 - allocated range in some rare cases. - - -4.58 beta 2008-05-05 -------------------------- -- ANSI-C LZMA Decoder was rewritten for speed optimizations. -- ANSI-C LZMA Encoder was included to LZMA SDK. -- C++ LZMA code now is just wrapper over ANSI-C code. - - -4.57 2007-12-12 -------------------------- -- Speed optimizations in =D1++ LZMA Decoder.=20 -- Small changes for more compatibility with some C/C++ compilers. - - -4.49 beta 2007-07-05 -------------------------- -- .7z ANSI-C Decoder: - - now it supports BCJ and BCJ2 filters - - now it supports files larger than 4 GB. - - now it supports "Last Write Time" field for files. -- C++ code for .7z archives compressing/decompressing from 7-zip=20 - was included to LZMA SDK. - =20 - -4.43 2006-06-04 -------------------------- -- Small changes for more compatibility with some C/C++ compilers. - =20 - -4.42 2006-05-15 -------------------------- -- Small changes in .h files in ANSI-C version. - =20 - -4.39 beta 2006-04-14 -------------------------- -- The bug in versions 4.33b:4.38b was fixed: - C++ version of LZMA encoder could not correctly compress=20 - files larger than 2 GB with HC4 match finder (-mfhc4). - =20 - -4.37 beta 2005-04-06 -------------------------- -- Fixes in C++ code: code could no be compiled if _NO_EXCEPTIONS was defin= ed.=20 - - -4.35 beta 2005-03-02 -------------------------- -- The bug was fixed in C++ version of LZMA Decoder: - If encoded stream was corrupted, decoder could access memory=20 - outside of allocated range. - - -4.34 beta 2006-02-27 -------------------------- -- Compressing speed and memory requirements for compressing were increased -- LZMA now can use only these match finders: HC4, BT2, BT3, BT4 - - -4.32 2005-12-09 -------------------------- -- Java version of LZMA SDK was included - - -4.30 2005-11-20 -------------------------- -- Compression ratio was improved in -a2 mode -- Speed optimizations for compressing in -a2 mode -- -fb switch now supports values up to 273 -- The bug in 7z_C (7zIn.c) was fixed: - It used Alloc/Free functions from different memory pools. - So if program used two memory pools, it worked incorrectly. -- 7z_C: .7z format supporting was improved -- LZMA# SDK (C#.NET version) was included - - -4.27 (Updated) 2005-09-21 -------------------------- -- Some GUIDs/interfaces in C++ were changed. - IStream.h: - ISequentialInStream::Read now works as old ReadPart - ISequentialOutStream::Write now works as old WritePart - - -4.27 2005-08-07 -------------------------- -- The bug in LzmaDecodeSize.c was fixed: - if _LZMA_IN_CB and _LZMA_OUT_READ were defined, - decompressing worked incorrectly. - - -4.26 2005-08-05 -------------------------- -- Fixes in 7z_C code and LzmaTest.c: - previous versions could work incorrectly, - if malloc(0) returns 0 - - -4.23 2005-06-29 -------------------------- -- Small fixes in C++ code - - -4.22 2005-06-10 -------------------------- -- Small fixes - - -4.21 2005-06-08 -------------------------- -- Interfaces for ANSI-C LZMA Decoder (LzmaDecode.c) were changed -- New additional version of ANSI-C LZMA Decoder with zlib-like interface: - - LzmaStateDecode.h - - LzmaStateDecode.c - - LzmaStateTest.c -- ANSI-C LZMA Decoder now can decompress files larger than 4 GB - - -4.17 2005-04-18 -------------------------- -- New example for RAM->RAM compressing/decompressing:=20 - LZMA + BCJ (filter for x86 code): - - LzmaRam.h - - LzmaRam.cpp - - LzmaRamDecode.h - - LzmaRamDecode.c - - -f86 switch for lzma.exe - - -4.16 2005-03-29 -------------------------- -- The bug was fixed in LzmaDecode.c (ANSI-C LZMA Decoder):=20 - If _LZMA_OUT_READ was defined, and if encoded stream was corrupted, - decoder could access memory outside of allocated range. -- Speed optimization of ANSI-C LZMA Decoder (now it's about 20% faster). - Old version of LZMA Decoder now is in file LzmaDecodeSize.c.=20 - LzmaDecodeSize.c can provide slightly smaller code than LzmaDecode.c -- Small speed optimization in LZMA C++ code -- filter for SPARC's code was added -- Simplified version of .7z ANSI-C Decoder was included - - -4.06 2004-09-05 -------------------------- -- The bug in v4.05 was fixed: - LZMA-Encoder didn't release output stream in some cases. - - -4.05 2004-08-25 -------------------------- -- Source code of filters for x86, IA-64, ARM, ARM-Thumb=20 - and PowerPC code was included to SDK -- Some internal minor changes - - -4.04 2004-07-28 -------------------------- -- More compatibility with some C++ compilers - - -4.03 2004-06-18 -------------------------- -- "Benchmark" command was added. It measures compressing=20 - and decompressing speed and shows rating values.=20 - Also it checks hardware errors. - - -4.02 2004-06-10 -------------------------- -- C++ LZMA Encoder/Decoder code now is more portable - and it can be compiled by GCC on Linux. - - -4.01 2004-02-15 -------------------------- -- Some detection of data corruption was enabled. - LzmaDecode.c / RangeDecoderReadByte - ..... - { - rd->ExtraBytes =3D 1; - return 0xFF; - } - - -4.00 2004-02-13 -------------------------- -- Original version of LZMA SDK - - - -HISTORY of the LZMA -------------------- - 2001-2008: Improvements to LZMA compressing/decompressing code,=20 - keeping compatibility with original LZMA format - 1996-2001: Development of LZMA compression format - - Some milestones: - - 2001-08-30: LZMA compression was added to 7-Zip - 1999-01-02: First version of 7-Zip was released - =20 - -End of document diff --git a/BaseTools/Source/C/LzmaCompress/Sdk/lzma.txt b/BaseTools/Sourc= e/C/LzmaCompress/Sdk/lzma.txt deleted file mode 100644 index d4f4af9..0000000 --- a/BaseTools/Source/C/LzmaCompress/Sdk/lzma.txt +++ /dev/null @@ -1,594 +0,0 @@ -LZMA SDK 4.65 -------------- - -LZMA SDK provides the documentation, samples, header files, libraries,=20 -and tools you need to develop applications that use LZMA compression. - -LZMA is default and general compression method of 7z format -in 7-Zip compression program (www.7-zip.org). LZMA provides high=20 -compression ratio and very fast decompression. - -LZMA is an improved version of famous LZ77 compression algorithm.=20 -It was improved in way of maximum increasing of compression ratio, -keeping high decompression speed and low memory requirements for=20 -decompressing. - - - -LICENSE -------- - -LZMA SDK is written and placed in the public domain by Igor Pavlov. - - -LZMA SDK Contents ------------------ - -LZMA SDK includes: - - - ANSI-C/C++/C#/Java source code for LZMA compressing and decompressing - - Compiled file->file LZMA compressing/decompressing program for Windows= system - - -UNIX/Linux version=20 ------------------- -To compile C++ version of file->file LZMA encoding, go to directory -C++/7zip/Compress/LZMA_Alone=20 -and call make to recompile it: - make -f makefile.gcc clean all - -In some UNIX/Linux versions you must compile LZMA with static libraries. -To compile with static libraries, you can use=20 -LIB =3D -lm -static - - -Files ---------------------- -lzma.txt - LZMA SDK description (this file) -7zFormat.txt - 7z Format description -7zC.txt - 7z ANSI-C Decoder description -methods.txt - Compression method IDs for .7z -lzma.exe - Compiled file->file LZMA encoder/decoder for Windows -history.txt - history of the LZMA SDK - - -Source code structure ---------------------- - -C/ - C files - 7zCrc*.* - CRC code - Alloc.* - Memory allocation functions - Bra*.* - Filters for x86, IA-64, ARM, ARM-Thumb, PowerPC and S= PARC code - LzFind.* - Match finder for LZ (LZMA) encoders=20 - LzFindMt.* - Match finder for LZ (LZMA) encoders for multithreadin= g encoding - LzHash.h - Additional file for LZ match finder - LzmaDec.* - LZMA decoding - LzmaEnc.* - LZMA encoding - LzmaLib.* - LZMA Library for DLL calling - Types.h - Basic types for another .c files - Threads.* - The code for multithreading. - - LzmaLib - LZMA Library (.DLL for Windows) - =20 - LzmaUtil - LZMA Utility (file->file LZMA encoder/decoder). - - Archive - files related to archiving - 7z - 7z ANSI-C Decoder - -CPP/ -- CPP files - - Common - common files for C++ projects - Windows - common files for Windows related code - - 7zip - files related to 7-Zip Project - - Common - common files for 7-Zip - - Compress - files related to compression/decompression - - Copy - Copy coder - RangeCoder - Range Coder (special code of compression/decompressio= n) - LZMA - LZMA compression/decompression on C++ - LZMA_Alone - file->file LZMA compression/decompression - Branch - Filters for x86, IA-64, ARM, ARM-Thumb, PowerPC and S= PARC code - - Archive - files related to archiving - - Common - common files for archive handling - 7z - 7z C++ Encoder/Decoder - - Bundles - Modules that are bundles of other modules - =20 - Alone7z - 7zr.exe: Standalone version of 7z.exe that suppo= rts only 7z/LZMA/BCJ/BCJ2 - Format7zR - 7zr.dll: Reduced version of 7za.dll: extracting/= compressing to 7z/LZMA/BCJ/BCJ2 - Format7zExtractR - 7zxr.dll: Reduced version of 7zxa.dll: extractin= g from 7z/LZMA/BCJ/BCJ2. - - UI - User Interface files - =20 - Client7z - Test application for 7za.dll, 7zr.dll, 7zxr.dll - Common - Common UI files - Console - Code for console archiver - - - -CS/ - C# files - 7zip - Common - some common files for 7-Zip - Compress - files related to compression/decompression - LZ - files related to LZ (Lempel-Ziv) compression algorithm - LZMA - LZMA compression/decompression - LzmaAlone - file->file LZMA compression/decompression - RangeCoder - Range Coder (special code of compression/decompressio= n) - -Java/ - Java files - SevenZip - Compression - files related to compression/decompression - LZ - files related to LZ (Lempel-Ziv) compression algorith= m - LZMA - LZMA compression/decompression - RangeCoder - Range Coder (special code of compression/decompressio= n) - - -C/C++ source code of LZMA SDK is part of 7-Zip project. -7-Zip source code can be downloaded from 7-Zip's SourceForge page: - - http://sourceforge.net/projects/sevenzip/ - - - -LZMA features -------------- - - Variable dictionary size (up to 1 GB) - - Estimated compressing speed: about 2 MB/s on 2 GHz CPU - - Estimated decompressing speed:=20 - - 20-30 MB/s on 2 GHz Core 2 or AMD Athlon 64 - - 1-2 MB/s on 200 MHz ARM, MIPS, PowerPC or other simple RISC - - Small memory requirements for decompressing (16 KB + DictionarySize) - - Small code size for decompressing: 5-8 KB - -LZMA decoder uses only integer operations and can be=20 -implemented in any modern 32-bit CPU (or on 16-bit CPU with some condition= s). - -Some critical operations that affect the speed of LZMA decompression: - 1) 32*16 bit integer multiply - 2) Misspredicted branches (penalty mostly depends from pipeline length) - 3) 32-bit shift and arithmetic operations - -The speed of LZMA decompressing mostly depends from CPU speed. -Memory speed has no big meaning. But if your CPU has small data cache,=20 -overall weight of memory speed will slightly increase. - - -How To Use ----------- - -Using LZMA encoder/decoder executable --------------------------------------- - -Usage: LZMA inputFile outputFile [...] - - e: encode file - - d: decode file - - b: Benchmark. There are two tests: compressing and decompressing=20 - with LZMA method. Benchmark shows rating in MIPS (million=20 - instructions per second). Rating value is calculated from=20 - measured speed and it is normalized with Intel's Core 2 results. - Also Benchmark checks possible hardware errors (RAM=20 - errors in most cases). Benchmark uses these settings: - (-a1, -d21, -fb32, -mfbt4). You can change only -d parameter.=20 - Also you can change the number of iterations. Example for 30 iteratio= ns: - LZMA b 30 - Default number of iterations is 10. - - - =20 - - -a{N}: set compression mode 0 =3D fast, 1 =3D normal - default: 1 (normal) - - d{N}: Sets Dictionary size - [0, 30], default: 23 (8MB) - The maximum value for dictionary size is 1 GB =3D 2^30 bytes. - Dictionary size is calculated as DictionarySize =3D 2^N bytes.=20 - For decompressing file compressed by LZMA method with dictionary= =20 - size D =3D 2^N you need about D bytes of memory (RAM). - - -fb{N}: set number of fast bytes - [5, 273], default: 128 - Usually big number gives a little bit better compression ratio=20 - and slower compression process. - - -lc{N}: set number of literal context bits - [0, 8], default: 3 - Sometimes lc=3D4 gives gain for big files. - - -lp{N}: set number of literal pos bits - [0, 4], default: 0 - lp switch is intended for periodical data when period is=20 - equal 2^N. For example, for 32-bit (4 bytes)=20 - periodical data you can use lp=3D2. Often it's better to set lc0= ,=20 - if you change lp switch. - - -pb{N}: set number of pos bits - [0, 4], default: 2 - pb switch is intended for periodical data=20 - when period is equal 2^N. - - -mf{MF_ID}: set Match Finder. Default: bt4.=20 - Algorithms from hc* group doesn't provide good compression=20 - ratio, but they often works pretty fast in combination with= =20 - fast mode (-a0). - - Memory requirements depend from dictionary size=20 - (parameter "d" in table below).=20 - - MF_ID Memory Description - - bt2 d * 9.5 + 4MB Binary Tree with 2 bytes hashing. - bt3 d * 11.5 + 4MB Binary Tree with 3 bytes hashing. - bt4 d * 11.5 + 4MB Binary Tree with 4 bytes hashing. - hc4 d * 7.5 + 4MB Hash Chain with 4 bytes hashing. - - -eos: write End Of Stream marker. By default LZMA doesn't write=20 - eos marker, since LZMA decoder knows uncompressed size=20 - stored in .lzma file header. - - -si: Read data from stdin (it will write End Of Stream marker). - -so: Write data to stdout - - -Examples: - -1) LZMA e file.bin file.lzma -d16 -lc0=20 - -compresses file.bin to file.lzma with 64 KB dictionary (2^16=3D64K) =20 -and 0 literal context bits. -lc0 allows to reduce memory requirements=20 -for decompression. - - -2) LZMA e file.bin file.lzma -lc0 -lp2 - -compresses file.bin to file.lzma with settings suitable=20 -for 32-bit periodical data (for example, ARM or MIPS code). - -3) LZMA d file.lzma file.bin - -decompresses file.lzma to file.bin. - - -Compression ratio hints ------------------------ - -Recommendations ---------------- - -To increase the compression ratio for LZMA compressing it's desirable=20 -to have aligned data (if it's possible) and also it's desirable to locate -data in such order, where code is grouped in one place and data is=20 -grouped in other place (it's better than such mixing: code, data, code, -data, ...). - - -Filters -------- -You can increase the compression ratio for some data types, using -special filters before compressing. For example, it's possible to=20 -increase the compression ratio on 5-10% for code for those CPU ISAs:=20 -x86, IA-64, ARM, ARM-Thumb, PowerPC, SPARC. - -You can find C source code of such filters in C/Bra*.* files - -You can check the compression ratio gain of these filters with such=20 -7-Zip commands (example for ARM code): -No filter: - 7z a a1.7z a.bin -m0=3Dlzma - -With filter for little-endian ARM code: - 7z a a2.7z a.bin -m0=3Darm -m1=3Dlzma =20 - -It works in such manner: -Compressing =3D Filter_encoding + LZMA_encoding -Decompressing =3D LZMA_decoding + Filter_decoding - -Compressing and decompressing speed of such filters is very high, -so it will not increase decompressing time too much. -Moreover, it reduces decompression time for LZMA_decoding,=20 -since compression ratio with filtering is higher. - -These filters convert CALL (calling procedure) instructions=20 -from relative offsets to absolute addresses, so such data becomes more=20 -compressible. - -For some ISAs (for example, for MIPS) it's impossible to get gain from suc= h filter. - - -LZMA compressed file format ---------------------------- -Offset Size Description - 0 1 Special LZMA properties (lc,lp, pb in encoded form) - 1 4 Dictionary size (little endian) - 5 8 Uncompressed size (little endian). -1 means unknown size - 13 Compressed data - - -ANSI-C LZMA Decoder -~~~~~~~~~~~~~~~~~~~ - -Please note that interfaces for ANSI-C code were changed in LZMA SDK 4.58. -If you want to use old interfaces you can download previous version of LZM= A SDK -from sourceforge.net site. - -To use ANSI-C LZMA Decoder you need the following files: -1) LzmaDec.h + LzmaDec.c + Types.h -LzmaUtil/LzmaUtil.c is example application that uses these files. - - -Memory requirements for LZMA decoding -------------------------------------- - -Stack usage of LZMA decoding function for local variables is not=20 -larger than 200-400 bytes. - -LZMA Decoder uses dictionary buffer and internal state structure. -Internal state structure consumes - state_size =3D (4 + (1.5 << (lc + lp))) KB -by default (lc=3D3, lp=3D0), state_size =3D 16 KB. - - -How To decompress data ----------------------- - -LZMA Decoder (ANSI-C version) now supports 2 interfaces: -1) Single-call Decompressing -2) Multi-call State Decompressing (zlib-like interface) - -You must use external allocator: -Example: -void *SzAlloc(void *p, size_t size) { p =3D p; return malloc(size); } -void SzFree(void *p, void *address) { p =3D p; free(address); } -ISzAlloc alloc =3D { SzAlloc, SzFree }; - -You can use p =3D p; operator to disable compiler warnings. - - -Single-call Decompressing -------------------------- -When to use: RAM->RAM decompressing -Compile files: LzmaDec.h + LzmaDec.c + Types.h -Compile defines: no defines -Memory Requirements: - - Input buffer: compressed size - - Output buffer: uncompressed size - - LZMA Internal Structures: state_size (16 KB for default settings)=20 - -Interface: - int LzmaDecode(Byte *dest, SizeT *destLen, const Byte *src, SizeT *srcLe= n, - const Byte *propData, unsigned propSize, ELzmaFinishMode finishMode,= =20 - ELzmaStatus *status, ISzAlloc *alloc); - In:=20 - dest - output data - destLen - output data size - src - input data - srcLen - input data size - propData - LZMA properties (5 bytes) - propSize - size of propData buffer (5 bytes) - finishMode - It has meaning only if the decoding reaches output limit = (*destLen). - LZMA_FINISH_ANY - Decode just destLen bytes. - LZMA_FINISH_END - Stream must be finished after (*destLen). - You can use LZMA_FINISH_END, when you know that= =20 - current output buffer covers last bytes of stre= am.=20 - alloc - Memory allocator. - - Out:=20 - destLen - processed output size=20 - srcLen - processed input size=20 - - Output: - SZ_OK - status: - LZMA_STATUS_FINISHED_WITH_MARK - LZMA_STATUS_NOT_FINISHED=20 - LZMA_STATUS_MAYBE_FINISHED_WITHOUT_MARK - SZ_ERROR_DATA - Data error - SZ_ERROR_MEM - Memory allocation error - SZ_ERROR_UNSUPPORTED - Unsupported properties - SZ_ERROR_INPUT_EOF - It needs more bytes in input buffer (src). - - If LZMA decoder sees end_marker before reaching output limit, it returns= OK result, - and output value of destLen will be less than output buffer size limit. - - You can use multiple checks to test data integrity after full decompress= ion: - 1) Check Result and "status" variable. - 2) Check that output(destLen) =3D uncompressedSize, if you know real u= ncompressedSize. - 3) Check that output(srcLen) =3D compressedSize, if you know real comp= ressedSize.=20 - You must use correct finish mode in that case. */=20 - - -Multi-call State Decompressing (zlib-like interface) ----------------------------------------------------- - -When to use: file->file decompressing=20 -Compile files: LzmaDec.h + LzmaDec.c + Types.h - -Memory Requirements: - - Buffer for input stream: any size (for example, 16 KB) - - Buffer for output stream: any size (for example, 16 KB) - - LZMA Internal Structures: state_size (16 KB for default settings)=20 - - LZMA dictionary (dictionary size is encoded in LZMA properties header) - -1) read LZMA properties (5 bytes) and uncompressed size (8 bytes, little-e= ndian) to header: - unsigned char header[LZMA_PROPS_SIZE + 8]; - ReadFile(inFile, header, sizeof(header) - -2) Allocate CLzmaDec structures (state + dictionary) using LZMA properties - - CLzmaDec state; - LzmaDec_Constr(&state); - res =3D LzmaDec_Allocate(&state, header, LZMA_PROPS_SIZE, &g_Alloc); - if (res !=3D SZ_OK) - return res; - -3) Init LzmaDec structure before any new LZMA stream. And call LzmaDec_Dec= odeToBuf in loop - - LzmaDec_Init(&state); - for (;;) - { - ...=20 - int res =3D LzmaDec_DecodeToBuf(CLzmaDec *p, Byte *dest, SizeT *destLe= n,=20 - const Byte *src, SizeT *srcLen, ELzmaFinishMode finishMode); - ... - } - - -4) Free all allocated structures - LzmaDec_Free(&state, &g_Alloc); - -For full code example, look at C/LzmaUtil/LzmaUtil.c code. - - -How To compress data --------------------- - -Compile files: LzmaEnc.h + LzmaEnc.c + Types.h + -LzFind.c + LzFind.h + LzFindMt.c + LzFindMt.h + LzHash.h - -Memory Requirements: - - (dictSize * 11.5 + 6 MB) + state_size - -Lzma Encoder can use two memory allocators: -1) alloc - for small arrays. -2) allocBig - for big arrays. - -For example, you can use Large RAM Pages (2 MB) in allocBig allocator for= =20 -better compression speed. Note that Windows has bad implementation for=20 -Large RAM Pages.=20 -It's OK to use same allocator for alloc and allocBig. - - -Single-call Compression with callbacks --------------------------------------- - -Check C/LzmaUtil/LzmaUtil.c as example,=20 - -When to use: file->file decompressing=20 - -1) you must implement callback structures for interfaces: -ISeqInStream -ISeqOutStream -ICompressProgress -ISzAlloc - -static void *SzAlloc(void *p, size_t size) { p =3D p; return MyAlloc(size)= ; } -static void SzFree(void *p, void *address) { p =3D p; MyFree(address); } -static ISzAlloc g_Alloc =3D { SzAlloc, SzFree }; - - CFileSeqInStream inStream; - CFileSeqOutStream outStream; - - inStream.funcTable.Read =3D MyRead; - inStream.file =3D inFile; - outStream.funcTable.Write =3D MyWrite; - outStream.file =3D outFile; - - -2) Create CLzmaEncHandle object; - - CLzmaEncHandle enc; - - enc =3D LzmaEnc_Create(&g_Alloc); - if (enc =3D=3D 0) - return SZ_ERROR_MEM; - - -3) initialize CLzmaEncProps properties; - - LzmaEncProps_Init(&props); - - Then you can change some properties in that structure. - -4) Send LZMA properties to LZMA Encoder - - res =3D LzmaEnc_SetProps(enc, &props); - -5) Write encoded properties to header - - Byte header[LZMA_PROPS_SIZE + 8]; - size_t headerSize =3D LZMA_PROPS_SIZE; - UInt64 fileSize; - int i; - - res =3D LzmaEnc_WriteProperties(enc, header, &headerSize); - fileSize =3D MyGetFileLength(inFile); - for (i =3D 0; i < 8; i++) - header[headerSize++] =3D (Byte)(fileSize >> (8 * i)); - MyWriteFileAndCheck(outFile, header, headerSize) - -6) Call encoding function: - res =3D LzmaEnc_Encode(enc, &outStream.funcTable, &inStream.funcTabl= e,=20 - NULL, &g_Alloc, &g_Alloc); - -7) Destroy LZMA Encoder Object - LzmaEnc_Destroy(enc, &g_Alloc, &g_Alloc); - - -If callback function return some error code, LzmaEnc_Encode also returns t= hat code. - - -Single-call RAM->RAM Compression --------------------------------- - -Single-call RAM->RAM Compression is similar to Compression with callbacks, -but you provide pointers to buffers instead of pointers to stream callback= s: - -HRes LzmaEncode(Byte *dest, SizeT *destLen, const Byte *src, SizeT srcLen, - CLzmaEncProps *props, Byte *propsEncoded, SizeT *propsSize, int writeE= ndMark,=20 - ICompressProgress *progress, ISzAlloc *alloc, ISzAlloc *allocBig); - -Return code: - SZ_OK - OK - SZ_ERROR_MEM - Memory allocation error=20 - SZ_ERROR_PARAM - Incorrect paramater - SZ_ERROR_OUTPUT_EOF - output buffer overflow - SZ_ERROR_THREAD - errors in multithreading functions (only for Mt ve= rsion) - - - -LZMA Defines ------------- - -_LZMA_SIZE_OPT - Enable some optimizations in LZMA Decoder to get smaller = executable code. - -_LZMA_PROB32 - It can increase the speed on some 32-bit CPUs, but memory= usage for=20 - some structures will be doubled in that case. - -_LZMA_UINT32_IS_ULONG - Define it if int is 16-bit on your compiler and l= ong is 32-bit. - -_LZMA_NO_SYSTEM_SIZE_T - Define it if you don't want to use size_t type. - - -C++ LZMA Encoder/Decoder=20 -~~~~~~~~~~~~~~~~~~~~~~~~ -C++ LZMA code use COM-like interfaces. So if you want to use it,=20 -you can study basics of COM/OLE. -C++ LZMA code is just wrapper over ANSI-C code. - - -C++ Notes -~~~~~~~~~~~~~~~~~~~~~~~~ -If you use some C++ code folders in 7-Zip (for example, C++ code for .7z h= andling), -you must check that you correctly work with "new" operator. -7-Zip can be compiled with MSVC 6.0 that doesn't throw "exception" from "n= ew" operator. -So 7-Zip uses "CPP\Common\NewHandler.cpp" that redefines "new" operator: -operator new(size_t size) -{ - void *p =3D ::malloc(size); - if (p =3D=3D 0) - throw CNewException(); - return p; -} -If you use MSCV that throws exception for "new" operator, you can compile = without=20 -"NewHandler.cpp". So standard exception will be used. Actually some code o= f=20 -7-Zip catches any exception in internal code and converts it to HRESULT co= de. -So you don't need to catch CNewException, if you call COM interfaces of 7-= Zip. - ---- - -http://www.7-zip.org -http://www.7-zip.org/sdk.html -http://www.7-zip.org/support.html --=20 2.8.0.windows.1